Re: [PATCH 11/11] bundle: unbundle promisor packs

From: Derrick Stolee <derrickstolee@github.com>
Date: Mar 7, 2022, 18:57 UTC
Message-ID: <85d434f8-0eb1-c6f7-dfd4-99bffbfbba57@github.com>
In-Reply-To: <xmqqv8wpu2fq.fsf@gitster.g>

On 3/7/2022 11:56 AM, Junio C Hamano wrote:

Show 16 quoted lines

> Derrick Stolee <derrickstolee@github.com> writes:
> 
>> Of course, looking closer at it... "git bundle unbundle" doesn't
>> actually store the refs directly in the refspace, but instead only
>> outputs the refs that it used.
> 
> True.  I was more thinking about equivalence between
> 
>     cd src_repo
>     git clone --no-local --filter=... . ../partial.network.cloned
>     git bundle create --filter=... partial.bndl
>     git clone partial.bndl ../partial.bundle.cloned
> 
> The two resulting repositories should look very similar except for
> that the remote.origin.* of the former would expect that it pushes
> back to where it was cloned from, while the latter would not.

Makes sense.

The one downside is that you list cloning form a partial bundle, but that currently does not work, even if we avoid a checkout. It fails because the clone command is not parsing the filter and properly setting repo-global promisor information. (Again, this is a bigger change to make this possible.)

I also had some struggles getting this to work since local clones
were actually ignoring the filter. I didn't think it was worth
setting up an HTTP or SSH server just for this test. See
workaround below.

Show 11 quoted lines

>> +		git init unbundled &&
>> +		(
>> +			cd unbundled &&
>> +			# This creates the first pack-file in the
>> +			# .git/objects/pack directory. Look for a .promisor.
>> +			git bundle unbundle ../partial.bdl >ref-list.txt &&
>> +			ls .git/objects/pack/pack-*.promisor >promisor &&
>> +			test_line_count = 1 promisor
> 
> And can we enumerate the objects we have in .git/objects, both loose
> and packed?

I can enumerate using 'git rev-list --objects' to compare the unbundled set to the full clone (adding --filter=$filter to the full clone's run and --missing=allow-any to the unbundled one).

Show 26 quoted lines

>> +		) &&
>> +
>> +		git clone --filter=blob:none --mirror "file://$(pwd)" cloned &&
>> +		git -C cloned for-each-ref \
>> +			--format="%(objectname) %(refname)" >cloned-refs.txt &&
>> +		echo "$(git -C cloned rev-parse HEAD) HEAD" >>cloned-refs.txt &&
>> +		test_cmp cloned-refs.txt unbundled/ref-list.txt
> 
> Likewise here?  I think the two should match, and that was what I
> was wondering if we should enforce.
> 
>>  	'
>>  done
>>  
>> --- >8 ---
>>
>> I also attempted doing a "git clone --bare partial.bdl unbundled.git" to
>> get the 'git clone' command to actually place the refs. However, 'git clone'
>> does not set up the repository filter based on the bundle, so it reports
>> missing blobs (even though there is no checkout).
> 
> Understandable, as cloning from a bundle, if I recall correctly, was
> done as yet another special case in "git clone", differently from
> the main "over the network" code path.  And from end-user's point of
> view, I think updating it is part of introducing the filtered
> bundle.

The reason I did not include that here is because of the lack of repository-global promisor/filter config. I do want to loop back and make those updates, but perhaps for this series we should add an error condition into 'git clone' to say "Cannot currently clone from a filtered bundle" to help users understand the issue?

Thanks, -Stolee