From: Derrick Stolee <stolee@gmail.com>
Date: Tue, 21 Apr 2026 18:06:09 GMT
Subject: Re: [PATCH v2 5/7] fetch: add --negotiation-require option for negotiation
Message-ID: <8f79a311-dbd8-4a86-8b46-1079ebd3c2ce@gmail.com>
In-Reply-To: <xmqqik9s57gc.fsf@gitster.g>

On 4/15/26 3:50 PM, Junio C Hamano wrote:
> "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:
...
>> +static struct string_list negotiation_require = STRING_LIST_INIT_NODUP;
> 
> I thought _tip was renamed to _restrict in an earlier step, but that
> was only in the transport in [3/7].  Perhaps we want to rename the
> file-scope static variable negotiation_tip to negotiation_restrict
> in an earlier step, like in [2/7]?

This one was missed but will be fixed in v3.

>> +	for_each_string_list_item(item, negotiation_require) {
>> +		if (!has_glob_specials(item->string)) {
>> +			struct object_id oid;
>> +			if (repo_get_oid(the_repository, item->string, &oid))
>> +				continue;
> 
> The configuration (or command line) says --nego-require=refs/heads/main
> but this old repository only has refs/heads/master; we do not want
> to error out in such a case.
> 
> Is it true, though?  nego-{require,restrict} feels quite tied to
> each project and unless the configuration or command line options
> are applied blindly regardless of the project, such an error should
> not happen.  Perhaps the user who gives a command line option
> "--nego-require=refs/heads/naster" may want to be reminded of a
> possible typo?

You're right here. We shouldn't die() on a bad ref passed this way.

This should be a best-effort attempt to include a "have" and continue
normally if it isn't found.

>> +			if (!odb_has_object(the_repository->objects, &oid, 0))
>> +				continue;
> 
> This is a bit curious.  When does the first condition holds but not
> the second?  A lazy clone whose ref-tip contains a missing commit
> promised by somebody else?

Good point. This would occur if a ref exists but points to a missing
object, which should mean the repo is corrupt and we can't trust that
the fetch will succeed (or should).

> In the presense of "promised objects are allowed to be missing"
> rule, silently skipping a missing object here is certainly
> conservative, but this is not an object that is buried deep in a
> tree hierarchy, but the top-level commit or tag that is directly
> pointed at by a ref, isn't it?  I am a bit uneasy that we ignore
> such potential repository corruption (i.e., a missing object may not
> be something a promisor remote promised but simply missing).

I'll update this to be an error.

>> @@ -474,7 +511,25 @@ static int find_common(struct fetch_negotiator *negotiator,
>>   	trace2_region_enter("fetch-pack", "negotiation_v0_v1", the_repository);
>>   	flushes = 0;
>>   	retval = -1;
>> +
>> +	/* Send unconditional haves from --negotiation-require */
>> +	resolve_negotiation_require(args->negotiation_require,
>> +				    &negotiation_require_oids);
>> +	if (oidset_size(&negotiation_require_oids)) {
>> +		struct oidset_iter iter;
>> +		oidset_iter_init(&negotiation_require_oids, &iter);
>> +
>> +		while ((oid = oidset_iter_next(&iter))) {
>> +			packet_buf_write(&req_buf, "have %s\n",
>> +					 oid_to_hex(oid));
>> +			print_verbose(args, "have %s", oid_to_hex(oid));
>> +		}
>> +	}
> 
> OK.  I think it makes sense to send these early.  We have already
> dealt with the usual "tips" by calling mark_tips() way earlier, but
> that hasn't produced any "have" yet, and these will go before the
> ones from traversal.  We do not traverse from these "require" and
> that may be why these are not called "_tips"?

It is correct that these required haves are not plugged into the
walk.

> And sending these early means the other side has much less chance to
> say "we've heard enough, stop!", so in a sense they are of much
> higher priority "have"s (I wonder what happens when they do want to
> say "stop!" while we are giving a lot of "have" from this loop,
> though).

I believe that we don't give the server an opportunity to say "stop"
until we've completed a "round" (see the 'if (flush_at <= ++count)'
case).

With this in mind, I should be incrementing 'count' while sending
the required haves.

>>   	while ((oid = negotiator->next(negotiator))) {
>> +		/* avoid duplicate oids from --negotiation-require */
>> +		if (oidset_contains(&negotiation_require_oids, oid))
>> +			continue;
> 
> If objects rechable from "require" are traversed like others, then
> this "avoid duplicate" would become unnecessary, right?

Yes, if we add the required things to the traversal, then we wouldn't
worry about duplicates. We'd also need to do things in a different
way:

1. The negotiator has a next() method that could do a number of
    things, but is responsible for walking history and ignoring IDs
    that are reachable from already-emitted haves.

2. This _could_ help us avoid sending required ID if we have
    emitted a have that could reach that ID.

3. However, this requires waiting until we flush a round of haves
    before determining if we should send the required IDs based on
    the walk to this point.

Perhaps the better way to incorporate things together would be to
mark the required IDs as COMMON as we emit them, which would signal
to the negotiation walks that they should not re-emit it as a have
(but since we don't add SEEN, they still walk through the commit to
its later ancestors).

Thanks,
-Stolee