Re: [PATCH v2 2/2] diff: restrict when prefetching occurs
- From
- Jonathan Tan <jonathantanmy@google.com>
- Date
- Apr 2, 2020, 23:09 UTC
- Message-ID
- <20200402230937.47323-1-jonathantanmy@google.com>
- In-Reply-To
- <xmqq7dyx3b1o.fsf@gitster.c.googlers.com>
Show 9 quoted lines
> > + int output_formats_to_prefetch = DIFF_FORMAT_DIFFSTAT | > > + DIFF_FORMAT_NUMSTAT | > > + DIFF_FORMAT_PATCH | > > + DIFF_FORMAT_SHORTSTAT | > > + DIFF_FORMAT_DIRSTAT; > > Would this want to be a "const int" (or even #define), I wonder. I > do not care too much between the two, but leaving it as a variable > makes me a bit nervous.
OK, will switch to "const int".
Show 33 quoted lines
> > + if (options->repo == the_repository && has_promisor_remote() &&
> > + (options->output_format & output_formats_to_prefetch ||
> > + (!options->found_follow && options->break_opt != -1))) {
> > int i;
> > struct diff_queue_struct *q = &diff_queued_diff;
> > struct oid_array to_fetch = OID_ARRAY_INIT;
> >
> > for (i = 0; i < q->nr; i++) {
> > struct diff_filepair *p = q->queue[i];
> > - add_if_missing(options->repo, &to_fetch, p->one);
> > - add_if_missing(options->repo, &to_fetch, p->two);
> > + diff_add_if_missing(options->repo, &to_fetch, p->one);
> > + diff_add_if_missing(options->repo, &to_fetch, p->two);
> > }
> > +
> > + prefetched = 1;
> > +
>
> Wouldn't it logically make more sense to do this after calling
> promisor_remote_get_direct() and if to_fetch.nr is not 0, ...
>
> > /*
> > * NEEDSWORK: Consider deduplicating the OIDs sent.
> > */
> > promisor_remote_get_direct(options->repo,
> > to_fetch.oid, to_fetch.nr);
> > +
>
> ... namely, here?
>
> When (q->nr != 0), to_fetch.nr may not be zero, I suspect, but the
> original code before [1/2] protected against to_fetch.nr==0 case, so
> ...?My idea is that this prefetch is a superset of what diffcore_rebase() wants to prefetch, so if we have already done the necessary logic here (even if nothing gets prefetched - which might be the case if we have all objects), we do not need to do it in diffcore_rebase().
Show 32 quoted lines
> > + if (!prefetched) {
> > + /*
> > + * At this point we know there's actual work to do: we have rename
> > + * destinations that didn't find an exact match, and we have potential
> > + * sources. So we'll have to do inexact rename detection, which
> > + * requires looking at the blobs.
> > + *
> > + * If we haven't already prefetched, it's worth pre-fetching
> > + * them as a group now.
> > + */
>
> This comment makes me wonder if it would be even better to
>
> - prepare an empty to_fetch OID array in the caller,
>
> - if the output format is one of the ones that wants prefetch, add
> object names to to_fetch in the caller, BUT not fetch there.
>
> - pass &to_fetch by the caller to this function, and this code here
> may add even more objects,
>
> - then do the prefetch here (so a single promisor interaction will
> grab objects the caller would have fetched before calling us and
> the ones we want here), and then clear the to_fetch array.
>
> - the caller, after seeing this function returns, checks to_fetch
> and if it is not empty, fetches (i.e. the caller prepared list of
> objects based on the output type, we ended up not calling this
> helper, and then finally the caller does the prefetch).
>
> That way, the "unless we have already prefetched" logic can go, and
> we can lose one indentation level, no?This means that the only prefetch occurs in diffcore_rename()? I don't think this will work for 2 reasons:
- diffcore_std() calls diffcore_break() (which also reads blobs) before diffcore_rename() - (more importantly) there's a code path in diffcore_std() that does not call diffcore_rename(), so we would still need some prefetching logic in diffcore_std() in case diffcore_rename() is not called
Show 5 quoted lines
> > + if (to_fetch.nr) > > + promisor_remote_get_direct(options->repo, > > + to_fetch.oid, to_fetch.nr); > > You no longer need the if(), no?
Ah...I'll remove the if().