Re: [External] Re: Missing Promisor Objects in Partial Repo Design Doc

From: Han Young <hanyang.tony@bytedance.com>
Date: Oct 9, 2024, 06:46 UTC
Message-ID: <CAG1j3zGcdfd6YRb=Fb1Aqt5kLajueagV+6upt6vwsGW9RxaR7Q@mail.gmail.com>
In-Reply-To: <CAFySSZCyoaKCGycYgJjCJGJ2mV1yfg+gVFb7RytGKmkjupkNkQ@mail.gmail.com>

On Wed, Oct 9, 2024 at 5:36 AM Calvin Wan <calvinwan@google.com> wrote:

Show 5 quoted lines

> Objects that are in promisor packs, specifically the ones that have the
> flag, packed_git::pack_promisor, set. However, since this design doc
> was sent out, it turns out the creation of a set of promisor pack objects
> in a large repository (such as Android or Chrome) is very expensive, so
> this design is infeasible in my opinion.

I wonder if a set of local loose/pack objects will be cheaper to construct? Normally loose objects are always non-promisor objects, unless the user running something like `unpack-objects`.

Show 12 quoted lines

> > After a lazy clone that omits a lot of objects acquires many objects
> > over time by fetching missing objects on demand, wouldn't we want to
> > have an option to "slim" the local repository by discarding some of
> > these objects (the ones that are least frequently used), relying on
> > the promise by the promisor remote that even if we did so, they can
> > be fetched again?  Can we treat loss of C2a/C2b/C2 as if such a
> > feature prematurely kicked in?  Or are we failing to refetch them
> > for some reason?
>
> Yes if such a feature existed, then it would be feasible and a possible
> solution for this issue (I'm leaning quite towards this now after testing
> out some of the other designs).

Since no partial clone filter omits commit objects, we always assume commits are available in the codebase. `merge` reports "cannot merge unrelated history" if one of the commits is missing, instead of trying to fetch it. Another problem is current lazy fetching code does not report "haves" to remote, so a lazy fetching of commit ended up pulling all the trees, blobs associated with that commit. I also prefer the "fetching the missing objects" approach, making sure the repo has all the "correct" objects is difficult to get right.