Re: RFC: Design and code of partial clones (now, missing commits and trees OK) (part 3)

From: Jonathan Tan <jonathantanmy@google.com>
Date: Sep 26, 2017, 17:32 UTC
Message-ID: <20170926103252.8330a266cf450e64dee27500@google.com>
In-Reply-To: <e05a7978-c312-9e26-79a2-fca3ae44f59a@jeffhostetler.com>

On Tue, 26 Sep 2017 10:25:16 -0400 Jeff Hostetler <git@jeffhostetler.com> wrote:

Show 20 quoted lines

> >> Perhaps you could augment the OID lookup to remember where the object
> >> was found (essentially a .promisor bit set).  Then you wouldn't need
> >> to touch them all.
> > 
> > Sorry - I don't understand this. Are you saying that missing promisor
> > objects should go into the global object hashtable, so that we can set a
> > flag on them?
> 
> I just meant could we add a bit to "struct object_info" to indicate
> that the object was found in a .promisor packfile ?  This could
> be set in sha1_object_info_extended().
> 
> Then the is_promised() calls in fsck and gc would just test that bit.
> 
> Given that that bit will be set on promisOR objects (and we won't
> have object_info for missing objects), you may need to adjust the
> iterator in the fsck/gc code slightly.
> 
> This is a bit of a handwave, but could something like that eliminate
> the need to build this oidset?

This oidset is meant to contain the missing objects, and is needed as the final check (attempt to read the object, then check this oidset). Admittedly, right now I add objects to it even if they are present in the DB, but that's because I think that it's better for the set to be bigger than to incur the repeated existence checks. But even if we only include truly missing objects in this oidset, we still need the oidset, or store information about missing objects in some equivalent data structure.

The bit that you mention being set on promisOR objects is already being set. See the invocation of mark_uninteresting() in rev-list.c.