Re: en/remerge-diff [Was: Re: What's cooking in git.git (Sep 2021, #09; Thu, 30)]

From: Jeff King <peff@peff.net>
Date: Oct 1, 2021, 07:01 UTC
Message-ID: <YVayMiOJPLFcTlY9@coredump.intra.peff.net>
In-Reply-To: <CABPp-BH2C5HFw_=NtK71oNGXkntzcJd2un0TgMb=UnJ5yJxAAA@mail.gmail.com>

On Thu, Sep 30, 2021 at 11:31:32PM -0700, Elijah Newren wrote:

>   * Do something different with the tmp-objdir usage or use
> pretend_object_file().  I'm slightly worried the latter was only
> designed for a few files that all fit into memory, which may not be
> valid for this usecase.

Just to make sure I was being clear on this final point: the pretend_object_file() interface _now_ is definitely not suitable for the sizes you'd want. I think you'd need to:

  - have it handle pointers to non-duplicated storage, instead of owning
    all buffers (and then the remerge-diff code would do something
    clever with mmaps and tempfiles).

  - use something better than the linear cached_objects array for
    storing the entries, so lookup doesn't suffer with many objects.
    Perhaps an oidmap.

  - teach it to remove objects. For your purposes, a complete clear
    function would be sufficient.

-Peff