Re: [PATCH] packfile: skip decompressing and hashing blobs in add_promisor_object()

From: Jeff King <peff@peff.net>
Date: Dec 5, 2025, 17:59 UTC
Message-ID: <20251205175957.GB18566@coredump.intra.peff.net>
In-Reply-To: <da52d3d8-f70e-4ab9-8752-0ddb7ad145f1@nvidia.com>

On Fri, Dec 05, 2025 at 08:55:19AM -0800, Aaron Plattner wrote:

Show 30 quoted lines

> +               if (we_parsed_object)
> +                       free_commit_buffer(pack->repo->parsed_objects,
> commit);
>         } else if (obj->type == OBJ_TAG) {
>                 struct tag *tag = (struct tag *) obj;
>                 oidset_insert(set, get_tagged_oid(tag));
> 
> --
> 
> 
> That said, the memory footprint improvement seems pretty minimal with this
> change:
> 
> Without free_commit_buffer():
> 
> $ /usr/bin/time ~/git/git/git-rev-list --objects --all
> --exclude-promisor-objects --quiet
> 66.19user 38.97system 2:17.46elapsed 76%CPU (0avgtext+0avgdata
> 8171072maxresident)k
> 307985728inputs+0outputs (151871major+1067727minor)pagefaults 0swaps
> 
> With free_commit_buffer():
> 
> $ /usr/bin/time ~/git/git/git-rev-list --objects --all
> --exclude-promisor-objects --quiet
> 66.47user 40.08system 2:18.72elapsed 76%CPU (0avgtext+0avgdata
> 8135640maxresident)k
> 307820424inputs+0outputs (100432major+1065152minor)pagefaults 0swaps
> 
> I'm inclined not to worry about it for now.

I don't think it will make a difference for that command because we already turn off the "save_commit_buffer" global in rev-list, unless we are going to show the contents. Perhaps:

  git rev-list --objects --all --format=%s

would show the difference.

If we take the skip-hash suggestion I wrote elsewhere in the thread, then we would usually not load the commit contents in the first place. But it could still be worth adding this free_commit_buffer() to catch commits not covered by the commit-graph.

Another way to do it is to temporarily unset save_commit_buffer in is_promisor_object() when we start walking the objects. That's how we do it in other places, like 359b01ca84 (ref-filter: disable save_commit_buffer while traversing, 2022-07-11). But in this case since the traversal code is custom it's pretty easy to just free immediately afterwards.

-Peff