git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] packfile: skip decompressing and hashing blobs in add_promisor_object()

From
Jeff King <peff@peff.net>
Date
Dec 5, 2025, 18:01 UTC
Message-ID
<20251205180106.GC18566@coredump.intra.peff.net>
In-Reply-To
<20251205174854.GA18566@coredump.intra.peff.net>
On Fri, Dec 05, 2025 at 12:48:54PM -0500, Jeff King wrote:
Show 30 quoted lines
> OK, so we are checking the type up front and then skipping
> parse_object() if we can. But there is already some logic inside
> parse_object() for these kinds of optimizations. If we tell it we are
> not interested in checking the hash of the objects, then it knows it can
> skip loading the blob entirely.
> 
> But it can _also_ use that flag for other things, like using the
> commit-graph rather than loading individual commit objects. So doing
> this:
>   
> diff --git a/packfile.c b/packfile.c
> index 9cc11b6dc5..01b992a4e1 100644
> --- a/packfile.c
> +++ b/packfile.c
> @@ -2310,7 +2310,8 @@ static int add_promisor_object(const struct object_id *oid,
>  		we_parsed_object = 0;
>  	} else {
>  		we_parsed_object = 1;
> -		obj = parse_object(pack->repo, oid);
> +		obj = parse_object_with_flags(pack->repo, oid,
> +					      PARSE_OBJECT_SKIP_HASH_CHECK);
>  	}
>  
>  	if (!obj)
> 
> drops my linux.git case down to 49s. It's skipping the blobs (with no
> need for your patch) and loading the commits out of the graph file. Note
> that you may need to "git commit-graph write --reachable" to see the
> effect (I think we do generate graphs by default in git-gc these days,
> but I'm not sure if we do so right after cloning).

Oh, and obviously it is skipping the hash computation on the objects, too. That's probably not as important as avoiding the object loads in the first place, but it may also be making a measurable difference on the ones we do load (notably trees here).

-Peff
Previous: Jeff KingNext: Aaron Plattner
Message 6 of 10 in “packfile: skip decompressing and hashing blobs in add_promisor_object()”
  1. packfile: skip decompressing and hashing blobs in add_promisor_object()Aaron Plattner, Dec 4, 2025
  2. Patrick SteinhardtDec 5, 2025
  3. Aaron PlattnerDec 5, 2025
  4. Jeff KingDec 5, 2025
  5. Jeff KingDec 5, 2025
  6. Jeff KingDec 5, 2025
  7. Aaron PlattnerDec 5, 2025
  8. Jeff KingDec 5, 2025
  9. Aaron PlattnerDec 5, 2025
  10. Jeff KingDec 6, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.