git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 5/8] builtin/pack-objects: simplify logic to find kept or nonlocal objects

From
Toon Claes <toon@iotcl.com>
Date
Oct 29, 2025, 14:55 UTC
Message-ID
<875xbxrc4q.fsf@iotcl.com>
In-Reply-To
<20251028-pks-packfiles-store-drop-list-v1-5-1a3b82030a7a@pks.im>
Patrick Steinhardt <ps@pks.im> writes:
Show 10 quoted lines
> The function `has_sha1_pack_kept_or_nonlocal()` takes an object ID and
> then searches through packed objects to figure out whether the object
> exists in a kept or non-local pack. As a performance optimization we
> remember the packfile that contains a given object ID so that the next
> call to the function first checks that same packfile again.
>
> The way this is written is rather hard to follow though, as the caching
> mechanism is intertwined with the loop that iterates through the packs.
> Consequently, we need to do some gymnastics to re-start the iteration if
> the cached pack does not contain the objects.

Okay, this took me while, but yes this function was really hard to understand. Thanks for simplifying.

Naive question, what's the point of keeping a "last_found"? We have one global "last_found" for the last time this function was called, and we have no control which OIDs get passed to this function. Why look into "last_found" first?

Show 53 quoted lines
> Refactor this so that we check the cached packfile at the beginning. We
> don't have to re-verify whether the packfile meets the properties as we
> have already verified those when storing the pack in `last_found` in the
> first place. So all we need to do is to use `find_pack_entry_one()` to
> check whether the pack contains the object ID, and to skip the cached
> pack in the loop so that we don't search it twice.
>
> This refactoring significantly simplifies the logic and makes it much
> easier to follow.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  builtin/pack-objects.c | 26 +++++++++++++-------------
>  1 file changed, 13 insertions(+), 13 deletions(-)
>
> diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
> index 5348aebbe9f..861fef3f38a 100644
> --- a/builtin/pack-objects.c
> +++ b/builtin/pack-objects.c
> @@ -4388,27 +4388,27 @@ static void add_unreachable_loose_objects(struct rev_info *revs)
>  
>  static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
>  {
> -	struct packfile_store *packs = the_repository->objects->packfiles;
>  	static struct packed_git *last_found = (void *)1;
>  	struct packed_git *p;
>  
> -	p = (last_found != (void *)1) ? last_found :
> -					packfile_store_get_packs(packs);
> +	if (last_found != (void *)1 && find_pack_entry_one(oid, last_found))
> +		return 1;
>  
> -	while (p) {
> -		if ((!p->pack_local || p->pack_keep ||
> -				p->pack_keep_in_core) &&
> -			find_pack_entry_one(oid, p)) {
> +	repo_for_each_pack(the_repository, p) {
> +		if ((!p->pack_local || p->pack_keep || p->pack_keep_in_core) &&
> +		    find_pack_entry_one(oid, p)) {
>  			last_found = p;
>  			return 1;
>  		}
> -		if (p == last_found)
> -			p = packfile_store_get_packs(packs);
> -		else
> -			p = p->next;
> -		if (p == last_found)
> -			p = p->next;
> +
> +		/*
> +		 * We have already checked `last_found`, so there is no need to
> +		 * re-check here.
> +		 */

I had to reason with myself why you need to extra `(void *)1` check, maybe you can extend the comment a bit:

		/*
		 * When `last_found` was set to something else then
		 * `(void *)1` we have already checked it,
		 * so there is no need to re-check here.
		 */
> +		if (p == last_found && last_found != (void *)1)
> +			continue;
-- 
Cheers,
Toon
Previous: Patrick SteinhardtNext: Taylor Blau
Message 14 of 34 in “packfiles: track pack lists via the packfile store”
  1. 0/8 packfiles: track pack lists via the packfile storePatrick Steinhardt, Oct 28, 2025
  2. 1/8 packfile: use a `strmap` to store packs by namePatrick Steinhardt, Oct 28, 2025
  3. Taylor BlauOct 29, 2025
  4. 2/8 packfile: move the MRU list into the packfile storePatrick Steinhardt, Oct 28, 2025
  5. Taylor BlauOct 29, 2025
  6. Patrick SteinhardtOct 30, 2025
  7. 3/8 http: refactor subsystem to use `packfile_list`sPatrick Steinhardt, Oct 28, 2025
  8. Toon ClaesOct 29, 2025
  9. Patrick SteinhardtOct 30, 2025
  10. 4/8 packfile: fix approximation of object countsPatrick Steinhardt, Oct 28, 2025
  11. Taylor BlauOct 29, 2025
  12. Patrick SteinhardtOct 30, 2025
  13. 5/8 builtin/pack-objects: simplify logic to find kept or nonlocal objectsPatrick Steinhardt, Oct 28, 2025
  14. Toon ClaesOct 29, 2025
  15. Taylor BlauOct 29, 2025
  16. Patrick SteinhardtOct 30, 2025
  17. Taylor BlauOct 29, 2025
  18. Patrick SteinhardtOct 30, 2025
  19. Toon ClaesOct 30, 2025
  20. Patrick SteinhardtOct 30, 2025
  21. 6/8 packfile: move list of packs into the packfile storePatrick Steinhardt, Oct 28, 2025
  22. 7/8 packfile: always add packfiles to MRU when adding a packPatrick Steinhardt, Oct 28, 2025
  23. Taylor BlauOct 29, 2025
  24. Patrick SteinhardtOct 30, 2025
  25. 8/8 packfile: track packs via the MRU list exclusivelyPatrick Steinhardt, Oct 28, 2025
  26. 0/8 packfiles: track pack lists via the packfile storePatrick Steinhardt, Oct 30, 2025
  27. 1/8 packfile: use a `strmap` to store packs by namePatrick Steinhardt, Oct 30, 2025
  28. 2/8 packfile: move the MRU list into the packfile storePatrick Steinhardt, Oct 30, 2025
  29. 3/8 http: refactor subsystem to use `packfile_list`sPatrick Steinhardt, Oct 30, 2025
  30. 4/8 packfile: fix approximation of object countsPatrick Steinhardt, Oct 30, 2025
  31. 5/8 builtin/pack-objects: simplify logic to find kept or nonlocal objectsPatrick Steinhardt, Oct 30, 2025
  32. 6/8 packfile: move list of packs into the packfile storePatrick Steinhardt, Oct 30, 2025
  33. 7/8 packfile: always add packfiles to MRU when adding a packPatrick Steinhardt, Oct 30, 2025
  34. 8/8 packfile: track packs via the MRU list exclusivelyPatrick Steinhardt, Oct 30, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.