git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 6/9] pack-bitmap: expose function to iterate over bitmapped objects

From
Taylor Blau <me@ttaylorr.com>
Date
Feb 27, 2025, 23:23 UTC
Message-ID
<Z8Dz6EkIpr/g3vuY@nand.local>
In-Reply-To
<xmqqseo35ic8.fsf@gitster.g>
On Mon, Feb 24, 2025 at 10:05:27AM -0800, Junio C Hamano wrote:
Show 23 quoted lines
> Patrick Steinhardt <ps@pks.im> writes:
>
> > Expose a function that allows the caller to iterate over all bitmapped
> > objects of a specific type. This mechanism allows us to use the object
> > type-specific bitmaps to enumerate all objects of that type without
> > having to scan through a complete packfile.
> >
> > This functionality will be used in a subsequent commit.
> >
> > Signed-off-by: Patrick Steinhardt <ps@pks.im>
> > ---
> >  builtin/pack-objects.c |  3 ++-
> >  builtin/rev-list.c     |  3 ++-
> >  pack-bitmap.c          | 65 +++++++++++++++++++++++++++++++-------------------
> >  pack-bitmap.h          | 12 +++++++++-
> >  reachable.c            |  3 ++-
> >  5 files changed, 57 insertions(+), 29 deletions(-)
>
> After 2189649b (pack-bitmap.c: keep track of each layer's type
> bitmaps, 2024-11-19) added <type>_all bitmaps to the bitmap_index
> struct, this step would need some adjustment, I am afraid.
>
> Taylor Cc'ed.
Thanks, I was going to respond with the same thing.

I was going to suggest leaving that function as-is to prevent future breakage and/or a messy integration into 'seen'. But stepping back I am not sure I understand the purpose of this commit in the first place.

It looks like the aim here is to introduce a function which executes a callback for each object of some type in a bitmap. That's a thin wrapper over the ewah_iterator, but it's not clear why we need a wrapper around that function since it is internal to pack-bitmap.c. Likewise, this is a performance critical area, so I am not sure I'm in favor of adding a function pointer to a hot path which executes once per object for some object type.

Thanks, Taylor

Previous: Patrick SteinhardtNext: Junio C Hamano
Message 22 of 44 in “builtin/cat-file: allow filtering objects in batch mode”
  1. 0/9 builtin/cat-file: allow filtering objects in batch modePatrick Steinhardt, Feb 21, 2025
  2. 1/9 builtin/cat-file: rename variable that tracks usagePatrick Steinhardt, Feb 21, 2025
  3. 3/9 builtin/cat-file: support "blob:none" objects filterPatrick Steinhardt, Feb 21, 2025
  4. Toon ClaesFeb 26, 2025
  5. Karthik NayakFeb 27, 2025
  6. 2/9 builtin/cat-file: wire up an option to filter objectsPatrick Steinhardt, Feb 21, 2025
  7. Toon ClaesFeb 26, 2025
  8. Patrick SteinhardtFeb 28, 2025
  9. Junio C HamanoFeb 28, 2025
  10. Patrick SteinhardtMar 3, 2025
  11. Karthik NayakFeb 27, 2025
  12. 4/9 builtin/cat-file: support "blob:limit=" objects filterPatrick Steinhardt, Feb 21, 2025
  13. 5/9 builtin/cat-file: support "object:type=" objects filterPatrick Steinhardt, Feb 21, 2025
  14. Toon ClaesFeb 26, 2025
  15. Patrick SteinhardtFeb 28, 2025
  16. 6/9 pack-bitmap: expose function to iterate over bitmapped objectsPatrick Steinhardt, Feb 21, 2025
  17. Junio C HamanoFeb 24, 2025
  18. Patrick SteinhardtFeb 25, 2025
  19. Junio C HamanoFeb 25, 2025
  20. Taylor BlauFeb 27, 2025
  21. Patrick SteinhardtFeb 28, 2025
  22. Taylor BlauFeb 27, 2025
  23. Junio C HamanoFeb 27, 2025
  24. Taylor BlauFeb 27, 2025
  25. 8/9 builtin/cat-file: deduplicate logic to iterate over all objectsPatrick Steinhardt, Feb 21, 2025
  26. 7/9 pack-bitmap: introduce function to check whether a pack is bitmappedPatrick Steinhardt, Feb 21, 2025
  27. Taylor BlauFeb 27, 2025
  28. 9/9 builtin/cat-file: use bitmaps to efficiently filter by object typePatrick Steinhardt, Feb 21, 2025
  29. Karthik NayakFeb 27, 2025
  30. Taylor BlauFeb 27, 2025
  31. 00/11 builtin/cat-file: allow filtering objects in batch modePatrick Steinhardt, Apr 2, 2025
  32. 01/11 builtin/cat-file: rename variable that tracks usagePatrick Steinhardt, Apr 2, 2025
  33. 02/11 builtin/cat-file: introduce function to report object statusPatrick Steinhardt, Apr 2, 2025
  34. 03/11 builtin/cat-file: wire up an option to filter objectsPatrick Steinhardt, Apr 2, 2025
  35. 04/11 builtin/cat-file: support "blob:none" objects filterPatrick Steinhardt, Apr 2, 2025
  36. 05/11 builtin/cat-file: support "blob:limit=" objects filterPatrick Steinhardt, Apr 2, 2025
  37. 06/11 builtin/cat-file: support "object:type=" objects filterPatrick Steinhardt, Apr 2, 2025
  38. 07/11 pack-bitmap: allow passing payloads to `show_reachable_fn()`Patrick Steinhardt, Apr 2, 2025
  39. 08/11 pack-bitmap: add function to iterate over filtered bitmapped objectsPatrick Steinhardt, Apr 2, 2025
  40. 09/11 pack-bitmap: introduce function to check whether a pack is bitmappedPatrick Steinhardt, Apr 2, 2025
  41. 10/11 builtin/cat-file: deduplicate logic to iterate over all objectsPatrick Steinhardt, Apr 2, 2025
  42. 11/11 builtin/cat-file: use bitmaps to efficiently filter by object typePatrick Steinhardt, Apr 2, 2025
  43. Karthik NayakApr 3, 2025
  44. Junio C HamanoApr 8, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.