threads / patch / 63984

patch, 16 partspackfile: carve out a new packfile store

Subject: [PATCH 00/16] packfile: carve out a new packfile store

## tl;dr

181 messages between Aug 19, 2025 and Sep 29, 2025. Diffs are folded; open one to read it.

replies: 180people: 6as markdown or json

Patrick Steinhardt· Aug 19, 2025, 08:19 UTC · lore
Hi,

information about a object database's packfiles is currently distributed across two different structures:

  - `struct packed_git` contains the `next` pointer as well as the
    `mru_head`, both of which serve to store the list of packfiles.
  - `struct object_database` contains several fields that relate to the
    packfiles.

So we don't really have a central data structure that tracks our packfiles, and consequently responsibilities aren't always clear cut. A consequence for the upcoming pluggable object databases is that this makes it very hard to move management of packfiles from the object database level down into the object database source.

This patch series introduces a new `struct packfile_store`, which is about to become the single source of truth for managing packfiles, and carves out the packfile store subsystem.

This is the first step to make packfiles work with pluggable object databases. Next steps will be to:

  - Move the `struct packed_git::next` and `struct packed::mru_head`
    pointers into the packfile store so that `struct packed_git` only
    tracks a single packfile.
  - Push the `struct packfile_store` down one level so that it's not
    hosted by the object database anymore, but instead by the object
    database source.
Thanks!
Patrick
---
Patrick Steinhardt (16):
      packfile: introduce a new `struct packfile_store`
      odb: move list of packfiles into `struct packfile_store`
      odb: move initialization bit into `struct packfile_store`
      odb: move packfile map into `struct packfile_store`
      odb: move MRU list of packfiles into `struct packfile_store`
      odb: move kept cache into `struct packfile_store`
      packfile: reorder functions to avoid function declaration
      packfile: refactor `prepare_packed_git()` to work on packfile store
      packfile: split up responsibilities of `reprepare_packed_git()`
      packfile: refactor `install_packed_git()` to work on packfile store
      packfile: always add packfiles to MRU when adding a pack
      packfile: introduce function to load and add packfiles
      packfile: move `get_multi_pack_index()` into "midx.c"
      packfile: remove `get_packed_git()`
      packfile: refactor `get_all_packs()` to work on packfile store
      packfile: refactor `get_packed_git_mru()` to work on packfile store
 builtin/backfill.c          |   2 +-
 builtin/cat-file.c          |   2 +-
 builtin/count-objects.c     |   2 +-
 builtin/fast-import.c       |   8 +-
 builtin/fsck.c              |   8 +-
 builtin/gc.c                |  12 +-
 builtin/grep.c              |   2 +-
 builtin/index-pack.c        |  10 +-
 builtin/pack-objects.c      |  22 ++--
 builtin/pack-redundant.c    |   4 +-
 builtin/receive-pack.c      |   2 +-
 builtin/repack.c            |   8 +-
 bulk-checkin.c              |   2 +-
 connected.c                 |   4 +-
 fetch-pack.c                |   4 +-
 http-backend.c              |   4 +-
 http.c                      |   4 +-
 http.h                      |   2 +-
 midx.c                      |  26 ++--
 midx.h                      |   2 +
 object-name.c               |   6 +-
 odb.c                       |  37 ++++--
 odb.h                       |  34 ++---
 pack-bitmap.c               |   4 +-
 pack-objects.c              |   2 +-
 packfile.c                  | 293 ++++++++++++++++++++++++--------------------
 packfile.h                  | 111 ++++++++++++++---
 server-info.c               |   2 +-
 t/helper/test-find-pack.c   |   2 +-
 t/helper/test-pack-mtimes.c |   2 +-
 transport-helper.c          |   2 +-
 31 files changed, 354 insertions(+), 271 deletions(-)

--- base-commit: c44beea485f0f2feaf460e2ac87fdd5608d63cf0 change-id: 20250806-b4-pks-packfiles-store-a44a608ca396

Patrick Steinhardt· Aug 19, 2025, 08:19 UTC · re: Patrick Steinhardt · lore

[PATCH 01/16] packfile: introduce a new `struct packfile_store`

Information about a object database's packfiles is currently distributed across two different structures:

  - `struct packed_git` contains the `next` pointer as well as the
    `mru_head`, both of which serve to store the list of packfiles.
  - `struct object_database` contains several fields that relate to the
    packfiles.

So we don't really have a central data structure that tracks our packfiles, and consequently responsibilities aren't always clear cut. A consequence for the upcoming pluggable object databases is that this makes it very hard to move management of packfiles from the object database level down into the object database source.

Introduce a new `struct packfile_store` which is about to become the single source of truth for managing packfiles. Right now this data structure doesn't yet contain anything, but in subsequent patches we will move all data structures that relate to packfiles and that are currently contained in `struct object_database` into this new home.

Note that this is only a first step: most importantly, we won't (yet) move the `struct packed_git::next` pointer around. This will happen in a subsequent patch series though so that `struct packed_git` will really only host information about the specific packfile it represents.

Further note that the new structure still sits at the wrong level at the end of this patch series: as mentioned, it should eventually sit at the level of the object database source, not at the object database level. But introducing the packfile store now already makes it way easier to eventually push down the now-selfcontained data structure by one level.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb.c      |  1 +
 odb.h      |  2 ++
 packfile.c | 13 +++++++++++++
 packfile.h | 18 ++++++++++++++++++
 4 files changed, 34 insertions(+)
Show changes to 4 files +34 −0

odb.c, odb.h, packfile.c, packfile.h

diff --git a/odb.c b/odb.c
index 2a92a018c4..34b70d0074 100644
--- a/odb.c
+++ b/odb.c
@@ -996,6 +996,7 @@ struct object_database *odb_new(struct repository *repo)
 
 	memset(o, 0, sizeof(*o));
 	o->repo = repo;
+	o->packfiles = packfile_store_new(o);
 	INIT_LIST_HEAD(&o->packed_git_mru);
 	hashmap_init(&o->pack_map, pack_map_entry_cmp, NULL, 0);
 	pthread_mutex_init(&o->replace_mutex, NULL);
diff --git a/odb.h b/odb.h
index 3dfc66d75a..026ba9386d 100644
--- a/odb.h
+++ b/odb.h
@@ -83,6 +83,7 @@ struct odb_source {
 };
 
 struct packed_git;
+struct packfile_store;
 struct cached_object_entry;
 
 /*
@@ -128,6 +129,7 @@ struct object_database {
 	 *
 	 * should only be accessed directly by packfile.c
 	 */
+	struct packfile_store *packfiles;
 
 	struct packed_git *packed_git;
 	/* A most-recently-used ordered version of the packed_git list. */
diff --git a/packfile.c b/packfile.c
index 5d73932f50..8fbf1cfc2d 100644
--- a/packfile.c
+++ b/packfile.c
@@ -2333,3 +2333,16 @@ int parse_pack_header_option(const char *in, unsigned char *out, unsigned int *l
 	*len = hdr - out;
 	return 0;
 }
+
+struct packfile_store *packfile_store_new(struct object_database *odb)
+{
+	struct packfile_store *store;
+	CALLOC_ARRAY(store, 1);
+	store->odb = odb;
+	return store;
+}
+
+void packfile_store_free(struct packfile_store *store)
+{
+	free(store);
+}
diff --git a/packfile.h b/packfile.h
index f16753f2a9..8d31fd619a 100644
--- a/packfile.h
+++ b/packfile.h
@@ -52,6 +52,24 @@ struct packed_git {
 	char pack_name[FLEX_ARRAY]; /* more */
 };
 
+/*
+ * A store that manages packfiles for a given object database.
+ */
+struct packfile_store {
+	struct object_database *odb;
+};
+
+/*
+ * Allocate and initialize a new empty packfile store for the given object
+ * database.
+ */
+struct packfile_store *packfile_store_new(struct object_database *odb);
+
+/*
+ * Free the packfile store and all its associated state.
+ */
+void packfile_store_free(struct packfile_store *store);
+
 static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
 				     const struct hashmap_entry *entry,
 				     const struct hashmap_entry *entry2,
-- 
2.51.0.261.g7ce5a0a67e.dirty
Karthik Nayak· Aug 19, 2025, 09:47 UTC · re: Patrick Steinhardt · lore

Re: [PATCH 01/16] packfile: introduce a new `struct packfile_store`

Patrick Steinhardt <ps@pks.im> writes:
Show 70 quoted lines
> Information about a object database's packfiles is currently distributed
> across two different structures:
>
>   - `struct packed_git` contains the `next` pointer as well as the
>     `mru_head`, both of which serve to store the list of packfiles.
>
>   - `struct object_database` contains several fields that relate to the
>     packfiles.
>
> So we don't really have a central data structure that tracks our
> packfiles, and consequently responsibilities aren't always clear cut.
> A consequence for the upcoming pluggable object databases is that this
> makes it very hard to move management of packfiles from the object
> database level down into the object database source.
>
> Introduce a new `struct packfile_store` which is about to become the
> single source of truth for managing packfiles. Right now this data
> structure doesn't yet contain anything, but in subsequent patches we
> will move all data structures that relate to packfiles and that are
> currently contained in `struct object_database` into this new home.
>
> Note that this is only a first step: most importantly, we won't (yet)
> move the `struct packed_git::next` pointer around. This will happen in a
> subsequent patch series though so that `struct packed_git` will really
> only host information about the specific packfile it represents.
>
> Further note that the new structure still sits at the wrong level at the
> end of this patch series: as mentioned, it should eventually sit at the
> level of the object database source, not at the object database level.
> But introducing the packfile store now already makes it way easier to
> eventually push down the now-selfcontained data structure by one level.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  odb.c      |  1 +
>  odb.h      |  2 ++
>  packfile.c | 13 +++++++++++++
>  packfile.h | 18 ++++++++++++++++++
>  4 files changed, 34 insertions(+)
>
> diff --git a/odb.c b/odb.c
> index 2a92a018c4..34b70d0074 100644
> --- a/odb.c
> +++ b/odb.c
> @@ -996,6 +996,7 @@ struct object_database *odb_new(struct repository *repo)
>
>  	memset(o, 0, sizeof(*o));
>  	o->repo = repo;
> +	o->packfiles = packfile_store_new(o);
>  	INIT_LIST_HEAD(&o->packed_git_mru);
>  	hashmap_init(&o->pack_map, pack_map_entry_cmp, NULL, 0);
>  	pthread_mutex_init(&o->replace_mutex, NULL);
> diff --git a/odb.h b/odb.h
> index 3dfc66d75a..026ba9386d 100644
> --- a/odb.h
> +++ b/odb.h
> @@ -83,6 +83,7 @@ struct odb_source {
>  };
>
>  struct packed_git;
> +struct packfile_store;
>  struct cached_object_entry;
>
>  /*
> @@ -128,6 +129,7 @@ struct object_database {
>  	 *
>  	 * should only be accessed directly by packfile.c
>  	 */
> +	struct packfile_store *packfiles;
>
Nit: The newline spacing makes it seem like the comment above only
applies to `struct packfile_store` while actually it also applies to
`struct packed_git`.
>  	struct packed_git *packed_git;
>  	/* A most-recently-used ordered version of the packed_git list. */
[snip]
Patrick Steinhardt· Aug 20, 2025, 04:58 UTC · re: Karthik Nayak · lore

Re: [PATCH 01/16] packfile: introduce a new `struct packfile_store`

On Tue, Aug 19, 2025 at 02:47:32AM -0700, Karthik Nayak wrote:
Show 15 quoted lines
> Patrick Steinhardt <ps@pks.im> writes:
> > diff --git a/odb.h b/odb.h
> > index 3dfc66d75a..026ba9386d 100644
> > --- a/odb.h
> > +++ b/odb.h
> > @@ -128,6 +129,7 @@ struct object_database {
> >  	 *
> >  	 * should only be accessed directly by packfile.c
> >  	 */
> > +	struct packfile_store *packfiles;
> >
> 
> Nit: The newline spacing makes it seem like the comment above only
> applies to `struct packfile_store` while actually it also applies to
> `struct packed_git`.

Fair. The remaining structs will go away over subsequent commits anyway, but we can still make this more obvious in this first step.

Patrick
Junio C Hamano· Aug 19, 2025, 17:32 UTC · re: Patrick Steinhardt · lore

Re: [PATCH 01/16] packfile: introduce a new `struct packfile_store`

Patrick Steinhardt <ps@pks.im> writes:
>  	memset(o, 0, sizeof(*o));
>  	o->repo = repo;
> +	o->packfiles = packfile_store_new(o);

Shouldn't this be called o->packfile_store? It is not like a packfile_store is merely an array of packfile struct, is it?

Show 5 quoted lines
> @@ -128,6 +129,7 @@ struct object_database {
>  	 *
>  	 * should only be accessed directly by packfile.c
>  	 */
> +	struct packfile_store *packfiles;

So odb has a pointer to packfile_store, which in turn has a pointer to a(nother) odb?

Hmph. It is unclear what this step has achieved (in other words, there is no obvious thing that the information stored in the new structure is used to achieve at this step). Let me read on.

Thanks.
Patrick Steinhardt· Aug 20, 2025, 04:58 UTC · re: Junio C Hamano · lore

Re: [PATCH 01/16] packfile: introduce a new `struct packfile_store`

On Tue, Aug 19, 2025 at 10:32:20AM -0700, Junio C Hamano wrote:
Show 8 quoted lines
> Patrick Steinhardt <ps@pks.im> writes:
> 
> >  	memset(o, 0, sizeof(*o));
> >  	o->repo = repo;
> > +	o->packfiles = packfile_store_new(o);
> 
> Shouldn't this be called o->packfile_store?  It is not like a
> packfile_store is merely an array of packfile struct, is it?

I was mostly aiming for brevity as there's going to be a bunch of sites that access the variable. But if you feel strongly about it I can adapt.

Show 8 quoted lines
> > @@ -128,6 +129,7 @@ struct object_database {
> >  	 *
> >  	 * should only be accessed directly by packfile.c
> >  	 */
> > +	struct packfile_store *packfiles;
> 
> So odb has a pointer to packfile_store, which in turn has a pointer
> to a(nother) odb?

The ODB has a pointer to a packfile store, and that store has a pointer to its owning ODB.

> Hmph.  It is unclear what this step has achieved (in other words,
> there is no obvious thing that the information stored in the new
> structure is used to achieve at this step).  Let me read on.

Well, this series really only cares about encapsulating access to packfiles so that it becomes easier in subsequent patch series to move the data structures around. So next to carving out the subsystem we don't achieve much, but it's a necessary prerequisite.

Patrick
Patrick Steinhardt· Aug 19, 2025, 08:19 UTC · re: Patrick Steinhardt · lore

[PATCH 02/16] odb: move list of packfiles into `struct packfile_store`

The object database tracks the list of packfiles it currently knows about. With the introduction of the `struct packfile_store` we have a better place to host this list though.

Move the list accordingly. Extract the logic from `odb_clear()` that knows to close all such packfiles and move it into the new subsystem, as well.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb.c      | 11 +----------
 odb.h      |  1 -
 packfile.c | 47 ++++++++++++++++++++++++++++++-----------------
 packfile.h | 16 +++++++++++++++-
 4 files changed, 46 insertions(+), 29 deletions(-)
Show changes to 4 files +46 −29

odb.c, odb.h, packfile.c, packfile.h

diff --git a/odb.c b/odb.c
index 34b70d0074..17a9135cbd 100644
--- a/odb.c
+++ b/odb.c
@@ -1038,16 +1038,7 @@ void odb_clear(struct object_database *o)
 
 	INIT_LIST_HEAD(&o->packed_git_mru);
 	close_object_store(o);
-
-	/*
-	 * `close_object_store()` only closes the packfiles, but doesn't free
-	 * them. We thus have to do this manually.
-	 */
-	for (struct packed_git *p = o->packed_git, *next; p; p = next) {
-		next = p->next;
-		free(p);
-	}
-	o->packed_git = NULL;
+	packfile_store_free(o->packfiles);
 
 	hashmap_clear(&o->pack_map);
 	string_list_clear(&o->submodule_source_paths, 0);
diff --git a/odb.h b/odb.h
index 026ba9386d..273ad0ceaa 100644
--- a/odb.h
+++ b/odb.h
@@ -131,7 +131,6 @@ struct object_database {
 	 */
 	struct packfile_store *packfiles;
 
-	struct packed_git *packed_git;
 	/* A most-recently-used ordered version of the packed_git list. */
 	struct list_head packed_git_mru;
 
diff --git a/packfile.c b/packfile.c
index 8fbf1cfc2d..6478e4cc30 100644
--- a/packfile.c
+++ b/packfile.c
@@ -278,7 +278,7 @@ static int unuse_one_window(struct packed_git *current)
 
 	if (current)
 		scan_windows(current, &lru_p, &lru_w, &lru_l);
-	for (p = current->repo->objects->packed_git; p; p = p->next)
+	for (p = current->repo->objects->packfiles->packs; p; p = p->next)
 		scan_windows(p, &lru_p, &lru_w, &lru_l);
 	if (lru_p) {
 		munmap(lru_w->base, lru_w->len);
@@ -362,13 +362,8 @@ void close_pack(struct packed_git *p)
 void close_object_store(struct object_database *o)
 {
 	struct odb_source *source;
-	struct packed_git *p;
 
-	for (p = o->packed_git; p; p = p->next)
-		if (p->do_not_close)
-			BUG("want to close pack marked 'do-not-close'");
-		else
-			close_pack(p);
+	packfile_store_close(o->packfiles);
 
 	for (source = o->sources; source; source = source->next) {
 		if (source->midx)
@@ -468,7 +463,7 @@ static int close_one_pack(struct repository *r)
 	struct pack_window *mru_w = NULL;
 	int accept_windows_inuse = 1;
 
-	for (p = r->objects->packed_git; p; p = p->next) {
+	for (p = r->objects->packfiles->packs; p; p = p->next) {
 		if (p->pack_fd == -1)
 			continue;
 		find_lru_pack(p, &lru_p, &mru_w, &accept_windows_inuse);
@@ -789,8 +784,8 @@ void install_packed_git(struct repository *r, struct packed_git *pack)
 	if (pack->pack_fd != -1)
 		pack_open_fds++;
 
-	pack->next = r->objects->packed_git;
-	r->objects->packed_git = pack;
+	pack->next = r->objects->packfiles->packs;
+	r->objects->packfiles->packs = pack;
 
 	hashmap_entry_init(&pack->packmap_ent, strhash(pack->pack_name));
 	hashmap_add(&r->objects->pack_map, &pack->packmap_ent);
@@ -974,7 +969,7 @@ unsigned long repo_approximate_object_count(struct repository *r)
 				count += m->num_objects;
 		}
 
-		for (p = r->objects->packed_git; p; p = p->next) {
+		for (p = r->objects->packfiles->packs; p; p = p->next) {
 			if (open_pack_index(p))
 				continue;
 			count += p->num_objects;
@@ -1015,7 +1010,7 @@ static int sort_pack(const struct packed_git *a, const struct packed_git *b)
 
 static void rearrange_packed_git(struct repository *r)
 {
-	sort_packs(&r->objects->packed_git, sort_pack);
+	sort_packs(&r->objects->packfiles->packs, sort_pack);
 }
 
 static void prepare_packed_git_mru(struct repository *r)
@@ -1024,7 +1019,7 @@ static void prepare_packed_git_mru(struct repository *r)
 
 	INIT_LIST_HEAD(&r->objects->packed_git_mru);
 
-	for (p = r->objects->packed_git; p; p = p->next)
+	for (p = r->objects->packfiles->packs; p; p = p->next)
 		list_add_tail(&p->mru, &r->objects->packed_git_mru);
 }
 
@@ -1074,7 +1069,7 @@ void reprepare_packed_git(struct repository *r)
 struct packed_git *get_packed_git(struct repository *r)
 {
 	prepare_packed_git(r);
-	return r->objects->packed_git;
+	return r->objects->packfiles->packs;
 }
 
 struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
@@ -1095,7 +1090,7 @@ struct packed_git *get_all_packs(struct repository *r)
 			prepare_midx_pack(r, m, i);
 	}
 
-	return r->objects->packed_git;
+	return r->objects->packfiles->packs;
 }
 
 struct list_head *get_packed_git_mru(struct repository *r)
@@ -1220,7 +1215,7 @@ const struct packed_git *has_packed_and_bad(struct repository *r,
 {
 	struct packed_git *p;
 
-	for (p = r->objects->packed_git; p; p = p->next)
+	for (p = r->objects->packfiles->packs; p; p = p->next)
 		if (oidset_contains(&p->bad_objects, oid))
 			return p;
 	return NULL;
@@ -2081,7 +2076,7 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 		if (source->midx && fill_midx_entry(r, oid, e, source->midx))
 			return 1;
 
-	if (!r->objects->packed_git)
+	if (!r->objects->packfiles->packs)
 		return 0;
 
 	list_for_each(pos, &r->objects->packed_git_mru) {
@@ -2344,5 +2339,23 @@ struct packfile_store *packfile_store_new(struct object_database *odb)
 
 void packfile_store_free(struct packfile_store *store)
 {
+	packfile_store_close(store);
+
+	for (struct packed_git *p = store->packs, *next; p; p = next) {
+		next = p->next;
+		free(p);
+	}
+
 	free(store);
 }
+
+void packfile_store_close(struct packfile_store *store)
+{
+	struct packed_git *p;
+
+	for (p = store->packs; p; p = p->next)
+		if (p->do_not_close)
+			BUG("want to close pack marked 'do-not-close'");
+		else
+			close_pack(p);
+}
diff --git a/packfile.h b/packfile.h
index 8d31fd619a..1404b80917 100644
--- a/packfile.h
+++ b/packfile.h
@@ -57,6 +57,13 @@ struct packed_git {
  */
 struct packfile_store {
 	struct object_database *odb;
+
+	/*
+	 * The list of packfiles in the order in which they are being added to
+	 * the store. The local packfile typically sits at the head of this
+	 * list.
+	 */
+	struct packed_git *packs;
 };
 
 /*
@@ -66,10 +73,17 @@ struct packfile_store {
 struct packfile_store *packfile_store_new(struct object_database *odb);
 
 /*
- * Free the packfile store and all its associated state.
+ * Free the packfile store and all its associated state. All packfiles
+ * tracked by the store will be closed.
  */
 void packfile_store_free(struct packfile_store *store);
 
+/*
+ * Close all packfiles associated with this store. The packfiles won't be
+ * free'd, so they can be re-opened at a later point in time.
+ */
+void packfile_store_close(struct packfile_store *store);
+
 static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
 				     const struct hashmap_entry *entry,
 				     const struct hashmap_entry *entry2,
-- 
2.51.0.261.g7ce5a0a67e.dirty
Patrick Steinhardt· Aug 19, 2025, 08:19 UTC · re: Patrick Steinhardt · lore

[PATCH 03/16] odb: move initialization bit into `struct packfile_store`

The object database knows to skip re-initializing the list of packfiles in case it's already been initialized. Whether or not that is the case is tracked via a separate `initialized` bit that is stored in the object database. With the introduction of the `struct packfile_store` we have a better place to host this bit though.

Move it accordingly.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb.h      | 6 ------
 packfile.c | 6 +++---
 packfile.h | 6 ++++++
 3 files changed, 9 insertions(+), 9 deletions(-)
Show changes to 3 files +9 −9

odb.h, packfile.c, packfile.h

diff --git a/odb.h b/odb.h
index 273ad0ceaa..970919403d 100644
--- a/odb.h
+++ b/odb.h
@@ -162,12 +162,6 @@ struct object_database {
 	unsigned long approximate_object_count;
 	unsigned approximate_object_count_valid : 1;
 
-	/*
-	 * Whether packed_git has already been populated with this repository's
-	 * packs.
-	 */
-	unsigned packed_git_initialized : 1;
-
 	/*
 	 * Submodule source paths that will be added as additional sources to
 	 * allow lookup of submodule objects via the main object database.
diff --git a/packfile.c b/packfile.c
index 6478e4cc30..4e5f84eb09 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1027,7 +1027,7 @@ static void prepare_packed_git(struct repository *r)
 {
 	struct odb_source *source;
 
-	if (r->objects->packed_git_initialized)
+	if (r->objects->packfiles->initialized)
 		return;
 
 	odb_prepare_alternates(r->objects);
@@ -1039,7 +1039,7 @@ static void prepare_packed_git(struct repository *r)
 	rearrange_packed_git(r);
 
 	prepare_packed_git_mru(r);
-	r->objects->packed_git_initialized = 1;
+	r->objects->packfiles->initialized = 1;
 }
 
 void reprepare_packed_git(struct repository *r)
@@ -1061,7 +1061,7 @@ void reprepare_packed_git(struct repository *r)
 		odb_clear_loose_cache(source);
 
 	r->objects->approximate_object_count_valid = 0;
-	r->objects->packed_git_initialized = 0;
+	r->objects->packfiles->initialized = 0;
 	prepare_packed_git(r);
 	obj_read_unlock();
 }
diff --git a/packfile.h b/packfile.h
index 1404b80917..573564b19e 100644
--- a/packfile.h
+++ b/packfile.h
@@ -64,6 +64,12 @@ struct packfile_store {
 	 * list.
 	 */
 	struct packed_git *packs;
+
+	/*
+	 * Whether packfiles have already been populated with this store's
+	 * packs.
+	 */
+	unsigned initialized : 1;
 };
 
 /*
-- 
2.51.0.261.g7ce5a0a67e.dirty
Karthik Nayak· Aug 19, 2025, 09:57 UTC · re: Patrick Steinhardt · lore

Re: [PATCH 03/16] odb: move initialization bit into `struct packfile_store`

Patrick Steinhardt <ps@pks.im> writes:
Show 16 quoted lines
> diff --git a/packfile.h b/packfile.h
> index 1404b80917..573564b19e 100644
> --- a/packfile.h
> +++ b/packfile.h
> @@ -64,6 +64,12 @@ struct packfile_store {
>  	 * list.
>  	 */
>  	struct packed_git *packs;
> +
> +	/*
> +	 * Whether packfiles have already been populated with this store's
> +	 * packs.
> +	 */
> +	unsigned initialized : 1;
>  };
>
Nit: I know this is moved from existing code, but might be nice to
adhere to our format rules here and remove spaces around the bit field.
Tangent: Also this is something that is only mentioned in the
'.clang-format' but not in any of our documentation, should we add it to
the documentation? Usage seems to be around the same for both types.
>  /*
>
> --
> 2.51.0.261.g7ce5a0a67e.dirty
Junio C Hamano· Aug 19, 2025, 16:24 UTC · re: Karthik Nayak · lore

Re: [PATCH 03/16] odb: move initialization bit into `struct packfile_store`

Karthik Nayak <karthik.188@gmail.com> writes:
> Tangent: Also this is something that is only mentioned in the
> '.clang-format' but not in any of our documentation, should we add it to
> the documentation? Usage seems to be around the same for both types.

It merely means that whatever .clang-format does is forcing changes to half of existing code base without any developer input, let alone conseusus, doesn't it?

A quick look around does indicate that with spaces around both sides were dominant in the code base early days (like 1.0.0), but that dominance eroded fairly quickly and by the time 1.6.0 was released it was already half-half (24 among 43 are with spaces). As you reported, among 216 hits for "^[ ]*unsigned .*:.*;" in header files (in 2.50.0), 105 of them are with a space after that colon, and the rest without, so it is really about the same, indeed.

I think it is a good idea to just pick one for new code and stick to it, and if we can do without churning existing code, that would be great.

I have personal preferences, and usually I'd like to hear from others first before mentioning my preference, but for something this small and does not affect readability very much, perhaps I can just pick and dictate? I dunno ;-).

Karthik Nayak· Aug 20, 2025, 08:04 UTC · re: Junio C Hamano · lore

Re: [PATCH 03/16] odb: move initialization bit into `struct packfile_store`

Junio C Hamano <gitster@pobox.com> writes:
Show 10 quoted lines
> Karthik Nayak <karthik.188@gmail.com> writes:
>
>> Tangent: Also this is something that is only mentioned in the
>> '.clang-format' but not in any of our documentation, should we add it to
>> the documentation? Usage seems to be around the same for both types.
>
> It merely means that whatever .clang-format does is forcing changes
> to half of existing code base without any developer input, let alone
> conseusus, doesn't it?
>
Yup indeed.
Show 12 quoted lines
> A quick look around does indicate that with spaces around both sides
> were dominant in the code base early days (like 1.0.0), but that
> dominance eroded fairly quickly and by the time 1.6.0 was released
> it was already half-half (24 among 43 are with spaces).  As you
> reported, among 216 hits for "^[ ]*unsigned .*:.*;" in header files
> (in 2.50.0), 105 of them are with a space after that colon, and the
> rest without, so it is really about the same, indeed.
>
> I think it is a good idea to just pick one for new code and stick to
> it, and if we can do without churning existing code, that would be
> great.
>

My thoughts here too. I do like the spaced version since it reads better for me, but in the end, I care more for consistency.

> I have personal preferences, and usually I'd like to hear from
> others first before mentioning my preference, but for something this
> small and does not affect readability very much, perhaps I can just
> pick and dictate?  I dunno ;-).

I wouldn't mind if you picked one over the other, like I mentioned, I care more that we make it consistent and that the formatter can notify or fix it for us.

Junio C Hamano· Aug 22, 2025, 23:50 UTC · re: Karthik Nayak · lore

Re: [PATCH 03/16] odb: move initialization bit into `struct packfile_store`

Karthik Nayak <karthik.188@gmail.com> writes:
Show 8 quoted lines
>> I have personal preferences, and usually I'd like to hear from
>> others first before mentioning my preference, but for something this
>> small and does not affect readability very much, perhaps I can just
>> pick and dictate?  I dunno ;-).
>
> I wouldn't mind if you picked one over the other, like I mentioned, I
> care more that we make it consistent and that the formatter can notify
> or fix it for us.
Then let's declare that these shall be written like so:
    unsigned my_field:1;
    unsigned other_field:1;
    unsigned field_with_longer_name:1;

without a space around the colon. It would allow us not to modify the clang-format file, and more importantly, discourage people from doing ugly alignment with spaces, i.e.

    unsigned my_field               : 1;
    unsigned            other_field : 1;
    unsigned field_with_longer_name : 1;
Karthik Nayak· Aug 26, 2025, 12:19 UTC · re: Junio C Hamano · lore

[PATCH] Documentation: note styling for bit fields

Our codebase uses a lot of bit field variables, generally to mark boolean type variables. While there is a formatting rule in the '.clang-format', there is no guideline specified in the 'CodingGuidelines'.

Since the '.clang-format' is not yet enforced, let's also add a guideline with the same rule as mentioned in the '.clang-format', which is to not use any spaces around the colon, like so:

    unsigned my_field:1;
    unsigned other_field:1;
    unsigned field_with_longer_name:1;

This would allow us not to modify the clang-format file, and more importantly, discourage people from doing ugly alignment with spaces, i.e.

    unsigned my_field               : 1;
    unsigned            other_field : 1;
    unsigned field_with_longer_name : 1;
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---

I think it would be worthwhile to also add this decision to the 'CodingGuidelines', but if you feel it is unnecessary, feel free to drop it.

 Documentation/CodingGuidelines | 6 ++++++
 1 file changed, 6 insertions(+)
Show changes to Documentation/CodingGuidelines +6 −0
diff --git a/Documentation/CodingGuidelines b/Documentation/CodingGuidelines
index 224f0978a8..df72fe0177 100644
--- a/Documentation/CodingGuidelines
+++ b/Documentation/CodingGuidelines
@@ -650,6 +650,12 @@ For C programs:
    cases. However, it is recommended to find a more descriptive name wherever
    possible to improve the readability and maintainability of the code.
 
+ - Bit fields should be defined without a space around the colon. E.g.
+
+   unsigned my_field:1;
+   unsigned other_field:1;
+   unsigned field_with_longer_name:1;
+
 For Perl programs:
 
  - Most of the C guidelines above apply.
-- 
2.50.1
Patrick Steinhardt· Aug 20, 2025, 04:58 UTC · re: Karthik Nayak · lore

Re: [PATCH 03/16] odb: move initialization bit into `struct packfile_store`

On Tue, Aug 19, 2025 at 02:57:54AM -0700, Karthik Nayak wrote:
Show 25 quoted lines
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > diff --git a/packfile.h b/packfile.h
> > index 1404b80917..573564b19e 100644
> > --- a/packfile.h
> > +++ b/packfile.h
> > @@ -64,6 +64,12 @@ struct packfile_store {
> >  	 * list.
> >  	 */
> >  	struct packed_git *packs;
> > +
> > +	/*
> > +	 * Whether packfiles have already been populated with this store's
> > +	 * packs.
> > +	 */
> > +	unsigned initialized : 1;
> >  };
> >
> 
> Nit: I know this is moved from existing code, but might be nice to
> adhere to our format rules here and remove spaces around the bit field.
> 
> Tangent: Also this is something that is only mentioned in the
> '.clang-format' but not in any of our documentation, should we add it to
> the documentation? Usage seems to be around the same for both types.

Well, now that booleans are allowed I think we should just stop using width specifiers like this altogether and instead use bool. There's probably still going to be cases where we use those, but I assume that the majority of users of this syntax is for flags.

Patrick
Junio C Hamano· Aug 20, 2025, 06:24 UTC · re: Patrick Steinhardt · lore

Re: [PATCH 03/16] odb: move initialization bit into `struct packfile_store`

Patrick Steinhardt <ps@pks.im> writes:
> Well, now that booleans are allowed I think we should just stop using
> width specifiers like this altogether and instead use bool. There's
> probably still going to be cases where we use those, but I assume that
> the majority of users of this syntax is for flags.

Yeah, unless the structure with these members are designed to exist in the millions in core (like "struct object" and its descendants), bool is fine.

Patrick Steinhardt· Aug 19, 2025, 08:19 UTC · re: Patrick Steinhardt · lore

[PATCH 04/16] odb: move packfile map into `struct packfile_store`

The object database tracks a map of packfiles by their respective paths, which is used to figure out whether a given packfile has already been loaded.With the introduction of the `struct packfile_store` we have a better place to host this list though.

Move the map accordingly. `pack_map_entry_cmp()` isn't used anywhere but in "packfile.c" anymore after this change, so we convert it to a static function, as well.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 midx.c     |  2 +-
 odb.c      |  2 --
 odb.h      |  6 ------
 packfile.c | 20 ++++++++++++++++++--
 packfile.h | 20 ++++++--------------
 5 files changed, 25 insertions(+), 25 deletions(-)
Show changes to 5 files +25 −25

midx.c, odb.c, odb.h, packfile.c, packfile.h

diff --git a/midx.c b/midx.c
index 7d407682e6..7f3f74ef2b 100644
--- a/midx.c
+++ b/midx.c
@@ -471,7 +471,7 @@ int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,
 	strbuf_addbuf(&key, &pack_name);
 	strbuf_strip_suffix(&key, ".idx");
 	strbuf_addstr(&key, ".pack");
-	p = hashmap_get_entry_from_hash(&r->objects->pack_map,
+	p = hashmap_get_entry_from_hash(&r->objects->packfiles->map,
 					strhash(key.buf), key.buf,
 					struct packed_git, packmap_ent);
 	if (!p) {
diff --git a/odb.c b/odb.c
index 17a9135cbd..568c820ef8 100644
--- a/odb.c
+++ b/odb.c
@@ -998,7 +998,6 @@ struct object_database *odb_new(struct repository *repo)
 	o->repo = repo;
 	o->packfiles = packfile_store_new(o);
 	INIT_LIST_HEAD(&o->packed_git_mru);
-	hashmap_init(&o->pack_map, pack_map_entry_cmp, NULL, 0);
 	pthread_mutex_init(&o->replace_mutex, NULL);
 	string_list_init_dup(&o->submodule_source_paths);
 	return o;
@@ -1040,6 +1039,5 @@ void odb_clear(struct object_database *o)
 	close_object_store(o);
 	packfile_store_free(o->packfiles);
 
-	hashmap_clear(&o->pack_map);
 	string_list_clear(&o->submodule_source_paths, 0);
 }
diff --git a/odb.h b/odb.h
index 970919403d..99c1ba7b77 100644
--- a/odb.h
+++ b/odb.h
@@ -148,12 +148,6 @@ struct object_database {
 	struct cached_object_entry *cached_objects;
 	size_t cached_object_nr, cached_object_alloc;
 
-	/*
-	 * A map of packfiles to packed_git structs for tracking which
-	 * packs have been loaded already.
-	 */
-	struct hashmap pack_map;
-
 	/*
 	 * A fast, rough count of the number of objects in the repository.
 	 * These two fields are not meant for direct access. Use
diff --git a/packfile.c b/packfile.c
index 4e5f84eb09..6582b0a479 100644
--- a/packfile.c
+++ b/packfile.c
@@ -788,7 +788,7 @@ void install_packed_git(struct repository *r, struct packed_git *pack)
 	r->objects->packfiles->packs = pack;
 
 	hashmap_entry_init(&pack->packmap_ent, strhash(pack->pack_name));
-	hashmap_add(&r->objects->pack_map, &pack->packmap_ent);
+	hashmap_add(&r->objects->packfiles->map, &pack->packmap_ent);
 }
 
 void (*report_garbage)(unsigned seen_bits, const char *path);
@@ -901,7 +901,7 @@ static void prepare_pack(const char *full_name, size_t full_name_len,
 		hashmap_entry_init(&hent, hash);
 
 		/* Don't reopen a pack we already have. */
-		if (!hashmap_get(&data->r->objects->pack_map, &hent, pack_name)) {
+		if (!hashmap_get(&data->r->objects->packfiles->map, &hent, pack_name)) {
 			p = add_packed_git(data->r, full_name, full_name_len, data->local);
 			if (p)
 				install_packed_git(data->r, p);
@@ -2329,11 +2329,26 @@ int parse_pack_header_option(const char *in, unsigned char *out, unsigned int *l
 	return 0;
 }
 
+static int pack_map_entry_cmp(const void *cmp_data UNUSED,
+			      const struct hashmap_entry *entry,
+			      const struct hashmap_entry *entry2,
+			      const void *keydata)
+{
+	const char *key = keydata;
+	const struct packed_git *pg1, *pg2;
+
+	pg1 = container_of(entry, const struct packed_git, packmap_ent);
+	pg2 = container_of(entry2, const struct packed_git, packmap_ent);
+
+	return strcmp(pg1->pack_name, key ? key : pg2->pack_name);
+}
+
 struct packfile_store *packfile_store_new(struct object_database *odb)
 {
 	struct packfile_store *store;
 	CALLOC_ARRAY(store, 1);
 	store->odb = odb;
+	hashmap_init(&store->map, pack_map_entry_cmp, NULL, 0);
 	return store;
 }
 
@@ -2346,6 +2361,7 @@ void packfile_store_free(struct packfile_store *store)
 		free(p);
 	}
 
+	hashmap_clear(&store->map);
 	free(store);
 }
 
diff --git a/packfile.h b/packfile.h
index 573564b19e..2f84d7d7e6 100644
--- a/packfile.h
+++ b/packfile.h
@@ -65,6 +65,12 @@ struct packfile_store {
 	 */
 	struct packed_git *packs;
 
+	/*
+	 * A map of packfile names to packed_git structs for tracking which
+	 * packs have been loaded already.
+	 */
+	struct hashmap map;
+
 	/*
 	 * Whether packfiles have already been populated with this store's
 	 * packs.
@@ -90,20 +96,6 @@ void packfile_store_free(struct packfile_store *store);
  */
 void packfile_store_close(struct packfile_store *store);
 
-static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
-				     const struct hashmap_entry *entry,
-				     const struct hashmap_entry *entry2,
-				     const void *keydata)
-{
-	const char *key = keydata;
-	const struct packed_git *pg1, *pg2;
-
-	pg1 = container_of(entry, const struct packed_git, packmap_ent);
-	pg2 = container_of(entry2, const struct packed_git, packmap_ent);
-
-	return strcmp(pg1->pack_name, key ? key : pg2->pack_name);
-}
-
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
-- 
2.51.0.261.g7ce5a0a67e.dirty
Patrick Steinhardt· Aug 19, 2025, 08:19 UTC · re: Patrick Steinhardt · lore

[PATCH 05/16] odb: move MRU list of packfiles into `struct packfile_store`

The object database tracks the list of packfiles in most-recently-used order, which is mostly used to favor reading from packfiles that contain most of the objects that we're currently accessing. With the introduction of the `struct packfile_store` we have a better place to host this list though.

Move the list accordingly.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 midx.c     |  2 +-
 odb.c      |  2 --
 odb.h      |  4 ----
 packfile.c | 11 ++++++-----
 packfile.h |  3 +++
 5 files changed, 10 insertions(+), 12 deletions(-)
Show changes to 5 files +10 −12

midx.c, odb.c, odb.h, packfile.c, packfile.h

diff --git a/midx.c b/midx.c
index 7f3f74ef2b..7fa2b8473a 100644
--- a/midx.c
+++ b/midx.c
@@ -478,7 +478,7 @@ int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,
 		p = add_packed_git(r, pack_name.buf, pack_name.len, m->local);
 		if (p) {
 			install_packed_git(r, p);
-			list_add_tail(&p->mru, &r->objects->packed_git_mru);
+			list_add_tail(&p->mru, &r->objects->packfiles->mru);
 		}
 	}
 
diff --git a/odb.c b/odb.c
index 568c820ef8..80ec6fc1fa 100644
--- a/odb.c
+++ b/odb.c
@@ -997,7 +997,6 @@ struct object_database *odb_new(struct repository *repo)
 	memset(o, 0, sizeof(*o));
 	o->repo = repo;
 	o->packfiles = packfile_store_new(o);
-	INIT_LIST_HEAD(&o->packed_git_mru);
 	pthread_mutex_init(&o->replace_mutex, NULL);
 	string_list_init_dup(&o->submodule_source_paths);
 	return o;
@@ -1035,7 +1034,6 @@ void odb_clear(struct object_database *o)
 		free((char *) o->cached_objects[i].value.buf);
 	FREE_AND_NULL(o->cached_objects);
 
-	INIT_LIST_HEAD(&o->packed_git_mru);
 	close_object_store(o);
 	packfile_store_free(o->packfiles);
 
diff --git a/odb.h b/odb.h
index 99c1ba7b77..2dc3bdc79d 100644
--- a/odb.h
+++ b/odb.h
@@ -3,7 +3,6 @@
 
 #include "hashmap.h"
 #include "object.h"
-#include "list.h"
 #include "oidset.h"
 #include "oidmap.h"
 #include "string-list.h"
@@ -131,9 +130,6 @@ struct object_database {
 	 */
 	struct packfile_store *packfiles;
 
-	/* A most-recently-used ordered version of the packed_git list. */
-	struct list_head packed_git_mru;
-
 	struct {
 		struct packed_git **packs;
 		unsigned flags;
diff --git a/packfile.c b/packfile.c
index 6582b0a479..f82856c19e 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1017,10 +1017,10 @@ static void prepare_packed_git_mru(struct repository *r)
 {
 	struct packed_git *p;
 
-	INIT_LIST_HEAD(&r->objects->packed_git_mru);
+	INIT_LIST_HEAD(&r->objects->packfiles->mru);
 
 	for (p = r->objects->packfiles->packs; p; p = p->next)
-		list_add_tail(&p->mru, &r->objects->packed_git_mru);
+		list_add_tail(&p->mru, &r->objects->packfiles->mru);
 }
 
 static void prepare_packed_git(struct repository *r)
@@ -1096,7 +1096,7 @@ struct packed_git *get_all_packs(struct repository *r)
 struct list_head *get_packed_git_mru(struct repository *r)
 {
 	prepare_packed_git(r);
-	return &r->objects->packed_git_mru;
+	return &r->objects->packfiles->mru;
 }
 
 unsigned long unpack_object_header_buffer(const unsigned char *buf,
@@ -2079,10 +2079,10 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 	if (!r->objects->packfiles->packs)
 		return 0;
 
-	list_for_each(pos, &r->objects->packed_git_mru) {
+	list_for_each(pos, &r->objects->packfiles->mru) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		if (!p->multi_pack_index && fill_pack_entry(oid, e, p)) {
-			list_move(&p->mru, &r->objects->packed_git_mru);
+			list_move(&p->mru, &r->objects->packfiles->mru);
 			return 1;
 		}
 	}
@@ -2348,6 +2348,7 @@ struct packfile_store *packfile_store_new(struct object_database *odb)
 	struct packfile_store *store;
 	CALLOC_ARRAY(store, 1);
 	store->odb = odb;
+	INIT_LIST_HEAD(&store->mru);
 	hashmap_init(&store->map, pack_map_entry_cmp, NULL, 0);
 	return store;
 }
diff --git a/packfile.h b/packfile.h
index 2f84d7d7e6..3022f3a19e 100644
--- a/packfile.h
+++ b/packfile.h
@@ -65,6 +65,9 @@ struct packfile_store {
 	 */
 	struct packed_git *packs;
 
+	/* A most-recently-used ordered version of the packs list. */
+	struct list_head mru;
+
 	/*
 	 * A map of packfile names to packed_git structs for tracking which
 	 * packs have been loaded already.
-- 
2.51.0.261.g7ce5a0a67e.dirty
Karthik Nayak· Aug 20, 2025, 12:44 UTC · re: Patrick Steinhardt · lore

Re: [PATCH 05/16] odb: move MRU list of packfiles into `struct packfile_store`

Patrick Steinhardt <ps@pks.im> writes:
Show 7 quoted lines
> The object database tracks the list of packfiles in most-recently-used
> order, which is mostly used to favor reading from packfiles that contain
> most of the objects that we're currently accessing. With the
> introduction of the `struct packfile_store` we have a better place to
> host this list though.
>
> Move the list accordingly.
[snip]
Show 17 quoted lines
> diff --git a/packfile.h b/packfile.h
> index 2f84d7d7e6..3022f3a19e 100644
> --- a/packfile.h
> +++ b/packfile.h
> @@ -65,6 +65,9 @@ struct packfile_store {
>  	 */
>  	struct packed_git *packs;
>
> +	/* A most-recently-used ordered version of the packs list. */
> +	struct list_head mru;
> +
>  	/*
>  	 * A map of packfile names to packed_git structs for tracking which
>  	 * packs have been loaded already.
>
> --
> 2.51.0.261.g7ce5a0a67e.dirty
Question: for my understanding, so we maintain a list of `packed_git`
packfiles in `packfile_store.packs` and then the same list is
available in a MRU form in `packfile_store.mru`?

I assume this is to optimize searches to use the mru form? Is there a reason to not pick the mru list?

Thanks
Jeff King· Aug 20, 2025, 19:20 UTC · re: Karthik Nayak · lore

Re: [PATCH 05/16] odb: move MRU list of packfiles into `struct packfile_store`

On Wed, Aug 20, 2025 at 05:44:36AM -0700, Karthik Nayak wrote:
Show 6 quoted lines
> Question: for my understanding, so we maintain a list of `packed_git`
> packfiles in `packfile_store.packs` and then the same list is
> available in a MRU form in `packfile_store.mru`?
> 
> I assume this is to optimize searches to use the mru form? Is there a
> reason to not pick the mru list?

Yeah, I believe you should find the same packfiles in the linked list currently formed by packed_git.next and the MRU list. When I introduced the MRU list long ago, I cowardly did not want to take the risk that somebody depended on the order of the original list, and left it in place.

I think it would _probably_ be OK to just keep a single list (and manipulate it to keep the MRU property). But whoever does should take a close look and make sure that is true. The biggest risk I can think of is that there could be some code iterating over the packfiles, coupled with object lookups in their loop body. If those lookups reorder the list, that would screw up the iteration.

IMHO it is probably better to do a change like that outside of Patrick's series. There already is a lot going on with moving fields around, and consolidating the lists can happen on top (and would be made easier for having pulled it out into two adjacent lists).

-Peff
Patrick Steinhardt· Aug 21, 2025, 06:40 UTC · re: Jeff King · lore

Re: [PATCH 05/16] odb: move MRU list of packfiles into `struct packfile_store`

On Wed, Aug 20, 2025 at 03:20:08PM -0400, Jeff King wrote:
Show 26 quoted lines
> On Wed, Aug 20, 2025 at 05:44:36AM -0700, Karthik Nayak wrote:
> 
> > Question: for my understanding, so we maintain a list of `packed_git`
> > packfiles in `packfile_store.packs` and then the same list is
> > available in a MRU form in `packfile_store.mru`?
> > 
> > I assume this is to optimize searches to use the mru form? Is there a
> > reason to not pick the mru list?
> 
> Yeah, I believe you should find the same packfiles in the linked list
> currently formed by packed_git.next and the MRU list. When I introduced
> the MRU list long ago, I cowardly did not want to take the risk that
> somebody depended on the order of the original list, and left it in
> place.
> 
> I think it would _probably_ be OK to just keep a single list (and
> manipulate it to keep the MRU property). But whoever does should take a
> close look and make sure that is true. The biggest risk I can think of
> is that there could be some code iterating over the packfiles, coupled
> with object lookups in their loop body. If those lookups reorder the
> list, that would screw up the iteration.
> 
> IMHO it is probably better to do a change like that outside of Patrick's
> series. There already is a lot going on with moving fields around, and
> consolidating the lists can happen on top (and would be made easier for
> having pulled it out into two adjacent lists).

There's in fact been a couple sites where we _didn't_ add packfiles to the MRU list: "builtin/fast-import.c", "builtin/index-pack.c", "packfile.c" via `prepare_pack()`. This is no longer going to be the case at the end of this patch series as we in a later patch adjust `packfile_store_store_add()` to handle this for us.

But I agree, this is something I'd rather want to push into a subsequent patch series. I've already got one cooking where I change how the lists are getting handled, so that's a good opportunity to do so.

Patrick
Patrick Steinhardt· Aug 19, 2025, 08:19 UTC · re: Patrick Steinhardt · lore

[PATCH 06/16] odb: move kept cache into `struct packfile_store`

The object database tracks a cache of "kept" packfiles, which is used by git-pack-objects(1) to handle cruft objects. With the introduction of the `struct packfile_store` we have a better place to host this cache though.

Move the cache accordingly.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb.h      |  9 +--------
 packfile.c | 16 ++++++++--------
 packfile.h |  5 +++++
 3 files changed, 14 insertions(+), 16 deletions(-)
Show changes to 3 files +14 −16

odb.h, packfile.c, packfile.h

diff --git a/odb.h b/odb.h
index 2dc3bdc79d..f1736b067c 100644
--- a/odb.h
+++ b/odb.h
@@ -124,17 +124,10 @@ struct object_database {
 	unsigned commit_graph_attempted : 1; /* if loading has been attempted */
 
 	/*
-	 * private data
-	 *
-	 * should only be accessed directly by packfile.c
+	 * Should only be accessed directly by packfile.c
 	 */
 	struct packfile_store *packfiles;
 
-	struct {
-		struct packed_git **packs;
-		unsigned flags;
-	} kept_pack_cache;
-
 	/*
 	 * This is meant to hold a *small* number of objects that you would
 	 * want odb_read_object() to be able to return, but yet you do not want
diff --git a/packfile.c b/packfile.c
index f82856c19e..f33445a5ff 100644
--- a/packfile.c
+++ b/packfile.c
@@ -2092,19 +2092,19 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 static void maybe_invalidate_kept_pack_cache(struct repository *r,
 					     unsigned flags)
 {
-	if (!r->objects->kept_pack_cache.packs)
+	if (!r->objects->packfiles->kept_cache.packs)
 		return;
-	if (r->objects->kept_pack_cache.flags == flags)
+	if (r->objects->packfiles->kept_cache.flags == flags)
 		return;
-	FREE_AND_NULL(r->objects->kept_pack_cache.packs);
-	r->objects->kept_pack_cache.flags = 0;
+	FREE_AND_NULL(r->objects->packfiles->kept_cache.packs);
+	r->objects->packfiles->kept_cache.flags = 0;
 }
 
 struct packed_git **kept_pack_cache(struct repository *r, unsigned flags)
 {
 	maybe_invalidate_kept_pack_cache(r, flags);
 
-	if (!r->objects->kept_pack_cache.packs) {
+	if (!r->objects->packfiles->kept_cache.packs) {
 		struct packed_git **packs = NULL;
 		size_t nr = 0, alloc = 0;
 		struct packed_git *p;
@@ -2127,11 +2127,11 @@ struct packed_git **kept_pack_cache(struct repository *r, unsigned flags)
 		ALLOC_GROW(packs, nr + 1, alloc);
 		packs[nr] = NULL;
 
-		r->objects->kept_pack_cache.packs = packs;
-		r->objects->kept_pack_cache.flags = flags;
+		r->objects->packfiles->kept_cache.packs = packs;
+		r->objects->packfiles->kept_cache.flags = flags;
 	}
 
-	return r->objects->kept_pack_cache.packs;
+	return r->objects->packfiles->kept_cache.packs;
 }
 
 int find_kept_pack_entry(struct repository *r,
diff --git a/packfile.h b/packfile.h
index 3022f3a19e..f46ea9ceec 100644
--- a/packfile.h
+++ b/packfile.h
@@ -65,6 +65,11 @@ struct packfile_store {
 	 */
 	struct packed_git *packs;
 
+	struct {
+		struct packed_git **packs;
+		unsigned flags;
+	} kept_cache;
+
 	/* A most-recently-used ordered version of the packs list. */
 	struct list_head mru;
 
-- 
2.51.0.261.g7ce5a0a67e.dirty
Junio C Hamano· Aug 19, 2025, 18:56 UTC · re: Patrick Steinhardt · lore

Re: [PATCH 06/16] odb: move kept cache into `struct packfile_store`

Patrick Steinhardt <ps@pks.im> writes:
Show 27 quoted lines
> The object database tracks a cache of "kept" packfiles, which is used by
> git-pack-objects(1) to handle cruft objects. With the introduction of
> the `struct packfile_store` we have a better place to host this cache
> though.
>
> Move the cache accordingly.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  odb.h      |  9 +--------
>  packfile.c | 16 ++++++++--------
>  packfile.h |  5 +++++
>  3 files changed, 14 insertions(+), 16 deletions(-)
>
> diff --git a/odb.h b/odb.h
> index 2dc3bdc79d..f1736b067c 100644
> --- a/odb.h
> +++ b/odb.h
> @@ -124,17 +124,10 @@ struct object_database {
>  	unsigned commit_graph_attempted : 1; /* if loading has been attempted */
>  
>  	/*
> -	 * private data
> -	 *
> -	 * should only be accessed directly by packfile.c
> +	 * Should only be accessed directly by packfile.c
>  	 */

Hmph, would this be better done in the step [01/16]? Or did the removal of kept_pack_cache make the last piece of "private data" disappear with this step?

Show 10 quoted lines
>  	struct packfile_store *packfiles;
>  
> -	struct {
> -		struct packed_git **packs;
> -		unsigned flags;
> -	} kept_pack_cache;
> -
>  	/*
>  	 * This is meant to hold a *small* number of objects that you would
>  	 * want odb_read_object() to be able to return, but yet you do not want
Patrick Steinhardt· Aug 20, 2025, 04:58 UTC · re: Junio C Hamano · lore

Re: [PATCH 06/16] odb: move kept cache into `struct packfile_store`

On Tue, Aug 19, 2025 at 11:56:54AM -0700, Junio C Hamano wrote:
Show 18 quoted lines
> Patrick Steinhardt <ps@pks.im> writes:
> > diff --git a/odb.h b/odb.h
> > index 2dc3bdc79d..f1736b067c 100644
> > --- a/odb.h
> > +++ b/odb.h
> > @@ -124,17 +124,10 @@ struct object_database {
> >  	unsigned commit_graph_attempted : 1; /* if loading has been attempted */
> >  
> >  	/*
> > -	 * private data
> > -	 *
> > -	 * should only be accessed directly by packfile.c
> > +	 * Should only be accessed directly by packfile.c
> >  	 */
> 
> Hmph, would this be better done in the step [01/16]?  Or did the
> removal of kept_pack_cache make the last piece of "private data"
> disappear with this step?

Yeah, the latter. All packfile-related private data is now encapsulated in `struct packfile_store`, so I felt like the comment became redundant with this commit here.

I'll mention this in the commit message.
Patrick
Patrick Steinhardt· Aug 19, 2025, 08:19 UTC · re: Patrick Steinhardt · lore

[PATCH 07/16] packfile: reorder functions to avoid function declaration

Reorder functions so that we can avoid an extra declaration of `prepare_packed_git()`.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 packfile.c | 67 +++++++++++++++++++++++++++++++-------------------------------
 1 file changed, 33 insertions(+), 34 deletions(-)
Show changes to packfile.c +33 −34
diff --git a/packfile.c b/packfile.c
index f33445a5ff..99f2d20812 100644
--- a/packfile.c
+++ b/packfile.c
@@ -946,40 +946,6 @@ static void prepare_packed_git_one(struct odb_source *source, int local)
 	string_list_clear(data.garbage, 0);
 }
 
-static void prepare_packed_git(struct repository *r);
-/*
- * Give a fast, rough count of the number of objects in the repository. This
- * ignores loose objects completely. If you have a lot of them, then either
- * you should repack because your performance will be awful, or they are
- * all unreachable objects about to be pruned, in which case they're not really
- * interesting as a measure of repo size in the first place.
- */
-unsigned long repo_approximate_object_count(struct repository *r)
-{
-	if (!r->objects->approximate_object_count_valid) {
-		struct odb_source *source;
-		unsigned long count = 0;
-		struct packed_git *p;
-
-		prepare_packed_git(r);
-
-		for (source = r->objects->sources; source; source = source->next) {
-			struct multi_pack_index *m = get_multi_pack_index(source);
-			if (m)
-				count += m->num_objects;
-		}
-
-		for (p = r->objects->packfiles->packs; p; p = p->next) {
-			if (open_pack_index(p))
-				continue;
-			count += p->num_objects;
-		}
-		r->objects->approximate_object_count = count;
-		r->objects->approximate_object_count_valid = 1;
-	}
-	return r->objects->approximate_object_count;
-}
-
 DEFINE_LIST_SORT(static, sort_packs, struct packed_git, next);
 
 static int sort_pack(const struct packed_git *a, const struct packed_git *b)
@@ -1099,6 +1065,39 @@ struct list_head *get_packed_git_mru(struct repository *r)
 	return &r->objects->packfiles->mru;
 }
 
+/*
+ * Give a fast, rough count of the number of objects in the repository. This
+ * ignores loose objects completely. If you have a lot of them, then either
+ * you should repack because your performance will be awful, or they are
+ * all unreachable objects about to be pruned, in which case they're not really
+ * interesting as a measure of repo size in the first place.
+ */
+unsigned long repo_approximate_object_count(struct repository *r)
+{
+	if (!r->objects->approximate_object_count_valid) {
+		struct odb_source *source;
+		unsigned long count = 0;
+		struct packed_git *p;
+
+		prepare_packed_git(r);
+
+		for (source = r->objects->sources; source; source = source->next) {
+			struct multi_pack_index *m = get_multi_pack_index(source);
+			if (m)
+				count += m->num_objects;
+		}
+
+		for (p = r->objects->packfiles->packs; p; p = p->next) {
+			if (open_pack_index(p))
+				continue;
+			count += p->num_objects;
+		}
+		r->objects->approximate_object_count = count;
+		r->objects->approximate_object_count_valid = 1;
+	}
+	return r->objects->approximate_object_count;
+}
+
 unsigned long unpack_object_header_buffer(const unsigned char *buf,
 		unsigned long len, enum object_type *type, unsigned long *sizep)
 {
-- 
2.51.0.261.g7ce5a0a67e.dirty
Junio C Hamano· Aug 19, 2025, 19:18 UTC · re: Patrick Steinhardt · lore

Re: [PATCH 07/16] packfile: reorder functions to avoid function declaration

Patrick Steinhardt <ps@pks.im> writes:
> Reorder functions so that we can avoid an extra declaration of
> `prepare_packed_git()`.

Makes sense. We usually call that "forward declaration" instead, though.

Patrick Steinhardt· Aug 19, 2025, 08:19 UTC · re: Patrick Steinhardt · lore

[PATCH 08/16] packfile: refactor `prepare_packed_git()` to work on packfile store

The `prepare_packed_git()` function and its friends are responsible for loading packfiles as well as the multi-pack index for a given object database. Refactor these functions to accept a packfile store instead of a repository to clarify their scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 packfile.c | 43 +++++++++++++++++++------------------------
 1 file changed, 19 insertions(+), 24 deletions(-)
Show changes to packfile.c +19 −24
diff --git a/packfile.c b/packfile.c
index 99f2d20812..58e50d7b30 100644
--- a/packfile.c
+++ b/packfile.c
@@ -974,38 +974,33 @@ static int sort_pack(const struct packed_git *a, const struct packed_git *b)
 	return -1;
 }
 
-static void rearrange_packed_git(struct repository *r)
-{
-	sort_packs(&r->objects->packfiles->packs, sort_pack);
-}
-
-static void prepare_packed_git_mru(struct repository *r)
+static void packfile_store_prepare_mru(struct packfile_store *store)
 {
 	struct packed_git *p;
 
-	INIT_LIST_HEAD(&r->objects->packfiles->mru);
+	INIT_LIST_HEAD(&store->mru);
 
-	for (p = r->objects->packfiles->packs; p; p = p->next)
-		list_add_tail(&p->mru, &r->objects->packfiles->mru);
+	for (p = store->packs; p; p = p->next)
+		list_add_tail(&p->mru, &store->mru);
 }
 
-static void prepare_packed_git(struct repository *r)
+static void packfile_store_prepare(struct packfile_store *store)
 {
 	struct odb_source *source;
 
-	if (r->objects->packfiles->initialized)
+	if (store->initialized)
 		return;
 
-	odb_prepare_alternates(r->objects);
-	for (source = r->objects->sources; source; source = source->next) {
-		int local = (source == r->objects->sources);
+	odb_prepare_alternates(store->odb);
+	for (source = store->odb->sources; source; source = source->next) {
+		int local = (source == store->odb->sources);
 		prepare_multi_pack_index_one(source, local);
 		prepare_packed_git_one(source, local);
 	}
-	rearrange_packed_git(r);
+	sort_packs(&store->packs, sort_pack);
 
-	prepare_packed_git_mru(r);
-	r->objects->packfiles->initialized = 1;
+	packfile_store_prepare_mru(store);
+	store->initialized = 1;
 }
 
 void reprepare_packed_git(struct repository *r)
@@ -1028,25 +1023,25 @@ void reprepare_packed_git(struct repository *r)
 
 	r->objects->approximate_object_count_valid = 0;
 	r->objects->packfiles->initialized = 0;
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 	obj_read_unlock();
 }
 
 struct packed_git *get_packed_git(struct repository *r)
 {
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 	return r->objects->packfiles->packs;
 }
 
 struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
 {
-	prepare_packed_git(source->odb->repo);
+	packfile_store_prepare(source->odb->packfiles);
 	return source->midx;
 }
 
 struct packed_git *get_all_packs(struct repository *r)
 {
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 
 	for (struct odb_source *source = r->objects->sources; source; source = source->next) {
 		struct multi_pack_index *m = source->midx;
@@ -1061,7 +1056,7 @@ struct packed_git *get_all_packs(struct repository *r)
 
 struct list_head *get_packed_git_mru(struct repository *r)
 {
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 	return &r->objects->packfiles->mru;
 }
 
@@ -1079,7 +1074,7 @@ unsigned long repo_approximate_object_count(struct repository *r)
 		unsigned long count = 0;
 		struct packed_git *p;
 
-		prepare_packed_git(r);
+		packfile_store_prepare(r->objects->packfiles);
 
 		for (source = r->objects->sources; source; source = source->next) {
 			struct multi_pack_index *m = get_multi_pack_index(source);
@@ -2069,7 +2064,7 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 {
 	struct list_head *pos;
 
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 
 	for (struct odb_source *source = r->objects->sources; source; source = source->next)
 		if (source->midx && fill_midx_entry(r, oid, e, source->midx))
-- 
2.51.0.261.g7ce5a0a67e.dirty
Patrick Steinhardt· Aug 19, 2025, 08:19 UTC · re: Patrick Steinhardt · lore

[PATCH 09/16] packfile: split up responsibilities of `reprepare_packed_git()`

In `reprepare_packed_git()` we perform a couple of operations:
  - We reload alternate object directories.
  - We clear the loose object cache.
  - We reprepare packfiles.

While the logic is hosted in "packfile.c", it clearly reaches into other subsystems that aren't related to packfiles.

Split up the responsibility and introduce `odb_reprepare()` which now becomes responsible for repreparing the whole object database. The existing `reprepare_packed_git()` function is refactored accordingly and only cares about reloading the packfile store now.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/backfill.c     |  2 +-
 builtin/gc.c           |  4 ++--
 builtin/receive-pack.c |  2 +-
 builtin/repack.c       |  2 +-
 bulk-checkin.c         |  2 +-
 connected.c            |  2 +-
 fetch-pack.c           |  4 ++--
 object-name.c          |  2 +-
 odb.c                  | 25 ++++++++++++++++++++++++-
 odb.h                  |  6 ++++++
 packfile.c             | 26 ++++----------------------
 packfile.h             |  9 ++++++++-
 transport-helper.c     |  2 +-
 13 files changed, 53 insertions(+), 35 deletions(-)
Show changes to 13 files +53 −35

builtin/backfill.c, builtin/gc.c, builtin/receive-pack.c, builtin/repack.c, bulk-checkin.c, connected.c, fetch-pack.c, object-name.c, odb.c, odb.h, packfile.c, packfile.h, transport-helper.c

diff --git a/builtin/backfill.c b/builtin/backfill.c
index 80056abe47..e80fc1b694 100644
--- a/builtin/backfill.c
+++ b/builtin/backfill.c
@@ -53,7 +53,7 @@ static void download_batch(struct backfill_context *ctx)
 	 * We likely have a new packfile. Add it to the packed list to
 	 * avoid possible duplicate downloads of the same objects.
 	 */
-	reprepare_packed_git(ctx->repo);
+	odb_reprepare(ctx->repo->objects);
 }
 
 static int fill_missing_blobs(const char *path UNUSED,
diff --git a/builtin/gc.c b/builtin/gc.c
index 0edd94a76f..1d30d1af2c 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1041,7 +1041,7 @@ int cmd_gc(int argc,
 		die(FAILED_RUN, "rerere");
 
 	report_garbage = report_pack_garbage;
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 	if (pack_garbage.nr > 0) {
 		close_object_store(the_repository->objects);
 		clean_pack_garbage();
@@ -1490,7 +1490,7 @@ static off_t get_auto_pack_size(void)
 	struct packed_git *p;
 	struct repository *r = the_repository;
 
-	reprepare_packed_git(r);
+	odb_reprepare(r->objects);
 	for (p = get_all_packs(r); p; p = p->next) {
 		if (p->pack_size > max_size) {
 			second_largest_size = max_size;
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 1113137a6f..c9288a9c7e 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -2389,7 +2389,7 @@ static const char *unpack(int err_fd, struct shallow_info *si)
 		status = finish_command(&child);
 		if (status)
 			return "index-pack abnormal exit";
-		reprepare_packed_git(the_repository);
+		odb_reprepare(the_repository->objects);
 	}
 	return NULL;
 }
diff --git a/builtin/repack.c b/builtin/repack.c
index a4def39197..ee8c80cd95 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -1684,7 +1684,7 @@ int cmd_repack(int argc,
 			goto cleanup;
 	}
 
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 
 	if (delete_redundant) {
 		int opts = 0;
diff --git a/bulk-checkin.c b/bulk-checkin.c
index b2809ab039..f65439a748 100644
--- a/bulk-checkin.c
+++ b/bulk-checkin.c
@@ -90,7 +90,7 @@ static void flush_bulk_checkin_packfile(struct bulk_checkin_packfile *state)
 
 	strbuf_release(&packname);
 	/* Make objects we just wrote available to ourselves */
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 }
 
 /*
diff --git a/connected.c b/connected.c
index 18c13245d8..d6e9682fd9 100644
--- a/connected.c
+++ b/connected.c
@@ -72,7 +72,7 @@ int check_connected(oid_iterate_fn fn, void *cb_data,
 		 * Before checking for promisor packs, be sure we have the
 		 * latest pack-files loaded into memory.
 		 */
-		reprepare_packed_git(the_repository);
+		odb_reprepare(the_repository->objects);
 		do {
 			struct packed_git *p;
 
diff --git a/fetch-pack.c b/fetch-pack.c
index 46c39f85c4..3b8960608c 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -1982,7 +1982,7 @@ static void update_shallow(struct fetch_pack_args *args,
 		 * remote is shallow, but this is a clone, there are
 		 * no objects in repo to worry about. Accept any
 		 * shallow points that exist in the pack (iow in repo
-		 * after get_pack() and reprepare_packed_git())
+		 * after get_pack() and odb_reprepare())
 		 */
 		struct oid_array extra = OID_ARRAY_INIT;
 		struct object_id *oid = si->shallow->oid;
@@ -2107,7 +2107,7 @@ struct ref *fetch_pack(struct fetch_pack_args *args,
 		ref_cpy = do_fetch_pack(args, fd, ref, sought, nr_sought,
 					&si, pack_lockfiles);
 	}
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 
 	if (!args->cloning && args->deepen) {
 		struct check_connected_options opt = CHECK_CONNECTED_INIT;
diff --git a/object-name.c b/object-name.c
index 11aa0e6afc..44b0d416ac 100644
--- a/object-name.c
+++ b/object-name.c
@@ -596,7 +596,7 @@ static enum get_oid_result get_short_oid(struct repository *r,
 	 * or migrated from loose to packed.
 	 */
 	if (status == MISSING_OBJECT) {
-		reprepare_packed_git(r);
+		odb_reprepare(r->objects);
 		find_short_object_filename(&ds);
 		find_short_packed_object(&ds);
 		status = finish_object_disambiguation(&ds, oid);
diff --git a/odb.c b/odb.c
index 80ec6fc1fa..37ed21f53b 100644
--- a/odb.c
+++ b/odb.c
@@ -694,7 +694,7 @@ static int do_oid_object_info_extended(struct object_database *odb,
 
 		/* Not a loose object; someone else may have just packed it. */
 		if (!(flags & OBJECT_INFO_QUICK)) {
-			reprepare_packed_git(odb->repo);
+			odb_reprepare(odb->repo->objects);
 			if (find_pack_entry(odb->repo, real, &e))
 				break;
 		}
@@ -1039,3 +1039,26 @@ void odb_clear(struct object_database *o)
 
 	string_list_clear(&o->submodule_source_paths, 0);
 }
+
+void odb_reprepare(struct object_database *o)
+{
+	struct odb_source *source;
+
+	/*
+	 * Reprepare alt odbs, in case the alternates file was modified
+	 * during the course of this process. This only _adds_ odbs to
+	 * the linked list, so existing odbs will continue to exist for
+	 * the lifetime of the process.
+	 */
+	o->loaded_alternates = 0;
+	odb_prepare_alternates(o);
+
+	for (source = o->sources; source; source = source->next)
+		odb_clear_loose_cache(source);
+
+	o->approximate_object_count_valid = 0;
+
+	packfile_store_reprepare(o->packfiles);
+
+	obj_read_unlock();
+}
diff --git a/odb.h b/odb.h
index f1736b067c..9810ec60a0 100644
--- a/odb.h
+++ b/odb.h
@@ -155,6 +155,12 @@ struct object_database {
 struct object_database *odb_new(struct repository *repo);
 void odb_clear(struct object_database *o);
 
+/*
+ * Clear caches, reload alternates and then reload object sources so that new
+ * objects may become accessible.
+ */
+void odb_reprepare(struct object_database *o);
+
 /*
  * Find source by its object directory path. Dies in case the source couldn't
  * be found.
diff --git a/packfile.c b/packfile.c
index 58e50d7b30..180c95ec1c 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1003,28 +1003,10 @@ static void packfile_store_prepare(struct packfile_store *store)
 	store->initialized = 1;
 }
 
-void reprepare_packed_git(struct repository *r)
+void packfile_store_reprepare(struct packfile_store *store)
 {
-	struct odb_source *source;
-
-	obj_read_lock();
-
-	/*
-	 * Reprepare alt odbs, in case the alternates file was modified
-	 * during the course of this process. This only _adds_ odbs to
-	 * the linked list, so existing odbs will continue to exist for
-	 * the lifetime of the process.
-	 */
-	r->objects->loaded_alternates = 0;
-	odb_prepare_alternates(r->objects);
-
-	for (source = r->objects->sources; source; source = source->next)
-		odb_clear_loose_cache(source);
-
-	r->objects->approximate_object_count_valid = 0;
-	r->objects->packfiles->initialized = 0;
-	packfile_store_prepare(r->objects->packfiles);
-	obj_read_unlock();
+	store->initialized = 0;
+	packfile_store_prepare(store);
 }
 
 struct packed_git *get_packed_git(struct repository *r)
@@ -1145,7 +1127,7 @@ unsigned long get_size_from_delta(struct packed_git *p,
 		 *
 		 * Other worrying sections could be the call to close_pack_fd(),
 		 * which can close packs even with in-use windows, and to
-		 * reprepare_packed_git(). Regarding the former, mmap doc says:
+		 * odb_reprepare(). Regarding the former, mmap doc says:
 		 * "closing the file descriptor does not unmap the region". And
 		 * for the latter, it won't re-open already available packs.
 		 */
diff --git a/packfile.h b/packfile.h
index f46ea9ceec..75672c808a 100644
--- a/packfile.h
+++ b/packfile.h
@@ -104,6 +104,14 @@ void packfile_store_free(struct packfile_store *store);
  */
 void packfile_store_close(struct packfile_store *store);
 
+/*
+ * Clear the packfile caches and try to look up any new packfiles that have
+ * appeared since last preparing the packfiles store.
+ *
+ * This function must be called under the `odb_read_lock()`.
+ */
+void packfile_store_reprepare(struct packfile_store *store);
+
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
@@ -180,7 +188,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-void reprepare_packed_git(struct repository *r);
 void install_packed_git(struct repository *r, struct packed_git *pack);
 
 struct packed_git *get_packed_git(struct repository *r);
diff --git a/transport-helper.c b/transport-helper.c
index 0789e5bca5..4d95d84f9e 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -450,7 +450,7 @@ static int fetch_with_fetch(struct transport *transport,
 	}
 	strbuf_release(&buf);
 
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 	return 0;
 }
 
-- 
2.51.0.261.g7ce5a0a67e.dirty
Karthik Nayak· Aug 20, 2025, 13:17 UTC · re: Patrick Steinhardt · lore

Re: [PATCH 09/16] packfile: split up responsibilities of `reprepare_packed_git()`

Patrick Steinhardt <ps@pks.im> writes:
Show 15 quoted lines
> In `reprepare_packed_git()` we perform a couple of operations:
>
>   - We reload alternate object directories.
>
>   - We clear the loose object cache.
>
>   - We reprepare packfiles.
>
> While the logic is hosted in "packfile.c", it clearly reaches into other
> subsystems that aren't related to packfiles.
>
> Split up the responsibility and introduce `odb_reprepare()` which now
> becomes responsible for repreparing the whole object database. The
> existing `reprepare_packed_git()` function is refactored accordingly and
> only cares about reloading the packfile store now.
[snip]
Show 14 quoted lines
> diff --git a/odb.h b/odb.h
> index f1736b067c..9810ec60a0 100644
> --- a/odb.h
> +++ b/odb.h
> @@ -155,6 +155,12 @@ struct object_database {
>  struct object_database *odb_new(struct repository *repo);
>  void odb_clear(struct object_database *o);
>
> +/*
> + * Clear caches, reload alternates and then reload object sources so that new
> + * objects may become accessible.
> + */
> +void odb_reprepare(struct object_database *o);
>

I was first wondering why you don't go into details like mentioning the packile cleanup. But since we eventually will move it into its own object source, this reads better.

Patrick Steinhardt· Aug 19, 2025, 08:19 UTC · re: Patrick Steinhardt · lore

[PATCH 10/16] packfile: refactor `install_packed_git()` to work on packfile store

The `install_packed_git()` functions adds a packfile to a specific object store. Refactor it to accept a packfile store instead of a repository to clarify its scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/fast-import.c |  2 +-
 builtin/index-pack.c  |  2 +-
 http.c                |  2 +-
 http.h                |  2 +-
 midx.c                |  2 +-
 packfile.c            | 11 ++++++-----
 packfile.h            |  9 +++++++--
 7 files changed, 18 insertions(+), 12 deletions(-)
Show changes to 7 files +18 −12

builtin/fast-import.c, builtin/index-pack.c, http.c, http.h, midx.c, packfile.c, packfile.h

diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index 2c35f9345d..e9d82b31c3 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -901,7 +901,7 @@ static void end_packfile(void)
 		if (!new_p)
 			die("core git rejected index %s", idx_name);
 		all_packs[pack_id] = new_p;
-		install_packed_git(the_repository, new_p);
+		packfile_store_add_pack(the_repository->objects->packfiles, new_p);
 		free(idx_name);
 
 		/* Print the boundary */
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index f91c301bba..ed490dfad4 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1645,7 +1645,7 @@ static void final(const char *final_pack_name, const char *curr_pack_name,
 		p = add_packed_git(the_repository, final_index_name,
 				   strlen(final_index_name), 0);
 		if (p)
-			install_packed_git(the_repository, p);
+			packfile_store_add_pack(the_repository->objects->packfiles, p);
 	}
 
 	if (!from_stdin) {
diff --git a/http.c b/http.c
index 98853d6483..af2120b64c 100644
--- a/http.c
+++ b/http.c
@@ -2541,7 +2541,7 @@ void http_install_packfile(struct packed_git *p,
 		lst = &((*lst)->next);
 	*lst = (*lst)->next;
 
-	install_packed_git(the_repository, p);
+	packfile_store_add_pack(the_repository->objects->packfiles, p);
 }
 
 struct http_pack_request *new_http_pack_request(
diff --git a/http.h b/http.h
index 36202139f4..e5a5380c6c 100644
--- a/http.h
+++ b/http.h
@@ -210,7 +210,7 @@ int finish_http_pack_request(struct http_pack_request *preq);
 void release_http_pack_request(struct http_pack_request *preq);
 
 /*
- * Remove p from the given list, and invoke install_packed_git() on it.
+ * Remove p from the given list, and invoke packfile_store_add_pack() on it.
  *
  * This is a convenience function for users that have obtained a list of packs
  * from http_get_info_packs() and have chosen a specific pack to fetch.
diff --git a/midx.c b/midx.c
index 7fa2b8473a..95e74c79c1 100644
--- a/midx.c
+++ b/midx.c
@@ -477,7 +477,7 @@ int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,
 	if (!p) {
 		p = add_packed_git(r, pack_name.buf, pack_name.len, m->local);
 		if (p) {
-			install_packed_git(r, p);
+			packfile_store_add_pack(r->objects->packfiles, p);
 			list_add_tail(&p->mru, &r->objects->packfiles->mru);
 		}
 	}
diff --git a/packfile.c b/packfile.c
index 180c95ec1c..186d182c7c 100644
--- a/packfile.c
+++ b/packfile.c
@@ -779,16 +779,17 @@ struct packed_git *add_packed_git(struct repository *r, const char *path,
 	return p;
 }
 
-void install_packed_git(struct repository *r, struct packed_git *pack)
+void packfile_store_add_pack(struct packfile_store *store,
+			     struct packed_git *pack)
 {
 	if (pack->pack_fd != -1)
 		pack_open_fds++;
 
-	pack->next = r->objects->packfiles->packs;
-	r->objects->packfiles->packs = pack;
+	pack->next = store->packs;
+	store->packs = pack;
 
 	hashmap_entry_init(&pack->packmap_ent, strhash(pack->pack_name));
-	hashmap_add(&r->objects->packfiles->map, &pack->packmap_ent);
+	hashmap_add(&store->map, &pack->packmap_ent);
 }
 
 void (*report_garbage)(unsigned seen_bits, const char *path);
@@ -904,7 +905,7 @@ static void prepare_pack(const char *full_name, size_t full_name_len,
 		if (!hashmap_get(&data->r->objects->packfiles->map, &hent, pack_name)) {
 			p = add_packed_git(data->r, full_name, full_name_len, data->local);
 			if (p)
-				install_packed_git(data->r, p);
+				packfile_store_add_pack(data->r->objects->packfiles, p);
 		}
 		free(pack_name);
 	}
diff --git a/packfile.h b/packfile.h
index 75672c808a..e751a5d93e 100644
--- a/packfile.h
+++ b/packfile.h
@@ -112,6 +112,13 @@ void packfile_store_close(struct packfile_store *store);
  */
 void packfile_store_reprepare(struct packfile_store *store);
 
+/*
+ * Add the pack to the store so that contained objects become accessible via
+ * the store. This moves ownership into the store.
+ */
+void packfile_store_add_pack(struct packfile_store *store,
+			     struct packed_git *pack);
+
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
@@ -188,8 +195,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-void install_packed_git(struct repository *r, struct packed_git *pack);
-
 struct packed_git *get_packed_git(struct repository *r);
 struct list_head *get_packed_git_mru(struct repository *r);
 struct multi_pack_index *get_multi_pack_index(struct odb_source *source);
-- 
2.51.0.261.g7ce5a0a67e.dirty
Patrick Steinhardt· Aug 19, 2025, 08:19 UTC · re: Patrick Steinhardt · lore

[PATCH 11/16] packfile: always add packfiles to MRU when adding a pack

When adding a packfile to it store we add it both to the list and map of packfiles, but we don't append it to the most-recently-used list of packs. We do know to add the packfile to the MRU list as soon as we access any of its objects, but in between we're being inconistent. It doesn't help that there are some subsystems that _do_ add the packfile to the MRU after having added it, which only adds to the confusion.

Refactor the code so that we unconditionally add packfiles to the MRU when adding them to a packfile store.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 midx.c     | 4 +---
 packfile.c | 1 +
 2 files changed, 2 insertions(+), 3 deletions(-)
Show changes to 2 files +2 −3

midx.c, packfile.c

diff --git a/midx.c b/midx.c
index 95e74c79c1..3cfe7884ad 100644
--- a/midx.c
+++ b/midx.c
@@ -476,10 +476,8 @@ int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,
 					struct packed_git, packmap_ent);
 	if (!p) {
 		p = add_packed_git(r, pack_name.buf, pack_name.len, m->local);
-		if (p) {
+		if (p)
 			packfile_store_add_pack(r->objects->packfiles, p);
-			list_add_tail(&p->mru, &r->objects->packfiles->mru);
-		}
 	}
 
 	strbuf_release(&pack_name);
diff --git a/packfile.c b/packfile.c
index 186d182c7c..8b5e6b96ce 100644
--- a/packfile.c
+++ b/packfile.c
@@ -790,6 +790,7 @@ void packfile_store_add_pack(struct packfile_store *store,
 
 	hashmap_entry_init(&pack->packmap_ent, strhash(pack->pack_name));
 	hashmap_add(&store->map, &pack->packmap_ent);
+	list_add_tail(&pack->mru, &store->mru);
 }
 
 void (*report_garbage)(unsigned seen_bits, const char *path);
-- 
2.51.0.261.g7ce5a0a67e.dirty
Karthik Nayak· Aug 20, 2025, 13:35 UTC · re: Patrick Steinhardt · lore

Re: [PATCH 11/16] packfile: always add packfiles to MRU when adding a pack

Patrick Steinhardt <ps@pks.im> writes:
Show 9 quoted lines
> When adding a packfile to it store we add it both to the list and map of
> packfiles, but we don't append it to the most-recently-used list of
> packs. We do know to add the packfile to the MRU list as soon as we
> access any of its objects, but in between we're being inconistent. It
> doesn't help that there are some subsystems that _do_ add the packfile
> to the MRU after having added it, which only adds to the confusion.
>
> Refactor the code so that we unconditionally add packfiles to the MRU
> when adding them to a packfile store.
This makes sense.

It would be really nice if the internals of packfile_store is private, that way users wouldn't know/care about the mru in the first place. Till this patch, we seem to be moving in that direction as more things move into `packfile_store` we can slowly abstract it behind a strong set of APIs and internals can be kept private.

[snip]
Patrick Steinhardt· Aug 19, 2025, 08:19 UTC · re: Patrick Steinhardt · lore

[PATCH 12/16] packfile: introduce function to load and add packfiles

We have a recurring pattern where we essentially perform an upsert of a packfile in case it isn't yet known by the packfile store. The logic to do so is non-trivial as we have to reconstruct the packfile's key, check the map of packfiles, then create the new packfile and finally add it to the store.

Introduce a new function that does this dance for us. Refactor callsites to use it.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/fast-import.c |  4 ++--
 builtin/index-pack.c  | 10 +++-------
 midx.c                | 18 ++----------------
 packfile.c            | 44 +++++++++++++++++++++++++++++++-------------
 packfile.h            |  8 ++++++++
 5 files changed, 46 insertions(+), 38 deletions(-)
Show changes to 5 files +46 −38

builtin/fast-import.c, builtin/index-pack.c, midx.c, packfile.c, packfile.h

diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index e9d82b31c3..a26e79689d 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -897,11 +897,11 @@ static void end_packfile(void)
 		idx_name = keep_pack(create_index());
 
 		/* Register the packfile with core git's machinery. */
-		new_p = add_packed_git(pack_data->repo, idx_name, strlen(idx_name), 1);
+		new_p = packfile_store_load_pack(pack_data->repo->objects->packfiles,
+						 idx_name, 1);
 		if (!new_p)
 			die("core git rejected index %s", idx_name);
 		all_packs[pack_id] = new_p;
-		packfile_store_add_pack(the_repository->objects->packfiles, new_p);
 		free(idx_name);
 
 		/* Print the boundary */
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index ed490dfad4..2b78ba7fe4 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1640,13 +1640,9 @@ static void final(const char *final_pack_name, const char *curr_pack_name,
 	rename_tmp_packfile(&final_index_name, curr_index_name, &index_name,
 			    hash, "idx", 1);
 
-	if (do_fsck_object) {
-		struct packed_git *p;
-		p = add_packed_git(the_repository, final_index_name,
-				   strlen(final_index_name), 0);
-		if (p)
-			packfile_store_add_pack(the_repository->objects->packfiles, p);
-	}
+	if (do_fsck_object)
+		packfile_store_load_pack(the_repository->objects->packfiles,
+					 final_index_name, 0);
 
 	if (!from_stdin) {
 		printf("%s\n", hash_to_hex(hash));
diff --git a/midx.c b/midx.c
index 3cfe7884ad..d30feda019 100644
--- a/midx.c
+++ b/midx.c
@@ -454,7 +454,6 @@ int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,
 		      uint32_t pack_int_id)
 {
 	struct strbuf pack_name = STRBUF_INIT;
-	struct strbuf key = STRBUF_INIT;
 	struct packed_git *p;
 
 	pack_int_id = midx_for_pack(&m, pack_int_id);
@@ -466,22 +465,9 @@ int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,
 
 	strbuf_addf(&pack_name, "%s/pack/%s", m->object_dir,
 		    m->pack_names[pack_int_id]);
-
-	/* pack_map holds the ".pack" name, but we have the .idx */
-	strbuf_addbuf(&key, &pack_name);
-	strbuf_strip_suffix(&key, ".idx");
-	strbuf_addstr(&key, ".pack");
-	p = hashmap_get_entry_from_hash(&r->objects->packfiles->map,
-					strhash(key.buf), key.buf,
-					struct packed_git, packmap_ent);
-	if (!p) {
-		p = add_packed_git(r, pack_name.buf, pack_name.len, m->local);
-		if (p)
-			packfile_store_add_pack(r->objects->packfiles, p);
-	}
-
+	p = packfile_store_load_pack(r->objects->packfiles,
+				     pack_name.buf, m->local);
 	strbuf_release(&pack_name);
-	strbuf_release(&key);
 
 	if (!p) {
 		m->packs[pack_int_id] = MIDX_PACK_ERROR;
diff --git a/packfile.c b/packfile.c
index 8b5e6b96ce..f7916543a6 100644
--- a/packfile.c
+++ b/packfile.c
@@ -793,6 +793,33 @@ void packfile_store_add_pack(struct packfile_store *store,
 	list_add_tail(&pack->mru, &store->mru);
 }
 
+struct packed_git *packfile_store_load_pack(struct packfile_store *store,
+					    const char *idx_path, int local)
+{
+	struct strbuf key = STRBUF_INIT;
+	struct packed_git *p;
+
+	/*
+	 * We're being called with the path to the index file, but `pack_map`
+	 * holds the path to the packfile itself.
+	 */
+	strbuf_addstr(&key, idx_path);
+	strbuf_strip_suffix(&key, ".idx");
+	strbuf_addstr(&key, ".pack");
+
+	p = hashmap_get_entry_from_hash(&store->map, strhash(key.buf), key.buf,
+					struct packed_git, packmap_ent);
+	if (!p) {
+		p = add_packed_git(store->odb->repo, idx_path,
+				   strlen(idx_path), local);
+		if (p)
+			packfile_store_add_pack(store, p);
+	}
+
+	strbuf_release(&key);
+	return p;
+}
+
 void (*report_garbage)(unsigned seen_bits, const char *path);
 
 static void report_helper(const struct string_list *list,
@@ -892,23 +919,14 @@ static void prepare_pack(const char *full_name, size_t full_name_len,
 			 const char *file_name, void *_data)
 {
 	struct prepare_pack_data *data = (struct prepare_pack_data *)_data;
-	struct packed_git *p;
 	size_t base_len = full_name_len;
 
 	if (strip_suffix_mem(full_name, &base_len, ".idx") &&
 	    !(data->m && midx_contains_pack(data->m, file_name))) {
-		struct hashmap_entry hent;
-		char *pack_name = xstrfmt("%.*s.pack", (int)base_len, full_name);
-		unsigned int hash = strhash(pack_name);
-		hashmap_entry_init(&hent, hash);
-
-		/* Don't reopen a pack we already have. */
-		if (!hashmap_get(&data->r->objects->packfiles->map, &hent, pack_name)) {
-			p = add_packed_git(data->r, full_name, full_name_len, data->local);
-			if (p)
-				packfile_store_add_pack(data->r->objects->packfiles, p);
-		}
-		free(pack_name);
+		char *trimmed_path = xstrndup(full_name, full_name_len);
+		packfile_store_load_pack(data->r->objects->packfiles,
+					 trimmed_path, data->local);
+		free(trimmed_path);
 	}
 
 	if (!report_garbage)
diff --git a/packfile.h b/packfile.h
index e751a5d93e..4971f18f51 100644
--- a/packfile.h
+++ b/packfile.h
@@ -119,6 +119,14 @@ void packfile_store_reprepare(struct packfile_store *store);
 void packfile_store_add_pack(struct packfile_store *store,
 			     struct packed_git *pack);
 
+/*
+ * Open the packfile and add it to the store if it isn't yet known. Returns
+ * either the newly opened packfile or the preexisting packfile. Returns a
+ * `NULL` pointer in case the packfile could not be opened.
+ */
+struct packed_git *packfile_store_load_pack(struct packfile_store *store,
+					    const char *idx_path, int local);
+
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
-- 
2.51.0.261.g7ce5a0a67e.dirty
Karthik Nayak· Aug 20, 2025, 13:41 UTC · re: Patrick Steinhardt · lore

Re: [PATCH 12/16] packfile: introduce function to load and add packfiles

Patrick Steinhardt <ps@pks.im> writes:
Show 6 quoted lines
> We have a recurring pattern where we essentially perform an upsert of a
> packfile in case it isn't yet known by the packfile store. The logic to
> do so is non-trivial as we have to reconstruct the packfile's key, check
> the map of packfiles, then create the new packfile and finally add it to
> the store.
>

I was just thinking about this in the previous patch and how it seemed weird that the midx.c file was checking and adding a packfile, so good to see this.

Show 24 quoted lines
> Introduce a new function that does this dance for us. Refactor callsites
> to use it.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  builtin/fast-import.c |  4 ++--
>  builtin/index-pack.c  | 10 +++-------
>  midx.c                | 18 ++----------------
>  packfile.c            | 44 +++++++++++++++++++++++++++++++-------------
>  packfile.h            |  8 ++++++++
>  5 files changed, 46 insertions(+), 38 deletions(-)
>
> diff --git a/builtin/fast-import.c b/builtin/fast-import.c
> index e9d82b31c3..a26e79689d 100644
> --- a/builtin/fast-import.c
> +++ b/builtin/fast-import.c
> @@ -897,11 +897,11 @@ static void end_packfile(void)
>  		idx_name = keep_pack(create_index());
>
>  		/* Register the packfile with core git's machinery. */
> -		new_p = add_packed_git(pack_data->repo, idx_name, strlen(idx_name), 1);
> +		new_p = packfile_store_load_pack(pack_data->repo->objects->packfiles,
> +						 idx_name, 1);
>

I assume that the 'packfile_store_load_pack' function here returns a new/existing packfile.

Show 89 quoted lines
>  		if (!new_p)
>  			die("core git rejected index %s", idx_name);
>  		all_packs[pack_id] = new_p;
> -		packfile_store_add_pack(the_repository->objects->packfiles, new_p);
>  		free(idx_name);
>
>  		/* Print the boundary */
> diff --git a/builtin/index-pack.c b/builtin/index-pack.c
> index ed490dfad4..2b78ba7fe4 100644
> --- a/builtin/index-pack.c
> +++ b/builtin/index-pack.c
> @@ -1640,13 +1640,9 @@ static void final(const char *final_pack_name, const char *curr_pack_name,
>  	rename_tmp_packfile(&final_index_name, curr_index_name, &index_name,
>  			    hash, "idx", 1);
>
> -	if (do_fsck_object) {
> -		struct packed_git *p;
> -		p = add_packed_git(the_repository, final_index_name,
> -				   strlen(final_index_name), 0);
> -		if (p)
> -			packfile_store_add_pack(the_repository->objects->packfiles, p);
> -	}
> +	if (do_fsck_object)
> +		packfile_store_load_pack(the_repository->objects->packfiles,
> +					 final_index_name, 0);
>
>  	if (!from_stdin) {
>  		printf("%s\n", hash_to_hex(hash));
> diff --git a/midx.c b/midx.c
> index 3cfe7884ad..d30feda019 100644
> --- a/midx.c
> +++ b/midx.c
> @@ -454,7 +454,6 @@ int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,
>  		      uint32_t pack_int_id)
>  {
>  	struct strbuf pack_name = STRBUF_INIT;
> -	struct strbuf key = STRBUF_INIT;
>  	struct packed_git *p;
>
>  	pack_int_id = midx_for_pack(&m, pack_int_id);
> @@ -466,22 +465,9 @@ int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,
>
>  	strbuf_addf(&pack_name, "%s/pack/%s", m->object_dir,
>  		    m->pack_names[pack_int_id]);
> -
> -	/* pack_map holds the ".pack" name, but we have the .idx */
> -	strbuf_addbuf(&key, &pack_name);
> -	strbuf_strip_suffix(&key, ".idx");
> -	strbuf_addstr(&key, ".pack");
> -	p = hashmap_get_entry_from_hash(&r->objects->packfiles->map,
> -					strhash(key.buf), key.buf,
> -					struct packed_git, packmap_ent);
> -	if (!p) {
> -		p = add_packed_git(r, pack_name.buf, pack_name.len, m->local);
> -		if (p)
> -			packfile_store_add_pack(r->objects->packfiles, p);
> -	}
> -
> +	p = packfile_store_load_pack(r->objects->packfiles,
> +				     pack_name.buf, m->local);
>  	strbuf_release(&pack_name);
> -	strbuf_release(&key);
>
>  	if (!p) {
>  		m->packs[pack_int_id] = MIDX_PACK_ERROR;
> diff --git a/packfile.c b/packfile.c
> index 8b5e6b96ce..f7916543a6 100644
> --- a/packfile.c
> +++ b/packfile.c
> @@ -793,6 +793,33 @@ void packfile_store_add_pack(struct packfile_store *store,
>  	list_add_tail(&pack->mru, &store->mru);
>  }
>
> +struct packed_git *packfile_store_load_pack(struct packfile_store *store,
> +					    const char *idx_path, int local)
> +{
> +	struct strbuf key = STRBUF_INIT;
> +	struct packed_git *p;
> +
> +	/*
> +	 * We're being called with the path to the index file, but `pack_map`
> +	 * holds the path to the packfile itself.
> +	 */
> +	strbuf_addstr(&key, idx_path);
> +	strbuf_strip_suffix(&key, ".idx");
> +	strbuf_addstr(&key, ".pack");
> +
> +	p = hashmap_get_entry_from_hash(&store->map, strhash(key.buf), key.buf,
> +					struct packed_git, packmap_ent);

I was wondering from an earlier patch too, is there a reason to simply not use 'strmap' for 'packfile_store.map'?

Show 58 quoted lines
> +	if (!p) {
> +		p = add_packed_git(store->odb->repo, idx_path,
> +				   strlen(idx_path), local);
> +		if (p)
> +			packfile_store_add_pack(store, p);
> +	}
> +
> +	strbuf_release(&key);
> +	return p;
> +}
> +
>  void (*report_garbage)(unsigned seen_bits, const char *path);
>
>  static void report_helper(const struct string_list *list,
> @@ -892,23 +919,14 @@ static void prepare_pack(const char *full_name, size_t full_name_len,
>  			 const char *file_name, void *_data)
>  {
>  	struct prepare_pack_data *data = (struct prepare_pack_data *)_data;
> -	struct packed_git *p;
>  	size_t base_len = full_name_len;
>
>  	if (strip_suffix_mem(full_name, &base_len, ".idx") &&
>  	    !(data->m && midx_contains_pack(data->m, file_name))) {
> -		struct hashmap_entry hent;
> -		char *pack_name = xstrfmt("%.*s.pack", (int)base_len, full_name);
> -		unsigned int hash = strhash(pack_name);
> -		hashmap_entry_init(&hent, hash);
> -
> -		/* Don't reopen a pack we already have. */
> -		if (!hashmap_get(&data->r->objects->packfiles->map, &hent, pack_name)) {
> -			p = add_packed_git(data->r, full_name, full_name_len, data->local);
> -			if (p)
> -				packfile_store_add_pack(data->r->objects->packfiles, p);
> -		}
> -		free(pack_name);
> +		char *trimmed_path = xstrndup(full_name, full_name_len);
> +		packfile_store_load_pack(data->r->objects->packfiles,
> +					 trimmed_path, data->local);
> +		free(trimmed_path);
>  	}
>
>  	if (!report_garbage)
> diff --git a/packfile.h b/packfile.h
> index e751a5d93e..4971f18f51 100644
> --- a/packfile.h
> +++ b/packfile.h
> @@ -119,6 +119,14 @@ void packfile_store_reprepare(struct packfile_store *store);
>  void packfile_store_add_pack(struct packfile_store *store,
>  			     struct packed_git *pack);
>
> +/*
> + * Open the packfile and add it to the store if it isn't yet known. Returns
> + * either the newly opened packfile or the preexisting packfile. Returns a
> + * `NULL` pointer in case the packfile could not be opened.
> + */
> +struct packed_git *packfile_store_load_pack(struct packfile_store *store,
> +					    const char *idx_path, int local);
> +
This seems inline with my expectations.
Show 6 quoted lines
>  struct pack_window {
>  	struct pack_window *next;
>  	unsigned char *base;
>
> --
> 2.51.0.261.g7ce5a0a67e.dirty
Patrick Steinhardt· Aug 21, 2025, 06:40 UTC · re: Karthik Nayak · lore

Re: [PATCH 12/16] packfile: introduce function to load and add packfiles

On Wed, Aug 20, 2025 at 06:41:23AM -0700, Karthik Nayak wrote:
Show 16 quoted lines
> Patrick Steinhardt <ps@pks.im> writes:
> > diff --git a/builtin/fast-import.c b/builtin/fast-import.c
> > index e9d82b31c3..a26e79689d 100644
> > --- a/builtin/fast-import.c
> > +++ b/builtin/fast-import.c
> > @@ -897,11 +897,11 @@ static void end_packfile(void)
> >  		idx_name = keep_pack(create_index());
> >
> >  		/* Register the packfile with core git's machinery. */
> > -		new_p = add_packed_git(pack_data->repo, idx_name, strlen(idx_name), 1);
> > +		new_p = packfile_store_load_pack(pack_data->repo->objects->packfiles,
> > +						 idx_name, 1);
> >
> 
> I assume that the 'packfile_store_load_pack' function here returns a
> new/existing packfile.
Yes, exactly.
Show 27 quoted lines
> > diff --git a/packfile.c b/packfile.c
> > index 8b5e6b96ce..f7916543a6 100644
> > --- a/packfile.c
> > +++ b/packfile.c
> > @@ -793,6 +793,33 @@ void packfile_store_add_pack(struct packfile_store *store,
> >  	list_add_tail(&pack->mru, &store->mru);
> >  }
> >
> > +struct packed_git *packfile_store_load_pack(struct packfile_store *store,
> > +					    const char *idx_path, int local)
> > +{
> > +	struct strbuf key = STRBUF_INIT;
> > +	struct packed_git *p;
> > +
> > +	/*
> > +	 * We're being called with the path to the index file, but `pack_map`
> > +	 * holds the path to the packfile itself.
> > +	 */
> > +	strbuf_addstr(&key, idx_path);
> > +	strbuf_strip_suffix(&key, ".idx");
> > +	strbuf_addstr(&key, ".pack");
> > +
> > +	p = hashmap_get_entry_from_hash(&store->map, strhash(key.buf), key.buf,
> > +					struct packed_git, packmap_ent);
> 
> I was wondering from an earlier patch too, is there a reason to simply
> not use 'strmap' for 'packfile_store.map'?

Hm. I cannot think of any, no. I'll leave this as-is in this patch series though and move it into the next one where I'm revamping how packfiles are stored.

Patrick
Patrick Steinhardt· Aug 19, 2025, 08:19 UTC · re: Patrick Steinhardt · lore

[PATCH 13/16] packfile: move `get_multi_pack_index()` into "midx.c"

The `get_multi_pack_index()` function is declared and implemented in the packfile subsystem, even though it really belongs into the multi-pack index subsystem. The reason for this is likely that it needs to call `packfile_store_prepare()`, which is not exposed by the packfile system. In a subsequent commit we're about to add another caller outside of the packfile system though, so we'll have to expose the function anyway.

Do so now already and move `get_multi_pack_index()` into the MIDX subsystem.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 midx.c     |  6 ++++++
 midx.h     |  2 ++
 packfile.c |  8 +-------
 packfile.h | 10 +++++++++-
 4 files changed, 18 insertions(+), 8 deletions(-)
Show changes to 4 files +18 −8

midx.c, midx.h, packfile.c, packfile.h

diff --git a/midx.c b/midx.c
index d30feda019..c1b2f141fa 100644
--- a/midx.c
+++ b/midx.c
@@ -95,6 +95,12 @@ static int midx_read_object_offsets(const unsigned char *chunk_start,
 	return 0;
 }
 
+struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
+{
+	packfile_store_prepare(source->odb->packfiles);
+	return source->midx;
+}
+
 static struct multi_pack_index *load_multi_pack_index_one(struct repository *r,
 							  const char *object_dir,
 							  const char *midx_name,
diff --git a/midx.h b/midx.h
index 076382de8a..8d6ea28682 100644
--- a/midx.h
+++ b/midx.h
@@ -100,6 +100,8 @@ void get_split_midx_filename_ext(const struct git_hash_algo *hash_algo,
 				 struct strbuf *buf, const char *object_dir,
 				 const unsigned char *hash, const char *ext);
 
+struct multi_pack_index *get_multi_pack_index(struct odb_source *source);
+
 struct multi_pack_index *load_multi_pack_index(struct repository *r,
 					       const char *object_dir,
 					       int local);
diff --git a/packfile.c b/packfile.c
index f7916543a6..bc32c45fe6 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1004,7 +1004,7 @@ static void packfile_store_prepare_mru(struct packfile_store *store)
 		list_add_tail(&p->mru, &store->mru);
 }
 
-static void packfile_store_prepare(struct packfile_store *store)
+void packfile_store_prepare(struct packfile_store *store)
 {
 	struct odb_source *source;
 
@@ -1035,12 +1035,6 @@ struct packed_git *get_packed_git(struct repository *r)
 	return r->objects->packfiles->packs;
 }
 
-struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
-{
-	packfile_store_prepare(source->odb->packfiles);
-	return source->midx;
-}
-
 struct packed_git *get_all_packs(struct repository *r)
 {
 	packfile_store_prepare(r->objects->packfiles);
diff --git a/packfile.h b/packfile.h
index 4971f18f51..1522da96f8 100644
--- a/packfile.h
+++ b/packfile.h
@@ -104,6 +104,15 @@ void packfile_store_free(struct packfile_store *store);
  */
 void packfile_store_close(struct packfile_store *store);
 
+/*
+ * Prepare the packfile store by loading packfiles and multi-pack indices for
+ * all alternates. This becomes a no-op if the store is already prepared.
+ *
+ * It shouldn't typically be necessary to call this function directly, as
+ * functions that access the store know to prepare it.
+ */
+void packfile_store_prepare(struct packfile_store *store);
+
 /*
  * Clear the packfile caches and try to look up any new packfiles that have
  * appeared since last preparing the packfiles store.
@@ -205,7 +214,6 @@ extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
 struct packed_git *get_packed_git(struct repository *r);
 struct list_head *get_packed_git_mru(struct repository *r);
-struct multi_pack_index *get_multi_pack_index(struct odb_source *source);
 struct packed_git *get_all_packs(struct repository *r);
 
 /*
-- 
2.51.0.261.g7ce5a0a67e.dirty
Patrick Steinhardt· Aug 19, 2025, 08:19 UTC · re: Patrick Steinhardt · lore

[PATCH 14/16] packfile: remove `get_packed_git()`

We have two different functions to retrieve packfiles for a packfile store:

  - `get_packed_git()` returns the list of packfiles directly.
  - `get_all_packs()` does more work and also prepares packfiles that
    are being indexed by a multi-pack-index.

The distinction is not immediately obvious. Furthermore, to make the situation even worse, `get_packed_git()` would return the same result as `get_all_packs()` once the latter has been called once as they both refer to the same list.

As it turns out, the distinction isn't necessary. We only have a couple of callers of `get_packed_git()`, and all of those callers are prepared to call `get_all_packs()` instead:

  - "builtin/gc.c": We explicitly check how many packfiles aren't
    contained in the multi-pack-index, so loading extra packfiles that
    are indexed by it won't change the result.
  - "builtin/grep.c": We only care `get_packed_git()` to prepare eagerly
    load packfiles. In the preceding commit we have started to expose
    `packfile_store_prepare()`, which is a more direct way of achieving
    the same result.
  - "object-name.c": `find_abbrev_len_for_pack()` and `unique_in_pack()`
    exit early in case the multi-pack index is set, so both callsites of
    `get_packed_git()` know to handle packs loaded via the MIDX already.

Convert all of these sites to use `get_all_packs()` instead and remove `get_packed_git()`.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/gc.c   | 2 +-
 builtin/grep.c | 2 +-
 object-name.c  | 4 ++--
 packfile.c     | 6 ------
 packfile.h     | 1 -
 5 files changed, 4 insertions(+), 11 deletions(-)
Show changes to 5 files +4 −11

builtin/gc.c, builtin/grep.c, object-name.c, packfile.c, packfile.h

diff --git a/builtin/gc.c b/builtin/gc.c
index 1d30d1af2c..565afda51f 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1422,7 +1422,7 @@ static int incremental_repack_auto_condition(struct gc_config *cfg UNUSED)
 	if (incremental_repack_auto_limit < 0)
 		return 1;
 
-	for (p = get_packed_git(the_repository);
+	for (p = get_all_packs(the_repository);
 	     count < incremental_repack_auto_limit && p;
 	     p = p->next) {
 		if (!p->multi_pack_index)
diff --git a/builtin/grep.c b/builtin/grep.c
index 5df6537333..8f0e21bd70 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -1214,7 +1214,7 @@ int cmd_grep(int argc,
 		if (recurse_submodules)
 			repo_read_gitmodules(the_repository, 1);
 		if (startup_info->have_repository)
-			(void)get_packed_git(the_repository);
+			packfile_store_prepare(the_repository->objects->packfiles);
 
 		start_threads(&opt);
 	} else {
diff --git a/object-name.c b/object-name.c
index 44b0d416ac..c87995cc1e 100644
--- a/object-name.c
+++ b/object-name.c
@@ -213,7 +213,7 @@ static void find_short_packed_object(struct disambiguate_state *ds)
 			unique_in_midx(m, ds);
 	}
 
-	for (p = get_packed_git(ds->repo); p && !ds->ambiguous;
+	for (p = get_all_packs(ds->repo); p && !ds->ambiguous;
 	     p = p->next)
 		unique_in_pack(p, ds);
 }
@@ -806,7 +806,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)
 			find_abbrev_len_for_midx(m, mad);
 	}
 
-	for (p = get_packed_git(mad->repo); p; p = p->next)
+	for (p = get_all_packs(mad->repo); p; p = p->next)
 		find_abbrev_len_for_pack(p, mad);
 }
 
diff --git a/packfile.c b/packfile.c
index bc32c45fe6..f1526e361c 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1029,12 +1029,6 @@ void packfile_store_reprepare(struct packfile_store *store)
 	packfile_store_prepare(store);
 }
 
-struct packed_git *get_packed_git(struct repository *r)
-{
-	packfile_store_prepare(r->objects->packfiles);
-	return r->objects->packfiles->packs;
-}
-
 struct packed_git *get_all_packs(struct repository *r)
 {
 	packfile_store_prepare(r->objects->packfiles);
diff --git a/packfile.h b/packfile.h
index 1522da96f8..dff0237092 100644
--- a/packfile.h
+++ b/packfile.h
@@ -212,7 +212,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-struct packed_git *get_packed_git(struct repository *r);
 struct list_head *get_packed_git_mru(struct repository *r);
 struct packed_git *get_all_packs(struct repository *r);
 
-- 
2.51.0.261.g7ce5a0a67e.dirty
Karthik Nayak· Aug 20, 2025, 13:50 UTC · re: Patrick Steinhardt · lore

Re: [PATCH 14/16] packfile: remove `get_packed_git()`

Patrick Steinhardt <ps@pks.im> writes:
Show 8 quoted lines
> We have two different functions to retrieve packfiles for a packfile
> store:
>
>   - `get_packed_git()` returns the list of packfiles directly.
>
>   - `get_all_packs()` does more work and also prepares packfiles that
>     are being indexed by a multi-pack-index.
>

Question, under what situation would a packfile not returned by `get_packed_git()` but be indexed by a multi-pack-index.

Show 15 quoted lines
> The distinction is not immediately obvious. Furthermore, to make the
> situation even worse, `get_packed_git()` would return the same result as
> `get_all_packs()` once the latter has been called once as they both
> refer to the same list.
>
> As it turns out, the distinction isn't necessary. We only have a couple
> of callers of `get_packed_git()`, and all of those callers are prepared
> to call `get_all_packs()` instead:
>
>   - "builtin/gc.c": We explicitly check how many packfiles aren't
>     contained in the multi-pack-index, so loading extra packfiles that
>     are indexed by it won't change the result.
>
>   - "builtin/grep.c": We only care `get_packed_git()` to prepare eagerly
>     load packfiles. In the preceding commit we have started to expose
Nit: the first sentence reads a bit weird.
Show 10 quoted lines
>     `packfile_store_prepare()`, which is a more direct way of achieving
>     the same result.
>
>   - "object-name.c": `find_abbrev_len_for_pack()` and `unique_in_pack()`
>     exit early in case the multi-pack index is set, so both callsites of
>     `get_packed_git()` know to handle packs loaded via the MIDX already.
>
> Convert all of these sites to use `get_all_packs()` instead and remove
> `get_packed_git()`.
>
[snip]
Patrick Steinhardt· Aug 21, 2025, 06:40 UTC · re: Karthik Nayak · lore

Re: [PATCH 14/16] packfile: remove `get_packed_git()`

On Wed, Aug 20, 2025 at 06:50:05AM -0700, Karthik Nayak wrote:
Show 13 quoted lines
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > We have two different functions to retrieve packfiles for a packfile
> > store:
> >
> >   - `get_packed_git()` returns the list of packfiles directly.
> >
> >   - `get_all_packs()` does more work and also prepares packfiles that
> >     are being indexed by a multi-pack-index.
> >
> 
> Question, under what situation would a packfile not returned by
> `get_packed_git()` but be indexed by a multi-pack-index.

Okay, this whole commit message isn't all that enlightening. The only difference between these two functions is that `get_packed_git()` only calls `packfile_store_prepare()` and then returns the list of packs, whereas `get_all_packs()` does the same and then also prepares the pack so that it's properly marked as being indexed by an MIDX.

Now that means that `get_packed_git()` does _less_ work. But both functions already load the MIDX via `packfile_store_prepare()` anyway, so the only difference is that we now also call `prepare_midx_pack()` for each indexed pack. And that function does not do a lot: we have already loaded the pack itself, so `packfile_store_load_pack()` returns the already-in-memory pack. So all we end up doing is to figure out whether the pack is indexed in the MIDX and then store this info in `p->multi_pack_index` as well as `m->packs[pack_int_id]`.

So the amount of extra work shouldn't really matter. I'll rephrase the commit message.

Show 17 quoted lines
> > The distinction is not immediately obvious. Furthermore, to make the
> > situation even worse, `get_packed_git()` would return the same result as
> > `get_all_packs()` once the latter has been called once as they both
> > refer to the same list.
> >
> > As it turns out, the distinction isn't necessary. We only have a couple
> > of callers of `get_packed_git()`, and all of those callers are prepared
> > to call `get_all_packs()` instead:
> >
> >   - "builtin/gc.c": We explicitly check how many packfiles aren't
> >     contained in the multi-pack-index, so loading extra packfiles that
> >     are indexed by it won't change the result.
> >
> >   - "builtin/grep.c": We only care `get_packed_git()` to prepare eagerly
> >     load packfiles. In the preceding commit we have started to expose
> 
> Nit: the first sentence reads a bit weird.
Ah, that was supposed to read "call", not "care".
Patrick
Karthik Nayak· Aug 20, 2025, 13:51 UTC · re: Patrick Steinhardt · lore

Re: [PATCH 14/16] packfile: remove `get_packed_git()`

Patrick Steinhardt <ps@pks.im> writes:
[snip]
Show 13 quoted lines
> diff --git a/builtin/gc.c b/builtin/gc.c
> index 1d30d1af2c..565afda51f 100644
> --- a/builtin/gc.c
> +++ b/builtin/gc.c
> @@ -1422,7 +1422,7 @@ static int incremental_repack_auto_condition(struct gc_config *cfg UNUSED)
>  	if (incremental_repack_auto_limit < 0)
>  		return 1;
>
> -	for (p = get_packed_git(the_repository);
> +	for (p = get_all_packs(the_repository);
>  	     count < incremental_repack_auto_limit && p;
>  	     p = p->next) {
>  		if (!p->multi_pack_index)
Nit: We can get rid of the curly braces here
[snip]
Patrick Steinhardt· Aug 19, 2025, 08:19 UTC · re: Patrick Steinhardt · lore

[PATCH 15/16] packfile: refactor `get_all_packs()` to work on packfile store

The `get_all_packs()` function prepares the packfile store and then returns its packfiles. Refactor it to accept a packfile store instead of a repository to clarify its scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/cat-file.c          |  2 +-
 builtin/count-objects.c     |  2 +-
 builtin/fast-import.c       |  4 ++--
 builtin/fsck.c              |  8 ++++----
 builtin/gc.c                |  8 ++++----
 builtin/pack-objects.c      | 18 +++++++++---------
 builtin/pack-redundant.c    |  4 ++--
 builtin/repack.c            |  6 +++---
 connected.c                 |  2 +-
 http-backend.c              |  4 ++--
 http.c                      |  2 +-
 object-name.c               |  4 ++--
 pack-bitmap.c               |  4 ++--
 pack-objects.c              |  2 +-
 packfile.c                  | 14 +++++++-------
 packfile.h                  |  7 ++++++-
 server-info.c               |  2 +-
 t/helper/test-find-pack.c   |  2 +-
 t/helper/test-pack-mtimes.c |  2 +-
 19 files changed, 51 insertions(+), 46 deletions(-)
Show changes to 19 files +51 −46

builtin/cat-file.c, builtin/count-objects.c, builtin/fast-import.c, builtin/fsck.c, builtin/gc.c, builtin/pack-objects.c, builtin/pack-redundant.c, builtin/repack.c, connected.c, http-backend.c, http.c, object-name.c, pack-bitmap.c, pack-objects.c, packfile.c, packfile.h, server-info.c, t/helper/test-find-pack.c, t/helper/test-pack-mtimes.c

diff --git a/builtin/cat-file.c b/builtin/cat-file.c
index fce0b06451c..7124c43fb14 100644
--- a/builtin/cat-file.c
+++ b/builtin/cat-file.c
@@ -854,7 +854,7 @@ static void batch_each_object(struct batch_options *opt,
 						 batch_one_object_bitmapped, &payload)) {
 		struct packed_git *pack;
 
-		for (pack = get_all_packs(the_repository); pack; pack = pack->next) {
+		for (pack = packfile_store_get_packs(the_repository->objects->packfiles); pack; pack = pack->next) {
 			if (bitmap_index_contains_pack(bitmap, pack) ||
 			    open_pack_index(pack))
 				continue;
diff --git a/builtin/count-objects.c b/builtin/count-objects.c
index a61d3b46aac..471d96a3089 100644
--- a/builtin/count-objects.c
+++ b/builtin/count-objects.c
@@ -129,7 +129,7 @@ int cmd_count_objects(int argc,
 		struct strbuf pack_buf = STRBUF_INIT;
 		struct strbuf garbage_buf = STRBUF_INIT;
 
-		for (p = get_all_packs(the_repository); p; p = p->next) {
+		for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 			if (!p->pack_local)
 				continue;
 			if (open_pack_index(p))
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index a26e79689d5..4f355118a10 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -975,7 +975,7 @@ static int store_object(
 	if (e->idx.offset) {
 		duplicate_count_by_type[type]++;
 		return 1;
-	} else if (find_oid_pack(&oid, get_all_packs(the_repository))) {
+	} else if (find_oid_pack(&oid, packfile_store_get_packs(the_repository->objects->packfiles))) {
 		e->type = type;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
@@ -1175,7 +1175,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 		duplicate_count_by_type[OBJ_BLOB]++;
 		truncate_pack(&checkpoint);
 
-	} else if (find_oid_pack(&oid, get_all_packs(the_repository))) {
+	} else if (find_oid_pack(&oid, packfile_store_get_packs(the_repository->objects->packfiles))) {
 		e->type = OBJ_BLOB;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
diff --git a/builtin/fsck.c b/builtin/fsck.c
index 543a2cdb5cd..e867fd510a3 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -873,14 +873,14 @@ static int check_pack_rev_indexes(struct repository *r, int show_progress)
 	int res = 0;
 
 	if (show_progress) {
-		for (struct packed_git *p = get_all_packs(r); p; p = p->next)
+		for (struct packed_git *p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next)
 			pack_count++;
 		progress = start_delayed_progress(the_repository,
 						  "Verifying reverse pack-indexes", pack_count);
 		pack_count = 0;
 	}
 
-	for (struct packed_git *p = get_all_packs(r); p; p = p->next) {
+	for (struct packed_git *p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next) {
 		int load_error = load_pack_revindex_from_disk(p);
 
 		if (load_error < 0) {
@@ -1010,7 +1010,7 @@ int cmd_fsck(int argc,
 			struct progress *progress = NULL;
 
 			if (show_progress) {
-				for (p = get_all_packs(the_repository); p;
+				for (p = packfile_store_get_packs(the_repository->objects->packfiles); p;
 				     p = p->next) {
 					if (open_pack_index(p))
 						continue;
@@ -1020,7 +1020,7 @@ int cmd_fsck(int argc,
 				progress = start_progress(the_repository,
 							  _("Checking objects"), total);
 			}
-			for (p = get_all_packs(the_repository); p;
+			for (p = packfile_store_get_packs(the_repository->objects->packfiles); p;
 			     p = p->next) {
 				/* verify gives error messages itself */
 				if (verify_pack(the_repository,
diff --git a/builtin/gc.c b/builtin/gc.c
index 565afda51fe..030d0b0c774 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -488,7 +488,7 @@ static struct packed_git *find_base_packs(struct string_list *packs,
 {
 	struct packed_git *p, *base = NULL;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		if (!p->pack_local || p->is_cruft)
 			continue;
 		if (limit) {
@@ -513,7 +513,7 @@ static int too_many_packs(struct gc_config *cfg)
 	if (cfg->gc_auto_pack_limit <= 0)
 		return 0;
 
-	for (cnt = 0, p = get_all_packs(the_repository); p; p = p->next) {
+	for (cnt = 0, p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		if (!p->pack_local)
 			continue;
 		if (p->pack_keep)
@@ -1422,7 +1422,7 @@ static int incremental_repack_auto_condition(struct gc_config *cfg UNUSED)
 	if (incremental_repack_auto_limit < 0)
 		return 1;
 
-	for (p = get_all_packs(the_repository);
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles);
 	     count < incremental_repack_auto_limit && p;
 	     p = p->next) {
 		if (!p->multi_pack_index)
@@ -1491,7 +1491,7 @@ static off_t get_auto_pack_size(void)
 	struct repository *r = the_repository;
 
 	odb_reprepare(r->objects);
-	for (p = get_all_packs(r); p; p = p->next) {
+	for (p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next) {
 		if (p->pack_size > max_size) {
 			second_largest_size = max_size;
 			max_size = p->pack_size;
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 53a22562503..1c24b84510e 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -3855,7 +3855,7 @@ static void read_packs_list_from_stdin(struct rev_info *revs)
 	string_list_sort(&exclude_packs);
 	string_list_remove_duplicates(&exclude_packs, 0);
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		const char *pack_name = pack_basename(p);
 
 		if ((item = string_list_lookup(&include_packs, pack_name)))
@@ -4105,7 +4105,7 @@ static void enumerate_and_traverse_cruft_objects(struct string_list *fresh_packs
 	 * Re-mark only the fresh packs as kept so that objects in
 	 * unknown packs do not halt the reachability traversal early.
 	 */
-	for (p = get_all_packs(the_repository); p; p = p->next)
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next)
 		p->pack_keep_in_core = 0;
 	mark_pack_kept_in_core(fresh_packs, 1);
 
@@ -4142,7 +4142,7 @@ static void read_cruft_objects(void)
 	string_list_sort(&discard_packs);
 	string_list_sort(&fresh_packs);
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		const char *pack_name = pack_basename(p);
 		struct string_list_item *item;
 
@@ -4394,7 +4394,7 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 	struct packed_git *p;
 
 	p = (last_found != (void *)1) ? last_found :
-					get_all_packs(the_repository);
+					packfile_store_get_packs(the_repository->objects->packfiles);
 
 	while (p) {
 		if ((!p->pack_local || p->pack_keep ||
@@ -4404,7 +4404,7 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 			return 1;
 		}
 		if (p == last_found)
-			p = get_all_packs(the_repository);
+			p = packfile_store_get_packs(the_repository->objects->packfiles);
 		else
 			p = p->next;
 		if (p == last_found)
@@ -4441,7 +4441,7 @@ static void loosen_unused_packed_objects(void)
 	uint32_t loosened_objects_nr = 0;
 	struct object_id oid;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		if (!p->pack_local || p->pack_keep || p->pack_keep_in_core)
 			continue;
 
@@ -4747,7 +4747,7 @@ static void add_extra_kept_packs(const struct string_list *names)
 	if (!names->nr)
 		return;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		const char *name = basename(p->pack_name);
 		int i;
 
@@ -5186,7 +5186,7 @@ int cmd_pack_objects(int argc,
 	add_extra_kept_packs(&keep_pack_list);
 	if (ignore_packed_keep_on_disk) {
 		struct packed_git *p;
-		for (p = get_all_packs(the_repository); p; p = p->next)
+		for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next)
 			if (p->pack_local && p->pack_keep)
 				break;
 		if (!p) /* no keep-able packs found */
@@ -5199,7 +5199,7 @@ int cmd_pack_objects(int argc,
 		 * it also covers non-local objects
 		 */
 		struct packed_git *p;
-		for (p = get_all_packs(the_repository); p; p = p->next) {
+		for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 			if (!p->pack_local) {
 				have_non_local_packs = 1;
 				break;
diff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c
index fe81c293e3a..7b2cb3ef1e2 100644
--- a/builtin/pack-redundant.c
+++ b/builtin/pack-redundant.c
@@ -566,7 +566,7 @@ static struct pack_list * add_pack(struct packed_git *p)
 
 static struct pack_list * add_pack_file(const char *filename)
 {
-	struct packed_git *p = get_all_packs(the_repository);
+	struct packed_git *p = packfile_store_get_packs(the_repository->objects->packfiles);
 
 	if (strlen(filename) < 40)
 		die("Bad pack filename: %s", filename);
@@ -581,7 +581,7 @@ static struct pack_list * add_pack_file(const char *filename)
 
 static void load_all(void)
 {
-	struct packed_git *p = get_all_packs(the_repository);
+	struct packed_git *p = packfile_store_get_packs(the_repository->objects->packfiles);
 
 	while (p) {
 		add_pack(p);
diff --git a/builtin/repack.c b/builtin/repack.c
index ee8c80cd95c..6119e236512 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -267,7 +267,7 @@ static void collect_pack_filenames(struct existing_packs *existing,
 	struct packed_git *p;
 	struct strbuf buf = STRBUF_INIT;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		int i;
 		const char *base;
 
@@ -499,7 +499,7 @@ static void init_pack_geometry(struct pack_geometry *geometry,
 	struct packed_git *p;
 	struct strbuf buf = STRBUF_INIT;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		if (args->local && !p->pack_local)
 			/*
 			 * When asked to only repack local packfiles we skip
@@ -1140,7 +1140,7 @@ static void combine_small_cruft_packs(FILE *in, size_t combine_cruft_below_size,
 	struct strbuf buf = STRBUF_INIT;
 	size_t i;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		if (!(p->is_cruft && p->pack_local))
 			continue;
 
diff --git a/connected.c b/connected.c
index d6e9682fd93..d7e07fa6b0d 100644
--- a/connected.c
+++ b/connected.c
@@ -76,7 +76,7 @@ int check_connected(oid_iterate_fn fn, void *cb_data,
 		do {
 			struct packed_git *p;
 
-			for (p = get_all_packs(the_repository); p; p = p->next) {
+			for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 				if (!p->pack_promisor)
 					continue;
 				if (find_pack_entry_one(oid, p))
diff --git a/http-backend.c b/http-backend.c
index d5dfe762bb5..be4d8263a58 100644
--- a/http-backend.c
+++ b/http-backend.c
@@ -608,13 +608,13 @@ static void get_info_packs(struct strbuf *hdr, char *arg UNUSED)
 	size_t cnt = 0;
 
 	select_getanyfile(hdr);
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		if (p->pack_local)
 			cnt++;
 	}
 
 	strbuf_grow(&buf, cnt * 53 + 2);
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		if (p->pack_local)
 			strbuf_addf(&buf, "P %s\n", p->pack_name + objdirlen + 6);
 	}
diff --git a/http.c b/http.c
index af2120b64c7..16a1ab54f34 100644
--- a/http.c
+++ b/http.c
@@ -2416,7 +2416,7 @@ static int fetch_and_setup_pack_index(struct packed_git **packs_head,
 	 * If we already have the pack locally, no need to fetch its index or
 	 * even add it to list; we already have all of its objects.
 	 */
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		if (hasheq(p->hash, sha1, the_repository->hash_algo))
 			return 0;
 	}
diff --git a/object-name.c b/object-name.c
index c87995cc1e6..e346075394d 100644
--- a/object-name.c
+++ b/object-name.c
@@ -213,7 +213,7 @@ static void find_short_packed_object(struct disambiguate_state *ds)
 			unique_in_midx(m, ds);
 	}
 
-	for (p = get_all_packs(ds->repo); p && !ds->ambiguous;
+	for (p = packfile_store_get_packs(ds->repo->objects->packfiles); p && !ds->ambiguous;
 	     p = p->next)
 		unique_in_pack(p, ds);
 }
@@ -806,7 +806,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)
 			find_abbrev_len_for_midx(m, mad);
 	}
 
-	for (p = get_all_packs(mad->repo); p; p = p->next)
+	for (p = packfile_store_get_packs(mad->repo->objects->packfiles); p; p = p->next)
 		find_abbrev_len_for_pack(p, mad);
 }
 
diff --git a/pack-bitmap.c b/pack-bitmap.c
index d14421ee204..67f9e92ec18 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -665,7 +665,7 @@ static int open_pack_bitmap(struct repository *r,
 	struct packed_git *p;
 	int ret = -1;
 
-	for (p = get_all_packs(r); p; p = p->next) {
+	for (p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next) {
 		if (open_pack_bitmap_1(bitmap_git, p) == 0) {
 			ret = 0;
 			/*
@@ -3363,7 +3363,7 @@ int verify_bitmap_files(struct repository *r)
 		free(midx_bitmap_name);
 	}
 
-	for (struct packed_git *p = get_all_packs(r);
+	for (struct packed_git *p = packfile_store_get_packs(r->objects->packfiles);
 	     p; p = p->next) {
 		char *pack_bitmap_name = pack_bitmap_filename(p);
 		res |= verify_bitmap_file(r->hash_algo, pack_bitmap_name);
diff --git a/pack-objects.c b/pack-objects.c
index a9d9855063a..5506f12293c 100644
--- a/pack-objects.c
+++ b/pack-objects.c
@@ -95,7 +95,7 @@ static void prepare_in_pack_by_idx(struct packing_data *pdata)
 	 * (i.e. in_pack_idx also zero) should return NULL.
 	 */
 	mapping[cnt++] = NULL;
-	for (p = get_all_packs(pdata->repo); p; p = p->next, cnt++) {
+	for (p = packfile_store_get_packs(pdata->repo->objects->packfiles); p; p = p->next, cnt++) {
 		if (cnt == nr) {
 			free(mapping);
 			return;
diff --git a/packfile.c b/packfile.c
index f1526e361c2..b60faf5c3e7 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1029,19 +1029,19 @@ void packfile_store_reprepare(struct packfile_store *store)
 	packfile_store_prepare(store);
 }
 
-struct packed_git *get_all_packs(struct repository *r)
+struct packed_git *packfile_store_get_packs(struct packfile_store *store)
 {
-	packfile_store_prepare(r->objects->packfiles);
+	packfile_store_prepare(store);
 
-	for (struct odb_source *source = r->objects->sources; source; source = source->next) {
+	for (struct odb_source *source = store->odb->sources; source; source = source->next) {
 		struct multi_pack_index *m = source->midx;
 		if (!m)
 			continue;
 		for (uint32_t i = 0; i < m->num_packs + m->num_packs_in_base; i++)
-			prepare_midx_pack(r, m, i);
+			prepare_midx_pack(store->odb->repo, m, i);
 	}
 
-	return r->objects->packfiles->packs;
+	return store->packs;
 }
 
 struct list_head *get_packed_git_mru(struct repository *r)
@@ -2101,7 +2101,7 @@ struct packed_git **kept_pack_cache(struct repository *r, unsigned flags)
 		 * covers, one kept and one not kept, but the midx returns only
 		 * the non-kept version.
 		 */
-		for (p = get_all_packs(r); p; p = p->next) {
+		for (p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next) {
 			if ((p->pack_keep && (flags & ON_DISK_KEEP_PACKS)) ||
 			    (p->pack_keep_in_core && (flags & IN_CORE_KEEP_PACKS))) {
 				ALLOC_GROW(packs, nr + 1, alloc);
@@ -2198,7 +2198,7 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 	int r = 0;
 	int pack_errors = 0;
 
-	for (p = get_all_packs(repo); p; p = p->next) {
+	for (p = packfile_store_get_packs(repo->objects->packfiles); p; p = p->next) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
 		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&
diff --git a/packfile.h b/packfile.h
index dff02370924..8f501f00947 100644
--- a/packfile.h
+++ b/packfile.h
@@ -128,6 +128,12 @@ void packfile_store_reprepare(struct packfile_store *store);
 void packfile_store_add_pack(struct packfile_store *store,
 			     struct packed_git *pack);
 
+/*
+ * Get all packs managed by the given store, including packfiles that are
+ * referenced by multi-pack indices.
+ */
+struct packed_git *packfile_store_get_packs(struct packfile_store *store);
+
 /*
  * Open the packfile and add it to the store if it isn't yet known. Returns
  * either the newly opened packfile or the preexisting packfile. Returns a
@@ -213,7 +219,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
 struct list_head *get_packed_git_mru(struct repository *r);
-struct packed_git *get_all_packs(struct repository *r);
 
 /*
  * Give a rough count of objects in the repository. This sacrifices accuracy
diff --git a/server-info.c b/server-info.c
index 9bb30d9ab71..79234c7fed3 100644
--- a/server-info.c
+++ b/server-info.c
@@ -292,7 +292,7 @@ static void init_pack_info(struct repository *r, const char *infofile, int force
 	int i;
 	size_t alloc = 0;
 
-	for (p = get_all_packs(r); p; p = p->next) {
+	for (p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next) {
 		/* we ignore things on alternate path since they are
 		 * not available to the pullers in general.
 		 */
diff --git a/t/helper/test-find-pack.c b/t/helper/test-find-pack.c
index 611a13a3261..183a777fc54 100644
--- a/t/helper/test-find-pack.c
+++ b/t/helper/test-find-pack.c
@@ -39,7 +39,7 @@ int cmd__find_pack(int argc, const char **argv)
 	if (repo_get_oid(the_repository, argv[0], &oid))
 		die("cannot parse %s as an object name", argv[0]);
 
-	for (p = get_all_packs(the_repository); p; p = p->next)
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next)
 		if (find_pack_entry_one(&oid, p)) {
 			printf("%s\n", p->pack_name);
 			actual_count++;
diff --git a/t/helper/test-pack-mtimes.c b/t/helper/test-pack-mtimes.c
index d51aaa3dc40..cfdfae77a6c 100644
--- a/t/helper/test-pack-mtimes.c
+++ b/t/helper/test-pack-mtimes.c
@@ -37,7 +37,7 @@ int cmd__pack_mtimes(int argc, const char **argv)
 	if (argc != 2)
 		usage(pack_mtimes_usage);
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		strbuf_addstr(&buf, basename(p->pack_name));
 		strbuf_strip_suffix(&buf, ".pack");
 		strbuf_addstr(&buf, ".mtimes");
-- 
2.51.0.261.g7ce5a0a67e.dirty
Karthik Nayak· Aug 20, 2025, 13:53 UTC · re: Patrick Steinhardt · lore

Re: [PATCH 15/16] packfile: refactor `get_all_packs()` to work on packfile store

Patrick Steinhardt <ps@pks.im> writes:
> The `get_all_packs()` function prepares the packfile store and then
> returns its packfiles. Refactor it to accept a packfile store instead of
> a repository to clarify its scope.
>
[snip]
Nit: From running the clang formatter, small cleanups:
Show changes to 3 files +5 −10

builtin/gc.c, http-backend.c, http.c

diff --git a/builtin/gc.c b/builtin/gc.c
index 030d0b0c77..41433b31ed 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1424,10 +1424,9 @@ static int
incremental_repack_auto_condition(struct gc_config *cfg UNUSED)

 	for (p = packfile_store_get_packs(the_repository->objects->packfiles);
 	     count < incremental_repack_auto_limit && p;
-	     p = p->next) {
+	     p = p->next)
 		if (!p->multi_pack_index)
 			count++;
-	}

 	return count >= incremental_repack_auto_limit;
 }
@@ -1491,13 +1490,12 @@ static off_t get_auto_pack_size(void)
 	struct repository *r = the_repository;

 	odb_reprepare(r->objects);
-	for (p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next) {
+	for (p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next)
 		if (p->pack_size > max_size) {
 			second_largest_size = max_size;
 			max_size = p->pack_size;
 		} else if (p->pack_size > second_largest_size)
 			second_largest_size = p->pack_size;
-	}

 	result_size = second_largest_size + 1;

diff --git a/http-backend.c b/http-backend.c
index be4d8263a5..c5779db79d 100644
--- a/http-backend.c
+++ b/http-backend.c
@@ -608,16 +608,14 @@ static void get_info_packs(struct strbuf *hdr,
char *arg UNUSED)
 	size_t cnt = 0;

 	select_getanyfile(hdr);
-	for (p = packfile_store_get_packs(the_repository->objects->packfiles);
p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles);
p; p = p->next)
 		if (p->pack_local)
 			cnt++;
-	}

 	strbuf_grow(&buf, cnt * 53 + 2);
-	for (p = packfile_store_get_packs(the_repository->objects->packfiles);
p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles);
p; p = p->next)
 		if (p->pack_local)
 			strbuf_addf(&buf, "P %s\n", p->pack_name + objdirlen + 6);
-	}
 	strbuf_addch(&buf, '\n');

 	hdr_nocache(hdr);
diff --git a/http.c b/http.c
index 16a1ab54f3..bf8711d6f8 100644
--- a/http.c
+++ b/http.c
@@ -2416,10 +2416,9 @@ static int fetch_and_setup_pack_index(struct
packed_git **packs_head,
 	 * If we already have the pack locally, no need to fetch its index or
 	 * even add it to list; we already have all of its objects.
 	 */
-	for (p = packfile_store_get_packs(the_repository->objects->packfiles);
p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles);
p; p = p->next)
 		if (hasheq(p->hash, sha1, the_repository->hash_algo))
 			return 0;
-	}

 	tmp_idx = fetch_pack_index(sha1, base_url);
 	if (!tmp_idx)
Patrick Steinhardt· Aug 21, 2025, 06:40 UTC · re: Karthik Nayak · lore

Re: [PATCH 15/16] packfile: refactor `get_all_packs()` to work on packfile store

On Wed, Aug 20, 2025 at 06:53:34AM -0700, Karthik Nayak wrote:
Show 10 quoted lines
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > The `get_all_packs()` function prepares the packfile store and then
> > returns its packfiles. Refactor it to accept a packfile store instead of
> > a repository to clarify its scope.
> >
> 
> [snip]
> 
> Nit: From running the clang formatter, small cleanups:

These would all be while-at-it changes though, so I'd prefer to punt on them for now.

Patrick
Patrick Steinhardt· Aug 19, 2025, 08:19 UTC · re: Patrick Steinhardt · lore

[PATCH 16/16] packfile: refactor `get_packed_git_mru()` to work on packfile store

The `get_packed_git_mru()` function prepares the packfile store and then returns its packfiles in most-recently-used order. Refactor it to accept a packfile store instead of a repository to clarify its scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/pack-objects.c | 4 ++--
 packfile.c             | 6 +++---
 packfile.h             | 7 +++++--
 3 files changed, 10 insertions(+), 7 deletions(-)
Show changes to 3 files +10 −7

builtin/pack-objects.c, packfile.c, packfile.h

diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 1c24b84510..4e75f14df1 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -1748,12 +1748,12 @@ static int want_object_in_pack_mtime(const struct object_id *oid,
 		}
 	}
 
-	list_for_each(pos, get_packed_git_mru(the_repository)) {
+	list_for_each(pos, packfile_store_get_packs_mru(the_repository->objects->packfiles)) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		want = want_object_in_pack_one(p, oid, exclude, found_pack, found_offset, found_mtime);
 		if (!exclude && want > 0)
 			list_move(&p->mru,
-				  get_packed_git_mru(the_repository));
+				  packfile_store_get_packs_mru(the_repository->objects->packfiles));
 		if (want != -1)
 			return want;
 	}
diff --git a/packfile.c b/packfile.c
index b60faf5c3e..69844dd6cc 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1044,10 +1044,10 @@ struct packed_git *packfile_store_get_packs(struct packfile_store *store)
 	return store->packs;
 }
 
-struct list_head *get_packed_git_mru(struct repository *r)
+struct list_head *packfile_store_get_packs_mru(struct packfile_store *store)
 {
-	packfile_store_prepare(r->objects->packfiles);
-	return &r->objects->packfiles->mru;
+	packfile_store_prepare(store);
+	return &store->mru;
 }
 
 /*
diff --git a/packfile.h b/packfile.h
index 8f501f0094..f6dc26d08a 100644
--- a/packfile.h
+++ b/packfile.h
@@ -134,6 +134,11 @@ void packfile_store_add_pack(struct packfile_store *store,
  */
 struct packed_git *packfile_store_get_packs(struct packfile_store *store);
 
+/*
+ * Get all packs in most-recently-used order.
+ */
+struct list_head *packfile_store_get_packs_mru(struct packfile_store *store);
+
 /*
  * Open the packfile and add it to the store if it isn't yet known. Returns
  * either the newly opened packfile or the preexisting packfile. Returns a
@@ -218,8 +223,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-struct list_head *get_packed_git_mru(struct repository *r);
-
 /*
  * Give a rough count of objects in the repository. This sacrifices accuracy
  * for speed.
-- 
2.51.0.261.g7ce5a0a67e.dirty
Junio C Hamano· Aug 19, 2025, 17:13 UTC · re: Patrick Steinhardt · lore

Re: [PATCH 00/16] packfile: carve out a new packfile store

Patrick Steinhardt <ps@pks.im> writes:
Show 9 quoted lines
> So we don't really have a central data structure that tracks our
> packfiles, and consequently responsibilities aren't always clear cut.
> A consequence for the upcoming pluggable object databases is that this
> makes it very hard to move management of packfiles from the object
> database level down into the object database source.
>
> This patch series introduces a new `struct packfile_store`, which is
> about to become the single source of truth for managing packfiles, and
> carves out the packfile store subsystem.
Nice.
Show 10 quoted lines
> This is the first step to make packfiles work with pluggable object
> databases. Next steps will be to:
>
>   - Move the `struct packed_git::next` and `struct packed::mru_head`
>     pointers into the packfile store so that `struct packed_git` only
>     tracks a single packfile.
>
>   - Push the `struct packfile_store` down one level so that it's not
>     hosted by the object database anymore, but instead by the object
>     database source.

Makes sense. Each packfile belong to a single $GIT_DIR/objects/ and together with loose object files in there form a set of objects in a single object store. When alternates are in effect, I think we still out of convenience link these packfiles taken from multiple places into a single list, which a series like this one may have to untangle.

Thanks.
Karthik Nayak· Aug 20, 2025, 13:55 UTC · re: Patrick Steinhardt · lore

Re: [PATCH 00/16] packfile: carve out a new packfile store

Patrick Steinhardt <ps@pks.im> writes:
Show 36 quoted lines
> Hi,
>
> information about a object database's packfiles is currently distributed
> across two different structures:
>
>   - `struct packed_git` contains the `next` pointer as well as the
>     `mru_head`, both of which serve to store the list of packfiles.
>
>   - `struct object_database` contains several fields that relate to the
>     packfiles.
>
> So we don't really have a central data structure that tracks our
> packfiles, and consequently responsibilities aren't always clear cut.
> A consequence for the upcoming pluggable object databases is that this
> makes it very hard to move management of packfiles from the object
> database level down into the object database source.
>
> This patch series introduces a new `struct packfile_store`, which is
> about to become the single source of truth for managing packfiles, and
> carves out the packfile store subsystem.
>
> This is the first step to make packfiles work with pluggable object
> databases. Next steps will be to:
>
>   - Move the `struct packed_git::next` and `struct packed::mru_head`
>     pointers into the packfile store so that `struct packed_git` only
>     tracks a single packfile.
>
>   - Push the `struct packfile_store` down one level so that it's not
>     hosted by the object database anymore, but instead by the object
>     database source.
>
> Thanks!
>
> Patrick
>
Hello Patrick,

I took some time to read through your patches and comment on them. I only had some small nits.

Overall they look good to me, but I must say I don't know much about this part of the codebase.

Thanks, Karthik

[snip]
Patrick Steinhardt· Aug 21, 2025, 07:38 UTC · re: Patrick Steinhardt · lore

[PATCH v2 00/16] packfile: carve out a new packfile store

Hi,

information about a object database's packfiles is currently distributed across two different structures:

  - `struct packed_git` contains the `next` pointer as well as the
    `mru_head`, both of which serve to store the list of packfiles.
  - `struct object_database` contains several fields that relate to the
    packfiles.

So we don't really have a central data structure that tracks our packfiles, and consequently responsibilities aren't always clear cut. A consequence for the upcoming pluggable object databases is that this makes it very hard to move management of packfiles from the object database level down into the object database source.

This patch series introduces a new `struct packfile_store`, which is about to become the single source of truth for managing packfiles, and carves out the packfile store subsystem.

This is the first step to make packfiles work with pluggable object databases. Next steps will be to:

  - Move the `struct packed_git::next` and `struct packed::mru_head`
    pointers into the packfile store so that `struct packed_git` only
    tracks a single packfile.
  - Push the `struct packfile_store` down one level so that it's not
    hosted by the object database anymore, but instead by the object
    database source.
Changes in v2:
  - Convert the `initialized` flag into a boolean.
  - Polish some commit messages.
  - Some smaller formatting changes to the layout of `struct
    object_database`.
  - Link to v1: https://lore.kernel.org/r/20250819-b4-pks-packfiles-store-v1-0-1660842e125a@pks.im
Thanks!
Patrick
---
Patrick Steinhardt (16):
      packfile: introduce a new `struct packfile_store`
      odb: move list of packfiles into `struct packfile_store`
      odb: move initialization bit into `struct packfile_store`
      odb: move packfile map into `struct packfile_store`
      odb: move MRU list of packfiles into `struct packfile_store`
      odb: move kept cache into `struct packfile_store`
      packfile: reorder functions to avoid function declaration
      packfile: refactor `prepare_packed_git()` to work on packfile store
      packfile: split up responsibilities of `reprepare_packed_git()`
      packfile: refactor `install_packed_git()` to work on packfile store
      packfile: always add packfiles to MRU when adding a pack
      packfile: introduce function to load and add packfiles
      packfile: move `get_multi_pack_index()` into "midx.c"
      packfile: remove `get_packed_git()`
      packfile: refactor `get_all_packs()` to work on packfile store
      packfile: refactor `get_packed_git_mru()` to work on packfile store
 builtin/backfill.c          |   2 +-
 builtin/cat-file.c          |   2 +-
 builtin/count-objects.c     |   2 +-
 builtin/fast-import.c       |   8 +-
 builtin/fsck.c              |   8 +-
 builtin/gc.c                |  12 +-
 builtin/grep.c              |   2 +-
 builtin/index-pack.c        |  10 +-
 builtin/pack-objects.c      |  22 ++--
 builtin/pack-redundant.c    |   4 +-
 builtin/receive-pack.c      |   2 +-
 builtin/repack.c            |   8 +-
 bulk-checkin.c              |   2 +-
 connected.c                 |   4 +-
 fetch-pack.c                |   4 +-
 http-backend.c              |   4 +-
 http.c                      |   4 +-
 http.h                      |   2 +-
 midx.c                      |  26 ++--
 midx.h                      |   2 +
 object-name.c               |   6 +-
 odb.c                       |  37 ++++--
 odb.h                       |  34 ++---
 pack-bitmap.c               |   4 +-
 pack-objects.c              |   2 +-
 packfile.c                  | 293 ++++++++++++++++++++++++--------------------
 packfile.h                  | 110 ++++++++++++++---
 server-info.c               |   2 +-
 t/helper/test-find-pack.c   |   2 +-
 t/helper/test-pack-mtimes.c |   2 +-
 transport-helper.c          |   2 +-
 31 files changed, 353 insertions(+), 271 deletions(-)
Range-diff versus v1:
 1:  49001587ad !  1:  5f89325948 packfile: introduce a new `struct packfile_store`
    @@ odb.h: struct object_database {
      	 *
      	 * should only be accessed directly by packfile.c
      	 */
    +-
     +	struct packfile_store *packfiles;
    - 
      	struct packed_git *packed_git;
      	/* A most-recently-used ordered version of the packed_git list. */
    + 	struct list_head packed_git_mru;
     
      ## packfile.c ##
     @@ packfile.c: int parse_pack_header_option(const char *in, unsigned char *out, unsigned int *l
 2:  40e8926e48 !  2:  bf3e61a3ac odb: move list of packfiles into `struct packfile_store`
    @@ odb.c: void odb_clear(struct object_database *o)
     
      ## odb.h ##
     @@ odb.h: struct object_database {
    + 	 * should only be accessed directly by packfile.c
      	 */
      	struct packfile_store *packfiles;
    - 
     -	struct packed_git *packed_git;
      	/* A most-recently-used ordered version of the packed_git list. */
      	struct list_head packed_git_mru;
    @@ packfile.h: struct packed_git {
     +
     +	/*
     +	 * The list of packfiles in the order in which they are being added to
    -+	 * the store. The local packfile typically sits at the head of this
    -+	 * list.
    ++	 * the store.
     +	 */
     +	struct packed_git *packs;
      };
 3:  7dacc2aeb3 !  3:  e0a167b952 odb: move initialization bit into `struct packfile_store`
    @@ Commit message
         database. With the introduction of the `struct packfile_store` we have a
         better place to host this bit though.
     
    -    Move it accordingly.
    +    Move it accordingly. While at it, convert the field into a boolean now
    +    that we're allowed to use them in our code base.
     
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
     
    @@ packfile.c: static void prepare_packed_git(struct repository *r)
      
      	prepare_packed_git_mru(r);
     -	r->objects->packed_git_initialized = 1;
    -+	r->objects->packfiles->initialized = 1;
    ++	r->objects->packfiles->initialized = true;
      }
      
      void reprepare_packed_git(struct repository *r)
    @@ packfile.c: void reprepare_packed_git(struct repository *r)
      
      	r->objects->approximate_object_count_valid = 0;
     -	r->objects->packed_git_initialized = 0;
    -+	r->objects->packfiles->initialized = 0;
    ++	r->objects->packfiles->initialized = false;
      	prepare_packed_git(r);
      	obj_read_unlock();
      }
     
      ## packfile.h ##
     @@ packfile.h: struct packfile_store {
    - 	 * list.
    + 	 * the store.
      	 */
      	struct packed_git *packs;
     +
    @@ packfile.h: struct packfile_store {
     +	 * Whether packfiles have already been populated with this store's
     +	 * packs.
     +	 */
    -+	unsigned initialized : 1;
    ++	bool initialized;
      };
      
      /*
 4:  970ddf62ba =  4:  5940ac0cc3 odb: move packfile map into `struct packfile_store`
 5:  300f68f5f3 !  5:  e1c81f4258 odb: move MRU list of packfiles into `struct packfile_store`
    @@ odb.h
      #include "oidmap.h"
      #include "string-list.h"
     @@ odb.h: struct object_database {
    + 	 * should only be accessed directly by packfile.c
      	 */
      	struct packfile_store *packfiles;
    - 
     -	/* A most-recently-used ordered version of the packed_git list. */
     -	struct list_head packed_git_mru;
     -
 6:  97d9a8322e !  6:  96eb610300 odb: move kept cache into `struct packfile_store`
    @@ Commit message
     
         Move the cache accordingly.
     
    +    This moves the last bit of packfile-related state from the object
    +    database into the packfile store. Adapt the comment for the `packfiles`
    +    pointer in `struct object_database` to reflect this.
    +
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
     
      ## odb.h ##
    @@ odb.h: struct object_database {
     +	 * Should only be accessed directly by packfile.c
      	 */
      	struct packfile_store *packfiles;
    - 
     -	struct {
     -		struct packed_git **packs;
     -		unsigned flags;
     -	} kept_pack_cache;
    --
    + 
      	/*
      	 * This is meant to hold a *small* number of objects that you would
    - 	 * want odb_read_object() to be able to return, but yet you do not want
     
      ## packfile.c ##
     @@ packfile.c: int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 7:  7e5c801326 !  7:  2a4392afeb packfile: reorder functions to avoid function declaration
    @@ Metadata
      ## Commit message ##
         packfile: reorder functions to avoid function declaration
     
    -    Reorder functions so that we can avoid an extra declaration of
    +    Reorder functions so that we can avoid a forward declaration of
         `prepare_packed_git()`.
     
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
 8:  967232564f !  8:  e92deae728 packfile: refactor `prepare_packed_git()` to work on packfile store
    @@ packfile.c: static int sort_pack(const struct packed_git *a, const struct packed
     +	sort_packs(&store->packs, sort_pack);
      
     -	prepare_packed_git_mru(r);
    --	r->objects->packfiles->initialized = 1;
    +-	r->objects->packfiles->initialized = true;
     +	packfile_store_prepare_mru(store);
    -+	store->initialized = 1;
    ++	store->initialized = true;
      }
      
      void reprepare_packed_git(struct repository *r)
     @@ packfile.c: void reprepare_packed_git(struct repository *r)
      
      	r->objects->approximate_object_count_valid = 0;
    - 	r->objects->packfiles->initialized = 0;
    + 	r->objects->packfiles->initialized = false;
     -	prepare_packed_git(r);
     +	packfile_store_prepare(r->objects->packfiles);
      	obj_read_unlock();
 9:  11860d282d !  9:  2fcc09332f packfile: split up responsibilities of `reprepare_packed_git()`
    @@ odb.h: struct object_database {
     
      ## packfile.c ##
     @@ packfile.c: static void packfile_store_prepare(struct packfile_store *store)
    - 	store->initialized = 1;
    + 	store->initialized = true;
      }
      
     -void reprepare_packed_git(struct repository *r)
    @@ packfile.c: static void packfile_store_prepare(struct packfile_store *store)
     -		odb_clear_loose_cache(source);
     -
     -	r->objects->approximate_object_count_valid = 0;
    --	r->objects->packfiles->initialized = 0;
    +-	r->objects->packfiles->initialized = false;
     -	packfile_store_prepare(r->objects->packfiles);
     -	obj_read_unlock();
    -+	store->initialized = 0;
    ++	store->initialized = false;
     +	packfile_store_prepare(store);
      }
      
10:  7472988a1f = 10:  3097fd3d85 packfile: refactor `install_packed_git()` to work on packfile store
11:  e705f1fbf4 = 11:  aff98b7a67 packfile: always add packfiles to MRU when adding a pack
12:  70b4d4921e = 12:  ab55a14dbc packfile: introduce function to load and add packfiles
13:  80326900fa = 13:  abb34ffc8a packfile: move `get_multi_pack_index()` into "midx.c"
14:  98219d6ed4 ! 14:  273f2d698e packfile: remove `get_packed_git()`
    @@ Commit message
         We have two different functions to retrieve packfiles for a packfile
         store:
     
    -      - `get_packed_git()` returns the list of packfiles directly.
    +      - `get_packed_git()` returns the list of packfiles after having called
    +        `prepare_packed_git()`.
     
    -      - `get_all_packs()` does more work and also prepares packfiles that
    -        are being indexed by a multi-pack-index.
    +      - `get_all_packs()` calls `prepare_packed_git()`, as well, but also
    +        calls `prepare_midx_pack()` for each pack.
     
    -    The distinction is not immediately obvious. Furthermore, to make the
    -    situation even worse, `get_packed_git()` would return the same result as
    -    `get_all_packs()` once the latter has been called once as they both
    -    refer to the same list.
    +    This means that the latter function also properly loads the info of
    +    whether or not a packfile is part of a multi-pack index. Preparing this
    +    extra information also shouldn't be significantly more expensive:
     
    -    As it turns out, the distinction isn't necessary. We only have a couple
    -    of callers of `get_packed_git()`, and all of those callers are prepared
    -    to call `get_all_packs()` instead:
    +      - We have already loaded all packfiles via `prepare_packed_git_one()`.
    +        So given that multi-pack indices may only refer to packfiles in the
    +        same object directory we know that we already loaded each packfile.
     
    -      - "builtin/gc.c": We explicitly check how many packfiles aren't
    -        contained in the multi-pack-index, so loading extra packfiles that
    -        are indexed by it won't change the result.
    +      - The multi-pack index was prepared via `packfile_store_prepare()`
    +        already, which calls `prepare_multi_pack_index_one()`.
     
    -      - "builtin/grep.c": We only care `get_packed_git()` to prepare eagerly
    -        load packfiles. In the preceding commit we have started to expose
    -        `packfile_store_prepare()`, which is a more direct way of achieving
    -        the same result.
    +      - So all that remains to be done is to look up the index of the pack
    +        in its multi-pack index so that we can store that info in both the
    +        pack itself and the MIDX.
     
    -      - "object-name.c": `find_abbrev_len_for_pack()` and `unique_in_pack()`
    -        exit early in case the multi-pack index is set, so both callsites of
    -        `get_packed_git()` know to handle packs loaded via the MIDX already.
    +    So it is somewhat confusing to readers that one of these two functions
    +    claims to load "all" packfiles while the other one doesn't, even though
    +    the ultimate difference is way more nuanced.
     
         Convert all of these sites to use `get_all_packs()` instead and remove
    -    `get_packed_git()`.
    +    `get_packed_git()`. There doesn't seem to be a good reason to discern
    +    these two functions.
     
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
     
15:  3b39e6c4c4 = 15:  15bc2b858c packfile: refactor `get_all_packs()` to work on packfile store
16:  4b2a3429d6 = 16:  0bfb6cf52a packfile: refactor `get_packed_git_mru()` to work on packfile store

--- base-commit: c44beea485f0f2feaf460e2ac87fdd5608d63cf0 change-id: 20250806-b4-pks-packfiles-store-a44a608ca396

Patrick Steinhardt· Aug 21, 2025, 07:38 UTC · re: Patrick Steinhardt · lore

[PATCH v2 01/16] packfile: introduce a new `struct packfile_store`

Information about a object database's packfiles is currently distributed across two different structures:

  - `struct packed_git` contains the `next` pointer as well as the
    `mru_head`, both of which serve to store the list of packfiles.
  - `struct object_database` contains several fields that relate to the
    packfiles.

So we don't really have a central data structure that tracks our packfiles, and consequently responsibilities aren't always clear cut. A consequence for the upcoming pluggable object databases is that this makes it very hard to move management of packfiles from the object database level down into the object database source.

Introduce a new `struct packfile_store` which is about to become the single source of truth for managing packfiles. Right now this data structure doesn't yet contain anything, but in subsequent patches we will move all data structures that relate to packfiles and that are currently contained in `struct object_database` into this new home.

Note that this is only a first step: most importantly, we won't (yet) move the `struct packed_git::next` pointer around. This will happen in a subsequent patch series though so that `struct packed_git` will really only host information about the specific packfile it represents.

Further note that the new structure still sits at the wrong level at the end of this patch series: as mentioned, it should eventually sit at the level of the object database source, not at the object database level. But introducing the packfile store now already makes it way easier to eventually push down the now-selfcontained data structure by one level.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb.c      |  1 +
 odb.h      |  3 ++-
 packfile.c | 13 +++++++++++++
 packfile.h | 18 ++++++++++++++++++
 4 files changed, 34 insertions(+), 1 deletion(-)
Show changes to 4 files +34 −1

odb.c, odb.h, packfile.c, packfile.h

diff --git a/odb.c b/odb.c
index 2a92a018c4..34b70d0074 100644
--- a/odb.c
+++ b/odb.c
@@ -996,6 +996,7 @@ struct object_database *odb_new(struct repository *repo)
 
 	memset(o, 0, sizeof(*o));
 	o->repo = repo;
+	o->packfiles = packfile_store_new(o);
 	INIT_LIST_HEAD(&o->packed_git_mru);
 	hashmap_init(&o->pack_map, pack_map_entry_cmp, NULL, 0);
 	pthread_mutex_init(&o->replace_mutex, NULL);
diff --git a/odb.h b/odb.h
index 3dfc66d75a..08c3a01f3b 100644
--- a/odb.h
+++ b/odb.h
@@ -83,6 +83,7 @@ struct odb_source {
 };
 
 struct packed_git;
+struct packfile_store;
 struct cached_object_entry;
 
 /*
@@ -128,7 +129,7 @@ struct object_database {
 	 *
 	 * should only be accessed directly by packfile.c
 	 */
-
+	struct packfile_store *packfiles;
 	struct packed_git *packed_git;
 	/* A most-recently-used ordered version of the packed_git list. */
 	struct list_head packed_git_mru;
diff --git a/packfile.c b/packfile.c
index 5d73932f50..8fbf1cfc2d 100644
--- a/packfile.c
+++ b/packfile.c
@@ -2333,3 +2333,16 @@ int parse_pack_header_option(const char *in, unsigned char *out, unsigned int *l
 	*len = hdr - out;
 	return 0;
 }
+
+struct packfile_store *packfile_store_new(struct object_database *odb)
+{
+	struct packfile_store *store;
+	CALLOC_ARRAY(store, 1);
+	store->odb = odb;
+	return store;
+}
+
+void packfile_store_free(struct packfile_store *store)
+{
+	free(store);
+}
diff --git a/packfile.h b/packfile.h
index f16753f2a9..8d31fd619a 100644
--- a/packfile.h
+++ b/packfile.h
@@ -52,6 +52,24 @@ struct packed_git {
 	char pack_name[FLEX_ARRAY]; /* more */
 };
 
+/*
+ * A store that manages packfiles for a given object database.
+ */
+struct packfile_store {
+	struct object_database *odb;
+};
+
+/*
+ * Allocate and initialize a new empty packfile store for the given object
+ * database.
+ */
+struct packfile_store *packfile_store_new(struct object_database *odb);
+
+/*
+ * Free the packfile store and all its associated state.
+ */
+void packfile_store_free(struct packfile_store *store);
+
 static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
 				     const struct hashmap_entry *entry,
 				     const struct hashmap_entry *entry2,
-- 
2.51.0.261.g7ce5a0a67e.dirty
Patrick Steinhardt· Aug 21, 2025, 07:39 UTC · re: Patrick Steinhardt · lore

[PATCH v2 02/16] odb: move list of packfiles into `struct packfile_store`

The object database tracks the list of packfiles it currently knows about. With the introduction of the `struct packfile_store` we have a better place to host this list though.

Move the list accordingly. Extract the logic from `odb_clear()` that knows to close all such packfiles and move it into the new subsystem, as well.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb.c      | 11 +----------
 odb.h      |  1 -
 packfile.c | 47 ++++++++++++++++++++++++++++++-----------------
 packfile.h | 15 ++++++++++++++-
 4 files changed, 45 insertions(+), 29 deletions(-)
Show changes to 4 files +45 −29

odb.c, odb.h, packfile.c, packfile.h

diff --git a/odb.c b/odb.c
index 34b70d0074..17a9135cbd 100644
--- a/odb.c
+++ b/odb.c
@@ -1038,16 +1038,7 @@ void odb_clear(struct object_database *o)
 
 	INIT_LIST_HEAD(&o->packed_git_mru);
 	close_object_store(o);
-
-	/*
-	 * `close_object_store()` only closes the packfiles, but doesn't free
-	 * them. We thus have to do this manually.
-	 */
-	for (struct packed_git *p = o->packed_git, *next; p; p = next) {
-		next = p->next;
-		free(p);
-	}
-	o->packed_git = NULL;
+	packfile_store_free(o->packfiles);
 
 	hashmap_clear(&o->pack_map);
 	string_list_clear(&o->submodule_source_paths, 0);
diff --git a/odb.h b/odb.h
index 08c3a01f3b..6f901c5ac0 100644
--- a/odb.h
+++ b/odb.h
@@ -130,7 +130,6 @@ struct object_database {
 	 * should only be accessed directly by packfile.c
 	 */
 	struct packfile_store *packfiles;
-	struct packed_git *packed_git;
 	/* A most-recently-used ordered version of the packed_git list. */
 	struct list_head packed_git_mru;
 
diff --git a/packfile.c b/packfile.c
index 8fbf1cfc2d..6478e4cc30 100644
--- a/packfile.c
+++ b/packfile.c
@@ -278,7 +278,7 @@ static int unuse_one_window(struct packed_git *current)
 
 	if (current)
 		scan_windows(current, &lru_p, &lru_w, &lru_l);
-	for (p = current->repo->objects->packed_git; p; p = p->next)
+	for (p = current->repo->objects->packfiles->packs; p; p = p->next)
 		scan_windows(p, &lru_p, &lru_w, &lru_l);
 	if (lru_p) {
 		munmap(lru_w->base, lru_w->len);
@@ -362,13 +362,8 @@ void close_pack(struct packed_git *p)
 void close_object_store(struct object_database *o)
 {
 	struct odb_source *source;
-	struct packed_git *p;
 
-	for (p = o->packed_git; p; p = p->next)
-		if (p->do_not_close)
-			BUG("want to close pack marked 'do-not-close'");
-		else
-			close_pack(p);
+	packfile_store_close(o->packfiles);
 
 	for (source = o->sources; source; source = source->next) {
 		if (source->midx)
@@ -468,7 +463,7 @@ static int close_one_pack(struct repository *r)
 	struct pack_window *mru_w = NULL;
 	int accept_windows_inuse = 1;
 
-	for (p = r->objects->packed_git; p; p = p->next) {
+	for (p = r->objects->packfiles->packs; p; p = p->next) {
 		if (p->pack_fd == -1)
 			continue;
 		find_lru_pack(p, &lru_p, &mru_w, &accept_windows_inuse);
@@ -789,8 +784,8 @@ void install_packed_git(struct repository *r, struct packed_git *pack)
 	if (pack->pack_fd != -1)
 		pack_open_fds++;
 
-	pack->next = r->objects->packed_git;
-	r->objects->packed_git = pack;
+	pack->next = r->objects->packfiles->packs;
+	r->objects->packfiles->packs = pack;
 
 	hashmap_entry_init(&pack->packmap_ent, strhash(pack->pack_name));
 	hashmap_add(&r->objects->pack_map, &pack->packmap_ent);
@@ -974,7 +969,7 @@ unsigned long repo_approximate_object_count(struct repository *r)
 				count += m->num_objects;
 		}
 
-		for (p = r->objects->packed_git; p; p = p->next) {
+		for (p = r->objects->packfiles->packs; p; p = p->next) {
 			if (open_pack_index(p))
 				continue;
 			count += p->num_objects;
@@ -1015,7 +1010,7 @@ static int sort_pack(const struct packed_git *a, const struct packed_git *b)
 
 static void rearrange_packed_git(struct repository *r)
 {
-	sort_packs(&r->objects->packed_git, sort_pack);
+	sort_packs(&r->objects->packfiles->packs, sort_pack);
 }
 
 static void prepare_packed_git_mru(struct repository *r)
@@ -1024,7 +1019,7 @@ static void prepare_packed_git_mru(struct repository *r)
 
 	INIT_LIST_HEAD(&r->objects->packed_git_mru);
 
-	for (p = r->objects->packed_git; p; p = p->next)
+	for (p = r->objects->packfiles->packs; p; p = p->next)
 		list_add_tail(&p->mru, &r->objects->packed_git_mru);
 }
 
@@ -1074,7 +1069,7 @@ void reprepare_packed_git(struct repository *r)
 struct packed_git *get_packed_git(struct repository *r)
 {
 	prepare_packed_git(r);
-	return r->objects->packed_git;
+	return r->objects->packfiles->packs;
 }
 
 struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
@@ -1095,7 +1090,7 @@ struct packed_git *get_all_packs(struct repository *r)
 			prepare_midx_pack(r, m, i);
 	}
 
-	return r->objects->packed_git;
+	return r->objects->packfiles->packs;
 }
 
 struct list_head *get_packed_git_mru(struct repository *r)
@@ -1220,7 +1215,7 @@ const struct packed_git *has_packed_and_bad(struct repository *r,
 {
 	struct packed_git *p;
 
-	for (p = r->objects->packed_git; p; p = p->next)
+	for (p = r->objects->packfiles->packs; p; p = p->next)
 		if (oidset_contains(&p->bad_objects, oid))
 			return p;
 	return NULL;
@@ -2081,7 +2076,7 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 		if (source->midx && fill_midx_entry(r, oid, e, source->midx))
 			return 1;
 
-	if (!r->objects->packed_git)
+	if (!r->objects->packfiles->packs)
 		return 0;
 
 	list_for_each(pos, &r->objects->packed_git_mru) {
@@ -2344,5 +2339,23 @@ struct packfile_store *packfile_store_new(struct object_database *odb)
 
 void packfile_store_free(struct packfile_store *store)
 {
+	packfile_store_close(store);
+
+	for (struct packed_git *p = store->packs, *next; p; p = next) {
+		next = p->next;
+		free(p);
+	}
+
 	free(store);
 }
+
+void packfile_store_close(struct packfile_store *store)
+{
+	struct packed_git *p;
+
+	for (p = store->packs; p; p = p->next)
+		if (p->do_not_close)
+			BUG("want to close pack marked 'do-not-close'");
+		else
+			close_pack(p);
+}
diff --git a/packfile.h b/packfile.h
index 8d31fd619a..d7ac8d24b4 100644
--- a/packfile.h
+++ b/packfile.h
@@ -57,6 +57,12 @@ struct packed_git {
  */
 struct packfile_store {
 	struct object_database *odb;
+
+	/*
+	 * The list of packfiles in the order in which they are being added to
+	 * the store.
+	 */
+	struct packed_git *packs;
 };
 
 /*
@@ -66,10 +72,17 @@ struct packfile_store {
 struct packfile_store *packfile_store_new(struct object_database *odb);
 
 /*
- * Free the packfile store and all its associated state.
+ * Free the packfile store and all its associated state. All packfiles
+ * tracked by the store will be closed.
  */
 void packfile_store_free(struct packfile_store *store);
 
+/*
+ * Close all packfiles associated with this store. The packfiles won't be
+ * free'd, so they can be re-opened at a later point in time.
+ */
+void packfile_store_close(struct packfile_store *store);
+
 static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
 				     const struct hashmap_entry *entry,
 				     const struct hashmap_entry *entry2,
-- 
2.51.0.261.g7ce5a0a67e.dirty
Taylor Blau· Aug 25, 2025, 23:42 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v2 02/16] odb: move list of packfiles into `struct packfile_store`

On Thu, Aug 21, 2025 at 09:39:00AM +0200, Patrick Steinhardt wrote:
Show 7 quoted lines
> The object database tracks the list of packfiles it currently knows
> about. With the introduction of the `struct packfile_store` we have a
> better place to host this list though.
>
> Move the list accordingly. Extract the logic from `odb_clear()` that
> knows to close all such packfiles and move it into the new subsystem, as
> well.

Not a comment on this patch itself, but as a meta-comment, I really appreciate you taking such an incremental approach here. The packfile machinery is quite fragile in my experience, so breaking it up into (what are so far) easily review-able chunks makes it much easier to build confidence in the correctness of these changes.

Show 27 quoted lines
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  odb.c      | 11 +----------
>  odb.h      |  1 -
>  packfile.c | 47 ++++++++++++++++++++++++++++++-----------------
>  packfile.h | 15 ++++++++++++++-
>  4 files changed, 45 insertions(+), 29 deletions(-)
>
> diff --git a/odb.c b/odb.c
> index 34b70d0074..17a9135cbd 100644
> --- a/odb.c
> +++ b/odb.c
> @@ -1038,16 +1038,7 @@ void odb_clear(struct object_database *o)
>
>  	INIT_LIST_HEAD(&o->packed_git_mru);
>  	close_object_store(o);
> -
> -	/*
> -	 * `close_object_store()` only closes the packfiles, but doesn't free
> -	 * them. We thus have to do this manually.
> -	 */
> -	for (struct packed_git *p = o->packed_git, *next; p; p = next) {
> -		next = p->next;
> -		free(p);
> -	}
> -	o->packed_git = NULL;
> +	packfile_store_free(o->packfiles);

Interesting. The movement of the for-loop here all looks correct to me. But I think the new packfile_store is creating a new awkardness here that we should consider.

In existing implementation, all of the ->next pointers here point to heap locations that have already been free()'d. But that's OK, since they aren't reachable at the moment that we do "o-packed_store = NULL".

Having a separate packfile_store changes that, since (from my reading of the code) o->packfiles will still be non-NULL even after calling odb_clear(), *and* those pointers will refer to free'd heap locations.

That seems like a potential footgun to me. I think that we could either:
 * Change packfile_store_free() to take in an object_database pointer,
   and NULL out the ->packs pointer after free'ing all of the packfiles.
   That would make it more similar to the existing behavior.
 * Leave packfile_store_free() as-is, document that it does NOT clear
   out the top-level pointer, and so callers are encouraged to NULL it
   out themselves after calling it. Likewise, we should change
   odb_clear() to do:
       packfile_store_free(o->packfiles);
       o->packfiles = NULL;
Let me know what you think.
Show 11 quoted lines
>  	hashmap_clear(&o->pack_map);
>  	string_list_clear(&o->submodule_source_paths, 0);
> diff --git a/odb.h b/odb.h
> index 08c3a01f3b..6f901c5ac0 100644
> --- a/odb.h
> +++ b/odb.h
> @@ -130,7 +130,6 @@ struct object_database {
>  	 * should only be accessed directly by packfile.c
>  	 */
>  	struct packfile_store *packfiles;
> -	struct packed_git *packed_git;
Makes sense.
Show 13 quoted lines
>  	/* A most-recently-used ordered version of the packed_git list. */
>  	struct list_head packed_git_mru;
>
> diff --git a/packfile.c b/packfile.c
> index 8fbf1cfc2d..6478e4cc30 100644
> --- a/packfile.c
> +++ b/packfile.c
> @@ -278,7 +278,7 @@ static int unuse_one_window(struct packed_git *current)
>
>  	if (current)
>  		scan_windows(current, &lru_p, &lru_w, &lru_l);
> -	for (p = current->repo->objects->packed_git; p; p = p->next)
> +	for (p = current->repo->objects->packfiles->packs; p; p = p->next)

Not a huge deal, but I do find "current->repo->objects->packfiles->packs" to be a bit unfortunate. I wonder if we should rename "packs" to "head" or "list_head" or similar since it's clear from "current->repo->objects->packfiles" that this is a list of packfiles.

Show 15 quoted lines
>  		scan_windows(p, &lru_p, &lru_w, &lru_l);
>  	if (lru_p) {
>  		munmap(lru_w->base, lru_w->len);
> @@ -362,13 +362,8 @@ void close_pack(struct packed_git *p)
>  void close_object_store(struct object_database *o)
>  {
>  	struct odb_source *source;
> -	struct packed_git *p;
>
> -	for (p = o->packed_git; p; p = p->next)
> -		if (p->do_not_close)
> -			BUG("want to close pack marked 'do-not-close'");
> -		else
> -			close_pack(p);
> +	packfile_store_close(o->packfiles);
Looks good.
Show 6 quoted lines
> @@ -468,7 +463,7 @@ static int close_one_pack(struct repository *r)
>  	struct pack_window *mru_w = NULL;
>  	int accept_windows_inuse = 1;
>
> -	for (p = r->objects->packed_git; p; p = p->next) {
> +	for (p = r->objects->packfiles->packs; p; p = p->next) {
Likewise.
Show 5 quoted lines
> @@ -2344,5 +2339,23 @@ struct packfile_store *packfile_store_new(struct object_database *odb)
>
>  void packfile_store_free(struct packfile_store *store)
>  {
> +	packfile_store_close(store);

Seeing a call to packfile_store_close() here was a little surprising to me. The code that you are moving has a comment that says:

   * `close_object_store()` only closes the packfiles, but doesn't free
   * them. We thus have to do this manually.

, so I would have expected to preserve that behavior. I *think* that this happens to be OK, since close_pack() is a noop if it is called more than once (though I had to double check through all of its leaf functions that that was indeed the case).

I would probably strike this from the new function, since the sole caller above already calls close_object_store() before calling packfile_store_free().

Show 8 quoted lines
> +
> +	for (struct packed_git *p = store->packs, *next; p; p = next) {
> +		next = p->next;
> +		free(p);
> +	}
> +
>  	free(store);
>  }
This part looks good.
Show 10 quoted lines
> +void packfile_store_close(struct packfile_store *store)
> +{
> +	struct packed_git *p;
> +
> +	for (p = store->packs; p; p = p->next)
> +		if (p->do_not_close)
> +			BUG("want to close pack marked 'do-not-close'");
> +		else
> +			close_pack(p);
> +}

And likewise this looks good to me. I do find the braceless for-loop a little hard to read, but it's (a) correct, and (b) consistent with the original implementation, so I don't feel strongly about changing it.

As a side-note, you could inline the declaration of "p" here into the for-loop, but I can understand not wanting to to make the diff more readable with --color-moved.

> diff --git a/packfile.h b/packfile.h
> index 8d31fd619a..d7ac8d24b4 100644
> --- a/packfile.h
> +++ b/packfile.h
The rest looks good to me.

Thanks, Taylor

Patrick Steinhardt· Sep 2, 2025, 08:50 UTC · re: Taylor Blau · lore

Re: [PATCH v2 02/16] odb: move list of packfiles into `struct packfile_store`

On Mon, Aug 25, 2025 at 07:42:53PM -0400, Taylor Blau wrote:
Show 14 quoted lines
> On Thu, Aug 21, 2025 at 09:39:00AM +0200, Patrick Steinhardt wrote:
> > The object database tracks the list of packfiles it currently knows
> > about. With the introduction of the `struct packfile_store` we have a
> > better place to host this list though.
> >
> > Move the list accordingly. Extract the logic from `odb_clear()` that
> > knows to close all such packfiles and move it into the new subsystem, as
> > well.
> 
> Not a comment on this patch itself, but as a meta-comment, I really
> appreciate you taking such an incremental approach here. The packfile
> machinery is quite fragile in my experience, so breaking it up into (what
> are so far) easily review-able chunks makes it much easier to build
> confidence in the correctness of these changes.

It certainly is fragile overall. I stared at code for way longer than I really want to admit in some cases.

Show 47 quoted lines
> > diff --git a/odb.c b/odb.c
> > index 34b70d0074..17a9135cbd 100644
> > --- a/odb.c
> > +++ b/odb.c
> > @@ -1038,16 +1038,7 @@ void odb_clear(struct object_database *o)
> >
> >  	INIT_LIST_HEAD(&o->packed_git_mru);
> >  	close_object_store(o);
> > -
> > -	/*
> > -	 * `close_object_store()` only closes the packfiles, but doesn't free
> > -	 * them. We thus have to do this manually.
> > -	 */
> > -	for (struct packed_git *p = o->packed_git, *next; p; p = next) {
> > -		next = p->next;
> > -		free(p);
> > -	}
> > -	o->packed_git = NULL;
> > +	packfile_store_free(o->packfiles);
> 
> Interesting. The movement of the for-loop here all looks correct to me.
> But I think the new packfile_store is creating a new awkardness here
> that we should consider.
> 
> In existing implementation, all of the ->next pointers here point to
> heap locations that have already been free()'d. But that's OK, since
> they aren't reachable at the moment that we do "o-packed_store = NULL".
> 
> Having a separate packfile_store changes that, since (from my reading of
> the code) o->packfiles will still be non-NULL even after calling
> odb_clear(), *and* those pointers will refer to free'd heap locations.
> 
> That seems like a potential footgun to me. I think that we could either:
> 
>  * Change packfile_store_free() to take in an object_database pointer,
>    and NULL out the ->packs pointer after free'ing all of the packfiles.
>    That would make it more similar to the existing behavior.
> 
>  * Leave packfile_store_free() as-is, document that it does NOT clear
>    out the top-level pointer, and so callers are encouraged to NULL it
>    out themselves after calling it. Likewise, we should change
>    odb_clear() to do:
> 
>        packfile_store_free(o->packfiles);
>        o->packfiles = NULL;
> 
> Let me know what you think.

Good point. I think it's unlikely to ever become a problem, but I don't see a reason why we shouldn't NULL out `o->packfiles`, either. So I'll do the second approach.

Show 15 quoted lines
> > diff --git a/packfile.c b/packfile.c
> > index 8fbf1cfc2d..6478e4cc30 100644
> > --- a/packfile.c
> > +++ b/packfile.c
> > @@ -278,7 +278,7 @@ static int unuse_one_window(struct packed_git *current)
> >
> >  	if (current)
> >  		scan_windows(current, &lru_p, &lru_w, &lru_l);
> > -	for (p = current->repo->objects->packed_git; p; p = p->next)
> > +	for (p = current->repo->objects->packfiles->packs; p; p = p->next)
> 
> Not a huge deal, but I do find "current->repo->objects->packfiles->packs"
> to be a bit unfortunate. I wonder if we should rename "packs" to "head"
> or "list_head" or similar since it's clear from
> "current->repo->objects->packfiles" that this is a list of packfiles.

I'd like to keep this part as-is for now if you don't mind. This is mostly because I've got a follow-up patch series that _does_ introduce `head` as part of making the `->next` pointer go away.

Show 13 quoted lines
> > @@ -2344,5 +2339,23 @@ struct packfile_store *packfile_store_new(struct object_database *odb)
> >
> >  void packfile_store_free(struct packfile_store *store)
> >  {
> > +	packfile_store_close(store);
> 
> Seeing a call to packfile_store_close() here was a little surprising to
> me. The code that you are moving has a comment that says:
> 
>    * `close_object_store()` only closes the packfiles, but doesn't free
>    * them. We thus have to do this manually.
> 
> , so I would have expected to preserve that behavior.

This behaviour is preserved though. Calling `packfile_store_close()` does not free the packfiles, it only closes them. And we continue to call `packfile_store_close()` in `close_object_store()`, so nothing changes.

The only change in behaviour is that we now also know to close packfiles when freeing the packfile store.

Show 8 quoted lines
> I *think* that
> this happens to be OK, since close_pack() is a noop if it is called more
> than once (though I had to double check through all of its leaf
> functions that that was indeed the case).
> 
> I would probably strike this from the new function, since the sole
> caller above already calls close_object_store() before calling
> packfile_store_free().

Calling `packfile_store_close()` is idempotent indeed, so it shouldn't be an issue to call this function twice. To me the question is whether there's ever a use case where you want to free the packfile store, but don't want to close the packfiles stored in it.

From all I've seen that is never the case, so I think it's sensible to ensure that we always close the packfile store before we free it to make things a tiny bit more self-contained.

Show 14 quoted lines
> > +void packfile_store_close(struct packfile_store *store)
> > +{
> > +	struct packed_git *p;
> > +
> > +	for (p = store->packs; p; p = p->next)
> > +		if (p->do_not_close)
> > +			BUG("want to close pack marked 'do-not-close'");
> > +		else
> > +			close_pack(p);
> > +}
> 
> And likewise this looks good to me. I do find the braceless for-loop a
> little hard to read, but it's (a) correct, and (b) consistent with the
> original implementation, so I don't feel strongly about changing it.

Agreed, it is a bit awkward. I feel like our coding style should be amended to say that we only do braceless bodies in case the body is a single statement.

> As a side-note, you could inline the declaration of "p" here into the
> for-loop, but I can understand not wanting to to make the diff more
> readable with --color-moved.
I wouldn't mind adapting this while at it, too.
Patrick
Taylor Blau· Sep 2, 2025, 17:21 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v2 02/16] odb: move list of packfiles into `struct packfile_store`

On Tue, Sep 02, 2025 at 10:50:41AM +0200, Patrick Steinhardt wrote:
Show 18 quoted lines
> > > +void packfile_store_close(struct packfile_store *store)
> > > +{
> > > +	struct packed_git *p;
> > > +
> > > +	for (p = store->packs; p; p = p->next)
> > > +		if (p->do_not_close)
> > > +			BUG("want to close pack marked 'do-not-close'");
> > > +		else
> > > +			close_pack(p);
> > > +}
> >
> > And likewise this looks good to me. I do find the braceless for-loop a
> > little hard to read, but it's (a) correct, and (b) consistent with the
> > original implementation, so I don't feel strongly about changing it.
>
> Agreed, it is a bit awkward. I feel like our coding style should be
> amended to say that we only do braceless bodies in case the body is a
> single statement.

I think that our CodingGuidelines cover this as of 1797dc5176 (CodingGuidelines: clarify multi-line brace style, 2017-01-17), which frowns upon statements that extend multiple lines.

So I think in this case, the CodingGuidelines would suggest that we write this as:

    for (p = store->packs; p; p = p->next) {
        if (p->do_not_close)
            BUG("want to close pack marked 'do-not-close'");
        else
            close_pack(p);
    }

, which from our discussion here seems like something that we both find more readable than the original.

Thanks, Taylor

Junio C Hamano· Sep 2, 2025, 17:42 UTC · re: Taylor Blau · lore

Re: [PATCH v2 02/16] odb: move list of packfiles into `struct packfile_store`

Taylor Blau <me@ttaylorr.com> writes:
Show 12 quoted lines
> So I think in this case, the CodingGuidelines would suggest that we
> write this as:
>
>     for (p = store->packs; p; p = p->next) {
>         if (p->do_not_close)
>             BUG("want to close pack marked 'do-not-close'");
>         else
>             close_pack(p);
>     }
>
> , which from our discussion here seems like something that we both find
> more readable than the original.

Yes. Technically the "if...else..." is still a single statement, so a rule like "do not use {} only if you would place a single statement in it", though.

I would actually write it more like this, though.
     for (p = store->packs; p; p = p->next) {
         if (p->do_not_close)
             BUG("want to close pack marked 'do-not-close'");
         close_pack(p);
     }

The first two lines in that block is a glorified assert(), and without a programming bug, what the loop wants to do is only to call close_pack() on eacn and every pack on the list. Not using "else" conveys that much clearer.

Patrick Steinhardt· Sep 3, 2025, 05:58 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 02/16] odb: move list of packfiles into `struct packfile_store`

On Tue, Sep 02, 2025 at 10:42:25AM -0700, Junio C Hamano wrote:
Show 32 quoted lines
> Taylor Blau <me@ttaylorr.com> writes:
> 
> > So I think in this case, the CodingGuidelines would suggest that we
> > write this as:
> >
> >     for (p = store->packs; p; p = p->next) {
> >         if (p->do_not_close)
> >             BUG("want to close pack marked 'do-not-close'");
> >         else
> >             close_pack(p);
> >     }
> >
> > , which from our discussion here seems like something that we both find
> > more readable than the original.
> 
> Yes.  Technically the "if...else..." is still a single statement, so
> a rule like "do not use {} only if you would place a single
> statement in it", though.
> 
> I would actually write it more like this, though.
> 
>      for (p = store->packs; p; p = p->next) {
>          if (p->do_not_close)
>              BUG("want to close pack marked 'do-not-close'");
> 
>          close_pack(p);
>      }
> 
> The first two lines in that block is a glorified assert(), and
> without a programming bug, what the loop wants to do is only to call
> close_pack() on eacn and every pack on the list.  Not using "else"
> conveys that much clearer.

That reads even better, agreed. Will queue this change locally and send it out with the next revision of this patch series.

Patrick
Taylor Blau· Sep 11, 2025, 23:16 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v2 02/16] odb: move list of packfiles into `struct packfile_store`

On Tue, Sep 02, 2025 at 10:50:41AM +0200, Patrick Steinhardt wrote:
Show 19 quoted lines
> > > diff --git a/packfile.c b/packfile.c
> > > index 8fbf1cfc2d..6478e4cc30 100644
> > > --- a/packfile.c
> > > +++ b/packfile.c
> > > @@ -278,7 +278,7 @@ static int unuse_one_window(struct packed_git *current)
> > >
> > >  	if (current)
> > >  		scan_windows(current, &lru_p, &lru_w, &lru_l);
> > > -	for (p = current->repo->objects->packed_git; p; p = p->next)
> > > +	for (p = current->repo->objects->packfiles->packs; p; p = p->next)
> >
> > Not a huge deal, but I do find "current->repo->objects->packfiles->packs"
> > to be a bit unfortunate. I wonder if we should rename "packs" to "head"
> > or "list_head" or similar since it's clear from
> > "current->repo->objects->packfiles" that this is a list of packfiles.
>
> I'd like to keep this part as-is for now if you don't mind. This is
> mostly because I've got a follow-up patch series that _does_ introduce
> `head` as part of making the `->next` pointer go away.

I'm OK with that approach provided that you have a plan to introduce "head" here ;-).

Show 21 quoted lines
> > > @@ -2344,5 +2339,23 @@ struct packfile_store *packfile_store_new(struct object_database *odb)
> > >
> > >  void packfile_store_free(struct packfile_store *store)
> > >  {
> > > +	packfile_store_close(store);
> >
> > Seeing a call to packfile_store_close() here was a little surprising to
> > me. The code that you are moving has a comment that says:
> >
> >    * `close_object_store()` only closes the packfiles, but doesn't free
> >    * them. We thus have to do this manually.
> >
> > , so I would have expected to preserve that behavior.
>
> This behaviour is preserved though. Calling `packfile_store_close()`
> does not free the packfiles, it only closes them. And we continue to
> call `packfile_store_close()` in `close_object_store()`, so nothing
> changes.
>
> The only change in behaviour is that we now also know to close packfiles
> when freeing the packfile store.

Why is that change necessary? I am not trying to be overly pedantic here, but as you note above this code is extremely fragile, so I am trying to avoid any changes that are not strictly necessary.

Thanks, Taylor

Patrick Steinhardt· Sep 15, 2025, 07:44 UTC · re: Taylor Blau · lore

Re: [PATCH v2 02/16] odb: move list of packfiles into `struct packfile_store`

On Thu, Sep 11, 2025 at 07:16:56PM -0400, Taylor Blau wrote:
Show 30 quoted lines
> On Tue, Sep 02, 2025 at 10:50:41AM +0200, Patrick Steinhardt wrote:
> > > > diff --git a/packfile.c b/packfile.c
> > > > index 8fbf1cfc2d..6478e4cc30 100644
> > > > --- a/packfile.c
> > > > +++ b/packfile.c
> > > > @@ -2344,5 +2339,23 @@ struct packfile_store *packfile_store_new(struct object_database *odb)
> > > >
> > > >  void packfile_store_free(struct packfile_store *store)
> > > >  {
> > > > +	packfile_store_close(store);
> > >
> > > Seeing a call to packfile_store_close() here was a little surprising to
> > > me. The code that you are moving has a comment that says:
> > >
> > >    * `close_object_store()` only closes the packfiles, but doesn't free
> > >    * them. We thus have to do this manually.
> > >
> > > , so I would have expected to preserve that behavior.
> >
> > This behaviour is preserved though. Calling `packfile_store_close()`
> > does not free the packfiles, it only closes them. And we continue to
> > call `packfile_store_close()` in `close_object_store()`, so nothing
> > changes.
> >
> > The only change in behaviour is that we now also know to close packfiles
> > when freeing the packfile store.
> 
> Why is that change necessary? I am not trying to be overly pedantic
> here, but as you note above this code is extremely fragile, so I am
> trying to avoid any changes that are not strictly necessary.

Hm, okay. I'll drop that part then. We can revisit this at a later point in time once we move the logic to close the object database into the respective sources.

Patrick
Patrick Steinhardt· Aug 21, 2025, 07:39 UTC · re: Patrick Steinhardt · lore

[PATCH v2 03/16] odb: move initialization bit into `struct packfile_store`

The object database knows to skip re-initializing the list of packfiles in case it's already been initialized. Whether or not that is the case is tracked via a separate `initialized` bit that is stored in the object database. With the introduction of the `struct packfile_store` we have a better place to host this bit though.

Move it accordingly. While at it, convert the field into a boolean now that we're allowed to use them in our code base.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb.h      | 6 ------
 packfile.c | 6 +++---
 packfile.h | 6 ++++++
 3 files changed, 9 insertions(+), 9 deletions(-)
Show changes to 3 files +9 −9

odb.h, packfile.c, packfile.h

diff --git a/odb.h b/odb.h
index 6f901c5ac0..98e038fa73 100644
--- a/odb.h
+++ b/odb.h
@@ -161,12 +161,6 @@ struct object_database {
 	unsigned long approximate_object_count;
 	unsigned approximate_object_count_valid : 1;
 
-	/*
-	 * Whether packed_git has already been populated with this repository's
-	 * packs.
-	 */
-	unsigned packed_git_initialized : 1;
-
 	/*
 	 * Submodule source paths that will be added as additional sources to
 	 * allow lookup of submodule objects via the main object database.
diff --git a/packfile.c b/packfile.c
index 6478e4cc30..17f770e0e0 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1027,7 +1027,7 @@ static void prepare_packed_git(struct repository *r)
 {
 	struct odb_source *source;
 
-	if (r->objects->packed_git_initialized)
+	if (r->objects->packfiles->initialized)
 		return;
 
 	odb_prepare_alternates(r->objects);
@@ -1039,7 +1039,7 @@ static void prepare_packed_git(struct repository *r)
 	rearrange_packed_git(r);
 
 	prepare_packed_git_mru(r);
-	r->objects->packed_git_initialized = 1;
+	r->objects->packfiles->initialized = true;
 }
 
 void reprepare_packed_git(struct repository *r)
@@ -1061,7 +1061,7 @@ void reprepare_packed_git(struct repository *r)
 		odb_clear_loose_cache(source);
 
 	r->objects->approximate_object_count_valid = 0;
-	r->objects->packed_git_initialized = 0;
+	r->objects->packfiles->initialized = false;
 	prepare_packed_git(r);
 	obj_read_unlock();
 }
diff --git a/packfile.h b/packfile.h
index d7ac8d24b4..cf81091175 100644
--- a/packfile.h
+++ b/packfile.h
@@ -63,6 +63,12 @@ struct packfile_store {
 	 * the store.
 	 */
 	struct packed_git *packs;
+
+	/*
+	 * Whether packfiles have already been populated with this store's
+	 * packs.
+	 */
+	bool initialized;
 };
 
 /*
-- 
2.51.0.261.g7ce5a0a67e.dirty
Taylor Blau· Aug 26, 2025, 01:40 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v2 03/16] odb: move initialization bit into `struct packfile_store`

On Thu, Aug 21, 2025 at 09:39:01AM +0200, Patrick Steinhardt wrote:
Show 14 quoted lines
> diff --git a/packfile.h b/packfile.h
> index d7ac8d24b4..cf81091175 100644
> --- a/packfile.h
> +++ b/packfile.h
> @@ -63,6 +63,12 @@ struct packfile_store {
>  	 * the store.
>  	 */
>  	struct packed_git *packs;
> +
> +	/*
> +	 * Whether packfiles have already been populated with this store's
> +	 * packs.
> +	 */
> +	bool initialized;
Exciting ;-).

Thanks, Taylor

Patrick Steinhardt· Aug 21, 2025, 07:39 UTC · re: Patrick Steinhardt · lore

[PATCH v2 04/16] odb: move packfile map into `struct packfile_store`

The object database tracks a map of packfiles by their respective paths, which is used to figure out whether a given packfile has already been loaded.With the introduction of the `struct packfile_store` we have a better place to host this list though.

Move the map accordingly. `pack_map_entry_cmp()` isn't used anywhere but in "packfile.c" anymore after this change, so we convert it to a static function, as well.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 midx.c     |  2 +-
 odb.c      |  2 --
 odb.h      |  6 ------
 packfile.c | 20 ++++++++++++++++++--
 packfile.h | 20 ++++++--------------
 5 files changed, 25 insertions(+), 25 deletions(-)
Show changes to 5 files +25 −25

midx.c, odb.c, odb.h, packfile.c, packfile.h

diff --git a/midx.c b/midx.c
index 7d407682e6..7f3f74ef2b 100644
--- a/midx.c
+++ b/midx.c
@@ -471,7 +471,7 @@ int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,
 	strbuf_addbuf(&key, &pack_name);
 	strbuf_strip_suffix(&key, ".idx");
 	strbuf_addstr(&key, ".pack");
-	p = hashmap_get_entry_from_hash(&r->objects->pack_map,
+	p = hashmap_get_entry_from_hash(&r->objects->packfiles->map,
 					strhash(key.buf), key.buf,
 					struct packed_git, packmap_ent);
 	if (!p) {
diff --git a/odb.c b/odb.c
index 17a9135cbd..568c820ef8 100644
--- a/odb.c
+++ b/odb.c
@@ -998,7 +998,6 @@ struct object_database *odb_new(struct repository *repo)
 	o->repo = repo;
 	o->packfiles = packfile_store_new(o);
 	INIT_LIST_HEAD(&o->packed_git_mru);
-	hashmap_init(&o->pack_map, pack_map_entry_cmp, NULL, 0);
 	pthread_mutex_init(&o->replace_mutex, NULL);
 	string_list_init_dup(&o->submodule_source_paths);
 	return o;
@@ -1040,6 +1039,5 @@ void odb_clear(struct object_database *o)
 	close_object_store(o);
 	packfile_store_free(o->packfiles);
 
-	hashmap_clear(&o->pack_map);
 	string_list_clear(&o->submodule_source_paths, 0);
 }
diff --git a/odb.h b/odb.h
index 98e038fa73..fb37c6ebce 100644
--- a/odb.h
+++ b/odb.h
@@ -147,12 +147,6 @@ struct object_database {
 	struct cached_object_entry *cached_objects;
 	size_t cached_object_nr, cached_object_alloc;
 
-	/*
-	 * A map of packfiles to packed_git structs for tracking which
-	 * packs have been loaded already.
-	 */
-	struct hashmap pack_map;
-
 	/*
 	 * A fast, rough count of the number of objects in the repository.
 	 * These two fields are not meant for direct access. Use
diff --git a/packfile.c b/packfile.c
index 17f770e0e0..752a0cee8d 100644
--- a/packfile.c
+++ b/packfile.c
@@ -788,7 +788,7 @@ void install_packed_git(struct repository *r, struct packed_git *pack)
 	r->objects->packfiles->packs = pack;
 
 	hashmap_entry_init(&pack->packmap_ent, strhash(pack->pack_name));
-	hashmap_add(&r->objects->pack_map, &pack->packmap_ent);
+	hashmap_add(&r->objects->packfiles->map, &pack->packmap_ent);
 }
 
 void (*report_garbage)(unsigned seen_bits, const char *path);
@@ -901,7 +901,7 @@ static void prepare_pack(const char *full_name, size_t full_name_len,
 		hashmap_entry_init(&hent, hash);
 
 		/* Don't reopen a pack we already have. */
-		if (!hashmap_get(&data->r->objects->pack_map, &hent, pack_name)) {
+		if (!hashmap_get(&data->r->objects->packfiles->map, &hent, pack_name)) {
 			p = add_packed_git(data->r, full_name, full_name_len, data->local);
 			if (p)
 				install_packed_git(data->r, p);
@@ -2329,11 +2329,26 @@ int parse_pack_header_option(const char *in, unsigned char *out, unsigned int *l
 	return 0;
 }
 
+static int pack_map_entry_cmp(const void *cmp_data UNUSED,
+			      const struct hashmap_entry *entry,
+			      const struct hashmap_entry *entry2,
+			      const void *keydata)
+{
+	const char *key = keydata;
+	const struct packed_git *pg1, *pg2;
+
+	pg1 = container_of(entry, const struct packed_git, packmap_ent);
+	pg2 = container_of(entry2, const struct packed_git, packmap_ent);
+
+	return strcmp(pg1->pack_name, key ? key : pg2->pack_name);
+}
+
 struct packfile_store *packfile_store_new(struct object_database *odb)
 {
 	struct packfile_store *store;
 	CALLOC_ARRAY(store, 1);
 	store->odb = odb;
+	hashmap_init(&store->map, pack_map_entry_cmp, NULL, 0);
 	return store;
 }
 
@@ -2346,6 +2361,7 @@ void packfile_store_free(struct packfile_store *store)
 		free(p);
 	}
 
+	hashmap_clear(&store->map);
 	free(store);
 }
 
diff --git a/packfile.h b/packfile.h
index cf81091175..9bbef51164 100644
--- a/packfile.h
+++ b/packfile.h
@@ -64,6 +64,12 @@ struct packfile_store {
 	 */
 	struct packed_git *packs;
 
+	/*
+	 * A map of packfile names to packed_git structs for tracking which
+	 * packs have been loaded already.
+	 */
+	struct hashmap map;
+
 	/*
 	 * Whether packfiles have already been populated with this store's
 	 * packs.
@@ -89,20 +95,6 @@ void packfile_store_free(struct packfile_store *store);
  */
 void packfile_store_close(struct packfile_store *store);
 
-static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
-				     const struct hashmap_entry *entry,
-				     const struct hashmap_entry *entry2,
-				     const void *keydata)
-{
-	const char *key = keydata;
-	const struct packed_git *pg1, *pg2;
-
-	pg1 = container_of(entry, const struct packed_git, packmap_ent);
-	pg2 = container_of(entry2, const struct packed_git, packmap_ent);
-
-	return strcmp(pg1->pack_name, key ? key : pg2->pack_name);
-}
-
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
-- 
2.51.0.261.g7ce5a0a67e.dirty
Taylor Blau· Aug 26, 2025, 01:41 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v2 04/16] odb: move packfile map into `struct packfile_store`

On Thu, Aug 21, 2025 at 09:39:02AM +0200, Patrick Steinhardt wrote:
Show 8 quoted lines
> The object database tracks a map of packfiles by their respective paths,
> which is used to figure out whether a given packfile has already been
> loaded.With the introduction of the `struct packfile_store` we have a
> better place to host this list though.
>
> Move the map accordingly. `pack_map_entry_cmp()` isn't used anywhere but
> in "packfile.c" anymore after this change, so we convert it to a static
> function, as well.
This patch looks exactly as expected, let's keep reading...

Thanks, Taylor

Patrick Steinhardt· Aug 21, 2025, 07:39 UTC · re: Patrick Steinhardt · lore

[PATCH v2 05/16] odb: move MRU list of packfiles into `struct packfile_store`

The object database tracks the list of packfiles in most-recently-used order, which is mostly used to favor reading from packfiles that contain most of the objects that we're currently accessing. With the introduction of the `struct packfile_store` we have a better place to host this list though.

Move the list accordingly.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 midx.c     |  2 +-
 odb.c      |  2 --
 odb.h      |  4 ----
 packfile.c | 11 ++++++-----
 packfile.h |  3 +++
 5 files changed, 10 insertions(+), 12 deletions(-)
Show changes to 5 files +10 −12

midx.c, odb.c, odb.h, packfile.c, packfile.h

diff --git a/midx.c b/midx.c
index 7f3f74ef2b..7fa2b8473a 100644
--- a/midx.c
+++ b/midx.c
@@ -478,7 +478,7 @@ int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,
 		p = add_packed_git(r, pack_name.buf, pack_name.len, m->local);
 		if (p) {
 			install_packed_git(r, p);
-			list_add_tail(&p->mru, &r->objects->packed_git_mru);
+			list_add_tail(&p->mru, &r->objects->packfiles->mru);
 		}
 	}
 
diff --git a/odb.c b/odb.c
index 568c820ef8..80ec6fc1fa 100644
--- a/odb.c
+++ b/odb.c
@@ -997,7 +997,6 @@ struct object_database *odb_new(struct repository *repo)
 	memset(o, 0, sizeof(*o));
 	o->repo = repo;
 	o->packfiles = packfile_store_new(o);
-	INIT_LIST_HEAD(&o->packed_git_mru);
 	pthread_mutex_init(&o->replace_mutex, NULL);
 	string_list_init_dup(&o->submodule_source_paths);
 	return o;
@@ -1035,7 +1034,6 @@ void odb_clear(struct object_database *o)
 		free((char *) o->cached_objects[i].value.buf);
 	FREE_AND_NULL(o->cached_objects);
 
-	INIT_LIST_HEAD(&o->packed_git_mru);
 	close_object_store(o);
 	packfile_store_free(o->packfiles);
 
diff --git a/odb.h b/odb.h
index fb37c6ebce..1505e39729 100644
--- a/odb.h
+++ b/odb.h
@@ -3,7 +3,6 @@
 
 #include "hashmap.h"
 #include "object.h"
-#include "list.h"
 #include "oidset.h"
 #include "oidmap.h"
 #include "string-list.h"
@@ -130,9 +129,6 @@ struct object_database {
 	 * should only be accessed directly by packfile.c
 	 */
 	struct packfile_store *packfiles;
-	/* A most-recently-used ordered version of the packed_git list. */
-	struct list_head packed_git_mru;
-
 	struct {
 		struct packed_git **packs;
 		unsigned flags;
diff --git a/packfile.c b/packfile.c
index 752a0cee8d..91a7a4064f 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1017,10 +1017,10 @@ static void prepare_packed_git_mru(struct repository *r)
 {
 	struct packed_git *p;
 
-	INIT_LIST_HEAD(&r->objects->packed_git_mru);
+	INIT_LIST_HEAD(&r->objects->packfiles->mru);
 
 	for (p = r->objects->packfiles->packs; p; p = p->next)
-		list_add_tail(&p->mru, &r->objects->packed_git_mru);
+		list_add_tail(&p->mru, &r->objects->packfiles->mru);
 }
 
 static void prepare_packed_git(struct repository *r)
@@ -1096,7 +1096,7 @@ struct packed_git *get_all_packs(struct repository *r)
 struct list_head *get_packed_git_mru(struct repository *r)
 {
 	prepare_packed_git(r);
-	return &r->objects->packed_git_mru;
+	return &r->objects->packfiles->mru;
 }
 
 unsigned long unpack_object_header_buffer(const unsigned char *buf,
@@ -2079,10 +2079,10 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 	if (!r->objects->packfiles->packs)
 		return 0;
 
-	list_for_each(pos, &r->objects->packed_git_mru) {
+	list_for_each(pos, &r->objects->packfiles->mru) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		if (!p->multi_pack_index && fill_pack_entry(oid, e, p)) {
-			list_move(&p->mru, &r->objects->packed_git_mru);
+			list_move(&p->mru, &r->objects->packfiles->mru);
 			return 1;
 		}
 	}
@@ -2348,6 +2348,7 @@ struct packfile_store *packfile_store_new(struct object_database *odb)
 	struct packfile_store *store;
 	CALLOC_ARRAY(store, 1);
 	store->odb = odb;
+	INIT_LIST_HEAD(&store->mru);
 	hashmap_init(&store->map, pack_map_entry_cmp, NULL, 0);
 	return store;
 }
diff --git a/packfile.h b/packfile.h
index 9bbef51164..d48d46cc1b 100644
--- a/packfile.h
+++ b/packfile.h
@@ -64,6 +64,9 @@ struct packfile_store {
 	 */
 	struct packed_git *packs;
 
+	/* A most-recently-used ordered version of the packs list. */
+	struct list_head mru;
+
 	/*
 	 * A map of packfile names to packed_git structs for tracking which
 	 * packs have been loaded already.
-- 
2.51.0.261.g7ce5a0a67e.dirty
Patrick Steinhardt· Aug 21, 2025, 07:39 UTC · re: Patrick Steinhardt · lore

[PATCH v2 06/16] odb: move kept cache into `struct packfile_store`

The object database tracks a cache of "kept" packfiles, which is used by git-pack-objects(1) to handle cruft objects. With the introduction of the `struct packfile_store` we have a better place to host this cache though.

Move the cache accordingly.

This moves the last bit of packfile-related state from the object database into the packfile store. Adapt the comment for the `packfiles` pointer in `struct object_database` to reflect this.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb.h      |  8 +-------
 packfile.c | 16 ++++++++--------
 packfile.h |  5 +++++
 3 files changed, 14 insertions(+), 15 deletions(-)
Show changes to 3 files +14 −15

odb.h, packfile.c, packfile.h

diff --git a/odb.h b/odb.h
index 1505e39729..f1736b067c 100644
--- a/odb.h
+++ b/odb.h
@@ -124,15 +124,9 @@ struct object_database {
 	unsigned commit_graph_attempted : 1; /* if loading has been attempted */
 
 	/*
-	 * private data
-	 *
-	 * should only be accessed directly by packfile.c
+	 * Should only be accessed directly by packfile.c
 	 */
 	struct packfile_store *packfiles;
-	struct {
-		struct packed_git **packs;
-		unsigned flags;
-	} kept_pack_cache;
 
 	/*
 	 * This is meant to hold a *small* number of objects that you would
diff --git a/packfile.c b/packfile.c
index 91a7a4064f..07c574f359 100644
--- a/packfile.c
+++ b/packfile.c
@@ -2092,19 +2092,19 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 static void maybe_invalidate_kept_pack_cache(struct repository *r,
 					     unsigned flags)
 {
-	if (!r->objects->kept_pack_cache.packs)
+	if (!r->objects->packfiles->kept_cache.packs)
 		return;
-	if (r->objects->kept_pack_cache.flags == flags)
+	if (r->objects->packfiles->kept_cache.flags == flags)
 		return;
-	FREE_AND_NULL(r->objects->kept_pack_cache.packs);
-	r->objects->kept_pack_cache.flags = 0;
+	FREE_AND_NULL(r->objects->packfiles->kept_cache.packs);
+	r->objects->packfiles->kept_cache.flags = 0;
 }
 
 struct packed_git **kept_pack_cache(struct repository *r, unsigned flags)
 {
 	maybe_invalidate_kept_pack_cache(r, flags);
 
-	if (!r->objects->kept_pack_cache.packs) {
+	if (!r->objects->packfiles->kept_cache.packs) {
 		struct packed_git **packs = NULL;
 		size_t nr = 0, alloc = 0;
 		struct packed_git *p;
@@ -2127,11 +2127,11 @@ struct packed_git **kept_pack_cache(struct repository *r, unsigned flags)
 		ALLOC_GROW(packs, nr + 1, alloc);
 		packs[nr] = NULL;
 
-		r->objects->kept_pack_cache.packs = packs;
-		r->objects->kept_pack_cache.flags = flags;
+		r->objects->packfiles->kept_cache.packs = packs;
+		r->objects->packfiles->kept_cache.flags = flags;
 	}
 
-	return r->objects->kept_pack_cache.packs;
+	return r->objects->packfiles->kept_cache.packs;
 }
 
 int find_kept_pack_entry(struct repository *r,
diff --git a/packfile.h b/packfile.h
index d48d46cc1b..74cea1a4a9 100644
--- a/packfile.h
+++ b/packfile.h
@@ -64,6 +64,11 @@ struct packfile_store {
 	 */
 	struct packed_git *packs;
 
+	struct {
+		struct packed_git **packs;
+		unsigned flags;
+	} kept_cache;
+
 	/* A most-recently-used ordered version of the packs list. */
 	struct list_head mru;
 
-- 
2.51.0.261.g7ce5a0a67e.dirty
Taylor Blau· Aug 26, 2025, 01:46 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v2 06/16] odb: move kept cache into `struct packfile_store`

On Thu, Aug 21, 2025 at 09:39:04AM +0200, Patrick Steinhardt wrote:
Show 6 quoted lines
> The object database tracks a cache of "kept" packfiles, which is used by
> git-pack-objects(1) to handle cruft objects. With the introduction of
> the `struct packfile_store` we have a better place to host this cache
> though.
>
> Move the cache accordingly.

This all looks good to me, thanks for taking care to preserve the kept-pack cache's behavior.

> This moves the last bit of packfile-related state from the object
> database into the packfile store. Adapt the comment for the `packfiles`
> pointer in `struct object_database` to reflect this.
Thanks for keeping the comment up-to-date :-).
Show 13 quoted lines
> diff --git a/packfile.h b/packfile.h
> index d48d46cc1b..74cea1a4a9 100644
> --- a/packfile.h
> +++ b/packfile.h
> @@ -64,6 +64,11 @@ struct packfile_store {
>  	 */
>  	struct packed_git *packs;
>
> +	struct {
> +		struct packed_git **packs;
> +		unsigned flags;
> +	} kept_cache;
> +

This wouldn't be a bad time to add a comment here explaining what the kept_cache is for and what each of the struct's members represent. We can blame (at least one of) the author(s) of 20b031fede (packfile: add kept-pack cache for find_kept_pack_entry(), 2021-02-22) for omitting it in the first place ;-).

Thanks, Taylor

Patrick Steinhardt· Sep 2, 2025, 08:50 UTC · re: Taylor Blau · lore

Re: [PATCH v2 06/16] odb: move kept cache into `struct packfile_store`

On Mon, Aug 25, 2025 at 09:46:07PM -0400, Taylor Blau wrote:
Show 20 quoted lines
> On Thu, Aug 21, 2025 at 09:39:04AM +0200, Patrick Steinhardt wrote:
> > diff --git a/packfile.h b/packfile.h
> > index d48d46cc1b..74cea1a4a9 100644
> > --- a/packfile.h
> > +++ b/packfile.h
> > @@ -64,6 +64,11 @@ struct packfile_store {
> >  	 */
> >  	struct packed_git *packs;
> >
> > +	struct {
> > +		struct packed_git **packs;
> > +		unsigned flags;
> > +	} kept_cache;
> > +
> 
> This wouldn't be a bad time to add a comment here explaining what the
> kept_cache is for and what each of the struct's members represent. We
> can blame (at least one of) the author(s) of 20b031fede (packfile: add
> kept-pack cache for find_kept_pack_entry(), 2021-02-22) for omitting it
> in the first place ;-).
Good idea. I'll try to puzzle something together.
Patrick
Patrick Steinhardt· Aug 21, 2025, 07:39 UTC · re: Patrick Steinhardt · lore

[PATCH v2 07/16] packfile: reorder functions to avoid function declaration

Reorder functions so that we can avoid a forward declaration of `prepare_packed_git()`.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 packfile.c | 67 +++++++++++++++++++++++++++++++-------------------------------
 1 file changed, 33 insertions(+), 34 deletions(-)
Show changes to packfile.c +33 −34
diff --git a/packfile.c b/packfile.c
index 07c574f359..90f15b0c20 100644
--- a/packfile.c
+++ b/packfile.c
@@ -946,40 +946,6 @@ static void prepare_packed_git_one(struct odb_source *source, int local)
 	string_list_clear(data.garbage, 0);
 }
 
-static void prepare_packed_git(struct repository *r);
-/*
- * Give a fast, rough count of the number of objects in the repository. This
- * ignores loose objects completely. If you have a lot of them, then either
- * you should repack because your performance will be awful, or they are
- * all unreachable objects about to be pruned, in which case they're not really
- * interesting as a measure of repo size in the first place.
- */
-unsigned long repo_approximate_object_count(struct repository *r)
-{
-	if (!r->objects->approximate_object_count_valid) {
-		struct odb_source *source;
-		unsigned long count = 0;
-		struct packed_git *p;
-
-		prepare_packed_git(r);
-
-		for (source = r->objects->sources; source; source = source->next) {
-			struct multi_pack_index *m = get_multi_pack_index(source);
-			if (m)
-				count += m->num_objects;
-		}
-
-		for (p = r->objects->packfiles->packs; p; p = p->next) {
-			if (open_pack_index(p))
-				continue;
-			count += p->num_objects;
-		}
-		r->objects->approximate_object_count = count;
-		r->objects->approximate_object_count_valid = 1;
-	}
-	return r->objects->approximate_object_count;
-}
-
 DEFINE_LIST_SORT(static, sort_packs, struct packed_git, next);
 
 static int sort_pack(const struct packed_git *a, const struct packed_git *b)
@@ -1099,6 +1065,39 @@ struct list_head *get_packed_git_mru(struct repository *r)
 	return &r->objects->packfiles->mru;
 }
 
+/*
+ * Give a fast, rough count of the number of objects in the repository. This
+ * ignores loose objects completely. If you have a lot of them, then either
+ * you should repack because your performance will be awful, or they are
+ * all unreachable objects about to be pruned, in which case they're not really
+ * interesting as a measure of repo size in the first place.
+ */
+unsigned long repo_approximate_object_count(struct repository *r)
+{
+	if (!r->objects->approximate_object_count_valid) {
+		struct odb_source *source;
+		unsigned long count = 0;
+		struct packed_git *p;
+
+		prepare_packed_git(r);
+
+		for (source = r->objects->sources; source; source = source->next) {
+			struct multi_pack_index *m = get_multi_pack_index(source);
+			if (m)
+				count += m->num_objects;
+		}
+
+		for (p = r->objects->packfiles->packs; p; p = p->next) {
+			if (open_pack_index(p))
+				continue;
+			count += p->num_objects;
+		}
+		r->objects->approximate_object_count = count;
+		r->objects->approximate_object_count_valid = 1;
+	}
+	return r->objects->approximate_object_count;
+}
+
 unsigned long unpack_object_header_buffer(const unsigned char *buf,
 		unsigned long len, enum object_type *type, unsigned long *sizep)
 {
-- 
2.51.0.261.g7ce5a0a67e.dirty
Taylor Blau· Aug 26, 2025, 01:47 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v2 07/16] packfile: reorder functions to avoid function declaration

On Thu, Aug 21, 2025 at 09:39:05AM +0200, Patrick Steinhardt wrote:
> ---
>  packfile.c | 67 +++++++++++++++++++++++++++++++-------------------------------
>  1 file changed, 33 insertions(+), 34 deletions(-)

Inspecting the diff locally with --color-moved shows that the changes are faithful here; thanks.

Thanks, Taylor

Patrick Steinhardt· Aug 21, 2025, 07:39 UTC · re: Patrick Steinhardt · lore

[PATCH v2 08/16] packfile: refactor `prepare_packed_git()` to work on packfile store

The `prepare_packed_git()` function and its friends are responsible for loading packfiles as well as the multi-pack index for a given object database. Refactor these functions to accept a packfile store instead of a repository to clarify their scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 packfile.c | 43 +++++++++++++++++++------------------------
 1 file changed, 19 insertions(+), 24 deletions(-)
Show changes to packfile.c +19 −24
diff --git a/packfile.c b/packfile.c
index 90f15b0c20..2e45a3a05f 100644
--- a/packfile.c
+++ b/packfile.c
@@ -974,38 +974,33 @@ static int sort_pack(const struct packed_git *a, const struct packed_git *b)
 	return -1;
 }
 
-static void rearrange_packed_git(struct repository *r)
-{
-	sort_packs(&r->objects->packfiles->packs, sort_pack);
-}
-
-static void prepare_packed_git_mru(struct repository *r)
+static void packfile_store_prepare_mru(struct packfile_store *store)
 {
 	struct packed_git *p;
 
-	INIT_LIST_HEAD(&r->objects->packfiles->mru);
+	INIT_LIST_HEAD(&store->mru);
 
-	for (p = r->objects->packfiles->packs; p; p = p->next)
-		list_add_tail(&p->mru, &r->objects->packfiles->mru);
+	for (p = store->packs; p; p = p->next)
+		list_add_tail(&p->mru, &store->mru);
 }
 
-static void prepare_packed_git(struct repository *r)
+static void packfile_store_prepare(struct packfile_store *store)
 {
 	struct odb_source *source;
 
-	if (r->objects->packfiles->initialized)
+	if (store->initialized)
 		return;
 
-	odb_prepare_alternates(r->objects);
-	for (source = r->objects->sources; source; source = source->next) {
-		int local = (source == r->objects->sources);
+	odb_prepare_alternates(store->odb);
+	for (source = store->odb->sources; source; source = source->next) {
+		int local = (source == store->odb->sources);
 		prepare_multi_pack_index_one(source, local);
 		prepare_packed_git_one(source, local);
 	}
-	rearrange_packed_git(r);
+	sort_packs(&store->packs, sort_pack);
 
-	prepare_packed_git_mru(r);
-	r->objects->packfiles->initialized = true;
+	packfile_store_prepare_mru(store);
+	store->initialized = true;
 }
 
 void reprepare_packed_git(struct repository *r)
@@ -1028,25 +1023,25 @@ void reprepare_packed_git(struct repository *r)
 
 	r->objects->approximate_object_count_valid = 0;
 	r->objects->packfiles->initialized = false;
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 	obj_read_unlock();
 }
 
 struct packed_git *get_packed_git(struct repository *r)
 {
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 	return r->objects->packfiles->packs;
 }
 
 struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
 {
-	prepare_packed_git(source->odb->repo);
+	packfile_store_prepare(source->odb->packfiles);
 	return source->midx;
 }
 
 struct packed_git *get_all_packs(struct repository *r)
 {
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 
 	for (struct odb_source *source = r->objects->sources; source; source = source->next) {
 		struct multi_pack_index *m = source->midx;
@@ -1061,7 +1056,7 @@ struct packed_git *get_all_packs(struct repository *r)
 
 struct list_head *get_packed_git_mru(struct repository *r)
 {
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 	return &r->objects->packfiles->mru;
 }
 
@@ -1079,7 +1074,7 @@ unsigned long repo_approximate_object_count(struct repository *r)
 		unsigned long count = 0;
 		struct packed_git *p;
 
-		prepare_packed_git(r);
+		packfile_store_prepare(r->objects->packfiles);
 
 		for (source = r->objects->sources; source; source = source->next) {
 			struct multi_pack_index *m = get_multi_pack_index(source);
@@ -2069,7 +2064,7 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 {
 	struct list_head *pos;
 
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 
 	for (struct odb_source *source = r->objects->sources; source; source = source->next)
 		if (source->midx && fill_midx_entry(r, oid, e, source->midx))
-- 
2.51.0.261.g7ce5a0a67e.dirty
Taylor Blau· Aug 26, 2025, 01:58 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v2 08/16] packfile: refactor `prepare_packed_git()` to work on packfile store

On Thu, Aug 21, 2025 at 09:39:06AM +0200, Patrick Steinhardt wrote:
Show 22 quoted lines
> The `prepare_packed_git()` function and its friends are responsible for
> loading packfiles as well as the multi-pack index for a given object
> database. Refactor these functions to accept a packfile store instead of
> a repository to clarify their scope.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  packfile.c | 43 +++++++++++++++++++------------------------
>  1 file changed, 19 insertions(+), 24 deletions(-)
>
> diff --git a/packfile.c b/packfile.c
> index 90f15b0c20..2e45a3a05f 100644
> --- a/packfile.c
> +++ b/packfile.c
> @@ -974,38 +974,33 @@ static int sort_pack(const struct packed_git *a, const struct packed_git *b)
>  	return -1;
>  }
>
> -static void rearrange_packed_git(struct repository *r)
> -{
> -	sort_packs(&r->objects->packfiles->packs, sort_pack);
> -}

OK, makes sense -- it looks like you inlined rearrange_packed_git() in its sole caller packfile_store_prepare() below. I think that could have been done as a separate step, but it's equally fine to include it here, too.

Show 12 quoted lines
> -static void prepare_packed_git_mru(struct repository *r)
> +static void packfile_store_prepare_mru(struct packfile_store *store)
>  {
>  	struct packed_git *p;
>
> -	INIT_LIST_HEAD(&r->objects->packfiles->mru);
> +	INIT_LIST_HEAD(&store->mru);
>
> -	for (p = r->objects->packfiles->packs; p; p = p->next)
> -		list_add_tail(&p->mru, &r->objects->packfiles->mru);
> +	for (p = store->packs; p; p = p->next)
> +		list_add_tail(&p->mru, &store->mru);
Looks all good.
Show 13 quoted lines
> -static void prepare_packed_git(struct repository *r)
> +static void packfile_store_prepare(struct packfile_store *store)
>  {
>  	struct odb_source *source;
>
> -	if (r->objects->packfiles->initialized)
> +	if (store->initialized)
>  		return;
>
> -	odb_prepare_alternates(r->objects);
> -	for (source = r->objects->sources; source; source = source->next) {
> -		int local = (source == r->objects->sources);
> +	odb_prepare_alternates(store->odb);

Hmmm. I admit that I don't love that the packfile_store knows about the object_store that it belongs to. This feels like a layering violation and makes me worry that we pushed too much down into the new packfile_store. I'm not sure I have a better idea off the top of my head, though.

Everything down from here looks good to me.

Thanks, Taylor

Patrick Steinhardt· Aug 21, 2025, 07:39 UTC · re: Patrick Steinhardt · lore

[PATCH v2 09/16] packfile: split up responsibilities of `reprepare_packed_git()`

In `reprepare_packed_git()` we perform a couple of operations:
  - We reload alternate object directories.
  - We clear the loose object cache.
  - We reprepare packfiles.

While the logic is hosted in "packfile.c", it clearly reaches into other subsystems that aren't related to packfiles.

Split up the responsibility and introduce `odb_reprepare()` which now becomes responsible for repreparing the whole object database. The existing `reprepare_packed_git()` function is refactored accordingly and only cares about reloading the packfile store now.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/backfill.c     |  2 +-
 builtin/gc.c           |  4 ++--
 builtin/receive-pack.c |  2 +-
 builtin/repack.c       |  2 +-
 bulk-checkin.c         |  2 +-
 connected.c            |  2 +-
 fetch-pack.c           |  4 ++--
 object-name.c          |  2 +-
 odb.c                  | 25 ++++++++++++++++++++++++-
 odb.h                  |  6 ++++++
 packfile.c             | 26 ++++----------------------
 packfile.h             |  9 ++++++++-
 transport-helper.c     |  2 +-
 13 files changed, 53 insertions(+), 35 deletions(-)
Show changes to 13 files +53 −35

builtin/backfill.c, builtin/gc.c, builtin/receive-pack.c, builtin/repack.c, bulk-checkin.c, connected.c, fetch-pack.c, object-name.c, odb.c, odb.h, packfile.c, packfile.h, transport-helper.c

diff --git a/builtin/backfill.c b/builtin/backfill.c
index 80056abe47..e80fc1b694 100644
--- a/builtin/backfill.c
+++ b/builtin/backfill.c
@@ -53,7 +53,7 @@ static void download_batch(struct backfill_context *ctx)
 	 * We likely have a new packfile. Add it to the packed list to
 	 * avoid possible duplicate downloads of the same objects.
 	 */
-	reprepare_packed_git(ctx->repo);
+	odb_reprepare(ctx->repo->objects);
 }
 
 static int fill_missing_blobs(const char *path UNUSED,
diff --git a/builtin/gc.c b/builtin/gc.c
index 0edd94a76f..1d30d1af2c 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1041,7 +1041,7 @@ int cmd_gc(int argc,
 		die(FAILED_RUN, "rerere");
 
 	report_garbage = report_pack_garbage;
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 	if (pack_garbage.nr > 0) {
 		close_object_store(the_repository->objects);
 		clean_pack_garbage();
@@ -1490,7 +1490,7 @@ static off_t get_auto_pack_size(void)
 	struct packed_git *p;
 	struct repository *r = the_repository;
 
-	reprepare_packed_git(r);
+	odb_reprepare(r->objects);
 	for (p = get_all_packs(r); p; p = p->next) {
 		if (p->pack_size > max_size) {
 			second_largest_size = max_size;
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 1113137a6f..c9288a9c7e 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -2389,7 +2389,7 @@ static const char *unpack(int err_fd, struct shallow_info *si)
 		status = finish_command(&child);
 		if (status)
 			return "index-pack abnormal exit";
-		reprepare_packed_git(the_repository);
+		odb_reprepare(the_repository->objects);
 	}
 	return NULL;
 }
diff --git a/builtin/repack.c b/builtin/repack.c
index a4def39197..ee8c80cd95 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -1684,7 +1684,7 @@ int cmd_repack(int argc,
 			goto cleanup;
 	}
 
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 
 	if (delete_redundant) {
 		int opts = 0;
diff --git a/bulk-checkin.c b/bulk-checkin.c
index b2809ab039..f65439a748 100644
--- a/bulk-checkin.c
+++ b/bulk-checkin.c
@@ -90,7 +90,7 @@ static void flush_bulk_checkin_packfile(struct bulk_checkin_packfile *state)
 
 	strbuf_release(&packname);
 	/* Make objects we just wrote available to ourselves */
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 }
 
 /*
diff --git a/connected.c b/connected.c
index 18c13245d8..d6e9682fd9 100644
--- a/connected.c
+++ b/connected.c
@@ -72,7 +72,7 @@ int check_connected(oid_iterate_fn fn, void *cb_data,
 		 * Before checking for promisor packs, be sure we have the
 		 * latest pack-files loaded into memory.
 		 */
-		reprepare_packed_git(the_repository);
+		odb_reprepare(the_repository->objects);
 		do {
 			struct packed_git *p;
 
diff --git a/fetch-pack.c b/fetch-pack.c
index 46c39f85c4..3b8960608c 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -1982,7 +1982,7 @@ static void update_shallow(struct fetch_pack_args *args,
 		 * remote is shallow, but this is a clone, there are
 		 * no objects in repo to worry about. Accept any
 		 * shallow points that exist in the pack (iow in repo
-		 * after get_pack() and reprepare_packed_git())
+		 * after get_pack() and odb_reprepare())
 		 */
 		struct oid_array extra = OID_ARRAY_INIT;
 		struct object_id *oid = si->shallow->oid;
@@ -2107,7 +2107,7 @@ struct ref *fetch_pack(struct fetch_pack_args *args,
 		ref_cpy = do_fetch_pack(args, fd, ref, sought, nr_sought,
 					&si, pack_lockfiles);
 	}
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 
 	if (!args->cloning && args->deepen) {
 		struct check_connected_options opt = CHECK_CONNECTED_INIT;
diff --git a/object-name.c b/object-name.c
index 11aa0e6afc..44b0d416ac 100644
--- a/object-name.c
+++ b/object-name.c
@@ -596,7 +596,7 @@ static enum get_oid_result get_short_oid(struct repository *r,
 	 * or migrated from loose to packed.
 	 */
 	if (status == MISSING_OBJECT) {
-		reprepare_packed_git(r);
+		odb_reprepare(r->objects);
 		find_short_object_filename(&ds);
 		find_short_packed_object(&ds);
 		status = finish_object_disambiguation(&ds, oid);
diff --git a/odb.c b/odb.c
index 80ec6fc1fa..37ed21f53b 100644
--- a/odb.c
+++ b/odb.c
@@ -694,7 +694,7 @@ static int do_oid_object_info_extended(struct object_database *odb,
 
 		/* Not a loose object; someone else may have just packed it. */
 		if (!(flags & OBJECT_INFO_QUICK)) {
-			reprepare_packed_git(odb->repo);
+			odb_reprepare(odb->repo->objects);
 			if (find_pack_entry(odb->repo, real, &e))
 				break;
 		}
@@ -1039,3 +1039,26 @@ void odb_clear(struct object_database *o)
 
 	string_list_clear(&o->submodule_source_paths, 0);
 }
+
+void odb_reprepare(struct object_database *o)
+{
+	struct odb_source *source;
+
+	/*
+	 * Reprepare alt odbs, in case the alternates file was modified
+	 * during the course of this process. This only _adds_ odbs to
+	 * the linked list, so existing odbs will continue to exist for
+	 * the lifetime of the process.
+	 */
+	o->loaded_alternates = 0;
+	odb_prepare_alternates(o);
+
+	for (source = o->sources; source; source = source->next)
+		odb_clear_loose_cache(source);
+
+	o->approximate_object_count_valid = 0;
+
+	packfile_store_reprepare(o->packfiles);
+
+	obj_read_unlock();
+}
diff --git a/odb.h b/odb.h
index f1736b067c..9810ec60a0 100644
--- a/odb.h
+++ b/odb.h
@@ -155,6 +155,12 @@ struct object_database {
 struct object_database *odb_new(struct repository *repo);
 void odb_clear(struct object_database *o);
 
+/*
+ * Clear caches, reload alternates and then reload object sources so that new
+ * objects may become accessible.
+ */
+void odb_reprepare(struct object_database *o);
+
 /*
  * Find source by its object directory path. Dies in case the source couldn't
  * be found.
diff --git a/packfile.c b/packfile.c
index 2e45a3a05f..8e446dda69 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1003,28 +1003,10 @@ static void packfile_store_prepare(struct packfile_store *store)
 	store->initialized = true;
 }
 
-void reprepare_packed_git(struct repository *r)
+void packfile_store_reprepare(struct packfile_store *store)
 {
-	struct odb_source *source;
-
-	obj_read_lock();
-
-	/*
-	 * Reprepare alt odbs, in case the alternates file was modified
-	 * during the course of this process. This only _adds_ odbs to
-	 * the linked list, so existing odbs will continue to exist for
-	 * the lifetime of the process.
-	 */
-	r->objects->loaded_alternates = 0;
-	odb_prepare_alternates(r->objects);
-
-	for (source = r->objects->sources; source; source = source->next)
-		odb_clear_loose_cache(source);
-
-	r->objects->approximate_object_count_valid = 0;
-	r->objects->packfiles->initialized = false;
-	packfile_store_prepare(r->objects->packfiles);
-	obj_read_unlock();
+	store->initialized = false;
+	packfile_store_prepare(store);
 }
 
 struct packed_git *get_packed_git(struct repository *r)
@@ -1145,7 +1127,7 @@ unsigned long get_size_from_delta(struct packed_git *p,
 		 *
 		 * Other worrying sections could be the call to close_pack_fd(),
 		 * which can close packs even with in-use windows, and to
-		 * reprepare_packed_git(). Regarding the former, mmap doc says:
+		 * odb_reprepare(). Regarding the former, mmap doc says:
 		 * "closing the file descriptor does not unmap the region". And
 		 * for the latter, it won't re-open already available packs.
 		 */
diff --git a/packfile.h b/packfile.h
index 74cea1a4a9..e9e60ec21b 100644
--- a/packfile.h
+++ b/packfile.h
@@ -103,6 +103,14 @@ void packfile_store_free(struct packfile_store *store);
  */
 void packfile_store_close(struct packfile_store *store);
 
+/*
+ * Clear the packfile caches and try to look up any new packfiles that have
+ * appeared since last preparing the packfiles store.
+ *
+ * This function must be called under the `odb_read_lock()`.
+ */
+void packfile_store_reprepare(struct packfile_store *store);
+
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
@@ -179,7 +187,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-void reprepare_packed_git(struct repository *r);
 void install_packed_git(struct repository *r, struct packed_git *pack);
 
 struct packed_git *get_packed_git(struct repository *r);
diff --git a/transport-helper.c b/transport-helper.c
index 0789e5bca5..4d95d84f9e 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -450,7 +450,7 @@ static int fetch_with_fetch(struct transport *transport,
 	}
 	strbuf_release(&buf);
 
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 	return 0;
 }
 
-- 
2.51.0.261.g7ce5a0a67e.dirty
Taylor Blau· Aug 26, 2025, 02:10 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v2 09/16] packfile: split up responsibilities of `reprepare_packed_git()`

On Thu, Aug 21, 2025 at 09:39:07AM +0200, Patrick Steinhardt wrote:
Show 7 quoted lines
> While the logic is hosted in "packfile.c", it clearly reaches into other
> subsystems that aren't related to packfiles.
>
> Split up the responsibility and introduce `odb_reprepare()` which now
> becomes responsible for repreparing the whole object database. The
> existing `reprepare_packed_git()` function is refactored accordingly and
> only cares about reloading the packfile store now.
Makes sense.
Show 19 quoted lines
> diff --git a/odb.c b/odb.c
> index 80ec6fc1fa..37ed21f53b 100644
> --- a/odb.c
> +++ b/odb.c
> @@ -694,7 +694,7 @@ static int do_oid_object_info_extended(struct object_database *odb,
>
>  		/* Not a loose object; someone else may have just packed it. */
>  		if (!(flags & OBJECT_INFO_QUICK)) {
> -			reprepare_packed_git(odb->repo);
> +			odb_reprepare(odb->repo->objects);
>  			if (find_pack_entry(odb->repo, real, &e))
>  				break;
>  		}
> @@ -1039,3 +1039,26 @@ void odb_clear(struct object_database *o)
>
>  	string_list_clear(&o->submodule_source_paths, 0);
>  }
> +
> +void odb_reprepare(struct object_database *o)

OK; so here is the new location for the non-packfile related portions of the former reprepare_packed_git() function. That makes sense, but...

Show 20 quoted lines
> +{
> +	struct odb_source *source;
> +
> +	/*
> +	 * Reprepare alt odbs, in case the alternates file was modified
> +	 * during the course of this process. This only _adds_ odbs to
> +	 * the linked list, so existing odbs will continue to exist for
> +	 * the lifetime of the process.
> +	 */
> +	o->loaded_alternates = 0;
> +	odb_prepare_alternates(o);
> +
> +	for (source = o->sources; source; source = source->next)
> +		odb_clear_loose_cache(source);
> +
> +	o->approximate_object_count_valid = 0;
> +
> +	packfile_store_reprepare(o->packfiles);
> +
> +	obj_read_unlock();

...I think I am missing where we call odb_read_lock(). The function packfile_store_reprepare() has a comment that it must be called under the odb_read_lock(), but I don't see where we acquire that lock here.

Are the callers of odb_reprepare() supposed to acquire that lock? If so, it seems a little awkward that the caller is supposed to acquire the lock, but the callee is the one to release it. Is this function missing a odb_read_lock() at the top?

I looked at a few callers here and none of them seem to be holding this lock. pthread_mutex_unlock() is supposed to check that the mutex lock is held for recursive and error-checking mutexes. IIRC we initialize the the odb mutex with PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP, so I am a little surprised that this did not cause a runtime error.

The rest of the patch looks good.

Thanks, Taylor

Patrick Steinhardt· Sep 2, 2025, 08:50 UTC · re: Taylor Blau · lore

Re: [PATCH v2 09/16] packfile: split up responsibilities of `reprepare_packed_git()`

On Mon, Aug 25, 2025 at 10:10:02PM -0400, Taylor Blau wrote:
Show 59 quoted lines
> On Thu, Aug 21, 2025 at 09:39:07AM +0200, Patrick Steinhardt wrote:
> > diff --git a/odb.c b/odb.c
> > index 80ec6fc1fa..37ed21f53b 100644
> > --- a/odb.c
> > +++ b/odb.c
> > @@ -694,7 +694,7 @@ static int do_oid_object_info_extended(struct object_database *odb,
> >
> >  		/* Not a loose object; someone else may have just packed it. */
> >  		if (!(flags & OBJECT_INFO_QUICK)) {
> > -			reprepare_packed_git(odb->repo);
> > +			odb_reprepare(odb->repo->objects);
> >  			if (find_pack_entry(odb->repo, real, &e))
> >  				break;
> >  		}
> > @@ -1039,3 +1039,26 @@ void odb_clear(struct object_database *o)
> >
> >  	string_list_clear(&o->submodule_source_paths, 0);
> >  }
> > +
> > +void odb_reprepare(struct object_database *o)
> 
> OK; so here is the new location for the non-packfile related portions of
> the former reprepare_packed_git() function. That makes sense, but...
> 
> > +{
> > +	struct odb_source *source;
> > +
> > +	/*
> > +	 * Reprepare alt odbs, in case the alternates file was modified
> > +	 * during the course of this process. This only _adds_ odbs to
> > +	 * the linked list, so existing odbs will continue to exist for
> > +	 * the lifetime of the process.
> > +	 */
> > +	o->loaded_alternates = 0;
> > +	odb_prepare_alternates(o);
> > +
> > +	for (source = o->sources; source; source = source->next)
> > +		odb_clear_loose_cache(source);
> > +
> > +	o->approximate_object_count_valid = 0;
> > +
> > +	packfile_store_reprepare(o->packfiles);
> > +
> > +	obj_read_unlock();
> 
> ...I think I am missing where we call odb_read_lock(). The function
> packfile_store_reprepare() has a comment that it must be called under
> the odb_read_lock(), but I don't see where we acquire that lock here.
> 
> Are the callers of odb_reprepare() supposed to acquire that lock? If so,
> it seems a little awkward that the caller is supposed to acquire the
> lock, but the callee is the one to release it. Is this function missing
> a odb_read_lock() at the top?
> 
> I looked at a few callers here and none of them seem to be holding this
> lock. pthread_mutex_unlock() is supposed to check that the mutex lock is
> held for recursive and error-checking mutexes. IIRC we initialize the
> the odb mutex with PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP, so I am a
> little surprised that this did not cause a runtime error.

Oh, that's a very good catch. And yes, surprising indeed that this does not cause an error anywhere. But if you look at the code it ultimately isn't all that surprising: there is only a single command in our code base that performs concurrent reads via the ODB, namely git-grep(1). That command explicitly opts into `obj_read_use_lock` by calling `enable_obj_read_lock()`. All the other commands never perform any locking whatsoever.

In any case, I think I merely missed to add the locking call into `odb_reprepare()`. It shouldn't be the responsibility of the caller to care about locking, it should be handled automatically.

Will fix.
Patrick
Patrick Steinhardt· Aug 21, 2025, 07:39 UTC · re: Patrick Steinhardt · lore

[PATCH v2 10/16] packfile: refactor `install_packed_git()` to work on packfile store

The `install_packed_git()` functions adds a packfile to a specific object store. Refactor it to accept a packfile store instead of a repository to clarify its scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/fast-import.c |  2 +-
 builtin/index-pack.c  |  2 +-
 http.c                |  2 +-
 http.h                |  2 +-
 midx.c                |  2 +-
 packfile.c            | 11 ++++++-----
 packfile.h            |  9 +++++++--
 7 files changed, 18 insertions(+), 12 deletions(-)
Show changes to 7 files +18 −12

builtin/fast-import.c, builtin/index-pack.c, http.c, http.h, midx.c, packfile.c, packfile.h

diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index 2c35f9345d..e9d82b31c3 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -901,7 +901,7 @@ static void end_packfile(void)
 		if (!new_p)
 			die("core git rejected index %s", idx_name);
 		all_packs[pack_id] = new_p;
-		install_packed_git(the_repository, new_p);
+		packfile_store_add_pack(the_repository->objects->packfiles, new_p);
 		free(idx_name);
 
 		/* Print the boundary */
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index f91c301bba..ed490dfad4 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1645,7 +1645,7 @@ static void final(const char *final_pack_name, const char *curr_pack_name,
 		p = add_packed_git(the_repository, final_index_name,
 				   strlen(final_index_name), 0);
 		if (p)
-			install_packed_git(the_repository, p);
+			packfile_store_add_pack(the_repository->objects->packfiles, p);
 	}
 
 	if (!from_stdin) {
diff --git a/http.c b/http.c
index 98853d6483..af2120b64c 100644
--- a/http.c
+++ b/http.c
@@ -2541,7 +2541,7 @@ void http_install_packfile(struct packed_git *p,
 		lst = &((*lst)->next);
 	*lst = (*lst)->next;
 
-	install_packed_git(the_repository, p);
+	packfile_store_add_pack(the_repository->objects->packfiles, p);
 }
 
 struct http_pack_request *new_http_pack_request(
diff --git a/http.h b/http.h
index 36202139f4..e5a5380c6c 100644
--- a/http.h
+++ b/http.h
@@ -210,7 +210,7 @@ int finish_http_pack_request(struct http_pack_request *preq);
 void release_http_pack_request(struct http_pack_request *preq);
 
 /*
- * Remove p from the given list, and invoke install_packed_git() on it.
+ * Remove p from the given list, and invoke packfile_store_add_pack() on it.
  *
  * This is a convenience function for users that have obtained a list of packs
  * from http_get_info_packs() and have chosen a specific pack to fetch.
diff --git a/midx.c b/midx.c
index 7fa2b8473a..95e74c79c1 100644
--- a/midx.c
+++ b/midx.c
@@ -477,7 +477,7 @@ int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,
 	if (!p) {
 		p = add_packed_git(r, pack_name.buf, pack_name.len, m->local);
 		if (p) {
-			install_packed_git(r, p);
+			packfile_store_add_pack(r->objects->packfiles, p);
 			list_add_tail(&p->mru, &r->objects->packfiles->mru);
 		}
 	}
diff --git a/packfile.c b/packfile.c
index 8e446dda69..c885046d9f 100644
--- a/packfile.c
+++ b/packfile.c
@@ -779,16 +779,17 @@ struct packed_git *add_packed_git(struct repository *r, const char *path,
 	return p;
 }
 
-void install_packed_git(struct repository *r, struct packed_git *pack)
+void packfile_store_add_pack(struct packfile_store *store,
+			     struct packed_git *pack)
 {
 	if (pack->pack_fd != -1)
 		pack_open_fds++;
 
-	pack->next = r->objects->packfiles->packs;
-	r->objects->packfiles->packs = pack;
+	pack->next = store->packs;
+	store->packs = pack;
 
 	hashmap_entry_init(&pack->packmap_ent, strhash(pack->pack_name));
-	hashmap_add(&r->objects->packfiles->map, &pack->packmap_ent);
+	hashmap_add(&store->map, &pack->packmap_ent);
 }
 
 void (*report_garbage)(unsigned seen_bits, const char *path);
@@ -904,7 +905,7 @@ static void prepare_pack(const char *full_name, size_t full_name_len,
 		if (!hashmap_get(&data->r->objects->packfiles->map, &hent, pack_name)) {
 			p = add_packed_git(data->r, full_name, full_name_len, data->local);
 			if (p)
-				install_packed_git(data->r, p);
+				packfile_store_add_pack(data->r->objects->packfiles, p);
 		}
 		free(pack_name);
 	}
diff --git a/packfile.h b/packfile.h
index e9e60ec21b..6641238796 100644
--- a/packfile.h
+++ b/packfile.h
@@ -111,6 +111,13 @@ void packfile_store_close(struct packfile_store *store);
  */
 void packfile_store_reprepare(struct packfile_store *store);
 
+/*
+ * Add the pack to the store so that contained objects become accessible via
+ * the store. This moves ownership into the store.
+ */
+void packfile_store_add_pack(struct packfile_store *store,
+			     struct packed_git *pack);
+
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
@@ -187,8 +194,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-void install_packed_git(struct repository *r, struct packed_git *pack);
-
 struct packed_git *get_packed_git(struct repository *r);
 struct list_head *get_packed_git_mru(struct repository *r);
 struct multi_pack_index *get_multi_pack_index(struct odb_source *source);
-- 
2.51.0.261.g7ce5a0a67e.dirty
Taylor Blau· Aug 26, 2025, 02:11 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v2 10/16] packfile: refactor `install_packed_git()` to work on packfile store

On Thu, Aug 21, 2025 at 09:39:08AM +0200, Patrick Steinhardt wrote:
> The `install_packed_git()` functions adds a packfile to a specific
> object store. Refactor it to accept a packfile store instead of a
> repository to clarify its scope.

All of the refactoring here looks straightforward and correct to me. I admit that I have a vague preference towards keeping the word "install" in the function name here, since it (to me) suggests that the packfile in question is going to be used for lookups, whereas "add" is a bit more generic.

I don't feel strongly about it, though, so if you have a preference towards "add" then I'm fine with that.

Thanks, Taylor

Patrick Steinhardt· Sep 2, 2025, 08:50 UTC · re: Taylor Blau · lore

Re: [PATCH v2 10/16] packfile: refactor `install_packed_git()` to work on packfile store

On Mon, Aug 25, 2025 at 10:11:52PM -0400, Taylor Blau wrote:
Show 13 quoted lines
> On Thu, Aug 21, 2025 at 09:39:08AM +0200, Patrick Steinhardt wrote:
> > The `install_packed_git()` functions adds a packfile to a specific
> > object store. Refactor it to accept a packfile store instead of a
> > repository to clarify its scope.
> 
> All of the refactoring here looks straightforward and correct to me. I
> admit that I have a vague preference towards keeping the word "install"
> in the function name here, since it (to me) suggests that the packfile
> in question is going to be used for lookups, whereas "add" is a bit
> more generic.
> 
> I don't feel strongly about it, though, so if you have a preference
> towards "add" then I'm fine with that.

I think "install" would make sense if there was a mode where you can set up a packfile via the store that is _inactive_, but that's not the case anymore. It was before the changes to introduce the packfile store, where "add" only instantiated the packfile whereas "install" added it to the store. But that difference is basically going away now, so as soon as you add a packfile to the store it immediately becomes active.

So I picked "add" because it's a bit shorter compared to "install", and because from my point of view the previous distinction isn't necessary anymore.

I'll keep this as-is for now, but same as you: I don't really feel all that strongly about it.

Patrick
Patrick Steinhardt· Aug 21, 2025, 07:39 UTC · re: Patrick Steinhardt · lore

[PATCH v2 11/16] packfile: always add packfiles to MRU when adding a pack

When adding a packfile to it store we add it both to the list and map of packfiles, but we don't append it to the most-recently-used list of packs. We do know to add the packfile to the MRU list as soon as we access any of its objects, but in between we're being inconistent. It doesn't help that there are some subsystems that _do_ add the packfile to the MRU after having added it, which only adds to the confusion.

Refactor the code so that we unconditionally add packfiles to the MRU when adding them to a packfile store.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 midx.c     | 4 +---
 packfile.c | 1 +
 2 files changed, 2 insertions(+), 3 deletions(-)
Show changes to 2 files +2 −3

midx.c, packfile.c

diff --git a/midx.c b/midx.c
index 95e74c79c1..3cfe7884ad 100644
--- a/midx.c
+++ b/midx.c
@@ -476,10 +476,8 @@ int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,
 					struct packed_git, packmap_ent);
 	if (!p) {
 		p = add_packed_git(r, pack_name.buf, pack_name.len, m->local);
-		if (p) {
+		if (p)
 			packfile_store_add_pack(r->objects->packfiles, p);
-			list_add_tail(&p->mru, &r->objects->packfiles->mru);
-		}
 	}
 
 	strbuf_release(&pack_name);
diff --git a/packfile.c b/packfile.c
index c885046d9f..a79d0fc1fa 100644
--- a/packfile.c
+++ b/packfile.c
@@ -790,6 +790,7 @@ void packfile_store_add_pack(struct packfile_store *store,
 
 	hashmap_entry_init(&pack->packmap_ent, strhash(pack->pack_name));
 	hashmap_add(&store->map, &pack->packmap_ent);
+	list_add_tail(&pack->mru, &store->mru);
 }
 
 void (*report_garbage)(unsigned seen_bits, const char *path);
-- 
2.51.0.261.g7ce5a0a67e.dirty
Taylor Blau· Aug 27, 2025, 01:04 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v2 11/16] packfile: always add packfiles to MRU when adding a pack

On Thu, Aug 21, 2025 at 09:39:09AM +0200, Patrick Steinhardt wrote:
Show 9 quoted lines
> When adding a packfile to it store we add it both to the list and map of
> packfiles, but we don't append it to the most-recently-used list of
> packs. We do know to add the packfile to the MRU list as soon as we
> access any of its objects, but in between we're being inconistent. It
> doesn't help that there are some subsystems that _do_ add the packfile
> to the MRU after having added it, which only adds to the confusion.
>
> Refactor the code so that we unconditionally add packfiles to the MRU
> when adding them to a packfile store.

I am a little confused why prepare_midx_pack() wants to add packs to the MRU cache so eagerly, and the commit which introduced that behavior (commit af96fe3392 (midx: add packs to packed_git linked list, 2019-04-29)) doesn't focus on that area in detail.

(Note that commit af96fe3392 *does* discuss a separate cache's behavior regarding the open file descriptor limit, but that LRU cache is a different one than the MRU cache we're discussing here.)

What I do wonder about is why af96fe3392 adds packs to the MRU cache in the first place. As far as I can tell, we never move MIDX'd packs to the front of the MRU cache at all. There are two spots that call list_move() on the MRU cache, which are:

 - packfile.c::find_pack_entry(), which enumerates MIDX'd
   packs in a separate loop earlier on in the function, and ignores
   packs in the MRU cache whose p->multi_pack_index bit is set.
 - builtin/pack-objects.c::want_object_in_pack_mtime(), which also
   enumerates MIDX'd packs in a separate loop, though it does not
   explicitly ignore packs in the MRU cache with the multi_pack_index
   bit set.

In practice, though, I think these two are equivalent, since want_object_in_pack_mtime() will return before it gets to the MRU cache if it found the object in a MIDX'd pack.

So I don't think we need to be adding MIDX'd packs to the MRU cache in the first place.

Show 19 quoted lines
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  midx.c     | 4 +---
>  packfile.c | 1 +
>  2 files changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/midx.c b/midx.c
> index 95e74c79c1..3cfe7884ad 100644
> --- a/midx.c
> +++ b/midx.c
> @@ -476,10 +476,8 @@ int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,
>  					struct packed_git, packmap_ent);
>  	if (!p) {
>  		p = add_packed_git(r, pack_name.buf, pack_name.len, m->local);
> -		if (p) {
> +		if (p)
>  			packfile_store_add_pack(r->objects->packfiles, p);
> -			list_add_tail(&p->mru, &r->objects->packfiles->mru);
> -		}

All of that aside, this portion of the diff is preserving the existing behavior, since it inlines the list_add_tail() call within packfile_store_add_pack(). OK.

Show 9 quoted lines
> diff --git a/packfile.c b/packfile.c
> index c885046d9f..a79d0fc1fa 100644
> --- a/packfile.c
> +++ b/packfile.c
> @@ -790,6 +790,7 @@ void packfile_store_add_pack(struct packfile_store *store,
>
>  	hashmap_entry_init(&pack->packmap_ent, strhash(pack->pack_name));
>  	hashmap_add(&store->map, &pack->packmap_ent);
> +	list_add_tail(&pack->mru, &store->mru);

But this spot makes me a little unsure. There are callers of what is now packfile_store_add_pack() that *don't* immediately add the pack to the MRU cache (e.g., packfile.c::prepare_pack()). I think that behavior is intentional, since we don't necessarily want to have packs in the MRU cache which haven't actually received an object lookup yet.

So I am not sure I understand the full extent of this change. I think it might be worth investigating the eagerness of prepare_midx_pack() to add packs to the MRU cache, and perhaps drop that behavior altogether, which I think would obviate the need for this patch in the series.

Thanks, Taylor

Patrick Steinhardt· Sep 2, 2025, 08:50 UTC · re: Taylor Blau · lore

Re: [PATCH v2 11/16] packfile: always add packfiles to MRU when adding a pack

On Tue, Aug 26, 2025 at 09:04:24PM -0400, Taylor Blau wrote:
Show 40 quoted lines
> On Thu, Aug 21, 2025 at 09:39:09AM +0200, Patrick Steinhardt wrote:
> > When adding a packfile to it store we add it both to the list and map of
> > packfiles, but we don't append it to the most-recently-used list of
> > packs. We do know to add the packfile to the MRU list as soon as we
> > access any of its objects, but in between we're being inconistent. It
> > doesn't help that there are some subsystems that _do_ add the packfile
> > to the MRU after having added it, which only adds to the confusion.
> >
> > Refactor the code so that we unconditionally add packfiles to the MRU
> > when adding them to a packfile store.
> 
> I am a little confused why prepare_midx_pack() wants to add packs to the
> MRU cache so eagerly, and the commit which introduced that behavior
> (commit af96fe3392 (midx: add packs to packed_git linked list,
> 2019-04-29)) doesn't focus on that area in detail.
> 
> (Note that commit af96fe3392 *does* discuss a separate cache's behavior
> regarding the open file descriptor limit, but that LRU cache is a
> different one than the MRU cache we're discussing here.)
> 
> What I do wonder about is why af96fe3392 adds packs to the MRU cache in
> the first place. As far as I can tell, we never move MIDX'd packs to
> the front of the MRU cache at all. There are two spots that call
> list_move() on the MRU cache, which are:
> 
>  - packfile.c::find_pack_entry(), which enumerates MIDX'd
>    packs in a separate loop earlier on in the function, and ignores
>    packs in the MRU cache whose p->multi_pack_index bit is set.
> 
>  - builtin/pack-objects.c::want_object_in_pack_mtime(), which also
>    enumerates MIDX'd packs in a separate loop, though it does not
>    explicitly ignore packs in the MRU cache with the multi_pack_index
>    bit set.
> 
> In practice, though, I think these two are equivalent, since
> want_object_in_pack_mtime() will return before it gets to the MRU cache
> if it found the object in a MIDX'd pack.
> 
> So I don't think we need to be adding MIDX'd packs to the MRU cache in
> the first place.

I think the status quo is quite confusing. There are callers which directly iterate through the list of packfiles in MRU order, and that list is not guaranteed right now to even contain all packfiles that are tracked in the packfile store. The list is complete when we only load packfiles from disk, but if we ever manually add a packfile to the store in-memory the list is not up-to-date anymore. I also couldn't find a reason for that distinction.

Despite being confusing, there's another motivation here as discussed with Peff in [1]: we can drop the distinction between the MRU list and the "normal" list altogether. Ensuring that all packfiles are always stored in the MRU is a prerequisite for that subsequent change. I already got a patch series pending that does this refactoring.

That being said, I wouldn't mind moving this change into that subsequent patch series, either. It doesn't really have a strong reason to exist yet, but once we remove the distinction between the two packfile lists we have a much stronger argument.

[1]: <20250820192008.GA1662788@coredump.intra.peff.net>
Patrick
Patrick Steinhardt· Aug 21, 2025, 07:39 UTC · re: Patrick Steinhardt · lore

[PATCH v2 12/16] packfile: introduce function to load and add packfiles

We have a recurring pattern where we essentially perform an upsert of a packfile in case it isn't yet known by the packfile store. The logic to do so is non-trivial as we have to reconstruct the packfile's key, check the map of packfiles, then create the new packfile and finally add it to the store.

Introduce a new function that does this dance for us. Refactor callsites to use it.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/fast-import.c |  4 ++--
 builtin/index-pack.c  | 10 +++-------
 midx.c                | 18 ++----------------
 packfile.c            | 44 +++++++++++++++++++++++++++++++-------------
 packfile.h            |  8 ++++++++
 5 files changed, 46 insertions(+), 38 deletions(-)
Show changes to 5 files +46 −38

builtin/fast-import.c, builtin/index-pack.c, midx.c, packfile.c, packfile.h

diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index e9d82b31c3..a26e79689d 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -897,11 +897,11 @@ static void end_packfile(void)
 		idx_name = keep_pack(create_index());
 
 		/* Register the packfile with core git's machinery. */
-		new_p = add_packed_git(pack_data->repo, idx_name, strlen(idx_name), 1);
+		new_p = packfile_store_load_pack(pack_data->repo->objects->packfiles,
+						 idx_name, 1);
 		if (!new_p)
 			die("core git rejected index %s", idx_name);
 		all_packs[pack_id] = new_p;
-		packfile_store_add_pack(the_repository->objects->packfiles, new_p);
 		free(idx_name);
 
 		/* Print the boundary */
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index ed490dfad4..2b78ba7fe4 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1640,13 +1640,9 @@ static void final(const char *final_pack_name, const char *curr_pack_name,
 	rename_tmp_packfile(&final_index_name, curr_index_name, &index_name,
 			    hash, "idx", 1);
 
-	if (do_fsck_object) {
-		struct packed_git *p;
-		p = add_packed_git(the_repository, final_index_name,
-				   strlen(final_index_name), 0);
-		if (p)
-			packfile_store_add_pack(the_repository->objects->packfiles, p);
-	}
+	if (do_fsck_object)
+		packfile_store_load_pack(the_repository->objects->packfiles,
+					 final_index_name, 0);
 
 	if (!from_stdin) {
 		printf("%s\n", hash_to_hex(hash));
diff --git a/midx.c b/midx.c
index 3cfe7884ad..d30feda019 100644
--- a/midx.c
+++ b/midx.c
@@ -454,7 +454,6 @@ int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,
 		      uint32_t pack_int_id)
 {
 	struct strbuf pack_name = STRBUF_INIT;
-	struct strbuf key = STRBUF_INIT;
 	struct packed_git *p;
 
 	pack_int_id = midx_for_pack(&m, pack_int_id);
@@ -466,22 +465,9 @@ int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,
 
 	strbuf_addf(&pack_name, "%s/pack/%s", m->object_dir,
 		    m->pack_names[pack_int_id]);
-
-	/* pack_map holds the ".pack" name, but we have the .idx */
-	strbuf_addbuf(&key, &pack_name);
-	strbuf_strip_suffix(&key, ".idx");
-	strbuf_addstr(&key, ".pack");
-	p = hashmap_get_entry_from_hash(&r->objects->packfiles->map,
-					strhash(key.buf), key.buf,
-					struct packed_git, packmap_ent);
-	if (!p) {
-		p = add_packed_git(r, pack_name.buf, pack_name.len, m->local);
-		if (p)
-			packfile_store_add_pack(r->objects->packfiles, p);
-	}
-
+	p = packfile_store_load_pack(r->objects->packfiles,
+				     pack_name.buf, m->local);
 	strbuf_release(&pack_name);
-	strbuf_release(&key);
 
 	if (!p) {
 		m->packs[pack_int_id] = MIDX_PACK_ERROR;
diff --git a/packfile.c b/packfile.c
index a79d0fc1fa..f7a9967c9d 100644
--- a/packfile.c
+++ b/packfile.c
@@ -793,6 +793,33 @@ void packfile_store_add_pack(struct packfile_store *store,
 	list_add_tail(&pack->mru, &store->mru);
 }
 
+struct packed_git *packfile_store_load_pack(struct packfile_store *store,
+					    const char *idx_path, int local)
+{
+	struct strbuf key = STRBUF_INIT;
+	struct packed_git *p;
+
+	/*
+	 * We're being called with the path to the index file, but `pack_map`
+	 * holds the path to the packfile itself.
+	 */
+	strbuf_addstr(&key, idx_path);
+	strbuf_strip_suffix(&key, ".idx");
+	strbuf_addstr(&key, ".pack");
+
+	p = hashmap_get_entry_from_hash(&store->map, strhash(key.buf), key.buf,
+					struct packed_git, packmap_ent);
+	if (!p) {
+		p = add_packed_git(store->odb->repo, idx_path,
+				   strlen(idx_path), local);
+		if (p)
+			packfile_store_add_pack(store, p);
+	}
+
+	strbuf_release(&key);
+	return p;
+}
+
 void (*report_garbage)(unsigned seen_bits, const char *path);
 
 static void report_helper(const struct string_list *list,
@@ -892,23 +919,14 @@ static void prepare_pack(const char *full_name, size_t full_name_len,
 			 const char *file_name, void *_data)
 {
 	struct prepare_pack_data *data = (struct prepare_pack_data *)_data;
-	struct packed_git *p;
 	size_t base_len = full_name_len;
 
 	if (strip_suffix_mem(full_name, &base_len, ".idx") &&
 	    !(data->m && midx_contains_pack(data->m, file_name))) {
-		struct hashmap_entry hent;
-		char *pack_name = xstrfmt("%.*s.pack", (int)base_len, full_name);
-		unsigned int hash = strhash(pack_name);
-		hashmap_entry_init(&hent, hash);
-
-		/* Don't reopen a pack we already have. */
-		if (!hashmap_get(&data->r->objects->packfiles->map, &hent, pack_name)) {
-			p = add_packed_git(data->r, full_name, full_name_len, data->local);
-			if (p)
-				packfile_store_add_pack(data->r->objects->packfiles, p);
-		}
-		free(pack_name);
+		char *trimmed_path = xstrndup(full_name, full_name_len);
+		packfile_store_load_pack(data->r->objects->packfiles,
+					 trimmed_path, data->local);
+		free(trimmed_path);
 	}
 
 	if (!report_garbage)
diff --git a/packfile.h b/packfile.h
index 6641238796..c4e5516f9e 100644
--- a/packfile.h
+++ b/packfile.h
@@ -118,6 +118,14 @@ void packfile_store_reprepare(struct packfile_store *store);
 void packfile_store_add_pack(struct packfile_store *store,
 			     struct packed_git *pack);
 
+/*
+ * Open the packfile and add it to the store if it isn't yet known. Returns
+ * either the newly opened packfile or the preexisting packfile. Returns a
+ * `NULL` pointer in case the packfile could not be opened.
+ */
+struct packed_git *packfile_store_load_pack(struct packfile_store *store,
+					    const char *idx_path, int local);
+
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
-- 
2.51.0.261.g7ce5a0a67e.dirty
Taylor Blau· Aug 27, 2025, 01:12 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v2 12/16] packfile: introduce function to load and add packfiles

On Thu, Aug 21, 2025 at 09:39:10AM +0200, Patrick Steinhardt wrote:
Show 8 quoted lines
> We have a recurring pattern where we essentially perform an upsert of a
> packfile in case it isn't yet known by the packfile store. The logic to
> do so is non-trivial as we have to reconstruct the packfile's key, check
> the map of packfiles, then create the new packfile and finally add it to
> the store.
>
> Introduce a new function that does this dance for us. Refactor callsites
> to use it.

Nice, I have definitely noticed this pattern before and thought it would be nice to DRY it up a bit, but never got around to doing so ;-).

Show 24 quoted lines
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  builtin/fast-import.c |  4 ++--
>  builtin/index-pack.c  | 10 +++-------
>  midx.c                | 18 ++----------------
>  packfile.c            | 44 +++++++++++++++++++++++++++++++-------------
>  packfile.h            |  8 ++++++++
>  5 files changed, 46 insertions(+), 38 deletions(-)
>
> diff --git a/builtin/fast-import.c b/builtin/fast-import.c
> index e9d82b31c3..a26e79689d 100644
> --- a/builtin/fast-import.c
> +++ b/builtin/fast-import.c
> @@ -897,11 +897,11 @@ static void end_packfile(void)
>  		idx_name = keep_pack(create_index());
>
>  		/* Register the packfile with core git's machinery. */
> -		new_p = add_packed_git(pack_data->repo, idx_name, strlen(idx_name), 1);
> +		new_p = packfile_store_load_pack(pack_data->repo->objects->packfiles,
> +						 idx_name, 1);
>  		if (!new_p)
>  			die("core git rejected index %s", idx_name);
>  		all_packs[pack_id] = new_p;
> -		packfile_store_add_pack(the_repository->objects->packfiles, new_p);

OK, we can now avoid calling packfile_store_add_pack() explicitly here, since that is part of the new packfile_store_load_pack() function which is called a few lines up. That does change the order of operations a little bit (previously the new pack would end up in 'all_packs' first before being installed, now it's the other way around), but not in a way that I think matters.

Show 18 quoted lines
> diff --git a/builtin/index-pack.c b/builtin/index-pack.c
> index ed490dfad4..2b78ba7fe4 100644
> --- a/builtin/index-pack.c
> +++ b/builtin/index-pack.c
> @@ -1640,13 +1640,9 @@ static void final(const char *final_pack_name, const char *curr_pack_name,
>  	rename_tmp_packfile(&final_index_name, curr_index_name, &index_name,
>  			    hash, "idx", 1);
>
> -	if (do_fsck_object) {
> -		struct packed_git *p;
> -		p = add_packed_git(the_repository, final_index_name,
> -				   strlen(final_index_name), 0);
> -		if (p)
> -			packfile_store_add_pack(the_repository->objects->packfiles, p);
> -	}
> +	if (do_fsck_object)
> +		packfile_store_load_pack(the_repository->objects->packfiles,
> +					 final_index_name, 0);
Looks obviously correct to me.
Show 32 quoted lines
> diff --git a/midx.c b/midx.c
> index 3cfe7884ad..d30feda019 100644
> --- a/midx.c
> +++ b/midx.c
> @@ -454,7 +454,6 @@ int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,
>  		      uint32_t pack_int_id)
>  {
>  	struct strbuf pack_name = STRBUF_INIT;
> -	struct strbuf key = STRBUF_INIT;
>  	struct packed_git *p;
>
>  	pack_int_id = midx_for_pack(&m, pack_int_id);
> @@ -466,22 +465,9 @@ int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,
>
>  	strbuf_addf(&pack_name, "%s/pack/%s", m->object_dir,
>  		    m->pack_names[pack_int_id]);
> -
> -	/* pack_map holds the ".pack" name, but we have the .idx */
> -	strbuf_addbuf(&key, &pack_name);
> -	strbuf_strip_suffix(&key, ".idx");
> -	strbuf_addstr(&key, ".pack");
> -	p = hashmap_get_entry_from_hash(&r->objects->packfiles->map,
> -					strhash(key.buf), key.buf,
> -					struct packed_git, packmap_ent);
> -	if (!p) {
> -		p = add_packed_git(r, pack_name.buf, pack_name.len, m->local);
> -		if (p)
> -			packfile_store_add_pack(r->objects->packfiles, p);
> -	}
> -
> +	p = packfile_store_load_pack(r->objects->packfiles,
> +				     pack_name.buf, m->local);

Nice. This all looks like it preserves the right behavior, and it's nice to see the "we have a thing that ends in '.pack', but we need one that ends in '.idx'" logic get inlined, too.

Show 35 quoted lines
> diff --git a/packfile.c b/packfile.c
> index a79d0fc1fa..f7a9967c9d 100644
> --- a/packfile.c
> +++ b/packfile.c
> @@ -793,6 +793,33 @@ void packfile_store_add_pack(struct packfile_store *store,
>  	list_add_tail(&pack->mru, &store->mru);
>  }
>
> +struct packed_git *packfile_store_load_pack(struct packfile_store *store,
> +					    const char *idx_path, int local)
> +{
> +	struct strbuf key = STRBUF_INIT;
> +	struct packed_git *p;
> +
> +	/*
> +	 * We're being called with the path to the index file, but `pack_map`
> +	 * holds the path to the packfile itself.
> +	 */
> +	strbuf_addstr(&key, idx_path);
> +	strbuf_strip_suffix(&key, ".idx");
> +	strbuf_addstr(&key, ".pack");
> +
> +	p = hashmap_get_entry_from_hash(&store->map, strhash(key.buf), key.buf,
> +					struct packed_git, packmap_ent);
> +	if (!p) {
> +		p = add_packed_git(store->odb->repo, idx_path,
> +				   strlen(idx_path), local);
> +		if (p)
> +			packfile_store_add_pack(store, p);
> +	}
> +
> +	strbuf_release(&key);
> +	return p;
> +}
> +

This all looks good too, and matches the behavior of the callees which are being refactored.

Show 27 quoted lines
>  void (*report_garbage)(unsigned seen_bits, const char *path);
>
>  static void report_helper(const struct string_list *list,
> @@ -892,23 +919,14 @@ static void prepare_pack(const char *full_name, size_t full_name_len,
>  			 const char *file_name, void *_data)
>  {
>  	struct prepare_pack_data *data = (struct prepare_pack_data *)_data;
> -	struct packed_git *p;
>  	size_t base_len = full_name_len;
>
>  	if (strip_suffix_mem(full_name, &base_len, ".idx") &&
>  	    !(data->m && midx_contains_pack(data->m, file_name))) {
> -		struct hashmap_entry hent;
> -		char *pack_name = xstrfmt("%.*s.pack", (int)base_len, full_name);
> -		unsigned int hash = strhash(pack_name);
> -		hashmap_entry_init(&hent, hash);
> -
> -		/* Don't reopen a pack we already have. */
> -		if (!hashmap_get(&data->r->objects->packfiles->map, &hent, pack_name)) {
> -			p = add_packed_git(data->r, full_name, full_name_len, data->local);
> -			if (p)
> -				packfile_store_add_pack(data->r->objects->packfiles, p);
> -		}
> -		free(pack_name);
> +		char *trimmed_path = xstrndup(full_name, full_name_len);
> +		packfile_store_load_pack(data->r->objects->packfiles,
> +					 trimmed_path, data->local);

I think we could avoid the allocation here by passing along the length of the string we want to use, as in:

    packfile_store_load_pack(data->r->objects->packfiles,
                             full_name, full_name_len,
                             data->local);
, but I prefer the way it is written here.

Thanks, Taylor

Patrick Steinhardt· Aug 21, 2025, 07:39 UTC · re: Patrick Steinhardt · lore

[PATCH v2 13/16] packfile: move `get_multi_pack_index()` into "midx.c"

The `get_multi_pack_index()` function is declared and implemented in the packfile subsystem, even though it really belongs into the multi-pack index subsystem. The reason for this is likely that it needs to call `packfile_store_prepare()`, which is not exposed by the packfile system. In a subsequent commit we're about to add another caller outside of the packfile system though, so we'll have to expose the function anyway.

Do so now already and move `get_multi_pack_index()` into the MIDX subsystem.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 midx.c     |  6 ++++++
 midx.h     |  2 ++
 packfile.c |  8 +-------
 packfile.h | 10 +++++++++-
 4 files changed, 18 insertions(+), 8 deletions(-)
Show changes to 4 files +18 −8

midx.c, midx.h, packfile.c, packfile.h

diff --git a/midx.c b/midx.c
index d30feda019..c1b2f141fa 100644
--- a/midx.c
+++ b/midx.c
@@ -95,6 +95,12 @@ static int midx_read_object_offsets(const unsigned char *chunk_start,
 	return 0;
 }
 
+struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
+{
+	packfile_store_prepare(source->odb->packfiles);
+	return source->midx;
+}
+
 static struct multi_pack_index *load_multi_pack_index_one(struct repository *r,
 							  const char *object_dir,
 							  const char *midx_name,
diff --git a/midx.h b/midx.h
index 076382de8a..8d6ea28682 100644
--- a/midx.h
+++ b/midx.h
@@ -100,6 +100,8 @@ void get_split_midx_filename_ext(const struct git_hash_algo *hash_algo,
 				 struct strbuf *buf, const char *object_dir,
 				 const unsigned char *hash, const char *ext);
 
+struct multi_pack_index *get_multi_pack_index(struct odb_source *source);
+
 struct multi_pack_index *load_multi_pack_index(struct repository *r,
 					       const char *object_dir,
 					       int local);
diff --git a/packfile.c b/packfile.c
index f7a9967c9d..16384e0865 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1004,7 +1004,7 @@ static void packfile_store_prepare_mru(struct packfile_store *store)
 		list_add_tail(&p->mru, &store->mru);
 }
 
-static void packfile_store_prepare(struct packfile_store *store)
+void packfile_store_prepare(struct packfile_store *store)
 {
 	struct odb_source *source;
 
@@ -1035,12 +1035,6 @@ struct packed_git *get_packed_git(struct repository *r)
 	return r->objects->packfiles->packs;
 }
 
-struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
-{
-	packfile_store_prepare(source->odb->packfiles);
-	return source->midx;
-}
-
 struct packed_git *get_all_packs(struct repository *r)
 {
 	packfile_store_prepare(r->objects->packfiles);
diff --git a/packfile.h b/packfile.h
index c4e5516f9e..816b762770 100644
--- a/packfile.h
+++ b/packfile.h
@@ -103,6 +103,15 @@ void packfile_store_free(struct packfile_store *store);
  */
 void packfile_store_close(struct packfile_store *store);
 
+/*
+ * Prepare the packfile store by loading packfiles and multi-pack indices for
+ * all alternates. This becomes a no-op if the store is already prepared.
+ *
+ * It shouldn't typically be necessary to call this function directly, as
+ * functions that access the store know to prepare it.
+ */
+void packfile_store_prepare(struct packfile_store *store);
+
 /*
  * Clear the packfile caches and try to look up any new packfiles that have
  * appeared since last preparing the packfiles store.
@@ -204,7 +213,6 @@ extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
 struct packed_git *get_packed_git(struct repository *r);
 struct list_head *get_packed_git_mru(struct repository *r);
-struct multi_pack_index *get_multi_pack_index(struct odb_source *source);
 struct packed_git *get_all_packs(struct repository *r);
 
 /*
-- 
2.51.0.261.g7ce5a0a67e.dirty
Taylor Blau· Aug 27, 2025, 01:20 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v2 13/16] packfile: move `get_multi_pack_index()` into "midx.c"

On Thu, Aug 21, 2025 at 09:39:11AM +0200, Patrick Steinhardt wrote:
Show 9 quoted lines
> The `get_multi_pack_index()` function is declared and implemented in the
> packfile subsystem, even though it really belongs into the multi-pack
> index subsystem. The reason for this is likely that it needs to call
> `packfile_store_prepare()`, which is not exposed by the packfile system.
> In a subsequent commit we're about to add another caller outside of the
> packfile system though, so we'll have to expose the function anyway.
>
> Do so now already and move `get_multi_pack_index()` into the MIDX
> subsystem.
Makes sense.
Show 7 quoted lines
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  midx.c     |  6 ++++++
>  midx.h     |  2 ++
>  packfile.c |  8 +-------
>  packfile.h | 10 +++++++++-
>  4 files changed, 18 insertions(+), 8 deletions(-)
And all looks good here.

Thanks, Taylor

Patrick Steinhardt· Aug 21, 2025, 07:39 UTC · re: Patrick Steinhardt · lore

[PATCH v2 14/16] packfile: remove `get_packed_git()`

We have two different functions to retrieve packfiles for a packfile store:

  - `get_packed_git()` returns the list of packfiles after having called
    `prepare_packed_git()`.
  - `get_all_packs()` calls `prepare_packed_git()`, as well, but also
    calls `prepare_midx_pack()` for each pack.

This means that the latter function also properly loads the info of whether or not a packfile is part of a multi-pack index. Preparing this extra information also shouldn't be significantly more expensive:

  - We have already loaded all packfiles via `prepare_packed_git_one()`.
    So given that multi-pack indices may only refer to packfiles in the
    same object directory we know that we already loaded each packfile.
  - The multi-pack index was prepared via `packfile_store_prepare()`
    already, which calls `prepare_multi_pack_index_one()`.
  - So all that remains to be done is to look up the index of the pack
    in its multi-pack index so that we can store that info in both the
    pack itself and the MIDX.

So it is somewhat confusing to readers that one of these two functions claims to load "all" packfiles while the other one doesn't, even though the ultimate difference is way more nuanced.

Convert all of these sites to use `get_all_packs()` instead and remove `get_packed_git()`. There doesn't seem to be a good reason to discern these two functions.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/gc.c   | 2 +-
 builtin/grep.c | 2 +-
 object-name.c  | 4 ++--
 packfile.c     | 6 ------
 packfile.h     | 1 -
 5 files changed, 4 insertions(+), 11 deletions(-)
Show changes to 5 files +4 −11

builtin/gc.c, builtin/grep.c, object-name.c, packfile.c, packfile.h

diff --git a/builtin/gc.c b/builtin/gc.c
index 1d30d1af2c..565afda51f 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1422,7 +1422,7 @@ static int incremental_repack_auto_condition(struct gc_config *cfg UNUSED)
 	if (incremental_repack_auto_limit < 0)
 		return 1;
 
-	for (p = get_packed_git(the_repository);
+	for (p = get_all_packs(the_repository);
 	     count < incremental_repack_auto_limit && p;
 	     p = p->next) {
 		if (!p->multi_pack_index)
diff --git a/builtin/grep.c b/builtin/grep.c
index 5df6537333..8f0e21bd70 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -1214,7 +1214,7 @@ int cmd_grep(int argc,
 		if (recurse_submodules)
 			repo_read_gitmodules(the_repository, 1);
 		if (startup_info->have_repository)
-			(void)get_packed_git(the_repository);
+			packfile_store_prepare(the_repository->objects->packfiles);
 
 		start_threads(&opt);
 	} else {
diff --git a/object-name.c b/object-name.c
index 44b0d416ac..c87995cc1e 100644
--- a/object-name.c
+++ b/object-name.c
@@ -213,7 +213,7 @@ static void find_short_packed_object(struct disambiguate_state *ds)
 			unique_in_midx(m, ds);
 	}
 
-	for (p = get_packed_git(ds->repo); p && !ds->ambiguous;
+	for (p = get_all_packs(ds->repo); p && !ds->ambiguous;
 	     p = p->next)
 		unique_in_pack(p, ds);
 }
@@ -806,7 +806,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)
 			find_abbrev_len_for_midx(m, mad);
 	}
 
-	for (p = get_packed_git(mad->repo); p; p = p->next)
+	for (p = get_all_packs(mad->repo); p; p = p->next)
 		find_abbrev_len_for_pack(p, mad);
 }
 
diff --git a/packfile.c b/packfile.c
index 16384e0865..523c30c5a7 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1029,12 +1029,6 @@ void packfile_store_reprepare(struct packfile_store *store)
 	packfile_store_prepare(store);
 }
 
-struct packed_git *get_packed_git(struct repository *r)
-{
-	packfile_store_prepare(r->objects->packfiles);
-	return r->objects->packfiles->packs;
-}
-
 struct packed_git *get_all_packs(struct repository *r)
 {
 	packfile_store_prepare(r->objects->packfiles);
diff --git a/packfile.h b/packfile.h
index 816b762770..15cb378781 100644
--- a/packfile.h
+++ b/packfile.h
@@ -211,7 +211,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-struct packed_git *get_packed_git(struct repository *r);
 struct list_head *get_packed_git_mru(struct repository *r);
 struct packed_git *get_all_packs(struct repository *r);
 
-- 
2.51.0.261.g7ce5a0a67e.dirty
Taylor Blau· Aug 27, 2025, 01:38 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v2 14/16] packfile: remove `get_packed_git()`

On Thu, Aug 21, 2025 at 09:39:12AM +0200, Patrick Steinhardt wrote:
Show 8 quoted lines
> We have two different functions to retrieve packfiles for a packfile
> store:
>
>   - `get_packed_git()` returns the list of packfiles after having called
>     `prepare_packed_git()`.
>
>   - `get_all_packs()` calls `prepare_packed_git()`, as well, but also
>     calls `prepare_midx_pack()` for each pack.

Yeah, having two of these functions that are named so similarly as to suggest they do the same thing (even though they don't) is unfortunate, and I am glad that we are looking at it here.

> This means that the latter function also properly loads the info of
> whether or not a packfile is part of a multi-pack index. Preparing this
> extra information also shouldn't be significantly more expensive:

Right; get_packed_git() only loads the non-MIDX'd packs, and get_all_packs() loads everything (regardless whether or not a pack is part of the MIDX or not).

Are all of the get_packed_git() callers prepared to handle packs that are in the MIDX? Looking through them:

 - builtin/gc.c::incremental_repack_auto_condition() skips over
  'p->multi_pack_index', so this one is fine to convert.
 - builtin/grep.c::cmd_grep() calls get_packed_git() but doesn't
   actually use the result, so this should be fine to convert, though I
   think there is some subtlty here.
 - builtin/pack-objects.c::want_object_in_pack_mtime() takes a separate
   pass over the MIDX'd packs before calling get_packed_git_mru() (which
   itself calls prepare_packed_git()). I think in practice this is OK,
   since we will have already handled the MIDX'd packs, but this
   function is now iterating over packs in the MIDX twice, so it may be
   worth adding a "if (p->multi_pack_index) continue;" in there.
 - object-name.c::find_short_packed_object() handles MIDX'd packs
   separately, and unique_in_pack() is a noop for MIDX'd packs, so this
   one is fine.
 - object-name.c::find_abbrev_len_packed() is OK for the same reasons.

So I think that want_object_in_pack_mtime() may need a small tweak, and I am not 100% certain that cmd_grep() is OK to convert.

Show 10 quoted lines
>   - We have already loaded all packfiles via `prepare_packed_git_one()`.
>     So given that multi-pack indices may only refer to packfiles in the
>     same object directory we know that we already loaded each packfile.
>
>   - The multi-pack index was prepared via `packfile_store_prepare()`
>     already, which calls `prepare_multi_pack_index_one()`.
>
>   - So all that remains to be done is to look up the index of the pack
>     in its multi-pack index so that we can store that info in both the
>     pack itself and the MIDX.

This clearly shows that get_all_packs() is not meaningfully more expensive than get_packed_git(), but I think that may be obscuring some of the details above on why it is OK (or not) to transition these calls over.

Show 7 quoted lines
> So it is somewhat confusing to readers that one of these two functions
> claims to load "all" packfiles while the other one doesn't, even though
> the ultimate difference is way more nuanced.
>
> Convert all of these sites to use `get_all_packs()` instead and remove
> `get_packed_git()`. There doesn't seem to be a good reason to discern
> these two functions.

The last sentence here threw me off a bit. I think the patch message would benefit from some of the reasoning above about why it is OK to transition callers over from one function to the other.

As an aside, I am not convinced that having one caller (in pack-objects) that cares about whether or not it sees a MIDX'd pack is that great of a reason to have separate functions. But I am also not convinced that it isn't either. If we kept both, I think they would benefit from having more distinct names, like get_all_packs() and get_non_midx_packs() or something.

Thanks, Taylor

Patrick Steinhardt· Sep 2, 2025, 08:50 UTC · re: Taylor Blau · lore

Re: [PATCH v2 14/16] packfile: remove `get_packed_git()`

On Tue, Aug 26, 2025 at 09:38:47PM -0400, Taylor Blau wrote:
Show 21 quoted lines
> On Thu, Aug 21, 2025 at 09:39:12AM +0200, Patrick Steinhardt wrote:
> > We have two different functions to retrieve packfiles for a packfile
> > store:
> >
> >   - `get_packed_git()` returns the list of packfiles after having called
> >     `prepare_packed_git()`.
> >
> >   - `get_all_packs()` calls `prepare_packed_git()`, as well, but also
> >     calls `prepare_midx_pack()` for each pack.
> 
> Yeah, having two of these functions that are named so similarly as to
> suggest they do the same thing (even though they don't) is unfortunate,
> and I am glad that we are looking at it here.
> 
> > This means that the latter function also properly loads the info of
> > whether or not a packfile is part of a multi-pack index. Preparing this
> > extra information also shouldn't be significantly more expensive:
> 
> Right; get_packed_git() only loads the non-MIDX'd packs, and
> get_all_packs() loads everything (regardless whether or not a pack is
> part of the MIDX or not).

I initially understood the distinction of these functions to be exactly this. But after looking further I don't think this is the actual distinction: both functions end up loading all packfiles in the repo, with the only distinction being that `get_all_packs()` also prepares the MIDX for each MIDX'd packfile.

The important thing to note is that an MIDX may only ever refer to packfiles in the same object directory. And as we have already loaded all packfiles in that object directory via `prepare_packed_git_one()` there isn't really any difference in the returned list of packfiles whatsoever. Both functions end up loading all packfiles, regardless of whether or not they have an MIDX.

Please double check my understanding though, I have been staring at this code for quite a while to figure out what the actual differences are, and whether or not `get_all_packs()` may actually end up loading more packs than `get_packed_git()`.

Show 25 quoted lines
> Are all of the get_packed_git() callers prepared to handle packs that
> are in the MIDX? Looking through them:
> 
>  - builtin/gc.c::incremental_repack_auto_condition() skips over
>   'p->multi_pack_index', so this one is fine to convert.
> 
>  - builtin/grep.c::cmd_grep() calls get_packed_git() but doesn't
>    actually use the result, so this should be fine to convert, though I
>    think there is some subtlty here.
> 
>  - builtin/pack-objects.c::want_object_in_pack_mtime() takes a separate
>    pass over the MIDX'd packs before calling get_packed_git_mru() (which
>    itself calls prepare_packed_git()). I think in practice this is OK,
>    since we will have already handled the MIDX'd packs, but this
>    function is now iterating over packs in the MIDX twice, so it may be
>    worth adding a "if (p->multi_pack_index) continue;" in there.
> 
>  - object-name.c::find_short_packed_object() handles MIDX'd packs
>    separately, and unique_in_pack() is a noop for MIDX'd packs, so this
>    one is fine.
> 
>  - object-name.c::find_abbrev_len_packed() is OK for the same reasons.
> 
> So I think that want_object_in_pack_mtime() may need a small tweak, and
> I am not 100% certain that cmd_grep() is OK to convert.

I initially misunderstood the distinction between these two functions the same as you did, and had a similar list to the above in the initial commit message. But with the adjusted understanding of the actual difference between these functions I think it shouldn't be necessary anymore to go through each caller one by one.

I'll adapt the commit message a bit.
Patrick
Taylor Blau· Sep 11, 2025, 23:25 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v2 14/16] packfile: remove `get_packed_git()`

On Tue, Sep 02, 2025 at 10:50:54AM +0200, Patrick Steinhardt wrote:
Show 28 quoted lines
> On Tue, Aug 26, 2025 at 09:38:47PM -0400, Taylor Blau wrote:
> > On Thu, Aug 21, 2025 at 09:39:12AM +0200, Patrick Steinhardt wrote:
> > > We have two different functions to retrieve packfiles for a packfile
> > > store:
> > >
> > >   - `get_packed_git()` returns the list of packfiles after having called
> > >     `prepare_packed_git()`.
> > >
> > >   - `get_all_packs()` calls `prepare_packed_git()`, as well, but also
> > >     calls `prepare_midx_pack()` for each pack.
> >
> > Yeah, having two of these functions that are named so similarly as to
> > suggest they do the same thing (even though they don't) is unfortunate,
> > and I am glad that we are looking at it here.
> >
> > > This means that the latter function also properly loads the info of
> > > whether or not a packfile is part of a multi-pack index. Preparing this
> > > extra information also shouldn't be significantly more expensive:
> >
> > Right; get_packed_git() only loads the non-MIDX'd packs, and
> > get_all_packs() loads everything (regardless whether or not a pack is
> > part of the MIDX or not).
>
> I initially understood the distinction of these functions to be exactly
> this. But after looking further I don't think this is the actual
> distinction: both functions end up loading all packfiles in the repo,
> with the only distinction being that `get_all_packs()` also prepares the
> MIDX for each MIDX'd packfile.
Calling both get_packed_git() and get_all_packs() both result in:
 - prepare_packed_git(), which calls
 - prepare_packed_git_one(), which calls
 - for_each_file_in_pack_dir(), which calls (via callback)
 - prepare_pack

prepare_pack() only calls add_packed_git() and install_packed_git() if the file it's look at ends in "*.idx", *and* there is either no MIDX, or (if there is one) the packfile is not listed in the MIDX.

get_all_packs(), on the other hand, *does* call prepare_midx_pack() on all MIDXs across all ODB sources, after having called prepare_packed_git(). And prepare_midx_pack() will add MIDX'd packs to the global list of packs, which is what differentiates these two calls.

I think this gets a little funky in practice if you have already called prepare_midx_pack() on one or more MIDX'd packfiles before calling get_packed_git(), because of the side effect of prepare_midx_pack() installing packs into the global list.

So the above behavior isn't guaranteed, but there definitely is a difference between those two functions depending on whether or not some other site has called prepare_midx_pack().

Show 8 quoted lines
> > So I think that want_object_in_pack_mtime() may need a small tweak, and
> > I am not 100% certain that cmd_grep() is OK to convert.
>
> I initially misunderstood the distinction between these two functions
> the same as you did, and had a similar list to the above in the initial
> commit message. But with the adjusted understanding of the actual
> difference between these functions I think it shouldn't be necessary
> anymore to go through each caller one by one.
From the above, I am not sure that this is true.

Thanks, Taylor

Patrick Steinhardt· Sep 15, 2025, 07:30 UTC · re: Taylor Blau · lore

Re: [PATCH v2 14/16] packfile: remove `get_packed_git()`

On Thu, Sep 11, 2025 at 07:25:59PM -0400, Taylor Blau wrote:
Show 65 quoted lines
> On Tue, Sep 02, 2025 at 10:50:54AM +0200, Patrick Steinhardt wrote:
> > On Tue, Aug 26, 2025 at 09:38:47PM -0400, Taylor Blau wrote:
> > > On Thu, Aug 21, 2025 at 09:39:12AM +0200, Patrick Steinhardt wrote:
> > > > We have two different functions to retrieve packfiles for a packfile
> > > > store:
> > > >
> > > >   - `get_packed_git()` returns the list of packfiles after having called
> > > >     `prepare_packed_git()`.
> > > >
> > > >   - `get_all_packs()` calls `prepare_packed_git()`, as well, but also
> > > >     calls `prepare_midx_pack()` for each pack.
> > >
> > > Yeah, having two of these functions that are named so similarly as to
> > > suggest they do the same thing (even though they don't) is unfortunate,
> > > and I am glad that we are looking at it here.
> > >
> > > > This means that the latter function also properly loads the info of
> > > > whether or not a packfile is part of a multi-pack index. Preparing this
> > > > extra information also shouldn't be significantly more expensive:
> > >
> > > Right; get_packed_git() only loads the non-MIDX'd packs, and
> > > get_all_packs() loads everything (regardless whether or not a pack is
> > > part of the MIDX or not).
> >
> > I initially understood the distinction of these functions to be exactly
> > this. But after looking further I don't think this is the actual
> > distinction: both functions end up loading all packfiles in the repo,
> > with the only distinction being that `get_all_packs()` also prepares the
> > MIDX for each MIDX'd packfile.
> 
> Calling both get_packed_git() and get_all_packs() both result in:
> 
>  - prepare_packed_git(), which calls
>  - prepare_packed_git_one(), which calls
>  - for_each_file_in_pack_dir(), which calls (via callback)
>  - prepare_pack
> 
> prepare_pack() only calls add_packed_git() and install_packed_git() if
> the file it's look at ends in "*.idx", *and* there is either no MIDX, or
> (if there is one) the packfile is not listed in the MIDX.
> 
> get_all_packs(), on the other hand, *does* call prepare_midx_pack() on
> all MIDXs across all ODB sources, after having called
> prepare_packed_git(). And prepare_midx_pack() will add MIDX'd packs to
> the global list of packs, which is what differentiates these two calls.
> 
> I think this gets a little funky in practice if you have already called
> prepare_midx_pack() on one or more MIDX'd packfiles before calling
> get_packed_git(), because of the side effect of prepare_midx_pack()
> installing packs into the global list.
> 
> So the above behavior isn't guaranteed, but there definitely is a
> difference between those two functions depending on whether or not some
> other site has called prepare_midx_pack().
> 
> > > So I think that want_object_in_pack_mtime() may need a small tweak, and
> > > I am not 100% certain that cmd_grep() is OK to convert.
> >
> > I initially misunderstood the distinction between these two functions
> > the same as you did, and had a similar list to the above in the initial
> > commit message. But with the adjusted understanding of the actual
> > difference between these functions I think it shouldn't be necessary
> > anymore to go through each caller one by one.
> 
> From the above, I am not sure that this is true.

Yeah, I'll drop that part from this patch series for now and will handle this in a subsequent series.

Patrick
Patrick Steinhardt· Aug 21, 2025, 07:39 UTC · re: Patrick Steinhardt · lore

[PATCH v2 15/16] packfile: refactor `get_all_packs()` to work on packfile store

The `get_all_packs()` function prepares the packfile store and then returns its packfiles. Refactor it to accept a packfile store instead of a repository to clarify its scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/cat-file.c          |  2 +-
 builtin/count-objects.c     |  2 +-
 builtin/fast-import.c       |  4 ++--
 builtin/fsck.c              |  8 ++++----
 builtin/gc.c                |  8 ++++----
 builtin/pack-objects.c      | 18 +++++++++---------
 builtin/pack-redundant.c    |  4 ++--
 builtin/repack.c            |  6 +++---
 connected.c                 |  2 +-
 http-backend.c              |  4 ++--
 http.c                      |  2 +-
 object-name.c               |  4 ++--
 pack-bitmap.c               |  4 ++--
 pack-objects.c              |  2 +-
 packfile.c                  | 14 +++++++-------
 packfile.h                  |  7 ++++++-
 server-info.c               |  2 +-
 t/helper/test-find-pack.c   |  2 +-
 t/helper/test-pack-mtimes.c |  2 +-
 19 files changed, 51 insertions(+), 46 deletions(-)
Show changes to 19 files +51 −46

builtin/cat-file.c, builtin/count-objects.c, builtin/fast-import.c, builtin/fsck.c, builtin/gc.c, builtin/pack-objects.c, builtin/pack-redundant.c, builtin/repack.c, connected.c, http-backend.c, http.c, object-name.c, pack-bitmap.c, pack-objects.c, packfile.c, packfile.h, server-info.c, t/helper/test-find-pack.c, t/helper/test-pack-mtimes.c

diff --git a/builtin/cat-file.c b/builtin/cat-file.c
index fce0b06451c..7124c43fb14 100644
--- a/builtin/cat-file.c
+++ b/builtin/cat-file.c
@@ -854,7 +854,7 @@ static void batch_each_object(struct batch_options *opt,
 						 batch_one_object_bitmapped, &payload)) {
 		struct packed_git *pack;
 
-		for (pack = get_all_packs(the_repository); pack; pack = pack->next) {
+		for (pack = packfile_store_get_packs(the_repository->objects->packfiles); pack; pack = pack->next) {
 			if (bitmap_index_contains_pack(bitmap, pack) ||
 			    open_pack_index(pack))
 				continue;
diff --git a/builtin/count-objects.c b/builtin/count-objects.c
index a61d3b46aac..471d96a3089 100644
--- a/builtin/count-objects.c
+++ b/builtin/count-objects.c
@@ -129,7 +129,7 @@ int cmd_count_objects(int argc,
 		struct strbuf pack_buf = STRBUF_INIT;
 		struct strbuf garbage_buf = STRBUF_INIT;
 
-		for (p = get_all_packs(the_repository); p; p = p->next) {
+		for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 			if (!p->pack_local)
 				continue;
 			if (open_pack_index(p))
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index a26e79689d5..4f355118a10 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -975,7 +975,7 @@ static int store_object(
 	if (e->idx.offset) {
 		duplicate_count_by_type[type]++;
 		return 1;
-	} else if (find_oid_pack(&oid, get_all_packs(the_repository))) {
+	} else if (find_oid_pack(&oid, packfile_store_get_packs(the_repository->objects->packfiles))) {
 		e->type = type;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
@@ -1175,7 +1175,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 		duplicate_count_by_type[OBJ_BLOB]++;
 		truncate_pack(&checkpoint);
 
-	} else if (find_oid_pack(&oid, get_all_packs(the_repository))) {
+	} else if (find_oid_pack(&oid, packfile_store_get_packs(the_repository->objects->packfiles))) {
 		e->type = OBJ_BLOB;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
diff --git a/builtin/fsck.c b/builtin/fsck.c
index 543a2cdb5cd..e867fd510a3 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -873,14 +873,14 @@ static int check_pack_rev_indexes(struct repository *r, int show_progress)
 	int res = 0;
 
 	if (show_progress) {
-		for (struct packed_git *p = get_all_packs(r); p; p = p->next)
+		for (struct packed_git *p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next)
 			pack_count++;
 		progress = start_delayed_progress(the_repository,
 						  "Verifying reverse pack-indexes", pack_count);
 		pack_count = 0;
 	}
 
-	for (struct packed_git *p = get_all_packs(r); p; p = p->next) {
+	for (struct packed_git *p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next) {
 		int load_error = load_pack_revindex_from_disk(p);
 
 		if (load_error < 0) {
@@ -1010,7 +1010,7 @@ int cmd_fsck(int argc,
 			struct progress *progress = NULL;
 
 			if (show_progress) {
-				for (p = get_all_packs(the_repository); p;
+				for (p = packfile_store_get_packs(the_repository->objects->packfiles); p;
 				     p = p->next) {
 					if (open_pack_index(p))
 						continue;
@@ -1020,7 +1020,7 @@ int cmd_fsck(int argc,
 				progress = start_progress(the_repository,
 							  _("Checking objects"), total);
 			}
-			for (p = get_all_packs(the_repository); p;
+			for (p = packfile_store_get_packs(the_repository->objects->packfiles); p;
 			     p = p->next) {
 				/* verify gives error messages itself */
 				if (verify_pack(the_repository,
diff --git a/builtin/gc.c b/builtin/gc.c
index 565afda51fe..030d0b0c774 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -488,7 +488,7 @@ static struct packed_git *find_base_packs(struct string_list *packs,
 {
 	struct packed_git *p, *base = NULL;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		if (!p->pack_local || p->is_cruft)
 			continue;
 		if (limit) {
@@ -513,7 +513,7 @@ static int too_many_packs(struct gc_config *cfg)
 	if (cfg->gc_auto_pack_limit <= 0)
 		return 0;
 
-	for (cnt = 0, p = get_all_packs(the_repository); p; p = p->next) {
+	for (cnt = 0, p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		if (!p->pack_local)
 			continue;
 		if (p->pack_keep)
@@ -1422,7 +1422,7 @@ static int incremental_repack_auto_condition(struct gc_config *cfg UNUSED)
 	if (incremental_repack_auto_limit < 0)
 		return 1;
 
-	for (p = get_all_packs(the_repository);
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles);
 	     count < incremental_repack_auto_limit && p;
 	     p = p->next) {
 		if (!p->multi_pack_index)
@@ -1491,7 +1491,7 @@ static off_t get_auto_pack_size(void)
 	struct repository *r = the_repository;
 
 	odb_reprepare(r->objects);
-	for (p = get_all_packs(r); p; p = p->next) {
+	for (p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next) {
 		if (p->pack_size > max_size) {
 			second_largest_size = max_size;
 			max_size = p->pack_size;
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 53a22562503..1c24b84510e 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -3855,7 +3855,7 @@ static void read_packs_list_from_stdin(struct rev_info *revs)
 	string_list_sort(&exclude_packs);
 	string_list_remove_duplicates(&exclude_packs, 0);
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		const char *pack_name = pack_basename(p);
 
 		if ((item = string_list_lookup(&include_packs, pack_name)))
@@ -4105,7 +4105,7 @@ static void enumerate_and_traverse_cruft_objects(struct string_list *fresh_packs
 	 * Re-mark only the fresh packs as kept so that objects in
 	 * unknown packs do not halt the reachability traversal early.
 	 */
-	for (p = get_all_packs(the_repository); p; p = p->next)
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next)
 		p->pack_keep_in_core = 0;
 	mark_pack_kept_in_core(fresh_packs, 1);
 
@@ -4142,7 +4142,7 @@ static void read_cruft_objects(void)
 	string_list_sort(&discard_packs);
 	string_list_sort(&fresh_packs);
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		const char *pack_name = pack_basename(p);
 		struct string_list_item *item;
 
@@ -4394,7 +4394,7 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 	struct packed_git *p;
 
 	p = (last_found != (void *)1) ? last_found :
-					get_all_packs(the_repository);
+					packfile_store_get_packs(the_repository->objects->packfiles);
 
 	while (p) {
 		if ((!p->pack_local || p->pack_keep ||
@@ -4404,7 +4404,7 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 			return 1;
 		}
 		if (p == last_found)
-			p = get_all_packs(the_repository);
+			p = packfile_store_get_packs(the_repository->objects->packfiles);
 		else
 			p = p->next;
 		if (p == last_found)
@@ -4441,7 +4441,7 @@ static void loosen_unused_packed_objects(void)
 	uint32_t loosened_objects_nr = 0;
 	struct object_id oid;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		if (!p->pack_local || p->pack_keep || p->pack_keep_in_core)
 			continue;
 
@@ -4747,7 +4747,7 @@ static void add_extra_kept_packs(const struct string_list *names)
 	if (!names->nr)
 		return;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		const char *name = basename(p->pack_name);
 		int i;
 
@@ -5186,7 +5186,7 @@ int cmd_pack_objects(int argc,
 	add_extra_kept_packs(&keep_pack_list);
 	if (ignore_packed_keep_on_disk) {
 		struct packed_git *p;
-		for (p = get_all_packs(the_repository); p; p = p->next)
+		for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next)
 			if (p->pack_local && p->pack_keep)
 				break;
 		if (!p) /* no keep-able packs found */
@@ -5199,7 +5199,7 @@ int cmd_pack_objects(int argc,
 		 * it also covers non-local objects
 		 */
 		struct packed_git *p;
-		for (p = get_all_packs(the_repository); p; p = p->next) {
+		for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 			if (!p->pack_local) {
 				have_non_local_packs = 1;
 				break;
diff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c
index fe81c293e3a..7b2cb3ef1e2 100644
--- a/builtin/pack-redundant.c
+++ b/builtin/pack-redundant.c
@@ -566,7 +566,7 @@ static struct pack_list * add_pack(struct packed_git *p)
 
 static struct pack_list * add_pack_file(const char *filename)
 {
-	struct packed_git *p = get_all_packs(the_repository);
+	struct packed_git *p = packfile_store_get_packs(the_repository->objects->packfiles);
 
 	if (strlen(filename) < 40)
 		die("Bad pack filename: %s", filename);
@@ -581,7 +581,7 @@ static struct pack_list * add_pack_file(const char *filename)
 
 static void load_all(void)
 {
-	struct packed_git *p = get_all_packs(the_repository);
+	struct packed_git *p = packfile_store_get_packs(the_repository->objects->packfiles);
 
 	while (p) {
 		add_pack(p);
diff --git a/builtin/repack.c b/builtin/repack.c
index ee8c80cd95c..6119e236512 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -267,7 +267,7 @@ static void collect_pack_filenames(struct existing_packs *existing,
 	struct packed_git *p;
 	struct strbuf buf = STRBUF_INIT;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		int i;
 		const char *base;
 
@@ -499,7 +499,7 @@ static void init_pack_geometry(struct pack_geometry *geometry,
 	struct packed_git *p;
 	struct strbuf buf = STRBUF_INIT;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		if (args->local && !p->pack_local)
 			/*
 			 * When asked to only repack local packfiles we skip
@@ -1140,7 +1140,7 @@ static void combine_small_cruft_packs(FILE *in, size_t combine_cruft_below_size,
 	struct strbuf buf = STRBUF_INIT;
 	size_t i;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		if (!(p->is_cruft && p->pack_local))
 			continue;
 
diff --git a/connected.c b/connected.c
index d6e9682fd93..d7e07fa6b0d 100644
--- a/connected.c
+++ b/connected.c
@@ -76,7 +76,7 @@ int check_connected(oid_iterate_fn fn, void *cb_data,
 		do {
 			struct packed_git *p;
 
-			for (p = get_all_packs(the_repository); p; p = p->next) {
+			for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 				if (!p->pack_promisor)
 					continue;
 				if (find_pack_entry_one(oid, p))
diff --git a/http-backend.c b/http-backend.c
index d5dfe762bb5..be4d8263a58 100644
--- a/http-backend.c
+++ b/http-backend.c
@@ -608,13 +608,13 @@ static void get_info_packs(struct strbuf *hdr, char *arg UNUSED)
 	size_t cnt = 0;
 
 	select_getanyfile(hdr);
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		if (p->pack_local)
 			cnt++;
 	}
 
 	strbuf_grow(&buf, cnt * 53 + 2);
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		if (p->pack_local)
 			strbuf_addf(&buf, "P %s\n", p->pack_name + objdirlen + 6);
 	}
diff --git a/http.c b/http.c
index af2120b64c7..16a1ab54f34 100644
--- a/http.c
+++ b/http.c
@@ -2416,7 +2416,7 @@ static int fetch_and_setup_pack_index(struct packed_git **packs_head,
 	 * If we already have the pack locally, no need to fetch its index or
 	 * even add it to list; we already have all of its objects.
 	 */
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		if (hasheq(p->hash, sha1, the_repository->hash_algo))
 			return 0;
 	}
diff --git a/object-name.c b/object-name.c
index c87995cc1e6..e346075394d 100644
--- a/object-name.c
+++ b/object-name.c
@@ -213,7 +213,7 @@ static void find_short_packed_object(struct disambiguate_state *ds)
 			unique_in_midx(m, ds);
 	}
 
-	for (p = get_all_packs(ds->repo); p && !ds->ambiguous;
+	for (p = packfile_store_get_packs(ds->repo->objects->packfiles); p && !ds->ambiguous;
 	     p = p->next)
 		unique_in_pack(p, ds);
 }
@@ -806,7 +806,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)
 			find_abbrev_len_for_midx(m, mad);
 	}
 
-	for (p = get_all_packs(mad->repo); p; p = p->next)
+	for (p = packfile_store_get_packs(mad->repo->objects->packfiles); p; p = p->next)
 		find_abbrev_len_for_pack(p, mad);
 }
 
diff --git a/pack-bitmap.c b/pack-bitmap.c
index d14421ee204..67f9e92ec18 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -665,7 +665,7 @@ static int open_pack_bitmap(struct repository *r,
 	struct packed_git *p;
 	int ret = -1;
 
-	for (p = get_all_packs(r); p; p = p->next) {
+	for (p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next) {
 		if (open_pack_bitmap_1(bitmap_git, p) == 0) {
 			ret = 0;
 			/*
@@ -3363,7 +3363,7 @@ int verify_bitmap_files(struct repository *r)
 		free(midx_bitmap_name);
 	}
 
-	for (struct packed_git *p = get_all_packs(r);
+	for (struct packed_git *p = packfile_store_get_packs(r->objects->packfiles);
 	     p; p = p->next) {
 		char *pack_bitmap_name = pack_bitmap_filename(p);
 		res |= verify_bitmap_file(r->hash_algo, pack_bitmap_name);
diff --git a/pack-objects.c b/pack-objects.c
index a9d9855063a..5506f12293c 100644
--- a/pack-objects.c
+++ b/pack-objects.c
@@ -95,7 +95,7 @@ static void prepare_in_pack_by_idx(struct packing_data *pdata)
 	 * (i.e. in_pack_idx also zero) should return NULL.
 	 */
 	mapping[cnt++] = NULL;
-	for (p = get_all_packs(pdata->repo); p; p = p->next, cnt++) {
+	for (p = packfile_store_get_packs(pdata->repo->objects->packfiles); p; p = p->next, cnt++) {
 		if (cnt == nr) {
 			free(mapping);
 			return;
diff --git a/packfile.c b/packfile.c
index 523c30c5a71..19227ea0b3c 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1029,19 +1029,19 @@ void packfile_store_reprepare(struct packfile_store *store)
 	packfile_store_prepare(store);
 }
 
-struct packed_git *get_all_packs(struct repository *r)
+struct packed_git *packfile_store_get_packs(struct packfile_store *store)
 {
-	packfile_store_prepare(r->objects->packfiles);
+	packfile_store_prepare(store);
 
-	for (struct odb_source *source = r->objects->sources; source; source = source->next) {
+	for (struct odb_source *source = store->odb->sources; source; source = source->next) {
 		struct multi_pack_index *m = source->midx;
 		if (!m)
 			continue;
 		for (uint32_t i = 0; i < m->num_packs + m->num_packs_in_base; i++)
-			prepare_midx_pack(r, m, i);
+			prepare_midx_pack(store->odb->repo, m, i);
 	}
 
-	return r->objects->packfiles->packs;
+	return store->packs;
 }
 
 struct list_head *get_packed_git_mru(struct repository *r)
@@ -2101,7 +2101,7 @@ struct packed_git **kept_pack_cache(struct repository *r, unsigned flags)
 		 * covers, one kept and one not kept, but the midx returns only
 		 * the non-kept version.
 		 */
-		for (p = get_all_packs(r); p; p = p->next) {
+		for (p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next) {
 			if ((p->pack_keep && (flags & ON_DISK_KEEP_PACKS)) ||
 			    (p->pack_keep_in_core && (flags & IN_CORE_KEEP_PACKS))) {
 				ALLOC_GROW(packs, nr + 1, alloc);
@@ -2198,7 +2198,7 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 	int r = 0;
 	int pack_errors = 0;
 
-	for (p = get_all_packs(repo); p; p = p->next) {
+	for (p = packfile_store_get_packs(repo->objects->packfiles); p; p = p->next) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
 		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&
diff --git a/packfile.h b/packfile.h
index 15cb378781d..86ab70eef9e 100644
--- a/packfile.h
+++ b/packfile.h
@@ -127,6 +127,12 @@ void packfile_store_reprepare(struct packfile_store *store);
 void packfile_store_add_pack(struct packfile_store *store,
 			     struct packed_git *pack);
 
+/*
+ * Get all packs managed by the given store, including packfiles that are
+ * referenced by multi-pack indices.
+ */
+struct packed_git *packfile_store_get_packs(struct packfile_store *store);
+
 /*
  * Open the packfile and add it to the store if it isn't yet known. Returns
  * either the newly opened packfile or the preexisting packfile. Returns a
@@ -212,7 +218,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
 struct list_head *get_packed_git_mru(struct repository *r);
-struct packed_git *get_all_packs(struct repository *r);
 
 /*
  * Give a rough count of objects in the repository. This sacrifices accuracy
diff --git a/server-info.c b/server-info.c
index 9bb30d9ab71..79234c7fed3 100644
--- a/server-info.c
+++ b/server-info.c
@@ -292,7 +292,7 @@ static void init_pack_info(struct repository *r, const char *infofile, int force
 	int i;
 	size_t alloc = 0;
 
-	for (p = get_all_packs(r); p; p = p->next) {
+	for (p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next) {
 		/* we ignore things on alternate path since they are
 		 * not available to the pullers in general.
 		 */
diff --git a/t/helper/test-find-pack.c b/t/helper/test-find-pack.c
index 611a13a3261..183a777fc54 100644
--- a/t/helper/test-find-pack.c
+++ b/t/helper/test-find-pack.c
@@ -39,7 +39,7 @@ int cmd__find_pack(int argc, const char **argv)
 	if (repo_get_oid(the_repository, argv[0], &oid))
 		die("cannot parse %s as an object name", argv[0]);
 
-	for (p = get_all_packs(the_repository); p; p = p->next)
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next)
 		if (find_pack_entry_one(&oid, p)) {
 			printf("%s\n", p->pack_name);
 			actual_count++;
diff --git a/t/helper/test-pack-mtimes.c b/t/helper/test-pack-mtimes.c
index d51aaa3dc40..cfdfae77a6c 100644
--- a/t/helper/test-pack-mtimes.c
+++ b/t/helper/test-pack-mtimes.c
@@ -37,7 +37,7 @@ int cmd__pack_mtimes(int argc, const char **argv)
 	if (argc != 2)
 		usage(pack_mtimes_usage);
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		strbuf_addstr(&buf, basename(p->pack_name));
 		strbuf_strip_suffix(&buf, ".pack");
 		strbuf_addstr(&buf, ".mtimes");
-- 
2.51.0.261.g7ce5a0a67e.dirty
Taylor Blau· Aug 27, 2025, 01:45 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v2 15/16] packfile: refactor `get_all_packs()` to work on packfile store

On Thu, Aug 21, 2025 at 09:39:13AM +0200, Patrick Steinhardt wrote:
> The `get_all_packs()` function prepares the packfile store and then
> returns its packfiles. Refactor it to accept a packfile store instead of
> a repository to clarify its scope.

I think that clarifying the scope here is a good idea. But I am a little sad to see this patch proposing that we drop get_all_packs(), which IMHO is a useful convenience function.

In effect this is pushing out the implementation details of the packfile_store out to every caller that wants to use get_all_packs(), which I am not sure is a win. Should those callers care where the array of packs is found, or have to write

    packfile_store_get_packs(the_repository->objects->packfiles)
each time they want to get the set of packs in a repository?

I could see an argument in the future where we have object stores that aren't packfile-based and thus calling "get_all_packs()" is not meaningful. But I don't think we are there yet, so I think that this patch is pushing the burden of that future hypothetical on all existing callers of get_all_packs().

Show 33 quoted lines
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  builtin/cat-file.c          |  2 +-
>  builtin/count-objects.c     |  2 +-
>  builtin/fast-import.c       |  4 ++--
>  builtin/fsck.c              |  8 ++++----
>  builtin/gc.c                |  8 ++++----
>  builtin/pack-objects.c      | 18 +++++++++---------
>  builtin/pack-redundant.c    |  4 ++--
>  builtin/repack.c            |  6 +++---
>  connected.c                 |  2 +-
>  http-backend.c              |  4 ++--
>  http.c                      |  2 +-
>  object-name.c               |  4 ++--
>  pack-bitmap.c               |  4 ++--
>  pack-objects.c              |  2 +-
>  packfile.c                  | 14 +++++++-------
>  packfile.h                  |  7 ++++++-
>  server-info.c               |  2 +-
>  t/helper/test-find-pack.c   |  2 +-
>  t/helper/test-pack-mtimes.c |  2 +-
>  19 files changed, 51 insertions(+), 46 deletions(-)
>
> diff --git a/builtin/cat-file.c b/builtin/cat-file.c
> index fce0b06451c..7124c43fb14 100644
> --- a/builtin/cat-file.c
> +++ b/builtin/cat-file.c
> @@ -854,7 +854,7 @@ static void batch_each_object(struct batch_options *opt,
>  						 batch_one_object_bitmapped, &payload)) {
>  		struct packed_git *pack;
>
> -		for (pack = get_all_packs(the_repository); pack; pack = pack->next) {
> +		for (pack = packfile_store_get_packs(the_repository->objects->packfiles); pack; pack = pack->next) {

If we do go this route, it might be nice to introduce the pattern of having a stack variable to hold the packfile_store pointer, since the line above here is getting a little long at >100 characters just to enumerate packs.

Thanks, Taylor

Patrick Steinhardt· Sep 2, 2025, 08:51 UTC · re: Taylor Blau · lore

Re: [PATCH v2 15/16] packfile: refactor `get_all_packs()` to work on packfile store

On Tue, Aug 26, 2025 at 09:45:15PM -0400, Taylor Blau wrote:
Show 23 quoted lines
> On Thu, Aug 21, 2025 at 09:39:13AM +0200, Patrick Steinhardt wrote:
> > The `get_all_packs()` function prepares the packfile store and then
> > returns its packfiles. Refactor it to accept a packfile store instead of
> > a repository to clarify its scope.
> 
> I think that clarifying the scope here is a good idea. But I am a little
> sad to see this patch proposing that we drop get_all_packs(), which IMHO
> is a useful convenience function.
> 
> In effect this is pushing out the implementation details of the
> packfile_store out to every caller that wants to use get_all_packs(),
> which I am not sure is a win. Should those callers care where the array
> of packs is found, or have to write
> 
>     packfile_store_get_packs(the_repository->objects->packfiles)
> 
> each time they want to get the set of packs in a repository?
> 
> I could see an argument in the future where we have object stores that
> aren't packfile-based and thus calling "get_all_packs()" is not
> meaningful. But I don't think we are there yet, so I think that this
> patch is pushing the burden of that future hypothetical on all existing
> callers of get_all_packs().

We aren't there yet indeed, but the entire goal of this patch series is to prepare for that future. So we have to do some steps into that direction that might not yet be entirely sensible by themselves, but that are necessary prerequisites.

In the end there ideally shouldn't be that many callers that want to access packfiles directly, but it should be the case that most of them go via the ODB. But many of the callers that we're adapting in this patch are callers that are deeply tied to the actual ODB on-disk layout, and we'll have to tear down the abstraction layer between ODB and the actual backend used to store objects. It's unfortunate, but we cannot really avoid that in a bunch of situations.

Show 15 quoted lines
> > diff --git a/builtin/cat-file.c b/builtin/cat-file.c
> > index fce0b06451c..7124c43fb14 100644
> > --- a/builtin/cat-file.c
> > +++ b/builtin/cat-file.c
> > @@ -854,7 +854,7 @@ static void batch_each_object(struct batch_options *opt,
> >  						 batch_one_object_bitmapped, &payload)) {
> >  		struct packed_git *pack;
> >
> > -		for (pack = get_all_packs(the_repository); pack; pack = pack->next) {
> > +		for (pack = packfile_store_get_packs(the_repository->objects->packfiles); pack; pack = pack->next) {
> 
> If we do go this route, it might be nice to introduce the pattern of
> having a stack variable to hold the packfile_store pointer, since the
> line above here is getting a little long at >100 characters just to
> enumerate packs.

Okay, will do for now. In a subsequent patch series I'm going to introduce a helper `packfile_store_for_each_pack()` that'll make this a bit less verbose, too.

Patrick
Taylor Blau· Sep 11, 2025, 23:33 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v2 15/16] packfile: refactor `get_all_packs()` to work on packfile store

On Tue, Sep 02, 2025 at 10:51:13AM +0200, Patrick Steinhardt wrote:
Show 29 quoted lines
> On Tue, Aug 26, 2025 at 09:45:15PM -0400, Taylor Blau wrote:
> > On Thu, Aug 21, 2025 at 09:39:13AM +0200, Patrick Steinhardt wrote:
> > > The `get_all_packs()` function prepares the packfile store and then
> > > returns its packfiles. Refactor it to accept a packfile store instead of
> > > a repository to clarify its scope.
> >
> > I think that clarifying the scope here is a good idea. But I am a little
> > sad to see this patch proposing that we drop get_all_packs(), which IMHO
> > is a useful convenience function.
> >
> > In effect this is pushing out the implementation details of the
> > packfile_store out to every caller that wants to use get_all_packs(),
> > which I am not sure is a win. Should those callers care where the array
> > of packs is found, or have to write
> >
> >     packfile_store_get_packs(the_repository->objects->packfiles)
> >
> > each time they want to get the set of packs in a repository?
> >
> > I could see an argument in the future where we have object stores that
> > aren't packfile-based and thus calling "get_all_packs()" is not
> > meaningful. But I don't think we are there yet, so I think that this
> > patch is pushing the burden of that future hypothetical on all existing
> > callers of get_all_packs().
>
> We aren't there yet indeed, but the entire goal of this patch series is
> to prepare for that future. So we have to do some steps into that
> direction that might not yet be entirely sensible by themselves, but
> that are necessary prerequisites.

I understand what you're saying, but I think this is the wrong place to be applying that trade-off.

Yes, we will eventually want to have callers handle object lookups differently depending on what ODB is in use. When that happens, yes, we should be pushing callers to go through the ODB abstraction layer. But we are not there yet, and I do not see a compelling reason to force this burden on *all* callers of get_all_packs() today.

Show 7 quoted lines
> In the end there ideally shouldn't be that many callers that want to
> access packfiles directly, but it should be the case that most of them
> go via the ODB. But many of the callers that we're adapting in this
> patch are callers that are deeply tied to the actual ODB on-disk layout,
> and we'll have to tear down the abstraction layer between ODB and the
> actual backend used to store objects. It's unfortunate, but we cannot
> really avoid that in a bunch of situations.

Maybe at some point, but that is definitely not the case today. I count nearly 50 mentions of either "get_all_packs()" or "get_packed_git()".

I am fine with evolving those callers one at a time, and think that the "the_repository" conversion has a good model for us to follow here. Replacing, say, get_oid() with repo_get_oid() over time instead for forcing all callers to do so at once was a good idea for a number of reasons, and I think we should follow the same model here.

Thanks, Taylor

Patrick Steinhardt· Sep 15, 2025, 07:44 UTC · re: Taylor Blau · lore

Re: [PATCH v2 15/16] packfile: refactor `get_all_packs()` to work on packfile store

On Thu, Sep 11, 2025 at 07:33:09PM -0400, Taylor Blau wrote:
Show 56 quoted lines
> On Tue, Sep 02, 2025 at 10:51:13AM +0200, Patrick Steinhardt wrote:
> > On Tue, Aug 26, 2025 at 09:45:15PM -0400, Taylor Blau wrote:
> > > On Thu, Aug 21, 2025 at 09:39:13AM +0200, Patrick Steinhardt wrote:
> > > > The `get_all_packs()` function prepares the packfile store and then
> > > > returns its packfiles. Refactor it to accept a packfile store instead of
> > > > a repository to clarify its scope.
> > >
> > > I think that clarifying the scope here is a good idea. But I am a little
> > > sad to see this patch proposing that we drop get_all_packs(), which IMHO
> > > is a useful convenience function.
> > >
> > > In effect this is pushing out the implementation details of the
> > > packfile_store out to every caller that wants to use get_all_packs(),
> > > which I am not sure is a win. Should those callers care where the array
> > > of packs is found, or have to write
> > >
> > >     packfile_store_get_packs(the_repository->objects->packfiles)
> > >
> > > each time they want to get the set of packs in a repository?
> > >
> > > I could see an argument in the future where we have object stores that
> > > aren't packfile-based and thus calling "get_all_packs()" is not
> > > meaningful. But I don't think we are there yet, so I think that this
> > > patch is pushing the burden of that future hypothetical on all existing
> > > callers of get_all_packs().
> >
> > We aren't there yet indeed, but the entire goal of this patch series is
> > to prepare for that future. So we have to do some steps into that
> > direction that might not yet be entirely sensible by themselves, but
> > that are necessary prerequisites.
> 
> I understand what you're saying, but I think this is the wrong place to
> be applying that trade-off.
> 
> Yes, we will eventually want to have callers handle object lookups
> differently depending on what ODB is in use. When that happens, yes, we
> should be pushing callers to go through the ODB abstraction layer. But
> we are not there yet, and I do not see a compelling reason to force this
> burden on *all* callers of get_all_packs() today.
> 
> > In the end there ideally shouldn't be that many callers that want to
> > access packfiles directly, but it should be the case that most of them
> > go via the ODB. But many of the callers that we're adapting in this
> > patch are callers that are deeply tied to the actual ODB on-disk layout,
> > and we'll have to tear down the abstraction layer between ODB and the
> > actual backend used to store objects. It's unfortunate, but we cannot
> > really avoid that in a bunch of situations.
> 
> Maybe at some point, but that is definitely not the case today. I count
> nearly 50 mentions of either "get_all_packs()" or "get_packed_git()".
> 
> I am fine with evolving those callers one at a time, and think that the
> "the_repository" conversion has a good model for us to follow here.
> Replacing, say, get_oid() with repo_get_oid() over time instead for
> forcing all callers to do so at once was a good idea for a number of
> reasons, and I think we should follow the same model here.

The problem though is that I'm now facing a chicken-and-egg situation. We'll have to move the packfiles into the object source to make it possible to abstract those away. So that's basically what I am working towards: to remove the global linked list of packfiles that contains packfiles from _all_ object sources. Moving the logic into the packfile store and make it the one place to access them is a natural step into that direction that makes all of this way easier.

What I don't quite understand is what your concern is. Sure, we are now a bit more verbose here because we have to pass the packfile store instead of `the_repository`. But that doesn't really change anything in the status quo, and it makes subsequent patch series way easier to implement. So what are you worried about?

Overall I'd really prefer to stick to this change, as subsequent steps build on this and as this allows us to iterate towards the goal.

Patrick
Patrick Steinhardt· Aug 21, 2025, 07:39 UTC · re: Patrick Steinhardt · lore

[PATCH v2 16/16] packfile: refactor `get_packed_git_mru()` to work on packfile store

The `get_packed_git_mru()` function prepares the packfile store and then returns its packfiles in most-recently-used order. Refactor it to accept a packfile store instead of a repository to clarify its scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/pack-objects.c | 4 ++--
 packfile.c             | 6 +++---
 packfile.h             | 7 +++++--
 3 files changed, 10 insertions(+), 7 deletions(-)
Show changes to 3 files +10 −7

builtin/pack-objects.c, packfile.c, packfile.h

diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 1c24b84510..4e75f14df1 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -1748,12 +1748,12 @@ static int want_object_in_pack_mtime(const struct object_id *oid,
 		}
 	}
 
-	list_for_each(pos, get_packed_git_mru(the_repository)) {
+	list_for_each(pos, packfile_store_get_packs_mru(the_repository->objects->packfiles)) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		want = want_object_in_pack_one(p, oid, exclude, found_pack, found_offset, found_mtime);
 		if (!exclude && want > 0)
 			list_move(&p->mru,
-				  get_packed_git_mru(the_repository));
+				  packfile_store_get_packs_mru(the_repository->objects->packfiles));
 		if (want != -1)
 			return want;
 	}
diff --git a/packfile.c b/packfile.c
index 19227ea0b3..8451467daf 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1044,10 +1044,10 @@ struct packed_git *packfile_store_get_packs(struct packfile_store *store)
 	return store->packs;
 }
 
-struct list_head *get_packed_git_mru(struct repository *r)
+struct list_head *packfile_store_get_packs_mru(struct packfile_store *store)
 {
-	packfile_store_prepare(r->objects->packfiles);
-	return &r->objects->packfiles->mru;
+	packfile_store_prepare(store);
+	return &store->mru;
 }
 
 /*
diff --git a/packfile.h b/packfile.h
index 86ab70eef9..b87fa2df59 100644
--- a/packfile.h
+++ b/packfile.h
@@ -133,6 +133,11 @@ void packfile_store_add_pack(struct packfile_store *store,
  */
 struct packed_git *packfile_store_get_packs(struct packfile_store *store);
 
+/*
+ * Get all packs in most-recently-used order.
+ */
+struct list_head *packfile_store_get_packs_mru(struct packfile_store *store);
+
 /*
  * Open the packfile and add it to the store if it isn't yet known. Returns
  * either the newly opened packfile or the preexisting packfile. Returns a
@@ -217,8 +222,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-struct list_head *get_packed_git_mru(struct repository *r);
-
 /*
  * Give a rough count of objects in the repository. This sacrifices accuracy
  * for speed.
-- 
2.51.0.261.g7ce5a0a67e.dirty
Patrick Steinhardt· Sep 2, 2025, 10:48 UTC · re: Patrick Steinhardt · lore

[PATCH v3 00/15] packfile: carve out a new packfile store

Hi,

information about a object database's packfiles is currently distributed across two different structures:

  - `struct packed_git` contains the `next` pointer as well as the
    `mru_head`, both of which serve to store the list of packfiles.
  - `struct object_database` contains several fields that relate to the
    packfiles.

So we don't really have a central data structure that tracks our packfiles, and consequently responsibilities aren't always clear cut. A consequence for the upcoming pluggable object databases is that this makes it very hard to move management of packfiles from the object database level down into the object database source.

This patch series introduces a new `struct packfile_store`, which is about to become the single source of truth for managing packfiles, and carves out the packfile store subsystem.

This is the first step to make packfiles work with pluggable object databases. Next steps will be to:

  - Move the `struct packed_git::next` and `struct packed::mru_head`
    pointers into the packfile store so that `struct packed_git` only
    tracks a single packfile.
  - Push the `struct packfile_store` down one level so that it's not
    hosted by the object database anymore, but instead by the object
    database source.
Changes in v2:
  - Convert the `initialized` flag into a boolean.
  - Polish some commit messages.
  - Some smaller formatting changes to the layout of `struct
    object_database`.
  - Link to v1: https://lore.kernel.org/r/20250819-b4-pks-packfiles-store-v1-0-1660842e125a@pks.im
Changes in v3:
  - Rebased on top of master at 6ad8021821 (The fifth batch, 2025-08-29)
    with ps/object-store-midx-dedup-info at 13296ac909 (midx: compute
    paths via their source, 2025-08-11) merged into it. This fixes
    various conflicts with "seen". There's still two conflicts: a
    trivial one with jt/de-global-bulk-checkin. And a more complex one
    with tb/prepare-midx-pack-cleanup. I don't think it's necessary to
    really address the first one, but I'm unsure how to proceed with the
    second one given that the patch series still seems to be cooking.
  - Set `struct object_database::packfiles` to `NULL` after free'ing it.
  - Add a comment to explain the kept cache.
  - Fix a missing `obj_read_lock()` call.
  - Drop the commit that always adds packfiles to the MRU. I've moved
    this into a subsequent patch series.
  - Avoid some overly long lines by storing the pointer to the packfile
    store on the stack.
  - Point out the difference between `get_all_packs()` and
    `get_packed_git()`.
  - Link to v2: https://lore.kernel.org/r/20250821-b4-pks-packfiles-store-v2-0-d10623355e9f@pks.im
Thanks!
Patrick
---
Patrick Steinhardt (15):
      packfile: introduce a new `struct packfile_store`
      odb: move list of packfiles into `struct packfile_store`
      odb: move initialization bit into `struct packfile_store`
      odb: move packfile map into `struct packfile_store`
      odb: move MRU list of packfiles into `struct packfile_store`
      odb: move kept cache into `struct packfile_store`
      packfile: reorder functions to avoid function declaration
      packfile: refactor `prepare_packed_git()` to work on packfile store
      packfile: split up responsibilities of `reprepare_packed_git()`
      packfile: refactor `install_packed_git()` to work on packfile store
      packfile: introduce function to load and add packfiles
      packfile: move `get_multi_pack_index()` into "midx.c"
      packfile: remove `get_packed_git()`
      packfile: refactor `get_all_packs()` to work on packfile store
      packfile: refactor `get_packed_git_mru()` to work on packfile store
 builtin/backfill.c          |   2 +-
 builtin/cat-file.c          |   3 +-
 builtin/count-objects.c     |   3 +-
 builtin/fast-import.c       |  10 +-
 builtin/fsck.c              |  11 +-
 builtin/gc.c                |  14 ++-
 builtin/grep.c              |   2 +-
 builtin/index-pack.c        |  10 +-
 builtin/pack-objects.c      |  32 +++--
 builtin/pack-redundant.c    |   6 +-
 builtin/receive-pack.c      |   2 +-
 builtin/repack.c            |  11 +-
 bulk-checkin.c              |   2 +-
 connected.c                 |   5 +-
 fetch-pack.c                |   4 +-
 http-backend.c              |   5 +-
 http.c                      |   5 +-
 http.h                      |   2 +-
 midx.c                      |  29 ++---
 midx.h                      |   1 +
 object-name.c               |   6 +-
 odb.c                       |  40 ++++--
 odb.h                       |  34 ++----
 pack-bitmap.c               |   4 +-
 pack-objects.c              |   3 +-
 packfile.c                  | 287 ++++++++++++++++++++++++--------------------
 packfile.h                  | 119 +++++++++++++++---
 server-info.c               |   3 +-
 t/helper/test-find-pack.c   |   2 +-
 t/helper/test-pack-mtimes.c |   2 +-
 transport-helper.c          |   2 +-
 31 files changed, 391 insertions(+), 270 deletions(-)
Range-diff versus v2:
 1:  9f205abd1e =  1:  efa41d9634 packfile: introduce a new `struct packfile_store`
 2:  a48391a618 !  2:  c4a45fdaf1 odb: move list of packfiles into `struct packfile_store`
    @@ odb.c: void odb_clear(struct object_database *o)
     -	}
     -	o->packed_git = NULL;
     +	packfile_store_free(o->packfiles);
    ++	o->packfiles = NULL;
      
      	hashmap_clear(&o->pack_map);
      	string_list_clear(&o->submodule_source_paths, 0);
    @@ packfile.c: void reprepare_packed_git(struct repository *r)
      
      struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
     @@ packfile.c: struct packed_git *get_all_packs(struct repository *r)
    - 			prepare_midx_pack(r, m, i);
    + 			prepare_midx_pack(m, i);
      	}
      
     -	return r->objects->packed_git;
    @@ packfile.c: const struct packed_git *has_packed_and_bad(struct repository *r,
      			return p;
      	return NULL;
     @@ packfile.c: int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
    - 		if (source->midx && fill_midx_entry(r, oid, e, source->midx))
    + 		if (source->midx && fill_midx_entry(source->midx, oid, e))
      			return 1;
      
     -	if (!r->objects->packed_git)
    @@ packfile.c: struct packfile_store *packfile_store_new(struct object_database *od
     +
     +void packfile_store_close(struct packfile_store *store)
     +{
    -+	struct packed_git *p;
    -+
    -+	for (p = store->packs; p; p = p->next)
    ++	for (struct packed_git *p = store->packs; p; p = p->next) {
     +		if (p->do_not_close)
     +			BUG("want to close pack marked 'do-not-close'");
     +		else
     +			close_pack(p);
    ++	}
     +}
     
      ## packfile.h ##
 3:  4a8cdfd936 =  3:  22125b27eb odb: move initialization bit into `struct packfile_store`
 4:  ba32eb35c4 !  4:  b927b57c6d odb: move packfile map into `struct packfile_store`
    @@ Commit message
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
     
      ## midx.c ##
    -@@ midx.c: int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,
    +@@ midx.c: int prepare_midx_pack(struct multi_pack_index *m,
      	strbuf_addbuf(&key, &pack_name);
      	strbuf_strip_suffix(&key, ".idx");
      	strbuf_addstr(&key, ".pack");
    @@ odb.c: struct object_database *odb_new(struct repository *repo)
      	string_list_init_dup(&o->submodule_source_paths);
      	return o;
     @@ odb.c: void odb_clear(struct object_database *o)
    - 	close_object_store(o);
      	packfile_store_free(o->packfiles);
    + 	o->packfiles = NULL;
      
     -	hashmap_clear(&o->pack_map);
      	string_list_clear(&o->submodule_source_paths, 0);
 5:  a87f102193 !  5:  030e34967d odb: move MRU list of packfiles into `struct packfile_store`
    @@ Commit message
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
     
      ## midx.c ##
    -@@ midx.c: int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,
    - 		p = add_packed_git(r, pack_name.buf, pack_name.len, m->local);
    +@@ midx.c: int prepare_midx_pack(struct multi_pack_index *m,
    + 				   m->source->local);
      		if (p) {
      			install_packed_git(r, p);
     -			list_add_tail(&p->mru, &r->objects->packed_git_mru);
    @@ odb.c: void odb_clear(struct object_database *o)
     -	INIT_LIST_HEAD(&o->packed_git_mru);
      	close_object_store(o);
      	packfile_store_free(o->packfiles);
    - 
    + 	o->packfiles = NULL;
     
      ## odb.h ##
     @@
 6:  7b43149530 !  6:  bda7ae0259 odb: move kept cache into `struct packfile_store`
    @@ packfile.h: struct packfile_store {
      	 */
      	struct packed_git *packs;
      
    ++	/*
    ++	 * Cache of packfiles which are marked as "kept", either because there
    ++	 * is an on-disk ".keep" file or because they are marked as "kept" in
    ++	 * memory.
    ++	 *
    ++	 * Should not be accessed directly, but via `kept_pack_cache()`. The
    ++	 * list of packs gets invalidated when the stored flags and the flags
    ++	 * passed to `kept_pack_cache()` mismatch.
    ++	 */
     +	struct {
     +		struct packed_git **packs;
     +		unsigned flags;
 7:  23d8d87330 !  7:  0ed3c2f690 packfile: reorder functions to avoid function declaration
    @@ Commit message
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
     
      ## packfile.c ##
    -@@ packfile.c: static void prepare_packed_git_one(struct odb_source *source, int local)
    +@@ packfile.c: static void prepare_packed_git_one(struct odb_source *source)
      	string_list_clear(data.garbage, 0);
      }
      
 8:  1623a5682e !  8:  c39180e7d2 packfile: refactor `prepare_packed_git()` to work on packfile store
    @@ packfile.c: static int sort_pack(const struct packed_git *a, const struct packed
      
     -	odb_prepare_alternates(r->objects);
     -	for (source = r->objects->sources; source; source = source->next) {
    --		int local = (source == r->objects->sources);
     +	odb_prepare_alternates(store->odb);
     +	for (source = store->odb->sources; source; source = source->next) {
    -+		int local = (source == store->odb->sources);
    - 		prepare_multi_pack_index_one(source, local);
    - 		prepare_packed_git_one(source, local);
    + 		prepare_multi_pack_index_one(source);
    + 		prepare_packed_git_one(source);
      	}
     -	rearrange_packed_git(r);
     +	sort_packs(&store->packs, sort_pack);
    @@ packfile.c: int find_pack_entry(struct repository *r, const struct object_id *oi
     +	packfile_store_prepare(r->objects->packfiles);
      
      	for (struct odb_source *source = r->objects->sources; source; source = source->next)
    - 		if (source->midx && fill_midx_entry(r, oid, e, source->midx))
    + 		if (source->midx && fill_midx_entry(source->midx, oid, e))
 9:  1ef4f6b1da !  9:  c39986a035 packfile: split up responsibilities of `reprepare_packed_git()`
    @@ odb.c: void odb_clear(struct object_database *o)
     +{
     +	struct odb_source *source;
     +
    ++	obj_read_lock();
    ++
     +	/*
     +	 * Reprepare alt odbs, in case the alternates file was modified
     +	 * during the course of this process. This only _adds_ odbs to
    @@ odb.h: struct object_database {
     +void odb_reprepare(struct object_database *o);
     +
      /*
    -  * Find source by its object directory path. Dies in case the source couldn't
    -  * be found.
    +  * Find source by its object directory path. Returns a `NULL` pointer in case
    +  * the source could not be found.
     
      ## packfile.c ##
     @@ packfile.c: static void packfile_store_prepare(struct packfile_store *store)
10:  3da8d69d11 ! 10:  8bc0e68c2f packfile: refactor `install_packed_git()` to work on packfile store
    @@ http.h: int finish_http_pack_request(struct http_pack_request *preq);
       * from http_get_info_packs() and have chosen a specific pack to fetch.
     
      ## midx.c ##
    -@@ midx.c: int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,
    - 	if (!p) {
    - 		p = add_packed_git(r, pack_name.buf, pack_name.len, m->local);
    +@@ midx.c: int prepare_midx_pack(struct multi_pack_index *m,
    + 		p = add_packed_git(r, pack_name.buf, pack_name.len,
    + 				   m->source->local);
      		if (p) {
     -			install_packed_git(r, p);
     +			packfile_store_add_pack(r->objects->packfiles, p);
11:  5c98d84581 <  -:  ---------- packfile: always add packfiles to MRU when adding a pack
12:  d9bca4e7cf ! 11:  0ea8488379 packfile: introduce function to load and add packfiles
    @@ builtin/index-pack.c: static void final(const char *final_pack_name, const char
      		printf("%s\n", hash_to_hex(hash));
     
      ## midx.c ##
    -@@ midx.c: int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,
    - 		      uint32_t pack_int_id)
    +@@ midx.c: int prepare_midx_pack(struct multi_pack_index *m,
      {
    + 	struct repository *r = m->source->odb->repo;
      	struct strbuf pack_name = STRBUF_INIT;
     -	struct strbuf key = STRBUF_INIT;
      	struct packed_git *p;
      
      	pack_int_id = midx_for_pack(&m, pack_int_id);
    -@@ midx.c: int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,
    +@@ midx.c: int prepare_midx_pack(struct multi_pack_index *m,
      
    - 	strbuf_addf(&pack_name, "%s/pack/%s", m->object_dir,
    + 	strbuf_addf(&pack_name, "%s/pack/%s", m->source->path,
      		    m->pack_names[pack_int_id]);
     -
     -	/* pack_map holds the ".pack" name, but we have the .idx */
    @@ midx.c: int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,
     -					strhash(key.buf), key.buf,
     -					struct packed_git, packmap_ent);
     -	if (!p) {
    --		p = add_packed_git(r, pack_name.buf, pack_name.len, m->local);
    --		if (p)
    +-		p = add_packed_git(r, pack_name.buf, pack_name.len,
    +-				   m->source->local);
    +-		if (p) {
     -			packfile_store_add_pack(r->objects->packfiles, p);
    +-			list_add_tail(&p->mru, &r->objects->packfiles->mru);
    +-		}
     -	}
     -
     +	p = packfile_store_load_pack(r->objects->packfiles,
    -+				     pack_name.buf, m->local);
    ++				     pack_name.buf, m->source->local);
    ++	if (p)
    ++		list_add_tail(&p->mru, &r->objects->packfiles->mru);
      	strbuf_release(&pack_name);
     -	strbuf_release(&key);
      
    @@ midx.c: int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,
     
      ## packfile.c ##
     @@ packfile.c: void packfile_store_add_pack(struct packfile_store *store,
    - 	list_add_tail(&pack->mru, &store->mru);
    + 	hashmap_add(&store->map, &pack->packmap_ent);
      }
      
     +struct packed_git *packfile_store_load_pack(struct packfile_store *store,
13:  bfda0a35f8 ! 12:  34a1b17424 packfile: move `get_multi_pack_index()` into "midx.c"
    @@ midx.c: static int midx_read_object_offsets(const unsigned char *chunk_start,
     +	return source->midx;
     +}
     +
    - static struct multi_pack_index *load_multi_pack_index_one(struct repository *r,
    - 							  const char *object_dir,
    - 							  const char *midx_name,
    + static struct multi_pack_index *load_multi_pack_index_one(struct odb_source *source,
    + 							  const char *midx_name)
    + {
     
      ## midx.h ##
    -@@ midx.h: void get_split_midx_filename_ext(const struct git_hash_algo *hash_algo,
    - 				 struct strbuf *buf, const char *object_dir,
    +@@ midx.h: void get_midx_chain_filename(struct odb_source *source, struct strbuf *out);
    + void get_split_midx_filename_ext(struct odb_source *source, struct strbuf *buf,
      				 const unsigned char *hash, const char *ext);
      
     +struct multi_pack_index *get_multi_pack_index(struct odb_source *source);
    -+
    - struct multi_pack_index *load_multi_pack_index(struct repository *r,
    - 					       const char *object_dir,
    - 					       int local);
    + struct multi_pack_index *load_multi_pack_index(struct odb_source *source);
    + int prepare_midx_pack(struct multi_pack_index *m, uint32_t pack_int_id);
    + struct packed_git *nth_midxed_pack(struct multi_pack_index *m,
     
      ## packfile.c ##
     @@ packfile.c: static void packfile_store_prepare_mru(struct packfile_store *store)
14:  8d54b4034b ! 13:  e039ab7770 packfile: remove `get_packed_git()`
    @@ Commit message
           - `get_all_packs()` calls `prepare_packed_git()`, as well, but also
             calls `prepare_midx_pack()` for each pack.
     
    -    This means that the latter function also properly loads the info of
    -    whether or not a packfile is part of a multi-pack index. Preparing this
    -    extra information also shouldn't be significantly more expensive:
    +    Based on the naming alone one might think that `get_all_packs()` would
    +    return more packs than `get_packed_git()`. But that's not the case: both
    +    functions end up returning the exact same list of packfiles. The real
    +    difference between those functions is that `get_all_packs()` also loads
    +    the info of whether or not a packfile is part of a multi-pack index.
    +
    +    Preparing this extra information also shouldn't be significantly more
    +    expensive:
     
           - We have already loaded all packfiles via `prepare_packed_git_one()`.
             So given that multi-pack indices may only refer to packfiles in the
15:  b04c2e4f7d ! 14:  65690526c2 packfile: refactor `get_all_packs()` to work on packfile store
    @@ Commit message
     
      ## builtin/cat-file.c ##
     @@ builtin/cat-file.c: static void batch_each_object(struct batch_options *opt,
    + 
    + 	if (bitmap && !for_each_bitmapped_object(bitmap, &opt->objects_filter,
      						 batch_one_object_bitmapped, &payload)) {
    ++		struct packfile_store *packs = the_repository->objects->packfiles;
      		struct packed_git *pack;
      
     -		for (pack = get_all_packs(the_repository); pack; pack = pack->next) {
    -+		for (pack = packfile_store_get_packs(the_repository->objects->packfiles); pack; pack = pack->next) {
    ++		for (pack = packfile_store_get_packs(packs); pack; pack = pack->next) {
      			if (bitmap_index_contains_pack(bitmap, pack) ||
      			    open_pack_index(pack))
      				continue;
     
      ## builtin/count-objects.c ##
    +@@ builtin/count-objects.c: int cmd_count_objects(int argc,
    + 				      count_loose, count_cruft, NULL, NULL);
    + 
    + 	if (verbose) {
    ++		struct packfile_store *packs = the_repository->objects->packfiles;
    + 		struct packed_git *p;
    + 		unsigned long num_pack = 0;
    + 		off_t size_pack = 0;
     @@ builtin/count-objects.c: int cmd_count_objects(int argc,
      		struct strbuf pack_buf = STRBUF_INIT;
      		struct strbuf garbage_buf = STRBUF_INIT;
      
     -		for (p = get_all_packs(the_repository); p; p = p->next) {
    -+		for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
    ++		for (p = packfile_store_get_packs(packs); p; p = p->next) {
      			if (!p->pack_local)
      				continue;
      			if (open_pack_index(p))
     
      ## builtin/fast-import.c ##
    +@@ builtin/fast-import.c: static int store_object(
    + 	struct object_id *oidout,
    + 	uintmax_t mark)
    + {
    ++	struct packfile_store *packs = the_repository->objects->packfiles;
    + 	void *out, *delta;
    + 	struct object_entry *e;
    + 	unsigned char hdr[96];
     @@ builtin/fast-import.c: static int store_object(
      	if (e->idx.offset) {
      		duplicate_count_by_type[type]++;
      		return 1;
     -	} else if (find_oid_pack(&oid, get_all_packs(the_repository))) {
    -+	} else if (find_oid_pack(&oid, packfile_store_get_packs(the_repository->objects->packfiles))) {
    ++	} else if (find_oid_pack(&oid, packfile_store_get_packs(packs))) {
      		e->type = type;
      		e->pack_id = MAX_PACK_ID;
      		e->idx.offset = 1; /* just not zero! */
    +@@ builtin/fast-import.c: static void truncate_pack(struct hashfile_checkpoint *checkpoint)
    + 
    + static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
    + {
    ++	struct packfile_store *packs = the_repository->objects->packfiles;
    + 	size_t in_sz = 64 * 1024, out_sz = 64 * 1024;
    + 	unsigned char *in_buf = xmalloc(in_sz);
    + 	unsigned char *out_buf = xmalloc(out_sz);
     @@ builtin/fast-import.c: static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
      		duplicate_count_by_type[OBJ_BLOB]++;
      		truncate_pack(&checkpoint);
      
     -	} else if (find_oid_pack(&oid, get_all_packs(the_repository))) {
    -+	} else if (find_oid_pack(&oid, packfile_store_get_packs(the_repository->objects->packfiles))) {
    ++	} else if (find_oid_pack(&oid, packfile_store_get_packs(packs))) {
      		e->type = OBJ_BLOB;
      		e->pack_id = MAX_PACK_ID;
      		e->idx.offset = 1; /* just not zero! */
     
      ## builtin/fsck.c ##
    -@@ builtin/fsck.c: static int check_pack_rev_indexes(struct repository *r, int show_progress)
    +@@ builtin/fsck.c: static int mark_packed_for_connectivity(const struct object_id *oid,
    + 
    + static int check_pack_rev_indexes(struct repository *r, int show_progress)
    + {
    ++	struct packfile_store *packs = r->objects->packfiles;
    + 	struct progress *progress = NULL;
    + 	uint32_t pack_count = 0;
      	int res = 0;
      
      	if (show_progress) {
     -		for (struct packed_git *p = get_all_packs(r); p; p = p->next)
    -+		for (struct packed_git *p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next)
    ++		for (struct packed_git *p = packfile_store_get_packs(packs); p; p = p->next)
      			pack_count++;
      		progress = start_delayed_progress(the_repository,
      						  "Verifying reverse pack-indexes", pack_count);
    @@ builtin/fsck.c: static int check_pack_rev_indexes(struct repository *r, int show
      	}
      
     -	for (struct packed_git *p = get_all_packs(r); p; p = p->next) {
    -+	for (struct packed_git *p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next) {
    ++	for (struct packed_git *p = packfile_store_get_packs(packs); p; p = p->next) {
      		int load_error = load_pack_revindex_from_disk(p);
      
      		if (load_error < 0) {
    +@@ builtin/fsck.c: int cmd_fsck(int argc,
    + 		for_each_packed_object(the_repository,
    + 				       mark_packed_for_connectivity, NULL, 0);
    + 	} else {
    ++		struct packfile_store *packs = the_repository->objects->packfiles;
    ++
    + 		odb_prepare_alternates(the_repository->objects);
    + 		for (source = the_repository->objects->sources; source; source = source->next)
    + 			fsck_source(source);
     @@ builtin/fsck.c: int cmd_fsck(int argc,
      			struct progress *progress = NULL;
      
      			if (show_progress) {
     -				for (p = get_all_packs(the_repository); p;
    -+				for (p = packfile_store_get_packs(the_repository->objects->packfiles); p;
    ++				for (p = packfile_store_get_packs(packs); p;
      				     p = p->next) {
      					if (open_pack_index(p))
      						continue;
    @@ builtin/fsck.c: int cmd_fsck(int argc,
      							  _("Checking objects"), total);
      			}
     -			for (p = get_all_packs(the_repository); p;
    -+			for (p = packfile_store_get_packs(the_repository->objects->packfiles); p;
    ++			for (p = packfile_store_get_packs(packs); p;
      			     p = p->next) {
      				/* verify gives error messages itself */
      				if (verify_pack(the_repository,
     
      ## builtin/gc.c ##
    -@@ builtin/gc.c: static struct packed_git *find_base_packs(struct string_list *packs,
    +@@ builtin/gc.c: static int too_many_loose_objects(struct gc_config *cfg)
    + static struct packed_git *find_base_packs(struct string_list *packs,
    + 					  unsigned long limit)
      {
    ++	struct packfile_store *packfiles = the_repository->objects->packfiles;
      	struct packed_git *p, *base = NULL;
      
     -	for (p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
    ++	for (p = packfile_store_get_packs(packfiles); p; p = p->next) {
      		if (!p->pack_local || p->is_cruft)
      			continue;
      		if (limit) {
    -@@ builtin/gc.c: static int too_many_packs(struct gc_config *cfg)
    +@@ builtin/gc.c: static struct packed_git *find_base_packs(struct string_list *packs,
    + 
    + static int too_many_packs(struct gc_config *cfg)
    + {
    ++	struct packfile_store *packs = the_repository->objects->packfiles;
    + 	struct packed_git *p;
    + 	int cnt;
    + 
      	if (cfg->gc_auto_pack_limit <= 0)
      		return 0;
      
     -	for (cnt = 0, p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (cnt = 0, p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
    ++	for (cnt = 0, p = packfile_store_get_packs(packs); p; p = p->next) {
      		if (!p->pack_local)
      			continue;
      		if (p->pack_keep)
    @@ builtin/gc.c: static off_t get_auto_pack_size(void)
      			max_size = p->pack_size;
     
      ## builtin/pack-objects.c ##
    +@@ builtin/pack-objects.c: static int pack_mtime_cmp(const void *_a, const void *_b)
    + 
    + static void read_packs_list_from_stdin(struct rev_info *revs)
    + {
    ++	struct packfile_store *packs = the_repository->objects->packfiles;
    + 	struct strbuf buf = STRBUF_INIT;
    + 	struct string_list include_packs = STRING_LIST_INIT_DUP;
    + 	struct string_list exclude_packs = STRING_LIST_INIT_DUP;
     @@ builtin/pack-objects.c: static void read_packs_list_from_stdin(struct rev_info *revs)
      	string_list_sort(&exclude_packs);
      	string_list_remove_duplicates(&exclude_packs, 0);
      
     -	for (p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
    ++	for (p = packfile_store_get_packs(packs); p; p = p->next) {
      		const char *pack_name = pack_basename(p);
      
      		if ((item = string_list_lookup(&include_packs, pack_name)))
    +@@ builtin/pack-objects.c: static void enumerate_cruft_objects(void)
    + 
    + static void enumerate_and_traverse_cruft_objects(struct string_list *fresh_packs)
    + {
    ++	struct packfile_store *packs = the_repository->objects->packfiles;
    + 	struct packed_git *p;
    + 	struct rev_info revs;
    + 	int ret;
     @@ builtin/pack-objects.c: static void enumerate_and_traverse_cruft_objects(struct string_list *fresh_packs
      	 * Re-mark only the fresh packs as kept so that objects in
      	 * unknown packs do not halt the reachability traversal early.
      	 */
     -	for (p = get_all_packs(the_repository); p; p = p->next)
    -+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next)
    ++	for (p = packfile_store_get_packs(packs); p; p = p->next)
      		p->pack_keep_in_core = 0;
      	mark_pack_kept_in_core(fresh_packs, 1);
      
    +@@ builtin/pack-objects.c: static void enumerate_and_traverse_cruft_objects(struct string_list *fresh_packs
    + 
    + static void read_cruft_objects(void)
    + {
    ++	struct packfile_store *packs = the_repository->objects->packfiles;
    + 	struct strbuf buf = STRBUF_INIT;
    + 	struct string_list discard_packs = STRING_LIST_INIT_DUP;
    + 	struct string_list fresh_packs = STRING_LIST_INIT_DUP;
     @@ builtin/pack-objects.c: static void read_cruft_objects(void)
      	string_list_sort(&discard_packs);
      	string_list_sort(&fresh_packs);
      
     -	for (p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
    ++	for (p = packfile_store_get_packs(packs); p; p = p->next) {
      		const char *pack_name = pack_basename(p);
      		struct string_list_item *item;
      
    -@@ builtin/pack-objects.c: static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
    +@@ builtin/pack-objects.c: static void add_unreachable_loose_objects(struct rev_info *revs)
    + 
    + static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
    + {
    ++	struct packfile_store *packs = the_repository->objects->packfiles;
    + 	static struct packed_git *last_found = (void *)1;
      	struct packed_git *p;
      
      	p = (last_found != (void *)1) ? last_found :
     -					get_all_packs(the_repository);
    -+					packfile_store_get_packs(the_repository->objects->packfiles);
    ++					packfile_store_get_packs(packs);
      
      	while (p) {
      		if ((!p->pack_local || p->pack_keep ||
    @@ builtin/pack-objects.c: static int has_sha1_pack_kept_or_nonlocal(const struct o
      		}
      		if (p == last_found)
     -			p = get_all_packs(the_repository);
    -+			p = packfile_store_get_packs(the_repository->objects->packfiles);
    ++			p = packfile_store_get_packs(packs);
      		else
      			p = p->next;
      		if (p == last_found)
    -@@ builtin/pack-objects.c: static void loosen_unused_packed_objects(void)
    +@@ builtin/pack-objects.c: static int loosened_object_can_be_discarded(const struct object_id *oid,
    + 
    + static void loosen_unused_packed_objects(void)
    + {
    ++	struct packfile_store *packs = the_repository->objects->packfiles;
    + 	struct packed_git *p;
    + 	uint32_t i;
      	uint32_t loosened_objects_nr = 0;
      	struct object_id oid;
      
     -	for (p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
    ++	for (p = packfile_store_get_packs(packs); p; p = p->next) {
      		if (!p->pack_local || p->pack_keep || p->pack_keep_in_core)
      			continue;
      
    -@@ builtin/pack-objects.c: static void add_extra_kept_packs(const struct string_list *names)
    +@@ builtin/pack-objects.c: static void get_object_list(struct rev_info *revs, int ac, const char **av)
    + 
    + static void add_extra_kept_packs(const struct string_list *names)
    + {
    ++	struct packfile_store *packs = the_repository->objects->packfiles;
    + 	struct packed_git *p;
    + 
      	if (!names->nr)
      		return;
      
     -	for (p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
    ++	for (p = packfile_store_get_packs(packs); p; p = p->next) {
      		const char *name = basename(p->pack_name);
      		int i;
      
     @@ builtin/pack-objects.c: int cmd_pack_objects(int argc,
    + 
      	add_extra_kept_packs(&keep_pack_list);
      	if (ignore_packed_keep_on_disk) {
    ++		struct packfile_store *packs = the_repository->objects->packfiles;
      		struct packed_git *p;
     -		for (p = get_all_packs(the_repository); p; p = p->next)
    -+		for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next)
    ++
    ++		for (p = packfile_store_get_packs(packs); p; p = p->next)
      			if (p->pack_local && p->pack_keep)
      				break;
      		if (!p) /* no keep-able packs found */
     @@ builtin/pack-objects.c: int cmd_pack_objects(int argc,
    + 		 * want to unset "local" based on looking at packs, as
      		 * it also covers non-local objects
      		 */
    ++		struct packfile_store *packs = the_repository->objects->packfiles;
      		struct packed_git *p;
     -		for (p = get_all_packs(the_repository); p; p = p->next) {
    -+		for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
    ++
    ++		for (p = packfile_store_get_packs(packs); p; p = p->next) {
      			if (!p->pack_local) {
      				have_non_local_packs = 1;
      				break;
    @@ builtin/pack-redundant.c: static struct pack_list * add_pack(struct packed_git *
      static struct pack_list * add_pack_file(const char *filename)
      {
     -	struct packed_git *p = get_all_packs(the_repository);
    -+	struct packed_git *p = packfile_store_get_packs(the_repository->objects->packfiles);
    ++	struct packfile_store *packs = the_repository->objects->packfiles;
    ++	struct packed_git *p = packfile_store_get_packs(packs);
      
      	if (strlen(filename) < 40)
      		die("Bad pack filename: %s", filename);
    @@ builtin/pack-redundant.c: static struct pack_list * add_pack_file(const char *fi
      static void load_all(void)
      {
     -	struct packed_git *p = get_all_packs(the_repository);
    -+	struct packed_git *p = packfile_store_get_packs(the_repository->objects->packfiles);
    ++	struct packfile_store *packs = the_repository->objects->packfiles;
    ++	struct packed_git *p = packfile_store_get_packs(packs);
      
      	while (p) {
      		add_pack(p);
     
      ## builtin/repack.c ##
    -@@ builtin/repack.c: static void collect_pack_filenames(struct existing_packs *existing,
    +@@ builtin/repack.c: static void existing_packs_release(struct existing_packs *existing)
    + static void collect_pack_filenames(struct existing_packs *existing,
    + 				   const struct string_list *extra_keep)
    + {
    ++	struct packfile_store *packs = the_repository->objects->packfiles;
      	struct packed_git *p;
      	struct strbuf buf = STRBUF_INIT;
      
     -	for (p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
    ++	for (p = packfile_store_get_packs(packs); p; p = p->next) {
      		int i;
      		const char *base;
      
     @@ builtin/repack.c: static void init_pack_geometry(struct pack_geometry *geometry,
    + 			       struct existing_packs *existing,
    + 			       const struct pack_objects_args *args)
    + {
    ++	struct packfile_store *packs = the_repository->objects->packfiles;
      	struct packed_git *p;
      	struct strbuf buf = STRBUF_INIT;
      
     -	for (p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
    ++	for (p = packfile_store_get_packs(packs); p; p = p->next) {
      		if (args->local && !p->pack_local)
      			/*
      			 * When asked to only repack local packfiles we skip
    -@@ builtin/repack.c: static void combine_small_cruft_packs(FILE *in, size_t combine_cruft_below_size,
    +@@ builtin/repack.c: static int write_filtered_pack(const struct pack_objects_args *args,
    + static void combine_small_cruft_packs(FILE *in, size_t combine_cruft_below_size,
    + 				      struct existing_packs *existing)
    + {
    ++	struct packfile_store *packs = the_repository->objects->packfiles;
    + 	struct packed_git *p;
      	struct strbuf buf = STRBUF_INIT;
      	size_t i;
      
     -	for (p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
    ++	for (p = packfile_store_get_packs(packs); p; p = p->next) {
      		if (!(p->is_cruft && p->pack_local))
      			continue;
      
     
      ## connected.c ##
     @@ connected.c: int check_connected(oid_iterate_fn fn, void *cb_data,
    + 		 */
    + 		odb_reprepare(the_repository->objects);
      		do {
    ++			struct packfile_store *packs = the_repository->objects->packfiles;
      			struct packed_git *p;
      
     -			for (p = get_all_packs(the_repository); p; p = p->next) {
    -+			for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
    ++			for (p = packfile_store_get_packs(packs); p; p = p->next) {
      				if (!p->pack_promisor)
      					continue;
      				if (find_pack_entry_one(oid, p))
     
      ## http-backend.c ##
    -@@ http-backend.c: static void get_info_packs(struct strbuf *hdr, char *arg UNUSED)
    +@@ http-backend.c: static void get_head(struct strbuf *hdr, char *arg UNUSED)
    + static void get_info_packs(struct strbuf *hdr, char *arg UNUSED)
    + {
    + 	size_t objdirlen = strlen(repo_get_object_directory(the_repository));
    ++	struct packfile_store *packs = the_repository->objects->packfiles;
    + 	struct strbuf buf = STRBUF_INIT;
    + 	struct packed_git *p;
      	size_t cnt = 0;
      
      	select_getanyfile(hdr);
     -	for (p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
    ++	for (p = packfile_store_get_packs(packs); p; p = p->next) {
      		if (p->pack_local)
      			cnt++;
      	}
      
      	strbuf_grow(&buf, cnt * 53 + 2);
     -	for (p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
    ++	for (p = packfile_store_get_packs(packs); p; p = p->next) {
      		if (p->pack_local)
      			strbuf_addf(&buf, "P %s\n", p->pack_name + objdirlen + 6);
      	}
     
      ## http.c ##
    +@@ http.c: static char *fetch_pack_index(unsigned char *hash, const char *base_url)
    + static int fetch_and_setup_pack_index(struct packed_git **packs_head,
    + 	unsigned char *sha1, const char *base_url)
    + {
    ++	struct packfile_store *packs = the_repository->objects->packfiles;
    + 	struct packed_git *new_pack, *p;
    + 	char *tmp_idx = NULL;
    + 	int ret;
     @@ http.c: static int fetch_and_setup_pack_index(struct packed_git **packs_head,
      	 * If we already have the pack locally, no need to fetch its index or
      	 * even add it to list; we already have all of its objects.
      	 */
     -	for (p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
    ++	for (p = packfile_store_get_packs(packs); p; p = p->next) {
      		if (hasheq(p->hash, sha1, the_repository->hash_algo))
      			return 0;
      	}
    @@ pack-bitmap.c: int verify_bitmap_files(struct repository *r)
      		res |= verify_bitmap_file(r->hash_algo, pack_bitmap_name);
     
      ## pack-objects.c ##
    +@@ pack-objects.c: struct object_entry *packlist_find(struct packing_data *pdata,
    + 
    + static void prepare_in_pack_by_idx(struct packing_data *pdata)
    + {
    ++	struct packfile_store *packs = pdata->repo->objects->packfiles;
    + 	struct packed_git **mapping, *p;
    + 	int cnt = 0, nr = 1U << OE_IN_PACK_BITS;
    + 
     @@ pack-objects.c: static void prepare_in_pack_by_idx(struct packing_data *pdata)
      	 * (i.e. in_pack_idx also zero) should return NULL.
      	 */
      	mapping[cnt++] = NULL;
     -	for (p = get_all_packs(pdata->repo); p; p = p->next, cnt++) {
    -+	for (p = packfile_store_get_packs(pdata->repo->objects->packfiles); p; p = p->next, cnt++) {
    ++	for (p = packfile_store_get_packs(packs); p; p = p->next, cnt++) {
      		if (cnt == nr) {
      			free(mapping);
      			return;
    @@ packfile.c: void packfile_store_reprepare(struct packfile_store *store)
      		struct multi_pack_index *m = source->midx;
      		if (!m)
      			continue;
    - 		for (uint32_t i = 0; i < m->num_packs + m->num_packs_in_base; i++)
    --			prepare_midx_pack(r, m, i);
    -+			prepare_midx_pack(store->odb->repo, m, i);
    +@@ packfile.c: struct packed_git *get_all_packs(struct repository *r)
    + 			prepare_midx_pack(m, i);
      	}
      
     -	return r->objects->packfiles->packs;
    @@ packfile.h: int for_each_packed_object(struct repository *repo, each_packed_obje
       * Give a rough count of objects in the repository. This sacrifices accuracy
     
      ## server-info.c ##
    -@@ server-info.c: static void init_pack_info(struct repository *r, const char *infofile, int force
    +@@ server-info.c: static int compare_info(const void *a_, const void *b_)
    + 
    + static void init_pack_info(struct repository *r, const char *infofile, int force)
    + {
    ++	struct packfile_store *packs = r->objects->packfiles;
    + 	struct packed_git *p;
    + 	int stale;
      	int i;
      	size_t alloc = 0;
      
     -	for (p = get_all_packs(r); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next) {
    ++	for (p = packfile_store_get_packs(packs); p; p = p->next) {
      		/* we ignore things on alternate path since they are
      		 * not available to the pullers in general.
      		 */
16:  594d14487e = 15:  1315247fd2 packfile: refactor `get_packed_git_mru()` to work on packfile store

--- base-commit: 337c7a0bbcf228ce11c87d066ecee352b3e52467 change-id: 20250806-b4-pks-packfiles-store-a44a608ca396

Patrick Steinhardt· Sep 2, 2025, 10:48 UTC · re: Patrick Steinhardt · lore

[PATCH v3 01/15] packfile: introduce a new `struct packfile_store`

Information about a object database's packfiles is currently distributed across two different structures:

  - `struct packed_git` contains the `next` pointer as well as the
    `mru_head`, both of which serve to store the list of packfiles.
  - `struct object_database` contains several fields that relate to the
    packfiles.

So we don't really have a central data structure that tracks our packfiles, and consequently responsibilities aren't always clear cut. A consequence for the upcoming pluggable object databases is that this makes it very hard to move management of packfiles from the object database level down into the object database source.

Introduce a new `struct packfile_store` which is about to become the single source of truth for managing packfiles. Right now this data structure doesn't yet contain anything, but in subsequent patches we will move all data structures that relate to packfiles and that are currently contained in `struct object_database` into this new home.

Note that this is only a first step: most importantly, we won't (yet) move the `struct packed_git::next` pointer around. This will happen in a subsequent patch series though so that `struct packed_git` will really only host information about the specific packfile it represents.

Further note that the new structure still sits at the wrong level at the end of this patch series: as mentioned, it should eventually sit at the level of the object database source, not at the object database level. But introducing the packfile store now already makes it way easier to eventually push down the now-selfcontained data structure by one level.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb.c      |  1 +
 odb.h      |  3 ++-
 packfile.c | 13 +++++++++++++
 packfile.h | 18 ++++++++++++++++++
 4 files changed, 34 insertions(+), 1 deletion(-)
Show changes to 4 files +34 −1

odb.c, odb.h, packfile.c, packfile.h

diff --git a/odb.c b/odb.c
index 75c443fe66..a2289ea97d 100644
--- a/odb.c
+++ b/odb.c
@@ -996,6 +996,7 @@ struct object_database *odb_new(struct repository *repo)
 
 	memset(o, 0, sizeof(*o));
 	o->repo = repo;
+	o->packfiles = packfile_store_new(o);
 	INIT_LIST_HEAD(&o->packed_git_mru);
 	hashmap_init(&o->pack_map, pack_map_entry_cmp, NULL, 0);
 	pthread_mutex_init(&o->replace_mutex, NULL);
diff --git a/odb.h b/odb.h
index 51fe8a5a92..33034eaf2f 100644
--- a/odb.h
+++ b/odb.h
@@ -91,6 +91,7 @@ struct odb_source {
 };
 
 struct packed_git;
+struct packfile_store;
 struct cached_object_entry;
 
 /*
@@ -136,7 +137,7 @@ struct object_database {
 	 *
 	 * should only be accessed directly by packfile.c
 	 */
-
+	struct packfile_store *packfiles;
 	struct packed_git *packed_git;
 	/* A most-recently-used ordered version of the packed_git list. */
 	struct list_head packed_git_mru;
diff --git a/packfile.c b/packfile.c
index acb680966d..130d3e2507 100644
--- a/packfile.c
+++ b/packfile.c
@@ -2332,3 +2332,16 @@ int parse_pack_header_option(const char *in, unsigned char *out, unsigned int *l
 	*len = hdr - out;
 	return 0;
 }
+
+struct packfile_store *packfile_store_new(struct object_database *odb)
+{
+	struct packfile_store *store;
+	CALLOC_ARRAY(store, 1);
+	store->odb = odb;
+	return store;
+}
+
+void packfile_store_free(struct packfile_store *store)
+{
+	free(store);
+}
diff --git a/packfile.h b/packfile.h
index f16753f2a9..8d31fd619a 100644
--- a/packfile.h
+++ b/packfile.h
@@ -52,6 +52,24 @@ struct packed_git {
 	char pack_name[FLEX_ARRAY]; /* more */
 };
 
+/*
+ * A store that manages packfiles for a given object database.
+ */
+struct packfile_store {
+	struct object_database *odb;
+};
+
+/*
+ * Allocate and initialize a new empty packfile store for the given object
+ * database.
+ */
+struct packfile_store *packfile_store_new(struct object_database *odb);
+
+/*
+ * Free the packfile store and all its associated state.
+ */
+void packfile_store_free(struct packfile_store *store);
+
 static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
 				     const struct hashmap_entry *entry,
 				     const struct hashmap_entry *entry2,
-- 
2.51.0.384.g4c02a37b29.dirty
Karthik Nayak· Sep 9, 2025, 07:49 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v3 01/15] packfile: introduce a new `struct packfile_store`

Patrick Steinhardt <ps@pks.im> writes:
Show 49 quoted lines
> Information about a object database's packfiles is currently distributed
> across two different structures:
>
>   - `struct packed_git` contains the `next` pointer as well as the
>     `mru_head`, both of which serve to store the list of packfiles.
>
>   - `struct object_database` contains several fields that relate to the
>     packfiles.
>
> So we don't really have a central data structure that tracks our
> packfiles, and consequently responsibilities aren't always clear cut.
> A consequence for the upcoming pluggable object databases is that this
> makes it very hard to move management of packfiles from the object
> database level down into the object database source.
>
> Introduce a new `struct packfile_store` which is about to become the
> single source of truth for managing packfiles. Right now this data
> structure doesn't yet contain anything, but in subsequent patches we
> will move all data structures that relate to packfiles and that are
> currently contained in `struct object_database` into this new home.
>
> Note that this is only a first step: most importantly, we won't (yet)
> move the `struct packed_git::next` pointer around. This will happen in a
> subsequent patch series though so that `struct packed_git` will really
> only host information about the specific packfile it represents.
>
> Further note that the new structure still sits at the wrong level at the
> end of this patch series: as mentioned, it should eventually sit at the
> level of the object database source, not at the object database level.
> But introducing the packfile store now already makes it way easier to
> eventually push down the now-selfcontained data structure by one level.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  odb.c      |  1 +
>  odb.h      |  3 ++-
>  packfile.c | 13 +++++++++++++
>  packfile.h | 18 ++++++++++++++++++
>  4 files changed, 34 insertions(+), 1 deletion(-)
>
> diff --git a/odb.c b/odb.c
> index 75c443fe66..a2289ea97d 100644
> --- a/odb.c
> +++ b/odb.c
> @@ -996,6 +996,7 @@ struct object_database *odb_new(struct repository *repo)
>
>  	memset(o, 0, sizeof(*o));
>  	o->repo = repo;
> +	o->packfiles = packfile_store_new(o);

I assume that the packfile_store needs a linkback to the odb for some reason which is utilized in the future commits. reading on.

[snip]
Patrick Steinhardt· Sep 2, 2025, 10:48 UTC · re: Patrick Steinhardt · lore

[PATCH v3 02/15] odb: move list of packfiles into `struct packfile_store`

The object database tracks the list of packfiles it currently knows about. With the introduction of the `struct packfile_store` we have a better place to host this list though.

Move the list accordingly. Extract the logic from `odb_clear()` that knows to close all such packfiles and move it into the new subsystem, as well.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb.c      | 12 ++----------
 odb.h      |  1 -
 packfile.c | 46 +++++++++++++++++++++++++++++-----------------
 packfile.h | 15 ++++++++++++++-
 4 files changed, 45 insertions(+), 29 deletions(-)
Show changes to 4 files +45 −29

odb.c, odb.h, packfile.c, packfile.h

diff --git a/odb.c b/odb.c
index a2289ea97d..7201d01406 100644
--- a/odb.c
+++ b/odb.c
@@ -1038,16 +1038,8 @@ void odb_clear(struct object_database *o)
 
 	INIT_LIST_HEAD(&o->packed_git_mru);
 	close_object_store(o);
-
-	/*
-	 * `close_object_store()` only closes the packfiles, but doesn't free
-	 * them. We thus have to do this manually.
-	 */
-	for (struct packed_git *p = o->packed_git, *next; p; p = next) {
-		next = p->next;
-		free(p);
-	}
-	o->packed_git = NULL;
+	packfile_store_free(o->packfiles);
+	o->packfiles = NULL;
 
 	hashmap_clear(&o->pack_map);
 	string_list_clear(&o->submodule_source_paths, 0);
diff --git a/odb.h b/odb.h
index 33034eaf2f..22a170b434 100644
--- a/odb.h
+++ b/odb.h
@@ -138,7 +138,6 @@ struct object_database {
 	 * should only be accessed directly by packfile.c
 	 */
 	struct packfile_store *packfiles;
-	struct packed_git *packed_git;
 	/* A most-recently-used ordered version of the packed_git list. */
 	struct list_head packed_git_mru;
 
diff --git a/packfile.c b/packfile.c
index 130d3e2507..43e9a7cb45 100644
--- a/packfile.c
+++ b/packfile.c
@@ -278,7 +278,7 @@ static int unuse_one_window(struct packed_git *current)
 
 	if (current)
 		scan_windows(current, &lru_p, &lru_w, &lru_l);
-	for (p = current->repo->objects->packed_git; p; p = p->next)
+	for (p = current->repo->objects->packfiles->packs; p; p = p->next)
 		scan_windows(p, &lru_p, &lru_w, &lru_l);
 	if (lru_p) {
 		munmap(lru_w->base, lru_w->len);
@@ -362,13 +362,8 @@ void close_pack(struct packed_git *p)
 void close_object_store(struct object_database *o)
 {
 	struct odb_source *source;
-	struct packed_git *p;
 
-	for (p = o->packed_git; p; p = p->next)
-		if (p->do_not_close)
-			BUG("want to close pack marked 'do-not-close'");
-		else
-			close_pack(p);
+	packfile_store_close(o->packfiles);
 
 	for (source = o->sources; source; source = source->next) {
 		if (source->midx)
@@ -468,7 +463,7 @@ static int close_one_pack(struct repository *r)
 	struct pack_window *mru_w = NULL;
 	int accept_windows_inuse = 1;
 
-	for (p = r->objects->packed_git; p; p = p->next) {
+	for (p = r->objects->packfiles->packs; p; p = p->next) {
 		if (p->pack_fd == -1)
 			continue;
 		find_lru_pack(p, &lru_p, &mru_w, &accept_windows_inuse);
@@ -789,8 +784,8 @@ void install_packed_git(struct repository *r, struct packed_git *pack)
 	if (pack->pack_fd != -1)
 		pack_open_fds++;
 
-	pack->next = r->objects->packed_git;
-	r->objects->packed_git = pack;
+	pack->next = r->objects->packfiles->packs;
+	r->objects->packfiles->packs = pack;
 
 	hashmap_entry_init(&pack->packmap_ent, strhash(pack->pack_name));
 	hashmap_add(&r->objects->pack_map, &pack->packmap_ent);
@@ -974,7 +969,7 @@ unsigned long repo_approximate_object_count(struct repository *r)
 				count += m->num_objects;
 		}
 
-		for (p = r->objects->packed_git; p; p = p->next) {
+		for (p = r->objects->packfiles->packs; p; p = p->next) {
 			if (open_pack_index(p))
 				continue;
 			count += p->num_objects;
@@ -1015,7 +1010,7 @@ static int sort_pack(const struct packed_git *a, const struct packed_git *b)
 
 static void rearrange_packed_git(struct repository *r)
 {
-	sort_packs(&r->objects->packed_git, sort_pack);
+	sort_packs(&r->objects->packfiles->packs, sort_pack);
 }
 
 static void prepare_packed_git_mru(struct repository *r)
@@ -1024,7 +1019,7 @@ static void prepare_packed_git_mru(struct repository *r)
 
 	INIT_LIST_HEAD(&r->objects->packed_git_mru);
 
-	for (p = r->objects->packed_git; p; p = p->next)
+	for (p = r->objects->packfiles->packs; p; p = p->next)
 		list_add_tail(&p->mru, &r->objects->packed_git_mru);
 }
 
@@ -1073,7 +1068,7 @@ void reprepare_packed_git(struct repository *r)
 struct packed_git *get_packed_git(struct repository *r)
 {
 	prepare_packed_git(r);
-	return r->objects->packed_git;
+	return r->objects->packfiles->packs;
 }
 
 struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
@@ -1094,7 +1089,7 @@ struct packed_git *get_all_packs(struct repository *r)
 			prepare_midx_pack(m, i);
 	}
 
-	return r->objects->packed_git;
+	return r->objects->packfiles->packs;
 }
 
 struct list_head *get_packed_git_mru(struct repository *r)
@@ -1219,7 +1214,7 @@ const struct packed_git *has_packed_and_bad(struct repository *r,
 {
 	struct packed_git *p;
 
-	for (p = r->objects->packed_git; p; p = p->next)
+	for (p = r->objects->packfiles->packs; p; p = p->next)
 		if (oidset_contains(&p->bad_objects, oid))
 			return p;
 	return NULL;
@@ -2080,7 +2075,7 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 		if (source->midx && fill_midx_entry(source->midx, oid, e))
 			return 1;
 
-	if (!r->objects->packed_git)
+	if (!r->objects->packfiles->packs)
 		return 0;
 
 	list_for_each(pos, &r->objects->packed_git_mru) {
@@ -2343,5 +2338,22 @@ struct packfile_store *packfile_store_new(struct object_database *odb)
 
 void packfile_store_free(struct packfile_store *store)
 {
+	packfile_store_close(store);
+
+	for (struct packed_git *p = store->packs, *next; p; p = next) {
+		next = p->next;
+		free(p);
+	}
+
 	free(store);
 }
+
+void packfile_store_close(struct packfile_store *store)
+{
+	for (struct packed_git *p = store->packs; p; p = p->next) {
+		if (p->do_not_close)
+			BUG("want to close pack marked 'do-not-close'");
+		else
+			close_pack(p);
+	}
+}
diff --git a/packfile.h b/packfile.h
index 8d31fd619a..d7ac8d24b4 100644
--- a/packfile.h
+++ b/packfile.h
@@ -57,6 +57,12 @@ struct packed_git {
  */
 struct packfile_store {
 	struct object_database *odb;
+
+	/*
+	 * The list of packfiles in the order in which they are being added to
+	 * the store.
+	 */
+	struct packed_git *packs;
 };
 
 /*
@@ -66,10 +72,17 @@ struct packfile_store {
 struct packfile_store *packfile_store_new(struct object_database *odb);
 
 /*
- * Free the packfile store and all its associated state.
+ * Free the packfile store and all its associated state. All packfiles
+ * tracked by the store will be closed.
  */
 void packfile_store_free(struct packfile_store *store);
 
+/*
+ * Close all packfiles associated with this store. The packfiles won't be
+ * free'd, so they can be re-opened at a later point in time.
+ */
+void packfile_store_close(struct packfile_store *store);
+
 static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
 				     const struct hashmap_entry *entry,
 				     const struct hashmap_entry *entry2,
-- 
2.51.0.384.g4c02a37b29.dirty
Karthik Nayak· Sep 9, 2025, 08:00 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v3 02/15] odb: move list of packfiles into `struct packfile_store`

Patrick Steinhardt <ps@pks.im> writes:
Show 9 quoted lines
> @@ -789,8 +784,8 @@ void install_packed_git(struct repository *r, struct packed_git *pack)
>  	if (pack->pack_fd != -1)
>  		pack_open_fds++;
>
> -	pack->next = r->objects->packed_git;
> -	r->objects->packed_git = pack;
> +	pack->next = r->objects->packfiles->packs;
> +	r->objects->packfiles->packs = pack;
>

So this is where we insert the packfile into the packfile_store. Okay makes sense.

Show 7 quoted lines
> +/*
> + * Close all packfiles associated with this store. The packfiles won't be
> + * free'd, so they can be re-opened at a later point in time.
> + */
> +void packfile_store_close(struct packfile_store *store);
> +
>

One immediate question is, how can someone reopen the closed packfiles. Is that something that the packfile store should offer?

Show 6 quoted lines
>  static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
>  				     const struct hashmap_entry *entry,
>  				     const struct hashmap_entry *entry2,
>
> --
> 2.51.0.384.g4c02a37b29.dirty
Patrick Steinhardt· Sep 9, 2025, 11:09 UTC · re: Karthik Nayak · lore

Re: [PATCH v3 02/15] odb: move list of packfiles into `struct packfile_store`

On Tue, Sep 09, 2025 at 01:00:53AM -0700, Karthik Nayak wrote:
Show 25 quoted lines
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > @@ -789,8 +784,8 @@ void install_packed_git(struct repository *r, struct packed_git *pack)
> >  	if (pack->pack_fd != -1)
> >  		pack_open_fds++;
> >
> > -	pack->next = r->objects->packed_git;
> > -	r->objects->packed_git = pack;
> > +	pack->next = r->objects->packfiles->packs;
> > +	r->objects->packfiles->packs = pack;
> >
> 
> So this is where we insert the packfile into the packfile_store. Okay
> makes sense.
> 
> > +/*
> > + * Close all packfiles associated with this store. The packfiles won't be
> > + * free'd, so they can be re-opened at a later point in time.
> > + */
> > +void packfile_store_close(struct packfile_store *store);
> > +
> >
> 
> One immediate question is, how can someone reopen the closed packfiles.
> Is that something that the packfile store should offer?

There is no specific use case for this right now. Packfiles get reopened automatically as needed when looking up objects.

Patrick
Patrick Steinhardt· Sep 2, 2025, 10:48 UTC · re: Patrick Steinhardt · lore

[PATCH v3 03/15] odb: move initialization bit into `struct packfile_store`

The object database knows to skip re-initializing the list of packfiles in case it's already been initialized. Whether or not that is the case is tracked via a separate `initialized` bit that is stored in the object database. With the introduction of the `struct packfile_store` we have a better place to host this bit though.

Move it accordingly. While at it, convert the field into a boolean now that we're allowed to use them in our code base.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb.h      | 6 ------
 packfile.c | 6 +++---
 packfile.h | 6 ++++++
 3 files changed, 9 insertions(+), 9 deletions(-)
Show changes to 3 files +9 −9

odb.h, packfile.c, packfile.h

diff --git a/odb.h b/odb.h
index 22a170b434..bf1b4d4677 100644
--- a/odb.h
+++ b/odb.h
@@ -169,12 +169,6 @@ struct object_database {
 	unsigned long approximate_object_count;
 	unsigned approximate_object_count_valid : 1;
 
-	/*
-	 * Whether packed_git has already been populated with this repository's
-	 * packs.
-	 */
-	unsigned packed_git_initialized : 1;
-
 	/*
 	 * Submodule source paths that will be added as additional sources to
 	 * allow lookup of submodule objects via the main object database.
diff --git a/packfile.c b/packfile.c
index 43e9a7cb45..0cfeb68b6b 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1027,7 +1027,7 @@ static void prepare_packed_git(struct repository *r)
 {
 	struct odb_source *source;
 
-	if (r->objects->packed_git_initialized)
+	if (r->objects->packfiles->initialized)
 		return;
 
 	odb_prepare_alternates(r->objects);
@@ -1038,7 +1038,7 @@ static void prepare_packed_git(struct repository *r)
 	rearrange_packed_git(r);
 
 	prepare_packed_git_mru(r);
-	r->objects->packed_git_initialized = 1;
+	r->objects->packfiles->initialized = true;
 }
 
 void reprepare_packed_git(struct repository *r)
@@ -1060,7 +1060,7 @@ void reprepare_packed_git(struct repository *r)
 		odb_clear_loose_cache(source);
 
 	r->objects->approximate_object_count_valid = 0;
-	r->objects->packed_git_initialized = 0;
+	r->objects->packfiles->initialized = false;
 	prepare_packed_git(r);
 	obj_read_unlock();
 }
diff --git a/packfile.h b/packfile.h
index d7ac8d24b4..cf81091175 100644
--- a/packfile.h
+++ b/packfile.h
@@ -63,6 +63,12 @@ struct packfile_store {
 	 * the store.
 	 */
 	struct packed_git *packs;
+
+	/*
+	 * Whether packfiles have already been populated with this store's
+	 * packs.
+	 */
+	bool initialized;
 };
 
 /*
-- 
2.51.0.384.g4c02a37b29.dirty
Patrick Steinhardt· Sep 2, 2025, 10:48 UTC · re: Patrick Steinhardt · lore

[PATCH v3 04/15] odb: move packfile map into `struct packfile_store`

The object database tracks a map of packfiles by their respective paths, which is used to figure out whether a given packfile has already been loaded.With the introduction of the `struct packfile_store` we have a better place to host this list though.

Move the map accordingly. `pack_map_entry_cmp()` isn't used anywhere but in "packfile.c" anymore after this change, so we convert it to a static function, as well.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 midx.c     |  2 +-
 odb.c      |  2 --
 odb.h      |  6 ------
 packfile.c | 20 ++++++++++++++++++--
 packfile.h | 20 ++++++--------------
 5 files changed, 25 insertions(+), 25 deletions(-)
Show changes to 5 files +25 −25

midx.c, odb.c, odb.h, packfile.c, packfile.h

diff --git a/midx.c b/midx.c
index 7726c13d7e..e96970efbf 100644
--- a/midx.c
+++ b/midx.c
@@ -460,7 +460,7 @@ int prepare_midx_pack(struct multi_pack_index *m,
 	strbuf_addbuf(&key, &pack_name);
 	strbuf_strip_suffix(&key, ".idx");
 	strbuf_addstr(&key, ".pack");
-	p = hashmap_get_entry_from_hash(&r->objects->pack_map,
+	p = hashmap_get_entry_from_hash(&r->objects->packfiles->map,
 					strhash(key.buf), key.buf,
 					struct packed_git, packmap_ent);
 	if (!p) {
diff --git a/odb.c b/odb.c
index 7201d01406..737d98c911 100644
--- a/odb.c
+++ b/odb.c
@@ -998,7 +998,6 @@ struct object_database *odb_new(struct repository *repo)
 	o->repo = repo;
 	o->packfiles = packfile_store_new(o);
 	INIT_LIST_HEAD(&o->packed_git_mru);
-	hashmap_init(&o->pack_map, pack_map_entry_cmp, NULL, 0);
 	pthread_mutex_init(&o->replace_mutex, NULL);
 	string_list_init_dup(&o->submodule_source_paths);
 	return o;
@@ -1041,6 +1040,5 @@ void odb_clear(struct object_database *o)
 	packfile_store_free(o->packfiles);
 	o->packfiles = NULL;
 
-	hashmap_clear(&o->pack_map);
 	string_list_clear(&o->submodule_source_paths, 0);
 }
diff --git a/odb.h b/odb.h
index bf1b4d4677..73a669b993 100644
--- a/odb.h
+++ b/odb.h
@@ -155,12 +155,6 @@ struct object_database {
 	struct cached_object_entry *cached_objects;
 	size_t cached_object_nr, cached_object_alloc;
 
-	/*
-	 * A map of packfiles to packed_git structs for tracking which
-	 * packs have been loaded already.
-	 */
-	struct hashmap pack_map;
-
 	/*
 	 * A fast, rough count of the number of objects in the repository.
 	 * These two fields are not meant for direct access. Use
diff --git a/packfile.c b/packfile.c
index 0cfeb68b6b..60ccdfaafb 100644
--- a/packfile.c
+++ b/packfile.c
@@ -788,7 +788,7 @@ void install_packed_git(struct repository *r, struct packed_git *pack)
 	r->objects->packfiles->packs = pack;
 
 	hashmap_entry_init(&pack->packmap_ent, strhash(pack->pack_name));
-	hashmap_add(&r->objects->pack_map, &pack->packmap_ent);
+	hashmap_add(&r->objects->packfiles->map, &pack->packmap_ent);
 }
 
 void (*report_garbage)(unsigned seen_bits, const char *path);
@@ -901,7 +901,7 @@ static void prepare_pack(const char *full_name, size_t full_name_len,
 		hashmap_entry_init(&hent, hash);
 
 		/* Don't reopen a pack we already have. */
-		if (!hashmap_get(&data->r->objects->pack_map, &hent, pack_name)) {
+		if (!hashmap_get(&data->r->objects->packfiles->map, &hent, pack_name)) {
 			p = add_packed_git(data->r, full_name, full_name_len, data->local);
 			if (p)
 				install_packed_git(data->r, p);
@@ -2328,11 +2328,26 @@ int parse_pack_header_option(const char *in, unsigned char *out, unsigned int *l
 	return 0;
 }
 
+static int pack_map_entry_cmp(const void *cmp_data UNUSED,
+			      const struct hashmap_entry *entry,
+			      const struct hashmap_entry *entry2,
+			      const void *keydata)
+{
+	const char *key = keydata;
+	const struct packed_git *pg1, *pg2;
+
+	pg1 = container_of(entry, const struct packed_git, packmap_ent);
+	pg2 = container_of(entry2, const struct packed_git, packmap_ent);
+
+	return strcmp(pg1->pack_name, key ? key : pg2->pack_name);
+}
+
 struct packfile_store *packfile_store_new(struct object_database *odb)
 {
 	struct packfile_store *store;
 	CALLOC_ARRAY(store, 1);
 	store->odb = odb;
+	hashmap_init(&store->map, pack_map_entry_cmp, NULL, 0);
 	return store;
 }
 
@@ -2345,6 +2360,7 @@ void packfile_store_free(struct packfile_store *store)
 		free(p);
 	}
 
+	hashmap_clear(&store->map);
 	free(store);
 }
 
diff --git a/packfile.h b/packfile.h
index cf81091175..9bbef51164 100644
--- a/packfile.h
+++ b/packfile.h
@@ -64,6 +64,12 @@ struct packfile_store {
 	 */
 	struct packed_git *packs;
 
+	/*
+	 * A map of packfile names to packed_git structs for tracking which
+	 * packs have been loaded already.
+	 */
+	struct hashmap map;
+
 	/*
 	 * Whether packfiles have already been populated with this store's
 	 * packs.
@@ -89,20 +95,6 @@ void packfile_store_free(struct packfile_store *store);
  */
 void packfile_store_close(struct packfile_store *store);
 
-static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
-				     const struct hashmap_entry *entry,
-				     const struct hashmap_entry *entry2,
-				     const void *keydata)
-{
-	const char *key = keydata;
-	const struct packed_git *pg1, *pg2;
-
-	pg1 = container_of(entry, const struct packed_git, packmap_ent);
-	pg2 = container_of(entry2, const struct packed_git, packmap_ent);
-
-	return strcmp(pg1->pack_name, key ? key : pg2->pack_name);
-}
-
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
-- 
2.51.0.384.g4c02a37b29.dirty
Karthik Nayak· Sep 9, 2025, 08:22 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v3 04/15] odb: move packfile map into `struct packfile_store`

Patrick Steinhardt <ps@pks.im> writes:
Show 5 quoted lines
> The object database tracks a map of packfiles by their respective paths,
> which is used to figure out whether a given packfile has already been
> loaded.With the introduction of the `struct packfile_store` we have a
> better place to host this list though.
>
Nit: missing space after 'loaded.'
> Move the map accordingly. `pack_map_entry_cmp()` isn't used anywhere but
> in "packfile.c" anymore after this change, so we convert it to a static
> function, as well.
>
Nit: Okay so we move it from the header to the '.c' file, since it is
only used there. But we do loose the 'inline' hint, we don't talk about
that. Any reason for that?
Patrick Steinhardt· Sep 9, 2025, 11:01 UTC · re: Karthik Nayak · lore

Re: [PATCH v3 04/15] odb: move packfile map into `struct packfile_store`

On Tue, Sep 09, 2025 at 01:22:33AM -0700, Karthik Nayak wrote:
Show 18 quoted lines
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > The object database tracks a map of packfiles by their respective paths,
> > which is used to figure out whether a given packfile has already been
> > loaded.With the introduction of the `struct packfile_store` we have a
> > better place to host this list though.
> >
> 
> Nit: missing space after 'loaded.'
> 
> > Move the map accordingly. `pack_map_entry_cmp()` isn't used anywhere but
> > in "packfile.c" anymore after this change, so we convert it to a static
> > function, as well.
> >
> 
> Nit: Okay so we move it from the header to the '.c' file, since it is
> only used there. But we do loose the 'inline' hint, we don't talk about
> that. Any reason for that?

Good catch. The `inline` hint didn't really make any sense in the first place: it's used as a callback function, so by definition it cannot be inlined anyway.

Will amend the commit message.
Patrick
Patrick Steinhardt· Sep 2, 2025, 10:48 UTC · re: Patrick Steinhardt · lore

[PATCH v3 05/15] odb: move MRU list of packfiles into `struct packfile_store`

The object database tracks the list of packfiles in most-recently-used order, which is mostly used to favor reading from packfiles that contain most of the objects that we're currently accessing. With the introduction of the `struct packfile_store` we have a better place to host this list though.

Move the list accordingly.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 midx.c     |  2 +-
 odb.c      |  2 --
 odb.h      |  4 ----
 packfile.c | 11 ++++++-----
 packfile.h |  3 +++
 5 files changed, 10 insertions(+), 12 deletions(-)
Show changes to 5 files +10 −12

midx.c, odb.c, odb.h, packfile.c, packfile.h

diff --git a/midx.c b/midx.c
index e96970efbf..91c7b3917d 100644
--- a/midx.c
+++ b/midx.c
@@ -468,7 +468,7 @@ int prepare_midx_pack(struct multi_pack_index *m,
 				   m->source->local);
 		if (p) {
 			install_packed_git(r, p);
-			list_add_tail(&p->mru, &r->objects->packed_git_mru);
+			list_add_tail(&p->mru, &r->objects->packfiles->mru);
 		}
 	}
 
diff --git a/odb.c b/odb.c
index 737d98c911..32e982bf0b 100644
--- a/odb.c
+++ b/odb.c
@@ -997,7 +997,6 @@ struct object_database *odb_new(struct repository *repo)
 	memset(o, 0, sizeof(*o));
 	o->repo = repo;
 	o->packfiles = packfile_store_new(o);
-	INIT_LIST_HEAD(&o->packed_git_mru);
 	pthread_mutex_init(&o->replace_mutex, NULL);
 	string_list_init_dup(&o->submodule_source_paths);
 	return o;
@@ -1035,7 +1034,6 @@ void odb_clear(struct object_database *o)
 		free((char *) o->cached_objects[i].value.buf);
 	FREE_AND_NULL(o->cached_objects);
 
-	INIT_LIST_HEAD(&o->packed_git_mru);
 	close_object_store(o);
 	packfile_store_free(o->packfiles);
 	o->packfiles = NULL;
diff --git a/odb.h b/odb.h
index 73a669b993..8ee1f8bb43 100644
--- a/odb.h
+++ b/odb.h
@@ -3,7 +3,6 @@
 
 #include "hashmap.h"
 #include "object.h"
-#include "list.h"
 #include "oidset.h"
 #include "oidmap.h"
 #include "string-list.h"
@@ -138,9 +137,6 @@ struct object_database {
 	 * should only be accessed directly by packfile.c
 	 */
 	struct packfile_store *packfiles;
-	/* A most-recently-used ordered version of the packed_git list. */
-	struct list_head packed_git_mru;
-
 	struct {
 		struct packed_git **packs;
 		unsigned flags;
diff --git a/packfile.c b/packfile.c
index 60ccdfaafb..98207aa380 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1017,10 +1017,10 @@ static void prepare_packed_git_mru(struct repository *r)
 {
 	struct packed_git *p;
 
-	INIT_LIST_HEAD(&r->objects->packed_git_mru);
+	INIT_LIST_HEAD(&r->objects->packfiles->mru);
 
 	for (p = r->objects->packfiles->packs; p; p = p->next)
-		list_add_tail(&p->mru, &r->objects->packed_git_mru);
+		list_add_tail(&p->mru, &r->objects->packfiles->mru);
 }
 
 static void prepare_packed_git(struct repository *r)
@@ -1095,7 +1095,7 @@ struct packed_git *get_all_packs(struct repository *r)
 struct list_head *get_packed_git_mru(struct repository *r)
 {
 	prepare_packed_git(r);
-	return &r->objects->packed_git_mru;
+	return &r->objects->packfiles->mru;
 }
 
 unsigned long unpack_object_header_buffer(const unsigned char *buf,
@@ -2078,10 +2078,10 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 	if (!r->objects->packfiles->packs)
 		return 0;
 
-	list_for_each(pos, &r->objects->packed_git_mru) {
+	list_for_each(pos, &r->objects->packfiles->mru) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		if (!p->multi_pack_index && fill_pack_entry(oid, e, p)) {
-			list_move(&p->mru, &r->objects->packed_git_mru);
+			list_move(&p->mru, &r->objects->packfiles->mru);
 			return 1;
 		}
 	}
@@ -2347,6 +2347,7 @@ struct packfile_store *packfile_store_new(struct object_database *odb)
 	struct packfile_store *store;
 	CALLOC_ARRAY(store, 1);
 	store->odb = odb;
+	INIT_LIST_HEAD(&store->mru);
 	hashmap_init(&store->map, pack_map_entry_cmp, NULL, 0);
 	return store;
 }
diff --git a/packfile.h b/packfile.h
index 9bbef51164..d48d46cc1b 100644
--- a/packfile.h
+++ b/packfile.h
@@ -64,6 +64,9 @@ struct packfile_store {
 	 */
 	struct packed_git *packs;
 
+	/* A most-recently-used ordered version of the packs list. */
+	struct list_head mru;
+
 	/*
 	 * A map of packfile names to packed_git structs for tracking which
 	 * packs have been loaded already.
-- 
2.51.0.384.g4c02a37b29.dirty
Patrick Steinhardt· Sep 2, 2025, 10:48 UTC · re: Patrick Steinhardt · lore

[PATCH v3 06/15] odb: move kept cache into `struct packfile_store`

The object database tracks a cache of "kept" packfiles, which is used by git-pack-objects(1) to handle cruft objects. With the introduction of the `struct packfile_store` we have a better place to host this cache though.

Move the cache accordingly.

This moves the last bit of packfile-related state from the object database into the packfile store. Adapt the comment for the `packfiles` pointer in `struct object_database` to reflect this.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb.h      |  8 +-------
 packfile.c | 16 ++++++++--------
 packfile.h | 14 ++++++++++++++
 3 files changed, 23 insertions(+), 15 deletions(-)
Show changes to 3 files +23 −15

odb.h, packfile.c, packfile.h

diff --git a/odb.h b/odb.h
index 8ee1f8bb43..1c998a2478 100644
--- a/odb.h
+++ b/odb.h
@@ -132,15 +132,9 @@ struct object_database {
 	unsigned commit_graph_attempted : 1; /* if loading has been attempted */
 
 	/*
-	 * private data
-	 *
-	 * should only be accessed directly by packfile.c
+	 * Should only be accessed directly by packfile.c
 	 */
 	struct packfile_store *packfiles;
-	struct {
-		struct packed_git **packs;
-		unsigned flags;
-	} kept_pack_cache;
 
 	/*
 	 * This is meant to hold a *small* number of objects that you would
diff --git a/packfile.c b/packfile.c
index 98207aa380..6ae7f22d65 100644
--- a/packfile.c
+++ b/packfile.c
@@ -2091,19 +2091,19 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 static void maybe_invalidate_kept_pack_cache(struct repository *r,
 					     unsigned flags)
 {
-	if (!r->objects->kept_pack_cache.packs)
+	if (!r->objects->packfiles->kept_cache.packs)
 		return;
-	if (r->objects->kept_pack_cache.flags == flags)
+	if (r->objects->packfiles->kept_cache.flags == flags)
 		return;
-	FREE_AND_NULL(r->objects->kept_pack_cache.packs);
-	r->objects->kept_pack_cache.flags = 0;
+	FREE_AND_NULL(r->objects->packfiles->kept_cache.packs);
+	r->objects->packfiles->kept_cache.flags = 0;
 }
 
 struct packed_git **kept_pack_cache(struct repository *r, unsigned flags)
 {
 	maybe_invalidate_kept_pack_cache(r, flags);
 
-	if (!r->objects->kept_pack_cache.packs) {
+	if (!r->objects->packfiles->kept_cache.packs) {
 		struct packed_git **packs = NULL;
 		size_t nr = 0, alloc = 0;
 		struct packed_git *p;
@@ -2126,11 +2126,11 @@ struct packed_git **kept_pack_cache(struct repository *r, unsigned flags)
 		ALLOC_GROW(packs, nr + 1, alloc);
 		packs[nr] = NULL;
 
-		r->objects->kept_pack_cache.packs = packs;
-		r->objects->kept_pack_cache.flags = flags;
+		r->objects->packfiles->kept_cache.packs = packs;
+		r->objects->packfiles->kept_cache.flags = flags;
 	}
 
-	return r->objects->kept_pack_cache.packs;
+	return r->objects->packfiles->kept_cache.packs;
 }
 
 int find_kept_pack_entry(struct repository *r,
diff --git a/packfile.h b/packfile.h
index d48d46cc1b..bf66211986 100644
--- a/packfile.h
+++ b/packfile.h
@@ -64,6 +64,20 @@ struct packfile_store {
 	 */
 	struct packed_git *packs;
 
+	/*
+	 * Cache of packfiles which are marked as "kept", either because there
+	 * is an on-disk ".keep" file or because they are marked as "kept" in
+	 * memory.
+	 *
+	 * Should not be accessed directly, but via `kept_pack_cache()`. The
+	 * list of packs gets invalidated when the stored flags and the flags
+	 * passed to `kept_pack_cache()` mismatch.
+	 */
+	struct {
+		struct packed_git **packs;
+		unsigned flags;
+	} kept_cache;
+
 	/* A most-recently-used ordered version of the packs list. */
 	struct list_head mru;
 
-- 
2.51.0.384.g4c02a37b29.dirty
Patrick Steinhardt· Sep 2, 2025, 10:48 UTC · re: Patrick Steinhardt · lore

[PATCH v3 07/15] packfile: reorder functions to avoid function declaration

Reorder functions so that we can avoid a forward declaration of `prepare_packed_git()`.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 packfile.c | 67 +++++++++++++++++++++++++++++++-------------------------------
 1 file changed, 33 insertions(+), 34 deletions(-)
Show changes to packfile.c +33 −34
diff --git a/packfile.c b/packfile.c
index 6ae7f22d65..771b58df8b 100644
--- a/packfile.c
+++ b/packfile.c
@@ -946,40 +946,6 @@ static void prepare_packed_git_one(struct odb_source *source)
 	string_list_clear(data.garbage, 0);
 }
 
-static void prepare_packed_git(struct repository *r);
-/*
- * Give a fast, rough count of the number of objects in the repository. This
- * ignores loose objects completely. If you have a lot of them, then either
- * you should repack because your performance will be awful, or they are
- * all unreachable objects about to be pruned, in which case they're not really
- * interesting as a measure of repo size in the first place.
- */
-unsigned long repo_approximate_object_count(struct repository *r)
-{
-	if (!r->objects->approximate_object_count_valid) {
-		struct odb_source *source;
-		unsigned long count = 0;
-		struct packed_git *p;
-
-		prepare_packed_git(r);
-
-		for (source = r->objects->sources; source; source = source->next) {
-			struct multi_pack_index *m = get_multi_pack_index(source);
-			if (m)
-				count += m->num_objects;
-		}
-
-		for (p = r->objects->packfiles->packs; p; p = p->next) {
-			if (open_pack_index(p))
-				continue;
-			count += p->num_objects;
-		}
-		r->objects->approximate_object_count = count;
-		r->objects->approximate_object_count_valid = 1;
-	}
-	return r->objects->approximate_object_count;
-}
-
 DEFINE_LIST_SORT(static, sort_packs, struct packed_git, next);
 
 static int sort_pack(const struct packed_git *a, const struct packed_git *b)
@@ -1098,6 +1064,39 @@ struct list_head *get_packed_git_mru(struct repository *r)
 	return &r->objects->packfiles->mru;
 }
 
+/*
+ * Give a fast, rough count of the number of objects in the repository. This
+ * ignores loose objects completely. If you have a lot of them, then either
+ * you should repack because your performance will be awful, or they are
+ * all unreachable objects about to be pruned, in which case they're not really
+ * interesting as a measure of repo size in the first place.
+ */
+unsigned long repo_approximate_object_count(struct repository *r)
+{
+	if (!r->objects->approximate_object_count_valid) {
+		struct odb_source *source;
+		unsigned long count = 0;
+		struct packed_git *p;
+
+		prepare_packed_git(r);
+
+		for (source = r->objects->sources; source; source = source->next) {
+			struct multi_pack_index *m = get_multi_pack_index(source);
+			if (m)
+				count += m->num_objects;
+		}
+
+		for (p = r->objects->packfiles->packs; p; p = p->next) {
+			if (open_pack_index(p))
+				continue;
+			count += p->num_objects;
+		}
+		r->objects->approximate_object_count = count;
+		r->objects->approximate_object_count_valid = 1;
+	}
+	return r->objects->approximate_object_count;
+}
+
 unsigned long unpack_object_header_buffer(const unsigned char *buf,
 		unsigned long len, enum object_type *type, unsigned long *sizep)
 {
-- 
2.51.0.384.g4c02a37b29.dirty
Patrick Steinhardt· Sep 2, 2025, 10:48 UTC · re: Patrick Steinhardt · lore

[PATCH v3 08/15] packfile: refactor `prepare_packed_git()` to work on packfile store

The `prepare_packed_git()` function and its friends are responsible for loading packfiles as well as the multi-pack index for a given object database. Refactor these functions to accept a packfile store instead of a repository to clarify their scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 packfile.c | 41 ++++++++++++++++++-----------------------
 1 file changed, 18 insertions(+), 23 deletions(-)
Show changes to packfile.c +18 −23
diff --git a/packfile.c b/packfile.c
index 771b58df8b..4564026658 100644
--- a/packfile.c
+++ b/packfile.c
@@ -974,37 +974,32 @@ static int sort_pack(const struct packed_git *a, const struct packed_git *b)
 	return -1;
 }
 
-static void rearrange_packed_git(struct repository *r)
-{
-	sort_packs(&r->objects->packfiles->packs, sort_pack);
-}
-
-static void prepare_packed_git_mru(struct repository *r)
+static void packfile_store_prepare_mru(struct packfile_store *store)
 {
 	struct packed_git *p;
 
-	INIT_LIST_HEAD(&r->objects->packfiles->mru);
+	INIT_LIST_HEAD(&store->mru);
 
-	for (p = r->objects->packfiles->packs; p; p = p->next)
-		list_add_tail(&p->mru, &r->objects->packfiles->mru);
+	for (p = store->packs; p; p = p->next)
+		list_add_tail(&p->mru, &store->mru);
 }
 
-static void prepare_packed_git(struct repository *r)
+static void packfile_store_prepare(struct packfile_store *store)
 {
 	struct odb_source *source;
 
-	if (r->objects->packfiles->initialized)
+	if (store->initialized)
 		return;
 
-	odb_prepare_alternates(r->objects);
-	for (source = r->objects->sources; source; source = source->next) {
+	odb_prepare_alternates(store->odb);
+	for (source = store->odb->sources; source; source = source->next) {
 		prepare_multi_pack_index_one(source);
 		prepare_packed_git_one(source);
 	}
-	rearrange_packed_git(r);
+	sort_packs(&store->packs, sort_pack);
 
-	prepare_packed_git_mru(r);
-	r->objects->packfiles->initialized = true;
+	packfile_store_prepare_mru(store);
+	store->initialized = true;
 }
 
 void reprepare_packed_git(struct repository *r)
@@ -1027,25 +1022,25 @@ void reprepare_packed_git(struct repository *r)
 
 	r->objects->approximate_object_count_valid = 0;
 	r->objects->packfiles->initialized = false;
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 	obj_read_unlock();
 }
 
 struct packed_git *get_packed_git(struct repository *r)
 {
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 	return r->objects->packfiles->packs;
 }
 
 struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
 {
-	prepare_packed_git(source->odb->repo);
+	packfile_store_prepare(source->odb->packfiles);
 	return source->midx;
 }
 
 struct packed_git *get_all_packs(struct repository *r)
 {
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 
 	for (struct odb_source *source = r->objects->sources; source; source = source->next) {
 		struct multi_pack_index *m = source->midx;
@@ -1060,7 +1055,7 @@ struct packed_git *get_all_packs(struct repository *r)
 
 struct list_head *get_packed_git_mru(struct repository *r)
 {
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 	return &r->objects->packfiles->mru;
 }
 
@@ -1078,7 +1073,7 @@ unsigned long repo_approximate_object_count(struct repository *r)
 		unsigned long count = 0;
 		struct packed_git *p;
 
-		prepare_packed_git(r);
+		packfile_store_prepare(r->objects->packfiles);
 
 		for (source = r->objects->sources; source; source = source->next) {
 			struct multi_pack_index *m = get_multi_pack_index(source);
@@ -2068,7 +2063,7 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 {
 	struct list_head *pos;
 
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 
 	for (struct odb_source *source = r->objects->sources; source; source = source->next)
 		if (source->midx && fill_midx_entry(source->midx, oid, e))
-- 
2.51.0.384.g4c02a37b29.dirty
Patrick Steinhardt· Sep 2, 2025, 10:48 UTC · re: Patrick Steinhardt · lore

[PATCH v3 09/15] packfile: split up responsibilities of `reprepare_packed_git()`

In `reprepare_packed_git()` we perform a couple of operations:
  - We reload alternate object directories.
  - We clear the loose object cache.
  - We reprepare packfiles.

While the logic is hosted in "packfile.c", it clearly reaches into other subsystems that aren't related to packfiles.

Split up the responsibility and introduce `odb_reprepare()` which now becomes responsible for repreparing the whole object database. The existing `reprepare_packed_git()` function is refactored accordingly and only cares about reloading the packfile store now.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/backfill.c     |  2 +-
 builtin/gc.c           |  4 ++--
 builtin/receive-pack.c |  2 +-
 builtin/repack.c       |  2 +-
 bulk-checkin.c         |  2 +-
 connected.c            |  2 +-
 fetch-pack.c           |  4 ++--
 object-name.c          |  2 +-
 odb.c                  | 27 ++++++++++++++++++++++++++-
 odb.h                  |  6 ++++++
 packfile.c             | 26 ++++----------------------
 packfile.h             |  9 ++++++++-
 transport-helper.c     |  2 +-
 13 files changed, 55 insertions(+), 35 deletions(-)
Show changes to 13 files +55 −35

builtin/backfill.c, builtin/gc.c, builtin/receive-pack.c, builtin/repack.c, bulk-checkin.c, connected.c, fetch-pack.c, object-name.c, odb.c, odb.h, packfile.c, packfile.h, transport-helper.c

diff --git a/builtin/backfill.c b/builtin/backfill.c
index 80056abe47..e80fc1b694 100644
--- a/builtin/backfill.c
+++ b/builtin/backfill.c
@@ -53,7 +53,7 @@ static void download_batch(struct backfill_context *ctx)
 	 * We likely have a new packfile. Add it to the packed list to
 	 * avoid possible duplicate downloads of the same objects.
 	 */
-	reprepare_packed_git(ctx->repo);
+	odb_reprepare(ctx->repo->objects);
 }
 
 static int fill_missing_blobs(const char *path UNUSED,
diff --git a/builtin/gc.c b/builtin/gc.c
index 03ae4926b2..aeca06a08b 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1042,7 +1042,7 @@ int cmd_gc(int argc,
 		die(FAILED_RUN, "rerere");
 
 	report_garbage = report_pack_garbage;
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 	if (pack_garbage.nr > 0) {
 		close_object_store(the_repository->objects);
 		clean_pack_garbage();
@@ -1491,7 +1491,7 @@ static off_t get_auto_pack_size(void)
 	struct packed_git *p;
 	struct repository *r = the_repository;
 
-	reprepare_packed_git(r);
+	odb_reprepare(r->objects);
 	for (p = get_all_packs(r); p; p = p->next) {
 		if (p->pack_size > max_size) {
 			second_largest_size = max_size;
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 1113137a6f..c9288a9c7e 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -2389,7 +2389,7 @@ static const char *unpack(int err_fd, struct shallow_info *si)
 		status = finish_command(&child);
 		if (status)
 			return "index-pack abnormal exit";
-		reprepare_packed_git(the_repository);
+		odb_reprepare(the_repository->objects);
 	}
 	return NULL;
 }
diff --git a/builtin/repack.c b/builtin/repack.c
index c490a51e91..5ff27fc8e2 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -1685,7 +1685,7 @@ int cmd_repack(int argc,
 			goto cleanup;
 	}
 
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 
 	if (delete_redundant) {
 		int opts = 0;
diff --git a/bulk-checkin.c b/bulk-checkin.c
index b2809ab039..f65439a748 100644
--- a/bulk-checkin.c
+++ b/bulk-checkin.c
@@ -90,7 +90,7 @@ static void flush_bulk_checkin_packfile(struct bulk_checkin_packfile *state)
 
 	strbuf_release(&packname);
 	/* Make objects we just wrote available to ourselves */
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 }
 
 /*
diff --git a/connected.c b/connected.c
index 18c13245d8..d6e9682fd9 100644
--- a/connected.c
+++ b/connected.c
@@ -72,7 +72,7 @@ int check_connected(oid_iterate_fn fn, void *cb_data,
 		 * Before checking for promisor packs, be sure we have the
 		 * latest pack-files loaded into memory.
 		 */
-		reprepare_packed_git(the_repository);
+		odb_reprepare(the_repository->objects);
 		do {
 			struct packed_git *p;
 
diff --git a/fetch-pack.c b/fetch-pack.c
index 6ed5662951..fe7a84bf2f 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -1983,7 +1983,7 @@ static void update_shallow(struct fetch_pack_args *args,
 		 * remote is shallow, but this is a clone, there are
 		 * no objects in repo to worry about. Accept any
 		 * shallow points that exist in the pack (iow in repo
-		 * after get_pack() and reprepare_packed_git())
+		 * after get_pack() and odb_reprepare())
 		 */
 		struct oid_array extra = OID_ARRAY_INIT;
 		struct object_id *oid = si->shallow->oid;
@@ -2108,7 +2108,7 @@ struct ref *fetch_pack(struct fetch_pack_args *args,
 		ref_cpy = do_fetch_pack(args, fd, ref, sought, nr_sought,
 					&si, pack_lockfiles);
 	}
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 
 	if (!args->cloning && args->deepen) {
 		struct check_connected_options opt = CHECK_CONNECTED_INIT;
diff --git a/object-name.c b/object-name.c
index 732056ff5e..df9e0c5f02 100644
--- a/object-name.c
+++ b/object-name.c
@@ -596,7 +596,7 @@ static enum get_oid_result get_short_oid(struct repository *r,
 	 * or migrated from loose to packed.
 	 */
 	if (status == MISSING_OBJECT) {
-		reprepare_packed_git(r);
+		odb_reprepare(r->objects);
 		find_short_object_filename(&ds);
 		find_short_packed_object(&ds);
 		status = finish_object_disambiguation(&ds, oid);
diff --git a/odb.c b/odb.c
index 32e982bf0b..65a6cc67b6 100644
--- a/odb.c
+++ b/odb.c
@@ -694,7 +694,7 @@ static int do_oid_object_info_extended(struct object_database *odb,
 
 		/* Not a loose object; someone else may have just packed it. */
 		if (!(flags & OBJECT_INFO_QUICK)) {
-			reprepare_packed_git(odb->repo);
+			odb_reprepare(odb->repo->objects);
 			if (find_pack_entry(odb->repo, real, &e))
 				break;
 		}
@@ -1040,3 +1040,28 @@ void odb_clear(struct object_database *o)
 
 	string_list_clear(&o->submodule_source_paths, 0);
 }
+
+void odb_reprepare(struct object_database *o)
+{
+	struct odb_source *source;
+
+	obj_read_lock();
+
+	/*
+	 * Reprepare alt odbs, in case the alternates file was modified
+	 * during the course of this process. This only _adds_ odbs to
+	 * the linked list, so existing odbs will continue to exist for
+	 * the lifetime of the process.
+	 */
+	o->loaded_alternates = 0;
+	odb_prepare_alternates(o);
+
+	for (source = o->sources; source; source = source->next)
+		odb_clear_loose_cache(source);
+
+	o->approximate_object_count_valid = 0;
+
+	packfile_store_reprepare(o->packfiles);
+
+	obj_read_unlock();
+}
diff --git a/odb.h b/odb.h
index 1c998a2478..ef34132c58 100644
--- a/odb.h
+++ b/odb.h
@@ -163,6 +163,12 @@ struct object_database {
 struct object_database *odb_new(struct repository *repo);
 void odb_clear(struct object_database *o);
 
+/*
+ * Clear caches, reload alternates and then reload object sources so that new
+ * objects may become accessible.
+ */
+void odb_reprepare(struct object_database *o);
+
 /*
  * Find source by its object directory path. Returns a `NULL` pointer in case
  * the source could not be found.
diff --git a/packfile.c b/packfile.c
index 4564026658..edd5ecc9cf 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1002,28 +1002,10 @@ static void packfile_store_prepare(struct packfile_store *store)
 	store->initialized = true;
 }
 
-void reprepare_packed_git(struct repository *r)
+void packfile_store_reprepare(struct packfile_store *store)
 {
-	struct odb_source *source;
-
-	obj_read_lock();
-
-	/*
-	 * Reprepare alt odbs, in case the alternates file was modified
-	 * during the course of this process. This only _adds_ odbs to
-	 * the linked list, so existing odbs will continue to exist for
-	 * the lifetime of the process.
-	 */
-	r->objects->loaded_alternates = 0;
-	odb_prepare_alternates(r->objects);
-
-	for (source = r->objects->sources; source; source = source->next)
-		odb_clear_loose_cache(source);
-
-	r->objects->approximate_object_count_valid = 0;
-	r->objects->packfiles->initialized = false;
-	packfile_store_prepare(r->objects->packfiles);
-	obj_read_unlock();
+	store->initialized = false;
+	packfile_store_prepare(store);
 }
 
 struct packed_git *get_packed_git(struct repository *r)
@@ -1144,7 +1126,7 @@ unsigned long get_size_from_delta(struct packed_git *p,
 		 *
 		 * Other worrying sections could be the call to close_pack_fd(),
 		 * which can close packs even with in-use windows, and to
-		 * reprepare_packed_git(). Regarding the former, mmap doc says:
+		 * odb_reprepare(). Regarding the former, mmap doc says:
 		 * "closing the file descriptor does not unmap the region". And
 		 * for the latter, it won't re-open already available packs.
 		 */
diff --git a/packfile.h b/packfile.h
index bf66211986..a85ff607fe 100644
--- a/packfile.h
+++ b/packfile.h
@@ -112,6 +112,14 @@ void packfile_store_free(struct packfile_store *store);
  */
 void packfile_store_close(struct packfile_store *store);
 
+/*
+ * Clear the packfile caches and try to look up any new packfiles that have
+ * appeared since last preparing the packfiles store.
+ *
+ * This function must be called under the `odb_read_lock()`.
+ */
+void packfile_store_reprepare(struct packfile_store *store);
+
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
@@ -188,7 +196,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-void reprepare_packed_git(struct repository *r);
 void install_packed_git(struct repository *r, struct packed_git *pack);
 
 struct packed_git *get_packed_git(struct repository *r);
diff --git a/transport-helper.c b/transport-helper.c
index 0789e5bca5..4d95d84f9e 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -450,7 +450,7 @@ static int fetch_with_fetch(struct transport *transport,
 	}
 	strbuf_release(&buf);
 
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 	return 0;
 }
 
-- 
2.51.0.384.g4c02a37b29.dirty
Patrick Steinhardt· Sep 2, 2025, 10:48 UTC · re: Patrick Steinhardt · lore

[PATCH v3 10/15] packfile: refactor `install_packed_git()` to work on packfile store

The `install_packed_git()` functions adds a packfile to a specific object store. Refactor it to accept a packfile store instead of a repository to clarify its scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/fast-import.c |  2 +-
 builtin/index-pack.c  |  2 +-
 http.c                |  2 +-
 http.h                |  2 +-
 midx.c                |  2 +-
 packfile.c            | 11 ++++++-----
 packfile.h            |  9 +++++++--
 7 files changed, 18 insertions(+), 12 deletions(-)
Show changes to 7 files +18 −12

builtin/fast-import.c, builtin/index-pack.c, http.c, http.h, midx.c, packfile.c, packfile.h

diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index 2c35f9345d..e9d82b31c3 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -901,7 +901,7 @@ static void end_packfile(void)
 		if (!new_p)
 			die("core git rejected index %s", idx_name);
 		all_packs[pack_id] = new_p;
-		install_packed_git(the_repository, new_p);
+		packfile_store_add_pack(the_repository->objects->packfiles, new_p);
 		free(idx_name);
 
 		/* Print the boundary */
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index f91c301bba..ed490dfad4 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1645,7 +1645,7 @@ static void final(const char *final_pack_name, const char *curr_pack_name,
 		p = add_packed_git(the_repository, final_index_name,
 				   strlen(final_index_name), 0);
 		if (p)
-			install_packed_git(the_repository, p);
+			packfile_store_add_pack(the_repository->objects->packfiles, p);
 	}
 
 	if (!from_stdin) {
diff --git a/http.c b/http.c
index 98853d6483..af2120b64c 100644
--- a/http.c
+++ b/http.c
@@ -2541,7 +2541,7 @@ void http_install_packfile(struct packed_git *p,
 		lst = &((*lst)->next);
 	*lst = (*lst)->next;
 
-	install_packed_git(the_repository, p);
+	packfile_store_add_pack(the_repository->objects->packfiles, p);
 }
 
 struct http_pack_request *new_http_pack_request(
diff --git a/http.h b/http.h
index 36202139f4..e5a5380c6c 100644
--- a/http.h
+++ b/http.h
@@ -210,7 +210,7 @@ int finish_http_pack_request(struct http_pack_request *preq);
 void release_http_pack_request(struct http_pack_request *preq);
 
 /*
- * Remove p from the given list, and invoke install_packed_git() on it.
+ * Remove p from the given list, and invoke packfile_store_add_pack() on it.
  *
  * This is a convenience function for users that have obtained a list of packs
  * from http_get_info_packs() and have chosen a specific pack to fetch.
diff --git a/midx.c b/midx.c
index 91c7b3917d..69c44be71c 100644
--- a/midx.c
+++ b/midx.c
@@ -467,7 +467,7 @@ int prepare_midx_pack(struct multi_pack_index *m,
 		p = add_packed_git(r, pack_name.buf, pack_name.len,
 				   m->source->local);
 		if (p) {
-			install_packed_git(r, p);
+			packfile_store_add_pack(r->objects->packfiles, p);
 			list_add_tail(&p->mru, &r->objects->packfiles->mru);
 		}
 	}
diff --git a/packfile.c b/packfile.c
index edd5ecc9cf..e1a3c0487c 100644
--- a/packfile.c
+++ b/packfile.c
@@ -779,16 +779,17 @@ struct packed_git *add_packed_git(struct repository *r, const char *path,
 	return p;
 }
 
-void install_packed_git(struct repository *r, struct packed_git *pack)
+void packfile_store_add_pack(struct packfile_store *store,
+			     struct packed_git *pack)
 {
 	if (pack->pack_fd != -1)
 		pack_open_fds++;
 
-	pack->next = r->objects->packfiles->packs;
-	r->objects->packfiles->packs = pack;
+	pack->next = store->packs;
+	store->packs = pack;
 
 	hashmap_entry_init(&pack->packmap_ent, strhash(pack->pack_name));
-	hashmap_add(&r->objects->packfiles->map, &pack->packmap_ent);
+	hashmap_add(&store->map, &pack->packmap_ent);
 }
 
 void (*report_garbage)(unsigned seen_bits, const char *path);
@@ -904,7 +905,7 @@ static void prepare_pack(const char *full_name, size_t full_name_len,
 		if (!hashmap_get(&data->r->objects->packfiles->map, &hent, pack_name)) {
 			p = add_packed_git(data->r, full_name, full_name_len, data->local);
 			if (p)
-				install_packed_git(data->r, p);
+				packfile_store_add_pack(data->r->objects->packfiles, p);
 		}
 		free(pack_name);
 	}
diff --git a/packfile.h b/packfile.h
index a85ff607fe..ba4b0cef9c 100644
--- a/packfile.h
+++ b/packfile.h
@@ -120,6 +120,13 @@ void packfile_store_close(struct packfile_store *store);
  */
 void packfile_store_reprepare(struct packfile_store *store);
 
+/*
+ * Add the pack to the store so that contained objects become accessible via
+ * the store. This moves ownership into the store.
+ */
+void packfile_store_add_pack(struct packfile_store *store,
+			     struct packed_git *pack);
+
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
@@ -196,8 +203,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-void install_packed_git(struct repository *r, struct packed_git *pack);
-
 struct packed_git *get_packed_git(struct repository *r);
 struct list_head *get_packed_git_mru(struct repository *r);
 struct multi_pack_index *get_multi_pack_index(struct odb_source *source);
-- 
2.51.0.384.g4c02a37b29.dirty
Patrick Steinhardt· Sep 2, 2025, 10:48 UTC · re: Patrick Steinhardt · lore

[PATCH v3 11/15] packfile: introduce function to load and add packfiles

We have a recurring pattern where we essentially perform an upsert of a packfile in case it isn't yet known by the packfile store. The logic to do so is non-trivial as we have to reconstruct the packfile's key, check the map of packfiles, then create the new packfile and finally add it to the store.

Introduce a new function that does this dance for us. Refactor callsites to use it.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/fast-import.c |  4 ++--
 builtin/index-pack.c  | 10 +++-------
 midx.c                | 23 ++++-------------------
 packfile.c            | 44 +++++++++++++++++++++++++++++++-------------
 packfile.h            |  8 ++++++++
 5 files changed, 48 insertions(+), 41 deletions(-)
Show changes to 5 files +48 −41

builtin/fast-import.c, builtin/index-pack.c, midx.c, packfile.c, packfile.h

diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index e9d82b31c3..a26e79689d 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -897,11 +897,11 @@ static void end_packfile(void)
 		idx_name = keep_pack(create_index());
 
 		/* Register the packfile with core git's machinery. */
-		new_p = add_packed_git(pack_data->repo, idx_name, strlen(idx_name), 1);
+		new_p = packfile_store_load_pack(pack_data->repo->objects->packfiles,
+						 idx_name, 1);
 		if (!new_p)
 			die("core git rejected index %s", idx_name);
 		all_packs[pack_id] = new_p;
-		packfile_store_add_pack(the_repository->objects->packfiles, new_p);
 		free(idx_name);
 
 		/* Print the boundary */
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index ed490dfad4..2b78ba7fe4 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1640,13 +1640,9 @@ static void final(const char *final_pack_name, const char *curr_pack_name,
 	rename_tmp_packfile(&final_index_name, curr_index_name, &index_name,
 			    hash, "idx", 1);
 
-	if (do_fsck_object) {
-		struct packed_git *p;
-		p = add_packed_git(the_repository, final_index_name,
-				   strlen(final_index_name), 0);
-		if (p)
-			packfile_store_add_pack(the_repository->objects->packfiles, p);
-	}
+	if (do_fsck_object)
+		packfile_store_load_pack(the_repository->objects->packfiles,
+					 final_index_name, 0);
 
 	if (!from_stdin) {
 		printf("%s\n", hash_to_hex(hash));
diff --git a/midx.c b/midx.c
index 69c44be71c..3faeaf2f8f 100644
--- a/midx.c
+++ b/midx.c
@@ -443,7 +443,6 @@ int prepare_midx_pack(struct multi_pack_index *m,
 {
 	struct repository *r = m->source->odb->repo;
 	struct strbuf pack_name = STRBUF_INIT;
-	struct strbuf key = STRBUF_INIT;
 	struct packed_git *p;
 
 	pack_int_id = midx_for_pack(&m, pack_int_id);
@@ -455,25 +454,11 @@ int prepare_midx_pack(struct multi_pack_index *m,
 
 	strbuf_addf(&pack_name, "%s/pack/%s", m->source->path,
 		    m->pack_names[pack_int_id]);
-
-	/* pack_map holds the ".pack" name, but we have the .idx */
-	strbuf_addbuf(&key, &pack_name);
-	strbuf_strip_suffix(&key, ".idx");
-	strbuf_addstr(&key, ".pack");
-	p = hashmap_get_entry_from_hash(&r->objects->packfiles->map,
-					strhash(key.buf), key.buf,
-					struct packed_git, packmap_ent);
-	if (!p) {
-		p = add_packed_git(r, pack_name.buf, pack_name.len,
-				   m->source->local);
-		if (p) {
-			packfile_store_add_pack(r->objects->packfiles, p);
-			list_add_tail(&p->mru, &r->objects->packfiles->mru);
-		}
-	}
-
+	p = packfile_store_load_pack(r->objects->packfiles,
+				     pack_name.buf, m->source->local);
+	if (p)
+		list_add_tail(&p->mru, &r->objects->packfiles->mru);
 	strbuf_release(&pack_name);
-	strbuf_release(&key);
 
 	if (!p) {
 		m->packs[pack_int_id] = MIDX_PACK_ERROR;
diff --git a/packfile.c b/packfile.c
index e1a3c0487c..e8b5be645c 100644
--- a/packfile.c
+++ b/packfile.c
@@ -792,6 +792,33 @@ void packfile_store_add_pack(struct packfile_store *store,
 	hashmap_add(&store->map, &pack->packmap_ent);
 }
 
+struct packed_git *packfile_store_load_pack(struct packfile_store *store,
+					    const char *idx_path, int local)
+{
+	struct strbuf key = STRBUF_INIT;
+	struct packed_git *p;
+
+	/*
+	 * We're being called with the path to the index file, but `pack_map`
+	 * holds the path to the packfile itself.
+	 */
+	strbuf_addstr(&key, idx_path);
+	strbuf_strip_suffix(&key, ".idx");
+	strbuf_addstr(&key, ".pack");
+
+	p = hashmap_get_entry_from_hash(&store->map, strhash(key.buf), key.buf,
+					struct packed_git, packmap_ent);
+	if (!p) {
+		p = add_packed_git(store->odb->repo, idx_path,
+				   strlen(idx_path), local);
+		if (p)
+			packfile_store_add_pack(store, p);
+	}
+
+	strbuf_release(&key);
+	return p;
+}
+
 void (*report_garbage)(unsigned seen_bits, const char *path);
 
 static void report_helper(const struct string_list *list,
@@ -891,23 +918,14 @@ static void prepare_pack(const char *full_name, size_t full_name_len,
 			 const char *file_name, void *_data)
 {
 	struct prepare_pack_data *data = (struct prepare_pack_data *)_data;
-	struct packed_git *p;
 	size_t base_len = full_name_len;
 
 	if (strip_suffix_mem(full_name, &base_len, ".idx") &&
 	    !(data->m && midx_contains_pack(data->m, file_name))) {
-		struct hashmap_entry hent;
-		char *pack_name = xstrfmt("%.*s.pack", (int)base_len, full_name);
-		unsigned int hash = strhash(pack_name);
-		hashmap_entry_init(&hent, hash);
-
-		/* Don't reopen a pack we already have. */
-		if (!hashmap_get(&data->r->objects->packfiles->map, &hent, pack_name)) {
-			p = add_packed_git(data->r, full_name, full_name_len, data->local);
-			if (p)
-				packfile_store_add_pack(data->r->objects->packfiles, p);
-		}
-		free(pack_name);
+		char *trimmed_path = xstrndup(full_name, full_name_len);
+		packfile_store_load_pack(data->r->objects->packfiles,
+					 trimmed_path, data->local);
+		free(trimmed_path);
 	}
 
 	if (!report_garbage)
diff --git a/packfile.h b/packfile.h
index ba4b0cef9c..fcefcbbef6 100644
--- a/packfile.h
+++ b/packfile.h
@@ -127,6 +127,14 @@ void packfile_store_reprepare(struct packfile_store *store);
 void packfile_store_add_pack(struct packfile_store *store,
 			     struct packed_git *pack);
 
+/*
+ * Open the packfile and add it to the store if it isn't yet known. Returns
+ * either the newly opened packfile or the preexisting packfile. Returns a
+ * `NULL` pointer in case the packfile could not be opened.
+ */
+struct packed_git *packfile_store_load_pack(struct packfile_store *store,
+					    const char *idx_path, int local);
+
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
-- 
2.51.0.384.g4c02a37b29.dirty
Patrick Steinhardt· Sep 2, 2025, 10:48 UTC · re: Patrick Steinhardt · lore

[PATCH v3 12/15] packfile: move `get_multi_pack_index()` into "midx.c"

The `get_multi_pack_index()` function is declared and implemented in the packfile subsystem, even though it really belongs into the multi-pack index subsystem. The reason for this is likely that it needs to call `packfile_store_prepare()`, which is not exposed by the packfile system. In a subsequent commit we're about to add another caller outside of the packfile system though, so we'll have to expose the function anyway.

Do so now already and move `get_multi_pack_index()` into the MIDX subsystem.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 midx.c     |  6 ++++++
 midx.h     |  1 +
 packfile.c |  8 +-------
 packfile.h | 10 +++++++++-
 4 files changed, 17 insertions(+), 8 deletions(-)
Show changes to 4 files +17 −8

midx.c, midx.h, packfile.c, packfile.h

diff --git a/midx.c b/midx.c
index 3faeaf2f8f..1d6269f957 100644
--- a/midx.c
+++ b/midx.c
@@ -93,6 +93,12 @@ static int midx_read_object_offsets(const unsigned char *chunk_start,
 	return 0;
 }
 
+struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
+{
+	packfile_store_prepare(source->odb->packfiles);
+	return source->midx;
+}
+
 static struct multi_pack_index *load_multi_pack_index_one(struct odb_source *source,
 							  const char *midx_name)
 {
diff --git a/midx.h b/midx.h
index e241d2d690..6e54d73503 100644
--- a/midx.h
+++ b/midx.h
@@ -94,6 +94,7 @@ void get_midx_chain_filename(struct odb_source *source, struct strbuf *out);
 void get_split_midx_filename_ext(struct odb_source *source, struct strbuf *buf,
 				 const unsigned char *hash, const char *ext);
 
+struct multi_pack_index *get_multi_pack_index(struct odb_source *source);
 struct multi_pack_index *load_multi_pack_index(struct odb_source *source);
 int prepare_midx_pack(struct multi_pack_index *m, uint32_t pack_int_id);
 struct packed_git *nth_midxed_pack(struct multi_pack_index *m,
diff --git a/packfile.c b/packfile.c
index e8b5be645c..70355ae92b 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1003,7 +1003,7 @@ static void packfile_store_prepare_mru(struct packfile_store *store)
 		list_add_tail(&p->mru, &store->mru);
 }
 
-static void packfile_store_prepare(struct packfile_store *store)
+void packfile_store_prepare(struct packfile_store *store)
 {
 	struct odb_source *source;
 
@@ -1033,12 +1033,6 @@ struct packed_git *get_packed_git(struct repository *r)
 	return r->objects->packfiles->packs;
 }
 
-struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
-{
-	packfile_store_prepare(source->odb->packfiles);
-	return source->midx;
-}
-
 struct packed_git *get_all_packs(struct repository *r)
 {
 	packfile_store_prepare(r->objects->packfiles);
diff --git a/packfile.h b/packfile.h
index fcefcbbef6..a9e561ac39 100644
--- a/packfile.h
+++ b/packfile.h
@@ -112,6 +112,15 @@ void packfile_store_free(struct packfile_store *store);
  */
 void packfile_store_close(struct packfile_store *store);
 
+/*
+ * Prepare the packfile store by loading packfiles and multi-pack indices for
+ * all alternates. This becomes a no-op if the store is already prepared.
+ *
+ * It shouldn't typically be necessary to call this function directly, as
+ * functions that access the store know to prepare it.
+ */
+void packfile_store_prepare(struct packfile_store *store);
+
 /*
  * Clear the packfile caches and try to look up any new packfiles that have
  * appeared since last preparing the packfiles store.
@@ -213,7 +222,6 @@ extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
 struct packed_git *get_packed_git(struct repository *r);
 struct list_head *get_packed_git_mru(struct repository *r);
-struct multi_pack_index *get_multi_pack_index(struct odb_source *source);
 struct packed_git *get_all_packs(struct repository *r);
 
 /*
-- 
2.51.0.384.g4c02a37b29.dirty
Patrick Steinhardt· Sep 2, 2025, 10:48 UTC · re: Patrick Steinhardt · lore

[PATCH v3 13/15] packfile: remove `get_packed_git()`

We have two different functions to retrieve packfiles for a packfile store:

  - `get_packed_git()` returns the list of packfiles after having called
    `prepare_packed_git()`.
  - `get_all_packs()` calls `prepare_packed_git()`, as well, but also
    calls `prepare_midx_pack()` for each pack.

Based on the naming alone one might think that `get_all_packs()` would return more packs than `get_packed_git()`. But that's not the case: both functions end up returning the exact same list of packfiles. The real difference between those functions is that `get_all_packs()` also loads the info of whether or not a packfile is part of a multi-pack index.

Preparing this extra information also shouldn't be significantly more expensive:

  - We have already loaded all packfiles via `prepare_packed_git_one()`.
    So given that multi-pack indices may only refer to packfiles in the
    same object directory we know that we already loaded each packfile.
  - The multi-pack index was prepared via `packfile_store_prepare()`
    already, which calls `prepare_multi_pack_index_one()`.
  - So all that remains to be done is to look up the index of the pack
    in its multi-pack index so that we can store that info in both the
    pack itself and the MIDX.

So it is somewhat confusing to readers that one of these two functions claims to load "all" packfiles while the other one doesn't, even though the ultimate difference is way more nuanced.

Convert all of these sites to use `get_all_packs()` instead and remove `get_packed_git()`. There doesn't seem to be a good reason to discern these two functions.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/gc.c   | 2 +-
 builtin/grep.c | 2 +-
 object-name.c  | 4 ++--
 packfile.c     | 6 ------
 packfile.h     | 1 -
 5 files changed, 4 insertions(+), 11 deletions(-)
Show changes to 5 files +4 −11

builtin/gc.c, builtin/grep.c, object-name.c, packfile.c, packfile.h

diff --git a/builtin/gc.c b/builtin/gc.c
index aeca06a08b..b3eec213d2 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1423,7 +1423,7 @@ static int incremental_repack_auto_condition(struct gc_config *cfg UNUSED)
 	if (incremental_repack_auto_limit < 0)
 		return 1;
 
-	for (p = get_packed_git(the_repository);
+	for (p = get_all_packs(the_repository);
 	     count < incremental_repack_auto_limit && p;
 	     p = p->next) {
 		if (!p->multi_pack_index)
diff --git a/builtin/grep.c b/builtin/grep.c
index 5df6537333..8f0e21bd70 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -1214,7 +1214,7 @@ int cmd_grep(int argc,
 		if (recurse_submodules)
 			repo_read_gitmodules(the_repository, 1);
 		if (startup_info->have_repository)
-			(void)get_packed_git(the_repository);
+			packfile_store_prepare(the_repository->objects->packfiles);
 
 		start_threads(&opt);
 	} else {
diff --git a/object-name.c b/object-name.c
index df9e0c5f02..ecffd2d5b1 100644
--- a/object-name.c
+++ b/object-name.c
@@ -213,7 +213,7 @@ static void find_short_packed_object(struct disambiguate_state *ds)
 			unique_in_midx(m, ds);
 	}
 
-	for (p = get_packed_git(ds->repo); p && !ds->ambiguous;
+	for (p = get_all_packs(ds->repo); p && !ds->ambiguous;
 	     p = p->next)
 		unique_in_pack(p, ds);
 }
@@ -806,7 +806,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)
 			find_abbrev_len_for_midx(m, mad);
 	}
 
-	for (p = get_packed_git(mad->repo); p; p = p->next)
+	for (p = get_all_packs(mad->repo); p; p = p->next)
 		find_abbrev_len_for_pack(p, mad);
 }
 
diff --git a/packfile.c b/packfile.c
index 70355ae92b..faa796f2a3 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1027,12 +1027,6 @@ void packfile_store_reprepare(struct packfile_store *store)
 	packfile_store_prepare(store);
 }
 
-struct packed_git *get_packed_git(struct repository *r)
-{
-	packfile_store_prepare(r->objects->packfiles);
-	return r->objects->packfiles->packs;
-}
-
 struct packed_git *get_all_packs(struct repository *r)
 {
 	packfile_store_prepare(r->objects->packfiles);
diff --git a/packfile.h b/packfile.h
index a9e561ac39..34c2132863 100644
--- a/packfile.h
+++ b/packfile.h
@@ -220,7 +220,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-struct packed_git *get_packed_git(struct repository *r);
 struct list_head *get_packed_git_mru(struct repository *r);
 struct packed_git *get_all_packs(struct repository *r);
 
-- 
2.51.0.384.g4c02a37b29.dirty
Patrick Steinhardt· Sep 2, 2025, 10:48 UTC · re: Patrick Steinhardt · lore

[PATCH v3 14/15] packfile: refactor `get_all_packs()` to work on packfile store

The `get_all_packs()` function prepares the packfile store and then returns its packfiles. Refactor it to accept a packfile store instead of a repository to clarify its scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/cat-file.c          |  3 ++-
 builtin/count-objects.c     |  3 ++-
 builtin/fast-import.c       |  6 ++++--
 builtin/fsck.c              | 11 +++++++----
 builtin/gc.c                | 10 ++++++----
 builtin/pack-objects.c      | 28 +++++++++++++++++++---------
 builtin/pack-redundant.c    |  6 ++++--
 builtin/repack.c            |  9 ++++++---
 connected.c                 |  3 ++-
 http-backend.c              |  5 +++--
 http.c                      |  3 ++-
 object-name.c               |  4 ++--
 pack-bitmap.c               |  4 ++--
 pack-objects.c              |  3 ++-
 packfile.c                  | 12 ++++++------
 packfile.h                  |  7 ++++++-
 server-info.c               |  3 ++-
 t/helper/test-find-pack.c   |  2 +-
 t/helper/test-pack-mtimes.c |  2 +-
 19 files changed, 79 insertions(+), 45 deletions(-)
Show changes to 19 files +79 −45

builtin/cat-file.c, builtin/count-objects.c, builtin/fast-import.c, builtin/fsck.c, builtin/gc.c, builtin/pack-objects.c, builtin/pack-redundant.c, builtin/repack.c, connected.c, http-backend.c, http.c, object-name.c, pack-bitmap.c, pack-objects.c, packfile.c, packfile.h, server-info.c, t/helper/test-find-pack.c, t/helper/test-pack-mtimes.c

diff --git a/builtin/cat-file.c b/builtin/cat-file.c
index fce0b06451c..461eb2a6e09 100644
--- a/builtin/cat-file.c
+++ b/builtin/cat-file.c
@@ -852,9 +852,10 @@ static void batch_each_object(struct batch_options *opt,
 
 	if (bitmap && !for_each_bitmapped_object(bitmap, &opt->objects_filter,
 						 batch_one_object_bitmapped, &payload)) {
+		struct packfile_store *packs = the_repository->objects->packfiles;
 		struct packed_git *pack;
 
-		for (pack = get_all_packs(the_repository); pack; pack = pack->next) {
+		for (pack = packfile_store_get_packs(packs); pack; pack = pack->next) {
 			if (bitmap_index_contains_pack(bitmap, pack) ||
 			    open_pack_index(pack))
 				continue;
diff --git a/builtin/count-objects.c b/builtin/count-objects.c
index a61d3b46aac..cb03c580553 100644
--- a/builtin/count-objects.c
+++ b/builtin/count-objects.c
@@ -122,6 +122,7 @@ int cmd_count_objects(int argc,
 				      count_loose, count_cruft, NULL, NULL);
 
 	if (verbose) {
+		struct packfile_store *packs = the_repository->objects->packfiles;
 		struct packed_git *p;
 		unsigned long num_pack = 0;
 		off_t size_pack = 0;
@@ -129,7 +130,7 @@ int cmd_count_objects(int argc,
 		struct strbuf pack_buf = STRBUF_INIT;
 		struct strbuf garbage_buf = STRBUF_INIT;
 
-		for (p = get_all_packs(the_repository); p; p = p->next) {
+		for (p = packfile_store_get_packs(packs); p; p = p->next) {
 			if (!p->pack_local)
 				continue;
 			if (open_pack_index(p))
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index a26e79689d5..fea914cf9eb 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -952,6 +952,7 @@ static int store_object(
 	struct object_id *oidout,
 	uintmax_t mark)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	void *out, *delta;
 	struct object_entry *e;
 	unsigned char hdr[96];
@@ -975,7 +976,7 @@ static int store_object(
 	if (e->idx.offset) {
 		duplicate_count_by_type[type]++;
 		return 1;
-	} else if (find_oid_pack(&oid, get_all_packs(the_repository))) {
+	} else if (find_oid_pack(&oid, packfile_store_get_packs(packs))) {
 		e->type = type;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
@@ -1092,6 +1093,7 @@ static void truncate_pack(struct hashfile_checkpoint *checkpoint)
 
 static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	size_t in_sz = 64 * 1024, out_sz = 64 * 1024;
 	unsigned char *in_buf = xmalloc(in_sz);
 	unsigned char *out_buf = xmalloc(out_sz);
@@ -1175,7 +1177,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 		duplicate_count_by_type[OBJ_BLOB]++;
 		truncate_pack(&checkpoint);
 
-	} else if (find_oid_pack(&oid, get_all_packs(the_repository))) {
+	} else if (find_oid_pack(&oid, packfile_store_get_packs(packs))) {
 		e->type = OBJ_BLOB;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
diff --git a/builtin/fsck.c b/builtin/fsck.c
index d2eb9d4fbe9..cabf5401c9f 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -867,19 +867,20 @@ static int mark_packed_for_connectivity(const struct object_id *oid,
 
 static int check_pack_rev_indexes(struct repository *r, int show_progress)
 {
+	struct packfile_store *packs = r->objects->packfiles;
 	struct progress *progress = NULL;
 	uint32_t pack_count = 0;
 	int res = 0;
 
 	if (show_progress) {
-		for (struct packed_git *p = get_all_packs(r); p; p = p->next)
+		for (struct packed_git *p = packfile_store_get_packs(packs); p; p = p->next)
 			pack_count++;
 		progress = start_delayed_progress(the_repository,
 						  "Verifying reverse pack-indexes", pack_count);
 		pack_count = 0;
 	}
 
-	for (struct packed_git *p = get_all_packs(r); p; p = p->next) {
+	for (struct packed_git *p = packfile_store_get_packs(packs); p; p = p->next) {
 		int load_error = load_pack_revindex_from_disk(p);
 
 		if (load_error < 0) {
@@ -999,6 +1000,8 @@ int cmd_fsck(int argc,
 		for_each_packed_object(the_repository,
 				       mark_packed_for_connectivity, NULL, 0);
 	} else {
+		struct packfile_store *packs = the_repository->objects->packfiles;
+
 		odb_prepare_alternates(the_repository->objects);
 		for (source = the_repository->objects->sources; source; source = source->next)
 			fsck_source(source);
@@ -1009,7 +1012,7 @@ int cmd_fsck(int argc,
 			struct progress *progress = NULL;
 
 			if (show_progress) {
-				for (p = get_all_packs(the_repository); p;
+				for (p = packfile_store_get_packs(packs); p;
 				     p = p->next) {
 					if (open_pack_index(p))
 						continue;
@@ -1019,7 +1022,7 @@ int cmd_fsck(int argc,
 				progress = start_progress(the_repository,
 							  _("Checking objects"), total);
 			}
-			for (p = get_all_packs(the_repository); p;
+			for (p = packfile_store_get_packs(packs); p;
 			     p = p->next) {
 				/* verify gives error messages itself */
 				if (verify_pack(the_repository,
diff --git a/builtin/gc.c b/builtin/gc.c
index b3eec213d25..c0a3b54ce74 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -487,9 +487,10 @@ static int too_many_loose_objects(struct gc_config *cfg)
 static struct packed_git *find_base_packs(struct string_list *packs,
 					  unsigned long limit)
 {
+	struct packfile_store *packfiles = the_repository->objects->packfiles;
 	struct packed_git *p, *base = NULL;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(packfiles); p; p = p->next) {
 		if (!p->pack_local || p->is_cruft)
 			continue;
 		if (limit) {
@@ -508,13 +509,14 @@ static struct packed_git *find_base_packs(struct string_list *packs,
 
 static int too_many_packs(struct gc_config *cfg)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 	int cnt;
 
 	if (cfg->gc_auto_pack_limit <= 0)
 		return 0;
 
-	for (cnt = 0, p = get_all_packs(the_repository); p; p = p->next) {
+	for (cnt = 0, p = packfile_store_get_packs(packs); p; p = p->next) {
 		if (!p->pack_local)
 			continue;
 		if (p->pack_keep)
@@ -1423,7 +1425,7 @@ static int incremental_repack_auto_condition(struct gc_config *cfg UNUSED)
 	if (incremental_repack_auto_limit < 0)
 		return 1;
 
-	for (p = get_all_packs(the_repository);
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles);
 	     count < incremental_repack_auto_limit && p;
 	     p = p->next) {
 		if (!p->multi_pack_index)
@@ -1492,7 +1494,7 @@ static off_t get_auto_pack_size(void)
 	struct repository *r = the_repository;
 
 	odb_reprepare(r->objects);
-	for (p = get_all_packs(r); p; p = p->next) {
+	for (p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next) {
 		if (p->pack_size > max_size) {
 			second_largest_size = max_size;
 			max_size = p->pack_size;
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 1494afcf3df..914c6e641d0 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -3831,6 +3831,7 @@ static int pack_mtime_cmp(const void *_a, const void *_b)
 
 static void read_packs_list_from_stdin(struct rev_info *revs)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct strbuf buf = STRBUF_INIT;
 	struct string_list include_packs = STRING_LIST_INIT_DUP;
 	struct string_list exclude_packs = STRING_LIST_INIT_DUP;
@@ -3855,7 +3856,7 @@ static void read_packs_list_from_stdin(struct rev_info *revs)
 	string_list_sort(&exclude_packs);
 	string_list_remove_duplicates(&exclude_packs, 0);
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
 		const char *pack_name = pack_basename(p);
 
 		if ((item = string_list_lookup(&include_packs, pack_name)))
@@ -4076,6 +4077,7 @@ static void enumerate_cruft_objects(void)
 
 static void enumerate_and_traverse_cruft_objects(struct string_list *fresh_packs)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 	struct rev_info revs;
 	int ret;
@@ -4105,7 +4107,7 @@ static void enumerate_and_traverse_cruft_objects(struct string_list *fresh_packs
 	 * Re-mark only the fresh packs as kept so that objects in
 	 * unknown packs do not halt the reachability traversal early.
 	 */
-	for (p = get_all_packs(the_repository); p; p = p->next)
+	for (p = packfile_store_get_packs(packs); p; p = p->next)
 		p->pack_keep_in_core = 0;
 	mark_pack_kept_in_core(fresh_packs, 1);
 
@@ -4122,6 +4124,7 @@ static void enumerate_and_traverse_cruft_objects(struct string_list *fresh_packs
 
 static void read_cruft_objects(void)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct strbuf buf = STRBUF_INIT;
 	struct string_list discard_packs = STRING_LIST_INIT_DUP;
 	struct string_list fresh_packs = STRING_LIST_INIT_DUP;
@@ -4142,7 +4145,7 @@ static void read_cruft_objects(void)
 	string_list_sort(&discard_packs);
 	string_list_sort(&fresh_packs);
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
 		const char *pack_name = pack_basename(p);
 		struct string_list_item *item;
 
@@ -4390,11 +4393,12 @@ static void add_unreachable_loose_objects(struct rev_info *revs)
 
 static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	static struct packed_git *last_found = (void *)1;
 	struct packed_git *p;
 
 	p = (last_found != (void *)1) ? last_found :
-					get_all_packs(the_repository);
+					packfile_store_get_packs(packs);
 
 	while (p) {
 		if ((!p->pack_local || p->pack_keep ||
@@ -4404,7 +4408,7 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 			return 1;
 		}
 		if (p == last_found)
-			p = get_all_packs(the_repository);
+			p = packfile_store_get_packs(packs);
 		else
 			p = p->next;
 		if (p == last_found)
@@ -4436,12 +4440,13 @@ static int loosened_object_can_be_discarded(const struct object_id *oid,
 
 static void loosen_unused_packed_objects(void)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 	uint32_t i;
 	uint32_t loosened_objects_nr = 0;
 	struct object_id oid;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
 		if (!p->pack_local || p->pack_keep || p->pack_keep_in_core)
 			continue;
 
@@ -4742,12 +4747,13 @@ static void get_object_list(struct rev_info *revs, int ac, const char **av)
 
 static void add_extra_kept_packs(const struct string_list *names)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 
 	if (!names->nr)
 		return;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
 		const char *name = basename(p->pack_name);
 		int i;
 
@@ -5185,8 +5191,10 @@ int cmd_pack_objects(int argc,
 
 	add_extra_kept_packs(&keep_pack_list);
 	if (ignore_packed_keep_on_disk) {
+		struct packfile_store *packs = the_repository->objects->packfiles;
 		struct packed_git *p;
-		for (p = get_all_packs(the_repository); p; p = p->next)
+
+		for (p = packfile_store_get_packs(packs); p; p = p->next)
 			if (p->pack_local && p->pack_keep)
 				break;
 		if (!p) /* no keep-able packs found */
@@ -5198,8 +5206,10 @@ int cmd_pack_objects(int argc,
 		 * want to unset "local" based on looking at packs, as
 		 * it also covers non-local objects
 		 */
+		struct packfile_store *packs = the_repository->objects->packfiles;
 		struct packed_git *p;
-		for (p = get_all_packs(the_repository); p; p = p->next) {
+
+		for (p = packfile_store_get_packs(packs); p; p = p->next) {
 			if (!p->pack_local) {
 				have_non_local_packs = 1;
 				break;
diff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c
index fe81c293e3a..721c4e04d5c 100644
--- a/builtin/pack-redundant.c
+++ b/builtin/pack-redundant.c
@@ -566,7 +566,8 @@ static struct pack_list * add_pack(struct packed_git *p)
 
 static struct pack_list * add_pack_file(const char *filename)
 {
-	struct packed_git *p = get_all_packs(the_repository);
+	struct packfile_store *packs = the_repository->objects->packfiles;
+	struct packed_git *p = packfile_store_get_packs(packs);
 
 	if (strlen(filename) < 40)
 		die("Bad pack filename: %s", filename);
@@ -581,7 +582,8 @@ static struct pack_list * add_pack_file(const char *filename)
 
 static void load_all(void)
 {
-	struct packed_git *p = get_all_packs(the_repository);
+	struct packfile_store *packs = the_repository->objects->packfiles;
+	struct packed_git *p = packfile_store_get_packs(packs);
 
 	while (p) {
 		add_pack(p);
diff --git a/builtin/repack.c b/builtin/repack.c
index 5ff27fc8e29..2579090abbd 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -265,10 +265,11 @@ static void existing_packs_release(struct existing_packs *existing)
 static void collect_pack_filenames(struct existing_packs *existing,
 				   const struct string_list *extra_keep)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 	struct strbuf buf = STRBUF_INIT;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
 		int i;
 		const char *base;
 
@@ -497,10 +498,11 @@ static void init_pack_geometry(struct pack_geometry *geometry,
 			       struct existing_packs *existing,
 			       const struct pack_objects_args *args)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 	struct strbuf buf = STRBUF_INIT;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
 		if (args->local && !p->pack_local)
 			/*
 			 * When asked to only repack local packfiles we skip
@@ -1137,11 +1139,12 @@ static int write_filtered_pack(const struct pack_objects_args *args,
 static void combine_small_cruft_packs(FILE *in, size_t combine_cruft_below_size,
 				      struct existing_packs *existing)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 	struct strbuf buf = STRBUF_INIT;
 	size_t i;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
 		if (!(p->is_cruft && p->pack_local))
 			continue;
 
diff --git a/connected.c b/connected.c
index d6e9682fd93..56fedb923d5 100644
--- a/connected.c
+++ b/connected.c
@@ -74,9 +74,10 @@ int check_connected(oid_iterate_fn fn, void *cb_data,
 		 */
 		odb_reprepare(the_repository->objects);
 		do {
+			struct packfile_store *packs = the_repository->objects->packfiles;
 			struct packed_git *p;
 
-			for (p = get_all_packs(the_repository); p; p = p->next) {
+			for (p = packfile_store_get_packs(packs); p; p = p->next) {
 				if (!p->pack_promisor)
 					continue;
 				if (find_pack_entry_one(oid, p))
diff --git a/http-backend.c b/http-backend.c
index d5dfe762bb5..c95aa51b45c 100644
--- a/http-backend.c
+++ b/http-backend.c
@@ -603,18 +603,19 @@ static void get_head(struct strbuf *hdr, char *arg UNUSED)
 static void get_info_packs(struct strbuf *hdr, char *arg UNUSED)
 {
 	size_t objdirlen = strlen(repo_get_object_directory(the_repository));
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct strbuf buf = STRBUF_INIT;
 	struct packed_git *p;
 	size_t cnt = 0;
 
 	select_getanyfile(hdr);
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
 		if (p->pack_local)
 			cnt++;
 	}
 
 	strbuf_grow(&buf, cnt * 53 + 2);
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
 		if (p->pack_local)
 			strbuf_addf(&buf, "P %s\n", p->pack_name + objdirlen + 6);
 	}
diff --git a/http.c b/http.c
index af2120b64c7..321648a81e6 100644
--- a/http.c
+++ b/http.c
@@ -2408,6 +2408,7 @@ static char *fetch_pack_index(unsigned char *hash, const char *base_url)
 static int fetch_and_setup_pack_index(struct packed_git **packs_head,
 	unsigned char *sha1, const char *base_url)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *new_pack, *p;
 	char *tmp_idx = NULL;
 	int ret;
@@ -2416,7 +2417,7 @@ static int fetch_and_setup_pack_index(struct packed_git **packs_head,
 	 * If we already have the pack locally, no need to fetch its index or
 	 * even add it to list; we already have all of its objects.
 	 */
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
 		if (hasheq(p->hash, sha1, the_repository->hash_algo))
 			return 0;
 	}
diff --git a/object-name.c b/object-name.c
index ecffd2d5b12..53356819a3d 100644
--- a/object-name.c
+++ b/object-name.c
@@ -213,7 +213,7 @@ static void find_short_packed_object(struct disambiguate_state *ds)
 			unique_in_midx(m, ds);
 	}
 
-	for (p = get_all_packs(ds->repo); p && !ds->ambiguous;
+	for (p = packfile_store_get_packs(ds->repo->objects->packfiles); p && !ds->ambiguous;
 	     p = p->next)
 		unique_in_pack(p, ds);
 }
@@ -806,7 +806,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)
 			find_abbrev_len_for_midx(m, mad);
 	}
 
-	for (p = get_all_packs(mad->repo); p; p = p->next)
+	for (p = packfile_store_get_packs(mad->repo->objects->packfiles); p; p = p->next)
 		find_abbrev_len_for_pack(p, mad);
 }
 
diff --git a/pack-bitmap.c b/pack-bitmap.c
index 058bdb5d7de..834c300aab7 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -664,7 +664,7 @@ static int open_pack_bitmap(struct repository *r,
 	struct packed_git *p;
 	int ret = -1;
 
-	for (p = get_all_packs(r); p; p = p->next) {
+	for (p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next) {
 		if (open_pack_bitmap_1(bitmap_git, p) == 0) {
 			ret = 0;
 			/*
@@ -3362,7 +3362,7 @@ int verify_bitmap_files(struct repository *r)
 		free(midx_bitmap_name);
 	}
 
-	for (struct packed_git *p = get_all_packs(r);
+	for (struct packed_git *p = packfile_store_get_packs(r->objects->packfiles);
 	     p; p = p->next) {
 		char *pack_bitmap_name = pack_bitmap_filename(p);
 		res |= verify_bitmap_file(r->hash_algo, pack_bitmap_name);
diff --git a/pack-objects.c b/pack-objects.c
index a9d9855063a..668c1136673 100644
--- a/pack-objects.c
+++ b/pack-objects.c
@@ -86,6 +86,7 @@ struct object_entry *packlist_find(struct packing_data *pdata,
 
 static void prepare_in_pack_by_idx(struct packing_data *pdata)
 {
+	struct packfile_store *packs = pdata->repo->objects->packfiles;
 	struct packed_git **mapping, *p;
 	int cnt = 0, nr = 1U << OE_IN_PACK_BITS;
 
@@ -95,7 +96,7 @@ static void prepare_in_pack_by_idx(struct packing_data *pdata)
 	 * (i.e. in_pack_idx also zero) should return NULL.
 	 */
 	mapping[cnt++] = NULL;
-	for (p = get_all_packs(pdata->repo); p; p = p->next, cnt++) {
+	for (p = packfile_store_get_packs(packs); p; p = p->next, cnt++) {
 		if (cnt == nr) {
 			free(mapping);
 			return;
diff --git a/packfile.c b/packfile.c
index faa796f2a3f..3e0d2a8f41e 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1027,11 +1027,11 @@ void packfile_store_reprepare(struct packfile_store *store)
 	packfile_store_prepare(store);
 }
 
-struct packed_git *get_all_packs(struct repository *r)
+struct packed_git *packfile_store_get_packs(struct packfile_store *store)
 {
-	packfile_store_prepare(r->objects->packfiles);
+	packfile_store_prepare(store);
 
-	for (struct odb_source *source = r->objects->sources; source; source = source->next) {
+	for (struct odb_source *source = store->odb->sources; source; source = source->next) {
 		struct multi_pack_index *m = source->midx;
 		if (!m)
 			continue;
@@ -1039,7 +1039,7 @@ struct packed_git *get_all_packs(struct repository *r)
 			prepare_midx_pack(m, i);
 	}
 
-	return r->objects->packfiles->packs;
+	return store->packs;
 }
 
 struct list_head *get_packed_git_mru(struct repository *r)
@@ -2099,7 +2099,7 @@ struct packed_git **kept_pack_cache(struct repository *r, unsigned flags)
 		 * covers, one kept and one not kept, but the midx returns only
 		 * the non-kept version.
 		 */
-		for (p = get_all_packs(r); p; p = p->next) {
+		for (p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next) {
 			if ((p->pack_keep && (flags & ON_DISK_KEEP_PACKS)) ||
 			    (p->pack_keep_in_core && (flags & IN_CORE_KEEP_PACKS))) {
 				ALLOC_GROW(packs, nr + 1, alloc);
@@ -2196,7 +2196,7 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 	int r = 0;
 	int pack_errors = 0;
 
-	for (p = get_all_packs(repo); p; p = p->next) {
+	for (p = packfile_store_get_packs(repo->objects->packfiles); p; p = p->next) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
 		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&
diff --git a/packfile.h b/packfile.h
index 34c2132863a..86f2c07101f 100644
--- a/packfile.h
+++ b/packfile.h
@@ -136,6 +136,12 @@ void packfile_store_reprepare(struct packfile_store *store);
 void packfile_store_add_pack(struct packfile_store *store,
 			     struct packed_git *pack);
 
+/*
+ * Get all packs managed by the given store, including packfiles that are
+ * referenced by multi-pack indices.
+ */
+struct packed_git *packfile_store_get_packs(struct packfile_store *store);
+
 /*
  * Open the packfile and add it to the store if it isn't yet known. Returns
  * either the newly opened packfile or the preexisting packfile. Returns a
@@ -221,7 +227,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
 struct list_head *get_packed_git_mru(struct repository *r);
-struct packed_git *get_all_packs(struct repository *r);
 
 /*
  * Give a rough count of objects in the repository. This sacrifices accuracy
diff --git a/server-info.c b/server-info.c
index 9bb30d9ab71..d949dea3094 100644
--- a/server-info.c
+++ b/server-info.c
@@ -287,12 +287,13 @@ static int compare_info(const void *a_, const void *b_)
 
 static void init_pack_info(struct repository *r, const char *infofile, int force)
 {
+	struct packfile_store *packs = r->objects->packfiles;
 	struct packed_git *p;
 	int stale;
 	int i;
 	size_t alloc = 0;
 
-	for (p = get_all_packs(r); p; p = p->next) {
+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
 		/* we ignore things on alternate path since they are
 		 * not available to the pullers in general.
 		 */
diff --git a/t/helper/test-find-pack.c b/t/helper/test-find-pack.c
index 611a13a3261..183a777fc54 100644
--- a/t/helper/test-find-pack.c
+++ b/t/helper/test-find-pack.c
@@ -39,7 +39,7 @@ int cmd__find_pack(int argc, const char **argv)
 	if (repo_get_oid(the_repository, argv[0], &oid))
 		die("cannot parse %s as an object name", argv[0]);
 
-	for (p = get_all_packs(the_repository); p; p = p->next)
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next)
 		if (find_pack_entry_one(&oid, p)) {
 			printf("%s\n", p->pack_name);
 			actual_count++;
diff --git a/t/helper/test-pack-mtimes.c b/t/helper/test-pack-mtimes.c
index d51aaa3dc40..cfdfae77a6c 100644
--- a/t/helper/test-pack-mtimes.c
+++ b/t/helper/test-pack-mtimes.c
@@ -37,7 +37,7 @@ int cmd__pack_mtimes(int argc, const char **argv)
 	if (argc != 2)
 		usage(pack_mtimes_usage);
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		strbuf_addstr(&buf, basename(p->pack_name));
 		strbuf_strip_suffix(&buf, ".pack");
 		strbuf_addstr(&buf, ".mtimes");
-- 
2.51.0.384.g4c02a37b29.dirty
Patrick Steinhardt· Sep 2, 2025, 10:48 UTC · re: Patrick Steinhardt · lore

[PATCH v3 15/15] packfile: refactor `get_packed_git_mru()` to work on packfile store

The `get_packed_git_mru()` function prepares the packfile store and then returns its packfiles in most-recently-used order. Refactor it to accept a packfile store instead of a repository to clarify its scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/pack-objects.c | 4 ++--
 packfile.c             | 6 +++---
 packfile.h             | 7 +++++--
 3 files changed, 10 insertions(+), 7 deletions(-)
Show changes to 3 files +10 −7

builtin/pack-objects.c, packfile.c, packfile.h

diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 914c6e641d..9558ab883e 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -1748,12 +1748,12 @@ static int want_object_in_pack_mtime(const struct object_id *oid,
 		}
 	}
 
-	list_for_each(pos, get_packed_git_mru(the_repository)) {
+	list_for_each(pos, packfile_store_get_packs_mru(the_repository->objects->packfiles)) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		want = want_object_in_pack_one(p, oid, exclude, found_pack, found_offset, found_mtime);
 		if (!exclude && want > 0)
 			list_move(&p->mru,
-				  get_packed_git_mru(the_repository));
+				  packfile_store_get_packs_mru(the_repository->objects->packfiles));
 		if (want != -1)
 			return want;
 	}
diff --git a/packfile.c b/packfile.c
index 3e0d2a8f41..91d11e0562 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1042,10 +1042,10 @@ struct packed_git *packfile_store_get_packs(struct packfile_store *store)
 	return store->packs;
 }
 
-struct list_head *get_packed_git_mru(struct repository *r)
+struct list_head *packfile_store_get_packs_mru(struct packfile_store *store)
 {
-	packfile_store_prepare(r->objects->packfiles);
-	return &r->objects->packfiles->mru;
+	packfile_store_prepare(store);
+	return &store->mru;
 }
 
 /*
diff --git a/packfile.h b/packfile.h
index 86f2c07101..e21ebd75d4 100644
--- a/packfile.h
+++ b/packfile.h
@@ -142,6 +142,11 @@ void packfile_store_add_pack(struct packfile_store *store,
  */
 struct packed_git *packfile_store_get_packs(struct packfile_store *store);
 
+/*
+ * Get all packs in most-recently-used order.
+ */
+struct list_head *packfile_store_get_packs_mru(struct packfile_store *store);
+
 /*
  * Open the packfile and add it to the store if it isn't yet known. Returns
  * either the newly opened packfile or the preexisting packfile. Returns a
@@ -226,8 +231,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-struct list_head *get_packed_git_mru(struct repository *r);
-
 /*
  * Give a rough count of objects in the repository. This sacrifices accuracy
  * for speed.
-- 
2.51.0.384.g4c02a37b29.dirty
Junio C Hamano· Sep 2, 2025, 16:40 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v3 00/15] packfile: carve out a new packfile store

Patrick Steinhardt <ps@pks.im> writes:
Show 8 quoted lines
>   - Rebased on top of master at 6ad8021821 (The fifth batch, 2025-08-29)
>     with ps/object-store-midx-dedup-info at 13296ac909 (midx: compute
>     paths via their source, 2025-08-11) merged into it. This fixes
>     various conflicts with "seen". There's still two conflicts: a
>     trivial one with jt/de-global-bulk-checkin. And a more complex one
>     with tb/prepare-midx-pack-cleanup. I don't think it's necessary to
>     really address the first one, but I'm unsure how to proceed with the
>     second one given that the patch series still seems to be cooking.

I think the second topic is not really cooking, but is expecting a reroll, so I'd say it is perfectly fine to drop it and expect it to come back, if it is still relevant, in future, in a shape that is friendlier to other topics in 'seen' when it happens.

Taylor Blau· Sep 11, 2025, 23:34 UTC · re: Junio C Hamano · lore

Re: [PATCH v3 00/15] packfile: carve out a new packfile store

On Tue, Sep 02, 2025 at 09:40:40AM -0700, Junio C Hamano wrote:
Show 15 quoted lines
> Patrick Steinhardt <ps@pks.im> writes:
>
> >   - Rebased on top of master at 6ad8021821 (The fifth batch, 2025-08-29)
> >     with ps/object-store-midx-dedup-info at 13296ac909 (midx: compute
> >     paths via their source, 2025-08-11) merged into it. This fixes
> >     various conflicts with "seen". There's still two conflicts: a
> >     trivial one with jt/de-global-bulk-checkin. And a more complex one
> >     with tb/prepare-midx-pack-cleanup. I don't think it's necessary to
> >     really address the first one, but I'm unsure how to proceed with the
> >     second one given that the patch series still seems to be cooking.
>
> I think the second topic is not really cooking, but is expecting a
> reroll, so I'd say it is perfectly fine to drop it and expect it to
> come back, if it is still relevant, in future, in a shape that is
> friendlier to other topics in 'seen' when it happens.
Agreed.

Thanks, Taylor

Karthik Nayak· Sep 9, 2025, 09:33 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v3 00/15] packfile: carve out a new packfile store

Patrick Steinhardt <ps@pks.im> writes:
Show 32 quoted lines
> Hi,
>
> information about a object database's packfiles is currently distributed
> across two different structures:
>
>   - `struct packed_git` contains the `next` pointer as well as the
>     `mru_head`, both of which serve to store the list of packfiles.
>
>   - `struct object_database` contains several fields that relate to the
>     packfiles.
>
> So we don't really have a central data structure that tracks our
> packfiles, and consequently responsibilities aren't always clear cut.
> A consequence for the upcoming pluggable object databases is that this
> makes it very hard to move management of packfiles from the object
> database level down into the object database source.
>
> This patch series introduces a new `struct packfile_store`, which is
> about to become the single source of truth for managing packfiles, and
> carves out the packfile store subsystem.
>
> This is the first step to make packfiles work with pluggable object
> databases. Next steps will be to:
>
>   - Move the `struct packed_git::next` and `struct packed::mru_head`
>     pointers into the packfile store so that `struct packed_git` only
>     tracks a single packfile.
>
>   - Push the `struct packfile_store` down one level so that it's not
>     hosted by the object database anymore, but instead by the object
>     database source.
>

I did an entire re-read of the series, since I didn't review v2 and there were a lot of changes in the middle. Overall it looks great, I've left some small questions/comments on the first few patches feel free to ignore!

Thanks, Karthik

Patrick Steinhardt· Sep 9, 2025, 11:02 UTC · re: Patrick Steinhardt · lore

[PATCH v4 00/15] packfile: carve out a new packfile store

Hi,

information about a object database's packfiles is currently distributed across two different structures:

  - `struct packed_git` contains the `next` pointer as well as the
    `mru_head`, both of which serve to store the list of packfiles.
  - `struct object_database` contains several fields that relate to the
    packfiles.

So we don't really have a central data structure that tracks our packfiles, and consequently responsibilities aren't always clear cut. A consequence for the upcoming pluggable object databases is that this makes it very hard to move management of packfiles from the object database level down into the object database source.

This patch series introduces a new `struct packfile_store`, which is about to become the single source of truth for managing packfiles, and carves out the packfile store subsystem.

This is the first step to make packfiles work with pluggable object databases. Next steps will be to:

  - Move the `struct packed_git::next` and `struct packed::mru_head`
    pointers into the packfile store so that `struct packed_git` only
    tracks a single packfile.
  - Push the `struct packfile_store` down one level so that it's not
    hosted by the object database anymore, but instead by the object
    database source.
Changes in v2:
  - Convert the `initialized` flag into a boolean.
  - Polish some commit messages.
  - Some smaller formatting changes to the layout of `struct
    object_database`.
  - Link to v1: https://lore.kernel.org/r/20250819-b4-pks-packfiles-store-v1-0-1660842e125a@pks.im
Changes in v3:
  - Rebased on top of master at 6ad8021821 (The fifth batch, 2025-08-29)
    with ps/object-store-midx-dedup-info at 13296ac909 (midx: compute
    paths via their source, 2025-08-11) merged into it. This fixes
    various conflicts with "seen". There's still two conflicts: a
    trivial one with jt/de-global-bulk-checkin. And a more complex one
    with tb/prepare-midx-pack-cleanup. I don't think it's necessary to
    really address the first one, but I'm unsure how to proceed with the
    second one given that the patch series still seems to be cooking.
  - Set `struct object_database::packfiles` to `NULL` after free'ing it.
  - Add a comment to explain the kept cache.
  - Fix a missing `obj_read_lock()` call.
  - Drop the commit that always adds packfiles to the MRU. I've moved
    this into a subsequent patch series.
  - Avoid some overly long lines by storing the pointer to the packfile
    store on the stack.
  - Point out the difference between `get_all_packs()` and
    `get_packed_git()`.
  - Link to v2: https://lore.kernel.org/r/20250821-b4-pks-packfiles-store-v2-0-d10623355e9f@pks.im
Changes in v4:
  - Small code style improvement as suggested by Junio.
  - Some commit message improvements as suggested by Karthik.
  - Link to v3: https://lore.kernel.org/r/20250902-b4-pks-packfiles-store-v3-0-6925278efeda@pks.im
Thanks!
Patrick
---
Patrick Steinhardt (15):
      packfile: introduce a new `struct packfile_store`
      odb: move list of packfiles into `struct packfile_store`
      odb: move initialization bit into `struct packfile_store`
      odb: move packfile map into `struct packfile_store`
      odb: move MRU list of packfiles into `struct packfile_store`
      odb: move kept cache into `struct packfile_store`
      packfile: reorder functions to avoid function declaration
      packfile: refactor `prepare_packed_git()` to work on packfile store
      packfile: split up responsibilities of `reprepare_packed_git()`
      packfile: refactor `install_packed_git()` to work on packfile store
      packfile: introduce function to load and add packfiles
      packfile: move `get_multi_pack_index()` into "midx.c"
      packfile: remove `get_packed_git()`
      packfile: refactor `get_all_packs()` to work on packfile store
      packfile: refactor `get_packed_git_mru()` to work on packfile store
 builtin/backfill.c          |   2 +-
 builtin/cat-file.c          |   3 +-
 builtin/count-objects.c     |   3 +-
 builtin/fast-import.c       |  10 +-
 builtin/fsck.c              |  11 +-
 builtin/gc.c                |  14 ++-
 builtin/grep.c              |   2 +-
 builtin/index-pack.c        |  10 +-
 builtin/pack-objects.c      |  32 +++--
 builtin/pack-redundant.c    |   6 +-
 builtin/receive-pack.c      |   2 +-
 builtin/repack.c            |  11 +-
 bulk-checkin.c              |   2 +-
 connected.c                 |   5 +-
 fetch-pack.c                |   4 +-
 http-backend.c              |   5 +-
 http.c                      |   5 +-
 http.h                      |   2 +-
 midx.c                      |  29 ++---
 midx.h                      |   1 +
 object-name.c               |   6 +-
 odb.c                       |  40 +++++--
 odb.h                       |  34 ++----
 pack-bitmap.c               |   4 +-
 pack-objects.c              |   3 +-
 packfile.c                  | 286 ++++++++++++++++++++++++--------------------
 packfile.h                  | 119 +++++++++++++++---
 server-info.c               |   3 +-
 t/helper/test-find-pack.c   |   2 +-
 t/helper/test-pack-mtimes.c |   2 +-
 transport-helper.c          |   2 +-
 31 files changed, 390 insertions(+), 270 deletions(-)
Range-diff versus v3:
 1:  30e662de96 =  1:  69bca2e49d packfile: introduce a new `struct packfile_store`
 2:  60840ec43f !  2:  ff52df37e6 odb: move list of packfiles into `struct packfile_store`
    @@ packfile.c: struct packfile_store *packfile_store_new(struct object_database *od
     +	for (struct packed_git *p = store->packs; p; p = p->next) {
     +		if (p->do_not_close)
     +			BUG("want to close pack marked 'do-not-close'");
    -+		else
    -+			close_pack(p);
    ++		close_pack(p);
     +	}
     +}
     
 3:  a593254243 =  3:  4ebb964950 odb: move initialization bit into `struct packfile_store`
 4:  291a410d78 !  4:  f481469434 odb: move packfile map into `struct packfile_store`
    @@ Commit message
     
         The object database tracks a map of packfiles by their respective paths,
         which is used to figure out whether a given packfile has already been
    -    loaded.With the introduction of the `struct packfile_store` we have a
    +    loaded. With the introduction of the `struct packfile_store` we have a
         better place to host this list though.
     
    -    Move the map accordingly. `pack_map_entry_cmp()` isn't used anywhere but
    -    in "packfile.c" anymore after this change, so we convert it to a static
    -    function, as well.
    +    Move the map accordingly.
    +
    +    `pack_map_entry_cmp()` isn't used anywhere but in "packfile.c" anymore
    +    after this change, so we convert it to a static function, as well. Note
    +    that we also drop the `inline` hint: the function is used as a callback
    +    function exclusively, and callbacks cannot be inlined.
     
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
     
 5:  8bc7d46524 =  5:  4d54056618 odb: move MRU list of packfiles into `struct packfile_store`
 6:  2e4e13e60b =  6:  c9a61cafc6 odb: move kept cache into `struct packfile_store`
 7:  7a873bf4d1 =  7:  4896d70b5c packfile: reorder functions to avoid function declaration
 8:  fc60fc0cae =  8:  c47bf63f5c packfile: refactor `prepare_packed_git()` to work on packfile store
 9:  f15c3569ae =  9:  99203234a1 packfile: split up responsibilities of `reprepare_packed_git()`
10:  114e9ac17a = 10:  163f9acf73 packfile: refactor `install_packed_git()` to work on packfile store
11:  17aefbdef9 = 11:  8c73603ad2 packfile: introduce function to load and add packfiles
12:  9666abfb08 = 12:  199e606682 packfile: move `get_multi_pack_index()` into "midx.c"
13:  7e90938546 = 13:  f37281543d packfile: remove `get_packed_git()`
14:  f378932538 = 14:  250a389b66 packfile: refactor `get_all_packs()` to work on packfile store
15:  0b3b613612 = 15:  1d194eba71 packfile: refactor `get_packed_git_mru()` to work on packfile store

--- base-commit: 337c7a0bbcf228ce11c87d066ecee352b3e52467 change-id: 20250806-b4-pks-packfiles-store-a44a608ca396

Patrick Steinhardt· Sep 9, 2025, 11:03 UTC · re: Patrick Steinhardt · lore

[PATCH v4 01/15] packfile: introduce a new `struct packfile_store`

Information about a object database's packfiles is currently distributed across two different structures:

  - `struct packed_git` contains the `next` pointer as well as the
    `mru_head`, both of which serve to store the list of packfiles.
  - `struct object_database` contains several fields that relate to the
    packfiles.

So we don't really have a central data structure that tracks our packfiles, and consequently responsibilities aren't always clear cut. A consequence for the upcoming pluggable object databases is that this makes it very hard to move management of packfiles from the object database level down into the object database source.

Introduce a new `struct packfile_store` which is about to become the single source of truth for managing packfiles. Right now this data structure doesn't yet contain anything, but in subsequent patches we will move all data structures that relate to packfiles and that are currently contained in `struct object_database` into this new home.

Note that this is only a first step: most importantly, we won't (yet) move the `struct packed_git::next` pointer around. This will happen in a subsequent patch series though so that `struct packed_git` will really only host information about the specific packfile it represents.

Further note that the new structure still sits at the wrong level at the end of this patch series: as mentioned, it should eventually sit at the level of the object database source, not at the object database level. But introducing the packfile store now already makes it way easier to eventually push down the now-selfcontained data structure by one level.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb.c      |  1 +
 odb.h      |  3 ++-
 packfile.c | 13 +++++++++++++
 packfile.h | 18 ++++++++++++++++++
 4 files changed, 34 insertions(+), 1 deletion(-)
Show changes to 4 files +34 −1

odb.c, odb.h, packfile.c, packfile.h

diff --git a/odb.c b/odb.c
index 75c443fe66..a2289ea97d 100644
--- a/odb.c
+++ b/odb.c
@@ -996,6 +996,7 @@ struct object_database *odb_new(struct repository *repo)
 
 	memset(o, 0, sizeof(*o));
 	o->repo = repo;
+	o->packfiles = packfile_store_new(o);
 	INIT_LIST_HEAD(&o->packed_git_mru);
 	hashmap_init(&o->pack_map, pack_map_entry_cmp, NULL, 0);
 	pthread_mutex_init(&o->replace_mutex, NULL);
diff --git a/odb.h b/odb.h
index 51fe8a5a92..33034eaf2f 100644
--- a/odb.h
+++ b/odb.h
@@ -91,6 +91,7 @@ struct odb_source {
 };
 
 struct packed_git;
+struct packfile_store;
 struct cached_object_entry;
 
 /*
@@ -136,7 +137,7 @@ struct object_database {
 	 *
 	 * should only be accessed directly by packfile.c
 	 */
-
+	struct packfile_store *packfiles;
 	struct packed_git *packed_git;
 	/* A most-recently-used ordered version of the packed_git list. */
 	struct list_head packed_git_mru;
diff --git a/packfile.c b/packfile.c
index acb680966d..130d3e2507 100644
--- a/packfile.c
+++ b/packfile.c
@@ -2332,3 +2332,16 @@ int parse_pack_header_option(const char *in, unsigned char *out, unsigned int *l
 	*len = hdr - out;
 	return 0;
 }
+
+struct packfile_store *packfile_store_new(struct object_database *odb)
+{
+	struct packfile_store *store;
+	CALLOC_ARRAY(store, 1);
+	store->odb = odb;
+	return store;
+}
+
+void packfile_store_free(struct packfile_store *store)
+{
+	free(store);
+}
diff --git a/packfile.h b/packfile.h
index f16753f2a9..8d31fd619a 100644
--- a/packfile.h
+++ b/packfile.h
@@ -52,6 +52,24 @@ struct packed_git {
 	char pack_name[FLEX_ARRAY]; /* more */
 };
 
+/*
+ * A store that manages packfiles for a given object database.
+ */
+struct packfile_store {
+	struct object_database *odb;
+};
+
+/*
+ * Allocate and initialize a new empty packfile store for the given object
+ * database.
+ */
+struct packfile_store *packfile_store_new(struct object_database *odb);
+
+/*
+ * Free the packfile store and all its associated state.
+ */
+void packfile_store_free(struct packfile_store *store);
+
 static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
 				     const struct hashmap_entry *entry,
 				     const struct hashmap_entry *entry2,
-- 
2.51.0.450.g87641ccf93.dirty
Patrick Steinhardt· Sep 9, 2025, 11:03 UTC · re: Patrick Steinhardt · lore

[PATCH v4 02/15] odb: move list of packfiles into `struct packfile_store`

The object database tracks the list of packfiles it currently knows about. With the introduction of the `struct packfile_store` we have a better place to host this list though.

Move the list accordingly. Extract the logic from `odb_clear()` that knows to close all such packfiles and move it into the new subsystem, as well.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb.c      | 12 ++----------
 odb.h      |  1 -
 packfile.c | 45 ++++++++++++++++++++++++++++-----------------
 packfile.h | 15 ++++++++++++++-
 4 files changed, 44 insertions(+), 29 deletions(-)
Show changes to 4 files +44 −29

odb.c, odb.h, packfile.c, packfile.h

diff --git a/odb.c b/odb.c
index a2289ea97d..7201d01406 100644
--- a/odb.c
+++ b/odb.c
@@ -1038,16 +1038,8 @@ void odb_clear(struct object_database *o)
 
 	INIT_LIST_HEAD(&o->packed_git_mru);
 	close_object_store(o);
-
-	/*
-	 * `close_object_store()` only closes the packfiles, but doesn't free
-	 * them. We thus have to do this manually.
-	 */
-	for (struct packed_git *p = o->packed_git, *next; p; p = next) {
-		next = p->next;
-		free(p);
-	}
-	o->packed_git = NULL;
+	packfile_store_free(o->packfiles);
+	o->packfiles = NULL;
 
 	hashmap_clear(&o->pack_map);
 	string_list_clear(&o->submodule_source_paths, 0);
diff --git a/odb.h b/odb.h
index 33034eaf2f..22a170b434 100644
--- a/odb.h
+++ b/odb.h
@@ -138,7 +138,6 @@ struct object_database {
 	 * should only be accessed directly by packfile.c
 	 */
 	struct packfile_store *packfiles;
-	struct packed_git *packed_git;
 	/* A most-recently-used ordered version of the packed_git list. */
 	struct list_head packed_git_mru;
 
diff --git a/packfile.c b/packfile.c
index 130d3e2507..e467948d28 100644
--- a/packfile.c
+++ b/packfile.c
@@ -278,7 +278,7 @@ static int unuse_one_window(struct packed_git *current)
 
 	if (current)
 		scan_windows(current, &lru_p, &lru_w, &lru_l);
-	for (p = current->repo->objects->packed_git; p; p = p->next)
+	for (p = current->repo->objects->packfiles->packs; p; p = p->next)
 		scan_windows(p, &lru_p, &lru_w, &lru_l);
 	if (lru_p) {
 		munmap(lru_w->base, lru_w->len);
@@ -362,13 +362,8 @@ void close_pack(struct packed_git *p)
 void close_object_store(struct object_database *o)
 {
 	struct odb_source *source;
-	struct packed_git *p;
 
-	for (p = o->packed_git; p; p = p->next)
-		if (p->do_not_close)
-			BUG("want to close pack marked 'do-not-close'");
-		else
-			close_pack(p);
+	packfile_store_close(o->packfiles);
 
 	for (source = o->sources; source; source = source->next) {
 		if (source->midx)
@@ -468,7 +463,7 @@ static int close_one_pack(struct repository *r)
 	struct pack_window *mru_w = NULL;
 	int accept_windows_inuse = 1;
 
-	for (p = r->objects->packed_git; p; p = p->next) {
+	for (p = r->objects->packfiles->packs; p; p = p->next) {
 		if (p->pack_fd == -1)
 			continue;
 		find_lru_pack(p, &lru_p, &mru_w, &accept_windows_inuse);
@@ -789,8 +784,8 @@ void install_packed_git(struct repository *r, struct packed_git *pack)
 	if (pack->pack_fd != -1)
 		pack_open_fds++;
 
-	pack->next = r->objects->packed_git;
-	r->objects->packed_git = pack;
+	pack->next = r->objects->packfiles->packs;
+	r->objects->packfiles->packs = pack;
 
 	hashmap_entry_init(&pack->packmap_ent, strhash(pack->pack_name));
 	hashmap_add(&r->objects->pack_map, &pack->packmap_ent);
@@ -974,7 +969,7 @@ unsigned long repo_approximate_object_count(struct repository *r)
 				count += m->num_objects;
 		}
 
-		for (p = r->objects->packed_git; p; p = p->next) {
+		for (p = r->objects->packfiles->packs; p; p = p->next) {
 			if (open_pack_index(p))
 				continue;
 			count += p->num_objects;
@@ -1015,7 +1010,7 @@ static int sort_pack(const struct packed_git *a, const struct packed_git *b)
 
 static void rearrange_packed_git(struct repository *r)
 {
-	sort_packs(&r->objects->packed_git, sort_pack);
+	sort_packs(&r->objects->packfiles->packs, sort_pack);
 }
 
 static void prepare_packed_git_mru(struct repository *r)
@@ -1024,7 +1019,7 @@ static void prepare_packed_git_mru(struct repository *r)
 
 	INIT_LIST_HEAD(&r->objects->packed_git_mru);
 
-	for (p = r->objects->packed_git; p; p = p->next)
+	for (p = r->objects->packfiles->packs; p; p = p->next)
 		list_add_tail(&p->mru, &r->objects->packed_git_mru);
 }
 
@@ -1073,7 +1068,7 @@ void reprepare_packed_git(struct repository *r)
 struct packed_git *get_packed_git(struct repository *r)
 {
 	prepare_packed_git(r);
-	return r->objects->packed_git;
+	return r->objects->packfiles->packs;
 }
 
 struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
@@ -1094,7 +1089,7 @@ struct packed_git *get_all_packs(struct repository *r)
 			prepare_midx_pack(m, i);
 	}
 
-	return r->objects->packed_git;
+	return r->objects->packfiles->packs;
 }
 
 struct list_head *get_packed_git_mru(struct repository *r)
@@ -1219,7 +1214,7 @@ const struct packed_git *has_packed_and_bad(struct repository *r,
 {
 	struct packed_git *p;
 
-	for (p = r->objects->packed_git; p; p = p->next)
+	for (p = r->objects->packfiles->packs; p; p = p->next)
 		if (oidset_contains(&p->bad_objects, oid))
 			return p;
 	return NULL;
@@ -2080,7 +2075,7 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 		if (source->midx && fill_midx_entry(source->midx, oid, e))
 			return 1;
 
-	if (!r->objects->packed_git)
+	if (!r->objects->packfiles->packs)
 		return 0;
 
 	list_for_each(pos, &r->objects->packed_git_mru) {
@@ -2343,5 +2338,21 @@ struct packfile_store *packfile_store_new(struct object_database *odb)
 
 void packfile_store_free(struct packfile_store *store)
 {
+	packfile_store_close(store);
+
+	for (struct packed_git *p = store->packs, *next; p; p = next) {
+		next = p->next;
+		free(p);
+	}
+
 	free(store);
 }
+
+void packfile_store_close(struct packfile_store *store)
+{
+	for (struct packed_git *p = store->packs; p; p = p->next) {
+		if (p->do_not_close)
+			BUG("want to close pack marked 'do-not-close'");
+		close_pack(p);
+	}
+}
diff --git a/packfile.h b/packfile.h
index 8d31fd619a..d7ac8d24b4 100644
--- a/packfile.h
+++ b/packfile.h
@@ -57,6 +57,12 @@ struct packed_git {
  */
 struct packfile_store {
 	struct object_database *odb;
+
+	/*
+	 * The list of packfiles in the order in which they are being added to
+	 * the store.
+	 */
+	struct packed_git *packs;
 };
 
 /*
@@ -66,10 +72,17 @@ struct packfile_store {
 struct packfile_store *packfile_store_new(struct object_database *odb);
 
 /*
- * Free the packfile store and all its associated state.
+ * Free the packfile store and all its associated state. All packfiles
+ * tracked by the store will be closed.
  */
 void packfile_store_free(struct packfile_store *store);
 
+/*
+ * Close all packfiles associated with this store. The packfiles won't be
+ * free'd, so they can be re-opened at a later point in time.
+ */
+void packfile_store_close(struct packfile_store *store);
+
 static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
 				     const struct hashmap_entry *entry,
 				     const struct hashmap_entry *entry2,
-- 
2.51.0.450.g87641ccf93.dirty
Patrick Steinhardt· Sep 9, 2025, 11:03 UTC · re: Patrick Steinhardt · lore

[PATCH v4 03/15] odb: move initialization bit into `struct packfile_store`

The object database knows to skip re-initializing the list of packfiles in case it's already been initialized. Whether or not that is the case is tracked via a separate `initialized` bit that is stored in the object database. With the introduction of the `struct packfile_store` we have a better place to host this bit though.

Move it accordingly. While at it, convert the field into a boolean now that we're allowed to use them in our code base.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb.h      | 6 ------
 packfile.c | 6 +++---
 packfile.h | 6 ++++++
 3 files changed, 9 insertions(+), 9 deletions(-)
Show changes to 3 files +9 −9

odb.h, packfile.c, packfile.h

diff --git a/odb.h b/odb.h
index 22a170b434..bf1b4d4677 100644
--- a/odb.h
+++ b/odb.h
@@ -169,12 +169,6 @@ struct object_database {
 	unsigned long approximate_object_count;
 	unsigned approximate_object_count_valid : 1;
 
-	/*
-	 * Whether packed_git has already been populated with this repository's
-	 * packs.
-	 */
-	unsigned packed_git_initialized : 1;
-
 	/*
 	 * Submodule source paths that will be added as additional sources to
 	 * allow lookup of submodule objects via the main object database.
diff --git a/packfile.c b/packfile.c
index e467948d28..711ccd95f1 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1027,7 +1027,7 @@ static void prepare_packed_git(struct repository *r)
 {
 	struct odb_source *source;
 
-	if (r->objects->packed_git_initialized)
+	if (r->objects->packfiles->initialized)
 		return;
 
 	odb_prepare_alternates(r->objects);
@@ -1038,7 +1038,7 @@ static void prepare_packed_git(struct repository *r)
 	rearrange_packed_git(r);
 
 	prepare_packed_git_mru(r);
-	r->objects->packed_git_initialized = 1;
+	r->objects->packfiles->initialized = true;
 }
 
 void reprepare_packed_git(struct repository *r)
@@ -1060,7 +1060,7 @@ void reprepare_packed_git(struct repository *r)
 		odb_clear_loose_cache(source);
 
 	r->objects->approximate_object_count_valid = 0;
-	r->objects->packed_git_initialized = 0;
+	r->objects->packfiles->initialized = false;
 	prepare_packed_git(r);
 	obj_read_unlock();
 }
diff --git a/packfile.h b/packfile.h
index d7ac8d24b4..cf81091175 100644
--- a/packfile.h
+++ b/packfile.h
@@ -63,6 +63,12 @@ struct packfile_store {
 	 * the store.
 	 */
 	struct packed_git *packs;
+
+	/*
+	 * Whether packfiles have already been populated with this store's
+	 * packs.
+	 */
+	bool initialized;
 };
 
 /*
-- 
2.51.0.450.g87641ccf93.dirty
Patrick Steinhardt· Sep 9, 2025, 11:03 UTC · re: Patrick Steinhardt · lore

[PATCH v4 04/15] odb: move packfile map into `struct packfile_store`

The object database tracks a map of packfiles by their respective paths, which is used to figure out whether a given packfile has already been loaded. With the introduction of the `struct packfile_store` we have a better place to host this list though.

Move the map accordingly.

`pack_map_entry_cmp()` isn't used anywhere but in "packfile.c" anymore after this change, so we convert it to a static function, as well. Note that we also drop the `inline` hint: the function is used as a callback function exclusively, and callbacks cannot be inlined.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 midx.c     |  2 +-
 odb.c      |  2 --
 odb.h      |  6 ------
 packfile.c | 20 ++++++++++++++++++--
 packfile.h | 20 ++++++--------------
 5 files changed, 25 insertions(+), 25 deletions(-)
Show changes to 5 files +25 −25

midx.c, odb.c, odb.h, packfile.c, packfile.h

diff --git a/midx.c b/midx.c
index 7726c13d7e..e96970efbf 100644
--- a/midx.c
+++ b/midx.c
@@ -460,7 +460,7 @@ int prepare_midx_pack(struct multi_pack_index *m,
 	strbuf_addbuf(&key, &pack_name);
 	strbuf_strip_suffix(&key, ".idx");
 	strbuf_addstr(&key, ".pack");
-	p = hashmap_get_entry_from_hash(&r->objects->pack_map,
+	p = hashmap_get_entry_from_hash(&r->objects->packfiles->map,
 					strhash(key.buf), key.buf,
 					struct packed_git, packmap_ent);
 	if (!p) {
diff --git a/odb.c b/odb.c
index 7201d01406..737d98c911 100644
--- a/odb.c
+++ b/odb.c
@@ -998,7 +998,6 @@ struct object_database *odb_new(struct repository *repo)
 	o->repo = repo;
 	o->packfiles = packfile_store_new(o);
 	INIT_LIST_HEAD(&o->packed_git_mru);
-	hashmap_init(&o->pack_map, pack_map_entry_cmp, NULL, 0);
 	pthread_mutex_init(&o->replace_mutex, NULL);
 	string_list_init_dup(&o->submodule_source_paths);
 	return o;
@@ -1041,6 +1040,5 @@ void odb_clear(struct object_database *o)
 	packfile_store_free(o->packfiles);
 	o->packfiles = NULL;
 
-	hashmap_clear(&o->pack_map);
 	string_list_clear(&o->submodule_source_paths, 0);
 }
diff --git a/odb.h b/odb.h
index bf1b4d4677..73a669b993 100644
--- a/odb.h
+++ b/odb.h
@@ -155,12 +155,6 @@ struct object_database {
 	struct cached_object_entry *cached_objects;
 	size_t cached_object_nr, cached_object_alloc;
 
-	/*
-	 * A map of packfiles to packed_git structs for tracking which
-	 * packs have been loaded already.
-	 */
-	struct hashmap pack_map;
-
 	/*
 	 * A fast, rough count of the number of objects in the repository.
 	 * These two fields are not meant for direct access. Use
diff --git a/packfile.c b/packfile.c
index 711ccd95f1..9006a9ea67 100644
--- a/packfile.c
+++ b/packfile.c
@@ -788,7 +788,7 @@ void install_packed_git(struct repository *r, struct packed_git *pack)
 	r->objects->packfiles->packs = pack;
 
 	hashmap_entry_init(&pack->packmap_ent, strhash(pack->pack_name));
-	hashmap_add(&r->objects->pack_map, &pack->packmap_ent);
+	hashmap_add(&r->objects->packfiles->map, &pack->packmap_ent);
 }
 
 void (*report_garbage)(unsigned seen_bits, const char *path);
@@ -901,7 +901,7 @@ static void prepare_pack(const char *full_name, size_t full_name_len,
 		hashmap_entry_init(&hent, hash);
 
 		/* Don't reopen a pack we already have. */
-		if (!hashmap_get(&data->r->objects->pack_map, &hent, pack_name)) {
+		if (!hashmap_get(&data->r->objects->packfiles->map, &hent, pack_name)) {
 			p = add_packed_git(data->r, full_name, full_name_len, data->local);
 			if (p)
 				install_packed_git(data->r, p);
@@ -2328,11 +2328,26 @@ int parse_pack_header_option(const char *in, unsigned char *out, unsigned int *l
 	return 0;
 }
 
+static int pack_map_entry_cmp(const void *cmp_data UNUSED,
+			      const struct hashmap_entry *entry,
+			      const struct hashmap_entry *entry2,
+			      const void *keydata)
+{
+	const char *key = keydata;
+	const struct packed_git *pg1, *pg2;
+
+	pg1 = container_of(entry, const struct packed_git, packmap_ent);
+	pg2 = container_of(entry2, const struct packed_git, packmap_ent);
+
+	return strcmp(pg1->pack_name, key ? key : pg2->pack_name);
+}
+
 struct packfile_store *packfile_store_new(struct object_database *odb)
 {
 	struct packfile_store *store;
 	CALLOC_ARRAY(store, 1);
 	store->odb = odb;
+	hashmap_init(&store->map, pack_map_entry_cmp, NULL, 0);
 	return store;
 }
 
@@ -2345,6 +2360,7 @@ void packfile_store_free(struct packfile_store *store)
 		free(p);
 	}
 
+	hashmap_clear(&store->map);
 	free(store);
 }
 
diff --git a/packfile.h b/packfile.h
index cf81091175..9bbef51164 100644
--- a/packfile.h
+++ b/packfile.h
@@ -64,6 +64,12 @@ struct packfile_store {
 	 */
 	struct packed_git *packs;
 
+	/*
+	 * A map of packfile names to packed_git structs for tracking which
+	 * packs have been loaded already.
+	 */
+	struct hashmap map;
+
 	/*
 	 * Whether packfiles have already been populated with this store's
 	 * packs.
@@ -89,20 +95,6 @@ void packfile_store_free(struct packfile_store *store);
  */
 void packfile_store_close(struct packfile_store *store);
 
-static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
-				     const struct hashmap_entry *entry,
-				     const struct hashmap_entry *entry2,
-				     const void *keydata)
-{
-	const char *key = keydata;
-	const struct packed_git *pg1, *pg2;
-
-	pg1 = container_of(entry, const struct packed_git, packmap_ent);
-	pg2 = container_of(entry2, const struct packed_git, packmap_ent);
-
-	return strcmp(pg1->pack_name, key ? key : pg2->pack_name);
-}
-
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
-- 
2.51.0.450.g87641ccf93.dirty
Patrick Steinhardt· Sep 9, 2025, 11:03 UTC · re: Patrick Steinhardt · lore

[PATCH v4 05/15] odb: move MRU list of packfiles into `struct packfile_store`

The object database tracks the list of packfiles in most-recently-used order, which is mostly used to favor reading from packfiles that contain most of the objects that we're currently accessing. With the introduction of the `struct packfile_store` we have a better place to host this list though.

Move the list accordingly.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 midx.c     |  2 +-
 odb.c      |  2 --
 odb.h      |  4 ----
 packfile.c | 11 ++++++-----
 packfile.h |  3 +++
 5 files changed, 10 insertions(+), 12 deletions(-)
Show changes to 5 files +10 −12

midx.c, odb.c, odb.h, packfile.c, packfile.h

diff --git a/midx.c b/midx.c
index e96970efbf..91c7b3917d 100644
--- a/midx.c
+++ b/midx.c
@@ -468,7 +468,7 @@ int prepare_midx_pack(struct multi_pack_index *m,
 				   m->source->local);
 		if (p) {
 			install_packed_git(r, p);
-			list_add_tail(&p->mru, &r->objects->packed_git_mru);
+			list_add_tail(&p->mru, &r->objects->packfiles->mru);
 		}
 	}
 
diff --git a/odb.c b/odb.c
index 737d98c911..32e982bf0b 100644
--- a/odb.c
+++ b/odb.c
@@ -997,7 +997,6 @@ struct object_database *odb_new(struct repository *repo)
 	memset(o, 0, sizeof(*o));
 	o->repo = repo;
 	o->packfiles = packfile_store_new(o);
-	INIT_LIST_HEAD(&o->packed_git_mru);
 	pthread_mutex_init(&o->replace_mutex, NULL);
 	string_list_init_dup(&o->submodule_source_paths);
 	return o;
@@ -1035,7 +1034,6 @@ void odb_clear(struct object_database *o)
 		free((char *) o->cached_objects[i].value.buf);
 	FREE_AND_NULL(o->cached_objects);
 
-	INIT_LIST_HEAD(&o->packed_git_mru);
 	close_object_store(o);
 	packfile_store_free(o->packfiles);
 	o->packfiles = NULL;
diff --git a/odb.h b/odb.h
index 73a669b993..8ee1f8bb43 100644
--- a/odb.h
+++ b/odb.h
@@ -3,7 +3,6 @@
 
 #include "hashmap.h"
 #include "object.h"
-#include "list.h"
 #include "oidset.h"
 #include "oidmap.h"
 #include "string-list.h"
@@ -138,9 +137,6 @@ struct object_database {
 	 * should only be accessed directly by packfile.c
 	 */
 	struct packfile_store *packfiles;
-	/* A most-recently-used ordered version of the packed_git list. */
-	struct list_head packed_git_mru;
-
 	struct {
 		struct packed_git **packs;
 		unsigned flags;
diff --git a/packfile.c b/packfile.c
index 9006a9ea67..b0bfe62b5d 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1017,10 +1017,10 @@ static void prepare_packed_git_mru(struct repository *r)
 {
 	struct packed_git *p;
 
-	INIT_LIST_HEAD(&r->objects->packed_git_mru);
+	INIT_LIST_HEAD(&r->objects->packfiles->mru);
 
 	for (p = r->objects->packfiles->packs; p; p = p->next)
-		list_add_tail(&p->mru, &r->objects->packed_git_mru);
+		list_add_tail(&p->mru, &r->objects->packfiles->mru);
 }
 
 static void prepare_packed_git(struct repository *r)
@@ -1095,7 +1095,7 @@ struct packed_git *get_all_packs(struct repository *r)
 struct list_head *get_packed_git_mru(struct repository *r)
 {
 	prepare_packed_git(r);
-	return &r->objects->packed_git_mru;
+	return &r->objects->packfiles->mru;
 }
 
 unsigned long unpack_object_header_buffer(const unsigned char *buf,
@@ -2078,10 +2078,10 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 	if (!r->objects->packfiles->packs)
 		return 0;
 
-	list_for_each(pos, &r->objects->packed_git_mru) {
+	list_for_each(pos, &r->objects->packfiles->mru) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		if (!p->multi_pack_index && fill_pack_entry(oid, e, p)) {
-			list_move(&p->mru, &r->objects->packed_git_mru);
+			list_move(&p->mru, &r->objects->packfiles->mru);
 			return 1;
 		}
 	}
@@ -2347,6 +2347,7 @@ struct packfile_store *packfile_store_new(struct object_database *odb)
 	struct packfile_store *store;
 	CALLOC_ARRAY(store, 1);
 	store->odb = odb;
+	INIT_LIST_HEAD(&store->mru);
 	hashmap_init(&store->map, pack_map_entry_cmp, NULL, 0);
 	return store;
 }
diff --git a/packfile.h b/packfile.h
index 9bbef51164..d48d46cc1b 100644
--- a/packfile.h
+++ b/packfile.h
@@ -64,6 +64,9 @@ struct packfile_store {
 	 */
 	struct packed_git *packs;
 
+	/* A most-recently-used ordered version of the packs list. */
+	struct list_head mru;
+
 	/*
 	 * A map of packfile names to packed_git structs for tracking which
 	 * packs have been loaded already.
-- 
2.51.0.450.g87641ccf93.dirty
Patrick Steinhardt· Sep 9, 2025, 11:03 UTC · re: Patrick Steinhardt · lore

[PATCH v4 06/15] odb: move kept cache into `struct packfile_store`

The object database tracks a cache of "kept" packfiles, which is used by git-pack-objects(1) to handle cruft objects. With the introduction of the `struct packfile_store` we have a better place to host this cache though.

Move the cache accordingly.

This moves the last bit of packfile-related state from the object database into the packfile store. Adapt the comment for the `packfiles` pointer in `struct object_database` to reflect this.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb.h      |  8 +-------
 packfile.c | 16 ++++++++--------
 packfile.h | 14 ++++++++++++++
 3 files changed, 23 insertions(+), 15 deletions(-)
Show changes to 3 files +23 −15

odb.h, packfile.c, packfile.h

diff --git a/odb.h b/odb.h
index 8ee1f8bb43..1c998a2478 100644
--- a/odb.h
+++ b/odb.h
@@ -132,15 +132,9 @@ struct object_database {
 	unsigned commit_graph_attempted : 1; /* if loading has been attempted */
 
 	/*
-	 * private data
-	 *
-	 * should only be accessed directly by packfile.c
+	 * Should only be accessed directly by packfile.c
 	 */
 	struct packfile_store *packfiles;
-	struct {
-		struct packed_git **packs;
-		unsigned flags;
-	} kept_pack_cache;
 
 	/*
 	 * This is meant to hold a *small* number of objects that you would
diff --git a/packfile.c b/packfile.c
index b0bfe62b5d..2145a4edb1 100644
--- a/packfile.c
+++ b/packfile.c
@@ -2091,19 +2091,19 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 static void maybe_invalidate_kept_pack_cache(struct repository *r,
 					     unsigned flags)
 {
-	if (!r->objects->kept_pack_cache.packs)
+	if (!r->objects->packfiles->kept_cache.packs)
 		return;
-	if (r->objects->kept_pack_cache.flags == flags)
+	if (r->objects->packfiles->kept_cache.flags == flags)
 		return;
-	FREE_AND_NULL(r->objects->kept_pack_cache.packs);
-	r->objects->kept_pack_cache.flags = 0;
+	FREE_AND_NULL(r->objects->packfiles->kept_cache.packs);
+	r->objects->packfiles->kept_cache.flags = 0;
 }
 
 struct packed_git **kept_pack_cache(struct repository *r, unsigned flags)
 {
 	maybe_invalidate_kept_pack_cache(r, flags);
 
-	if (!r->objects->kept_pack_cache.packs) {
+	if (!r->objects->packfiles->kept_cache.packs) {
 		struct packed_git **packs = NULL;
 		size_t nr = 0, alloc = 0;
 		struct packed_git *p;
@@ -2126,11 +2126,11 @@ struct packed_git **kept_pack_cache(struct repository *r, unsigned flags)
 		ALLOC_GROW(packs, nr + 1, alloc);
 		packs[nr] = NULL;
 
-		r->objects->kept_pack_cache.packs = packs;
-		r->objects->kept_pack_cache.flags = flags;
+		r->objects->packfiles->kept_cache.packs = packs;
+		r->objects->packfiles->kept_cache.flags = flags;
 	}
 
-	return r->objects->kept_pack_cache.packs;
+	return r->objects->packfiles->kept_cache.packs;
 }
 
 int find_kept_pack_entry(struct repository *r,
diff --git a/packfile.h b/packfile.h
index d48d46cc1b..bf66211986 100644
--- a/packfile.h
+++ b/packfile.h
@@ -64,6 +64,20 @@ struct packfile_store {
 	 */
 	struct packed_git *packs;
 
+	/*
+	 * Cache of packfiles which are marked as "kept", either because there
+	 * is an on-disk ".keep" file or because they are marked as "kept" in
+	 * memory.
+	 *
+	 * Should not be accessed directly, but via `kept_pack_cache()`. The
+	 * list of packs gets invalidated when the stored flags and the flags
+	 * passed to `kept_pack_cache()` mismatch.
+	 */
+	struct {
+		struct packed_git **packs;
+		unsigned flags;
+	} kept_cache;
+
 	/* A most-recently-used ordered version of the packs list. */
 	struct list_head mru;
 
-- 
2.51.0.450.g87641ccf93.dirty
Patrick Steinhardt· Sep 9, 2025, 11:03 UTC · re: Patrick Steinhardt · lore

[PATCH v4 07/15] packfile: reorder functions to avoid function declaration

Reorder functions so that we can avoid a forward declaration of `prepare_packed_git()`.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 packfile.c | 67 +++++++++++++++++++++++++++++++-------------------------------
 1 file changed, 33 insertions(+), 34 deletions(-)
Show changes to packfile.c +33 −34
diff --git a/packfile.c b/packfile.c
index 2145a4edb1..59bc02575f 100644
--- a/packfile.c
+++ b/packfile.c
@@ -946,40 +946,6 @@ static void prepare_packed_git_one(struct odb_source *source)
 	string_list_clear(data.garbage, 0);
 }
 
-static void prepare_packed_git(struct repository *r);
-/*
- * Give a fast, rough count of the number of objects in the repository. This
- * ignores loose objects completely. If you have a lot of them, then either
- * you should repack because your performance will be awful, or they are
- * all unreachable objects about to be pruned, in which case they're not really
- * interesting as a measure of repo size in the first place.
- */
-unsigned long repo_approximate_object_count(struct repository *r)
-{
-	if (!r->objects->approximate_object_count_valid) {
-		struct odb_source *source;
-		unsigned long count = 0;
-		struct packed_git *p;
-
-		prepare_packed_git(r);
-
-		for (source = r->objects->sources; source; source = source->next) {
-			struct multi_pack_index *m = get_multi_pack_index(source);
-			if (m)
-				count += m->num_objects;
-		}
-
-		for (p = r->objects->packfiles->packs; p; p = p->next) {
-			if (open_pack_index(p))
-				continue;
-			count += p->num_objects;
-		}
-		r->objects->approximate_object_count = count;
-		r->objects->approximate_object_count_valid = 1;
-	}
-	return r->objects->approximate_object_count;
-}
-
 DEFINE_LIST_SORT(static, sort_packs, struct packed_git, next);
 
 static int sort_pack(const struct packed_git *a, const struct packed_git *b)
@@ -1098,6 +1064,39 @@ struct list_head *get_packed_git_mru(struct repository *r)
 	return &r->objects->packfiles->mru;
 }
 
+/*
+ * Give a fast, rough count of the number of objects in the repository. This
+ * ignores loose objects completely. If you have a lot of them, then either
+ * you should repack because your performance will be awful, or they are
+ * all unreachable objects about to be pruned, in which case they're not really
+ * interesting as a measure of repo size in the first place.
+ */
+unsigned long repo_approximate_object_count(struct repository *r)
+{
+	if (!r->objects->approximate_object_count_valid) {
+		struct odb_source *source;
+		unsigned long count = 0;
+		struct packed_git *p;
+
+		prepare_packed_git(r);
+
+		for (source = r->objects->sources; source; source = source->next) {
+			struct multi_pack_index *m = get_multi_pack_index(source);
+			if (m)
+				count += m->num_objects;
+		}
+
+		for (p = r->objects->packfiles->packs; p; p = p->next) {
+			if (open_pack_index(p))
+				continue;
+			count += p->num_objects;
+		}
+		r->objects->approximate_object_count = count;
+		r->objects->approximate_object_count_valid = 1;
+	}
+	return r->objects->approximate_object_count;
+}
+
 unsigned long unpack_object_header_buffer(const unsigned char *buf,
 		unsigned long len, enum object_type *type, unsigned long *sizep)
 {
-- 
2.51.0.450.g87641ccf93.dirty
Patrick Steinhardt· Sep 9, 2025, 11:03 UTC · re: Patrick Steinhardt · lore

[PATCH v4 08/15] packfile: refactor `prepare_packed_git()` to work on packfile store

The `prepare_packed_git()` function and its friends are responsible for loading packfiles as well as the multi-pack index for a given object database. Refactor these functions to accept a packfile store instead of a repository to clarify their scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 packfile.c | 41 ++++++++++++++++++-----------------------
 1 file changed, 18 insertions(+), 23 deletions(-)
Show changes to packfile.c +18 −23
diff --git a/packfile.c b/packfile.c
index 59bc02575f..87bd8c7864 100644
--- a/packfile.c
+++ b/packfile.c
@@ -974,37 +974,32 @@ static int sort_pack(const struct packed_git *a, const struct packed_git *b)
 	return -1;
 }
 
-static void rearrange_packed_git(struct repository *r)
-{
-	sort_packs(&r->objects->packfiles->packs, sort_pack);
-}
-
-static void prepare_packed_git_mru(struct repository *r)
+static void packfile_store_prepare_mru(struct packfile_store *store)
 {
 	struct packed_git *p;
 
-	INIT_LIST_HEAD(&r->objects->packfiles->mru);
+	INIT_LIST_HEAD(&store->mru);
 
-	for (p = r->objects->packfiles->packs; p; p = p->next)
-		list_add_tail(&p->mru, &r->objects->packfiles->mru);
+	for (p = store->packs; p; p = p->next)
+		list_add_tail(&p->mru, &store->mru);
 }
 
-static void prepare_packed_git(struct repository *r)
+static void packfile_store_prepare(struct packfile_store *store)
 {
 	struct odb_source *source;
 
-	if (r->objects->packfiles->initialized)
+	if (store->initialized)
 		return;
 
-	odb_prepare_alternates(r->objects);
-	for (source = r->objects->sources; source; source = source->next) {
+	odb_prepare_alternates(store->odb);
+	for (source = store->odb->sources; source; source = source->next) {
 		prepare_multi_pack_index_one(source);
 		prepare_packed_git_one(source);
 	}
-	rearrange_packed_git(r);
+	sort_packs(&store->packs, sort_pack);
 
-	prepare_packed_git_mru(r);
-	r->objects->packfiles->initialized = true;
+	packfile_store_prepare_mru(store);
+	store->initialized = true;
 }
 
 void reprepare_packed_git(struct repository *r)
@@ -1027,25 +1022,25 @@ void reprepare_packed_git(struct repository *r)
 
 	r->objects->approximate_object_count_valid = 0;
 	r->objects->packfiles->initialized = false;
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 	obj_read_unlock();
 }
 
 struct packed_git *get_packed_git(struct repository *r)
 {
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 	return r->objects->packfiles->packs;
 }
 
 struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
 {
-	prepare_packed_git(source->odb->repo);
+	packfile_store_prepare(source->odb->packfiles);
 	return source->midx;
 }
 
 struct packed_git *get_all_packs(struct repository *r)
 {
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 
 	for (struct odb_source *source = r->objects->sources; source; source = source->next) {
 		struct multi_pack_index *m = source->midx;
@@ -1060,7 +1055,7 @@ struct packed_git *get_all_packs(struct repository *r)
 
 struct list_head *get_packed_git_mru(struct repository *r)
 {
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 	return &r->objects->packfiles->mru;
 }
 
@@ -1078,7 +1073,7 @@ unsigned long repo_approximate_object_count(struct repository *r)
 		unsigned long count = 0;
 		struct packed_git *p;
 
-		prepare_packed_git(r);
+		packfile_store_prepare(r->objects->packfiles);
 
 		for (source = r->objects->sources; source; source = source->next) {
 			struct multi_pack_index *m = get_multi_pack_index(source);
@@ -2068,7 +2063,7 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 {
 	struct list_head *pos;
 
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 
 	for (struct odb_source *source = r->objects->sources; source; source = source->next)
 		if (source->midx && fill_midx_entry(source->midx, oid, e))
-- 
2.51.0.450.g87641ccf93.dirty
Patrick Steinhardt· Sep 9, 2025, 11:03 UTC · re: Patrick Steinhardt · lore

[PATCH v4 09/15] packfile: split up responsibilities of `reprepare_packed_git()`

In `reprepare_packed_git()` we perform a couple of operations:
  - We reload alternate object directories.
  - We clear the loose object cache.
  - We reprepare packfiles.

While the logic is hosted in "packfile.c", it clearly reaches into other subsystems that aren't related to packfiles.

Split up the responsibility and introduce `odb_reprepare()` which now becomes responsible for repreparing the whole object database. The existing `reprepare_packed_git()` function is refactored accordingly and only cares about reloading the packfile store now.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/backfill.c     |  2 +-
 builtin/gc.c           |  4 ++--
 builtin/receive-pack.c |  2 +-
 builtin/repack.c       |  2 +-
 bulk-checkin.c         |  2 +-
 connected.c            |  2 +-
 fetch-pack.c           |  4 ++--
 object-name.c          |  2 +-
 odb.c                  | 27 ++++++++++++++++++++++++++-
 odb.h                  |  6 ++++++
 packfile.c             | 26 ++++----------------------
 packfile.h             |  9 ++++++++-
 transport-helper.c     |  2 +-
 13 files changed, 55 insertions(+), 35 deletions(-)
Show changes to 13 files +55 −35

builtin/backfill.c, builtin/gc.c, builtin/receive-pack.c, builtin/repack.c, bulk-checkin.c, connected.c, fetch-pack.c, object-name.c, odb.c, odb.h, packfile.c, packfile.h, transport-helper.c

diff --git a/builtin/backfill.c b/builtin/backfill.c
index 80056abe47..e80fc1b694 100644
--- a/builtin/backfill.c
+++ b/builtin/backfill.c
@@ -53,7 +53,7 @@ static void download_batch(struct backfill_context *ctx)
 	 * We likely have a new packfile. Add it to the packed list to
 	 * avoid possible duplicate downloads of the same objects.
 	 */
-	reprepare_packed_git(ctx->repo);
+	odb_reprepare(ctx->repo->objects);
 }
 
 static int fill_missing_blobs(const char *path UNUSED,
diff --git a/builtin/gc.c b/builtin/gc.c
index 03ae4926b2..aeca06a08b 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1042,7 +1042,7 @@ int cmd_gc(int argc,
 		die(FAILED_RUN, "rerere");
 
 	report_garbage = report_pack_garbage;
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 	if (pack_garbage.nr > 0) {
 		close_object_store(the_repository->objects);
 		clean_pack_garbage();
@@ -1491,7 +1491,7 @@ static off_t get_auto_pack_size(void)
 	struct packed_git *p;
 	struct repository *r = the_repository;
 
-	reprepare_packed_git(r);
+	odb_reprepare(r->objects);
 	for (p = get_all_packs(r); p; p = p->next) {
 		if (p->pack_size > max_size) {
 			second_largest_size = max_size;
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 1113137a6f..c9288a9c7e 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -2389,7 +2389,7 @@ static const char *unpack(int err_fd, struct shallow_info *si)
 		status = finish_command(&child);
 		if (status)
 			return "index-pack abnormal exit";
-		reprepare_packed_git(the_repository);
+		odb_reprepare(the_repository->objects);
 	}
 	return NULL;
 }
diff --git a/builtin/repack.c b/builtin/repack.c
index c490a51e91..5ff27fc8e2 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -1685,7 +1685,7 @@ int cmd_repack(int argc,
 			goto cleanup;
 	}
 
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 
 	if (delete_redundant) {
 		int opts = 0;
diff --git a/bulk-checkin.c b/bulk-checkin.c
index b2809ab039..f65439a748 100644
--- a/bulk-checkin.c
+++ b/bulk-checkin.c
@@ -90,7 +90,7 @@ static void flush_bulk_checkin_packfile(struct bulk_checkin_packfile *state)
 
 	strbuf_release(&packname);
 	/* Make objects we just wrote available to ourselves */
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 }
 
 /*
diff --git a/connected.c b/connected.c
index 18c13245d8..d6e9682fd9 100644
--- a/connected.c
+++ b/connected.c
@@ -72,7 +72,7 @@ int check_connected(oid_iterate_fn fn, void *cb_data,
 		 * Before checking for promisor packs, be sure we have the
 		 * latest pack-files loaded into memory.
 		 */
-		reprepare_packed_git(the_repository);
+		odb_reprepare(the_repository->objects);
 		do {
 			struct packed_git *p;
 
diff --git a/fetch-pack.c b/fetch-pack.c
index 6ed5662951..fe7a84bf2f 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -1983,7 +1983,7 @@ static void update_shallow(struct fetch_pack_args *args,
 		 * remote is shallow, but this is a clone, there are
 		 * no objects in repo to worry about. Accept any
 		 * shallow points that exist in the pack (iow in repo
-		 * after get_pack() and reprepare_packed_git())
+		 * after get_pack() and odb_reprepare())
 		 */
 		struct oid_array extra = OID_ARRAY_INIT;
 		struct object_id *oid = si->shallow->oid;
@@ -2108,7 +2108,7 @@ struct ref *fetch_pack(struct fetch_pack_args *args,
 		ref_cpy = do_fetch_pack(args, fd, ref, sought, nr_sought,
 					&si, pack_lockfiles);
 	}
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 
 	if (!args->cloning && args->deepen) {
 		struct check_connected_options opt = CHECK_CONNECTED_INIT;
diff --git a/object-name.c b/object-name.c
index 732056ff5e..df9e0c5f02 100644
--- a/object-name.c
+++ b/object-name.c
@@ -596,7 +596,7 @@ static enum get_oid_result get_short_oid(struct repository *r,
 	 * or migrated from loose to packed.
 	 */
 	if (status == MISSING_OBJECT) {
-		reprepare_packed_git(r);
+		odb_reprepare(r->objects);
 		find_short_object_filename(&ds);
 		find_short_packed_object(&ds);
 		status = finish_object_disambiguation(&ds, oid);
diff --git a/odb.c b/odb.c
index 32e982bf0b..65a6cc67b6 100644
--- a/odb.c
+++ b/odb.c
@@ -694,7 +694,7 @@ static int do_oid_object_info_extended(struct object_database *odb,
 
 		/* Not a loose object; someone else may have just packed it. */
 		if (!(flags & OBJECT_INFO_QUICK)) {
-			reprepare_packed_git(odb->repo);
+			odb_reprepare(odb->repo->objects);
 			if (find_pack_entry(odb->repo, real, &e))
 				break;
 		}
@@ -1040,3 +1040,28 @@ void odb_clear(struct object_database *o)
 
 	string_list_clear(&o->submodule_source_paths, 0);
 }
+
+void odb_reprepare(struct object_database *o)
+{
+	struct odb_source *source;
+
+	obj_read_lock();
+
+	/*
+	 * Reprepare alt odbs, in case the alternates file was modified
+	 * during the course of this process. This only _adds_ odbs to
+	 * the linked list, so existing odbs will continue to exist for
+	 * the lifetime of the process.
+	 */
+	o->loaded_alternates = 0;
+	odb_prepare_alternates(o);
+
+	for (source = o->sources; source; source = source->next)
+		odb_clear_loose_cache(source);
+
+	o->approximate_object_count_valid = 0;
+
+	packfile_store_reprepare(o->packfiles);
+
+	obj_read_unlock();
+}
diff --git a/odb.h b/odb.h
index 1c998a2478..ef34132c58 100644
--- a/odb.h
+++ b/odb.h
@@ -163,6 +163,12 @@ struct object_database {
 struct object_database *odb_new(struct repository *repo);
 void odb_clear(struct object_database *o);
 
+/*
+ * Clear caches, reload alternates and then reload object sources so that new
+ * objects may become accessible.
+ */
+void odb_reprepare(struct object_database *o);
+
 /*
  * Find source by its object directory path. Returns a `NULL` pointer in case
  * the source could not be found.
diff --git a/packfile.c b/packfile.c
index 87bd8c7864..d0fe1a19f0 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1002,28 +1002,10 @@ static void packfile_store_prepare(struct packfile_store *store)
 	store->initialized = true;
 }
 
-void reprepare_packed_git(struct repository *r)
+void packfile_store_reprepare(struct packfile_store *store)
 {
-	struct odb_source *source;
-
-	obj_read_lock();
-
-	/*
-	 * Reprepare alt odbs, in case the alternates file was modified
-	 * during the course of this process. This only _adds_ odbs to
-	 * the linked list, so existing odbs will continue to exist for
-	 * the lifetime of the process.
-	 */
-	r->objects->loaded_alternates = 0;
-	odb_prepare_alternates(r->objects);
-
-	for (source = r->objects->sources; source; source = source->next)
-		odb_clear_loose_cache(source);
-
-	r->objects->approximate_object_count_valid = 0;
-	r->objects->packfiles->initialized = false;
-	packfile_store_prepare(r->objects->packfiles);
-	obj_read_unlock();
+	store->initialized = false;
+	packfile_store_prepare(store);
 }
 
 struct packed_git *get_packed_git(struct repository *r)
@@ -1144,7 +1126,7 @@ unsigned long get_size_from_delta(struct packed_git *p,
 		 *
 		 * Other worrying sections could be the call to close_pack_fd(),
 		 * which can close packs even with in-use windows, and to
-		 * reprepare_packed_git(). Regarding the former, mmap doc says:
+		 * odb_reprepare(). Regarding the former, mmap doc says:
 		 * "closing the file descriptor does not unmap the region". And
 		 * for the latter, it won't re-open already available packs.
 		 */
diff --git a/packfile.h b/packfile.h
index bf66211986..a85ff607fe 100644
--- a/packfile.h
+++ b/packfile.h
@@ -112,6 +112,14 @@ void packfile_store_free(struct packfile_store *store);
  */
 void packfile_store_close(struct packfile_store *store);
 
+/*
+ * Clear the packfile caches and try to look up any new packfiles that have
+ * appeared since last preparing the packfiles store.
+ *
+ * This function must be called under the `odb_read_lock()`.
+ */
+void packfile_store_reprepare(struct packfile_store *store);
+
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
@@ -188,7 +196,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-void reprepare_packed_git(struct repository *r);
 void install_packed_git(struct repository *r, struct packed_git *pack);
 
 struct packed_git *get_packed_git(struct repository *r);
diff --git a/transport-helper.c b/transport-helper.c
index 0789e5bca5..4d95d84f9e 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -450,7 +450,7 @@ static int fetch_with_fetch(struct transport *transport,
 	}
 	strbuf_release(&buf);
 
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 	return 0;
 }
 
-- 
2.51.0.450.g87641ccf93.dirty
Patrick Steinhardt· Sep 9, 2025, 11:03 UTC · re: Patrick Steinhardt · lore

[PATCH v4 10/15] packfile: refactor `install_packed_git()` to work on packfile store

The `install_packed_git()` functions adds a packfile to a specific object store. Refactor it to accept a packfile store instead of a repository to clarify its scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/fast-import.c |  2 +-
 builtin/index-pack.c  |  2 +-
 http.c                |  2 +-
 http.h                |  2 +-
 midx.c                |  2 +-
 packfile.c            | 11 ++++++-----
 packfile.h            |  9 +++++++--
 7 files changed, 18 insertions(+), 12 deletions(-)
Show changes to 7 files +18 −12

builtin/fast-import.c, builtin/index-pack.c, http.c, http.h, midx.c, packfile.c, packfile.h

diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index 2c35f9345d..e9d82b31c3 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -901,7 +901,7 @@ static void end_packfile(void)
 		if (!new_p)
 			die("core git rejected index %s", idx_name);
 		all_packs[pack_id] = new_p;
-		install_packed_git(the_repository, new_p);
+		packfile_store_add_pack(the_repository->objects->packfiles, new_p);
 		free(idx_name);
 
 		/* Print the boundary */
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index f91c301bba..ed490dfad4 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1645,7 +1645,7 @@ static void final(const char *final_pack_name, const char *curr_pack_name,
 		p = add_packed_git(the_repository, final_index_name,
 				   strlen(final_index_name), 0);
 		if (p)
-			install_packed_git(the_repository, p);
+			packfile_store_add_pack(the_repository->objects->packfiles, p);
 	}
 
 	if (!from_stdin) {
diff --git a/http.c b/http.c
index 98853d6483..af2120b64c 100644
--- a/http.c
+++ b/http.c
@@ -2541,7 +2541,7 @@ void http_install_packfile(struct packed_git *p,
 		lst = &((*lst)->next);
 	*lst = (*lst)->next;
 
-	install_packed_git(the_repository, p);
+	packfile_store_add_pack(the_repository->objects->packfiles, p);
 }
 
 struct http_pack_request *new_http_pack_request(
diff --git a/http.h b/http.h
index 36202139f4..e5a5380c6c 100644
--- a/http.h
+++ b/http.h
@@ -210,7 +210,7 @@ int finish_http_pack_request(struct http_pack_request *preq);
 void release_http_pack_request(struct http_pack_request *preq);
 
 /*
- * Remove p from the given list, and invoke install_packed_git() on it.
+ * Remove p from the given list, and invoke packfile_store_add_pack() on it.
  *
  * This is a convenience function for users that have obtained a list of packs
  * from http_get_info_packs() and have chosen a specific pack to fetch.
diff --git a/midx.c b/midx.c
index 91c7b3917d..69c44be71c 100644
--- a/midx.c
+++ b/midx.c
@@ -467,7 +467,7 @@ int prepare_midx_pack(struct multi_pack_index *m,
 		p = add_packed_git(r, pack_name.buf, pack_name.len,
 				   m->source->local);
 		if (p) {
-			install_packed_git(r, p);
+			packfile_store_add_pack(r->objects->packfiles, p);
 			list_add_tail(&p->mru, &r->objects->packfiles->mru);
 		}
 	}
diff --git a/packfile.c b/packfile.c
index d0fe1a19f0..599bad1279 100644
--- a/packfile.c
+++ b/packfile.c
@@ -779,16 +779,17 @@ struct packed_git *add_packed_git(struct repository *r, const char *path,
 	return p;
 }
 
-void install_packed_git(struct repository *r, struct packed_git *pack)
+void packfile_store_add_pack(struct packfile_store *store,
+			     struct packed_git *pack)
 {
 	if (pack->pack_fd != -1)
 		pack_open_fds++;
 
-	pack->next = r->objects->packfiles->packs;
-	r->objects->packfiles->packs = pack;
+	pack->next = store->packs;
+	store->packs = pack;
 
 	hashmap_entry_init(&pack->packmap_ent, strhash(pack->pack_name));
-	hashmap_add(&r->objects->packfiles->map, &pack->packmap_ent);
+	hashmap_add(&store->map, &pack->packmap_ent);
 }
 
 void (*report_garbage)(unsigned seen_bits, const char *path);
@@ -904,7 +905,7 @@ static void prepare_pack(const char *full_name, size_t full_name_len,
 		if (!hashmap_get(&data->r->objects->packfiles->map, &hent, pack_name)) {
 			p = add_packed_git(data->r, full_name, full_name_len, data->local);
 			if (p)
-				install_packed_git(data->r, p);
+				packfile_store_add_pack(data->r->objects->packfiles, p);
 		}
 		free(pack_name);
 	}
diff --git a/packfile.h b/packfile.h
index a85ff607fe..ba4b0cef9c 100644
--- a/packfile.h
+++ b/packfile.h
@@ -120,6 +120,13 @@ void packfile_store_close(struct packfile_store *store);
  */
 void packfile_store_reprepare(struct packfile_store *store);
 
+/*
+ * Add the pack to the store so that contained objects become accessible via
+ * the store. This moves ownership into the store.
+ */
+void packfile_store_add_pack(struct packfile_store *store,
+			     struct packed_git *pack);
+
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
@@ -196,8 +203,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-void install_packed_git(struct repository *r, struct packed_git *pack);
-
 struct packed_git *get_packed_git(struct repository *r);
 struct list_head *get_packed_git_mru(struct repository *r);
 struct multi_pack_index *get_multi_pack_index(struct odb_source *source);
-- 
2.51.0.450.g87641ccf93.dirty
Patrick Steinhardt· Sep 9, 2025, 11:03 UTC · re: Patrick Steinhardt · lore

[PATCH v4 11/15] packfile: introduce function to load and add packfiles

We have a recurring pattern where we essentially perform an upsert of a packfile in case it isn't yet known by the packfile store. The logic to do so is non-trivial as we have to reconstruct the packfile's key, check the map of packfiles, then create the new packfile and finally add it to the store.

Introduce a new function that does this dance for us. Refactor callsites to use it.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/fast-import.c |  4 ++--
 builtin/index-pack.c  | 10 +++-------
 midx.c                | 23 ++++-------------------
 packfile.c            | 44 +++++++++++++++++++++++++++++++-------------
 packfile.h            |  8 ++++++++
 5 files changed, 48 insertions(+), 41 deletions(-)
Show changes to 5 files +48 −41

builtin/fast-import.c, builtin/index-pack.c, midx.c, packfile.c, packfile.h

diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index e9d82b31c3..a26e79689d 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -897,11 +897,11 @@ static void end_packfile(void)
 		idx_name = keep_pack(create_index());
 
 		/* Register the packfile with core git's machinery. */
-		new_p = add_packed_git(pack_data->repo, idx_name, strlen(idx_name), 1);
+		new_p = packfile_store_load_pack(pack_data->repo->objects->packfiles,
+						 idx_name, 1);
 		if (!new_p)
 			die("core git rejected index %s", idx_name);
 		all_packs[pack_id] = new_p;
-		packfile_store_add_pack(the_repository->objects->packfiles, new_p);
 		free(idx_name);
 
 		/* Print the boundary */
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index ed490dfad4..2b78ba7fe4 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1640,13 +1640,9 @@ static void final(const char *final_pack_name, const char *curr_pack_name,
 	rename_tmp_packfile(&final_index_name, curr_index_name, &index_name,
 			    hash, "idx", 1);
 
-	if (do_fsck_object) {
-		struct packed_git *p;
-		p = add_packed_git(the_repository, final_index_name,
-				   strlen(final_index_name), 0);
-		if (p)
-			packfile_store_add_pack(the_repository->objects->packfiles, p);
-	}
+	if (do_fsck_object)
+		packfile_store_load_pack(the_repository->objects->packfiles,
+					 final_index_name, 0);
 
 	if (!from_stdin) {
 		printf("%s\n", hash_to_hex(hash));
diff --git a/midx.c b/midx.c
index 69c44be71c..3faeaf2f8f 100644
--- a/midx.c
+++ b/midx.c
@@ -443,7 +443,6 @@ int prepare_midx_pack(struct multi_pack_index *m,
 {
 	struct repository *r = m->source->odb->repo;
 	struct strbuf pack_name = STRBUF_INIT;
-	struct strbuf key = STRBUF_INIT;
 	struct packed_git *p;
 
 	pack_int_id = midx_for_pack(&m, pack_int_id);
@@ -455,25 +454,11 @@ int prepare_midx_pack(struct multi_pack_index *m,
 
 	strbuf_addf(&pack_name, "%s/pack/%s", m->source->path,
 		    m->pack_names[pack_int_id]);
-
-	/* pack_map holds the ".pack" name, but we have the .idx */
-	strbuf_addbuf(&key, &pack_name);
-	strbuf_strip_suffix(&key, ".idx");
-	strbuf_addstr(&key, ".pack");
-	p = hashmap_get_entry_from_hash(&r->objects->packfiles->map,
-					strhash(key.buf), key.buf,
-					struct packed_git, packmap_ent);
-	if (!p) {
-		p = add_packed_git(r, pack_name.buf, pack_name.len,
-				   m->source->local);
-		if (p) {
-			packfile_store_add_pack(r->objects->packfiles, p);
-			list_add_tail(&p->mru, &r->objects->packfiles->mru);
-		}
-	}
-
+	p = packfile_store_load_pack(r->objects->packfiles,
+				     pack_name.buf, m->source->local);
+	if (p)
+		list_add_tail(&p->mru, &r->objects->packfiles->mru);
 	strbuf_release(&pack_name);
-	strbuf_release(&key);
 
 	if (!p) {
 		m->packs[pack_int_id] = MIDX_PACK_ERROR;
diff --git a/packfile.c b/packfile.c
index 599bad1279..acfa6042ba 100644
--- a/packfile.c
+++ b/packfile.c
@@ -792,6 +792,33 @@ void packfile_store_add_pack(struct packfile_store *store,
 	hashmap_add(&store->map, &pack->packmap_ent);
 }
 
+struct packed_git *packfile_store_load_pack(struct packfile_store *store,
+					    const char *idx_path, int local)
+{
+	struct strbuf key = STRBUF_INIT;
+	struct packed_git *p;
+
+	/*
+	 * We're being called with the path to the index file, but `pack_map`
+	 * holds the path to the packfile itself.
+	 */
+	strbuf_addstr(&key, idx_path);
+	strbuf_strip_suffix(&key, ".idx");
+	strbuf_addstr(&key, ".pack");
+
+	p = hashmap_get_entry_from_hash(&store->map, strhash(key.buf), key.buf,
+					struct packed_git, packmap_ent);
+	if (!p) {
+		p = add_packed_git(store->odb->repo, idx_path,
+				   strlen(idx_path), local);
+		if (p)
+			packfile_store_add_pack(store, p);
+	}
+
+	strbuf_release(&key);
+	return p;
+}
+
 void (*report_garbage)(unsigned seen_bits, const char *path);
 
 static void report_helper(const struct string_list *list,
@@ -891,23 +918,14 @@ static void prepare_pack(const char *full_name, size_t full_name_len,
 			 const char *file_name, void *_data)
 {
 	struct prepare_pack_data *data = (struct prepare_pack_data *)_data;
-	struct packed_git *p;
 	size_t base_len = full_name_len;
 
 	if (strip_suffix_mem(full_name, &base_len, ".idx") &&
 	    !(data->m && midx_contains_pack(data->m, file_name))) {
-		struct hashmap_entry hent;
-		char *pack_name = xstrfmt("%.*s.pack", (int)base_len, full_name);
-		unsigned int hash = strhash(pack_name);
-		hashmap_entry_init(&hent, hash);
-
-		/* Don't reopen a pack we already have. */
-		if (!hashmap_get(&data->r->objects->packfiles->map, &hent, pack_name)) {
-			p = add_packed_git(data->r, full_name, full_name_len, data->local);
-			if (p)
-				packfile_store_add_pack(data->r->objects->packfiles, p);
-		}
-		free(pack_name);
+		char *trimmed_path = xstrndup(full_name, full_name_len);
+		packfile_store_load_pack(data->r->objects->packfiles,
+					 trimmed_path, data->local);
+		free(trimmed_path);
 	}
 
 	if (!report_garbage)
diff --git a/packfile.h b/packfile.h
index ba4b0cef9c..fcefcbbef6 100644
--- a/packfile.h
+++ b/packfile.h
@@ -127,6 +127,14 @@ void packfile_store_reprepare(struct packfile_store *store);
 void packfile_store_add_pack(struct packfile_store *store,
 			     struct packed_git *pack);
 
+/*
+ * Open the packfile and add it to the store if it isn't yet known. Returns
+ * either the newly opened packfile or the preexisting packfile. Returns a
+ * `NULL` pointer in case the packfile could not be opened.
+ */
+struct packed_git *packfile_store_load_pack(struct packfile_store *store,
+					    const char *idx_path, int local);
+
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
-- 
2.51.0.450.g87641ccf93.dirty
Patrick Steinhardt· Sep 9, 2025, 11:03 UTC · re: Patrick Steinhardt · lore

[PATCH v4 12/15] packfile: move `get_multi_pack_index()` into "midx.c"

The `get_multi_pack_index()` function is declared and implemented in the packfile subsystem, even though it really belongs into the multi-pack index subsystem. The reason for this is likely that it needs to call `packfile_store_prepare()`, which is not exposed by the packfile system. In a subsequent commit we're about to add another caller outside of the packfile system though, so we'll have to expose the function anyway.

Do so now already and move `get_multi_pack_index()` into the MIDX subsystem.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 midx.c     |  6 ++++++
 midx.h     |  1 +
 packfile.c |  8 +-------
 packfile.h | 10 +++++++++-
 4 files changed, 17 insertions(+), 8 deletions(-)
Show changes to 4 files +17 −8

midx.c, midx.h, packfile.c, packfile.h

diff --git a/midx.c b/midx.c
index 3faeaf2f8f..1d6269f957 100644
--- a/midx.c
+++ b/midx.c
@@ -93,6 +93,12 @@ static int midx_read_object_offsets(const unsigned char *chunk_start,
 	return 0;
 }
 
+struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
+{
+	packfile_store_prepare(source->odb->packfiles);
+	return source->midx;
+}
+
 static struct multi_pack_index *load_multi_pack_index_one(struct odb_source *source,
 							  const char *midx_name)
 {
diff --git a/midx.h b/midx.h
index e241d2d690..6e54d73503 100644
--- a/midx.h
+++ b/midx.h
@@ -94,6 +94,7 @@ void get_midx_chain_filename(struct odb_source *source, struct strbuf *out);
 void get_split_midx_filename_ext(struct odb_source *source, struct strbuf *buf,
 				 const unsigned char *hash, const char *ext);
 
+struct multi_pack_index *get_multi_pack_index(struct odb_source *source);
 struct multi_pack_index *load_multi_pack_index(struct odb_source *source);
 int prepare_midx_pack(struct multi_pack_index *m, uint32_t pack_int_id);
 struct packed_git *nth_midxed_pack(struct multi_pack_index *m,
diff --git a/packfile.c b/packfile.c
index acfa6042ba..e3ac7a2993 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1003,7 +1003,7 @@ static void packfile_store_prepare_mru(struct packfile_store *store)
 		list_add_tail(&p->mru, &store->mru);
 }
 
-static void packfile_store_prepare(struct packfile_store *store)
+void packfile_store_prepare(struct packfile_store *store)
 {
 	struct odb_source *source;
 
@@ -1033,12 +1033,6 @@ struct packed_git *get_packed_git(struct repository *r)
 	return r->objects->packfiles->packs;
 }
 
-struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
-{
-	packfile_store_prepare(source->odb->packfiles);
-	return source->midx;
-}
-
 struct packed_git *get_all_packs(struct repository *r)
 {
 	packfile_store_prepare(r->objects->packfiles);
diff --git a/packfile.h b/packfile.h
index fcefcbbef6..a9e561ac39 100644
--- a/packfile.h
+++ b/packfile.h
@@ -112,6 +112,15 @@ void packfile_store_free(struct packfile_store *store);
  */
 void packfile_store_close(struct packfile_store *store);
 
+/*
+ * Prepare the packfile store by loading packfiles and multi-pack indices for
+ * all alternates. This becomes a no-op if the store is already prepared.
+ *
+ * It shouldn't typically be necessary to call this function directly, as
+ * functions that access the store know to prepare it.
+ */
+void packfile_store_prepare(struct packfile_store *store);
+
 /*
  * Clear the packfile caches and try to look up any new packfiles that have
  * appeared since last preparing the packfiles store.
@@ -213,7 +222,6 @@ extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
 struct packed_git *get_packed_git(struct repository *r);
 struct list_head *get_packed_git_mru(struct repository *r);
-struct multi_pack_index *get_multi_pack_index(struct odb_source *source);
 struct packed_git *get_all_packs(struct repository *r);
 
 /*
-- 
2.51.0.450.g87641ccf93.dirty
Patrick Steinhardt· Sep 9, 2025, 11:03 UTC · re: Patrick Steinhardt · lore

[PATCH v4 13/15] packfile: remove `get_packed_git()`

We have two different functions to retrieve packfiles for a packfile store:

  - `get_packed_git()` returns the list of packfiles after having called
    `prepare_packed_git()`.
  - `get_all_packs()` calls `prepare_packed_git()`, as well, but also
    calls `prepare_midx_pack()` for each pack.

Based on the naming alone one might think that `get_all_packs()` would return more packs than `get_packed_git()`. But that's not the case: both functions end up returning the exact same list of packfiles. The real difference between those functions is that `get_all_packs()` also loads the info of whether or not a packfile is part of a multi-pack index.

Preparing this extra information also shouldn't be significantly more expensive:

  - We have already loaded all packfiles via `prepare_packed_git_one()`.
    So given that multi-pack indices may only refer to packfiles in the
    same object directory we know that we already loaded each packfile.
  - The multi-pack index was prepared via `packfile_store_prepare()`
    already, which calls `prepare_multi_pack_index_one()`.
  - So all that remains to be done is to look up the index of the pack
    in its multi-pack index so that we can store that info in both the
    pack itself and the MIDX.

So it is somewhat confusing to readers that one of these two functions claims to load "all" packfiles while the other one doesn't, even though the ultimate difference is way more nuanced.

Convert all of these sites to use `get_all_packs()` instead and remove `get_packed_git()`. There doesn't seem to be a good reason to discern these two functions.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/gc.c   | 2 +-
 builtin/grep.c | 2 +-
 object-name.c  | 4 ++--
 packfile.c     | 6 ------
 packfile.h     | 1 -
 5 files changed, 4 insertions(+), 11 deletions(-)
Show changes to 5 files +4 −11

builtin/gc.c, builtin/grep.c, object-name.c, packfile.c, packfile.h

diff --git a/builtin/gc.c b/builtin/gc.c
index aeca06a08b..b3eec213d2 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1423,7 +1423,7 @@ static int incremental_repack_auto_condition(struct gc_config *cfg UNUSED)
 	if (incremental_repack_auto_limit < 0)
 		return 1;
 
-	for (p = get_packed_git(the_repository);
+	for (p = get_all_packs(the_repository);
 	     count < incremental_repack_auto_limit && p;
 	     p = p->next) {
 		if (!p->multi_pack_index)
diff --git a/builtin/grep.c b/builtin/grep.c
index 5df6537333..8f0e21bd70 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -1214,7 +1214,7 @@ int cmd_grep(int argc,
 		if (recurse_submodules)
 			repo_read_gitmodules(the_repository, 1);
 		if (startup_info->have_repository)
-			(void)get_packed_git(the_repository);
+			packfile_store_prepare(the_repository->objects->packfiles);
 
 		start_threads(&opt);
 	} else {
diff --git a/object-name.c b/object-name.c
index df9e0c5f02..ecffd2d5b1 100644
--- a/object-name.c
+++ b/object-name.c
@@ -213,7 +213,7 @@ static void find_short_packed_object(struct disambiguate_state *ds)
 			unique_in_midx(m, ds);
 	}
 
-	for (p = get_packed_git(ds->repo); p && !ds->ambiguous;
+	for (p = get_all_packs(ds->repo); p && !ds->ambiguous;
 	     p = p->next)
 		unique_in_pack(p, ds);
 }
@@ -806,7 +806,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)
 			find_abbrev_len_for_midx(m, mad);
 	}
 
-	for (p = get_packed_git(mad->repo); p; p = p->next)
+	for (p = get_all_packs(mad->repo); p; p = p->next)
 		find_abbrev_len_for_pack(p, mad);
 }
 
diff --git a/packfile.c b/packfile.c
index e3ac7a2993..3d975e47f5 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1027,12 +1027,6 @@ void packfile_store_reprepare(struct packfile_store *store)
 	packfile_store_prepare(store);
 }
 
-struct packed_git *get_packed_git(struct repository *r)
-{
-	packfile_store_prepare(r->objects->packfiles);
-	return r->objects->packfiles->packs;
-}
-
 struct packed_git *get_all_packs(struct repository *r)
 {
 	packfile_store_prepare(r->objects->packfiles);
diff --git a/packfile.h b/packfile.h
index a9e561ac39..34c2132863 100644
--- a/packfile.h
+++ b/packfile.h
@@ -220,7 +220,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-struct packed_git *get_packed_git(struct repository *r);
 struct list_head *get_packed_git_mru(struct repository *r);
 struct packed_git *get_all_packs(struct repository *r);
 
-- 
2.51.0.450.g87641ccf93.dirty
Taylor Blau· Sep 11, 2025, 23:37 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v4 13/15] packfile: remove `get_packed_git()`

On Tue, Sep 09, 2025 at 01:03:12PM +0200, Patrick Steinhardt wrote:
Show 14 quoted lines
> We have two different functions to retrieve packfiles for a packfile
> store:
>
>   - `get_packed_git()` returns the list of packfiles after having called
>     `prepare_packed_git()`.
>
>   - `get_all_packs()` calls `prepare_packed_git()`, as well, but also
>     calls `prepare_midx_pack()` for each pack.
>
> Based on the naming alone one might think that `get_all_packs()` would
> return more packs than `get_packed_git()`. But that's not the case: both
> functions end up returning the exact same list of packfiles. The real
> difference between those functions is that `get_all_packs()` also loads
> the info of whether or not a packfile is part of a multi-pack index.

Just reading the newer versions and catching up on some of the discussion here, I want to point out that my earlier comments here explain a case where these two functions do *not* return the same list of packs.

A future version of this series should either prepare all "get_packed_git()" callers to handle packs that are part of the MIDX, or drop this patch and address it later on.

Speaking personally, I think that there is already enough going on in this series, that punting the clean-up out to a separate series where we can discuss the implementation changes in more detail would be the way that I'd suggest approaching it.

Thanks, Taylor

Patrick Steinhardt· Sep 9, 2025, 11:03 UTC · re: Patrick Steinhardt · lore

[PATCH v4 14/15] packfile: refactor `get_all_packs()` to work on packfile store

The `get_all_packs()` function prepares the packfile store and then returns its packfiles. Refactor it to accept a packfile store instead of a repository to clarify its scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/cat-file.c          |  3 ++-
 builtin/count-objects.c     |  3 ++-
 builtin/fast-import.c       |  6 ++++--
 builtin/fsck.c              | 11 +++++++----
 builtin/gc.c                | 10 ++++++----
 builtin/pack-objects.c      | 28 +++++++++++++++++++---------
 builtin/pack-redundant.c    |  6 ++++--
 builtin/repack.c            |  9 ++++++---
 connected.c                 |  3 ++-
 http-backend.c              |  5 +++--
 http.c                      |  3 ++-
 object-name.c               |  4 ++--
 pack-bitmap.c               |  4 ++--
 pack-objects.c              |  3 ++-
 packfile.c                  | 12 ++++++------
 packfile.h                  |  7 ++++++-
 server-info.c               |  3 ++-
 t/helper/test-find-pack.c   |  2 +-
 t/helper/test-pack-mtimes.c |  2 +-
 19 files changed, 79 insertions(+), 45 deletions(-)
Show changes to 19 files +79 −45

builtin/cat-file.c, builtin/count-objects.c, builtin/fast-import.c, builtin/fsck.c, builtin/gc.c, builtin/pack-objects.c, builtin/pack-redundant.c, builtin/repack.c, connected.c, http-backend.c, http.c, object-name.c, pack-bitmap.c, pack-objects.c, packfile.c, packfile.h, server-info.c, t/helper/test-find-pack.c, t/helper/test-pack-mtimes.c

diff --git a/builtin/cat-file.c b/builtin/cat-file.c
index fce0b06451c..461eb2a6e09 100644
--- a/builtin/cat-file.c
+++ b/builtin/cat-file.c
@@ -852,9 +852,10 @@ static void batch_each_object(struct batch_options *opt,
 
 	if (bitmap && !for_each_bitmapped_object(bitmap, &opt->objects_filter,
 						 batch_one_object_bitmapped, &payload)) {
+		struct packfile_store *packs = the_repository->objects->packfiles;
 		struct packed_git *pack;
 
-		for (pack = get_all_packs(the_repository); pack; pack = pack->next) {
+		for (pack = packfile_store_get_packs(packs); pack; pack = pack->next) {
 			if (bitmap_index_contains_pack(bitmap, pack) ||
 			    open_pack_index(pack))
 				continue;
diff --git a/builtin/count-objects.c b/builtin/count-objects.c
index a61d3b46aac..cb03c580553 100644
--- a/builtin/count-objects.c
+++ b/builtin/count-objects.c
@@ -122,6 +122,7 @@ int cmd_count_objects(int argc,
 				      count_loose, count_cruft, NULL, NULL);
 
 	if (verbose) {
+		struct packfile_store *packs = the_repository->objects->packfiles;
 		struct packed_git *p;
 		unsigned long num_pack = 0;
 		off_t size_pack = 0;
@@ -129,7 +130,7 @@ int cmd_count_objects(int argc,
 		struct strbuf pack_buf = STRBUF_INIT;
 		struct strbuf garbage_buf = STRBUF_INIT;
 
-		for (p = get_all_packs(the_repository); p; p = p->next) {
+		for (p = packfile_store_get_packs(packs); p; p = p->next) {
 			if (!p->pack_local)
 				continue;
 			if (open_pack_index(p))
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index a26e79689d5..fea914cf9eb 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -952,6 +952,7 @@ static int store_object(
 	struct object_id *oidout,
 	uintmax_t mark)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	void *out, *delta;
 	struct object_entry *e;
 	unsigned char hdr[96];
@@ -975,7 +976,7 @@ static int store_object(
 	if (e->idx.offset) {
 		duplicate_count_by_type[type]++;
 		return 1;
-	} else if (find_oid_pack(&oid, get_all_packs(the_repository))) {
+	} else if (find_oid_pack(&oid, packfile_store_get_packs(packs))) {
 		e->type = type;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
@@ -1092,6 +1093,7 @@ static void truncate_pack(struct hashfile_checkpoint *checkpoint)
 
 static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	size_t in_sz = 64 * 1024, out_sz = 64 * 1024;
 	unsigned char *in_buf = xmalloc(in_sz);
 	unsigned char *out_buf = xmalloc(out_sz);
@@ -1175,7 +1177,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 		duplicate_count_by_type[OBJ_BLOB]++;
 		truncate_pack(&checkpoint);
 
-	} else if (find_oid_pack(&oid, get_all_packs(the_repository))) {
+	} else if (find_oid_pack(&oid, packfile_store_get_packs(packs))) {
 		e->type = OBJ_BLOB;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
diff --git a/builtin/fsck.c b/builtin/fsck.c
index d2eb9d4fbe9..cabf5401c9f 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -867,19 +867,20 @@ static int mark_packed_for_connectivity(const struct object_id *oid,
 
 static int check_pack_rev_indexes(struct repository *r, int show_progress)
 {
+	struct packfile_store *packs = r->objects->packfiles;
 	struct progress *progress = NULL;
 	uint32_t pack_count = 0;
 	int res = 0;
 
 	if (show_progress) {
-		for (struct packed_git *p = get_all_packs(r); p; p = p->next)
+		for (struct packed_git *p = packfile_store_get_packs(packs); p; p = p->next)
 			pack_count++;
 		progress = start_delayed_progress(the_repository,
 						  "Verifying reverse pack-indexes", pack_count);
 		pack_count = 0;
 	}
 
-	for (struct packed_git *p = get_all_packs(r); p; p = p->next) {
+	for (struct packed_git *p = packfile_store_get_packs(packs); p; p = p->next) {
 		int load_error = load_pack_revindex_from_disk(p);
 
 		if (load_error < 0) {
@@ -999,6 +1000,8 @@ int cmd_fsck(int argc,
 		for_each_packed_object(the_repository,
 				       mark_packed_for_connectivity, NULL, 0);
 	} else {
+		struct packfile_store *packs = the_repository->objects->packfiles;
+
 		odb_prepare_alternates(the_repository->objects);
 		for (source = the_repository->objects->sources; source; source = source->next)
 			fsck_source(source);
@@ -1009,7 +1012,7 @@ int cmd_fsck(int argc,
 			struct progress *progress = NULL;
 
 			if (show_progress) {
-				for (p = get_all_packs(the_repository); p;
+				for (p = packfile_store_get_packs(packs); p;
 				     p = p->next) {
 					if (open_pack_index(p))
 						continue;
@@ -1019,7 +1022,7 @@ int cmd_fsck(int argc,
 				progress = start_progress(the_repository,
 							  _("Checking objects"), total);
 			}
-			for (p = get_all_packs(the_repository); p;
+			for (p = packfile_store_get_packs(packs); p;
 			     p = p->next) {
 				/* verify gives error messages itself */
 				if (verify_pack(the_repository,
diff --git a/builtin/gc.c b/builtin/gc.c
index b3eec213d25..c0a3b54ce74 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -487,9 +487,10 @@ static int too_many_loose_objects(struct gc_config *cfg)
 static struct packed_git *find_base_packs(struct string_list *packs,
 					  unsigned long limit)
 {
+	struct packfile_store *packfiles = the_repository->objects->packfiles;
 	struct packed_git *p, *base = NULL;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(packfiles); p; p = p->next) {
 		if (!p->pack_local || p->is_cruft)
 			continue;
 		if (limit) {
@@ -508,13 +509,14 @@ static struct packed_git *find_base_packs(struct string_list *packs,
 
 static int too_many_packs(struct gc_config *cfg)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 	int cnt;
 
 	if (cfg->gc_auto_pack_limit <= 0)
 		return 0;
 
-	for (cnt = 0, p = get_all_packs(the_repository); p; p = p->next) {
+	for (cnt = 0, p = packfile_store_get_packs(packs); p; p = p->next) {
 		if (!p->pack_local)
 			continue;
 		if (p->pack_keep)
@@ -1423,7 +1425,7 @@ static int incremental_repack_auto_condition(struct gc_config *cfg UNUSED)
 	if (incremental_repack_auto_limit < 0)
 		return 1;
 
-	for (p = get_all_packs(the_repository);
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles);
 	     count < incremental_repack_auto_limit && p;
 	     p = p->next) {
 		if (!p->multi_pack_index)
@@ -1492,7 +1494,7 @@ static off_t get_auto_pack_size(void)
 	struct repository *r = the_repository;
 
 	odb_reprepare(r->objects);
-	for (p = get_all_packs(r); p; p = p->next) {
+	for (p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next) {
 		if (p->pack_size > max_size) {
 			second_largest_size = max_size;
 			max_size = p->pack_size;
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 1494afcf3df..914c6e641d0 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -3831,6 +3831,7 @@ static int pack_mtime_cmp(const void *_a, const void *_b)
 
 static void read_packs_list_from_stdin(struct rev_info *revs)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct strbuf buf = STRBUF_INIT;
 	struct string_list include_packs = STRING_LIST_INIT_DUP;
 	struct string_list exclude_packs = STRING_LIST_INIT_DUP;
@@ -3855,7 +3856,7 @@ static void read_packs_list_from_stdin(struct rev_info *revs)
 	string_list_sort(&exclude_packs);
 	string_list_remove_duplicates(&exclude_packs, 0);
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
 		const char *pack_name = pack_basename(p);
 
 		if ((item = string_list_lookup(&include_packs, pack_name)))
@@ -4076,6 +4077,7 @@ static void enumerate_cruft_objects(void)
 
 static void enumerate_and_traverse_cruft_objects(struct string_list *fresh_packs)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 	struct rev_info revs;
 	int ret;
@@ -4105,7 +4107,7 @@ static void enumerate_and_traverse_cruft_objects(struct string_list *fresh_packs
 	 * Re-mark only the fresh packs as kept so that objects in
 	 * unknown packs do not halt the reachability traversal early.
 	 */
-	for (p = get_all_packs(the_repository); p; p = p->next)
+	for (p = packfile_store_get_packs(packs); p; p = p->next)
 		p->pack_keep_in_core = 0;
 	mark_pack_kept_in_core(fresh_packs, 1);
 
@@ -4122,6 +4124,7 @@ static void enumerate_and_traverse_cruft_objects(struct string_list *fresh_packs
 
 static void read_cruft_objects(void)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct strbuf buf = STRBUF_INIT;
 	struct string_list discard_packs = STRING_LIST_INIT_DUP;
 	struct string_list fresh_packs = STRING_LIST_INIT_DUP;
@@ -4142,7 +4145,7 @@ static void read_cruft_objects(void)
 	string_list_sort(&discard_packs);
 	string_list_sort(&fresh_packs);
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
 		const char *pack_name = pack_basename(p);
 		struct string_list_item *item;
 
@@ -4390,11 +4393,12 @@ static void add_unreachable_loose_objects(struct rev_info *revs)
 
 static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	static struct packed_git *last_found = (void *)1;
 	struct packed_git *p;
 
 	p = (last_found != (void *)1) ? last_found :
-					get_all_packs(the_repository);
+					packfile_store_get_packs(packs);
 
 	while (p) {
 		if ((!p->pack_local || p->pack_keep ||
@@ -4404,7 +4408,7 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 			return 1;
 		}
 		if (p == last_found)
-			p = get_all_packs(the_repository);
+			p = packfile_store_get_packs(packs);
 		else
 			p = p->next;
 		if (p == last_found)
@@ -4436,12 +4440,13 @@ static int loosened_object_can_be_discarded(const struct object_id *oid,
 
 static void loosen_unused_packed_objects(void)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 	uint32_t i;
 	uint32_t loosened_objects_nr = 0;
 	struct object_id oid;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
 		if (!p->pack_local || p->pack_keep || p->pack_keep_in_core)
 			continue;
 
@@ -4742,12 +4747,13 @@ static void get_object_list(struct rev_info *revs, int ac, const char **av)
 
 static void add_extra_kept_packs(const struct string_list *names)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 
 	if (!names->nr)
 		return;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
 		const char *name = basename(p->pack_name);
 		int i;
 
@@ -5185,8 +5191,10 @@ int cmd_pack_objects(int argc,
 
 	add_extra_kept_packs(&keep_pack_list);
 	if (ignore_packed_keep_on_disk) {
+		struct packfile_store *packs = the_repository->objects->packfiles;
 		struct packed_git *p;
-		for (p = get_all_packs(the_repository); p; p = p->next)
+
+		for (p = packfile_store_get_packs(packs); p; p = p->next)
 			if (p->pack_local && p->pack_keep)
 				break;
 		if (!p) /* no keep-able packs found */
@@ -5198,8 +5206,10 @@ int cmd_pack_objects(int argc,
 		 * want to unset "local" based on looking at packs, as
 		 * it also covers non-local objects
 		 */
+		struct packfile_store *packs = the_repository->objects->packfiles;
 		struct packed_git *p;
-		for (p = get_all_packs(the_repository); p; p = p->next) {
+
+		for (p = packfile_store_get_packs(packs); p; p = p->next) {
 			if (!p->pack_local) {
 				have_non_local_packs = 1;
 				break;
diff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c
index fe81c293e3a..721c4e04d5c 100644
--- a/builtin/pack-redundant.c
+++ b/builtin/pack-redundant.c
@@ -566,7 +566,8 @@ static struct pack_list * add_pack(struct packed_git *p)
 
 static struct pack_list * add_pack_file(const char *filename)
 {
-	struct packed_git *p = get_all_packs(the_repository);
+	struct packfile_store *packs = the_repository->objects->packfiles;
+	struct packed_git *p = packfile_store_get_packs(packs);
 
 	if (strlen(filename) < 40)
 		die("Bad pack filename: %s", filename);
@@ -581,7 +582,8 @@ static struct pack_list * add_pack_file(const char *filename)
 
 static void load_all(void)
 {
-	struct packed_git *p = get_all_packs(the_repository);
+	struct packfile_store *packs = the_repository->objects->packfiles;
+	struct packed_git *p = packfile_store_get_packs(packs);
 
 	while (p) {
 		add_pack(p);
diff --git a/builtin/repack.c b/builtin/repack.c
index 5ff27fc8e29..2579090abbd 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -265,10 +265,11 @@ static void existing_packs_release(struct existing_packs *existing)
 static void collect_pack_filenames(struct existing_packs *existing,
 				   const struct string_list *extra_keep)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 	struct strbuf buf = STRBUF_INIT;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
 		int i;
 		const char *base;
 
@@ -497,10 +498,11 @@ static void init_pack_geometry(struct pack_geometry *geometry,
 			       struct existing_packs *existing,
 			       const struct pack_objects_args *args)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 	struct strbuf buf = STRBUF_INIT;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
 		if (args->local && !p->pack_local)
 			/*
 			 * When asked to only repack local packfiles we skip
@@ -1137,11 +1139,12 @@ static int write_filtered_pack(const struct pack_objects_args *args,
 static void combine_small_cruft_packs(FILE *in, size_t combine_cruft_below_size,
 				      struct existing_packs *existing)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 	struct strbuf buf = STRBUF_INIT;
 	size_t i;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
 		if (!(p->is_cruft && p->pack_local))
 			continue;
 
diff --git a/connected.c b/connected.c
index d6e9682fd93..56fedb923d5 100644
--- a/connected.c
+++ b/connected.c
@@ -74,9 +74,10 @@ int check_connected(oid_iterate_fn fn, void *cb_data,
 		 */
 		odb_reprepare(the_repository->objects);
 		do {
+			struct packfile_store *packs = the_repository->objects->packfiles;
 			struct packed_git *p;
 
-			for (p = get_all_packs(the_repository); p; p = p->next) {
+			for (p = packfile_store_get_packs(packs); p; p = p->next) {
 				if (!p->pack_promisor)
 					continue;
 				if (find_pack_entry_one(oid, p))
diff --git a/http-backend.c b/http-backend.c
index d5dfe762bb5..c95aa51b45c 100644
--- a/http-backend.c
+++ b/http-backend.c
@@ -603,18 +603,19 @@ static void get_head(struct strbuf *hdr, char *arg UNUSED)
 static void get_info_packs(struct strbuf *hdr, char *arg UNUSED)
 {
 	size_t objdirlen = strlen(repo_get_object_directory(the_repository));
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct strbuf buf = STRBUF_INIT;
 	struct packed_git *p;
 	size_t cnt = 0;
 
 	select_getanyfile(hdr);
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
 		if (p->pack_local)
 			cnt++;
 	}
 
 	strbuf_grow(&buf, cnt * 53 + 2);
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
 		if (p->pack_local)
 			strbuf_addf(&buf, "P %s\n", p->pack_name + objdirlen + 6);
 	}
diff --git a/http.c b/http.c
index af2120b64c7..321648a81e6 100644
--- a/http.c
+++ b/http.c
@@ -2408,6 +2408,7 @@ static char *fetch_pack_index(unsigned char *hash, const char *base_url)
 static int fetch_and_setup_pack_index(struct packed_git **packs_head,
 	unsigned char *sha1, const char *base_url)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *new_pack, *p;
 	char *tmp_idx = NULL;
 	int ret;
@@ -2416,7 +2417,7 @@ static int fetch_and_setup_pack_index(struct packed_git **packs_head,
 	 * If we already have the pack locally, no need to fetch its index or
 	 * even add it to list; we already have all of its objects.
 	 */
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
 		if (hasheq(p->hash, sha1, the_repository->hash_algo))
 			return 0;
 	}
diff --git a/object-name.c b/object-name.c
index ecffd2d5b12..53356819a3d 100644
--- a/object-name.c
+++ b/object-name.c
@@ -213,7 +213,7 @@ static void find_short_packed_object(struct disambiguate_state *ds)
 			unique_in_midx(m, ds);
 	}
 
-	for (p = get_all_packs(ds->repo); p && !ds->ambiguous;
+	for (p = packfile_store_get_packs(ds->repo->objects->packfiles); p && !ds->ambiguous;
 	     p = p->next)
 		unique_in_pack(p, ds);
 }
@@ -806,7 +806,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)
 			find_abbrev_len_for_midx(m, mad);
 	}
 
-	for (p = get_all_packs(mad->repo); p; p = p->next)
+	for (p = packfile_store_get_packs(mad->repo->objects->packfiles); p; p = p->next)
 		find_abbrev_len_for_pack(p, mad);
 }
 
diff --git a/pack-bitmap.c b/pack-bitmap.c
index 058bdb5d7de..834c300aab7 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -664,7 +664,7 @@ static int open_pack_bitmap(struct repository *r,
 	struct packed_git *p;
 	int ret = -1;
 
-	for (p = get_all_packs(r); p; p = p->next) {
+	for (p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next) {
 		if (open_pack_bitmap_1(bitmap_git, p) == 0) {
 			ret = 0;
 			/*
@@ -3362,7 +3362,7 @@ int verify_bitmap_files(struct repository *r)
 		free(midx_bitmap_name);
 	}
 
-	for (struct packed_git *p = get_all_packs(r);
+	for (struct packed_git *p = packfile_store_get_packs(r->objects->packfiles);
 	     p; p = p->next) {
 		char *pack_bitmap_name = pack_bitmap_filename(p);
 		res |= verify_bitmap_file(r->hash_algo, pack_bitmap_name);
diff --git a/pack-objects.c b/pack-objects.c
index a9d9855063a..668c1136673 100644
--- a/pack-objects.c
+++ b/pack-objects.c
@@ -86,6 +86,7 @@ struct object_entry *packlist_find(struct packing_data *pdata,
 
 static void prepare_in_pack_by_idx(struct packing_data *pdata)
 {
+	struct packfile_store *packs = pdata->repo->objects->packfiles;
 	struct packed_git **mapping, *p;
 	int cnt = 0, nr = 1U << OE_IN_PACK_BITS;
 
@@ -95,7 +96,7 @@ static void prepare_in_pack_by_idx(struct packing_data *pdata)
 	 * (i.e. in_pack_idx also zero) should return NULL.
 	 */
 	mapping[cnt++] = NULL;
-	for (p = get_all_packs(pdata->repo); p; p = p->next, cnt++) {
+	for (p = packfile_store_get_packs(packs); p; p = p->next, cnt++) {
 		if (cnt == nr) {
 			free(mapping);
 			return;
diff --git a/packfile.c b/packfile.c
index 3d975e47f52..8c45d7d1168 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1027,11 +1027,11 @@ void packfile_store_reprepare(struct packfile_store *store)
 	packfile_store_prepare(store);
 }
 
-struct packed_git *get_all_packs(struct repository *r)
+struct packed_git *packfile_store_get_packs(struct packfile_store *store)
 {
-	packfile_store_prepare(r->objects->packfiles);
+	packfile_store_prepare(store);
 
-	for (struct odb_source *source = r->objects->sources; source; source = source->next) {
+	for (struct odb_source *source = store->odb->sources; source; source = source->next) {
 		struct multi_pack_index *m = source->midx;
 		if (!m)
 			continue;
@@ -1039,7 +1039,7 @@ struct packed_git *get_all_packs(struct repository *r)
 			prepare_midx_pack(m, i);
 	}
 
-	return r->objects->packfiles->packs;
+	return store->packs;
 }
 
 struct list_head *get_packed_git_mru(struct repository *r)
@@ -2099,7 +2099,7 @@ struct packed_git **kept_pack_cache(struct repository *r, unsigned flags)
 		 * covers, one kept and one not kept, but the midx returns only
 		 * the non-kept version.
 		 */
-		for (p = get_all_packs(r); p; p = p->next) {
+		for (p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next) {
 			if ((p->pack_keep && (flags & ON_DISK_KEEP_PACKS)) ||
 			    (p->pack_keep_in_core && (flags & IN_CORE_KEEP_PACKS))) {
 				ALLOC_GROW(packs, nr + 1, alloc);
@@ -2196,7 +2196,7 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 	int r = 0;
 	int pack_errors = 0;
 
-	for (p = get_all_packs(repo); p; p = p->next) {
+	for (p = packfile_store_get_packs(repo->objects->packfiles); p; p = p->next) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
 		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&
diff --git a/packfile.h b/packfile.h
index 34c2132863a..86f2c07101f 100644
--- a/packfile.h
+++ b/packfile.h
@@ -136,6 +136,12 @@ void packfile_store_reprepare(struct packfile_store *store);
 void packfile_store_add_pack(struct packfile_store *store,
 			     struct packed_git *pack);
 
+/*
+ * Get all packs managed by the given store, including packfiles that are
+ * referenced by multi-pack indices.
+ */
+struct packed_git *packfile_store_get_packs(struct packfile_store *store);
+
 /*
  * Open the packfile and add it to the store if it isn't yet known. Returns
  * either the newly opened packfile or the preexisting packfile. Returns a
@@ -221,7 +227,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
 struct list_head *get_packed_git_mru(struct repository *r);
-struct packed_git *get_all_packs(struct repository *r);
 
 /*
  * Give a rough count of objects in the repository. This sacrifices accuracy
diff --git a/server-info.c b/server-info.c
index 9bb30d9ab71..d949dea3094 100644
--- a/server-info.c
+++ b/server-info.c
@@ -287,12 +287,13 @@ static int compare_info(const void *a_, const void *b_)
 
 static void init_pack_info(struct repository *r, const char *infofile, int force)
 {
+	struct packfile_store *packs = r->objects->packfiles;
 	struct packed_git *p;
 	int stale;
 	int i;
 	size_t alloc = 0;
 
-	for (p = get_all_packs(r); p; p = p->next) {
+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
 		/* we ignore things on alternate path since they are
 		 * not available to the pullers in general.
 		 */
diff --git a/t/helper/test-find-pack.c b/t/helper/test-find-pack.c
index 611a13a3261..183a777fc54 100644
--- a/t/helper/test-find-pack.c
+++ b/t/helper/test-find-pack.c
@@ -39,7 +39,7 @@ int cmd__find_pack(int argc, const char **argv)
 	if (repo_get_oid(the_repository, argv[0], &oid))
 		die("cannot parse %s as an object name", argv[0]);
 
-	for (p = get_all_packs(the_repository); p; p = p->next)
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next)
 		if (find_pack_entry_one(&oid, p)) {
 			printf("%s\n", p->pack_name);
 			actual_count++;
diff --git a/t/helper/test-pack-mtimes.c b/t/helper/test-pack-mtimes.c
index d51aaa3dc40..cfdfae77a6c 100644
--- a/t/helper/test-pack-mtimes.c
+++ b/t/helper/test-pack-mtimes.c
@@ -37,7 +37,7 @@ int cmd__pack_mtimes(int argc, const char **argv)
 	if (argc != 2)
 		usage(pack_mtimes_usage);
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
 		strbuf_addstr(&buf, basename(p->pack_name));
 		strbuf_strip_suffix(&buf, ".pack");
 		strbuf_addstr(&buf, ".mtimes");
-- 
2.51.0.450.g87641ccf93.dirty
Patrick Steinhardt· Sep 9, 2025, 11:03 UTC · re: Patrick Steinhardt · lore

[PATCH v4 15/15] packfile: refactor `get_packed_git_mru()` to work on packfile store

The `get_packed_git_mru()` function prepares the packfile store and then returns its packfiles in most-recently-used order. Refactor it to accept a packfile store instead of a repository to clarify its scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/pack-objects.c | 4 ++--
 packfile.c             | 6 +++---
 packfile.h             | 7 +++++--
 3 files changed, 10 insertions(+), 7 deletions(-)
Show changes to 3 files +10 −7

builtin/pack-objects.c, packfile.c, packfile.h

diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 914c6e641d..9558ab883e 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -1748,12 +1748,12 @@ static int want_object_in_pack_mtime(const struct object_id *oid,
 		}
 	}
 
-	list_for_each(pos, get_packed_git_mru(the_repository)) {
+	list_for_each(pos, packfile_store_get_packs_mru(the_repository->objects->packfiles)) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		want = want_object_in_pack_one(p, oid, exclude, found_pack, found_offset, found_mtime);
 		if (!exclude && want > 0)
 			list_move(&p->mru,
-				  get_packed_git_mru(the_repository));
+				  packfile_store_get_packs_mru(the_repository->objects->packfiles));
 		if (want != -1)
 			return want;
 	}
diff --git a/packfile.c b/packfile.c
index 8c45d7d116..2f7c277d4a 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1042,10 +1042,10 @@ struct packed_git *packfile_store_get_packs(struct packfile_store *store)
 	return store->packs;
 }
 
-struct list_head *get_packed_git_mru(struct repository *r)
+struct list_head *packfile_store_get_packs_mru(struct packfile_store *store)
 {
-	packfile_store_prepare(r->objects->packfiles);
-	return &r->objects->packfiles->mru;
+	packfile_store_prepare(store);
+	return &store->mru;
 }
 
 /*
diff --git a/packfile.h b/packfile.h
index 86f2c07101..e21ebd75d4 100644
--- a/packfile.h
+++ b/packfile.h
@@ -142,6 +142,11 @@ void packfile_store_add_pack(struct packfile_store *store,
  */
 struct packed_git *packfile_store_get_packs(struct packfile_store *store);
 
+/*
+ * Get all packs in most-recently-used order.
+ */
+struct list_head *packfile_store_get_packs_mru(struct packfile_store *store);
+
 /*
  * Open the packfile and add it to the store if it isn't yet known. Returns
  * either the newly opened packfile or the preexisting packfile. Returns a
@@ -226,8 +231,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-struct list_head *get_packed_git_mru(struct repository *r);
-
 /*
  * Give a rough count of objects in the repository. This sacrifices accuracy
  * for speed.
-- 
2.51.0.450.g87641ccf93.dirty
Karthik Nayak· Sep 10, 2025, 07:35 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v4 00/15] packfile: carve out a new packfile store

Patrick Steinhardt <ps@pks.im> writes:
Show 64 quoted lines
> Hi,
>
> information about a object database's packfiles is currently distributed
> across two different structures:
>
>   - `struct packed_git` contains the `next` pointer as well as the
>     `mru_head`, both of which serve to store the list of packfiles.
>
>   - `struct object_database` contains several fields that relate to the
>     packfiles.
>
> So we don't really have a central data structure that tracks our
> packfiles, and consequently responsibilities aren't always clear cut.
> A consequence for the upcoming pluggable object databases is that this
> makes it very hard to move management of packfiles from the object
> database level down into the object database source.
>
> This patch series introduces a new `struct packfile_store`, which is
> about to become the single source of truth for managing packfiles, and
> carves out the packfile store subsystem.
>
> This is the first step to make packfiles work with pluggable object
> databases. Next steps will be to:
>
>   - Move the `struct packed_git::next` and `struct packed::mru_head`
>     pointers into the packfile store so that `struct packed_git` only
>     tracks a single packfile.
>
>   - Push the `struct packfile_store` down one level so that it's not
>     hosted by the object database anymore, but instead by the object
>     database source.
>
> Changes in v2:
>   - Convert the `initialized` flag into a boolean.
>   - Polish some commit messages.
>   - Some smaller formatting changes to the layout of `struct
>     object_database`.
>   - Link to v1: https://lore.kernel.org/r/20250819-b4-pks-packfiles-store-v1-0-1660842e125a@pks.im
>
> Changes in v3:
>   - Rebased on top of master at 6ad8021821 (The fifth batch, 2025-08-29)
>     with ps/object-store-midx-dedup-info at 13296ac909 (midx: compute
>     paths via their source, 2025-08-11) merged into it. This fixes
>     various conflicts with "seen". There's still two conflicts: a
>     trivial one with jt/de-global-bulk-checkin. And a more complex one
>     with tb/prepare-midx-pack-cleanup. I don't think it's necessary to
>     really address the first one, but I'm unsure how to proceed with the
>     second one given that the patch series still seems to be cooking.
>   - Set `struct object_database::packfiles` to `NULL` after free'ing it.
>   - Add a comment to explain the kept cache.
>   - Fix a missing `obj_read_lock()` call.
>   - Drop the commit that always adds packfiles to the MRU. I've moved
>     this into a subsequent patch series.
>   - Avoid some overly long lines by storing the pointer to the packfile
>     store on the stack.
>   - Point out the difference between `get_all_packs()` and
>     `get_packed_git()`.
>   - Link to v2: https://lore.kernel.org/r/20250821-b4-pks-packfiles-store-v2-0-d10623355e9f@pks.im
>
> Changes in v4:
>   - Small code style improvement as suggested by Junio.
>   - Some commit message improvements as suggested by Karthik.
>   - Link to v3: https://lore.kernel.org/r/20250902-b4-pks-packfiles-store-v3-0-6925278efeda@pks.im
>
The range-diff looked good to me. Thanks,
Karthik
[snip]
Taylor Blau· Sep 11, 2025, 23:40 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v4 00/15] packfile: carve out a new packfile store

On Tue, Sep 09, 2025 at 01:02:59PM +0200, Patrick Steinhardt wrote:
> Changes in v4:
>   - Small code style improvement as suggested by Junio.
>   - Some commit message improvements as suggested by Karthik.
>   - Link to v3: https://lore.kernel.org/r/20250902-b4-pks-packfiles-store-v3-0-6925278efeda@pks.im

Thanks for these changes. I think that this series is getting closer, but I do not think that it is quite ready yet.

The largest outstanding topic is that this round claims that get_all_packs() behaves identically to get_packed_git(), but this is not the case. I mentioned in a response to the patch that removes the latter function, but I think we should:

 - Permit both get_all_packs() and get_packed_git() to coexist for now.
 - In a follow-up series, transition get_packed_git() callers one-by-one
   to use get_all_packs() instead. Each of these commits should include
   IMHO a justification that the change is safe as-is, or include fixes
   to make it safe. I think any "fixes" here are limited to "if
   (p->multi_pack_index) continue;".
 - Once there are no longer any callers of get_packed_git(), we can
   remove it.

If you want to pursue that in this series, I am happy to review and discuss it, but IMHO there is already enough going on here that I think it makes more sense to do that in a separate follow-up.

Thanks, Taylor

Taylor Blau· Sep 11, 2025, 23:42 UTC · re: Taylor Blau · lore

Re: [PATCH v4 00/15] packfile: carve out a new packfile store

On Thu, Sep 11, 2025 at 07:40:01PM -0400, Taylor Blau wrote:
Show 28 quoted lines
> On Tue, Sep 09, 2025 at 01:02:59PM +0200, Patrick Steinhardt wrote:
> > Changes in v4:
> >   - Small code style improvement as suggested by Junio.
> >   - Some commit message improvements as suggested by Karthik.
> >   - Link to v3: https://lore.kernel.org/r/20250902-b4-pks-packfiles-store-v3-0-6925278efeda@pks.im
>
> Thanks for these changes. I think that this series is getting closer,
> but I do not think that it is quite ready yet.
>
> The largest outstanding topic is that this round claims that
> get_all_packs() behaves identically to get_packed_git(), but this is not
> the case. I mentioned in a response to the patch that removes the latter
> function, but I think we should:
>
>  - Permit both get_all_packs() and get_packed_git() to coexist for now.
>
>  - In a follow-up series, transition get_packed_git() callers one-by-one
>    to use get_all_packs() instead. Each of these commits should include
>    IMHO a justification that the change is safe as-is, or include fixes
>    to make it safe. I think any "fixes" here are limited to "if
>    (p->multi_pack_index) continue;".
>
>  - Once there are no longer any callers of get_packed_git(), we can
>    remove it.
>
> If you want to pursue that in this series, I am happy to review and
> discuss it, but IMHO there is already enough going on here that I think
> it makes more sense to do that in a separate follow-up.
Oops, hit "send" too early.

The other outstanding topic that I want to raise is that I think transitioning away from get_all_packs() to the new API in all callers is premature. I would *much* rather see us do this call-by-call when it is necessary to do so rather than forcing all callers onto the new API.

If there is a compelling reason that we must force callers to all use the new API now, I'm happy to discuss that, but as-is I worry that we are changing things too quickly here.

Thanks, Taylor

Patrick Steinhardt· Sep 15, 2025, 07:25 UTC · re: Taylor Blau · lore

Re: [PATCH v4 00/15] packfile: carve out a new packfile store

On Thu, Sep 11, 2025 at 07:42:16PM -0400, Taylor Blau wrote:
Show 40 quoted lines
> On Thu, Sep 11, 2025 at 07:40:01PM -0400, Taylor Blau wrote:
> > On Tue, Sep 09, 2025 at 01:02:59PM +0200, Patrick Steinhardt wrote:
> > > Changes in v4:
> > >   - Small code style improvement as suggested by Junio.
> > >   - Some commit message improvements as suggested by Karthik.
> > >   - Link to v3: https://lore.kernel.org/r/20250902-b4-pks-packfiles-store-v3-0-6925278efeda@pks.im
> >
> > Thanks for these changes. I think that this series is getting closer,
> > but I do not think that it is quite ready yet.
> >
> > The largest outstanding topic is that this round claims that
> > get_all_packs() behaves identically to get_packed_git(), but this is not
> > the case. I mentioned in a response to the patch that removes the latter
> > function, but I think we should:
> >
> >  - Permit both get_all_packs() and get_packed_git() to coexist for now.
> >
> >  - In a follow-up series, transition get_packed_git() callers one-by-one
> >    to use get_all_packs() instead. Each of these commits should include
> >    IMHO a justification that the change is safe as-is, or include fixes
> >    to make it safe. I think any "fixes" here are limited to "if
> >    (p->multi_pack_index) continue;".
> >
> >  - Once there are no longer any callers of get_packed_git(), we can
> >    remove it.
> >
> > If you want to pursue that in this series, I am happy to review and
> > discuss it, but IMHO there is already enough going on here that I think
> > it makes more sense to do that in a separate follow-up.
> 
> Oops, hit "send" too early.
> 
> The other outstanding topic that I want to raise is that I think
> transitioning away from get_all_packs() to the new API in all callers is
> premature. I would *much* rather see us do this call-by-call when it is
> necessary to do so rather than forcing all callers onto the new API.
> 
> If there is a compelling reason that we must force callers to all use
> the new API now, I'm happy to discuss that, but as-is I worry that we
> are changing things too quickly here.

Okay, that's fair. I'll rework this patch then to not drop that function and instead make it work on top of the packfile store, as well. I'll then have another look at this in the next patch series.

Thanks!
Patrick
Patrick Steinhardt· Sep 15, 2025, 08:54 UTC · re: Patrick Steinhardt · lore

[PATCH v5 00/15] packfile: carve out a new packfile store

Hi,

information about a object database's packfiles is currently distributed across two different structures:

  - `struct packed_git` contains the `next` pointer as well as the
    `mru_head`, both of which serve to store the list of packfiles.
  - `struct object_database` contains several fields that relate to the
    packfiles.

So we don't really have a central data structure that tracks our packfiles, and consequently responsibilities aren't always clear cut. A consequence for the upcoming pluggable object databases is that this makes it very hard to move management of packfiles from the object database level down into the object database source.

This patch series introduces a new `struct packfile_store`, which is about to become the single source of truth for managing packfiles, and carves out the packfile store subsystem.

This is the first step to make packfiles work with pluggable object databases. Next steps will be to:

  - Move the `struct packed_git::next` and `struct packed::mru_head`
    pointers into the packfile store so that `struct packed_git` only
    tracks a single packfile.
  - Push the `struct packfile_store` down one level so that it's not
    hosted by the object database anymore, but instead by the object
    database source.
Changes in v2:
  - Convert the `initialized` flag into a boolean.
  - Polish some commit messages.
  - Some smaller formatting changes to the layout of `struct
    object_database`.
  - Link to v1: https://lore.kernel.org/r/20250819-b4-pks-packfiles-store-v1-0-1660842e125a@pks.im
Changes in v3:
  - Rebased on top of master at 6ad8021821 (The fifth batch, 2025-08-29)
    with ps/object-store-midx-dedup-info at 13296ac909 (midx: compute
    paths via their source, 2025-08-11) merged into it. This fixes
    various conflicts with "seen". There's still two conflicts: a
    trivial one with jt/de-global-bulk-checkin. And a more complex one
    with tb/prepare-midx-pack-cleanup. I don't think it's necessary to
    really address the first one, but I'm unsure how to proceed with the
    second one given that the patch series still seems to be cooking.
  - Set `struct object_database::packfiles` to `NULL` after free'ing it.
  - Add a comment to explain the kept cache.
  - Fix a missing `obj_read_lock()` call.
  - Drop the commit that always adds packfiles to the MRU. I've moved
    this into a subsequent patch series.
  - Avoid some overly long lines by storing the pointer to the packfile
    store on the stack.
  - Point out the difference between `get_all_packs()` and
    `get_packed_git()`.
  - Link to v2: https://lore.kernel.org/r/20250821-b4-pks-packfiles-store-v2-0-d10623355e9f@pks.im
Changes in v4:
  - Small code style improvement as suggested by Junio.
  - Some commit message improvements as suggested by Karthik.
  - Link to v3: https://lore.kernel.org/r/20250902-b4-pks-packfiles-store-v3-0-6925278efeda@pks.im
Changes in v5:
  - Stop closing packfiles when freeing the packfile store.
  - Don't drop `get_packed_git()` for now. This will be done in a
    subsequent patch series.
  - Link to v4: https://lore.kernel.org/r/20250909-b4-pks-packfiles-store-v4-0-151c4ba3619f@pks.im
Thanks!
Patrick
---
Patrick Steinhardt (15):
      packfile: introduce a new `struct packfile_store`
      odb: move list of packfiles into `struct packfile_store`
      odb: move initialization bit into `struct packfile_store`
      odb: move packfile map into `struct packfile_store`
      odb: move MRU list of packfiles into `struct packfile_store`
      odb: move kept cache into `struct packfile_store`
      packfile: reorder functions to avoid function declaration
      packfile: refactor `prepare_packed_git()` to work on packfile store
      packfile: split up responsibilities of `reprepare_packed_git()`
      packfile: refactor `install_packed_git()` to work on packfile store
      packfile: introduce function to load and add packfiles
      packfile: move `get_multi_pack_index()` into "midx.c"
      packfile: refactor `get_packed_git()` to work on packfile store
      packfile: refactor `get_all_packs()` to work on packfile store
      packfile: refactor `get_packed_git_mru()` to work on packfile store
 builtin/backfill.c          |   2 +-
 builtin/cat-file.c          |   3 +-
 builtin/count-objects.c     |   3 +-
 builtin/fast-import.c       |  10 +-
 builtin/fsck.c              |  11 +-
 builtin/gc.c                |  14 ++-
 builtin/grep.c              |   2 +-
 builtin/index-pack.c        |  10 +-
 builtin/pack-objects.c      |  32 +++--
 builtin/pack-redundant.c    |   6 +-
 builtin/receive-pack.c      |   2 +-
 builtin/repack.c            |  11 +-
 bulk-checkin.c              |   2 +-
 connected.c                 |   5 +-
 fetch-pack.c                |   4 +-
 http-backend.c              |   5 +-
 http.c                      |   5 +-
 http.h                      |   2 +-
 midx.c                      |  29 ++---
 midx.h                      |   1 +
 object-name.c               |   6 +-
 odb.c                       |  40 +++++--
 odb.h                       |  34 ++----
 pack-bitmap.c               |   4 +-
 pack-objects.c              |   3 +-
 packfile.c                  | 283 ++++++++++++++++++++++++--------------------
 packfile.h                  | 125 ++++++++++++++++---
 server-info.c               |   3 +-
 t/helper/test-find-pack.c   |   2 +-
 t/helper/test-pack-mtimes.c |   2 +-
 transport-helper.c          |   2 +-
 31 files changed, 396 insertions(+), 267 deletions(-)
Range-diff versus v4:
 1:  b006e4f919 =  1:  6dbb423d97 packfile: introduce a new `struct packfile_store`
 2:  078a3aebf1 !  2:  1f402667fd odb: move list of packfiles into `struct packfile_store`
    @@ packfile.c: struct packfile_store *packfile_store_new(struct object_database *od
      
      void packfile_store_free(struct packfile_store *store)
      {
    -+	packfile_store_close(store);
    -+
     +	for (struct packed_git *p = store->packs, *next; p; p = next) {
     +		next = p->next;
     +		free(p);
     +	}
    -+
      	free(store);
      }
     +
 3:  8bbb9b5957 =  3:  b3bae09159 odb: move initialization bit into `struct packfile_store`
 4:  499826d25e !  4:  91d3c6016b odb: move packfile map into `struct packfile_store`
    @@ packfile.c: int parse_pack_header_option(const char *in, unsigned char *out, uns
      }
      
     @@ packfile.c: void packfile_store_free(struct packfile_store *store)
    + 		next = p->next;
      		free(p);
      	}
    - 
     +	hashmap_clear(&store->map);
      	free(store);
      }
 5:  54c02314ac =  5:  5aa029b482 odb: move MRU list of packfiles into `struct packfile_store`
 6:  7810d28c81 =  6:  d2f430f86f odb: move kept cache into `struct packfile_store`
 7:  c5cc67c267 =  7:  1f35516b99 packfile: reorder functions to avoid function declaration
 8:  012059000e =  8:  2c298068b5 packfile: refactor `prepare_packed_git()` to work on packfile store
 9:  f05aa1f700 =  9:  eece281f07 packfile: split up responsibilities of `reprepare_packed_git()`
10:  62ad7cfc0c = 10:  68d057c18c packfile: refactor `install_packed_git()` to work on packfile store
11:  ed795182ac = 11:  0700b475ce packfile: introduce function to load and add packfiles
12:  fc97ab562c = 12:  7d2e76e127 packfile: move `get_multi_pack_index()` into "midx.c"
13:  ececdd0a23 ! 13:  cea2433d84 packfile: remove `get_packed_git()`
    @@ Metadata
     Author: Patrick Steinhardt <ps@pks.im>
     
      ## Commit message ##
    -    packfile: remove `get_packed_git()`
    +    packfile: refactor `get_packed_git()` to work on packfile store
     
    -    We have two different functions to retrieve packfiles for a packfile
    -    store:
    -
    -      - `get_packed_git()` returns the list of packfiles after having called
    -        `prepare_packed_git()`.
    -
    -      - `get_all_packs()` calls `prepare_packed_git()`, as well, but also
    -        calls `prepare_midx_pack()` for each pack.
    -
    -    Based on the naming alone one might think that `get_all_packs()` would
    -    return more packs than `get_packed_git()`. But that's not the case: both
    -    functions end up returning the exact same list of packfiles. The real
    -    difference between those functions is that `get_all_packs()` also loads
    -    the info of whether or not a packfile is part of a multi-pack index.
    -
    -    Preparing this extra information also shouldn't be significantly more
    -    expensive:
    -
    -      - We have already loaded all packfiles via `prepare_packed_git_one()`.
    -        So given that multi-pack indices may only refer to packfiles in the
    -        same object directory we know that we already loaded each packfile.
    -
    -      - The multi-pack index was prepared via `packfile_store_prepare()`
    -        already, which calls `prepare_multi_pack_index_one()`.
    -
    -      - So all that remains to be done is to look up the index of the pack
    -        in its multi-pack index so that we can store that info in both the
    -        pack itself and the MIDX.
    -
    -    So it is somewhat confusing to readers that one of these two functions
    -    claims to load "all" packfiles while the other one doesn't, even though
    -    the ultimate difference is way more nuanced.
    -
    -    Convert all of these sites to use `get_all_packs()` instead and remove
    -    `get_packed_git()`. There doesn't seem to be a good reason to discern
    -    these two functions.
    +    The `get_packed_git()` function prepares the packfile store and then
    +    returns its packfiles. Refactor it to accept a packfile store instead of
    +    a repository to clarify its scope.
     
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
     
    @@ builtin/gc.c: static int incremental_repack_auto_condition(struct gc_config *cfg
      		return 1;
      
     -	for (p = get_packed_git(the_repository);
    -+	for (p = get_all_packs(the_repository);
    ++	for (p = packfile_store_get_packs(the_repository->objects->packfiles);
      	     count < incremental_repack_auto_limit && p;
      	     p = p->next) {
      		if (!p->multi_pack_index)
    @@ builtin/grep.c: int cmd_grep(int argc,
      			repo_read_gitmodules(the_repository, 1);
      		if (startup_info->have_repository)
     -			(void)get_packed_git(the_repository);
    -+			packfile_store_prepare(the_repository->objects->packfiles);
    ++			(void)packfile_store_get_packs(the_repository->objects->packfiles);
      
      		start_threads(&opt);
      	} else {
    @@ object-name.c: static void find_short_packed_object(struct disambiguate_state *d
      	}
      
     -	for (p = get_packed_git(ds->repo); p && !ds->ambiguous;
    -+	for (p = get_all_packs(ds->repo); p && !ds->ambiguous;
    ++	for (p = packfile_store_get_packs(ds->repo->objects->packfiles); p && !ds->ambiguous;
      	     p = p->next)
      		unique_in_pack(p, ds);
      }
    @@ object-name.c: static void find_abbrev_len_packed(struct min_abbrev_data *mad)
      	}
      
     -	for (p = get_packed_git(mad->repo); p; p = p->next)
    -+	for (p = get_all_packs(mad->repo); p; p = p->next)
    ++	for (p = packfile_store_get_packs(mad->repo->objects->packfiles); p; p = p->next)
      		find_abbrev_len_for_pack(p, mad);
      }
      
    @@ packfile.c: void packfile_store_reprepare(struct packfile_store *store)
      }
      
     -struct packed_git *get_packed_git(struct repository *r)
    --{
    ++struct packed_git *packfile_store_get_packs(struct packfile_store *store)
    + {
     -	packfile_store_prepare(r->objects->packfiles);
     -	return r->objects->packfiles->packs;
    --}
    --
    ++	packfile_store_prepare(store);
    ++	return store->packs;
    + }
    + 
      struct packed_git *get_all_packs(struct repository *r)
    - {
    - 	packfile_store_prepare(r->objects->packfiles);
     
      ## packfile.h ##
    +@@ packfile.h: void packfile_store_reprepare(struct packfile_store *store);
    + void packfile_store_add_pack(struct packfile_store *store,
    + 			     struct packed_git *pack);
    + 
    ++/*
    ++ * Get packs managed by the given store. Does not load the MIDX or any packs
    ++ * referenced by it.
    ++ */
    ++struct packed_git *packfile_store_get_packs(struct packfile_store *store);
    ++
    + /*
    +  * Open the packfile and add it to the store if it isn't yet known. Returns
    +  * either the newly opened packfile or the preexisting packfile. Returns a
     @@ packfile.h: int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
      #define PACKDIR_FILE_GARBAGE 4
      extern void (*report_garbage)(unsigned seen_bits, const char *path);
14:  ed7a0d5457 ! 14:  8d5296bbbe packfile: refactor `get_all_packs()` to work on packfile store
    @@ builtin/cat-file.c: static void batch_each_object(struct batch_options *opt,
      		struct packed_git *pack;
      
     -		for (pack = get_all_packs(the_repository); pack; pack = pack->next) {
    -+		for (pack = packfile_store_get_packs(packs); pack; pack = pack->next) {
    ++		for (pack = packfile_store_get_all_packs(packs); pack; pack = pack->next) {
      			if (bitmap_index_contains_pack(bitmap, pack) ||
      			    open_pack_index(pack))
      				continue;
    @@ builtin/count-objects.c: int cmd_count_objects(int argc,
      		struct strbuf garbage_buf = STRBUF_INIT;
      
     -		for (p = get_all_packs(the_repository); p; p = p->next) {
    -+		for (p = packfile_store_get_packs(packs); p; p = p->next) {
    ++		for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
      			if (!p->pack_local)
      				continue;
      			if (open_pack_index(p))
    @@ builtin/fast-import.c: static int store_object(
      		duplicate_count_by_type[type]++;
      		return 1;
     -	} else if (find_oid_pack(&oid, get_all_packs(the_repository))) {
    -+	} else if (find_oid_pack(&oid, packfile_store_get_packs(packs))) {
    ++	} else if (find_oid_pack(&oid, packfile_store_get_all_packs(packs))) {
      		e->type = type;
      		e->pack_id = MAX_PACK_ID;
      		e->idx.offset = 1; /* just not zero! */
    @@ builtin/fast-import.c: static void stream_blob(uintmax_t len, struct object_id *
      		truncate_pack(&checkpoint);
      
     -	} else if (find_oid_pack(&oid, get_all_packs(the_repository))) {
    -+	} else if (find_oid_pack(&oid, packfile_store_get_packs(packs))) {
    ++	} else if (find_oid_pack(&oid, packfile_store_get_all_packs(packs))) {
      		e->type = OBJ_BLOB;
      		e->pack_id = MAX_PACK_ID;
      		e->idx.offset = 1; /* just not zero! */
    @@ builtin/fsck.c: static int mark_packed_for_connectivity(const struct object_id *
      
      	if (show_progress) {
     -		for (struct packed_git *p = get_all_packs(r); p; p = p->next)
    -+		for (struct packed_git *p = packfile_store_get_packs(packs); p; p = p->next)
    ++		for (struct packed_git *p = packfile_store_get_all_packs(packs); p; p = p->next)
      			pack_count++;
      		progress = start_delayed_progress(the_repository,
      						  "Verifying reverse pack-indexes", pack_count);
    @@ builtin/fsck.c: static int mark_packed_for_connectivity(const struct object_id *
      	}
      
     -	for (struct packed_git *p = get_all_packs(r); p; p = p->next) {
    -+	for (struct packed_git *p = packfile_store_get_packs(packs); p; p = p->next) {
    ++	for (struct packed_git *p = packfile_store_get_all_packs(packs); p; p = p->next) {
      		int load_error = load_pack_revindex_from_disk(p);
      
      		if (load_error < 0) {
    @@ builtin/fsck.c: int cmd_fsck(int argc,
      
      			if (show_progress) {
     -				for (p = get_all_packs(the_repository); p;
    -+				for (p = packfile_store_get_packs(packs); p;
    ++				for (p = packfile_store_get_all_packs(packs); p;
      				     p = p->next) {
      					if (open_pack_index(p))
      						continue;
    @@ builtin/fsck.c: int cmd_fsck(int argc,
      							  _("Checking objects"), total);
      			}
     -			for (p = get_all_packs(the_repository); p;
    -+			for (p = packfile_store_get_packs(packs); p;
    ++			for (p = packfile_store_get_all_packs(packs); p;
      			     p = p->next) {
      				/* verify gives error messages itself */
      				if (verify_pack(the_repository,
    @@ builtin/gc.c: static int too_many_loose_objects(struct gc_config *cfg)
      	struct packed_git *p, *base = NULL;
      
     -	for (p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(packfiles); p; p = p->next) {
    ++	for (p = packfile_store_get_all_packs(packfiles); p; p = p->next) {
      		if (!p->pack_local || p->is_cruft)
      			continue;
      		if (limit) {
    @@ builtin/gc.c: static struct packed_git *find_base_packs(struct string_list *pack
      		return 0;
      
     -	for (cnt = 0, p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (cnt = 0, p = packfile_store_get_packs(packs); p; p = p->next) {
    ++	for (cnt = 0, p = packfile_store_get_all_packs(packs); p; p = p->next) {
      		if (!p->pack_local)
      			continue;
      		if (p->pack_keep)
    -@@ builtin/gc.c: static int incremental_repack_auto_condition(struct gc_config *cfg UNUSED)
    - 	if (incremental_repack_auto_limit < 0)
    - 		return 1;
    - 
    --	for (p = get_all_packs(the_repository);
    -+	for (p = packfile_store_get_packs(the_repository->objects->packfiles);
    - 	     count < incremental_repack_auto_limit && p;
    - 	     p = p->next) {
    - 		if (!p->multi_pack_index)
     @@ builtin/gc.c: static off_t get_auto_pack_size(void)
      	struct repository *r = the_repository;
      
      	odb_reprepare(r->objects);
     -	for (p = get_all_packs(r); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next) {
    ++	for (p = packfile_store_get_all_packs(r->objects->packfiles); p; p = p->next) {
      		if (p->pack_size > max_size) {
      			second_largest_size = max_size;
      			max_size = p->pack_size;
    @@ builtin/pack-objects.c: static void read_packs_list_from_stdin(struct rev_info *
      	string_list_remove_duplicates(&exclude_packs, 0);
      
     -	for (p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
    ++	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
      		const char *pack_name = pack_basename(p);
      
      		if ((item = string_list_lookup(&include_packs, pack_name)))
    @@ builtin/pack-objects.c: static void enumerate_and_traverse_cruft_objects(struct
      	 * unknown packs do not halt the reachability traversal early.
      	 */
     -	for (p = get_all_packs(the_repository); p; p = p->next)
    -+	for (p = packfile_store_get_packs(packs); p; p = p->next)
    ++	for (p = packfile_store_get_all_packs(packs); p; p = p->next)
      		p->pack_keep_in_core = 0;
      	mark_pack_kept_in_core(fresh_packs, 1);
      
    @@ builtin/pack-objects.c: static void read_cruft_objects(void)
      	string_list_sort(&fresh_packs);
      
     -	for (p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
    ++	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
      		const char *pack_name = pack_basename(p);
      		struct string_list_item *item;
      
    @@ builtin/pack-objects.c: static void add_unreachable_loose_objects(struct rev_inf
      
      	p = (last_found != (void *)1) ? last_found :
     -					get_all_packs(the_repository);
    -+					packfile_store_get_packs(packs);
    ++					packfile_store_get_all_packs(packs);
      
      	while (p) {
      		if ((!p->pack_local || p->pack_keep ||
    @@ builtin/pack-objects.c: static int has_sha1_pack_kept_or_nonlocal(const struct o
      		}
      		if (p == last_found)
     -			p = get_all_packs(the_repository);
    -+			p = packfile_store_get_packs(packs);
    ++			p = packfile_store_get_all_packs(packs);
      		else
      			p = p->next;
      		if (p == last_found)
    @@ builtin/pack-objects.c: static int loosened_object_can_be_discarded(const struct
      	struct object_id oid;
      
     -	for (p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
    ++	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
      		if (!p->pack_local || p->pack_keep || p->pack_keep_in_core)
      			continue;
      
    @@ builtin/pack-objects.c: static void get_object_list(struct rev_info *revs, int a
      		return;
      
     -	for (p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
    ++	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
      		const char *name = basename(p->pack_name);
      		int i;
      
    @@ builtin/pack-objects.c: int cmd_pack_objects(int argc,
      		struct packed_git *p;
     -		for (p = get_all_packs(the_repository); p; p = p->next)
     +
    -+		for (p = packfile_store_get_packs(packs); p; p = p->next)
    ++		for (p = packfile_store_get_all_packs(packs); p; p = p->next)
      			if (p->pack_local && p->pack_keep)
      				break;
      		if (!p) /* no keep-able packs found */
    @@ builtin/pack-objects.c: int cmd_pack_objects(int argc,
      		struct packed_git *p;
     -		for (p = get_all_packs(the_repository); p; p = p->next) {
     +
    -+		for (p = packfile_store_get_packs(packs); p; p = p->next) {
    ++		for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
      			if (!p->pack_local) {
      				have_non_local_packs = 1;
      				break;
    @@ builtin/pack-redundant.c: static struct pack_list * add_pack(struct packed_git *
      {
     -	struct packed_git *p = get_all_packs(the_repository);
     +	struct packfile_store *packs = the_repository->objects->packfiles;
    -+	struct packed_git *p = packfile_store_get_packs(packs);
    ++	struct packed_git *p = packfile_store_get_all_packs(packs);
      
      	if (strlen(filename) < 40)
      		die("Bad pack filename: %s", filename);
    @@ builtin/pack-redundant.c: static struct pack_list * add_pack_file(const char *fi
      {
     -	struct packed_git *p = get_all_packs(the_repository);
     +	struct packfile_store *packs = the_repository->objects->packfiles;
    -+	struct packed_git *p = packfile_store_get_packs(packs);
    ++	struct packed_git *p = packfile_store_get_all_packs(packs);
      
      	while (p) {
      		add_pack(p);
    @@ builtin/repack.c: static void existing_packs_release(struct existing_packs *exis
      	struct strbuf buf = STRBUF_INIT;
      
     -	for (p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
    ++	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
      		int i;
      		const char *base;
      
    @@ builtin/repack.c: static void init_pack_geometry(struct pack_geometry *geometry,
      	struct strbuf buf = STRBUF_INIT;
      
     -	for (p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
    ++	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
      		if (args->local && !p->pack_local)
      			/*
      			 * When asked to only repack local packfiles we skip
    @@ builtin/repack.c: static int write_filtered_pack(const struct pack_objects_args
      	size_t i;
      
     -	for (p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
    ++	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
      		if (!(p->is_cruft && p->pack_local))
      			continue;
      
    @@ connected.c: int check_connected(oid_iterate_fn fn, void *cb_data,
      			struct packed_git *p;
      
     -			for (p = get_all_packs(the_repository); p; p = p->next) {
    -+			for (p = packfile_store_get_packs(packs); p; p = p->next) {
    ++			for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
      				if (!p->pack_promisor)
      					continue;
      				if (find_pack_entry_one(oid, p))
    @@ http-backend.c: static void get_head(struct strbuf *hdr, char *arg UNUSED)
      
      	select_getanyfile(hdr);
     -	for (p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
    ++	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
      		if (p->pack_local)
      			cnt++;
      	}
      
      	strbuf_grow(&buf, cnt * 53 + 2);
     -	for (p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
    ++	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
      		if (p->pack_local)
      			strbuf_addf(&buf, "P %s\n", p->pack_name + objdirlen + 6);
      	}
    @@ http.c: static int fetch_and_setup_pack_index(struct packed_git **packs_head,
      	 * even add it to list; we already have all of its objects.
      	 */
     -	for (p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
    ++	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
      		if (hasheq(p->hash, sha1, the_repository->hash_algo))
      			return 0;
      	}
     
    - ## object-name.c ##
    -@@ object-name.c: static void find_short_packed_object(struct disambiguate_state *ds)
    - 			unique_in_midx(m, ds);
    - 	}
    - 
    --	for (p = get_all_packs(ds->repo); p && !ds->ambiguous;
    -+	for (p = packfile_store_get_packs(ds->repo->objects->packfiles); p && !ds->ambiguous;
    - 	     p = p->next)
    - 		unique_in_pack(p, ds);
    - }
    -@@ object-name.c: static void find_abbrev_len_packed(struct min_abbrev_data *mad)
    - 			find_abbrev_len_for_midx(m, mad);
    - 	}
    - 
    --	for (p = get_all_packs(mad->repo); p; p = p->next)
    -+	for (p = packfile_store_get_packs(mad->repo->objects->packfiles); p; p = p->next)
    - 		find_abbrev_len_for_pack(p, mad);
    - }
    - 
    -
      ## pack-bitmap.c ##
     @@ pack-bitmap.c: static int open_pack_bitmap(struct repository *r,
      	struct packed_git *p;
      	int ret = -1;
      
     -	for (p = get_all_packs(r); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next) {
    ++	for (p = packfile_store_get_all_packs(r->objects->packfiles); p; p = p->next) {
      		if (open_pack_bitmap_1(bitmap_git, p) == 0) {
      			ret = 0;
      			/*
    @@ pack-bitmap.c: int verify_bitmap_files(struct repository *r)
      	}
      
     -	for (struct packed_git *p = get_all_packs(r);
    -+	for (struct packed_git *p = packfile_store_get_packs(r->objects->packfiles);
    ++	for (struct packed_git *p = packfile_store_get_all_packs(r->objects->packfiles);
      	     p; p = p->next) {
      		char *pack_bitmap_name = pack_bitmap_filename(p);
      		res |= verify_bitmap_file(r->hash_algo, pack_bitmap_name);
    @@ pack-objects.c: static void prepare_in_pack_by_idx(struct packing_data *pdata)
      	 */
      	mapping[cnt++] = NULL;
     -	for (p = get_all_packs(pdata->repo); p; p = p->next, cnt++) {
    -+	for (p = packfile_store_get_packs(packs); p; p = p->next, cnt++) {
    ++	for (p = packfile_store_get_all_packs(packs); p; p = p->next, cnt++) {
      		if (cnt == nr) {
      			free(mapping);
      			return;
     
      ## packfile.c ##
    -@@ packfile.c: void packfile_store_reprepare(struct packfile_store *store)
    - 	packfile_store_prepare(store);
    +@@ packfile.c: struct packed_git *packfile_store_get_packs(struct packfile_store *store)
    + 	return store->packs;
      }
      
     -struct packed_git *get_all_packs(struct repository *r)
    -+struct packed_git *packfile_store_get_packs(struct packfile_store *store)
    ++struct packed_git *packfile_store_get_all_packs(struct packfile_store *store)
      {
     -	packfile_store_prepare(r->objects->packfiles);
     +	packfile_store_prepare(store);
    @@ packfile.c: struct packed_git **kept_pack_cache(struct repository *r, unsigned f
      		 * the non-kept version.
      		 */
     -		for (p = get_all_packs(r); p; p = p->next) {
    -+		for (p = packfile_store_get_packs(r->objects->packfiles); p; p = p->next) {
    ++		for (p = packfile_store_get_all_packs(r->objects->packfiles); p; p = p->next) {
      			if ((p->pack_keep && (flags & ON_DISK_KEEP_PACKS)) ||
      			    (p->pack_keep_in_core && (flags & IN_CORE_KEEP_PACKS))) {
      				ALLOC_GROW(packs, nr + 1, alloc);
    @@ packfile.c: int for_each_packed_object(struct repository *repo, each_packed_obje
      	int pack_errors = 0;
      
     -	for (p = get_all_packs(repo); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(repo->objects->packfiles); p; p = p->next) {
    ++	for (p = packfile_store_get_all_packs(repo->objects->packfiles); p; p = p->next) {
      		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
      			continue;
      		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&
     
      ## packfile.h ##
    -@@ packfile.h: void packfile_store_reprepare(struct packfile_store *store);
    - void packfile_store_add_pack(struct packfile_store *store,
    - 			     struct packed_git *pack);
    +@@ packfile.h: void packfile_store_add_pack(struct packfile_store *store,
    +  */
    + struct packed_git *packfile_store_get_packs(struct packfile_store *store);
      
     +/*
     + * Get all packs managed by the given store, including packfiles that are
     + * referenced by multi-pack indices.
     + */
    -+struct packed_git *packfile_store_get_packs(struct packfile_store *store);
    ++struct packed_git *packfile_store_get_all_packs(struct packfile_store *store);
     +
      /*
       * Open the packfile and add it to the store if it isn't yet known. Returns
    @@ server-info.c: static int compare_info(const void *a_, const void *b_)
      	size_t alloc = 0;
      
     -	for (p = get_all_packs(r); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(packs); p; p = p->next) {
    ++	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
      		/* we ignore things on alternate path since they are
      		 * not available to the pullers in general.
      		 */
    @@ t/helper/test-find-pack.c: int cmd__find_pack(int argc, const char **argv)
      		die("cannot parse %s as an object name", argv[0]);
      
     -	for (p = get_all_packs(the_repository); p; p = p->next)
    -+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next)
    ++	for (p = packfile_store_get_all_packs(the_repository->objects->packfiles); p; p = p->next)
      		if (find_pack_entry_one(&oid, p)) {
      			printf("%s\n", p->pack_name);
      			actual_count++;
    @@ t/helper/test-pack-mtimes.c: int cmd__pack_mtimes(int argc, const char **argv)
      		usage(pack_mtimes_usage);
      
     -	for (p = get_all_packs(the_repository); p; p = p->next) {
    -+	for (p = packfile_store_get_packs(the_repository->objects->packfiles); p; p = p->next) {
    ++	for (p = packfile_store_get_all_packs(the_repository->objects->packfiles); p; p = p->next) {
      		strbuf_addstr(&buf, basename(p->pack_name));
      		strbuf_strip_suffix(&buf, ".pack");
      		strbuf_addstr(&buf, ".mtimes");
15:  6af591f605 ! 15:  444f7f3b3f packfile: refactor `get_packed_git_mru()` to work on packfile store
    @@ builtin/pack-objects.c: static int want_object_in_pack_mtime(const struct object
      	}
     
      ## packfile.c ##
    -@@ packfile.c: struct packed_git *packfile_store_get_packs(struct packfile_store *store)
    +@@ packfile.c: struct packed_git *packfile_store_get_all_packs(struct packfile_store *store)
      	return store->packs;
      }
      
    @@ packfile.c: struct packed_git *packfile_store_get_packs(struct packfile_store *s
      /*
     
      ## packfile.h ##
    -@@ packfile.h: void packfile_store_add_pack(struct packfile_store *store,
    +@@ packfile.h: struct packed_git *packfile_store_get_packs(struct packfile_store *store);
       */
    - struct packed_git *packfile_store_get_packs(struct packfile_store *store);
    + struct packed_git *packfile_store_get_all_packs(struct packfile_store *store);
      
     +/*
     + * Get all packs in most-recently-used order.

--- base-commit: 337c7a0bbcf228ce11c87d066ecee352b3e52467 change-id: 20250806-b4-pks-packfiles-store-a44a608ca396

Patrick Steinhardt· Sep 15, 2025, 08:54 UTC · re: Patrick Steinhardt · lore

[PATCH v5 01/15] packfile: introduce a new `struct packfile_store`

Information about a object database's packfiles is currently distributed across two different structures:

  - `struct packed_git` contains the `next` pointer as well as the
    `mru_head`, both of which serve to store the list of packfiles.
  - `struct object_database` contains several fields that relate to the
    packfiles.

So we don't really have a central data structure that tracks our packfiles, and consequently responsibilities aren't always clear cut. A consequence for the upcoming pluggable object databases is that this makes it very hard to move management of packfiles from the object database level down into the object database source.

Introduce a new `struct packfile_store` which is about to become the single source of truth for managing packfiles. Right now this data structure doesn't yet contain anything, but in subsequent patches we will move all data structures that relate to packfiles and that are currently contained in `struct object_database` into this new home.

Note that this is only a first step: most importantly, we won't (yet) move the `struct packed_git::next` pointer around. This will happen in a subsequent patch series though so that `struct packed_git` will really only host information about the specific packfile it represents.

Further note that the new structure still sits at the wrong level at the end of this patch series: as mentioned, it should eventually sit at the level of the object database source, not at the object database level. But introducing the packfile store now already makes it way easier to eventually push down the now-selfcontained data structure by one level.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb.c      |  1 +
 odb.h      |  3 ++-
 packfile.c | 13 +++++++++++++
 packfile.h | 18 ++++++++++++++++++
 4 files changed, 34 insertions(+), 1 deletion(-)
Show changes to 4 files +34 −1

odb.c, odb.h, packfile.c, packfile.h

diff --git a/odb.c b/odb.c
index 75c443fe66..a2289ea97d 100644
--- a/odb.c
+++ b/odb.c
@@ -996,6 +996,7 @@ struct object_database *odb_new(struct repository *repo)
 
 	memset(o, 0, sizeof(*o));
 	o->repo = repo;
+	o->packfiles = packfile_store_new(o);
 	INIT_LIST_HEAD(&o->packed_git_mru);
 	hashmap_init(&o->pack_map, pack_map_entry_cmp, NULL, 0);
 	pthread_mutex_init(&o->replace_mutex, NULL);
diff --git a/odb.h b/odb.h
index 51fe8a5a92..33034eaf2f 100644
--- a/odb.h
+++ b/odb.h
@@ -91,6 +91,7 @@ struct odb_source {
 };
 
 struct packed_git;
+struct packfile_store;
 struct cached_object_entry;
 
 /*
@@ -136,7 +137,7 @@ struct object_database {
 	 *
 	 * should only be accessed directly by packfile.c
 	 */
-
+	struct packfile_store *packfiles;
 	struct packed_git *packed_git;
 	/* A most-recently-used ordered version of the packed_git list. */
 	struct list_head packed_git_mru;
diff --git a/packfile.c b/packfile.c
index acb680966d..130d3e2507 100644
--- a/packfile.c
+++ b/packfile.c
@@ -2332,3 +2332,16 @@ int parse_pack_header_option(const char *in, unsigned char *out, unsigned int *l
 	*len = hdr - out;
 	return 0;
 }
+
+struct packfile_store *packfile_store_new(struct object_database *odb)
+{
+	struct packfile_store *store;
+	CALLOC_ARRAY(store, 1);
+	store->odb = odb;
+	return store;
+}
+
+void packfile_store_free(struct packfile_store *store)
+{
+	free(store);
+}
diff --git a/packfile.h b/packfile.h
index f16753f2a9..8d31fd619a 100644
--- a/packfile.h
+++ b/packfile.h
@@ -52,6 +52,24 @@ struct packed_git {
 	char pack_name[FLEX_ARRAY]; /* more */
 };
 
+/*
+ * A store that manages packfiles for a given object database.
+ */
+struct packfile_store {
+	struct object_database *odb;
+};
+
+/*
+ * Allocate and initialize a new empty packfile store for the given object
+ * database.
+ */
+struct packfile_store *packfile_store_new(struct object_database *odb);
+
+/*
+ * Free the packfile store and all its associated state.
+ */
+void packfile_store_free(struct packfile_store *store);
+
 static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
 				     const struct hashmap_entry *entry,
 				     const struct hashmap_entry *entry2,
-- 
2.51.0.450.g87641ccf93.dirty
Justin Tobler· Sep 17, 2025, 21:26 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v5 01/15] packfile: introduce a new `struct packfile_store`

On 25/09/15 10:54AM, Patrick Steinhardt wrote:
> Information about a object database's packfiles is currently distributed
s/a object/an object/
Show 33 quoted lines
> across two different structures:
> 
>   - `struct packed_git` contains the `next` pointer as well as the
>     `mru_head`, both of which serve to store the list of packfiles.
> 
>   - `struct object_database` contains several fields that relate to the
>     packfiles.
> 
> So we don't really have a central data structure that tracks our
> packfiles, and consequently responsibilities aren't always clear cut.
> A consequence for the upcoming pluggable object databases is that this
> makes it very hard to move management of packfiles from the object
> database level down into the object database source.
> 
> Introduce a new `struct packfile_store` which is about to become the
> single source of truth for managing packfiles. Right now this data
> structure doesn't yet contain anything, but in subsequent patches we
> will move all data structures that relate to packfiles and that are
> currently contained in `struct object_database` into this new home.
> 
> Note that this is only a first step: most importantly, we won't (yet)
> move the `struct packed_git::next` pointer around. This will happen in a
> subsequent patch series though so that `struct packed_git` will really
> only host information about the specific packfile it represents.
> 
> Further note that the new structure still sits at the wrong level at the
> end of this patch series: as mentioned, it should eventually sit at the
> level of the object database source, not at the object database level.
> But introducing the packfile store now already makes it way easier to
> eventually push down the now-selfcontained data structure by one level.
> 
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
[snip]
Show 5 quoted lines
> @@ -91,6 +91,7 @@ struct odb_source {
>  };
>  
>  struct packed_git;
> +struct packfile_store;

I find it a bit suprising that we already add a `struct packfile_store` to `struct odb_source` since it doesn't get wired up yet. I suppose it's just acting as a placeholder for what's to come though.

>  struct cached_object_entry;
-Justin
Patrick Steinhardt· Sep 23, 2025, 09:34 UTC · re: Justin Tobler · lore

Re: [PATCH v5 01/15] packfile: introduce a new `struct packfile_store`

On Wed, Sep 17, 2025 at 04:26:52PM -0500, Justin Tobler wrote:
Show 10 quoted lines
> On 25/09/15 10:54AM, Patrick Steinhardt wrote:
> > @@ -91,6 +91,7 @@ struct odb_source {
> >  };
> >  
> >  struct packed_git;
> > +struct packfile_store;
> 
> I find it a bit suprising that we already add a `struct packfile_store`
> to `struct odb_source` since it doesn't get wired up yet. I suppose it's
> just acting as a placeholder for what's to come though.

You're probably misreading this: we don't add it to `struct odb_source`. This is only a forward declaration so that we can include the struct in `struct object_database`, which happens in this same commit.

Patrick
Justin Tobler· Sep 24, 2025, 21:56 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v5 01/15] packfile: introduce a new `struct packfile_store`

On 25/09/23 11:34AM, Patrick Steinhardt wrote:
Show 15 quoted lines
> On Wed, Sep 17, 2025 at 04:26:52PM -0500, Justin Tobler wrote:
> > On 25/09/15 10:54AM, Patrick Steinhardt wrote:
> > > @@ -91,6 +91,7 @@ struct odb_source {
> > >  };
> > >  
> > >  struct packed_git;
> > > +struct packfile_store;
> > 
> > I find it a bit suprising that we already add a `struct packfile_store`
> > to `struct odb_source` since it doesn't get wired up yet. I suppose it's
> > just acting as a placeholder for what's to come though.
> 
> You're probably misreading this: we don't add it to `struct odb_source`.
> This is only a forward declaration so that we can include the struct in
> `struct object_database`, which happens in this same commit.
Ah, I was indeed misreading this. Thanks for the correction.
-Justin
Patrick Steinhardt· Sep 15, 2025, 08:54 UTC · re: Patrick Steinhardt · lore

[PATCH v5 02/15] odb: move list of packfiles into `struct packfile_store`

The object database tracks the list of packfiles it currently knows about. With the introduction of the `struct packfile_store` we have a better place to host this list though.

Move the list accordingly. Extract the logic from `odb_clear()` that knows to close all such packfiles and move it into the new subsystem, as well.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb.c      | 12 ++----------
 odb.h      |  1 -
 packfile.c | 42 +++++++++++++++++++++++++-----------------
 packfile.h | 15 ++++++++++++++-
 4 files changed, 41 insertions(+), 29 deletions(-)
Show changes to 4 files +41 −29

odb.c, odb.h, packfile.c, packfile.h

diff --git a/odb.c b/odb.c
index a2289ea97d..7201d01406 100644
--- a/odb.c
+++ b/odb.c
@@ -1038,16 +1038,8 @@ void odb_clear(struct object_database *o)
 
 	INIT_LIST_HEAD(&o->packed_git_mru);
 	close_object_store(o);
-
-	/*
-	 * `close_object_store()` only closes the packfiles, but doesn't free
-	 * them. We thus have to do this manually.
-	 */
-	for (struct packed_git *p = o->packed_git, *next; p; p = next) {
-		next = p->next;
-		free(p);
-	}
-	o->packed_git = NULL;
+	packfile_store_free(o->packfiles);
+	o->packfiles = NULL;
 
 	hashmap_clear(&o->pack_map);
 	string_list_clear(&o->submodule_source_paths, 0);
diff --git a/odb.h b/odb.h
index 33034eaf2f..22a170b434 100644
--- a/odb.h
+++ b/odb.h
@@ -138,7 +138,6 @@ struct object_database {
 	 * should only be accessed directly by packfile.c
 	 */
 	struct packfile_store *packfiles;
-	struct packed_git *packed_git;
 	/* A most-recently-used ordered version of the packed_git list. */
 	struct list_head packed_git_mru;
 
diff --git a/packfile.c b/packfile.c
index 130d3e2507..36bc240107 100644
--- a/packfile.c
+++ b/packfile.c
@@ -278,7 +278,7 @@ static int unuse_one_window(struct packed_git *current)
 
 	if (current)
 		scan_windows(current, &lru_p, &lru_w, &lru_l);
-	for (p = current->repo->objects->packed_git; p; p = p->next)
+	for (p = current->repo->objects->packfiles->packs; p; p = p->next)
 		scan_windows(p, &lru_p, &lru_w, &lru_l);
 	if (lru_p) {
 		munmap(lru_w->base, lru_w->len);
@@ -362,13 +362,8 @@ void close_pack(struct packed_git *p)
 void close_object_store(struct object_database *o)
 {
 	struct odb_source *source;
-	struct packed_git *p;
 
-	for (p = o->packed_git; p; p = p->next)
-		if (p->do_not_close)
-			BUG("want to close pack marked 'do-not-close'");
-		else
-			close_pack(p);
+	packfile_store_close(o->packfiles);
 
 	for (source = o->sources; source; source = source->next) {
 		if (source->midx)
@@ -468,7 +463,7 @@ static int close_one_pack(struct repository *r)
 	struct pack_window *mru_w = NULL;
 	int accept_windows_inuse = 1;
 
-	for (p = r->objects->packed_git; p; p = p->next) {
+	for (p = r->objects->packfiles->packs; p; p = p->next) {
 		if (p->pack_fd == -1)
 			continue;
 		find_lru_pack(p, &lru_p, &mru_w, &accept_windows_inuse);
@@ -789,8 +784,8 @@ void install_packed_git(struct repository *r, struct packed_git *pack)
 	if (pack->pack_fd != -1)
 		pack_open_fds++;
 
-	pack->next = r->objects->packed_git;
-	r->objects->packed_git = pack;
+	pack->next = r->objects->packfiles->packs;
+	r->objects->packfiles->packs = pack;
 
 	hashmap_entry_init(&pack->packmap_ent, strhash(pack->pack_name));
 	hashmap_add(&r->objects->pack_map, &pack->packmap_ent);
@@ -974,7 +969,7 @@ unsigned long repo_approximate_object_count(struct repository *r)
 				count += m->num_objects;
 		}
 
-		for (p = r->objects->packed_git; p; p = p->next) {
+		for (p = r->objects->packfiles->packs; p; p = p->next) {
 			if (open_pack_index(p))
 				continue;
 			count += p->num_objects;
@@ -1015,7 +1010,7 @@ static int sort_pack(const struct packed_git *a, const struct packed_git *b)
 
 static void rearrange_packed_git(struct repository *r)
 {
-	sort_packs(&r->objects->packed_git, sort_pack);
+	sort_packs(&r->objects->packfiles->packs, sort_pack);
 }
 
 static void prepare_packed_git_mru(struct repository *r)
@@ -1024,7 +1019,7 @@ static void prepare_packed_git_mru(struct repository *r)
 
 	INIT_LIST_HEAD(&r->objects->packed_git_mru);
 
-	for (p = r->objects->packed_git; p; p = p->next)
+	for (p = r->objects->packfiles->packs; p; p = p->next)
 		list_add_tail(&p->mru, &r->objects->packed_git_mru);
 }
 
@@ -1073,7 +1068,7 @@ void reprepare_packed_git(struct repository *r)
 struct packed_git *get_packed_git(struct repository *r)
 {
 	prepare_packed_git(r);
-	return r->objects->packed_git;
+	return r->objects->packfiles->packs;
 }
 
 struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
@@ -1094,7 +1089,7 @@ struct packed_git *get_all_packs(struct repository *r)
 			prepare_midx_pack(m, i);
 	}
 
-	return r->objects->packed_git;
+	return r->objects->packfiles->packs;
 }
 
 struct list_head *get_packed_git_mru(struct repository *r)
@@ -1219,7 +1214,7 @@ const struct packed_git *has_packed_and_bad(struct repository *r,
 {
 	struct packed_git *p;
 
-	for (p = r->objects->packed_git; p; p = p->next)
+	for (p = r->objects->packfiles->packs; p; p = p->next)
 		if (oidset_contains(&p->bad_objects, oid))
 			return p;
 	return NULL;
@@ -2080,7 +2075,7 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 		if (source->midx && fill_midx_entry(source->midx, oid, e))
 			return 1;
 
-	if (!r->objects->packed_git)
+	if (!r->objects->packfiles->packs)
 		return 0;
 
 	list_for_each(pos, &r->objects->packed_git_mru) {
@@ -2343,5 +2338,18 @@ struct packfile_store *packfile_store_new(struct object_database *odb)
 
 void packfile_store_free(struct packfile_store *store)
 {
+	for (struct packed_git *p = store->packs, *next; p; p = next) {
+		next = p->next;
+		free(p);
+	}
 	free(store);
 }
+
+void packfile_store_close(struct packfile_store *store)
+{
+	for (struct packed_git *p = store->packs; p; p = p->next) {
+		if (p->do_not_close)
+			BUG("want to close pack marked 'do-not-close'");
+		close_pack(p);
+	}
+}
diff --git a/packfile.h b/packfile.h
index 8d31fd619a..d7ac8d24b4 100644
--- a/packfile.h
+++ b/packfile.h
@@ -57,6 +57,12 @@ struct packed_git {
  */
 struct packfile_store {
 	struct object_database *odb;
+
+	/*
+	 * The list of packfiles in the order in which they are being added to
+	 * the store.
+	 */
+	struct packed_git *packs;
 };
 
 /*
@@ -66,10 +72,17 @@ struct packfile_store {
 struct packfile_store *packfile_store_new(struct object_database *odb);
 
 /*
- * Free the packfile store and all its associated state.
+ * Free the packfile store and all its associated state. All packfiles
+ * tracked by the store will be closed.
  */
 void packfile_store_free(struct packfile_store *store);
 
+/*
+ * Close all packfiles associated with this store. The packfiles won't be
+ * free'd, so they can be re-opened at a later point in time.
+ */
+void packfile_store_close(struct packfile_store *store);
+
 static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
 				     const struct hashmap_entry *entry,
 				     const struct hashmap_entry *entry2,
-- 
2.51.0.450.g87641ccf93.dirty
Patrick Steinhardt· Sep 15, 2025, 08:54 UTC · re: Patrick Steinhardt · lore

[PATCH v5 03/15] odb: move initialization bit into `struct packfile_store`

The object database knows to skip re-initializing the list of packfiles in case it's already been initialized. Whether or not that is the case is tracked via a separate `initialized` bit that is stored in the object database. With the introduction of the `struct packfile_store` we have a better place to host this bit though.

Move it accordingly. While at it, convert the field into a boolean now that we're allowed to use them in our code base.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb.h      | 6 ------
 packfile.c | 6 +++---
 packfile.h | 6 ++++++
 3 files changed, 9 insertions(+), 9 deletions(-)
Show changes to 3 files +9 −9

odb.h, packfile.c, packfile.h

diff --git a/odb.h b/odb.h
index 22a170b434..bf1b4d4677 100644
--- a/odb.h
+++ b/odb.h
@@ -169,12 +169,6 @@ struct object_database {
 	unsigned long approximate_object_count;
 	unsigned approximate_object_count_valid : 1;
 
-	/*
-	 * Whether packed_git has already been populated with this repository's
-	 * packs.
-	 */
-	unsigned packed_git_initialized : 1;
-
 	/*
 	 * Submodule source paths that will be added as additional sources to
 	 * allow lookup of submodule objects via the main object database.
diff --git a/packfile.c b/packfile.c
index 36bc240107..f37557eac5 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1027,7 +1027,7 @@ static void prepare_packed_git(struct repository *r)
 {
 	struct odb_source *source;
 
-	if (r->objects->packed_git_initialized)
+	if (r->objects->packfiles->initialized)
 		return;
 
 	odb_prepare_alternates(r->objects);
@@ -1038,7 +1038,7 @@ static void prepare_packed_git(struct repository *r)
 	rearrange_packed_git(r);
 
 	prepare_packed_git_mru(r);
-	r->objects->packed_git_initialized = 1;
+	r->objects->packfiles->initialized = true;
 }
 
 void reprepare_packed_git(struct repository *r)
@@ -1060,7 +1060,7 @@ void reprepare_packed_git(struct repository *r)
 		odb_clear_loose_cache(source);
 
 	r->objects->approximate_object_count_valid = 0;
-	r->objects->packed_git_initialized = 0;
+	r->objects->packfiles->initialized = false;
 	prepare_packed_git(r);
 	obj_read_unlock();
 }
diff --git a/packfile.h b/packfile.h
index d7ac8d24b4..cf81091175 100644
--- a/packfile.h
+++ b/packfile.h
@@ -63,6 +63,12 @@ struct packfile_store {
 	 * the store.
 	 */
 	struct packed_git *packs;
+
+	/*
+	 * Whether packfiles have already been populated with this store's
+	 * packs.
+	 */
+	bool initialized;
 };
 
 /*
-- 
2.51.0.450.g87641ccf93.dirty
Patrick Steinhardt· Sep 15, 2025, 08:54 UTC · re: Patrick Steinhardt · lore

[PATCH v5 04/15] odb: move packfile map into `struct packfile_store`

The object database tracks a map of packfiles by their respective paths, which is used to figure out whether a given packfile has already been loaded. With the introduction of the `struct packfile_store` we have a better place to host this list though.

Move the map accordingly.

`pack_map_entry_cmp()` isn't used anywhere but in "packfile.c" anymore after this change, so we convert it to a static function, as well. Note that we also drop the `inline` hint: the function is used as a callback function exclusively, and callbacks cannot be inlined.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 midx.c     |  2 +-
 odb.c      |  2 --
 odb.h      |  6 ------
 packfile.c | 20 ++++++++++++++++++--
 packfile.h | 20 ++++++--------------
 5 files changed, 25 insertions(+), 25 deletions(-)
Show changes to 5 files +25 −25

midx.c, odb.c, odb.h, packfile.c, packfile.h

diff --git a/midx.c b/midx.c
index 7726c13d7e..e96970efbf 100644
--- a/midx.c
+++ b/midx.c
@@ -460,7 +460,7 @@ int prepare_midx_pack(struct multi_pack_index *m,
 	strbuf_addbuf(&key, &pack_name);
 	strbuf_strip_suffix(&key, ".idx");
 	strbuf_addstr(&key, ".pack");
-	p = hashmap_get_entry_from_hash(&r->objects->pack_map,
+	p = hashmap_get_entry_from_hash(&r->objects->packfiles->map,
 					strhash(key.buf), key.buf,
 					struct packed_git, packmap_ent);
 	if (!p) {
diff --git a/odb.c b/odb.c
index 7201d01406..737d98c911 100644
--- a/odb.c
+++ b/odb.c
@@ -998,7 +998,6 @@ struct object_database *odb_new(struct repository *repo)
 	o->repo = repo;
 	o->packfiles = packfile_store_new(o);
 	INIT_LIST_HEAD(&o->packed_git_mru);
-	hashmap_init(&o->pack_map, pack_map_entry_cmp, NULL, 0);
 	pthread_mutex_init(&o->replace_mutex, NULL);
 	string_list_init_dup(&o->submodule_source_paths);
 	return o;
@@ -1041,6 +1040,5 @@ void odb_clear(struct object_database *o)
 	packfile_store_free(o->packfiles);
 	o->packfiles = NULL;
 
-	hashmap_clear(&o->pack_map);
 	string_list_clear(&o->submodule_source_paths, 0);
 }
diff --git a/odb.h b/odb.h
index bf1b4d4677..73a669b993 100644
--- a/odb.h
+++ b/odb.h
@@ -155,12 +155,6 @@ struct object_database {
 	struct cached_object_entry *cached_objects;
 	size_t cached_object_nr, cached_object_alloc;
 
-	/*
-	 * A map of packfiles to packed_git structs for tracking which
-	 * packs have been loaded already.
-	 */
-	struct hashmap pack_map;
-
 	/*
 	 * A fast, rough count of the number of objects in the repository.
 	 * These two fields are not meant for direct access. Use
diff --git a/packfile.c b/packfile.c
index f37557eac5..17e0b8ab27 100644
--- a/packfile.c
+++ b/packfile.c
@@ -788,7 +788,7 @@ void install_packed_git(struct repository *r, struct packed_git *pack)
 	r->objects->packfiles->packs = pack;
 
 	hashmap_entry_init(&pack->packmap_ent, strhash(pack->pack_name));
-	hashmap_add(&r->objects->pack_map, &pack->packmap_ent);
+	hashmap_add(&r->objects->packfiles->map, &pack->packmap_ent);
 }
 
 void (*report_garbage)(unsigned seen_bits, const char *path);
@@ -901,7 +901,7 @@ static void prepare_pack(const char *full_name, size_t full_name_len,
 		hashmap_entry_init(&hent, hash);
 
 		/* Don't reopen a pack we already have. */
-		if (!hashmap_get(&data->r->objects->pack_map, &hent, pack_name)) {
+		if (!hashmap_get(&data->r->objects->packfiles->map, &hent, pack_name)) {
 			p = add_packed_git(data->r, full_name, full_name_len, data->local);
 			if (p)
 				install_packed_git(data->r, p);
@@ -2328,11 +2328,26 @@ int parse_pack_header_option(const char *in, unsigned char *out, unsigned int *l
 	return 0;
 }
 
+static int pack_map_entry_cmp(const void *cmp_data UNUSED,
+			      const struct hashmap_entry *entry,
+			      const struct hashmap_entry *entry2,
+			      const void *keydata)
+{
+	const char *key = keydata;
+	const struct packed_git *pg1, *pg2;
+
+	pg1 = container_of(entry, const struct packed_git, packmap_ent);
+	pg2 = container_of(entry2, const struct packed_git, packmap_ent);
+
+	return strcmp(pg1->pack_name, key ? key : pg2->pack_name);
+}
+
 struct packfile_store *packfile_store_new(struct object_database *odb)
 {
 	struct packfile_store *store;
 	CALLOC_ARRAY(store, 1);
 	store->odb = odb;
+	hashmap_init(&store->map, pack_map_entry_cmp, NULL, 0);
 	return store;
 }
 
@@ -2342,6 +2357,7 @@ void packfile_store_free(struct packfile_store *store)
 		next = p->next;
 		free(p);
 	}
+	hashmap_clear(&store->map);
 	free(store);
 }
 
diff --git a/packfile.h b/packfile.h
index cf81091175..9bbef51164 100644
--- a/packfile.h
+++ b/packfile.h
@@ -64,6 +64,12 @@ struct packfile_store {
 	 */
 	struct packed_git *packs;
 
+	/*
+	 * A map of packfile names to packed_git structs for tracking which
+	 * packs have been loaded already.
+	 */
+	struct hashmap map;
+
 	/*
 	 * Whether packfiles have already been populated with this store's
 	 * packs.
@@ -89,20 +95,6 @@ void packfile_store_free(struct packfile_store *store);
  */
 void packfile_store_close(struct packfile_store *store);
 
-static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
-				     const struct hashmap_entry *entry,
-				     const struct hashmap_entry *entry2,
-				     const void *keydata)
-{
-	const char *key = keydata;
-	const struct packed_git *pg1, *pg2;
-
-	pg1 = container_of(entry, const struct packed_git, packmap_ent);
-	pg2 = container_of(entry2, const struct packed_git, packmap_ent);
-
-	return strcmp(pg1->pack_name, key ? key : pg2->pack_name);
-}
-
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
-- 
2.51.0.450.g87641ccf93.dirty
Justin Tobler· Sep 17, 2025, 22:15 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v5 04/15] odb: move packfile map into `struct packfile_store`

On 25/09/15 10:54AM, Patrick Steinhardt wrote:
> The object database tracks a map of packfiles by their respective paths,
> which is used to figure out whether a given packfile has already been
> loaded. With the introduction of the `struct packfile_store` we have a
> better place to host this list though.
Makes sense.
Show 9 quoted lines
> Move the map accordingly.
> 
> `pack_map_entry_cmp()` isn't used anywhere but in "packfile.c" anymore
> after this change, so we convert it to a static function, as well. Note
> that we also drop the `inline` hint: the function is used as a callback
> function exclusively, and callbacks cannot be inlined.
> 
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
[snip]
Show 12 quoted lines
> diff --git a/midx.c b/midx.c
> index 7726c13d7e..e96970efbf 100644
> --- a/midx.c
> +++ b/midx.c
> @@ -460,7 +460,7 @@ int prepare_midx_pack(struct multi_pack_index *m,
>  	strbuf_addbuf(&key, &pack_name);
>  	strbuf_strip_suffix(&key, ".idx");
>  	strbuf_addstr(&key, ".pack");
> -	p = hashmap_get_entry_from_hash(&r->objects->pack_map,
> +	p = hashmap_get_entry_from_hash(&r->objects->packfiles->map,
>  					strhash(key.buf), key.buf,
>  					struct packed_git, packmap_ent);

In `struct object_database`, the comment above the defined `struct packfile_store *packfiles` says it "should only be accessed directly by packfile.c". Now that the packfile map has been moved into `struct packfile_store`, it looks like "midx.c" now reaches into this structure.

Is this something we should consider?
-Justin
Patrick Steinhardt· Sep 23, 2025, 09:35 UTC · re: Justin Tobler · lore

Re: [PATCH v5 04/15] odb: move packfile map into `struct packfile_store`

On Wed, Sep 17, 2025 at 05:15:44PM -0500, Justin Tobler wrote:
Show 20 quoted lines
> On 25/09/15 10:54AM, Patrick Steinhardt wrote:
> > diff --git a/midx.c b/midx.c
> > index 7726c13d7e..e96970efbf 100644
> > --- a/midx.c
> > +++ b/midx.c
> > @@ -460,7 +460,7 @@ int prepare_midx_pack(struct multi_pack_index *m,
> >  	strbuf_addbuf(&key, &pack_name);
> >  	strbuf_strip_suffix(&key, ".idx");
> >  	strbuf_addstr(&key, ".pack");
> > -	p = hashmap_get_entry_from_hash(&r->objects->pack_map,
> > +	p = hashmap_get_entry_from_hash(&r->objects->packfiles->map,
> >  					strhash(key.buf), key.buf,
> >  					struct packed_git, packmap_ent);
> 
> In `struct object_database`, the comment above the defined `struct
> packfile_store *packfiles` says it "should only be accessed directly by
> packfile.c". Now that the packfile map has been moved into `struct
> packfile_store`, it looks like "midx.c" now reaches into this structure.
> 
> Is this something we should consider?

Hm, probably I guess. I think overall the comment doesn't add that much value anymore now that things are a bit more self-contained, but let's keep it for now and update it accordingly.

Eventually, it will be dropped anyway as we move the data structures into the object source backend, as it becomes obvious that this is private data at that point.

Patrick
Patrick Steinhardt· Sep 15, 2025, 08:54 UTC · re: Patrick Steinhardt · lore

[PATCH v5 05/15] odb: move MRU list of packfiles into `struct packfile_store`

The object database tracks the list of packfiles in most-recently-used order, which is mostly used to favor reading from packfiles that contain most of the objects that we're currently accessing. With the introduction of the `struct packfile_store` we have a better place to host this list though.

Move the list accordingly.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 midx.c     |  2 +-
 odb.c      |  2 --
 odb.h      |  4 ----
 packfile.c | 11 ++++++-----
 packfile.h |  3 +++
 5 files changed, 10 insertions(+), 12 deletions(-)
Show changes to 5 files +10 −12

midx.c, odb.c, odb.h, packfile.c, packfile.h

diff --git a/midx.c b/midx.c
index e96970efbf..91c7b3917d 100644
--- a/midx.c
+++ b/midx.c
@@ -468,7 +468,7 @@ int prepare_midx_pack(struct multi_pack_index *m,
 				   m->source->local);
 		if (p) {
 			install_packed_git(r, p);
-			list_add_tail(&p->mru, &r->objects->packed_git_mru);
+			list_add_tail(&p->mru, &r->objects->packfiles->mru);
 		}
 	}
 
diff --git a/odb.c b/odb.c
index 737d98c911..32e982bf0b 100644
--- a/odb.c
+++ b/odb.c
@@ -997,7 +997,6 @@ struct object_database *odb_new(struct repository *repo)
 	memset(o, 0, sizeof(*o));
 	o->repo = repo;
 	o->packfiles = packfile_store_new(o);
-	INIT_LIST_HEAD(&o->packed_git_mru);
 	pthread_mutex_init(&o->replace_mutex, NULL);
 	string_list_init_dup(&o->submodule_source_paths);
 	return o;
@@ -1035,7 +1034,6 @@ void odb_clear(struct object_database *o)
 		free((char *) o->cached_objects[i].value.buf);
 	FREE_AND_NULL(o->cached_objects);
 
-	INIT_LIST_HEAD(&o->packed_git_mru);
 	close_object_store(o);
 	packfile_store_free(o->packfiles);
 	o->packfiles = NULL;
diff --git a/odb.h b/odb.h
index 73a669b993..8ee1f8bb43 100644
--- a/odb.h
+++ b/odb.h
@@ -3,7 +3,6 @@
 
 #include "hashmap.h"
 #include "object.h"
-#include "list.h"
 #include "oidset.h"
 #include "oidmap.h"
 #include "string-list.h"
@@ -138,9 +137,6 @@ struct object_database {
 	 * should only be accessed directly by packfile.c
 	 */
 	struct packfile_store *packfiles;
-	/* A most-recently-used ordered version of the packed_git list. */
-	struct list_head packed_git_mru;
-
 	struct {
 		struct packed_git **packs;
 		unsigned flags;
diff --git a/packfile.c b/packfile.c
index 17e0b8ab27..861d7ffd6f 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1017,10 +1017,10 @@ static void prepare_packed_git_mru(struct repository *r)
 {
 	struct packed_git *p;
 
-	INIT_LIST_HEAD(&r->objects->packed_git_mru);
+	INIT_LIST_HEAD(&r->objects->packfiles->mru);
 
 	for (p = r->objects->packfiles->packs; p; p = p->next)
-		list_add_tail(&p->mru, &r->objects->packed_git_mru);
+		list_add_tail(&p->mru, &r->objects->packfiles->mru);
 }
 
 static void prepare_packed_git(struct repository *r)
@@ -1095,7 +1095,7 @@ struct packed_git *get_all_packs(struct repository *r)
 struct list_head *get_packed_git_mru(struct repository *r)
 {
 	prepare_packed_git(r);
-	return &r->objects->packed_git_mru;
+	return &r->objects->packfiles->mru;
 }
 
 unsigned long unpack_object_header_buffer(const unsigned char *buf,
@@ -2078,10 +2078,10 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 	if (!r->objects->packfiles->packs)
 		return 0;
 
-	list_for_each(pos, &r->objects->packed_git_mru) {
+	list_for_each(pos, &r->objects->packfiles->mru) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		if (!p->multi_pack_index && fill_pack_entry(oid, e, p)) {
-			list_move(&p->mru, &r->objects->packed_git_mru);
+			list_move(&p->mru, &r->objects->packfiles->mru);
 			return 1;
 		}
 	}
@@ -2347,6 +2347,7 @@ struct packfile_store *packfile_store_new(struct object_database *odb)
 	struct packfile_store *store;
 	CALLOC_ARRAY(store, 1);
 	store->odb = odb;
+	INIT_LIST_HEAD(&store->mru);
 	hashmap_init(&store->map, pack_map_entry_cmp, NULL, 0);
 	return store;
 }
diff --git a/packfile.h b/packfile.h
index 9bbef51164..d48d46cc1b 100644
--- a/packfile.h
+++ b/packfile.h
@@ -64,6 +64,9 @@ struct packfile_store {
 	 */
 	struct packed_git *packs;
 
+	/* A most-recently-used ordered version of the packs list. */
+	struct list_head mru;
+
 	/*
 	 * A map of packfile names to packed_git structs for tracking which
 	 * packs have been loaded already.
-- 
2.51.0.450.g87641ccf93.dirty
Justin Tobler· Sep 17, 2025, 21:59 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v5 05/15] odb: move MRU list of packfiles into `struct packfile_store`

On 25/09/15 10:54AM, Patrick Steinhardt wrote:
Show 10 quoted lines
> The object database tracks the list of packfiles in most-recently-used
> order, which is mostly used to favor reading from packfiles that contain
> most of the objects that we're currently accessing. With the
> introduction of the `struct packfile_store` we have a better place to
> host this list though.
> 
> Move the list accordingly.
> 
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
[snip]
Show 5 quoted lines
> @@ -1035,7 +1034,6 @@ void odb_clear(struct object_database *o)
>  		free((char *) o->cached_objects[i].value.buf);
>  	FREE_AND_NULL(o->cached_objects);
>  
> -	INIT_LIST_HEAD(&o->packed_git_mru);

Now that the mru list lives in `struct packfile_store`, it cleanup automatically gets handled via packfile_store_free(). Makes sense.

>  	close_object_store(o);
>  	packfile_store_free(o->packfiles);
>  	o->packfiles = NULL;
I quite like all of the reorganization in this series so far. :)
-Justin
Patrick Steinhardt· Sep 15, 2025, 08:54 UTC · re: Patrick Steinhardt · lore

[PATCH v5 06/15] odb: move kept cache into `struct packfile_store`

The object database tracks a cache of "kept" packfiles, which is used by git-pack-objects(1) to handle cruft objects. With the introduction of the `struct packfile_store` we have a better place to host this cache though.

Move the cache accordingly.

This moves the last bit of packfile-related state from the object database into the packfile store. Adapt the comment for the `packfiles` pointer in `struct object_database` to reflect this.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb.h      |  8 +-------
 packfile.c | 16 ++++++++--------
 packfile.h | 14 ++++++++++++++
 3 files changed, 23 insertions(+), 15 deletions(-)
Show changes to 3 files +23 −15

odb.h, packfile.c, packfile.h

diff --git a/odb.h b/odb.h
index 8ee1f8bb43..1c998a2478 100644
--- a/odb.h
+++ b/odb.h
@@ -132,15 +132,9 @@ struct object_database {
 	unsigned commit_graph_attempted : 1; /* if loading has been attempted */
 
 	/*
-	 * private data
-	 *
-	 * should only be accessed directly by packfile.c
+	 * Should only be accessed directly by packfile.c
 	 */
 	struct packfile_store *packfiles;
-	struct {
-		struct packed_git **packs;
-		unsigned flags;
-	} kept_pack_cache;
 
 	/*
 	 * This is meant to hold a *small* number of objects that you would
diff --git a/packfile.c b/packfile.c
index 861d7ffd6f..95a78f267f 100644
--- a/packfile.c
+++ b/packfile.c
@@ -2091,19 +2091,19 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 static void maybe_invalidate_kept_pack_cache(struct repository *r,
 					     unsigned flags)
 {
-	if (!r->objects->kept_pack_cache.packs)
+	if (!r->objects->packfiles->kept_cache.packs)
 		return;
-	if (r->objects->kept_pack_cache.flags == flags)
+	if (r->objects->packfiles->kept_cache.flags == flags)
 		return;
-	FREE_AND_NULL(r->objects->kept_pack_cache.packs);
-	r->objects->kept_pack_cache.flags = 0;
+	FREE_AND_NULL(r->objects->packfiles->kept_cache.packs);
+	r->objects->packfiles->kept_cache.flags = 0;
 }
 
 struct packed_git **kept_pack_cache(struct repository *r, unsigned flags)
 {
 	maybe_invalidate_kept_pack_cache(r, flags);
 
-	if (!r->objects->kept_pack_cache.packs) {
+	if (!r->objects->packfiles->kept_cache.packs) {
 		struct packed_git **packs = NULL;
 		size_t nr = 0, alloc = 0;
 		struct packed_git *p;
@@ -2126,11 +2126,11 @@ struct packed_git **kept_pack_cache(struct repository *r, unsigned flags)
 		ALLOC_GROW(packs, nr + 1, alloc);
 		packs[nr] = NULL;
 
-		r->objects->kept_pack_cache.packs = packs;
-		r->objects->kept_pack_cache.flags = flags;
+		r->objects->packfiles->kept_cache.packs = packs;
+		r->objects->packfiles->kept_cache.flags = flags;
 	}
 
-	return r->objects->kept_pack_cache.packs;
+	return r->objects->packfiles->kept_cache.packs;
 }
 
 int find_kept_pack_entry(struct repository *r,
diff --git a/packfile.h b/packfile.h
index d48d46cc1b..bf66211986 100644
--- a/packfile.h
+++ b/packfile.h
@@ -64,6 +64,20 @@ struct packfile_store {
 	 */
 	struct packed_git *packs;
 
+	/*
+	 * Cache of packfiles which are marked as "kept", either because there
+	 * is an on-disk ".keep" file or because they are marked as "kept" in
+	 * memory.
+	 *
+	 * Should not be accessed directly, but via `kept_pack_cache()`. The
+	 * list of packs gets invalidated when the stored flags and the flags
+	 * passed to `kept_pack_cache()` mismatch.
+	 */
+	struct {
+		struct packed_git **packs;
+		unsigned flags;
+	} kept_cache;
+
 	/* A most-recently-used ordered version of the packs list. */
 	struct list_head mru;
 
-- 
2.51.0.450.g87641ccf93.dirty
Patrick Steinhardt· Sep 15, 2025, 08:54 UTC · re: Patrick Steinhardt · lore

[PATCH v5 07/15] packfile: reorder functions to avoid function declaration

Reorder functions so that we can avoid a forward declaration of `prepare_packed_git()`.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 packfile.c | 67 +++++++++++++++++++++++++++++++-------------------------------
 1 file changed, 33 insertions(+), 34 deletions(-)
Show changes to packfile.c +33 −34
diff --git a/packfile.c b/packfile.c
index 95a78f267f..5588a7ad6d 100644
--- a/packfile.c
+++ b/packfile.c
@@ -946,40 +946,6 @@ static void prepare_packed_git_one(struct odb_source *source)
 	string_list_clear(data.garbage, 0);
 }
 
-static void prepare_packed_git(struct repository *r);
-/*
- * Give a fast, rough count of the number of objects in the repository. This
- * ignores loose objects completely. If you have a lot of them, then either
- * you should repack because your performance will be awful, or they are
- * all unreachable objects about to be pruned, in which case they're not really
- * interesting as a measure of repo size in the first place.
- */
-unsigned long repo_approximate_object_count(struct repository *r)
-{
-	if (!r->objects->approximate_object_count_valid) {
-		struct odb_source *source;
-		unsigned long count = 0;
-		struct packed_git *p;
-
-		prepare_packed_git(r);
-
-		for (source = r->objects->sources; source; source = source->next) {
-			struct multi_pack_index *m = get_multi_pack_index(source);
-			if (m)
-				count += m->num_objects;
-		}
-
-		for (p = r->objects->packfiles->packs; p; p = p->next) {
-			if (open_pack_index(p))
-				continue;
-			count += p->num_objects;
-		}
-		r->objects->approximate_object_count = count;
-		r->objects->approximate_object_count_valid = 1;
-	}
-	return r->objects->approximate_object_count;
-}
-
 DEFINE_LIST_SORT(static, sort_packs, struct packed_git, next);
 
 static int sort_pack(const struct packed_git *a, const struct packed_git *b)
@@ -1098,6 +1064,39 @@ struct list_head *get_packed_git_mru(struct repository *r)
 	return &r->objects->packfiles->mru;
 }
 
+/*
+ * Give a fast, rough count of the number of objects in the repository. This
+ * ignores loose objects completely. If you have a lot of them, then either
+ * you should repack because your performance will be awful, or they are
+ * all unreachable objects about to be pruned, in which case they're not really
+ * interesting as a measure of repo size in the first place.
+ */
+unsigned long repo_approximate_object_count(struct repository *r)
+{
+	if (!r->objects->approximate_object_count_valid) {
+		struct odb_source *source;
+		unsigned long count = 0;
+		struct packed_git *p;
+
+		prepare_packed_git(r);
+
+		for (source = r->objects->sources; source; source = source->next) {
+			struct multi_pack_index *m = get_multi_pack_index(source);
+			if (m)
+				count += m->num_objects;
+		}
+
+		for (p = r->objects->packfiles->packs; p; p = p->next) {
+			if (open_pack_index(p))
+				continue;
+			count += p->num_objects;
+		}
+		r->objects->approximate_object_count = count;
+		r->objects->approximate_object_count_valid = 1;
+	}
+	return r->objects->approximate_object_count;
+}
+
 unsigned long unpack_object_header_buffer(const unsigned char *buf,
 		unsigned long len, enum object_type *type, unsigned long *sizep)
 {
-- 
2.51.0.450.g87641ccf93.dirty
Patrick Steinhardt· Sep 15, 2025, 08:54 UTC · re: Patrick Steinhardt · lore

[PATCH v5 08/15] packfile: refactor `prepare_packed_git()` to work on packfile store

The `prepare_packed_git()` function and its friends are responsible for loading packfiles as well as the multi-pack index for a given object database. Refactor these functions to accept a packfile store instead of a repository to clarify their scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 packfile.c | 41 ++++++++++++++++++-----------------------
 1 file changed, 18 insertions(+), 23 deletions(-)
Show changes to packfile.c +18 −23
diff --git a/packfile.c b/packfile.c
index 5588a7ad6d..095c85919b 100644
--- a/packfile.c
+++ b/packfile.c
@@ -974,37 +974,32 @@ static int sort_pack(const struct packed_git *a, const struct packed_git *b)
 	return -1;
 }
 
-static void rearrange_packed_git(struct repository *r)
-{
-	sort_packs(&r->objects->packfiles->packs, sort_pack);
-}
-
-static void prepare_packed_git_mru(struct repository *r)
+static void packfile_store_prepare_mru(struct packfile_store *store)
 {
 	struct packed_git *p;
 
-	INIT_LIST_HEAD(&r->objects->packfiles->mru);
+	INIT_LIST_HEAD(&store->mru);
 
-	for (p = r->objects->packfiles->packs; p; p = p->next)
-		list_add_tail(&p->mru, &r->objects->packfiles->mru);
+	for (p = store->packs; p; p = p->next)
+		list_add_tail(&p->mru, &store->mru);
 }
 
-static void prepare_packed_git(struct repository *r)
+static void packfile_store_prepare(struct packfile_store *store)
 {
 	struct odb_source *source;
 
-	if (r->objects->packfiles->initialized)
+	if (store->initialized)
 		return;
 
-	odb_prepare_alternates(r->objects);
-	for (source = r->objects->sources; source; source = source->next) {
+	odb_prepare_alternates(store->odb);
+	for (source = store->odb->sources; source; source = source->next) {
 		prepare_multi_pack_index_one(source);
 		prepare_packed_git_one(source);
 	}
-	rearrange_packed_git(r);
+	sort_packs(&store->packs, sort_pack);
 
-	prepare_packed_git_mru(r);
-	r->objects->packfiles->initialized = true;
+	packfile_store_prepare_mru(store);
+	store->initialized = true;
 }
 
 void reprepare_packed_git(struct repository *r)
@@ -1027,25 +1022,25 @@ void reprepare_packed_git(struct repository *r)
 
 	r->objects->approximate_object_count_valid = 0;
 	r->objects->packfiles->initialized = false;
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 	obj_read_unlock();
 }
 
 struct packed_git *get_packed_git(struct repository *r)
 {
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 	return r->objects->packfiles->packs;
 }
 
 struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
 {
-	prepare_packed_git(source->odb->repo);
+	packfile_store_prepare(source->odb->packfiles);
 	return source->midx;
 }
 
 struct packed_git *get_all_packs(struct repository *r)
 {
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 
 	for (struct odb_source *source = r->objects->sources; source; source = source->next) {
 		struct multi_pack_index *m = source->midx;
@@ -1060,7 +1055,7 @@ struct packed_git *get_all_packs(struct repository *r)
 
 struct list_head *get_packed_git_mru(struct repository *r)
 {
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 	return &r->objects->packfiles->mru;
 }
 
@@ -1078,7 +1073,7 @@ unsigned long repo_approximate_object_count(struct repository *r)
 		unsigned long count = 0;
 		struct packed_git *p;
 
-		prepare_packed_git(r);
+		packfile_store_prepare(r->objects->packfiles);
 
 		for (source = r->objects->sources; source; source = source->next) {
 			struct multi_pack_index *m = get_multi_pack_index(source);
@@ -2068,7 +2063,7 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 {
 	struct list_head *pos;
 
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 
 	for (struct odb_source *source = r->objects->sources; source; source = source->next)
 		if (source->midx && fill_midx_entry(source->midx, oid, e))
-- 
2.51.0.450.g87641ccf93.dirty
Patrick Steinhardt· Sep 15, 2025, 08:54 UTC · re: Patrick Steinhardt · lore

[PATCH v5 09/15] packfile: split up responsibilities of `reprepare_packed_git()`

In `reprepare_packed_git()` we perform a couple of operations:
  - We reload alternate object directories.
  - We clear the loose object cache.
  - We reprepare packfiles.

While the logic is hosted in "packfile.c", it clearly reaches into other subsystems that aren't related to packfiles.

Split up the responsibility and introduce `odb_reprepare()` which now becomes responsible for repreparing the whole object database. The existing `reprepare_packed_git()` function is refactored accordingly and only cares about reloading the packfile store now.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/backfill.c     |  2 +-
 builtin/gc.c           |  4 ++--
 builtin/receive-pack.c |  2 +-
 builtin/repack.c       |  2 +-
 bulk-checkin.c         |  2 +-
 connected.c            |  2 +-
 fetch-pack.c           |  4 ++--
 object-name.c          |  2 +-
 odb.c                  | 27 ++++++++++++++++++++++++++-
 odb.h                  |  6 ++++++
 packfile.c             | 26 ++++----------------------
 packfile.h             |  9 ++++++++-
 transport-helper.c     |  2 +-
 13 files changed, 55 insertions(+), 35 deletions(-)
Show changes to 13 files +55 −35

builtin/backfill.c, builtin/gc.c, builtin/receive-pack.c, builtin/repack.c, bulk-checkin.c, connected.c, fetch-pack.c, object-name.c, odb.c, odb.h, packfile.c, packfile.h, transport-helper.c

diff --git a/builtin/backfill.c b/builtin/backfill.c
index 80056abe47..e80fc1b694 100644
--- a/builtin/backfill.c
+++ b/builtin/backfill.c
@@ -53,7 +53,7 @@ static void download_batch(struct backfill_context *ctx)
 	 * We likely have a new packfile. Add it to the packed list to
 	 * avoid possible duplicate downloads of the same objects.
 	 */
-	reprepare_packed_git(ctx->repo);
+	odb_reprepare(ctx->repo->objects);
 }
 
 static int fill_missing_blobs(const char *path UNUSED,
diff --git a/builtin/gc.c b/builtin/gc.c
index 03ae4926b2..aeca06a08b 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1042,7 +1042,7 @@ int cmd_gc(int argc,
 		die(FAILED_RUN, "rerere");
 
 	report_garbage = report_pack_garbage;
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 	if (pack_garbage.nr > 0) {
 		close_object_store(the_repository->objects);
 		clean_pack_garbage();
@@ -1491,7 +1491,7 @@ static off_t get_auto_pack_size(void)
 	struct packed_git *p;
 	struct repository *r = the_repository;
 
-	reprepare_packed_git(r);
+	odb_reprepare(r->objects);
 	for (p = get_all_packs(r); p; p = p->next) {
 		if (p->pack_size > max_size) {
 			second_largest_size = max_size;
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 1113137a6f..c9288a9c7e 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -2389,7 +2389,7 @@ static const char *unpack(int err_fd, struct shallow_info *si)
 		status = finish_command(&child);
 		if (status)
 			return "index-pack abnormal exit";
-		reprepare_packed_git(the_repository);
+		odb_reprepare(the_repository->objects);
 	}
 	return NULL;
 }
diff --git a/builtin/repack.c b/builtin/repack.c
index c490a51e91..5ff27fc8e2 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -1685,7 +1685,7 @@ int cmd_repack(int argc,
 			goto cleanup;
 	}
 
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 
 	if (delete_redundant) {
 		int opts = 0;
diff --git a/bulk-checkin.c b/bulk-checkin.c
index b2809ab039..f65439a748 100644
--- a/bulk-checkin.c
+++ b/bulk-checkin.c
@@ -90,7 +90,7 @@ static void flush_bulk_checkin_packfile(struct bulk_checkin_packfile *state)
 
 	strbuf_release(&packname);
 	/* Make objects we just wrote available to ourselves */
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 }
 
 /*
diff --git a/connected.c b/connected.c
index 18c13245d8..d6e9682fd9 100644
--- a/connected.c
+++ b/connected.c
@@ -72,7 +72,7 @@ int check_connected(oid_iterate_fn fn, void *cb_data,
 		 * Before checking for promisor packs, be sure we have the
 		 * latest pack-files loaded into memory.
 		 */
-		reprepare_packed_git(the_repository);
+		odb_reprepare(the_repository->objects);
 		do {
 			struct packed_git *p;
 
diff --git a/fetch-pack.c b/fetch-pack.c
index 6ed5662951..fe7a84bf2f 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -1983,7 +1983,7 @@ static void update_shallow(struct fetch_pack_args *args,
 		 * remote is shallow, but this is a clone, there are
 		 * no objects in repo to worry about. Accept any
 		 * shallow points that exist in the pack (iow in repo
-		 * after get_pack() and reprepare_packed_git())
+		 * after get_pack() and odb_reprepare())
 		 */
 		struct oid_array extra = OID_ARRAY_INIT;
 		struct object_id *oid = si->shallow->oid;
@@ -2108,7 +2108,7 @@ struct ref *fetch_pack(struct fetch_pack_args *args,
 		ref_cpy = do_fetch_pack(args, fd, ref, sought, nr_sought,
 					&si, pack_lockfiles);
 	}
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 
 	if (!args->cloning && args->deepen) {
 		struct check_connected_options opt = CHECK_CONNECTED_INIT;
diff --git a/object-name.c b/object-name.c
index 732056ff5e..df9e0c5f02 100644
--- a/object-name.c
+++ b/object-name.c
@@ -596,7 +596,7 @@ static enum get_oid_result get_short_oid(struct repository *r,
 	 * or migrated from loose to packed.
 	 */
 	if (status == MISSING_OBJECT) {
-		reprepare_packed_git(r);
+		odb_reprepare(r->objects);
 		find_short_object_filename(&ds);
 		find_short_packed_object(&ds);
 		status = finish_object_disambiguation(&ds, oid);
diff --git a/odb.c b/odb.c
index 32e982bf0b..65a6cc67b6 100644
--- a/odb.c
+++ b/odb.c
@@ -694,7 +694,7 @@ static int do_oid_object_info_extended(struct object_database *odb,
 
 		/* Not a loose object; someone else may have just packed it. */
 		if (!(flags & OBJECT_INFO_QUICK)) {
-			reprepare_packed_git(odb->repo);
+			odb_reprepare(odb->repo->objects);
 			if (find_pack_entry(odb->repo, real, &e))
 				break;
 		}
@@ -1040,3 +1040,28 @@ void odb_clear(struct object_database *o)
 
 	string_list_clear(&o->submodule_source_paths, 0);
 }
+
+void odb_reprepare(struct object_database *o)
+{
+	struct odb_source *source;
+
+	obj_read_lock();
+
+	/*
+	 * Reprepare alt odbs, in case the alternates file was modified
+	 * during the course of this process. This only _adds_ odbs to
+	 * the linked list, so existing odbs will continue to exist for
+	 * the lifetime of the process.
+	 */
+	o->loaded_alternates = 0;
+	odb_prepare_alternates(o);
+
+	for (source = o->sources; source; source = source->next)
+		odb_clear_loose_cache(source);
+
+	o->approximate_object_count_valid = 0;
+
+	packfile_store_reprepare(o->packfiles);
+
+	obj_read_unlock();
+}
diff --git a/odb.h b/odb.h
index 1c998a2478..ef34132c58 100644
--- a/odb.h
+++ b/odb.h
@@ -163,6 +163,12 @@ struct object_database {
 struct object_database *odb_new(struct repository *repo);
 void odb_clear(struct object_database *o);
 
+/*
+ * Clear caches, reload alternates and then reload object sources so that new
+ * objects may become accessible.
+ */
+void odb_reprepare(struct object_database *o);
+
 /*
  * Find source by its object directory path. Returns a `NULL` pointer in case
  * the source could not be found.
diff --git a/packfile.c b/packfile.c
index 095c85919b..950b98aac5 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1002,28 +1002,10 @@ static void packfile_store_prepare(struct packfile_store *store)
 	store->initialized = true;
 }
 
-void reprepare_packed_git(struct repository *r)
+void packfile_store_reprepare(struct packfile_store *store)
 {
-	struct odb_source *source;
-
-	obj_read_lock();
-
-	/*
-	 * Reprepare alt odbs, in case the alternates file was modified
-	 * during the course of this process. This only _adds_ odbs to
-	 * the linked list, so existing odbs will continue to exist for
-	 * the lifetime of the process.
-	 */
-	r->objects->loaded_alternates = 0;
-	odb_prepare_alternates(r->objects);
-
-	for (source = r->objects->sources; source; source = source->next)
-		odb_clear_loose_cache(source);
-
-	r->objects->approximate_object_count_valid = 0;
-	r->objects->packfiles->initialized = false;
-	packfile_store_prepare(r->objects->packfiles);
-	obj_read_unlock();
+	store->initialized = false;
+	packfile_store_prepare(store);
 }
 
 struct packed_git *get_packed_git(struct repository *r)
@@ -1144,7 +1126,7 @@ unsigned long get_size_from_delta(struct packed_git *p,
 		 *
 		 * Other worrying sections could be the call to close_pack_fd(),
 		 * which can close packs even with in-use windows, and to
-		 * reprepare_packed_git(). Regarding the former, mmap doc says:
+		 * odb_reprepare(). Regarding the former, mmap doc says:
 		 * "closing the file descriptor does not unmap the region". And
 		 * for the latter, it won't re-open already available packs.
 		 */
diff --git a/packfile.h b/packfile.h
index bf66211986..a85ff607fe 100644
--- a/packfile.h
+++ b/packfile.h
@@ -112,6 +112,14 @@ void packfile_store_free(struct packfile_store *store);
  */
 void packfile_store_close(struct packfile_store *store);
 
+/*
+ * Clear the packfile caches and try to look up any new packfiles that have
+ * appeared since last preparing the packfiles store.
+ *
+ * This function must be called under the `odb_read_lock()`.
+ */
+void packfile_store_reprepare(struct packfile_store *store);
+
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
@@ -188,7 +196,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-void reprepare_packed_git(struct repository *r);
 void install_packed_git(struct repository *r, struct packed_git *pack);
 
 struct packed_git *get_packed_git(struct repository *r);
diff --git a/transport-helper.c b/transport-helper.c
index 0789e5bca5..4d95d84f9e 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -450,7 +450,7 @@ static int fetch_with_fetch(struct transport *transport,
 	}
 	strbuf_release(&buf);
 
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 	return 0;
 }
 
-- 
2.51.0.450.g87641ccf93.dirty
Justin Tobler· Sep 17, 2025, 22:32 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v5 09/15] packfile: split up responsibilities of `reprepare_packed_git()`

On 25/09/15 10:54AM, Patrick Steinhardt wrote:
Show 18 quoted lines
> In `reprepare_packed_git()` we perform a couple of operations:
> 
>   - We reload alternate object directories.
> 
>   - We clear the loose object cache.
> 
>   - We reprepare packfiles.
> 
> While the logic is hosted in "packfile.c", it clearly reaches into other
> subsystems that aren't related to packfiles.
> 
> Split up the responsibility and introduce `odb_reprepare()` which now
> becomes responsible for repreparing the whole object database. The
> existing `reprepare_packed_git()` function is refactored accordingly and
> only cares about reloading the packfile store now.
> 
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
[snip]
Show 13 quoted lines
> diff --git a/odb.h b/odb.h
> index 1c998a2478..ef34132c58 100644
> --- a/odb.h
> +++ b/odb.h
> @@ -163,6 +163,12 @@ struct object_database {
>  struct object_database *odb_new(struct repository *repo);
>  void odb_clear(struct object_database *o);
>  
> +/*
> + * Clear caches, reload alternates and then reload object sources so that new
> + * objects may become accessible.
> + */
> +void odb_reprepare(struct object_database *o);

nit: IMO, something like `odb_reload()` sounds a bit better, but it's not a big deal either way.

[snip]
Show 15 quoted lines
> diff --git a/packfile.h b/packfile.h
> index bf66211986..a85ff607fe 100644
> --- a/packfile.h
> +++ b/packfile.h
> @@ -112,6 +112,14 @@ void packfile_store_free(struct packfile_store *store);
>   */
>  void packfile_store_close(struct packfile_store *store);
>  
> +/*
> + * Clear the packfile caches and try to look up any new packfiles that have
> + * appeared since last preparing the packfiles store.
> + *
> + * This function must be called under the `odb_read_lock()`.
> + */
> +void packfile_store_reprepare(struct packfile_store *store);
nit: I also think `packfile_store_reload()` sounds a bit nicer here.
Overall this patch looks good. :)
-Justin
Patrick Steinhardt· Sep 23, 2025, 09:34 UTC · re: Justin Tobler · lore

Re: [PATCH v5 09/15] packfile: split up responsibilities of `reprepare_packed_git()`

On Wed, Sep 17, 2025 at 05:32:43PM -0500, Justin Tobler wrote:
Show 38 quoted lines
> On 25/09/15 10:54AM, Patrick Steinhardt wrote:
> > diff --git a/odb.h b/odb.h
> > index 1c998a2478..ef34132c58 100644
> > --- a/odb.h
> > +++ b/odb.h
> > @@ -163,6 +163,12 @@ struct object_database {
> >  struct object_database *odb_new(struct repository *repo);
> >  void odb_clear(struct object_database *o);
> >  
> > +/*
> > + * Clear caches, reload alternates and then reload object sources so that new
> > + * objects may become accessible.
> > + */
> > +void odb_reprepare(struct object_database *o);
> 
> nit: IMO, something like `odb_reload()` sounds a bit better, but it's
> not a big deal either way.
> 
> [snip]
> > diff --git a/packfile.h b/packfile.h
> > index bf66211986..a85ff607fe 100644
> > --- a/packfile.h
> > +++ b/packfile.h
> > @@ -112,6 +112,14 @@ void packfile_store_free(struct packfile_store *store);
> >   */
> >  void packfile_store_close(struct packfile_store *store);
> >  
> > +/*
> > + * Clear the packfile caches and try to look up any new packfiles that have
> > + * appeared since last preparing the packfiles store.
> > + *
> > + * This function must be called under the `odb_read_lock()`.
> > + */
> > +void packfile_store_reprepare(struct packfile_store *store);
> 
> nit: I also think `packfile_store_reload()` sounds a bit nicer here.
> 
> Overall this patch looks good. :)

Seen in a void I'd agree, but "preparing" and "repreparing" have been terms that we already used for quite a long time in this context. So I'd rather continue using them for now.

Patrick
Patrick Steinhardt· Sep 15, 2025, 08:54 UTC · re: Patrick Steinhardt · lore

[PATCH v5 10/15] packfile: refactor `install_packed_git()` to work on packfile store

The `install_packed_git()` functions adds a packfile to a specific object store. Refactor it to accept a packfile store instead of a repository to clarify its scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/fast-import.c |  2 +-
 builtin/index-pack.c  |  2 +-
 http.c                |  2 +-
 http.h                |  2 +-
 midx.c                |  2 +-
 packfile.c            | 11 ++++++-----
 packfile.h            |  9 +++++++--
 7 files changed, 18 insertions(+), 12 deletions(-)
Show changes to 7 files +18 −12

builtin/fast-import.c, builtin/index-pack.c, http.c, http.h, midx.c, packfile.c, packfile.h

diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index 2c35f9345d..e9d82b31c3 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -901,7 +901,7 @@ static void end_packfile(void)
 		if (!new_p)
 			die("core git rejected index %s", idx_name);
 		all_packs[pack_id] = new_p;
-		install_packed_git(the_repository, new_p);
+		packfile_store_add_pack(the_repository->objects->packfiles, new_p);
 		free(idx_name);
 
 		/* Print the boundary */
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index f91c301bba..ed490dfad4 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1645,7 +1645,7 @@ static void final(const char *final_pack_name, const char *curr_pack_name,
 		p = add_packed_git(the_repository, final_index_name,
 				   strlen(final_index_name), 0);
 		if (p)
-			install_packed_git(the_repository, p);
+			packfile_store_add_pack(the_repository->objects->packfiles, p);
 	}
 
 	if (!from_stdin) {
diff --git a/http.c b/http.c
index 98853d6483..af2120b64c 100644
--- a/http.c
+++ b/http.c
@@ -2541,7 +2541,7 @@ void http_install_packfile(struct packed_git *p,
 		lst = &((*lst)->next);
 	*lst = (*lst)->next;
 
-	install_packed_git(the_repository, p);
+	packfile_store_add_pack(the_repository->objects->packfiles, p);
 }
 
 struct http_pack_request *new_http_pack_request(
diff --git a/http.h b/http.h
index 36202139f4..e5a5380c6c 100644
--- a/http.h
+++ b/http.h
@@ -210,7 +210,7 @@ int finish_http_pack_request(struct http_pack_request *preq);
 void release_http_pack_request(struct http_pack_request *preq);
 
 /*
- * Remove p from the given list, and invoke install_packed_git() on it.
+ * Remove p from the given list, and invoke packfile_store_add_pack() on it.
  *
  * This is a convenience function for users that have obtained a list of packs
  * from http_get_info_packs() and have chosen a specific pack to fetch.
diff --git a/midx.c b/midx.c
index 91c7b3917d..69c44be71c 100644
--- a/midx.c
+++ b/midx.c
@@ -467,7 +467,7 @@ int prepare_midx_pack(struct multi_pack_index *m,
 		p = add_packed_git(r, pack_name.buf, pack_name.len,
 				   m->source->local);
 		if (p) {
-			install_packed_git(r, p);
+			packfile_store_add_pack(r->objects->packfiles, p);
 			list_add_tail(&p->mru, &r->objects->packfiles->mru);
 		}
 	}
diff --git a/packfile.c b/packfile.c
index 950b98aac5..af806aba09 100644
--- a/packfile.c
+++ b/packfile.c
@@ -779,16 +779,17 @@ struct packed_git *add_packed_git(struct repository *r, const char *path,
 	return p;
 }
 
-void install_packed_git(struct repository *r, struct packed_git *pack)
+void packfile_store_add_pack(struct packfile_store *store,
+			     struct packed_git *pack)
 {
 	if (pack->pack_fd != -1)
 		pack_open_fds++;
 
-	pack->next = r->objects->packfiles->packs;
-	r->objects->packfiles->packs = pack;
+	pack->next = store->packs;
+	store->packs = pack;
 
 	hashmap_entry_init(&pack->packmap_ent, strhash(pack->pack_name));
-	hashmap_add(&r->objects->packfiles->map, &pack->packmap_ent);
+	hashmap_add(&store->map, &pack->packmap_ent);
 }
 
 void (*report_garbage)(unsigned seen_bits, const char *path);
@@ -904,7 +905,7 @@ static void prepare_pack(const char *full_name, size_t full_name_len,
 		if (!hashmap_get(&data->r->objects->packfiles->map, &hent, pack_name)) {
 			p = add_packed_git(data->r, full_name, full_name_len, data->local);
 			if (p)
-				install_packed_git(data->r, p);
+				packfile_store_add_pack(data->r->objects->packfiles, p);
 		}
 		free(pack_name);
 	}
diff --git a/packfile.h b/packfile.h
index a85ff607fe..ba4b0cef9c 100644
--- a/packfile.h
+++ b/packfile.h
@@ -120,6 +120,13 @@ void packfile_store_close(struct packfile_store *store);
  */
 void packfile_store_reprepare(struct packfile_store *store);
 
+/*
+ * Add the pack to the store so that contained objects become accessible via
+ * the store. This moves ownership into the store.
+ */
+void packfile_store_add_pack(struct packfile_store *store,
+			     struct packed_git *pack);
+
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
@@ -196,8 +203,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-void install_packed_git(struct repository *r, struct packed_git *pack);
-
 struct packed_git *get_packed_git(struct repository *r);
 struct list_head *get_packed_git_mru(struct repository *r);
 struct multi_pack_index *get_multi_pack_index(struct odb_source *source);
-- 
2.51.0.450.g87641ccf93.dirty
Patrick Steinhardt· Sep 15, 2025, 08:54 UTC · re: Patrick Steinhardt · lore

[PATCH v5 11/15] packfile: introduce function to load and add packfiles

We have a recurring pattern where we essentially perform an upsert of a packfile in case it isn't yet known by the packfile store. The logic to do so is non-trivial as we have to reconstruct the packfile's key, check the map of packfiles, then create the new packfile and finally add it to the store.

Introduce a new function that does this dance for us. Refactor callsites to use it.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/fast-import.c |  4 ++--
 builtin/index-pack.c  | 10 +++-------
 midx.c                | 23 ++++-------------------
 packfile.c            | 44 +++++++++++++++++++++++++++++++-------------
 packfile.h            |  8 ++++++++
 5 files changed, 48 insertions(+), 41 deletions(-)
Show changes to 5 files +48 −41

builtin/fast-import.c, builtin/index-pack.c, midx.c, packfile.c, packfile.h

diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index e9d82b31c3..a26e79689d 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -897,11 +897,11 @@ static void end_packfile(void)
 		idx_name = keep_pack(create_index());
 
 		/* Register the packfile with core git's machinery. */
-		new_p = add_packed_git(pack_data->repo, idx_name, strlen(idx_name), 1);
+		new_p = packfile_store_load_pack(pack_data->repo->objects->packfiles,
+						 idx_name, 1);
 		if (!new_p)
 			die("core git rejected index %s", idx_name);
 		all_packs[pack_id] = new_p;
-		packfile_store_add_pack(the_repository->objects->packfiles, new_p);
 		free(idx_name);
 
 		/* Print the boundary */
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index ed490dfad4..2b78ba7fe4 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1640,13 +1640,9 @@ static void final(const char *final_pack_name, const char *curr_pack_name,
 	rename_tmp_packfile(&final_index_name, curr_index_name, &index_name,
 			    hash, "idx", 1);
 
-	if (do_fsck_object) {
-		struct packed_git *p;
-		p = add_packed_git(the_repository, final_index_name,
-				   strlen(final_index_name), 0);
-		if (p)
-			packfile_store_add_pack(the_repository->objects->packfiles, p);
-	}
+	if (do_fsck_object)
+		packfile_store_load_pack(the_repository->objects->packfiles,
+					 final_index_name, 0);
 
 	if (!from_stdin) {
 		printf("%s\n", hash_to_hex(hash));
diff --git a/midx.c b/midx.c
index 69c44be71c..3faeaf2f8f 100644
--- a/midx.c
+++ b/midx.c
@@ -443,7 +443,6 @@ int prepare_midx_pack(struct multi_pack_index *m,
 {
 	struct repository *r = m->source->odb->repo;
 	struct strbuf pack_name = STRBUF_INIT;
-	struct strbuf key = STRBUF_INIT;
 	struct packed_git *p;
 
 	pack_int_id = midx_for_pack(&m, pack_int_id);
@@ -455,25 +454,11 @@ int prepare_midx_pack(struct multi_pack_index *m,
 
 	strbuf_addf(&pack_name, "%s/pack/%s", m->source->path,
 		    m->pack_names[pack_int_id]);
-
-	/* pack_map holds the ".pack" name, but we have the .idx */
-	strbuf_addbuf(&key, &pack_name);
-	strbuf_strip_suffix(&key, ".idx");
-	strbuf_addstr(&key, ".pack");
-	p = hashmap_get_entry_from_hash(&r->objects->packfiles->map,
-					strhash(key.buf), key.buf,
-					struct packed_git, packmap_ent);
-	if (!p) {
-		p = add_packed_git(r, pack_name.buf, pack_name.len,
-				   m->source->local);
-		if (p) {
-			packfile_store_add_pack(r->objects->packfiles, p);
-			list_add_tail(&p->mru, &r->objects->packfiles->mru);
-		}
-	}
-
+	p = packfile_store_load_pack(r->objects->packfiles,
+				     pack_name.buf, m->source->local);
+	if (p)
+		list_add_tail(&p->mru, &r->objects->packfiles->mru);
 	strbuf_release(&pack_name);
-	strbuf_release(&key);
 
 	if (!p) {
 		m->packs[pack_int_id] = MIDX_PACK_ERROR;
diff --git a/packfile.c b/packfile.c
index af806aba09..9224ca424c 100644
--- a/packfile.c
+++ b/packfile.c
@@ -792,6 +792,33 @@ void packfile_store_add_pack(struct packfile_store *store,
 	hashmap_add(&store->map, &pack->packmap_ent);
 }
 
+struct packed_git *packfile_store_load_pack(struct packfile_store *store,
+					    const char *idx_path, int local)
+{
+	struct strbuf key = STRBUF_INIT;
+	struct packed_git *p;
+
+	/*
+	 * We're being called with the path to the index file, but `pack_map`
+	 * holds the path to the packfile itself.
+	 */
+	strbuf_addstr(&key, idx_path);
+	strbuf_strip_suffix(&key, ".idx");
+	strbuf_addstr(&key, ".pack");
+
+	p = hashmap_get_entry_from_hash(&store->map, strhash(key.buf), key.buf,
+					struct packed_git, packmap_ent);
+	if (!p) {
+		p = add_packed_git(store->odb->repo, idx_path,
+				   strlen(idx_path), local);
+		if (p)
+			packfile_store_add_pack(store, p);
+	}
+
+	strbuf_release(&key);
+	return p;
+}
+
 void (*report_garbage)(unsigned seen_bits, const char *path);
 
 static void report_helper(const struct string_list *list,
@@ -891,23 +918,14 @@ static void prepare_pack(const char *full_name, size_t full_name_len,
 			 const char *file_name, void *_data)
 {
 	struct prepare_pack_data *data = (struct prepare_pack_data *)_data;
-	struct packed_git *p;
 	size_t base_len = full_name_len;
 
 	if (strip_suffix_mem(full_name, &base_len, ".idx") &&
 	    !(data->m && midx_contains_pack(data->m, file_name))) {
-		struct hashmap_entry hent;
-		char *pack_name = xstrfmt("%.*s.pack", (int)base_len, full_name);
-		unsigned int hash = strhash(pack_name);
-		hashmap_entry_init(&hent, hash);
-
-		/* Don't reopen a pack we already have. */
-		if (!hashmap_get(&data->r->objects->packfiles->map, &hent, pack_name)) {
-			p = add_packed_git(data->r, full_name, full_name_len, data->local);
-			if (p)
-				packfile_store_add_pack(data->r->objects->packfiles, p);
-		}
-		free(pack_name);
+		char *trimmed_path = xstrndup(full_name, full_name_len);
+		packfile_store_load_pack(data->r->objects->packfiles,
+					 trimmed_path, data->local);
+		free(trimmed_path);
 	}
 
 	if (!report_garbage)
diff --git a/packfile.h b/packfile.h
index ba4b0cef9c..fcefcbbef6 100644
--- a/packfile.h
+++ b/packfile.h
@@ -127,6 +127,14 @@ void packfile_store_reprepare(struct packfile_store *store);
 void packfile_store_add_pack(struct packfile_store *store,
 			     struct packed_git *pack);
 
+/*
+ * Open the packfile and add it to the store if it isn't yet known. Returns
+ * either the newly opened packfile or the preexisting packfile. Returns a
+ * `NULL` pointer in case the packfile could not be opened.
+ */
+struct packed_git *packfile_store_load_pack(struct packfile_store *store,
+					    const char *idx_path, int local);
+
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
-- 
2.51.0.450.g87641ccf93.dirty
Patrick Steinhardt· Sep 15, 2025, 08:54 UTC · re: Patrick Steinhardt · lore

[PATCH v5 12/15] packfile: move `get_multi_pack_index()` into "midx.c"

The `get_multi_pack_index()` function is declared and implemented in the packfile subsystem, even though it really belongs into the multi-pack index subsystem. The reason for this is likely that it needs to call `packfile_store_prepare()`, which is not exposed by the packfile system. In a subsequent commit we're about to add another caller outside of the packfile system though, so we'll have to expose the function anyway.

Do so now already and move `get_multi_pack_index()` into the MIDX subsystem.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 midx.c     |  6 ++++++
 midx.h     |  1 +
 packfile.c |  8 +-------
 packfile.h | 10 +++++++++-
 4 files changed, 17 insertions(+), 8 deletions(-)
Show changes to 4 files +17 −8

midx.c, midx.h, packfile.c, packfile.h

diff --git a/midx.c b/midx.c
index 3faeaf2f8f..1d6269f957 100644
--- a/midx.c
+++ b/midx.c
@@ -93,6 +93,12 @@ static int midx_read_object_offsets(const unsigned char *chunk_start,
 	return 0;
 }
 
+struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
+{
+	packfile_store_prepare(source->odb->packfiles);
+	return source->midx;
+}
+
 static struct multi_pack_index *load_multi_pack_index_one(struct odb_source *source,
 							  const char *midx_name)
 {
diff --git a/midx.h b/midx.h
index e241d2d690..6e54d73503 100644
--- a/midx.h
+++ b/midx.h
@@ -94,6 +94,7 @@ void get_midx_chain_filename(struct odb_source *source, struct strbuf *out);
 void get_split_midx_filename_ext(struct odb_source *source, struct strbuf *buf,
 				 const unsigned char *hash, const char *ext);
 
+struct multi_pack_index *get_multi_pack_index(struct odb_source *source);
 struct multi_pack_index *load_multi_pack_index(struct odb_source *source);
 int prepare_midx_pack(struct multi_pack_index *m, uint32_t pack_int_id);
 struct packed_git *nth_midxed_pack(struct multi_pack_index *m,
diff --git a/packfile.c b/packfile.c
index 9224ca424c..7a9193e5ef 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1003,7 +1003,7 @@ static void packfile_store_prepare_mru(struct packfile_store *store)
 		list_add_tail(&p->mru, &store->mru);
 }
 
-static void packfile_store_prepare(struct packfile_store *store)
+void packfile_store_prepare(struct packfile_store *store)
 {
 	struct odb_source *source;
 
@@ -1033,12 +1033,6 @@ struct packed_git *get_packed_git(struct repository *r)
 	return r->objects->packfiles->packs;
 }
 
-struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
-{
-	packfile_store_prepare(source->odb->packfiles);
-	return source->midx;
-}
-
 struct packed_git *get_all_packs(struct repository *r)
 {
 	packfile_store_prepare(r->objects->packfiles);
diff --git a/packfile.h b/packfile.h
index fcefcbbef6..a9e561ac39 100644
--- a/packfile.h
+++ b/packfile.h
@@ -112,6 +112,15 @@ void packfile_store_free(struct packfile_store *store);
  */
 void packfile_store_close(struct packfile_store *store);
 
+/*
+ * Prepare the packfile store by loading packfiles and multi-pack indices for
+ * all alternates. This becomes a no-op if the store is already prepared.
+ *
+ * It shouldn't typically be necessary to call this function directly, as
+ * functions that access the store know to prepare it.
+ */
+void packfile_store_prepare(struct packfile_store *store);
+
 /*
  * Clear the packfile caches and try to look up any new packfiles that have
  * appeared since last preparing the packfiles store.
@@ -213,7 +222,6 @@ extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
 struct packed_git *get_packed_git(struct repository *r);
 struct list_head *get_packed_git_mru(struct repository *r);
-struct multi_pack_index *get_multi_pack_index(struct odb_source *source);
 struct packed_git *get_all_packs(struct repository *r);
 
 /*
-- 
2.51.0.450.g87641ccf93.dirty
Patrick Steinhardt· Sep 15, 2025, 08:54 UTC · re: Patrick Steinhardt · lore

[PATCH v5 13/15] packfile: refactor `get_packed_git()` to work on packfile store

The `get_packed_git()` function prepares the packfile store and then returns its packfiles. Refactor it to accept a packfile store instead of a repository to clarify its scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/gc.c   | 2 +-
 builtin/grep.c | 2 +-
 object-name.c  | 4 ++--
 packfile.c     | 6 +++---
 packfile.h     | 7 ++++++-
 5 files changed, 13 insertions(+), 8 deletions(-)
Show changes to 5 files +13 −8

builtin/gc.c, builtin/grep.c, object-name.c, packfile.c, packfile.h

diff --git a/builtin/gc.c b/builtin/gc.c
index aeca06a08b..ec6735a540 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1423,7 +1423,7 @@ static int incremental_repack_auto_condition(struct gc_config *cfg UNUSED)
 	if (incremental_repack_auto_limit < 0)
 		return 1;
 
-	for (p = get_packed_git(the_repository);
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles);
 	     count < incremental_repack_auto_limit && p;
 	     p = p->next) {
 		if (!p->multi_pack_index)
diff --git a/builtin/grep.c b/builtin/grep.c
index 5df6537333..63a4959568 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -1214,7 +1214,7 @@ int cmd_grep(int argc,
 		if (recurse_submodules)
 			repo_read_gitmodules(the_repository, 1);
 		if (startup_info->have_repository)
-			(void)get_packed_git(the_repository);
+			(void)packfile_store_get_packs(the_repository->objects->packfiles);
 
 		start_threads(&opt);
 	} else {
diff --git a/object-name.c b/object-name.c
index df9e0c5f02..53356819a3 100644
--- a/object-name.c
+++ b/object-name.c
@@ -213,7 +213,7 @@ static void find_short_packed_object(struct disambiguate_state *ds)
 			unique_in_midx(m, ds);
 	}
 
-	for (p = get_packed_git(ds->repo); p && !ds->ambiguous;
+	for (p = packfile_store_get_packs(ds->repo->objects->packfiles); p && !ds->ambiguous;
 	     p = p->next)
 		unique_in_pack(p, ds);
 }
@@ -806,7 +806,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)
 			find_abbrev_len_for_midx(m, mad);
 	}
 
-	for (p = get_packed_git(mad->repo); p; p = p->next)
+	for (p = packfile_store_get_packs(mad->repo->objects->packfiles); p; p = p->next)
 		find_abbrev_len_for_pack(p, mad);
 }
 
diff --git a/packfile.c b/packfile.c
index 7a9193e5ef..b37f43afb5 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1027,10 +1027,10 @@ void packfile_store_reprepare(struct packfile_store *store)
 	packfile_store_prepare(store);
 }
 
-struct packed_git *get_packed_git(struct repository *r)
+struct packed_git *packfile_store_get_packs(struct packfile_store *store)
 {
-	packfile_store_prepare(r->objects->packfiles);
-	return r->objects->packfiles->packs;
+	packfile_store_prepare(store);
+	return store->packs;
 }
 
 struct packed_git *get_all_packs(struct repository *r)
diff --git a/packfile.h b/packfile.h
index a9e561ac39..0b691ded7e 100644
--- a/packfile.h
+++ b/packfile.h
@@ -136,6 +136,12 @@ void packfile_store_reprepare(struct packfile_store *store);
 void packfile_store_add_pack(struct packfile_store *store,
 			     struct packed_git *pack);
 
+/*
+ * Get packs managed by the given store. Does not load the MIDX or any packs
+ * referenced by it.
+ */
+struct packed_git *packfile_store_get_packs(struct packfile_store *store);
+
 /*
  * Open the packfile and add it to the store if it isn't yet known. Returns
  * either the newly opened packfile or the preexisting packfile. Returns a
@@ -220,7 +226,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-struct packed_git *get_packed_git(struct repository *r);
 struct list_head *get_packed_git_mru(struct repository *r);
 struct packed_git *get_all_packs(struct repository *r);
 
-- 
2.51.0.450.g87641ccf93.dirty
Patrick Steinhardt· Sep 15, 2025, 08:54 UTC · re: Patrick Steinhardt · lore

[PATCH v5 14/15] packfile: refactor `get_all_packs()` to work on packfile store

The `get_all_packs()` function prepares the packfile store and then returns its packfiles. Refactor it to accept a packfile store instead of a repository to clarify its scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/cat-file.c          |  3 ++-
 builtin/count-objects.c     |  3 ++-
 builtin/fast-import.c       |  6 ++++--
 builtin/fsck.c              | 11 +++++++----
 builtin/gc.c                |  8 +++++---
 builtin/pack-objects.c      | 28 +++++++++++++++++++---------
 builtin/pack-redundant.c    |  6 ++++--
 builtin/repack.c            |  9 ++++++---
 connected.c                 |  3 ++-
 http-backend.c              |  5 +++--
 http.c                      |  3 ++-
 pack-bitmap.c               |  4 ++--
 pack-objects.c              |  3 ++-
 packfile.c                  | 12 ++++++------
 packfile.h                  |  7 ++++++-
 server-info.c               |  3 ++-
 t/helper/test-find-pack.c   |  2 +-
 t/helper/test-pack-mtimes.c |  2 +-
 18 files changed, 76 insertions(+), 42 deletions(-)
Show changes to 18 files +76 −42

builtin/cat-file.c, builtin/count-objects.c, builtin/fast-import.c, builtin/fsck.c, builtin/gc.c, builtin/pack-objects.c, builtin/pack-redundant.c, builtin/repack.c, connected.c, http-backend.c, http.c, pack-bitmap.c, pack-objects.c, packfile.c, packfile.h, server-info.c, t/helper/test-find-pack.c, t/helper/test-pack-mtimes.c

diff --git a/builtin/cat-file.c b/builtin/cat-file.c
index fce0b06451c..ee6715fa523 100644
--- a/builtin/cat-file.c
+++ b/builtin/cat-file.c
@@ -852,9 +852,10 @@ static void batch_each_object(struct batch_options *opt,
 
 	if (bitmap && !for_each_bitmapped_object(bitmap, &opt->objects_filter,
 						 batch_one_object_bitmapped, &payload)) {
+		struct packfile_store *packs = the_repository->objects->packfiles;
 		struct packed_git *pack;
 
-		for (pack = get_all_packs(the_repository); pack; pack = pack->next) {
+		for (pack = packfile_store_get_all_packs(packs); pack; pack = pack->next) {
 			if (bitmap_index_contains_pack(bitmap, pack) ||
 			    open_pack_index(pack))
 				continue;
diff --git a/builtin/count-objects.c b/builtin/count-objects.c
index a61d3b46aac..f2f407c2a78 100644
--- a/builtin/count-objects.c
+++ b/builtin/count-objects.c
@@ -122,6 +122,7 @@ int cmd_count_objects(int argc,
 				      count_loose, count_cruft, NULL, NULL);
 
 	if (verbose) {
+		struct packfile_store *packs = the_repository->objects->packfiles;
 		struct packed_git *p;
 		unsigned long num_pack = 0;
 		off_t size_pack = 0;
@@ -129,7 +130,7 @@ int cmd_count_objects(int argc,
 		struct strbuf pack_buf = STRBUF_INIT;
 		struct strbuf garbage_buf = STRBUF_INIT;
 
-		for (p = get_all_packs(the_repository); p; p = p->next) {
+		for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 			if (!p->pack_local)
 				continue;
 			if (open_pack_index(p))
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index a26e79689d5..b1d5549815a 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -952,6 +952,7 @@ static int store_object(
 	struct object_id *oidout,
 	uintmax_t mark)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	void *out, *delta;
 	struct object_entry *e;
 	unsigned char hdr[96];
@@ -975,7 +976,7 @@ static int store_object(
 	if (e->idx.offset) {
 		duplicate_count_by_type[type]++;
 		return 1;
-	} else if (find_oid_pack(&oid, get_all_packs(the_repository))) {
+	} else if (find_oid_pack(&oid, packfile_store_get_all_packs(packs))) {
 		e->type = type;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
@@ -1092,6 +1093,7 @@ static void truncate_pack(struct hashfile_checkpoint *checkpoint)
 
 static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	size_t in_sz = 64 * 1024, out_sz = 64 * 1024;
 	unsigned char *in_buf = xmalloc(in_sz);
 	unsigned char *out_buf = xmalloc(out_sz);
@@ -1175,7 +1177,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 		duplicate_count_by_type[OBJ_BLOB]++;
 		truncate_pack(&checkpoint);
 
-	} else if (find_oid_pack(&oid, get_all_packs(the_repository))) {
+	} else if (find_oid_pack(&oid, packfile_store_get_all_packs(packs))) {
 		e->type = OBJ_BLOB;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
diff --git a/builtin/fsck.c b/builtin/fsck.c
index d2eb9d4fbe9..8ee95e0d67c 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -867,19 +867,20 @@ static int mark_packed_for_connectivity(const struct object_id *oid,
 
 static int check_pack_rev_indexes(struct repository *r, int show_progress)
 {
+	struct packfile_store *packs = r->objects->packfiles;
 	struct progress *progress = NULL;
 	uint32_t pack_count = 0;
 	int res = 0;
 
 	if (show_progress) {
-		for (struct packed_git *p = get_all_packs(r); p; p = p->next)
+		for (struct packed_git *p = packfile_store_get_all_packs(packs); p; p = p->next)
 			pack_count++;
 		progress = start_delayed_progress(the_repository,
 						  "Verifying reverse pack-indexes", pack_count);
 		pack_count = 0;
 	}
 
-	for (struct packed_git *p = get_all_packs(r); p; p = p->next) {
+	for (struct packed_git *p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		int load_error = load_pack_revindex_from_disk(p);
 
 		if (load_error < 0) {
@@ -999,6 +1000,8 @@ int cmd_fsck(int argc,
 		for_each_packed_object(the_repository,
 				       mark_packed_for_connectivity, NULL, 0);
 	} else {
+		struct packfile_store *packs = the_repository->objects->packfiles;
+
 		odb_prepare_alternates(the_repository->objects);
 		for (source = the_repository->objects->sources; source; source = source->next)
 			fsck_source(source);
@@ -1009,7 +1012,7 @@ int cmd_fsck(int argc,
 			struct progress *progress = NULL;
 
 			if (show_progress) {
-				for (p = get_all_packs(the_repository); p;
+				for (p = packfile_store_get_all_packs(packs); p;
 				     p = p->next) {
 					if (open_pack_index(p))
 						continue;
@@ -1019,7 +1022,7 @@ int cmd_fsck(int argc,
 				progress = start_progress(the_repository,
 							  _("Checking objects"), total);
 			}
-			for (p = get_all_packs(the_repository); p;
+			for (p = packfile_store_get_all_packs(packs); p;
 			     p = p->next) {
 				/* verify gives error messages itself */
 				if (verify_pack(the_repository,
diff --git a/builtin/gc.c b/builtin/gc.c
index ec6735a540a..e19e13d9788 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -487,9 +487,10 @@ static int too_many_loose_objects(struct gc_config *cfg)
 static struct packed_git *find_base_packs(struct string_list *packs,
 					  unsigned long limit)
 {
+	struct packfile_store *packfiles = the_repository->objects->packfiles;
 	struct packed_git *p, *base = NULL;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(packfiles); p; p = p->next) {
 		if (!p->pack_local || p->is_cruft)
 			continue;
 		if (limit) {
@@ -508,13 +509,14 @@ static struct packed_git *find_base_packs(struct string_list *packs,
 
 static int too_many_packs(struct gc_config *cfg)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 	int cnt;
 
 	if (cfg->gc_auto_pack_limit <= 0)
 		return 0;
 
-	for (cnt = 0, p = get_all_packs(the_repository); p; p = p->next) {
+	for (cnt = 0, p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		if (!p->pack_local)
 			continue;
 		if (p->pack_keep)
@@ -1492,7 +1494,7 @@ static off_t get_auto_pack_size(void)
 	struct repository *r = the_repository;
 
 	odb_reprepare(r->objects);
-	for (p = get_all_packs(r); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(r->objects->packfiles); p; p = p->next) {
 		if (p->pack_size > max_size) {
 			second_largest_size = max_size;
 			max_size = p->pack_size;
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 1494afcf3df..de351b757ae 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -3831,6 +3831,7 @@ static int pack_mtime_cmp(const void *_a, const void *_b)
 
 static void read_packs_list_from_stdin(struct rev_info *revs)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct strbuf buf = STRBUF_INIT;
 	struct string_list include_packs = STRING_LIST_INIT_DUP;
 	struct string_list exclude_packs = STRING_LIST_INIT_DUP;
@@ -3855,7 +3856,7 @@ static void read_packs_list_from_stdin(struct rev_info *revs)
 	string_list_sort(&exclude_packs);
 	string_list_remove_duplicates(&exclude_packs, 0);
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		const char *pack_name = pack_basename(p);
 
 		if ((item = string_list_lookup(&include_packs, pack_name)))
@@ -4076,6 +4077,7 @@ static void enumerate_cruft_objects(void)
 
 static void enumerate_and_traverse_cruft_objects(struct string_list *fresh_packs)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 	struct rev_info revs;
 	int ret;
@@ -4105,7 +4107,7 @@ static void enumerate_and_traverse_cruft_objects(struct string_list *fresh_packs
 	 * Re-mark only the fresh packs as kept so that objects in
 	 * unknown packs do not halt the reachability traversal early.
 	 */
-	for (p = get_all_packs(the_repository); p; p = p->next)
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next)
 		p->pack_keep_in_core = 0;
 	mark_pack_kept_in_core(fresh_packs, 1);
 
@@ -4122,6 +4124,7 @@ static void enumerate_and_traverse_cruft_objects(struct string_list *fresh_packs
 
 static void read_cruft_objects(void)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct strbuf buf = STRBUF_INIT;
 	struct string_list discard_packs = STRING_LIST_INIT_DUP;
 	struct string_list fresh_packs = STRING_LIST_INIT_DUP;
@@ -4142,7 +4145,7 @@ static void read_cruft_objects(void)
 	string_list_sort(&discard_packs);
 	string_list_sort(&fresh_packs);
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		const char *pack_name = pack_basename(p);
 		struct string_list_item *item;
 
@@ -4390,11 +4393,12 @@ static void add_unreachable_loose_objects(struct rev_info *revs)
 
 static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	static struct packed_git *last_found = (void *)1;
 	struct packed_git *p;
 
 	p = (last_found != (void *)1) ? last_found :
-					get_all_packs(the_repository);
+					packfile_store_get_all_packs(packs);
 
 	while (p) {
 		if ((!p->pack_local || p->pack_keep ||
@@ -4404,7 +4408,7 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 			return 1;
 		}
 		if (p == last_found)
-			p = get_all_packs(the_repository);
+			p = packfile_store_get_all_packs(packs);
 		else
 			p = p->next;
 		if (p == last_found)
@@ -4436,12 +4440,13 @@ static int loosened_object_can_be_discarded(const struct object_id *oid,
 
 static void loosen_unused_packed_objects(void)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 	uint32_t i;
 	uint32_t loosened_objects_nr = 0;
 	struct object_id oid;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		if (!p->pack_local || p->pack_keep || p->pack_keep_in_core)
 			continue;
 
@@ -4742,12 +4747,13 @@ static void get_object_list(struct rev_info *revs, int ac, const char **av)
 
 static void add_extra_kept_packs(const struct string_list *names)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 
 	if (!names->nr)
 		return;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		const char *name = basename(p->pack_name);
 		int i;
 
@@ -5185,8 +5191,10 @@ int cmd_pack_objects(int argc,
 
 	add_extra_kept_packs(&keep_pack_list);
 	if (ignore_packed_keep_on_disk) {
+		struct packfile_store *packs = the_repository->objects->packfiles;
 		struct packed_git *p;
-		for (p = get_all_packs(the_repository); p; p = p->next)
+
+		for (p = packfile_store_get_all_packs(packs); p; p = p->next)
 			if (p->pack_local && p->pack_keep)
 				break;
 		if (!p) /* no keep-able packs found */
@@ -5198,8 +5206,10 @@ int cmd_pack_objects(int argc,
 		 * want to unset "local" based on looking at packs, as
 		 * it also covers non-local objects
 		 */
+		struct packfile_store *packs = the_repository->objects->packfiles;
 		struct packed_git *p;
-		for (p = get_all_packs(the_repository); p; p = p->next) {
+
+		for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 			if (!p->pack_local) {
 				have_non_local_packs = 1;
 				break;
diff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c
index fe81c293e3a..dd28171f0a1 100644
--- a/builtin/pack-redundant.c
+++ b/builtin/pack-redundant.c
@@ -566,7 +566,8 @@ static struct pack_list * add_pack(struct packed_git *p)
 
 static struct pack_list * add_pack_file(const char *filename)
 {
-	struct packed_git *p = get_all_packs(the_repository);
+	struct packfile_store *packs = the_repository->objects->packfiles;
+	struct packed_git *p = packfile_store_get_all_packs(packs);
 
 	if (strlen(filename) < 40)
 		die("Bad pack filename: %s", filename);
@@ -581,7 +582,8 @@ static struct pack_list * add_pack_file(const char *filename)
 
 static void load_all(void)
 {
-	struct packed_git *p = get_all_packs(the_repository);
+	struct packfile_store *packs = the_repository->objects->packfiles;
+	struct packed_git *p = packfile_store_get_all_packs(packs);
 
 	while (p) {
 		add_pack(p);
diff --git a/builtin/repack.c b/builtin/repack.c
index 5ff27fc8e29..e8730808c53 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -265,10 +265,11 @@ static void existing_packs_release(struct existing_packs *existing)
 static void collect_pack_filenames(struct existing_packs *existing,
 				   const struct string_list *extra_keep)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 	struct strbuf buf = STRBUF_INIT;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		int i;
 		const char *base;
 
@@ -497,10 +498,11 @@ static void init_pack_geometry(struct pack_geometry *geometry,
 			       struct existing_packs *existing,
 			       const struct pack_objects_args *args)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 	struct strbuf buf = STRBUF_INIT;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		if (args->local && !p->pack_local)
 			/*
 			 * When asked to only repack local packfiles we skip
@@ -1137,11 +1139,12 @@ static int write_filtered_pack(const struct pack_objects_args *args,
 static void combine_small_cruft_packs(FILE *in, size_t combine_cruft_below_size,
 				      struct existing_packs *existing)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 	struct strbuf buf = STRBUF_INIT;
 	size_t i;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		if (!(p->is_cruft && p->pack_local))
 			continue;
 
diff --git a/connected.c b/connected.c
index d6e9682fd93..b288a18b17c 100644
--- a/connected.c
+++ b/connected.c
@@ -74,9 +74,10 @@ int check_connected(oid_iterate_fn fn, void *cb_data,
 		 */
 		odb_reprepare(the_repository->objects);
 		do {
+			struct packfile_store *packs = the_repository->objects->packfiles;
 			struct packed_git *p;
 
-			for (p = get_all_packs(the_repository); p; p = p->next) {
+			for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 				if (!p->pack_promisor)
 					continue;
 				if (find_pack_entry_one(oid, p))
diff --git a/http-backend.c b/http-backend.c
index d5dfe762bb5..9084058f1e9 100644
--- a/http-backend.c
+++ b/http-backend.c
@@ -603,18 +603,19 @@ static void get_head(struct strbuf *hdr, char *arg UNUSED)
 static void get_info_packs(struct strbuf *hdr, char *arg UNUSED)
 {
 	size_t objdirlen = strlen(repo_get_object_directory(the_repository));
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct strbuf buf = STRBUF_INIT;
 	struct packed_git *p;
 	size_t cnt = 0;
 
 	select_getanyfile(hdr);
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		if (p->pack_local)
 			cnt++;
 	}
 
 	strbuf_grow(&buf, cnt * 53 + 2);
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		if (p->pack_local)
 			strbuf_addf(&buf, "P %s\n", p->pack_name + objdirlen + 6);
 	}
diff --git a/http.c b/http.c
index af2120b64c7..077e879de9e 100644
--- a/http.c
+++ b/http.c
@@ -2408,6 +2408,7 @@ static char *fetch_pack_index(unsigned char *hash, const char *base_url)
 static int fetch_and_setup_pack_index(struct packed_git **packs_head,
 	unsigned char *sha1, const char *base_url)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *new_pack, *p;
 	char *tmp_idx = NULL;
 	int ret;
@@ -2416,7 +2417,7 @@ static int fetch_and_setup_pack_index(struct packed_git **packs_head,
 	 * If we already have the pack locally, no need to fetch its index or
 	 * even add it to list; we already have all of its objects.
 	 */
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		if (hasheq(p->hash, sha1, the_repository->hash_algo))
 			return 0;
 	}
diff --git a/pack-bitmap.c b/pack-bitmap.c
index 058bdb5d7de..ac71035d771 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -664,7 +664,7 @@ static int open_pack_bitmap(struct repository *r,
 	struct packed_git *p;
 	int ret = -1;
 
-	for (p = get_all_packs(r); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(r->objects->packfiles); p; p = p->next) {
 		if (open_pack_bitmap_1(bitmap_git, p) == 0) {
 			ret = 0;
 			/*
@@ -3362,7 +3362,7 @@ int verify_bitmap_files(struct repository *r)
 		free(midx_bitmap_name);
 	}
 
-	for (struct packed_git *p = get_all_packs(r);
+	for (struct packed_git *p = packfile_store_get_all_packs(r->objects->packfiles);
 	     p; p = p->next) {
 		char *pack_bitmap_name = pack_bitmap_filename(p);
 		res |= verify_bitmap_file(r->hash_algo, pack_bitmap_name);
diff --git a/pack-objects.c b/pack-objects.c
index a9d9855063a..d8eb6797354 100644
--- a/pack-objects.c
+++ b/pack-objects.c
@@ -86,6 +86,7 @@ struct object_entry *packlist_find(struct packing_data *pdata,
 
 static void prepare_in_pack_by_idx(struct packing_data *pdata)
 {
+	struct packfile_store *packs = pdata->repo->objects->packfiles;
 	struct packed_git **mapping, *p;
 	int cnt = 0, nr = 1U << OE_IN_PACK_BITS;
 
@@ -95,7 +96,7 @@ static void prepare_in_pack_by_idx(struct packing_data *pdata)
 	 * (i.e. in_pack_idx also zero) should return NULL.
 	 */
 	mapping[cnt++] = NULL;
-	for (p = get_all_packs(pdata->repo); p; p = p->next, cnt++) {
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next, cnt++) {
 		if (cnt == nr) {
 			free(mapping);
 			return;
diff --git a/packfile.c b/packfile.c
index b37f43afb58..cd5431b6aa1 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1033,11 +1033,11 @@ struct packed_git *packfile_store_get_packs(struct packfile_store *store)
 	return store->packs;
 }
 
-struct packed_git *get_all_packs(struct repository *r)
+struct packed_git *packfile_store_get_all_packs(struct packfile_store *store)
 {
-	packfile_store_prepare(r->objects->packfiles);
+	packfile_store_prepare(store);
 
-	for (struct odb_source *source = r->objects->sources; source; source = source->next) {
+	for (struct odb_source *source = store->odb->sources; source; source = source->next) {
 		struct multi_pack_index *m = source->midx;
 		if (!m)
 			continue;
@@ -1045,7 +1045,7 @@ struct packed_git *get_all_packs(struct repository *r)
 			prepare_midx_pack(m, i);
 	}
 
-	return r->objects->packfiles->packs;
+	return store->packs;
 }
 
 struct list_head *get_packed_git_mru(struct repository *r)
@@ -2105,7 +2105,7 @@ struct packed_git **kept_pack_cache(struct repository *r, unsigned flags)
 		 * covers, one kept and one not kept, but the midx returns only
 		 * the non-kept version.
 		 */
-		for (p = get_all_packs(r); p; p = p->next) {
+		for (p = packfile_store_get_all_packs(r->objects->packfiles); p; p = p->next) {
 			if ((p->pack_keep && (flags & ON_DISK_KEEP_PACKS)) ||
 			    (p->pack_keep_in_core && (flags & IN_CORE_KEEP_PACKS))) {
 				ALLOC_GROW(packs, nr + 1, alloc);
@@ -2202,7 +2202,7 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 	int r = 0;
 	int pack_errors = 0;
 
-	for (p = get_all_packs(repo); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(repo->objects->packfiles); p; p = p->next) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
 		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&
diff --git a/packfile.h b/packfile.h
index 0b691ded7ef..1afb9cd6641 100644
--- a/packfile.h
+++ b/packfile.h
@@ -142,6 +142,12 @@ void packfile_store_add_pack(struct packfile_store *store,
  */
 struct packed_git *packfile_store_get_packs(struct packfile_store *store);
 
+/*
+ * Get all packs managed by the given store, including packfiles that are
+ * referenced by multi-pack indices.
+ */
+struct packed_git *packfile_store_get_all_packs(struct packfile_store *store);
+
 /*
  * Open the packfile and add it to the store if it isn't yet known. Returns
  * either the newly opened packfile or the preexisting packfile. Returns a
@@ -227,7 +233,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
 struct list_head *get_packed_git_mru(struct repository *r);
-struct packed_git *get_all_packs(struct repository *r);
 
 /*
  * Give a rough count of objects in the repository. This sacrifices accuracy
diff --git a/server-info.c b/server-info.c
index 9bb30d9ab71..1d33de821e9 100644
--- a/server-info.c
+++ b/server-info.c
@@ -287,12 +287,13 @@ static int compare_info(const void *a_, const void *b_)
 
 static void init_pack_info(struct repository *r, const char *infofile, int force)
 {
+	struct packfile_store *packs = r->objects->packfiles;
 	struct packed_git *p;
 	int stale;
 	int i;
 	size_t alloc = 0;
 
-	for (p = get_all_packs(r); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		/* we ignore things on alternate path since they are
 		 * not available to the pullers in general.
 		 */
diff --git a/t/helper/test-find-pack.c b/t/helper/test-find-pack.c
index 611a13a3261..e001dc3066d 100644
--- a/t/helper/test-find-pack.c
+++ b/t/helper/test-find-pack.c
@@ -39,7 +39,7 @@ int cmd__find_pack(int argc, const char **argv)
 	if (repo_get_oid(the_repository, argv[0], &oid))
 		die("cannot parse %s as an object name", argv[0]);
 
-	for (p = get_all_packs(the_repository); p; p = p->next)
+	for (p = packfile_store_get_all_packs(the_repository->objects->packfiles); p; p = p->next)
 		if (find_pack_entry_one(&oid, p)) {
 			printf("%s\n", p->pack_name);
 			actual_count++;
diff --git a/t/helper/test-pack-mtimes.c b/t/helper/test-pack-mtimes.c
index d51aaa3dc40..7c428c16011 100644
--- a/t/helper/test-pack-mtimes.c
+++ b/t/helper/test-pack-mtimes.c
@@ -37,7 +37,7 @@ int cmd__pack_mtimes(int argc, const char **argv)
 	if (argc != 2)
 		usage(pack_mtimes_usage);
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(the_repository->objects->packfiles); p; p = p->next) {
 		strbuf_addstr(&buf, basename(p->pack_name));
 		strbuf_strip_suffix(&buf, ".pack");
 		strbuf_addstr(&buf, ".mtimes");
-- 
2.51.0.450.g87641ccf93.dirty
Patrick Steinhardt· Sep 15, 2025, 08:54 UTC · re: Patrick Steinhardt · lore

[PATCH v5 15/15] packfile: refactor `get_packed_git_mru()` to work on packfile store

The `get_packed_git_mru()` function prepares the packfile store and then returns its packfiles in most-recently-used order. Refactor it to accept a packfile store instead of a repository to clarify its scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/pack-objects.c | 4 ++--
 packfile.c             | 6 +++---
 packfile.h             | 7 +++++--
 3 files changed, 10 insertions(+), 7 deletions(-)
Show changes to 3 files +10 −7

builtin/pack-objects.c, packfile.c, packfile.h

diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index de351b757a..61bbbdfb83 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -1748,12 +1748,12 @@ static int want_object_in_pack_mtime(const struct object_id *oid,
 		}
 	}
 
-	list_for_each(pos, get_packed_git_mru(the_repository)) {
+	list_for_each(pos, packfile_store_get_packs_mru(the_repository->objects->packfiles)) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		want = want_object_in_pack_one(p, oid, exclude, found_pack, found_offset, found_mtime);
 		if (!exclude && want > 0)
 			list_move(&p->mru,
-				  get_packed_git_mru(the_repository));
+				  packfile_store_get_packs_mru(the_repository->objects->packfiles));
 		if (want != -1)
 			return want;
 	}
diff --git a/packfile.c b/packfile.c
index cd5431b6aa..5a7caec292 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1048,10 +1048,10 @@ struct packed_git *packfile_store_get_all_packs(struct packfile_store *store)
 	return store->packs;
 }
 
-struct list_head *get_packed_git_mru(struct repository *r)
+struct list_head *packfile_store_get_packs_mru(struct packfile_store *store)
 {
-	packfile_store_prepare(r->objects->packfiles);
-	return &r->objects->packfiles->mru;
+	packfile_store_prepare(store);
+	return &store->mru;
 }
 
 /*
diff --git a/packfile.h b/packfile.h
index 1afb9cd664..e7a5792b6c 100644
--- a/packfile.h
+++ b/packfile.h
@@ -148,6 +148,11 @@ struct packed_git *packfile_store_get_packs(struct packfile_store *store);
  */
 struct packed_git *packfile_store_get_all_packs(struct packfile_store *store);
 
+/*
+ * Get all packs in most-recently-used order.
+ */
+struct list_head *packfile_store_get_packs_mru(struct packfile_store *store);
+
 /*
  * Open the packfile and add it to the store if it isn't yet known. Returns
  * either the newly opened packfile or the preexisting packfile. Returns a
@@ -232,8 +237,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-struct list_head *get_packed_git_mru(struct repository *r);
-
 /*
  * Give a rough count of objects in the repository. This sacrifices accuracy
  * for speed.
-- 
2.51.0.450.g87641ccf93.dirty
Patrick Steinhardt· Sep 23, 2025, 10:16 UTC · re: Patrick Steinhardt · lore

[PATCH v6 00/15] packfile: carve out a new packfile store

Hi,

information about a object database's packfiles is currently distributed across two different structures:

  - `struct packed_git` contains the `next` pointer as well as the
    `mru_head`, both of which serve to store the list of packfiles.
  - `struct object_database` contains several fields that relate to the
    packfiles.

So we don't really have a central data structure that tracks our packfiles, and consequently responsibilities aren't always clear cut. A consequence for the upcoming pluggable object databases is that this makes it very hard to move management of packfiles from the object database level down into the object database source.

This patch series introduces a new `struct packfile_store`, which is about to become the single source of truth for managing packfiles, and carves out the packfile store subsystem.

This is the first step to make packfiles work with pluggable object databases. Next steps will be to:

  - Move the `struct packed_git::next` and `struct packed::mru_head`
    pointers into the packfile store so that `struct packed_git` only
    tracks a single packfile.
  - Push the `struct packfile_store` down one level so that it's not
    hosted by the object database anymore, but instead by the object
    database source.
Changes in v2:
  - Convert the `initialized` flag into a boolean.
  - Polish some commit messages.
  - Some smaller formatting changes to the layout of `struct
    object_database`.
  - Link to v1: https://lore.kernel.org/r/20250819-b4-pks-packfiles-store-v1-0-1660842e125a@pks.im
Changes in v3:
  - Rebased on top of master at 6ad8021821 (The fifth batch, 2025-08-29)
    with ps/object-store-midx-dedup-info at 13296ac909 (midx: compute
    paths via their source, 2025-08-11) merged into it. This fixes
    various conflicts with "seen". There's still two conflicts: a
    trivial one with jt/de-global-bulk-checkin. And a more complex one
    with tb/prepare-midx-pack-cleanup. I don't think it's necessary to
    really address the first one, but I'm unsure how to proceed with the
    second one given that the patch series still seems to be cooking.
  - Set `struct object_database::packfiles` to `NULL` after free'ing it.
  - Add a comment to explain the kept cache.
  - Fix a missing `obj_read_lock()` call.
  - Drop the commit that always adds packfiles to the MRU. I've moved
    this into a subsequent patch series.
  - Avoid some overly long lines by storing the pointer to the packfile
    store on the stack.
  - Point out the difference between `get_all_packs()` and
    `get_packed_git()`.
  - Link to v2: https://lore.kernel.org/r/20250821-b4-pks-packfiles-store-v2-0-d10623355e9f@pks.im
Changes in v4:
  - Small code style improvement as suggested by Junio.
  - Some commit message improvements as suggested by Karthik.
  - Link to v3: https://lore.kernel.org/r/20250902-b4-pks-packfiles-store-v3-0-6925278efeda@pks.im
Changes in v5:
  - Stop closing packfiles when freeing the packfile store.
  - Don't drop `get_packed_git()` for now. This will be done in a
    subsequent patch series.
  - Link to v4: https://lore.kernel.org/r/20250909-b4-pks-packfiles-store-v4-0-151c4ba3619f@pks.im
Changes in v6:
  - Fix a grammar issue in a commit message.
  - Update a comment to reflect that we access `struct packfile_store`
    in "midx.c" directly, as well.
  - Link to v5: https://lore.kernel.org/r/20250915-b4-pks-packfiles-store-v5-0-d6340350934f@pks.im
Thanks!
Patrick
---
Patrick Steinhardt (15):
      packfile: introduce a new `struct packfile_store`
      odb: move list of packfiles into `struct packfile_store`
      odb: move initialization bit into `struct packfile_store`
      odb: move packfile map into `struct packfile_store`
      odb: move MRU list of packfiles into `struct packfile_store`
      odb: move kept cache into `struct packfile_store`
      packfile: reorder functions to avoid function declaration
      packfile: refactor `prepare_packed_git()` to work on packfile store
      packfile: split up responsibilities of `reprepare_packed_git()`
      packfile: refactor `install_packed_git()` to work on packfile store
      packfile: introduce function to load and add packfiles
      packfile: move `get_multi_pack_index()` into "midx.c"
      packfile: refactor `get_packed_git()` to work on packfile store
      packfile: refactor `get_all_packs()` to work on packfile store
      packfile: refactor `get_packed_git_mru()` to work on packfile store
 builtin/backfill.c          |   2 +-
 builtin/cat-file.c          |   3 +-
 builtin/count-objects.c     |   3 +-
 builtin/fast-import.c       |  10 +-
 builtin/fsck.c              |  11 +-
 builtin/gc.c                |  14 ++-
 builtin/grep.c              |   2 +-
 builtin/index-pack.c        |  10 +-
 builtin/pack-objects.c      |  32 +++--
 builtin/pack-redundant.c    |   6 +-
 builtin/receive-pack.c      |   2 +-
 builtin/repack.c            |  11 +-
 bulk-checkin.c              |   2 +-
 connected.c                 |   5 +-
 fetch-pack.c                |   4 +-
 http-backend.c              |   5 +-
 http.c                      |   5 +-
 http.h                      |   2 +-
 midx.c                      |  29 ++---
 midx.h                      |   1 +
 object-name.c               |   6 +-
 odb.c                       |  40 +++++--
 odb.h                       |  36 ++----
 pack-bitmap.c               |   4 +-
 pack-objects.c              |   3 +-
 packfile.c                  | 283 ++++++++++++++++++++++++--------------------
 packfile.h                  | 125 ++++++++++++++++---
 server-info.c               |   3 +-
 t/helper/test-find-pack.c   |   2 +-
 t/helper/test-pack-mtimes.c |   2 +-
 transport-helper.c          |   2 +-
 31 files changed, 396 insertions(+), 269 deletions(-)
Range-diff versus v5:
 1:  00b1c404db !  1:  976df0d77d packfile: introduce a new `struct packfile_store`
    @@ Metadata
      ## Commit message ##
         packfile: introduce a new `struct packfile_store`
     
    -    Information about a object database's packfiles is currently distributed
    -    across two different structures:
    +    Information about an object database's packfiles is currently
    +    distributed across two different structures:
     
           - `struct packed_git` contains the `next` pointer as well as the
             `mru_head`, both of which serve to store the list of packfiles.
 2:  bd31834585 =  2:  0aa64a5a70 odb: move list of packfiles into `struct packfile_store`
 3:  d5d99e619b =  3:  7681f7d72d odb: move initialization bit into `struct packfile_store`
 4:  8d905f3e58 !  4:  b65334c581 odb: move packfile map into `struct packfile_store`
    @@ odb.c: void odb_clear(struct object_database *o)
      }
     
      ## odb.h ##
    +@@ odb.h: struct object_database {
    + 	/*
    + 	 * private data
    + 	 *
    +-	 * should only be accessed directly by packfile.c
    ++	 * Should only be accessed directly by packfile.c and midx.c.
    + 	 */
    + 	struct packfile_store *packfiles;
    + 	/* A most-recently-used ordered version of the packed_git list. */
     @@ odb.h: struct object_database {
      	struct cached_object_entry *cached_objects;
      	size_t cached_object_nr, cached_object_alloc;
 5:  a5d14ca02c !  5:  5d067ef530 odb: move MRU list of packfiles into `struct packfile_store`
    @@ odb.h
      #include "oidmap.h"
      #include "string-list.h"
     @@ odb.h: struct object_database {
    - 	 * should only be accessed directly by packfile.c
    + 	 * Should only be accessed directly by packfile.c and midx.c.
      	 */
      	struct packfile_store *packfiles;
     -	/* A most-recently-used ordered version of the packed_git list. */
 6:  a957203b3c !  6:  2d4cd7e170 odb: move kept cache into `struct packfile_store`
    @@ Commit message
     
      ## odb.h ##
     @@ odb.h: struct object_database {
    + 	struct commit_graph *commit_graph;
      	unsigned commit_graph_attempted : 1; /* if loading has been attempted */
      
    - 	/*
    +-	/*
     -	 * private data
     -	 *
    --	 * should only be accessed directly by packfile.c
    -+	 * Should only be accessed directly by packfile.c
    - 	 */
    +-	 * Should only be accessed directly by packfile.c and midx.c.
    +-	 */
    ++	/* Should only be accessed directly by packfile.c and midx.c. */
      	struct packfile_store *packfiles;
     -	struct {
     -		struct packed_git **packs;
 7:  b49fffa17f =  7:  dd515d94dd packfile: reorder functions to avoid function declaration
 8:  015d0de331 =  8:  7c7d116314 packfile: refactor `prepare_packed_git()` to work on packfile store
 9:  29540e3690 =  9:  b75323ece8 packfile: split up responsibilities of `reprepare_packed_git()`
10:  e3283daf07 = 10:  7863cd6bae packfile: refactor `install_packed_git()` to work on packfile store
11:  257a6f8458 = 11:  f65025d54e packfile: introduce function to load and add packfiles
12:  081bb4d348 = 12:  15e962daea packfile: move `get_multi_pack_index()` into "midx.c"
13:  39f70c5cca = 13:  7ba39efd3a packfile: refactor `get_packed_git()` to work on packfile store
14:  36f5d08128 = 14:  505d32ea35 packfile: refactor `get_all_packs()` to work on packfile store
15:  a558e95cf0 = 15:  c474d336be packfile: refactor `get_packed_git_mru()` to work on packfile store

--- base-commit: 337c7a0bbcf228ce11c87d066ecee352b3e52467 change-id: 20250806-b4-pks-packfiles-store-a44a608ca396

Patrick Steinhardt· Sep 23, 2025, 10:17 UTC · re: Patrick Steinhardt · lore

[PATCH v6 01/15] packfile: introduce a new `struct packfile_store`

Information about an object database's packfiles is currently distributed across two different structures:

  - `struct packed_git` contains the `next` pointer as well as the
    `mru_head`, both of which serve to store the list of packfiles.
  - `struct object_database` contains several fields that relate to the
    packfiles.

So we don't really have a central data structure that tracks our packfiles, and consequently responsibilities aren't always clear cut. A consequence for the upcoming pluggable object databases is that this makes it very hard to move management of packfiles from the object database level down into the object database source.

Introduce a new `struct packfile_store` which is about to become the single source of truth for managing packfiles. Right now this data structure doesn't yet contain anything, but in subsequent patches we will move all data structures that relate to packfiles and that are currently contained in `struct object_database` into this new home.

Note that this is only a first step: most importantly, we won't (yet) move the `struct packed_git::next` pointer around. This will happen in a subsequent patch series though so that `struct packed_git` will really only host information about the specific packfile it represents.

Further note that the new structure still sits at the wrong level at the end of this patch series: as mentioned, it should eventually sit at the level of the object database source, not at the object database level. But introducing the packfile store now already makes it way easier to eventually push down the now-selfcontained data structure by one level.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb.c      |  1 +
 odb.h      |  3 ++-
 packfile.c | 13 +++++++++++++
 packfile.h | 18 ++++++++++++++++++
 4 files changed, 34 insertions(+), 1 deletion(-)
Show changes to 4 files +34 −1

odb.c, odb.h, packfile.c, packfile.h

diff --git a/odb.c b/odb.c
index 75c443fe66..a2289ea97d 100644
--- a/odb.c
+++ b/odb.c
@@ -996,6 +996,7 @@ struct object_database *odb_new(struct repository *repo)
 
 	memset(o, 0, sizeof(*o));
 	o->repo = repo;
+	o->packfiles = packfile_store_new(o);
 	INIT_LIST_HEAD(&o->packed_git_mru);
 	hashmap_init(&o->pack_map, pack_map_entry_cmp, NULL, 0);
 	pthread_mutex_init(&o->replace_mutex, NULL);
diff --git a/odb.h b/odb.h
index 51fe8a5a92..33034eaf2f 100644
--- a/odb.h
+++ b/odb.h
@@ -91,6 +91,7 @@ struct odb_source {
 };
 
 struct packed_git;
+struct packfile_store;
 struct cached_object_entry;
 
 /*
@@ -136,7 +137,7 @@ struct object_database {
 	 *
 	 * should only be accessed directly by packfile.c
 	 */
-
+	struct packfile_store *packfiles;
 	struct packed_git *packed_git;
 	/* A most-recently-used ordered version of the packed_git list. */
 	struct list_head packed_git_mru;
diff --git a/packfile.c b/packfile.c
index acb680966d..130d3e2507 100644
--- a/packfile.c
+++ b/packfile.c
@@ -2332,3 +2332,16 @@ int parse_pack_header_option(const char *in, unsigned char *out, unsigned int *l
 	*len = hdr - out;
 	return 0;
 }
+
+struct packfile_store *packfile_store_new(struct object_database *odb)
+{
+	struct packfile_store *store;
+	CALLOC_ARRAY(store, 1);
+	store->odb = odb;
+	return store;
+}
+
+void packfile_store_free(struct packfile_store *store)
+{
+	free(store);
+}
diff --git a/packfile.h b/packfile.h
index f16753f2a9..8d31fd619a 100644
--- a/packfile.h
+++ b/packfile.h
@@ -52,6 +52,24 @@ struct packed_git {
 	char pack_name[FLEX_ARRAY]; /* more */
 };
 
+/*
+ * A store that manages packfiles for a given object database.
+ */
+struct packfile_store {
+	struct object_database *odb;
+};
+
+/*
+ * Allocate and initialize a new empty packfile store for the given object
+ * database.
+ */
+struct packfile_store *packfile_store_new(struct object_database *odb);
+
+/*
+ * Free the packfile store and all its associated state.
+ */
+void packfile_store_free(struct packfile_store *store);
+
 static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
 				     const struct hashmap_entry *entry,
 				     const struct hashmap_entry *entry2,
-- 
2.51.0.536.g15c5d4f767.dirty
Patrick Steinhardt· Sep 23, 2025, 10:17 UTC · re: Patrick Steinhardt · lore

[PATCH v6 02/15] odb: move list of packfiles into `struct packfile_store`

The object database tracks the list of packfiles it currently knows about. With the introduction of the `struct packfile_store` we have a better place to host this list though.

Move the list accordingly. Extract the logic from `odb_clear()` that knows to close all such packfiles and move it into the new subsystem, as well.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb.c      | 12 ++----------
 odb.h      |  1 -
 packfile.c | 42 +++++++++++++++++++++++++-----------------
 packfile.h | 15 ++++++++++++++-
 4 files changed, 41 insertions(+), 29 deletions(-)
Show changes to 4 files +41 −29

odb.c, odb.h, packfile.c, packfile.h

diff --git a/odb.c b/odb.c
index a2289ea97d..7201d01406 100644
--- a/odb.c
+++ b/odb.c
@@ -1038,16 +1038,8 @@ void odb_clear(struct object_database *o)
 
 	INIT_LIST_HEAD(&o->packed_git_mru);
 	close_object_store(o);
-
-	/*
-	 * `close_object_store()` only closes the packfiles, but doesn't free
-	 * them. We thus have to do this manually.
-	 */
-	for (struct packed_git *p = o->packed_git, *next; p; p = next) {
-		next = p->next;
-		free(p);
-	}
-	o->packed_git = NULL;
+	packfile_store_free(o->packfiles);
+	o->packfiles = NULL;
 
 	hashmap_clear(&o->pack_map);
 	string_list_clear(&o->submodule_source_paths, 0);
diff --git a/odb.h b/odb.h
index 33034eaf2f..22a170b434 100644
--- a/odb.h
+++ b/odb.h
@@ -138,7 +138,6 @@ struct object_database {
 	 * should only be accessed directly by packfile.c
 	 */
 	struct packfile_store *packfiles;
-	struct packed_git *packed_git;
 	/* A most-recently-used ordered version of the packed_git list. */
 	struct list_head packed_git_mru;
 
diff --git a/packfile.c b/packfile.c
index 130d3e2507..36bc240107 100644
--- a/packfile.c
+++ b/packfile.c
@@ -278,7 +278,7 @@ static int unuse_one_window(struct packed_git *current)
 
 	if (current)
 		scan_windows(current, &lru_p, &lru_w, &lru_l);
-	for (p = current->repo->objects->packed_git; p; p = p->next)
+	for (p = current->repo->objects->packfiles->packs; p; p = p->next)
 		scan_windows(p, &lru_p, &lru_w, &lru_l);
 	if (lru_p) {
 		munmap(lru_w->base, lru_w->len);
@@ -362,13 +362,8 @@ void close_pack(struct packed_git *p)
 void close_object_store(struct object_database *o)
 {
 	struct odb_source *source;
-	struct packed_git *p;
 
-	for (p = o->packed_git; p; p = p->next)
-		if (p->do_not_close)
-			BUG("want to close pack marked 'do-not-close'");
-		else
-			close_pack(p);
+	packfile_store_close(o->packfiles);
 
 	for (source = o->sources; source; source = source->next) {
 		if (source->midx)
@@ -468,7 +463,7 @@ static int close_one_pack(struct repository *r)
 	struct pack_window *mru_w = NULL;
 	int accept_windows_inuse = 1;
 
-	for (p = r->objects->packed_git; p; p = p->next) {
+	for (p = r->objects->packfiles->packs; p; p = p->next) {
 		if (p->pack_fd == -1)
 			continue;
 		find_lru_pack(p, &lru_p, &mru_w, &accept_windows_inuse);
@@ -789,8 +784,8 @@ void install_packed_git(struct repository *r, struct packed_git *pack)
 	if (pack->pack_fd != -1)
 		pack_open_fds++;
 
-	pack->next = r->objects->packed_git;
-	r->objects->packed_git = pack;
+	pack->next = r->objects->packfiles->packs;
+	r->objects->packfiles->packs = pack;
 
 	hashmap_entry_init(&pack->packmap_ent, strhash(pack->pack_name));
 	hashmap_add(&r->objects->pack_map, &pack->packmap_ent);
@@ -974,7 +969,7 @@ unsigned long repo_approximate_object_count(struct repository *r)
 				count += m->num_objects;
 		}
 
-		for (p = r->objects->packed_git; p; p = p->next) {
+		for (p = r->objects->packfiles->packs; p; p = p->next) {
 			if (open_pack_index(p))
 				continue;
 			count += p->num_objects;
@@ -1015,7 +1010,7 @@ static int sort_pack(const struct packed_git *a, const struct packed_git *b)
 
 static void rearrange_packed_git(struct repository *r)
 {
-	sort_packs(&r->objects->packed_git, sort_pack);
+	sort_packs(&r->objects->packfiles->packs, sort_pack);
 }
 
 static void prepare_packed_git_mru(struct repository *r)
@@ -1024,7 +1019,7 @@ static void prepare_packed_git_mru(struct repository *r)
 
 	INIT_LIST_HEAD(&r->objects->packed_git_mru);
 
-	for (p = r->objects->packed_git; p; p = p->next)
+	for (p = r->objects->packfiles->packs; p; p = p->next)
 		list_add_tail(&p->mru, &r->objects->packed_git_mru);
 }
 
@@ -1073,7 +1068,7 @@ void reprepare_packed_git(struct repository *r)
 struct packed_git *get_packed_git(struct repository *r)
 {
 	prepare_packed_git(r);
-	return r->objects->packed_git;
+	return r->objects->packfiles->packs;
 }
 
 struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
@@ -1094,7 +1089,7 @@ struct packed_git *get_all_packs(struct repository *r)
 			prepare_midx_pack(m, i);
 	}
 
-	return r->objects->packed_git;
+	return r->objects->packfiles->packs;
 }
 
 struct list_head *get_packed_git_mru(struct repository *r)
@@ -1219,7 +1214,7 @@ const struct packed_git *has_packed_and_bad(struct repository *r,
 {
 	struct packed_git *p;
 
-	for (p = r->objects->packed_git; p; p = p->next)
+	for (p = r->objects->packfiles->packs; p; p = p->next)
 		if (oidset_contains(&p->bad_objects, oid))
 			return p;
 	return NULL;
@@ -2080,7 +2075,7 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 		if (source->midx && fill_midx_entry(source->midx, oid, e))
 			return 1;
 
-	if (!r->objects->packed_git)
+	if (!r->objects->packfiles->packs)
 		return 0;
 
 	list_for_each(pos, &r->objects->packed_git_mru) {
@@ -2343,5 +2338,18 @@ struct packfile_store *packfile_store_new(struct object_database *odb)
 
 void packfile_store_free(struct packfile_store *store)
 {
+	for (struct packed_git *p = store->packs, *next; p; p = next) {
+		next = p->next;
+		free(p);
+	}
 	free(store);
 }
+
+void packfile_store_close(struct packfile_store *store)
+{
+	for (struct packed_git *p = store->packs; p; p = p->next) {
+		if (p->do_not_close)
+			BUG("want to close pack marked 'do-not-close'");
+		close_pack(p);
+	}
+}
diff --git a/packfile.h b/packfile.h
index 8d31fd619a..d7ac8d24b4 100644
--- a/packfile.h
+++ b/packfile.h
@@ -57,6 +57,12 @@ struct packed_git {
  */
 struct packfile_store {
 	struct object_database *odb;
+
+	/*
+	 * The list of packfiles in the order in which they are being added to
+	 * the store.
+	 */
+	struct packed_git *packs;
 };
 
 /*
@@ -66,10 +72,17 @@ struct packfile_store {
 struct packfile_store *packfile_store_new(struct object_database *odb);
 
 /*
- * Free the packfile store and all its associated state.
+ * Free the packfile store and all its associated state. All packfiles
+ * tracked by the store will be closed.
  */
 void packfile_store_free(struct packfile_store *store);
 
+/*
+ * Close all packfiles associated with this store. The packfiles won't be
+ * free'd, so they can be re-opened at a later point in time.
+ */
+void packfile_store_close(struct packfile_store *store);
+
 static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
 				     const struct hashmap_entry *entry,
 				     const struct hashmap_entry *entry2,
-- 
2.51.0.536.g15c5d4f767.dirty
Patrick Steinhardt· Sep 23, 2025, 10:17 UTC · re: Patrick Steinhardt · lore

[PATCH v6 03/15] odb: move initialization bit into `struct packfile_store`

The object database knows to skip re-initializing the list of packfiles in case it's already been initialized. Whether or not that is the case is tracked via a separate `initialized` bit that is stored in the object database. With the introduction of the `struct packfile_store` we have a better place to host this bit though.

Move it accordingly. While at it, convert the field into a boolean now that we're allowed to use them in our code base.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb.h      | 6 ------
 packfile.c | 6 +++---
 packfile.h | 6 ++++++
 3 files changed, 9 insertions(+), 9 deletions(-)
Show changes to 3 files +9 −9

odb.h, packfile.c, packfile.h

diff --git a/odb.h b/odb.h
index 22a170b434..bf1b4d4677 100644
--- a/odb.h
+++ b/odb.h
@@ -169,12 +169,6 @@ struct object_database {
 	unsigned long approximate_object_count;
 	unsigned approximate_object_count_valid : 1;
 
-	/*
-	 * Whether packed_git has already been populated with this repository's
-	 * packs.
-	 */
-	unsigned packed_git_initialized : 1;
-
 	/*
 	 * Submodule source paths that will be added as additional sources to
 	 * allow lookup of submodule objects via the main object database.
diff --git a/packfile.c b/packfile.c
index 36bc240107..f37557eac5 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1027,7 +1027,7 @@ static void prepare_packed_git(struct repository *r)
 {
 	struct odb_source *source;
 
-	if (r->objects->packed_git_initialized)
+	if (r->objects->packfiles->initialized)
 		return;
 
 	odb_prepare_alternates(r->objects);
@@ -1038,7 +1038,7 @@ static void prepare_packed_git(struct repository *r)
 	rearrange_packed_git(r);
 
 	prepare_packed_git_mru(r);
-	r->objects->packed_git_initialized = 1;
+	r->objects->packfiles->initialized = true;
 }
 
 void reprepare_packed_git(struct repository *r)
@@ -1060,7 +1060,7 @@ void reprepare_packed_git(struct repository *r)
 		odb_clear_loose_cache(source);
 
 	r->objects->approximate_object_count_valid = 0;
-	r->objects->packed_git_initialized = 0;
+	r->objects->packfiles->initialized = false;
 	prepare_packed_git(r);
 	obj_read_unlock();
 }
diff --git a/packfile.h b/packfile.h
index d7ac8d24b4..cf81091175 100644
--- a/packfile.h
+++ b/packfile.h
@@ -63,6 +63,12 @@ struct packfile_store {
 	 * the store.
 	 */
 	struct packed_git *packs;
+
+	/*
+	 * Whether packfiles have already been populated with this store's
+	 * packs.
+	 */
+	bool initialized;
 };
 
 /*
-- 
2.51.0.536.g15c5d4f767.dirty
Patrick Steinhardt· Sep 23, 2025, 10:17 UTC · re: Patrick Steinhardt · lore

[PATCH v6 04/15] odb: move packfile map into `struct packfile_store`

The object database tracks a map of packfiles by their respective paths, which is used to figure out whether a given packfile has already been loaded. With the introduction of the `struct packfile_store` we have a better place to host this list though.

Move the map accordingly.

`pack_map_entry_cmp()` isn't used anywhere but in "packfile.c" anymore after this change, so we convert it to a static function, as well. Note that we also drop the `inline` hint: the function is used as a callback function exclusively, and callbacks cannot be inlined.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 midx.c     |  2 +-
 odb.c      |  2 --
 odb.h      |  8 +-------
 packfile.c | 20 ++++++++++++++++++--
 packfile.h | 20 ++++++--------------
 5 files changed, 26 insertions(+), 26 deletions(-)
Show changes to 5 files +26 −26

midx.c, odb.c, odb.h, packfile.c, packfile.h

diff --git a/midx.c b/midx.c
index 7726c13d7e..e96970efbf 100644
--- a/midx.c
+++ b/midx.c
@@ -460,7 +460,7 @@ int prepare_midx_pack(struct multi_pack_index *m,
 	strbuf_addbuf(&key, &pack_name);
 	strbuf_strip_suffix(&key, ".idx");
 	strbuf_addstr(&key, ".pack");
-	p = hashmap_get_entry_from_hash(&r->objects->pack_map,
+	p = hashmap_get_entry_from_hash(&r->objects->packfiles->map,
 					strhash(key.buf), key.buf,
 					struct packed_git, packmap_ent);
 	if (!p) {
diff --git a/odb.c b/odb.c
index 7201d01406..737d98c911 100644
--- a/odb.c
+++ b/odb.c
@@ -998,7 +998,6 @@ struct object_database *odb_new(struct repository *repo)
 	o->repo = repo;
 	o->packfiles = packfile_store_new(o);
 	INIT_LIST_HEAD(&o->packed_git_mru);
-	hashmap_init(&o->pack_map, pack_map_entry_cmp, NULL, 0);
 	pthread_mutex_init(&o->replace_mutex, NULL);
 	string_list_init_dup(&o->submodule_source_paths);
 	return o;
@@ -1041,6 +1040,5 @@ void odb_clear(struct object_database *o)
 	packfile_store_free(o->packfiles);
 	o->packfiles = NULL;
 
-	hashmap_clear(&o->pack_map);
 	string_list_clear(&o->submodule_source_paths, 0);
 }
diff --git a/odb.h b/odb.h
index bf1b4d4677..b79e7280c1 100644
--- a/odb.h
+++ b/odb.h
@@ -135,7 +135,7 @@ struct object_database {
 	/*
 	 * private data
 	 *
-	 * should only be accessed directly by packfile.c
+	 * Should only be accessed directly by packfile.c and midx.c.
 	 */
 	struct packfile_store *packfiles;
 	/* A most-recently-used ordered version of the packed_git list. */
@@ -155,12 +155,6 @@ struct object_database {
 	struct cached_object_entry *cached_objects;
 	size_t cached_object_nr, cached_object_alloc;
 
-	/*
-	 * A map of packfiles to packed_git structs for tracking which
-	 * packs have been loaded already.
-	 */
-	struct hashmap pack_map;
-
 	/*
 	 * A fast, rough count of the number of objects in the repository.
 	 * These two fields are not meant for direct access. Use
diff --git a/packfile.c b/packfile.c
index f37557eac5..17e0b8ab27 100644
--- a/packfile.c
+++ b/packfile.c
@@ -788,7 +788,7 @@ void install_packed_git(struct repository *r, struct packed_git *pack)
 	r->objects->packfiles->packs = pack;
 
 	hashmap_entry_init(&pack->packmap_ent, strhash(pack->pack_name));
-	hashmap_add(&r->objects->pack_map, &pack->packmap_ent);
+	hashmap_add(&r->objects->packfiles->map, &pack->packmap_ent);
 }
 
 void (*report_garbage)(unsigned seen_bits, const char *path);
@@ -901,7 +901,7 @@ static void prepare_pack(const char *full_name, size_t full_name_len,
 		hashmap_entry_init(&hent, hash);
 
 		/* Don't reopen a pack we already have. */
-		if (!hashmap_get(&data->r->objects->pack_map, &hent, pack_name)) {
+		if (!hashmap_get(&data->r->objects->packfiles->map, &hent, pack_name)) {
 			p = add_packed_git(data->r, full_name, full_name_len, data->local);
 			if (p)
 				install_packed_git(data->r, p);
@@ -2328,11 +2328,26 @@ int parse_pack_header_option(const char *in, unsigned char *out, unsigned int *l
 	return 0;
 }
 
+static int pack_map_entry_cmp(const void *cmp_data UNUSED,
+			      const struct hashmap_entry *entry,
+			      const struct hashmap_entry *entry2,
+			      const void *keydata)
+{
+	const char *key = keydata;
+	const struct packed_git *pg1, *pg2;
+
+	pg1 = container_of(entry, const struct packed_git, packmap_ent);
+	pg2 = container_of(entry2, const struct packed_git, packmap_ent);
+
+	return strcmp(pg1->pack_name, key ? key : pg2->pack_name);
+}
+
 struct packfile_store *packfile_store_new(struct object_database *odb)
 {
 	struct packfile_store *store;
 	CALLOC_ARRAY(store, 1);
 	store->odb = odb;
+	hashmap_init(&store->map, pack_map_entry_cmp, NULL, 0);
 	return store;
 }
 
@@ -2342,6 +2357,7 @@ void packfile_store_free(struct packfile_store *store)
 		next = p->next;
 		free(p);
 	}
+	hashmap_clear(&store->map);
 	free(store);
 }
 
diff --git a/packfile.h b/packfile.h
index cf81091175..9bbef51164 100644
--- a/packfile.h
+++ b/packfile.h
@@ -64,6 +64,12 @@ struct packfile_store {
 	 */
 	struct packed_git *packs;
 
+	/*
+	 * A map of packfile names to packed_git structs for tracking which
+	 * packs have been loaded already.
+	 */
+	struct hashmap map;
+
 	/*
 	 * Whether packfiles have already been populated with this store's
 	 * packs.
@@ -89,20 +95,6 @@ void packfile_store_free(struct packfile_store *store);
  */
 void packfile_store_close(struct packfile_store *store);
 
-static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
-				     const struct hashmap_entry *entry,
-				     const struct hashmap_entry *entry2,
-				     const void *keydata)
-{
-	const char *key = keydata;
-	const struct packed_git *pg1, *pg2;
-
-	pg1 = container_of(entry, const struct packed_git, packmap_ent);
-	pg2 = container_of(entry2, const struct packed_git, packmap_ent);
-
-	return strcmp(pg1->pack_name, key ? key : pg2->pack_name);
-}
-
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
-- 
2.51.0.536.g15c5d4f767.dirty
Patrick Steinhardt· Sep 23, 2025, 10:17 UTC · re: Patrick Steinhardt · lore

[PATCH v6 05/15] odb: move MRU list of packfiles into `struct packfile_store`

The object database tracks the list of packfiles in most-recently-used order, which is mostly used to favor reading from packfiles that contain most of the objects that we're currently accessing. With the introduction of the `struct packfile_store` we have a better place to host this list though.

Move the list accordingly.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 midx.c     |  2 +-
 odb.c      |  2 --
 odb.h      |  4 ----
 packfile.c | 11 ++++++-----
 packfile.h |  3 +++
 5 files changed, 10 insertions(+), 12 deletions(-)
Show changes to 5 files +10 −12

midx.c, odb.c, odb.h, packfile.c, packfile.h

diff --git a/midx.c b/midx.c
index e96970efbf..91c7b3917d 100644
--- a/midx.c
+++ b/midx.c
@@ -468,7 +468,7 @@ int prepare_midx_pack(struct multi_pack_index *m,
 				   m->source->local);
 		if (p) {
 			install_packed_git(r, p);
-			list_add_tail(&p->mru, &r->objects->packed_git_mru);
+			list_add_tail(&p->mru, &r->objects->packfiles->mru);
 		}
 	}
 
diff --git a/odb.c b/odb.c
index 737d98c911..32e982bf0b 100644
--- a/odb.c
+++ b/odb.c
@@ -997,7 +997,6 @@ struct object_database *odb_new(struct repository *repo)
 	memset(o, 0, sizeof(*o));
 	o->repo = repo;
 	o->packfiles = packfile_store_new(o);
-	INIT_LIST_HEAD(&o->packed_git_mru);
 	pthread_mutex_init(&o->replace_mutex, NULL);
 	string_list_init_dup(&o->submodule_source_paths);
 	return o;
@@ -1035,7 +1034,6 @@ void odb_clear(struct object_database *o)
 		free((char *) o->cached_objects[i].value.buf);
 	FREE_AND_NULL(o->cached_objects);
 
-	INIT_LIST_HEAD(&o->packed_git_mru);
 	close_object_store(o);
 	packfile_store_free(o->packfiles);
 	o->packfiles = NULL;
diff --git a/odb.h b/odb.h
index b79e7280c1..3044b6a661 100644
--- a/odb.h
+++ b/odb.h
@@ -3,7 +3,6 @@
 
 #include "hashmap.h"
 #include "object.h"
-#include "list.h"
 #include "oidset.h"
 #include "oidmap.h"
 #include "string-list.h"
@@ -138,9 +137,6 @@ struct object_database {
 	 * Should only be accessed directly by packfile.c and midx.c.
 	 */
 	struct packfile_store *packfiles;
-	/* A most-recently-used ordered version of the packed_git list. */
-	struct list_head packed_git_mru;
-
 	struct {
 		struct packed_git **packs;
 		unsigned flags;
diff --git a/packfile.c b/packfile.c
index 17e0b8ab27..861d7ffd6f 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1017,10 +1017,10 @@ static void prepare_packed_git_mru(struct repository *r)
 {
 	struct packed_git *p;
 
-	INIT_LIST_HEAD(&r->objects->packed_git_mru);
+	INIT_LIST_HEAD(&r->objects->packfiles->mru);
 
 	for (p = r->objects->packfiles->packs; p; p = p->next)
-		list_add_tail(&p->mru, &r->objects->packed_git_mru);
+		list_add_tail(&p->mru, &r->objects->packfiles->mru);
 }
 
 static void prepare_packed_git(struct repository *r)
@@ -1095,7 +1095,7 @@ struct packed_git *get_all_packs(struct repository *r)
 struct list_head *get_packed_git_mru(struct repository *r)
 {
 	prepare_packed_git(r);
-	return &r->objects->packed_git_mru;
+	return &r->objects->packfiles->mru;
 }
 
 unsigned long unpack_object_header_buffer(const unsigned char *buf,
@@ -2078,10 +2078,10 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 	if (!r->objects->packfiles->packs)
 		return 0;
 
-	list_for_each(pos, &r->objects->packed_git_mru) {
+	list_for_each(pos, &r->objects->packfiles->mru) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		if (!p->multi_pack_index && fill_pack_entry(oid, e, p)) {
-			list_move(&p->mru, &r->objects->packed_git_mru);
+			list_move(&p->mru, &r->objects->packfiles->mru);
 			return 1;
 		}
 	}
@@ -2347,6 +2347,7 @@ struct packfile_store *packfile_store_new(struct object_database *odb)
 	struct packfile_store *store;
 	CALLOC_ARRAY(store, 1);
 	store->odb = odb;
+	INIT_LIST_HEAD(&store->mru);
 	hashmap_init(&store->map, pack_map_entry_cmp, NULL, 0);
 	return store;
 }
diff --git a/packfile.h b/packfile.h
index 9bbef51164..d48d46cc1b 100644
--- a/packfile.h
+++ b/packfile.h
@@ -64,6 +64,9 @@ struct packfile_store {
 	 */
 	struct packed_git *packs;
 
+	/* A most-recently-used ordered version of the packs list. */
+	struct list_head mru;
+
 	/*
 	 * A map of packfile names to packed_git structs for tracking which
 	 * packs have been loaded already.
-- 
2.51.0.536.g15c5d4f767.dirty
Patrick Steinhardt· Sep 23, 2025, 10:17 UTC · re: Patrick Steinhardt · lore

[PATCH v6 06/15] odb: move kept cache into `struct packfile_store`

The object database tracks a cache of "kept" packfiles, which is used by git-pack-objects(1) to handle cruft objects. With the introduction of the `struct packfile_store` we have a better place to host this cache though.

Move the cache accordingly.

This moves the last bit of packfile-related state from the object database into the packfile store. Adapt the comment for the `packfiles` pointer in `struct object_database` to reflect this.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb.h      | 10 +---------
 packfile.c | 16 ++++++++--------
 packfile.h | 14 ++++++++++++++
 3 files changed, 23 insertions(+), 17 deletions(-)
Show changes to 3 files +23 −17

odb.h, packfile.c, packfile.h

diff --git a/odb.h b/odb.h
index 3044b6a661..9dd7bb6bc3 100644
--- a/odb.h
+++ b/odb.h
@@ -131,16 +131,8 @@ struct object_database {
 	struct commit_graph *commit_graph;
 	unsigned commit_graph_attempted : 1; /* if loading has been attempted */
 
-	/*
-	 * private data
-	 *
-	 * Should only be accessed directly by packfile.c and midx.c.
-	 */
+	/* Should only be accessed directly by packfile.c and midx.c. */
 	struct packfile_store *packfiles;
-	struct {
-		struct packed_git **packs;
-		unsigned flags;
-	} kept_pack_cache;
 
 	/*
 	 * This is meant to hold a *small* number of objects that you would
diff --git a/packfile.c b/packfile.c
index 861d7ffd6f..95a78f267f 100644
--- a/packfile.c
+++ b/packfile.c
@@ -2091,19 +2091,19 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 static void maybe_invalidate_kept_pack_cache(struct repository *r,
 					     unsigned flags)
 {
-	if (!r->objects->kept_pack_cache.packs)
+	if (!r->objects->packfiles->kept_cache.packs)
 		return;
-	if (r->objects->kept_pack_cache.flags == flags)
+	if (r->objects->packfiles->kept_cache.flags == flags)
 		return;
-	FREE_AND_NULL(r->objects->kept_pack_cache.packs);
-	r->objects->kept_pack_cache.flags = 0;
+	FREE_AND_NULL(r->objects->packfiles->kept_cache.packs);
+	r->objects->packfiles->kept_cache.flags = 0;
 }
 
 struct packed_git **kept_pack_cache(struct repository *r, unsigned flags)
 {
 	maybe_invalidate_kept_pack_cache(r, flags);
 
-	if (!r->objects->kept_pack_cache.packs) {
+	if (!r->objects->packfiles->kept_cache.packs) {
 		struct packed_git **packs = NULL;
 		size_t nr = 0, alloc = 0;
 		struct packed_git *p;
@@ -2126,11 +2126,11 @@ struct packed_git **kept_pack_cache(struct repository *r, unsigned flags)
 		ALLOC_GROW(packs, nr + 1, alloc);
 		packs[nr] = NULL;
 
-		r->objects->kept_pack_cache.packs = packs;
-		r->objects->kept_pack_cache.flags = flags;
+		r->objects->packfiles->kept_cache.packs = packs;
+		r->objects->packfiles->kept_cache.flags = flags;
 	}
 
-	return r->objects->kept_pack_cache.packs;
+	return r->objects->packfiles->kept_cache.packs;
 }
 
 int find_kept_pack_entry(struct repository *r,
diff --git a/packfile.h b/packfile.h
index d48d46cc1b..bf66211986 100644
--- a/packfile.h
+++ b/packfile.h
@@ -64,6 +64,20 @@ struct packfile_store {
 	 */
 	struct packed_git *packs;
 
+	/*
+	 * Cache of packfiles which are marked as "kept", either because there
+	 * is an on-disk ".keep" file or because they are marked as "kept" in
+	 * memory.
+	 *
+	 * Should not be accessed directly, but via `kept_pack_cache()`. The
+	 * list of packs gets invalidated when the stored flags and the flags
+	 * passed to `kept_pack_cache()` mismatch.
+	 */
+	struct {
+		struct packed_git **packs;
+		unsigned flags;
+	} kept_cache;
+
 	/* A most-recently-used ordered version of the packs list. */
 	struct list_head mru;
 
-- 
2.51.0.536.g15c5d4f767.dirty
Patrick Steinhardt· Sep 23, 2025, 10:17 UTC · re: Patrick Steinhardt · lore

[PATCH v6 07/15] packfile: reorder functions to avoid function declaration

Reorder functions so that we can avoid a forward declaration of `prepare_packed_git()`.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 packfile.c | 67 +++++++++++++++++++++++++++++++-------------------------------
 1 file changed, 33 insertions(+), 34 deletions(-)
Show changes to packfile.c +33 −34
diff --git a/packfile.c b/packfile.c
index 95a78f267f..5588a7ad6d 100644
--- a/packfile.c
+++ b/packfile.c
@@ -946,40 +946,6 @@ static void prepare_packed_git_one(struct odb_source *source)
 	string_list_clear(data.garbage, 0);
 }
 
-static void prepare_packed_git(struct repository *r);
-/*
- * Give a fast, rough count of the number of objects in the repository. This
- * ignores loose objects completely. If you have a lot of them, then either
- * you should repack because your performance will be awful, or they are
- * all unreachable objects about to be pruned, in which case they're not really
- * interesting as a measure of repo size in the first place.
- */
-unsigned long repo_approximate_object_count(struct repository *r)
-{
-	if (!r->objects->approximate_object_count_valid) {
-		struct odb_source *source;
-		unsigned long count = 0;
-		struct packed_git *p;
-
-		prepare_packed_git(r);
-
-		for (source = r->objects->sources; source; source = source->next) {
-			struct multi_pack_index *m = get_multi_pack_index(source);
-			if (m)
-				count += m->num_objects;
-		}
-
-		for (p = r->objects->packfiles->packs; p; p = p->next) {
-			if (open_pack_index(p))
-				continue;
-			count += p->num_objects;
-		}
-		r->objects->approximate_object_count = count;
-		r->objects->approximate_object_count_valid = 1;
-	}
-	return r->objects->approximate_object_count;
-}
-
 DEFINE_LIST_SORT(static, sort_packs, struct packed_git, next);
 
 static int sort_pack(const struct packed_git *a, const struct packed_git *b)
@@ -1098,6 +1064,39 @@ struct list_head *get_packed_git_mru(struct repository *r)
 	return &r->objects->packfiles->mru;
 }
 
+/*
+ * Give a fast, rough count of the number of objects in the repository. This
+ * ignores loose objects completely. If you have a lot of them, then either
+ * you should repack because your performance will be awful, or they are
+ * all unreachable objects about to be pruned, in which case they're not really
+ * interesting as a measure of repo size in the first place.
+ */
+unsigned long repo_approximate_object_count(struct repository *r)
+{
+	if (!r->objects->approximate_object_count_valid) {
+		struct odb_source *source;
+		unsigned long count = 0;
+		struct packed_git *p;
+
+		prepare_packed_git(r);
+
+		for (source = r->objects->sources; source; source = source->next) {
+			struct multi_pack_index *m = get_multi_pack_index(source);
+			if (m)
+				count += m->num_objects;
+		}
+
+		for (p = r->objects->packfiles->packs; p; p = p->next) {
+			if (open_pack_index(p))
+				continue;
+			count += p->num_objects;
+		}
+		r->objects->approximate_object_count = count;
+		r->objects->approximate_object_count_valid = 1;
+	}
+	return r->objects->approximate_object_count;
+}
+
 unsigned long unpack_object_header_buffer(const unsigned char *buf,
 		unsigned long len, enum object_type *type, unsigned long *sizep)
 {
-- 
2.51.0.536.g15c5d4f767.dirty
Patrick Steinhardt· Sep 23, 2025, 10:17 UTC · re: Patrick Steinhardt · lore

[PATCH v6 08/15] packfile: refactor `prepare_packed_git()` to work on packfile store

The `prepare_packed_git()` function and its friends are responsible for loading packfiles as well as the multi-pack index for a given object database. Refactor these functions to accept a packfile store instead of a repository to clarify their scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 packfile.c | 41 ++++++++++++++++++-----------------------
 1 file changed, 18 insertions(+), 23 deletions(-)
Show changes to packfile.c +18 −23
diff --git a/packfile.c b/packfile.c
index 5588a7ad6d..095c85919b 100644
--- a/packfile.c
+++ b/packfile.c
@@ -974,37 +974,32 @@ static int sort_pack(const struct packed_git *a, const struct packed_git *b)
 	return -1;
 }
 
-static void rearrange_packed_git(struct repository *r)
-{
-	sort_packs(&r->objects->packfiles->packs, sort_pack);
-}
-
-static void prepare_packed_git_mru(struct repository *r)
+static void packfile_store_prepare_mru(struct packfile_store *store)
 {
 	struct packed_git *p;
 
-	INIT_LIST_HEAD(&r->objects->packfiles->mru);
+	INIT_LIST_HEAD(&store->mru);
 
-	for (p = r->objects->packfiles->packs; p; p = p->next)
-		list_add_tail(&p->mru, &r->objects->packfiles->mru);
+	for (p = store->packs; p; p = p->next)
+		list_add_tail(&p->mru, &store->mru);
 }
 
-static void prepare_packed_git(struct repository *r)
+static void packfile_store_prepare(struct packfile_store *store)
 {
 	struct odb_source *source;
 
-	if (r->objects->packfiles->initialized)
+	if (store->initialized)
 		return;
 
-	odb_prepare_alternates(r->objects);
-	for (source = r->objects->sources; source; source = source->next) {
+	odb_prepare_alternates(store->odb);
+	for (source = store->odb->sources; source; source = source->next) {
 		prepare_multi_pack_index_one(source);
 		prepare_packed_git_one(source);
 	}
-	rearrange_packed_git(r);
+	sort_packs(&store->packs, sort_pack);
 
-	prepare_packed_git_mru(r);
-	r->objects->packfiles->initialized = true;
+	packfile_store_prepare_mru(store);
+	store->initialized = true;
 }
 
 void reprepare_packed_git(struct repository *r)
@@ -1027,25 +1022,25 @@ void reprepare_packed_git(struct repository *r)
 
 	r->objects->approximate_object_count_valid = 0;
 	r->objects->packfiles->initialized = false;
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 	obj_read_unlock();
 }
 
 struct packed_git *get_packed_git(struct repository *r)
 {
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 	return r->objects->packfiles->packs;
 }
 
 struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
 {
-	prepare_packed_git(source->odb->repo);
+	packfile_store_prepare(source->odb->packfiles);
 	return source->midx;
 }
 
 struct packed_git *get_all_packs(struct repository *r)
 {
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 
 	for (struct odb_source *source = r->objects->sources; source; source = source->next) {
 		struct multi_pack_index *m = source->midx;
@@ -1060,7 +1055,7 @@ struct packed_git *get_all_packs(struct repository *r)
 
 struct list_head *get_packed_git_mru(struct repository *r)
 {
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 	return &r->objects->packfiles->mru;
 }
 
@@ -1078,7 +1073,7 @@ unsigned long repo_approximate_object_count(struct repository *r)
 		unsigned long count = 0;
 		struct packed_git *p;
 
-		prepare_packed_git(r);
+		packfile_store_prepare(r->objects->packfiles);
 
 		for (source = r->objects->sources; source; source = source->next) {
 			struct multi_pack_index *m = get_multi_pack_index(source);
@@ -2068,7 +2063,7 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa
 {
 	struct list_head *pos;
 
-	prepare_packed_git(r);
+	packfile_store_prepare(r->objects->packfiles);
 
 	for (struct odb_source *source = r->objects->sources; source; source = source->next)
 		if (source->midx && fill_midx_entry(source->midx, oid, e))
-- 
2.51.0.536.g15c5d4f767.dirty
Patrick Steinhardt· Sep 23, 2025, 10:17 UTC · re: Patrick Steinhardt · lore

[PATCH v6 09/15] packfile: split up responsibilities of `reprepare_packed_git()`

In `reprepare_packed_git()` we perform a couple of operations:
  - We reload alternate object directories.
  - We clear the loose object cache.
  - We reprepare packfiles.

While the logic is hosted in "packfile.c", it clearly reaches into other subsystems that aren't related to packfiles.

Split up the responsibility and introduce `odb_reprepare()` which now becomes responsible for repreparing the whole object database. The existing `reprepare_packed_git()` function is refactored accordingly and only cares about reloading the packfile store now.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/backfill.c     |  2 +-
 builtin/gc.c           |  4 ++--
 builtin/receive-pack.c |  2 +-
 builtin/repack.c       |  2 +-
 bulk-checkin.c         |  2 +-
 connected.c            |  2 +-
 fetch-pack.c           |  4 ++--
 object-name.c          |  2 +-
 odb.c                  | 27 ++++++++++++++++++++++++++-
 odb.h                  |  6 ++++++
 packfile.c             | 26 ++++----------------------
 packfile.h             |  9 ++++++++-
 transport-helper.c     |  2 +-
 13 files changed, 55 insertions(+), 35 deletions(-)
Show changes to 13 files +55 −35

builtin/backfill.c, builtin/gc.c, builtin/receive-pack.c, builtin/repack.c, bulk-checkin.c, connected.c, fetch-pack.c, object-name.c, odb.c, odb.h, packfile.c, packfile.h, transport-helper.c

diff --git a/builtin/backfill.c b/builtin/backfill.c
index 80056abe47..e80fc1b694 100644
--- a/builtin/backfill.c
+++ b/builtin/backfill.c
@@ -53,7 +53,7 @@ static void download_batch(struct backfill_context *ctx)
 	 * We likely have a new packfile. Add it to the packed list to
 	 * avoid possible duplicate downloads of the same objects.
 	 */
-	reprepare_packed_git(ctx->repo);
+	odb_reprepare(ctx->repo->objects);
 }
 
 static int fill_missing_blobs(const char *path UNUSED,
diff --git a/builtin/gc.c b/builtin/gc.c
index 03ae4926b2..aeca06a08b 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1042,7 +1042,7 @@ int cmd_gc(int argc,
 		die(FAILED_RUN, "rerere");
 
 	report_garbage = report_pack_garbage;
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 	if (pack_garbage.nr > 0) {
 		close_object_store(the_repository->objects);
 		clean_pack_garbage();
@@ -1491,7 +1491,7 @@ static off_t get_auto_pack_size(void)
 	struct packed_git *p;
 	struct repository *r = the_repository;
 
-	reprepare_packed_git(r);
+	odb_reprepare(r->objects);
 	for (p = get_all_packs(r); p; p = p->next) {
 		if (p->pack_size > max_size) {
 			second_largest_size = max_size;
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 1113137a6f..c9288a9c7e 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -2389,7 +2389,7 @@ static const char *unpack(int err_fd, struct shallow_info *si)
 		status = finish_command(&child);
 		if (status)
 			return "index-pack abnormal exit";
-		reprepare_packed_git(the_repository);
+		odb_reprepare(the_repository->objects);
 	}
 	return NULL;
 }
diff --git a/builtin/repack.c b/builtin/repack.c
index c490a51e91..5ff27fc8e2 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -1685,7 +1685,7 @@ int cmd_repack(int argc,
 			goto cleanup;
 	}
 
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 
 	if (delete_redundant) {
 		int opts = 0;
diff --git a/bulk-checkin.c b/bulk-checkin.c
index b2809ab039..f65439a748 100644
--- a/bulk-checkin.c
+++ b/bulk-checkin.c
@@ -90,7 +90,7 @@ static void flush_bulk_checkin_packfile(struct bulk_checkin_packfile *state)
 
 	strbuf_release(&packname);
 	/* Make objects we just wrote available to ourselves */
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 }
 
 /*
diff --git a/connected.c b/connected.c
index 18c13245d8..d6e9682fd9 100644
--- a/connected.c
+++ b/connected.c
@@ -72,7 +72,7 @@ int check_connected(oid_iterate_fn fn, void *cb_data,
 		 * Before checking for promisor packs, be sure we have the
 		 * latest pack-files loaded into memory.
 		 */
-		reprepare_packed_git(the_repository);
+		odb_reprepare(the_repository->objects);
 		do {
 			struct packed_git *p;
 
diff --git a/fetch-pack.c b/fetch-pack.c
index 6ed5662951..fe7a84bf2f 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -1983,7 +1983,7 @@ static void update_shallow(struct fetch_pack_args *args,
 		 * remote is shallow, but this is a clone, there are
 		 * no objects in repo to worry about. Accept any
 		 * shallow points that exist in the pack (iow in repo
-		 * after get_pack() and reprepare_packed_git())
+		 * after get_pack() and odb_reprepare())
 		 */
 		struct oid_array extra = OID_ARRAY_INIT;
 		struct object_id *oid = si->shallow->oid;
@@ -2108,7 +2108,7 @@ struct ref *fetch_pack(struct fetch_pack_args *args,
 		ref_cpy = do_fetch_pack(args, fd, ref, sought, nr_sought,
 					&si, pack_lockfiles);
 	}
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 
 	if (!args->cloning && args->deepen) {
 		struct check_connected_options opt = CHECK_CONNECTED_INIT;
diff --git a/object-name.c b/object-name.c
index 732056ff5e..df9e0c5f02 100644
--- a/object-name.c
+++ b/object-name.c
@@ -596,7 +596,7 @@ static enum get_oid_result get_short_oid(struct repository *r,
 	 * or migrated from loose to packed.
 	 */
 	if (status == MISSING_OBJECT) {
-		reprepare_packed_git(r);
+		odb_reprepare(r->objects);
 		find_short_object_filename(&ds);
 		find_short_packed_object(&ds);
 		status = finish_object_disambiguation(&ds, oid);
diff --git a/odb.c b/odb.c
index 32e982bf0b..65a6cc67b6 100644
--- a/odb.c
+++ b/odb.c
@@ -694,7 +694,7 @@ static int do_oid_object_info_extended(struct object_database *odb,
 
 		/* Not a loose object; someone else may have just packed it. */
 		if (!(flags & OBJECT_INFO_QUICK)) {
-			reprepare_packed_git(odb->repo);
+			odb_reprepare(odb->repo->objects);
 			if (find_pack_entry(odb->repo, real, &e))
 				break;
 		}
@@ -1040,3 +1040,28 @@ void odb_clear(struct object_database *o)
 
 	string_list_clear(&o->submodule_source_paths, 0);
 }
+
+void odb_reprepare(struct object_database *o)
+{
+	struct odb_source *source;
+
+	obj_read_lock();
+
+	/*
+	 * Reprepare alt odbs, in case the alternates file was modified
+	 * during the course of this process. This only _adds_ odbs to
+	 * the linked list, so existing odbs will continue to exist for
+	 * the lifetime of the process.
+	 */
+	o->loaded_alternates = 0;
+	odb_prepare_alternates(o);
+
+	for (source = o->sources; source; source = source->next)
+		odb_clear_loose_cache(source);
+
+	o->approximate_object_count_valid = 0;
+
+	packfile_store_reprepare(o->packfiles);
+
+	obj_read_unlock();
+}
diff --git a/odb.h b/odb.h
index 9dd7bb6bc3..ab39e3605d 100644
--- a/odb.h
+++ b/odb.h
@@ -161,6 +161,12 @@ struct object_database {
 struct object_database *odb_new(struct repository *repo);
 void odb_clear(struct object_database *o);
 
+/*
+ * Clear caches, reload alternates and then reload object sources so that new
+ * objects may become accessible.
+ */
+void odb_reprepare(struct object_database *o);
+
 /*
  * Find source by its object directory path. Returns a `NULL` pointer in case
  * the source could not be found.
diff --git a/packfile.c b/packfile.c
index 095c85919b..950b98aac5 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1002,28 +1002,10 @@ static void packfile_store_prepare(struct packfile_store *store)
 	store->initialized = true;
 }
 
-void reprepare_packed_git(struct repository *r)
+void packfile_store_reprepare(struct packfile_store *store)
 {
-	struct odb_source *source;
-
-	obj_read_lock();
-
-	/*
-	 * Reprepare alt odbs, in case the alternates file was modified
-	 * during the course of this process. This only _adds_ odbs to
-	 * the linked list, so existing odbs will continue to exist for
-	 * the lifetime of the process.
-	 */
-	r->objects->loaded_alternates = 0;
-	odb_prepare_alternates(r->objects);
-
-	for (source = r->objects->sources; source; source = source->next)
-		odb_clear_loose_cache(source);
-
-	r->objects->approximate_object_count_valid = 0;
-	r->objects->packfiles->initialized = false;
-	packfile_store_prepare(r->objects->packfiles);
-	obj_read_unlock();
+	store->initialized = false;
+	packfile_store_prepare(store);
 }
 
 struct packed_git *get_packed_git(struct repository *r)
@@ -1144,7 +1126,7 @@ unsigned long get_size_from_delta(struct packed_git *p,
 		 *
 		 * Other worrying sections could be the call to close_pack_fd(),
 		 * which can close packs even with in-use windows, and to
-		 * reprepare_packed_git(). Regarding the former, mmap doc says:
+		 * odb_reprepare(). Regarding the former, mmap doc says:
 		 * "closing the file descriptor does not unmap the region". And
 		 * for the latter, it won't re-open already available packs.
 		 */
diff --git a/packfile.h b/packfile.h
index bf66211986..a85ff607fe 100644
--- a/packfile.h
+++ b/packfile.h
@@ -112,6 +112,14 @@ void packfile_store_free(struct packfile_store *store);
  */
 void packfile_store_close(struct packfile_store *store);
 
+/*
+ * Clear the packfile caches and try to look up any new packfiles that have
+ * appeared since last preparing the packfiles store.
+ *
+ * This function must be called under the `odb_read_lock()`.
+ */
+void packfile_store_reprepare(struct packfile_store *store);
+
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
@@ -188,7 +196,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-void reprepare_packed_git(struct repository *r);
 void install_packed_git(struct repository *r, struct packed_git *pack);
 
 struct packed_git *get_packed_git(struct repository *r);
diff --git a/transport-helper.c b/transport-helper.c
index 0789e5bca5..4d95d84f9e 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -450,7 +450,7 @@ static int fetch_with_fetch(struct transport *transport,
 	}
 	strbuf_release(&buf);
 
-	reprepare_packed_git(the_repository);
+	odb_reprepare(the_repository->objects);
 	return 0;
 }
 
-- 
2.51.0.536.g15c5d4f767.dirty
Patrick Steinhardt· Sep 23, 2025, 10:17 UTC · re: Patrick Steinhardt · lore

[PATCH v6 10/15] packfile: refactor `install_packed_git()` to work on packfile store

The `install_packed_git()` functions adds a packfile to a specific object store. Refactor it to accept a packfile store instead of a repository to clarify its scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/fast-import.c |  2 +-
 builtin/index-pack.c  |  2 +-
 http.c                |  2 +-
 http.h                |  2 +-
 midx.c                |  2 +-
 packfile.c            | 11 ++++++-----
 packfile.h            |  9 +++++++--
 7 files changed, 18 insertions(+), 12 deletions(-)
Show changes to 7 files +18 −12

builtin/fast-import.c, builtin/index-pack.c, http.c, http.h, midx.c, packfile.c, packfile.h

diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index 2c35f9345d..e9d82b31c3 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -901,7 +901,7 @@ static void end_packfile(void)
 		if (!new_p)
 			die("core git rejected index %s", idx_name);
 		all_packs[pack_id] = new_p;
-		install_packed_git(the_repository, new_p);
+		packfile_store_add_pack(the_repository->objects->packfiles, new_p);
 		free(idx_name);
 
 		/* Print the boundary */
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index f91c301bba..ed490dfad4 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1645,7 +1645,7 @@ static void final(const char *final_pack_name, const char *curr_pack_name,
 		p = add_packed_git(the_repository, final_index_name,
 				   strlen(final_index_name), 0);
 		if (p)
-			install_packed_git(the_repository, p);
+			packfile_store_add_pack(the_repository->objects->packfiles, p);
 	}
 
 	if (!from_stdin) {
diff --git a/http.c b/http.c
index 98853d6483..af2120b64c 100644
--- a/http.c
+++ b/http.c
@@ -2541,7 +2541,7 @@ void http_install_packfile(struct packed_git *p,
 		lst = &((*lst)->next);
 	*lst = (*lst)->next;
 
-	install_packed_git(the_repository, p);
+	packfile_store_add_pack(the_repository->objects->packfiles, p);
 }
 
 struct http_pack_request *new_http_pack_request(
diff --git a/http.h b/http.h
index 36202139f4..e5a5380c6c 100644
--- a/http.h
+++ b/http.h
@@ -210,7 +210,7 @@ int finish_http_pack_request(struct http_pack_request *preq);
 void release_http_pack_request(struct http_pack_request *preq);
 
 /*
- * Remove p from the given list, and invoke install_packed_git() on it.
+ * Remove p from the given list, and invoke packfile_store_add_pack() on it.
  *
  * This is a convenience function for users that have obtained a list of packs
  * from http_get_info_packs() and have chosen a specific pack to fetch.
diff --git a/midx.c b/midx.c
index 91c7b3917d..69c44be71c 100644
--- a/midx.c
+++ b/midx.c
@@ -467,7 +467,7 @@ int prepare_midx_pack(struct multi_pack_index *m,
 		p = add_packed_git(r, pack_name.buf, pack_name.len,
 				   m->source->local);
 		if (p) {
-			install_packed_git(r, p);
+			packfile_store_add_pack(r->objects->packfiles, p);
 			list_add_tail(&p->mru, &r->objects->packfiles->mru);
 		}
 	}
diff --git a/packfile.c b/packfile.c
index 950b98aac5..af806aba09 100644
--- a/packfile.c
+++ b/packfile.c
@@ -779,16 +779,17 @@ struct packed_git *add_packed_git(struct repository *r, const char *path,
 	return p;
 }
 
-void install_packed_git(struct repository *r, struct packed_git *pack)
+void packfile_store_add_pack(struct packfile_store *store,
+			     struct packed_git *pack)
 {
 	if (pack->pack_fd != -1)
 		pack_open_fds++;
 
-	pack->next = r->objects->packfiles->packs;
-	r->objects->packfiles->packs = pack;
+	pack->next = store->packs;
+	store->packs = pack;
 
 	hashmap_entry_init(&pack->packmap_ent, strhash(pack->pack_name));
-	hashmap_add(&r->objects->packfiles->map, &pack->packmap_ent);
+	hashmap_add(&store->map, &pack->packmap_ent);
 }
 
 void (*report_garbage)(unsigned seen_bits, const char *path);
@@ -904,7 +905,7 @@ static void prepare_pack(const char *full_name, size_t full_name_len,
 		if (!hashmap_get(&data->r->objects->packfiles->map, &hent, pack_name)) {
 			p = add_packed_git(data->r, full_name, full_name_len, data->local);
 			if (p)
-				install_packed_git(data->r, p);
+				packfile_store_add_pack(data->r->objects->packfiles, p);
 		}
 		free(pack_name);
 	}
diff --git a/packfile.h b/packfile.h
index a85ff607fe..ba4b0cef9c 100644
--- a/packfile.h
+++ b/packfile.h
@@ -120,6 +120,13 @@ void packfile_store_close(struct packfile_store *store);
  */
 void packfile_store_reprepare(struct packfile_store *store);
 
+/*
+ * Add the pack to the store so that contained objects become accessible via
+ * the store. This moves ownership into the store.
+ */
+void packfile_store_add_pack(struct packfile_store *store,
+			     struct packed_git *pack);
+
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
@@ -196,8 +203,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-void install_packed_git(struct repository *r, struct packed_git *pack);
-
 struct packed_git *get_packed_git(struct repository *r);
 struct list_head *get_packed_git_mru(struct repository *r);
 struct multi_pack_index *get_multi_pack_index(struct odb_source *source);
-- 
2.51.0.536.g15c5d4f767.dirty
Patrick Steinhardt· Sep 23, 2025, 10:17 UTC · re: Patrick Steinhardt · lore

[PATCH v6 11/15] packfile: introduce function to load and add packfiles

We have a recurring pattern where we essentially perform an upsert of a packfile in case it isn't yet known by the packfile store. The logic to do so is non-trivial as we have to reconstruct the packfile's key, check the map of packfiles, then create the new packfile and finally add it to the store.

Introduce a new function that does this dance for us. Refactor callsites to use it.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/fast-import.c |  4 ++--
 builtin/index-pack.c  | 10 +++-------
 midx.c                | 23 ++++-------------------
 packfile.c            | 44 +++++++++++++++++++++++++++++++-------------
 packfile.h            |  8 ++++++++
 5 files changed, 48 insertions(+), 41 deletions(-)
Show changes to 5 files +48 −41

builtin/fast-import.c, builtin/index-pack.c, midx.c, packfile.c, packfile.h

diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index e9d82b31c3..a26e79689d 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -897,11 +897,11 @@ static void end_packfile(void)
 		idx_name = keep_pack(create_index());
 
 		/* Register the packfile with core git's machinery. */
-		new_p = add_packed_git(pack_data->repo, idx_name, strlen(idx_name), 1);
+		new_p = packfile_store_load_pack(pack_data->repo->objects->packfiles,
+						 idx_name, 1);
 		if (!new_p)
 			die("core git rejected index %s", idx_name);
 		all_packs[pack_id] = new_p;
-		packfile_store_add_pack(the_repository->objects->packfiles, new_p);
 		free(idx_name);
 
 		/* Print the boundary */
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index ed490dfad4..2b78ba7fe4 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1640,13 +1640,9 @@ static void final(const char *final_pack_name, const char *curr_pack_name,
 	rename_tmp_packfile(&final_index_name, curr_index_name, &index_name,
 			    hash, "idx", 1);
 
-	if (do_fsck_object) {
-		struct packed_git *p;
-		p = add_packed_git(the_repository, final_index_name,
-				   strlen(final_index_name), 0);
-		if (p)
-			packfile_store_add_pack(the_repository->objects->packfiles, p);
-	}
+	if (do_fsck_object)
+		packfile_store_load_pack(the_repository->objects->packfiles,
+					 final_index_name, 0);
 
 	if (!from_stdin) {
 		printf("%s\n", hash_to_hex(hash));
diff --git a/midx.c b/midx.c
index 69c44be71c..3faeaf2f8f 100644
--- a/midx.c
+++ b/midx.c
@@ -443,7 +443,6 @@ int prepare_midx_pack(struct multi_pack_index *m,
 {
 	struct repository *r = m->source->odb->repo;
 	struct strbuf pack_name = STRBUF_INIT;
-	struct strbuf key = STRBUF_INIT;
 	struct packed_git *p;
 
 	pack_int_id = midx_for_pack(&m, pack_int_id);
@@ -455,25 +454,11 @@ int prepare_midx_pack(struct multi_pack_index *m,
 
 	strbuf_addf(&pack_name, "%s/pack/%s", m->source->path,
 		    m->pack_names[pack_int_id]);
-
-	/* pack_map holds the ".pack" name, but we have the .idx */
-	strbuf_addbuf(&key, &pack_name);
-	strbuf_strip_suffix(&key, ".idx");
-	strbuf_addstr(&key, ".pack");
-	p = hashmap_get_entry_from_hash(&r->objects->packfiles->map,
-					strhash(key.buf), key.buf,
-					struct packed_git, packmap_ent);
-	if (!p) {
-		p = add_packed_git(r, pack_name.buf, pack_name.len,
-				   m->source->local);
-		if (p) {
-			packfile_store_add_pack(r->objects->packfiles, p);
-			list_add_tail(&p->mru, &r->objects->packfiles->mru);
-		}
-	}
-
+	p = packfile_store_load_pack(r->objects->packfiles,
+				     pack_name.buf, m->source->local);
+	if (p)
+		list_add_tail(&p->mru, &r->objects->packfiles->mru);
 	strbuf_release(&pack_name);
-	strbuf_release(&key);
 
 	if (!p) {
 		m->packs[pack_int_id] = MIDX_PACK_ERROR;
diff --git a/packfile.c b/packfile.c
index af806aba09..9224ca424c 100644
--- a/packfile.c
+++ b/packfile.c
@@ -792,6 +792,33 @@ void packfile_store_add_pack(struct packfile_store *store,
 	hashmap_add(&store->map, &pack->packmap_ent);
 }
 
+struct packed_git *packfile_store_load_pack(struct packfile_store *store,
+					    const char *idx_path, int local)
+{
+	struct strbuf key = STRBUF_INIT;
+	struct packed_git *p;
+
+	/*
+	 * We're being called with the path to the index file, but `pack_map`
+	 * holds the path to the packfile itself.
+	 */
+	strbuf_addstr(&key, idx_path);
+	strbuf_strip_suffix(&key, ".idx");
+	strbuf_addstr(&key, ".pack");
+
+	p = hashmap_get_entry_from_hash(&store->map, strhash(key.buf), key.buf,
+					struct packed_git, packmap_ent);
+	if (!p) {
+		p = add_packed_git(store->odb->repo, idx_path,
+				   strlen(idx_path), local);
+		if (p)
+			packfile_store_add_pack(store, p);
+	}
+
+	strbuf_release(&key);
+	return p;
+}
+
 void (*report_garbage)(unsigned seen_bits, const char *path);
 
 static void report_helper(const struct string_list *list,
@@ -891,23 +918,14 @@ static void prepare_pack(const char *full_name, size_t full_name_len,
 			 const char *file_name, void *_data)
 {
 	struct prepare_pack_data *data = (struct prepare_pack_data *)_data;
-	struct packed_git *p;
 	size_t base_len = full_name_len;
 
 	if (strip_suffix_mem(full_name, &base_len, ".idx") &&
 	    !(data->m && midx_contains_pack(data->m, file_name))) {
-		struct hashmap_entry hent;
-		char *pack_name = xstrfmt("%.*s.pack", (int)base_len, full_name);
-		unsigned int hash = strhash(pack_name);
-		hashmap_entry_init(&hent, hash);
-
-		/* Don't reopen a pack we already have. */
-		if (!hashmap_get(&data->r->objects->packfiles->map, &hent, pack_name)) {
-			p = add_packed_git(data->r, full_name, full_name_len, data->local);
-			if (p)
-				packfile_store_add_pack(data->r->objects->packfiles, p);
-		}
-		free(pack_name);
+		char *trimmed_path = xstrndup(full_name, full_name_len);
+		packfile_store_load_pack(data->r->objects->packfiles,
+					 trimmed_path, data->local);
+		free(trimmed_path);
 	}
 
 	if (!report_garbage)
diff --git a/packfile.h b/packfile.h
index ba4b0cef9c..fcefcbbef6 100644
--- a/packfile.h
+++ b/packfile.h
@@ -127,6 +127,14 @@ void packfile_store_reprepare(struct packfile_store *store);
 void packfile_store_add_pack(struct packfile_store *store,
 			     struct packed_git *pack);
 
+/*
+ * Open the packfile and add it to the store if it isn't yet known. Returns
+ * either the newly opened packfile or the preexisting packfile. Returns a
+ * `NULL` pointer in case the packfile could not be opened.
+ */
+struct packed_git *packfile_store_load_pack(struct packfile_store *store,
+					    const char *idx_path, int local);
+
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
-- 
2.51.0.536.g15c5d4f767.dirty
Patrick Steinhardt· Sep 23, 2025, 10:17 UTC · re: Patrick Steinhardt · lore

[PATCH v6 12/15] packfile: move `get_multi_pack_index()` into "midx.c"

The `get_multi_pack_index()` function is declared and implemented in the packfile subsystem, even though it really belongs into the multi-pack index subsystem. The reason for this is likely that it needs to call `packfile_store_prepare()`, which is not exposed by the packfile system. In a subsequent commit we're about to add another caller outside of the packfile system though, so we'll have to expose the function anyway.

Do so now already and move `get_multi_pack_index()` into the MIDX subsystem.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 midx.c     |  6 ++++++
 midx.h     |  1 +
 packfile.c |  8 +-------
 packfile.h | 10 +++++++++-
 4 files changed, 17 insertions(+), 8 deletions(-)
Show changes to 4 files +17 −8

midx.c, midx.h, packfile.c, packfile.h

diff --git a/midx.c b/midx.c
index 3faeaf2f8f..1d6269f957 100644
--- a/midx.c
+++ b/midx.c
@@ -93,6 +93,12 @@ static int midx_read_object_offsets(const unsigned char *chunk_start,
 	return 0;
 }
 
+struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
+{
+	packfile_store_prepare(source->odb->packfiles);
+	return source->midx;
+}
+
 static struct multi_pack_index *load_multi_pack_index_one(struct odb_source *source,
 							  const char *midx_name)
 {
diff --git a/midx.h b/midx.h
index e241d2d690..6e54d73503 100644
--- a/midx.h
+++ b/midx.h
@@ -94,6 +94,7 @@ void get_midx_chain_filename(struct odb_source *source, struct strbuf *out);
 void get_split_midx_filename_ext(struct odb_source *source, struct strbuf *buf,
 				 const unsigned char *hash, const char *ext);
 
+struct multi_pack_index *get_multi_pack_index(struct odb_source *source);
 struct multi_pack_index *load_multi_pack_index(struct odb_source *source);
 int prepare_midx_pack(struct multi_pack_index *m, uint32_t pack_int_id);
 struct packed_git *nth_midxed_pack(struct multi_pack_index *m,
diff --git a/packfile.c b/packfile.c
index 9224ca424c..7a9193e5ef 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1003,7 +1003,7 @@ static void packfile_store_prepare_mru(struct packfile_store *store)
 		list_add_tail(&p->mru, &store->mru);
 }
 
-static void packfile_store_prepare(struct packfile_store *store)
+void packfile_store_prepare(struct packfile_store *store)
 {
 	struct odb_source *source;
 
@@ -1033,12 +1033,6 @@ struct packed_git *get_packed_git(struct repository *r)
 	return r->objects->packfiles->packs;
 }
 
-struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
-{
-	packfile_store_prepare(source->odb->packfiles);
-	return source->midx;
-}
-
 struct packed_git *get_all_packs(struct repository *r)
 {
 	packfile_store_prepare(r->objects->packfiles);
diff --git a/packfile.h b/packfile.h
index fcefcbbef6..a9e561ac39 100644
--- a/packfile.h
+++ b/packfile.h
@@ -112,6 +112,15 @@ void packfile_store_free(struct packfile_store *store);
  */
 void packfile_store_close(struct packfile_store *store);
 
+/*
+ * Prepare the packfile store by loading packfiles and multi-pack indices for
+ * all alternates. This becomes a no-op if the store is already prepared.
+ *
+ * It shouldn't typically be necessary to call this function directly, as
+ * functions that access the store know to prepare it.
+ */
+void packfile_store_prepare(struct packfile_store *store);
+
 /*
  * Clear the packfile caches and try to look up any new packfiles that have
  * appeared since last preparing the packfiles store.
@@ -213,7 +222,6 @@ extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
 struct packed_git *get_packed_git(struct repository *r);
 struct list_head *get_packed_git_mru(struct repository *r);
-struct multi_pack_index *get_multi_pack_index(struct odb_source *source);
 struct packed_git *get_all_packs(struct repository *r);
 
 /*
-- 
2.51.0.536.g15c5d4f767.dirty
Patrick Steinhardt· Sep 23, 2025, 10:17 UTC · re: Patrick Steinhardt · lore

[PATCH v6 13/15] packfile: refactor `get_packed_git()` to work on packfile store

The `get_packed_git()` function prepares the packfile store and then returns its packfiles. Refactor it to accept a packfile store instead of a repository to clarify its scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/gc.c   | 2 +-
 builtin/grep.c | 2 +-
 object-name.c  | 4 ++--
 packfile.c     | 6 +++---
 packfile.h     | 7 ++++++-
 5 files changed, 13 insertions(+), 8 deletions(-)
Show changes to 5 files +13 −8

builtin/gc.c, builtin/grep.c, object-name.c, packfile.c, packfile.h

diff --git a/builtin/gc.c b/builtin/gc.c
index aeca06a08b..ec6735a540 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1423,7 +1423,7 @@ static int incremental_repack_auto_condition(struct gc_config *cfg UNUSED)
 	if (incremental_repack_auto_limit < 0)
 		return 1;
 
-	for (p = get_packed_git(the_repository);
+	for (p = packfile_store_get_packs(the_repository->objects->packfiles);
 	     count < incremental_repack_auto_limit && p;
 	     p = p->next) {
 		if (!p->multi_pack_index)
diff --git a/builtin/grep.c b/builtin/grep.c
index 5df6537333..63a4959568 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -1214,7 +1214,7 @@ int cmd_grep(int argc,
 		if (recurse_submodules)
 			repo_read_gitmodules(the_repository, 1);
 		if (startup_info->have_repository)
-			(void)get_packed_git(the_repository);
+			(void)packfile_store_get_packs(the_repository->objects->packfiles);
 
 		start_threads(&opt);
 	} else {
diff --git a/object-name.c b/object-name.c
index df9e0c5f02..53356819a3 100644
--- a/object-name.c
+++ b/object-name.c
@@ -213,7 +213,7 @@ static void find_short_packed_object(struct disambiguate_state *ds)
 			unique_in_midx(m, ds);
 	}
 
-	for (p = get_packed_git(ds->repo); p && !ds->ambiguous;
+	for (p = packfile_store_get_packs(ds->repo->objects->packfiles); p && !ds->ambiguous;
 	     p = p->next)
 		unique_in_pack(p, ds);
 }
@@ -806,7 +806,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)
 			find_abbrev_len_for_midx(m, mad);
 	}
 
-	for (p = get_packed_git(mad->repo); p; p = p->next)
+	for (p = packfile_store_get_packs(mad->repo->objects->packfiles); p; p = p->next)
 		find_abbrev_len_for_pack(p, mad);
 }
 
diff --git a/packfile.c b/packfile.c
index 7a9193e5ef..b37f43afb5 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1027,10 +1027,10 @@ void packfile_store_reprepare(struct packfile_store *store)
 	packfile_store_prepare(store);
 }
 
-struct packed_git *get_packed_git(struct repository *r)
+struct packed_git *packfile_store_get_packs(struct packfile_store *store)
 {
-	packfile_store_prepare(r->objects->packfiles);
-	return r->objects->packfiles->packs;
+	packfile_store_prepare(store);
+	return store->packs;
 }
 
 struct packed_git *get_all_packs(struct repository *r)
diff --git a/packfile.h b/packfile.h
index a9e561ac39..0b691ded7e 100644
--- a/packfile.h
+++ b/packfile.h
@@ -136,6 +136,12 @@ void packfile_store_reprepare(struct packfile_store *store);
 void packfile_store_add_pack(struct packfile_store *store,
 			     struct packed_git *pack);
 
+/*
+ * Get packs managed by the given store. Does not load the MIDX or any packs
+ * referenced by it.
+ */
+struct packed_git *packfile_store_get_packs(struct packfile_store *store);
+
 /*
  * Open the packfile and add it to the store if it isn't yet known. Returns
  * either the newly opened packfile or the preexisting packfile. Returns a
@@ -220,7 +226,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-struct packed_git *get_packed_git(struct repository *r);
 struct list_head *get_packed_git_mru(struct repository *r);
 struct packed_git *get_all_packs(struct repository *r);
 
-- 
2.51.0.536.g15c5d4f767.dirty
Patrick Steinhardt· Sep 23, 2025, 10:17 UTC · re: Patrick Steinhardt · lore

[PATCH v6 14/15] packfile: refactor `get_all_packs()` to work on packfile store

The `get_all_packs()` function prepares the packfile store and then returns its packfiles. Refactor it to accept a packfile store instead of a repository to clarify its scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/cat-file.c          |  3 ++-
 builtin/count-objects.c     |  3 ++-
 builtin/fast-import.c       |  6 ++++--
 builtin/fsck.c              | 11 +++++++----
 builtin/gc.c                |  8 +++++---
 builtin/pack-objects.c      | 28 +++++++++++++++++++---------
 builtin/pack-redundant.c    |  6 ++++--
 builtin/repack.c            |  9 ++++++---
 connected.c                 |  3 ++-
 http-backend.c              |  5 +++--
 http.c                      |  3 ++-
 pack-bitmap.c               |  4 ++--
 pack-objects.c              |  3 ++-
 packfile.c                  | 12 ++++++------
 packfile.h                  |  7 ++++++-
 server-info.c               |  3 ++-
 t/helper/test-find-pack.c   |  2 +-
 t/helper/test-pack-mtimes.c |  2 +-
 18 files changed, 76 insertions(+), 42 deletions(-)
Show changes to 18 files +76 −42

builtin/cat-file.c, builtin/count-objects.c, builtin/fast-import.c, builtin/fsck.c, builtin/gc.c, builtin/pack-objects.c, builtin/pack-redundant.c, builtin/repack.c, connected.c, http-backend.c, http.c, pack-bitmap.c, pack-objects.c, packfile.c, packfile.h, server-info.c, t/helper/test-find-pack.c, t/helper/test-pack-mtimes.c

diff --git a/builtin/cat-file.c b/builtin/cat-file.c
index fce0b06451c..ee6715fa523 100644
--- a/builtin/cat-file.c
+++ b/builtin/cat-file.c
@@ -852,9 +852,10 @@ static void batch_each_object(struct batch_options *opt,
 
 	if (bitmap && !for_each_bitmapped_object(bitmap, &opt->objects_filter,
 						 batch_one_object_bitmapped, &payload)) {
+		struct packfile_store *packs = the_repository->objects->packfiles;
 		struct packed_git *pack;
 
-		for (pack = get_all_packs(the_repository); pack; pack = pack->next) {
+		for (pack = packfile_store_get_all_packs(packs); pack; pack = pack->next) {
 			if (bitmap_index_contains_pack(bitmap, pack) ||
 			    open_pack_index(pack))
 				continue;
diff --git a/builtin/count-objects.c b/builtin/count-objects.c
index a61d3b46aac..f2f407c2a78 100644
--- a/builtin/count-objects.c
+++ b/builtin/count-objects.c
@@ -122,6 +122,7 @@ int cmd_count_objects(int argc,
 				      count_loose, count_cruft, NULL, NULL);
 
 	if (verbose) {
+		struct packfile_store *packs = the_repository->objects->packfiles;
 		struct packed_git *p;
 		unsigned long num_pack = 0;
 		off_t size_pack = 0;
@@ -129,7 +130,7 @@ int cmd_count_objects(int argc,
 		struct strbuf pack_buf = STRBUF_INIT;
 		struct strbuf garbage_buf = STRBUF_INIT;
 
-		for (p = get_all_packs(the_repository); p; p = p->next) {
+		for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 			if (!p->pack_local)
 				continue;
 			if (open_pack_index(p))
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index a26e79689d5..b1d5549815a 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -952,6 +952,7 @@ static int store_object(
 	struct object_id *oidout,
 	uintmax_t mark)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	void *out, *delta;
 	struct object_entry *e;
 	unsigned char hdr[96];
@@ -975,7 +976,7 @@ static int store_object(
 	if (e->idx.offset) {
 		duplicate_count_by_type[type]++;
 		return 1;
-	} else if (find_oid_pack(&oid, get_all_packs(the_repository))) {
+	} else if (find_oid_pack(&oid, packfile_store_get_all_packs(packs))) {
 		e->type = type;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
@@ -1092,6 +1093,7 @@ static void truncate_pack(struct hashfile_checkpoint *checkpoint)
 
 static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	size_t in_sz = 64 * 1024, out_sz = 64 * 1024;
 	unsigned char *in_buf = xmalloc(in_sz);
 	unsigned char *out_buf = xmalloc(out_sz);
@@ -1175,7 +1177,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 		duplicate_count_by_type[OBJ_BLOB]++;
 		truncate_pack(&checkpoint);
 
-	} else if (find_oid_pack(&oid, get_all_packs(the_repository))) {
+	} else if (find_oid_pack(&oid, packfile_store_get_all_packs(packs))) {
 		e->type = OBJ_BLOB;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
diff --git a/builtin/fsck.c b/builtin/fsck.c
index d2eb9d4fbe9..8ee95e0d67c 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -867,19 +867,20 @@ static int mark_packed_for_connectivity(const struct object_id *oid,
 
 static int check_pack_rev_indexes(struct repository *r, int show_progress)
 {
+	struct packfile_store *packs = r->objects->packfiles;
 	struct progress *progress = NULL;
 	uint32_t pack_count = 0;
 	int res = 0;
 
 	if (show_progress) {
-		for (struct packed_git *p = get_all_packs(r); p; p = p->next)
+		for (struct packed_git *p = packfile_store_get_all_packs(packs); p; p = p->next)
 			pack_count++;
 		progress = start_delayed_progress(the_repository,
 						  "Verifying reverse pack-indexes", pack_count);
 		pack_count = 0;
 	}
 
-	for (struct packed_git *p = get_all_packs(r); p; p = p->next) {
+	for (struct packed_git *p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		int load_error = load_pack_revindex_from_disk(p);
 
 		if (load_error < 0) {
@@ -999,6 +1000,8 @@ int cmd_fsck(int argc,
 		for_each_packed_object(the_repository,
 				       mark_packed_for_connectivity, NULL, 0);
 	} else {
+		struct packfile_store *packs = the_repository->objects->packfiles;
+
 		odb_prepare_alternates(the_repository->objects);
 		for (source = the_repository->objects->sources; source; source = source->next)
 			fsck_source(source);
@@ -1009,7 +1012,7 @@ int cmd_fsck(int argc,
 			struct progress *progress = NULL;
 
 			if (show_progress) {
-				for (p = get_all_packs(the_repository); p;
+				for (p = packfile_store_get_all_packs(packs); p;
 				     p = p->next) {
 					if (open_pack_index(p))
 						continue;
@@ -1019,7 +1022,7 @@ int cmd_fsck(int argc,
 				progress = start_progress(the_repository,
 							  _("Checking objects"), total);
 			}
-			for (p = get_all_packs(the_repository); p;
+			for (p = packfile_store_get_all_packs(packs); p;
 			     p = p->next) {
 				/* verify gives error messages itself */
 				if (verify_pack(the_repository,
diff --git a/builtin/gc.c b/builtin/gc.c
index ec6735a540a..e19e13d9788 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -487,9 +487,10 @@ static int too_many_loose_objects(struct gc_config *cfg)
 static struct packed_git *find_base_packs(struct string_list *packs,
 					  unsigned long limit)
 {
+	struct packfile_store *packfiles = the_repository->objects->packfiles;
 	struct packed_git *p, *base = NULL;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(packfiles); p; p = p->next) {
 		if (!p->pack_local || p->is_cruft)
 			continue;
 		if (limit) {
@@ -508,13 +509,14 @@ static struct packed_git *find_base_packs(struct string_list *packs,
 
 static int too_many_packs(struct gc_config *cfg)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 	int cnt;
 
 	if (cfg->gc_auto_pack_limit <= 0)
 		return 0;
 
-	for (cnt = 0, p = get_all_packs(the_repository); p; p = p->next) {
+	for (cnt = 0, p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		if (!p->pack_local)
 			continue;
 		if (p->pack_keep)
@@ -1492,7 +1494,7 @@ static off_t get_auto_pack_size(void)
 	struct repository *r = the_repository;
 
 	odb_reprepare(r->objects);
-	for (p = get_all_packs(r); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(r->objects->packfiles); p; p = p->next) {
 		if (p->pack_size > max_size) {
 			second_largest_size = max_size;
 			max_size = p->pack_size;
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 1494afcf3df..de351b757ae 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -3831,6 +3831,7 @@ static int pack_mtime_cmp(const void *_a, const void *_b)
 
 static void read_packs_list_from_stdin(struct rev_info *revs)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct strbuf buf = STRBUF_INIT;
 	struct string_list include_packs = STRING_LIST_INIT_DUP;
 	struct string_list exclude_packs = STRING_LIST_INIT_DUP;
@@ -3855,7 +3856,7 @@ static void read_packs_list_from_stdin(struct rev_info *revs)
 	string_list_sort(&exclude_packs);
 	string_list_remove_duplicates(&exclude_packs, 0);
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		const char *pack_name = pack_basename(p);
 
 		if ((item = string_list_lookup(&include_packs, pack_name)))
@@ -4076,6 +4077,7 @@ static void enumerate_cruft_objects(void)
 
 static void enumerate_and_traverse_cruft_objects(struct string_list *fresh_packs)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 	struct rev_info revs;
 	int ret;
@@ -4105,7 +4107,7 @@ static void enumerate_and_traverse_cruft_objects(struct string_list *fresh_packs
 	 * Re-mark only the fresh packs as kept so that objects in
 	 * unknown packs do not halt the reachability traversal early.
 	 */
-	for (p = get_all_packs(the_repository); p; p = p->next)
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next)
 		p->pack_keep_in_core = 0;
 	mark_pack_kept_in_core(fresh_packs, 1);
 
@@ -4122,6 +4124,7 @@ static void enumerate_and_traverse_cruft_objects(struct string_list *fresh_packs
 
 static void read_cruft_objects(void)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct strbuf buf = STRBUF_INIT;
 	struct string_list discard_packs = STRING_LIST_INIT_DUP;
 	struct string_list fresh_packs = STRING_LIST_INIT_DUP;
@@ -4142,7 +4145,7 @@ static void read_cruft_objects(void)
 	string_list_sort(&discard_packs);
 	string_list_sort(&fresh_packs);
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		const char *pack_name = pack_basename(p);
 		struct string_list_item *item;
 
@@ -4390,11 +4393,12 @@ static void add_unreachable_loose_objects(struct rev_info *revs)
 
 static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	static struct packed_git *last_found = (void *)1;
 	struct packed_git *p;
 
 	p = (last_found != (void *)1) ? last_found :
-					get_all_packs(the_repository);
+					packfile_store_get_all_packs(packs);
 
 	while (p) {
 		if ((!p->pack_local || p->pack_keep ||
@@ -4404,7 +4408,7 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 			return 1;
 		}
 		if (p == last_found)
-			p = get_all_packs(the_repository);
+			p = packfile_store_get_all_packs(packs);
 		else
 			p = p->next;
 		if (p == last_found)
@@ -4436,12 +4440,13 @@ static int loosened_object_can_be_discarded(const struct object_id *oid,
 
 static void loosen_unused_packed_objects(void)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 	uint32_t i;
 	uint32_t loosened_objects_nr = 0;
 	struct object_id oid;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		if (!p->pack_local || p->pack_keep || p->pack_keep_in_core)
 			continue;
 
@@ -4742,12 +4747,13 @@ static void get_object_list(struct rev_info *revs, int ac, const char **av)
 
 static void add_extra_kept_packs(const struct string_list *names)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 
 	if (!names->nr)
 		return;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		const char *name = basename(p->pack_name);
 		int i;
 
@@ -5185,8 +5191,10 @@ int cmd_pack_objects(int argc,
 
 	add_extra_kept_packs(&keep_pack_list);
 	if (ignore_packed_keep_on_disk) {
+		struct packfile_store *packs = the_repository->objects->packfiles;
 		struct packed_git *p;
-		for (p = get_all_packs(the_repository); p; p = p->next)
+
+		for (p = packfile_store_get_all_packs(packs); p; p = p->next)
 			if (p->pack_local && p->pack_keep)
 				break;
 		if (!p) /* no keep-able packs found */
@@ -5198,8 +5206,10 @@ int cmd_pack_objects(int argc,
 		 * want to unset "local" based on looking at packs, as
 		 * it also covers non-local objects
 		 */
+		struct packfile_store *packs = the_repository->objects->packfiles;
 		struct packed_git *p;
-		for (p = get_all_packs(the_repository); p; p = p->next) {
+
+		for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 			if (!p->pack_local) {
 				have_non_local_packs = 1;
 				break;
diff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c
index fe81c293e3a..dd28171f0a1 100644
--- a/builtin/pack-redundant.c
+++ b/builtin/pack-redundant.c
@@ -566,7 +566,8 @@ static struct pack_list * add_pack(struct packed_git *p)
 
 static struct pack_list * add_pack_file(const char *filename)
 {
-	struct packed_git *p = get_all_packs(the_repository);
+	struct packfile_store *packs = the_repository->objects->packfiles;
+	struct packed_git *p = packfile_store_get_all_packs(packs);
 
 	if (strlen(filename) < 40)
 		die("Bad pack filename: %s", filename);
@@ -581,7 +582,8 @@ static struct pack_list * add_pack_file(const char *filename)
 
 static void load_all(void)
 {
-	struct packed_git *p = get_all_packs(the_repository);
+	struct packfile_store *packs = the_repository->objects->packfiles;
+	struct packed_git *p = packfile_store_get_all_packs(packs);
 
 	while (p) {
 		add_pack(p);
diff --git a/builtin/repack.c b/builtin/repack.c
index 5ff27fc8e29..e8730808c53 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -265,10 +265,11 @@ static void existing_packs_release(struct existing_packs *existing)
 static void collect_pack_filenames(struct existing_packs *existing,
 				   const struct string_list *extra_keep)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 	struct strbuf buf = STRBUF_INIT;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		int i;
 		const char *base;
 
@@ -497,10 +498,11 @@ static void init_pack_geometry(struct pack_geometry *geometry,
 			       struct existing_packs *existing,
 			       const struct pack_objects_args *args)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 	struct strbuf buf = STRBUF_INIT;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		if (args->local && !p->pack_local)
 			/*
 			 * When asked to only repack local packfiles we skip
@@ -1137,11 +1139,12 @@ static int write_filtered_pack(const struct pack_objects_args *args,
 static void combine_small_cruft_packs(FILE *in, size_t combine_cruft_below_size,
 				      struct existing_packs *existing)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *p;
 	struct strbuf buf = STRBUF_INIT;
 	size_t i;
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		if (!(p->is_cruft && p->pack_local))
 			continue;
 
diff --git a/connected.c b/connected.c
index d6e9682fd93..b288a18b17c 100644
--- a/connected.c
+++ b/connected.c
@@ -74,9 +74,10 @@ int check_connected(oid_iterate_fn fn, void *cb_data,
 		 */
 		odb_reprepare(the_repository->objects);
 		do {
+			struct packfile_store *packs = the_repository->objects->packfiles;
 			struct packed_git *p;
 
-			for (p = get_all_packs(the_repository); p; p = p->next) {
+			for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 				if (!p->pack_promisor)
 					continue;
 				if (find_pack_entry_one(oid, p))
diff --git a/http-backend.c b/http-backend.c
index d5dfe762bb5..9084058f1e9 100644
--- a/http-backend.c
+++ b/http-backend.c
@@ -603,18 +603,19 @@ static void get_head(struct strbuf *hdr, char *arg UNUSED)
 static void get_info_packs(struct strbuf *hdr, char *arg UNUSED)
 {
 	size_t objdirlen = strlen(repo_get_object_directory(the_repository));
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct strbuf buf = STRBUF_INIT;
 	struct packed_git *p;
 	size_t cnt = 0;
 
 	select_getanyfile(hdr);
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		if (p->pack_local)
 			cnt++;
 	}
 
 	strbuf_grow(&buf, cnt * 53 + 2);
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		if (p->pack_local)
 			strbuf_addf(&buf, "P %s\n", p->pack_name + objdirlen + 6);
 	}
diff --git a/http.c b/http.c
index af2120b64c7..077e879de9e 100644
--- a/http.c
+++ b/http.c
@@ -2408,6 +2408,7 @@ static char *fetch_pack_index(unsigned char *hash, const char *base_url)
 static int fetch_and_setup_pack_index(struct packed_git **packs_head,
 	unsigned char *sha1, const char *base_url)
 {
+	struct packfile_store *packs = the_repository->objects->packfiles;
 	struct packed_git *new_pack, *p;
 	char *tmp_idx = NULL;
 	int ret;
@@ -2416,7 +2417,7 @@ static int fetch_and_setup_pack_index(struct packed_git **packs_head,
 	 * If we already have the pack locally, no need to fetch its index or
 	 * even add it to list; we already have all of its objects.
 	 */
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		if (hasheq(p->hash, sha1, the_repository->hash_algo))
 			return 0;
 	}
diff --git a/pack-bitmap.c b/pack-bitmap.c
index 058bdb5d7de..ac71035d771 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -664,7 +664,7 @@ static int open_pack_bitmap(struct repository *r,
 	struct packed_git *p;
 	int ret = -1;
 
-	for (p = get_all_packs(r); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(r->objects->packfiles); p; p = p->next) {
 		if (open_pack_bitmap_1(bitmap_git, p) == 0) {
 			ret = 0;
 			/*
@@ -3362,7 +3362,7 @@ int verify_bitmap_files(struct repository *r)
 		free(midx_bitmap_name);
 	}
 
-	for (struct packed_git *p = get_all_packs(r);
+	for (struct packed_git *p = packfile_store_get_all_packs(r->objects->packfiles);
 	     p; p = p->next) {
 		char *pack_bitmap_name = pack_bitmap_filename(p);
 		res |= verify_bitmap_file(r->hash_algo, pack_bitmap_name);
diff --git a/pack-objects.c b/pack-objects.c
index a9d9855063a..d8eb6797354 100644
--- a/pack-objects.c
+++ b/pack-objects.c
@@ -86,6 +86,7 @@ struct object_entry *packlist_find(struct packing_data *pdata,
 
 static void prepare_in_pack_by_idx(struct packing_data *pdata)
 {
+	struct packfile_store *packs = pdata->repo->objects->packfiles;
 	struct packed_git **mapping, *p;
 	int cnt = 0, nr = 1U << OE_IN_PACK_BITS;
 
@@ -95,7 +96,7 @@ static void prepare_in_pack_by_idx(struct packing_data *pdata)
 	 * (i.e. in_pack_idx also zero) should return NULL.
 	 */
 	mapping[cnt++] = NULL;
-	for (p = get_all_packs(pdata->repo); p; p = p->next, cnt++) {
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next, cnt++) {
 		if (cnt == nr) {
 			free(mapping);
 			return;
diff --git a/packfile.c b/packfile.c
index b37f43afb58..cd5431b6aa1 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1033,11 +1033,11 @@ struct packed_git *packfile_store_get_packs(struct packfile_store *store)
 	return store->packs;
 }
 
-struct packed_git *get_all_packs(struct repository *r)
+struct packed_git *packfile_store_get_all_packs(struct packfile_store *store)
 {
-	packfile_store_prepare(r->objects->packfiles);
+	packfile_store_prepare(store);
 
-	for (struct odb_source *source = r->objects->sources; source; source = source->next) {
+	for (struct odb_source *source = store->odb->sources; source; source = source->next) {
 		struct multi_pack_index *m = source->midx;
 		if (!m)
 			continue;
@@ -1045,7 +1045,7 @@ struct packed_git *get_all_packs(struct repository *r)
 			prepare_midx_pack(m, i);
 	}
 
-	return r->objects->packfiles->packs;
+	return store->packs;
 }
 
 struct list_head *get_packed_git_mru(struct repository *r)
@@ -2105,7 +2105,7 @@ struct packed_git **kept_pack_cache(struct repository *r, unsigned flags)
 		 * covers, one kept and one not kept, but the midx returns only
 		 * the non-kept version.
 		 */
-		for (p = get_all_packs(r); p; p = p->next) {
+		for (p = packfile_store_get_all_packs(r->objects->packfiles); p; p = p->next) {
 			if ((p->pack_keep && (flags & ON_DISK_KEEP_PACKS)) ||
 			    (p->pack_keep_in_core && (flags & IN_CORE_KEEP_PACKS))) {
 				ALLOC_GROW(packs, nr + 1, alloc);
@@ -2202,7 +2202,7 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 	int r = 0;
 	int pack_errors = 0;
 
-	for (p = get_all_packs(repo); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(repo->objects->packfiles); p; p = p->next) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
 		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&
diff --git a/packfile.h b/packfile.h
index 0b691ded7ef..1afb9cd6641 100644
--- a/packfile.h
+++ b/packfile.h
@@ -142,6 +142,12 @@ void packfile_store_add_pack(struct packfile_store *store,
  */
 struct packed_git *packfile_store_get_packs(struct packfile_store *store);
 
+/*
+ * Get all packs managed by the given store, including packfiles that are
+ * referenced by multi-pack indices.
+ */
+struct packed_git *packfile_store_get_all_packs(struct packfile_store *store);
+
 /*
  * Open the packfile and add it to the store if it isn't yet known. Returns
  * either the newly opened packfile or the preexisting packfile. Returns a
@@ -227,7 +233,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
 struct list_head *get_packed_git_mru(struct repository *r);
-struct packed_git *get_all_packs(struct repository *r);
 
 /*
  * Give a rough count of objects in the repository. This sacrifices accuracy
diff --git a/server-info.c b/server-info.c
index 9bb30d9ab71..1d33de821e9 100644
--- a/server-info.c
+++ b/server-info.c
@@ -287,12 +287,13 @@ static int compare_info(const void *a_, const void *b_)
 
 static void init_pack_info(struct repository *r, const char *infofile, int force)
 {
+	struct packfile_store *packs = r->objects->packfiles;
 	struct packed_git *p;
 	int stale;
 	int i;
 	size_t alloc = 0;
 
-	for (p = get_all_packs(r); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(packs); p; p = p->next) {
 		/* we ignore things on alternate path since they are
 		 * not available to the pullers in general.
 		 */
diff --git a/t/helper/test-find-pack.c b/t/helper/test-find-pack.c
index 611a13a3261..e001dc3066d 100644
--- a/t/helper/test-find-pack.c
+++ b/t/helper/test-find-pack.c
@@ -39,7 +39,7 @@ int cmd__find_pack(int argc, const char **argv)
 	if (repo_get_oid(the_repository, argv[0], &oid))
 		die("cannot parse %s as an object name", argv[0]);
 
-	for (p = get_all_packs(the_repository); p; p = p->next)
+	for (p = packfile_store_get_all_packs(the_repository->objects->packfiles); p; p = p->next)
 		if (find_pack_entry_one(&oid, p)) {
 			printf("%s\n", p->pack_name);
 			actual_count++;
diff --git a/t/helper/test-pack-mtimes.c b/t/helper/test-pack-mtimes.c
index d51aaa3dc40..7c428c16011 100644
--- a/t/helper/test-pack-mtimes.c
+++ b/t/helper/test-pack-mtimes.c
@@ -37,7 +37,7 @@ int cmd__pack_mtimes(int argc, const char **argv)
 	if (argc != 2)
 		usage(pack_mtimes_usage);
 
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for (p = packfile_store_get_all_packs(the_repository->objects->packfiles); p; p = p->next) {
 		strbuf_addstr(&buf, basename(p->pack_name));
 		strbuf_strip_suffix(&buf, ".pack");
 		strbuf_addstr(&buf, ".mtimes");
-- 
2.51.0.536.g15c5d4f767.dirty
Patrick Steinhardt· Sep 23, 2025, 10:17 UTC · re: Patrick Steinhardt · lore

[PATCH v6 15/15] packfile: refactor `get_packed_git_mru()` to work on packfile store

The `get_packed_git_mru()` function prepares the packfile store and then returns its packfiles in most-recently-used order. Refactor it to accept a packfile store instead of a repository to clarify its scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/pack-objects.c | 4 ++--
 packfile.c             | 6 +++---
 packfile.h             | 7 +++++--
 3 files changed, 10 insertions(+), 7 deletions(-)
Show changes to 3 files +10 −7

builtin/pack-objects.c, packfile.c, packfile.h

diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index de351b757a..61bbbdfb83 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -1748,12 +1748,12 @@ static int want_object_in_pack_mtime(const struct object_id *oid,
 		}
 	}
 
-	list_for_each(pos, get_packed_git_mru(the_repository)) {
+	list_for_each(pos, packfile_store_get_packs_mru(the_repository->objects->packfiles)) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		want = want_object_in_pack_one(p, oid, exclude, found_pack, found_offset, found_mtime);
 		if (!exclude && want > 0)
 			list_move(&p->mru,
-				  get_packed_git_mru(the_repository));
+				  packfile_store_get_packs_mru(the_repository->objects->packfiles));
 		if (want != -1)
 			return want;
 	}
diff --git a/packfile.c b/packfile.c
index cd5431b6aa..5a7caec292 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1048,10 +1048,10 @@ struct packed_git *packfile_store_get_all_packs(struct packfile_store *store)
 	return store->packs;
 }
 
-struct list_head *get_packed_git_mru(struct repository *r)
+struct list_head *packfile_store_get_packs_mru(struct packfile_store *store)
 {
-	packfile_store_prepare(r->objects->packfiles);
-	return &r->objects->packfiles->mru;
+	packfile_store_prepare(store);
+	return &store->mru;
 }
 
 /*
diff --git a/packfile.h b/packfile.h
index 1afb9cd664..e7a5792b6c 100644
--- a/packfile.h
+++ b/packfile.h
@@ -148,6 +148,11 @@ struct packed_git *packfile_store_get_packs(struct packfile_store *store);
  */
 struct packed_git *packfile_store_get_all_packs(struct packfile_store *store);
 
+/*
+ * Get all packs in most-recently-used order.
+ */
+struct list_head *packfile_store_get_packs_mru(struct packfile_store *store);
+
 /*
  * Open the packfile and add it to the store if it isn't yet known. Returns
  * either the newly opened packfile or the preexisting packfile. Returns a
@@ -232,8 +237,6 @@ int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-struct list_head *get_packed_git_mru(struct repository *r);
-
 /*
  * Give a rough count of objects in the repository. This sacrifices accuracy
  * for speed.
-- 
2.51.0.536.g15c5d4f767.dirty
Justin Tobler· Sep 24, 2025, 21:58 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v6 00/15] packfile: carve out a new packfile store

On 25/09/23 12:16PM, Patrick Steinhardt wrote:
Show 5 quoted lines
> Changes in v6:
>   - Fix a grammar issue in a commit message.
>   - Update a comment to reflect that we access `struct packfile_store`
>     in "midx.c" directly, as well.
>   - Link to v5: https://lore.kernel.org/r/20250915-b4-pks-packfiles-store-v5-0-d6340350934f@pks.im
The changes in the version look good. Thanks
-Justin
Junio C Hamano· Sep 25, 2025, 16:08 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v6 00/15] packfile: carve out a new packfile store

Patrick Steinhardt <ps@pks.im> writes:
Show 14 quoted lines
> information about a object database's packfiles is currently distributed
> across two different structures:
>
>   - `struct packed_git` contains the `next` pointer as well as the
>     `mru_head`, both of which serve to store the list of packfiles.
>
>   - `struct object_database` contains several fields that relate to the
>     packfiles.
> ...
> Changes in v6:
>   - Fix a grammar issue in a commit message.
>   - Update a comment to reflect that we access `struct packfile_store`
>     in "midx.c" directly, as well.
>   - Link to v5: https://lore.kernel.org/r/20250915-b4-pks-packfiles-store-v5-0-d6340350934f@pks.im

It seems that we are basically done with the topic and strayed into nitpicking territory? Shall we declare victory and mark the topic for 'next'?

Thanks.
Patrick Steinhardt· Sep 26, 2025, 05:26 UTC · re: Junio C Hamano · lore

Re: [PATCH v6 00/15] packfile: carve out a new packfile store

On Thu, Sep 25, 2025 at 09:08:19AM -0700, Junio C Hamano wrote:
Show 20 quoted lines
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > information about a object database's packfiles is currently distributed
> > across two different structures:
> >
> >   - `struct packed_git` contains the `next` pointer as well as the
> >     `mru_head`, both of which serve to store the list of packfiles.
> >
> >   - `struct object_database` contains several fields that relate to the
> >     packfiles.
> > ...
> > Changes in v6:
> >   - Fix a grammar issue in a commit message.
> >   - Update a comment to reflect that we access `struct packfile_store`
> >     in "midx.c" directly, as well.
> >   - Link to v5: https://lore.kernel.org/r/20250915-b4-pks-packfiles-store-v5-0-d6340350934f@pks.im
> 
> It seems that we are basically done with the topic and strayed into
> nitpicking territory?   Shall we declare victory and mark the topic
> for 'next'?
From my point of view it should be ready to go, yeah. Thanks!
Patrick
Taylor Blau· Sep 28, 2025, 22:05 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v6 00/15] packfile: carve out a new packfile store

On Fri, Sep 26, 2025 at 07:26:14AM +0200, Patrick Steinhardt wrote:
Show 23 quoted lines
> On Thu, Sep 25, 2025 at 09:08:19AM -0700, Junio C Hamano wrote:
> > Patrick Steinhardt <ps@pks.im> writes:
> >
> > > information about a object database's packfiles is currently distributed
> > > across two different structures:
> > >
> > >   - `struct packed_git` contains the `next` pointer as well as the
> > >     `mru_head`, both of which serve to store the list of packfiles.
> > >
> > >   - `struct object_database` contains several fields that relate to the
> > >     packfiles.
> > > ...
> > > Changes in v6:
> > >   - Fix a grammar issue in a commit message.
> > >   - Update a comment to reflect that we access `struct packfile_store`
> > >     in "midx.c" directly, as well.
> > >   - Link to v5: https://lore.kernel.org/r/20250915-b4-pks-packfiles-store-v5-0-d6340350934f@pks.im
> >
> > It seems that we are basically done with the topic and strayed into
> > nitpicking territory?   Shall we declare victory and mark the topic
> > for 'next'?
>
> >From my point of view it should be ready to go, yeah. Thanks!

Yeah, this version looks good to me. I apologize for the delay in reading v5, but looking at the range-diffs between v4 and v6, the result looks good to me.

  Acked-by: Taylor Blau <me@ttaylorr.com>

Thanks, Taylor

Patrick Steinhardt· Sep 29, 2025, 21:39 UTC · re: Taylor Blau · lore

Re: [PATCH v6 00/15] packfile: carve out a new packfile store

On Sun, Sep 28, 2025 at 06:05:54PM -0400, Taylor Blau wrote:
Show 30 quoted lines
> On Fri, Sep 26, 2025 at 07:26:14AM +0200, Patrick Steinhardt wrote:
> > On Thu, Sep 25, 2025 at 09:08:19AM -0700, Junio C Hamano wrote:
> > > Patrick Steinhardt <ps@pks.im> writes:
> > >
> > > > information about a object database's packfiles is currently distributed
> > > > across two different structures:
> > > >
> > > >   - `struct packed_git` contains the `next` pointer as well as the
> > > >     `mru_head`, both of which serve to store the list of packfiles.
> > > >
> > > >   - `struct object_database` contains several fields that relate to the
> > > >     packfiles.
> > > > ...
> > > > Changes in v6:
> > > >   - Fix a grammar issue in a commit message.
> > > >   - Update a comment to reflect that we access `struct packfile_store`
> > > >     in "midx.c" directly, as well.
> > > >   - Link to v5: https://lore.kernel.org/r/20250915-b4-pks-packfiles-store-v5-0-d6340350934f@pks.im
> > >
> > > It seems that we are basically done with the topic and strayed into
> > > nitpicking territory?   Shall we declare victory and mark the topic
> > > for 'next'?
> >
> > >From my point of view it should be ready to go, yeah. Thanks!
> 
> Yeah, this version looks good to me. I apologize for the delay in
> reading v5, but looking at the range-diffs between v4 and v6, the result
> looks good to me.
> 
>   Acked-by: Taylor Blau <me@ttaylorr.com>
Thanks!
Patrick

← back to recent threads