# [PATCH 0/6] packfile: some lifecycle improvements

7 messages from 2026-11-20 to 2026-11-20. Participants: Patrick Steinhardt.
Thread: https://gitlist.dev/t/66896

## Patrick Steinhardt, 2026-11-20 12:02

Subject: [PATCH 0/6] packfile: some lifecycle improvements
Message-ID: <20261120-pks-packfile-global-statistics-v1-0-19beb83b145f@pks.im>
URL: https://gitlist.dev/e/20261120-pks-packfile-global-statistics-v1-0-19beb83b145f%40pks.im

```
Hi,

this small patch series polishes our lifecycle handling of packfiles a
bit. The original motivation of this patch series is really only the
last patch, which fixes how we handle process-wide limitations. But I
took the chance to also polish this subsystem a bit, as a lot of its
parts feel rather dated nowadays.

There's a minor semantic merge conflict with "seen" that can be sovled
like this:

diff --git a/odb/source-packed.c b/odb/source-packed.c
index 77f8227ae52..e24f6cc7ef9 100644
--- a/odb/source-packed.c
+++ b/odb/source-packed.c
@@ -898,7 +898,7 @@ static int odb_source_packed_map_oid(struct odb_source *source,
 						       &hash_algos[src->algo] :
 						       repo->hash_algo;
 
-		if (open_pack_index(p) || !p->num_objects)
+		if (packfile_open_index(p) || !p->num_objects)
 			continue;
 		/* Only v3 indices store object IDs in both algorithms. */
 		if (p->index_version < 3)

Thanks!

Patrick

---
Patrick Steinhardt (6):
      builtin/fast-import: stop reporting packfile statistics
      packfile: rename lifecycle functions
      packfile: inline `open_packed_git_1()`
      packfile: introduce function to free a packfile
      packfile: maintain process-wide limits globally
      packfile: skip closing packs marked as do-not-close

 Documentation/git-fast-import.adoc |   5 +-
 builtin/count-objects.c            |   2 +-
 builtin/fast-import.c              |   8 +-
 builtin/index-pack.c               |  16 +--
 builtin/pack-objects.c             |   2 +-
 builtin/pack-redundant.c           |   2 +-
 commit-graph.c                     |  12 +-
 connected.c                        |  10 +-
 http-push.c                        |   2 +-
 http-walker.c                      |   8 +-
 http.c                             |   4 +-
 midx-write.c                       |  23 ++--
 midx.c                             |   6 +-
 odb/source-packed.c                |  14 +--
 pack-check.c                       |   2 +-
 pack-mtimes.c                      |   2 +-
 pack-revindex.c                    |   4 +-
 packfile.c                         | 233 ++++++++++++++++++-------------------
 packfile.h                         |  30 ++---
 repack-geometry.c                  |   2 +-
 repack-midx.c                      |   2 +-
 t/helper/test-bitmap.c             |   2 +-
 22 files changed, 182 insertions(+), 209 deletions(-)


---
base-commit: f7de81456940dd6f3274e461149bba12965de609
change-id: 20261120-pks-packfile-global-statistics-92a9be97a11f



```

## Patrick Steinhardt, 2026-11-20 12:02

Subject: [PATCH 1/6] builtin/fast-import: stop reporting packfile statistics
Message-ID: <20261120-pks-packfile-global-statistics-v1-1-19beb83b145f@pks.im>
URL: https://gitlist.dev/e/20261120-pks-packfile-global-statistics-v1-1-19beb83b145f%40pks.im
In-Reply-To: <20261120-pks-packfile-global-statistics-v1-0-19beb83b145f@pks.im>

```
By default, git-fast-import(1) prints a bunch of statistics about all
kinds of internal details of Git. This includes for example how many
objects have been allocated, the number of refs processed, or the amount
of memory that was allocated.

Reporting this kind information is highly questionable, doubly so as it
is done by default, as a user really shoudln't have to worry about such
internal implementation details. For some of the above metrics it could
be argued though that it's indeed interesting to learn about the number
of objects processed to get a rough sense of what git-fast-import(1) has
done. But for other kind of information that's not the case, as it is
without a doubt purely internal implementation details.

One of these reported implementation details is our packfile statistics.
These include information on the system's page size, the number of packs
that were opened, the number of calls to mmap(3) and how many pack
windows were created. Reporting that kind of information, and that even
by default, is just completely uncalled for. This information may be
useful to a developer, but a user really shouldn't have to care about
any of this.

More importantly though, with pluggable object databases we shouldn't
assume that we're using packfiles in the first place anymore. The
current implementation of git-fast-import(1) still does, but we will
convert it to use transactions sooner rather than later.

Let's just unceremoniously drop reporting packfile information to the
user. If anyone complains (which I highly doubt), then we may consider
reintroducing this as trace2 data. The same is arguably true for many of
the other reported statistics. But for now I care about the packfile
subsystem, only, and hence those are left as-is.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/git-fast-import.adoc |  5 ++---
 builtin/fast-import.c              |  4 +---
 packfile.c                         | 36 ------------------------------------
 packfile.h                         |  2 --
 4 files changed, 3 insertions(+), 44 deletions(-)

diff --git a/Documentation/git-fast-import.adoc b/Documentation/git-fast-import.adoc
index 5201e493345..6afe14b650c 100644
--- a/Documentation/git-fast-import.adoc
+++ b/Documentation/git-fast-import.adoc
@@ -48,9 +48,8 @@ OPTIONS
 
 --stats::
 	Display some basic statistics about the objects fast-import has
-	created, the packfiles they were stored into, and the
-	memory used by fast-import during this run.  Showing this output
-	is currently the default, but can be disabled with --quiet.
+	created, and the memory used by fast-import during this run.  Showing
+	this output is currently the default, but can be disabled with --quiet.
 
 --allow-unsafe-features::
 	Many command-line options can be provided as part of the
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index ab5d2a23448..14e75149415 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -4153,7 +4153,7 @@ static int option_parse_early_allow_unsafe(const struct option *option,
 int cmd_fast_import(int argc,
 		    const char **argv,
 		    const char *prefix,
-		    struct repository *repo)
+		    struct repository *repo UNUSED)
 {
 	struct fast_import_state state;
 
@@ -4346,8 +4346,6 @@ int cmd_fast_import(int argc,
 		fprintf(stderr, "       pools:    %10lu KiB\n", (unsigned long)((tree_entry_allocd + fi_mem_pool.pool_alloc) /1024));
 		fprintf(stderr, "     objects:    %10" PRIuMAX " KiB\n", (alloc_count*sizeof(struct object_entry))/1024);
 		fprintf(stderr, "---------------------------------------------------------------------\n");
-		pack_report(repo);
-		fprintf(stderr, "---------------------------------------------------------------------\n");
 		fprintf(stderr, "\n");
 	}
 
diff --git a/packfile.c b/packfile.c
index b32b4cd75da..5aa9b009174 100644
--- a/packfile.c
+++ b/packfile.c
@@ -36,38 +36,10 @@ char *odb_pack_name(struct repository *r, struct strbuf *buf,
 }
 
 static unsigned int pack_used_ctr;
-static unsigned int pack_mmap_calls;
-static unsigned int peak_pack_open_windows;
-static unsigned int pack_open_windows;
 static unsigned int pack_open_fds;
 static unsigned int pack_max_fds;
-static size_t peak_pack_mapped;
 static size_t pack_mapped;
 
-#define SZ_FMT PRIuMAX
-static inline uintmax_t sz_fmt(size_t s) { return s; }
-
-void pack_report(struct repository *repo)
-{
-	fprintf(stderr,
-		"pack_report: getpagesize()            = %10" SZ_FMT "\n"
-		"pack_report: core.packedGitWindowSize = %10" SZ_FMT "\n"
-		"pack_report: core.packedGitLimit      = %10" SZ_FMT "\n",
-		sz_fmt(getpagesize()),
-		sz_fmt(repo->settings.packed_git_window_size),
-		sz_fmt(repo->settings.packed_git_limit));
-	fprintf(stderr,
-		"pack_report: pack_used_ctr            = %10u\n"
-		"pack_report: pack_mmap_calls          = %10u\n"
-		"pack_report: pack_open_windows        = %10u / %10u\n"
-		"pack_report: pack_mapped              = "
-			"%10" SZ_FMT " / %10" SZ_FMT "\n",
-		pack_used_ctr,
-		pack_mmap_calls,
-		pack_open_windows, peak_pack_open_windows,
-		sz_fmt(pack_mapped), sz_fmt(peak_pack_mapped));
-}
-
 /*
  * Open and mmap the index file at path, perform a couple of
  * consistency checks, then record its information to p.  Return 0 on
@@ -291,7 +263,6 @@ static int unuse_one_window(struct object_database *odb)
 		else
 			lru_p->windows = lru_w->next;
 		free(lru_w);
-		pack_open_windows--;
 		return 1;
 	}
 	return 0;
@@ -307,7 +278,6 @@ void close_pack_windows(struct packed_git *p)
 			    p->pack_name);
 		munmap(w->base, w->len);
 		pack_mapped -= w->len;
-		pack_open_windows--;
 		p->windows = w->next;
 		free(w);
 	}
@@ -665,12 +635,6 @@ unsigned char *use_pack(struct packed_git *p,
 			if (!win->offset && win->len == p->pack_size
 				&& !p->do_not_close)
 				close_pack_fd(p);
-			pack_mmap_calls++;
-			pack_open_windows++;
-			if (pack_mapped > peak_pack_mapped)
-				peak_pack_mapped = pack_mapped;
-			if (pack_open_windows > peak_pack_open_windows)
-				peak_pack_open_windows = pack_open_windows;
 			win->next = p->windows;
 			p->windows = win;
 		}
diff --git a/packfile.h b/packfile.h
index ed3378c4576..bab9931d00e 100644
--- a/packfile.h
+++ b/packfile.h
@@ -217,8 +217,6 @@ int for_each_object_in_pack(struct packed_git *p,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-void pack_report(struct repository *repo);
-
 /*
  * mmap the index file for the specified packfile (if it is not
  * already mmapped).  Return 0 on success.

-- 
2.56.1.831.g720f3f5937.dirty



```

## Patrick Steinhardt, 2026-11-20 12:02

Subject: [PATCH 2/6] packfile: rename lifecycle functions
Message-ID: <20261120-pks-packfile-global-statistics-v1-2-19beb83b145f@pks.im>
URL: https://gitlist.dev/e/20261120-pks-packfile-global-statistics-v1-2-19beb83b145f%40pks.im
In-Reply-To: <20261120-pks-packfile-global-statistics-v1-0-19beb83b145f@pks.im>

```
The packfile subsystem is kind of the wild west when it comes to
function naming. None of the functions have proper subsystem prefixes
and the naming of `struct packed_git` feels like a relict from the past.

Ideally we'd modernize all of that, but that would lead to a huge amount
of churn. Instead, let's improve the subsystem one step at a time and
rename lifecycle-related functions to match modern best practices.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/count-objects.c  |  2 +-
 builtin/fast-import.c    |  4 ++--
 builtin/index-pack.c     |  8 ++++----
 builtin/pack-objects.c   |  2 +-
 builtin/pack-redundant.c |  2 +-
 commit-graph.c           | 10 +++++-----
 connected.c              |  6 +++---
 http-push.c              |  2 +-
 http-walker.c            |  4 ++--
 http.c                   |  2 +-
 midx-write.c             | 18 +++++++++---------
 midx.c                   |  6 +++---
 odb/source-packed.c      | 12 ++++++------
 pack-check.c             |  2 +-
 pack-mtimes.c            |  2 +-
 pack-revindex.c          |  4 ++--
 packfile.c               | 46 +++++++++++++++++++++++-----------------------
 packfile.h               | 29 +++++++++++++++--------------
 repack-geometry.c        |  2 +-
 repack-midx.c            |  2 +-
 t/helper/test-bitmap.c   |  2 +-
 21 files changed, 84 insertions(+), 83 deletions(-)

diff --git a/builtin/count-objects.c b/builtin/count-objects.c
index 3f7b20fde53..09922ae4a70 100644
--- a/builtin/count-objects.c
+++ b/builtin/count-objects.c
@@ -132,7 +132,7 @@ int cmd_count_objects(int argc,
 		repo_for_each_pack(the_repository, p) {
 			if (!p->pack_local)
 				continue;
-			if (open_pack_index(p))
+			if (packfile_open_index(p))
 				continue;
 			packed += p->num_objects;
 			size_pack += p->pack_size + p->index_size;
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index 14e75149415..5f034d39a71 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -906,7 +906,7 @@ static void end_packfile(void)
 		struct branch *b;
 		struct tag *t;
 
-		close_pack_windows(pack_data);
+		packfile_close_windows(pack_data);
 		finalize_hashfile(pack_file, cur_pack_oid.hash, FSYNC_COMPONENT_PACK, 0);
 		fixup_pack_header_footer(the_hash_algo, pack_data->pack_fd,
 					 pack_data->hash, pack_data->pack_name,
@@ -1273,7 +1273,7 @@ static void *gfi_unpack_entry(
 		 * and updating the packfile length ensures we can read
 		 * the newly written data.
 		 */
-		close_pack_windows(p);
+		packfile_close_windows(p);
 		hashflush(pack_file);
 
 		/* We have to offer rawsz bytes additional on the end of
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 0264c18c5f8..71048a5a8af 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1741,12 +1741,12 @@ static void read_v2_anomalous_offsets(struct packed_git *p,
 
 static void read_idx_option(struct pack_idx_option *opts, const char *pack_name)
 {
-	struct packed_git *p = add_packed_git(the_repository, pack_name,
-					      strlen(pack_name), 1);
+	struct packed_git *p = packfile_from_idx(the_repository, pack_name,
+						 strlen(pack_name), 1);
 
 	if (!p)
 		die(_("Cannot open existing pack file '%s'"), pack_name);
-	if (open_pack_index(p))
+	if (packfile_open_index(p))
 		die(_("Cannot open existing pack idx file for '%s'"), pack_name);
 
 	/* Read the attributes from the existing idx file */
@@ -1762,7 +1762,7 @@ static void read_idx_option(struct pack_idx_option *opts, const char *pack_name)
 	 * know we haven't installed this pack (hence we never have
 	 * read anything from it).
 	 */
-	close_pack_index(p);
+	packfile_close_index(p);
 	free(p);
 }
 
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 87c3102e622..9c375811937 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -4724,7 +4724,7 @@ static void loosen_unused_packed_objects(void)
 		if (!p->pack_local || p->pack_keep || p->pack_keep_in_core)
 			continue;
 
-		if (open_pack_index(p))
+		if (packfile_open_index(p))
 			die(_("cannot open pack index"));
 
 		for (i = 0; i < p->num_objects; i++) {
diff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c
index 86749bb7e77..1b437bb6565 100644
--- a/builtin/pack-redundant.c
+++ b/builtin/pack-redundant.c
@@ -546,7 +546,7 @@ static struct pack_list * add_pack(struct packed_git *p)
 	l.pack = p;
 	llist_init(&l.remaining_objects);
 
-	if (open_pack_index(p)) {
+	if (packfile_open_index(p)) {
 		llist_free(l.remaining_objects);
 		return NULL;
 	}
diff --git a/commit-graph.c b/commit-graph.c
index da65cd3deb9..c3b04e8c1f4 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -1950,21 +1950,21 @@ static int fill_oids_from_packs(struct write_commit_graph_context *ctx,
 	for (i = 0; i < pack_indexes->nr; i++) {
 		struct packed_git *p;
 
-		p = add_packed_git(ctx->r, pack_indexes->items[i].string,
-				   strlen(pack_indexes->items[i].string), 1);
+		p = packfile_from_idx(ctx->r, pack_indexes->items[i].string,
+				      strlen(pack_indexes->items[i].string), 1);
 		if (!p) {
 			ret = error(_("error adding pack %s"), pack_indexes->items[i].string);
 			goto cleanup;
 		}
-		if (open_pack_index(p)) {
+		if (packfile_open_index(p)) {
 			ret = error(_("error opening index for %s"), pack_indexes->items[i].string);
-			close_pack(p);
+			packfile_close(p);
 			free(p);
 			goto cleanup;
 		}
 		for_each_object_in_pack(p, add_packed_commits, ctx,
 					ODB_FOR_EACH_OBJECT_PACK_ORDER);
-		close_pack(p);
+		packfile_close(p);
 		free(p);
 	}
 
diff --git a/connected.c b/connected.c
index bfb771e4695..fbacff734b6 100644
--- a/connected.c
+++ b/connected.c
@@ -207,8 +207,8 @@ int check_connected(struct repository *repo,
 		strbuf_add(&idx_file, transport->pack_lockfiles.items[0].string,
 			   base_len);
 		strbuf_addstr(&idx_file, ".idx");
-		new_pack = add_packed_git(repo, idx_file.buf,
-					  idx_file.len, 1);
+		new_pack = packfile_from_idx(repo, idx_file.buf,
+					     idx_file.len, 1);
 		strbuf_release(&idx_file);
 	}
 
@@ -239,7 +239,7 @@ int check_connected(struct repository *repo,
 
 	sigchain_pop(SIGPIPE);
 	if (new_pack) {
-		close_pack(new_pack);
+		packfile_close(new_pack);
 		free(new_pack);
 	}
 	return finish_command(&rev_list) || err;
diff --git a/http-push.c b/http-push.c
index b8f3faaed95..d5335c023c8 100644
--- a/http-push.c
+++ b/http-push.c
@@ -318,7 +318,7 @@ static void start_fetch_packed(struct transfer_request *request)
 		release_request(request);
 		return;
 	}
-	close_pack_index(target);
+	packfile_close_index(target);
 	request->target = target;
 
 	fprintf(stderr,	"Fetching pack %s\n",
diff --git a/http-walker.c b/http-walker.c
index 4f50dd9b8d0..de43d9cf673 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -435,7 +435,7 @@ static int http_fetch_pack(struct walker *walker, struct alt_base *repo,
 	target = packfile_list_find_oid(repo->packs.head, oid);
 	if (!target)
 		return -1;
-	close_pack_index(target);
+	packfile_close_index(target);
 
 	if (walker->get_verbosely) {
 		fprintf(stderr, "Getting pack %s\n",
@@ -588,7 +588,7 @@ static void cleanup(struct walker *walker)
 			alt_next = alt->next;
 
 			for (e = alt->packs.head; e; e = e->next) {
-				close_pack(e->pack);
+				packfile_close(e->pack);
 				free(e->pack);
 			}
 			packfile_list_clear(&alt->packs);
diff --git a/http.c b/http.c
index d79678d01f0..e86bebdffb2 100644
--- a/http.c
+++ b/http.c
@@ -2658,7 +2658,7 @@ static int fetch_and_setup_pack_index(struct packfile_list *packs,
 
 	ret = verify_pack_index(new_pack);
 
-	close_pack_index(new_pack);
+	packfile_close_index(new_pack);
 	free(tmp_idx);
 	if (ret) {
 		free(new_pack);
diff --git a/midx-write.c b/midx-write.c
index 3038bbfad26..d5be4e40843 100644
--- a/midx-write.c
+++ b/midx-write.c
@@ -172,17 +172,17 @@ static void add_pack_to_midx(const char *full_path, size_t full_path_len,
 			return;
 
 		ALLOC_GROW(ctx->info, ctx->nr + 1, ctx->alloc);
-		p = add_packed_git(ctx->repo, full_path, full_path_len, 0);
+		p = packfile_from_idx(ctx->repo, full_path, full_path_len, 0);
 		if (!p) {
 			warning(_("failed to add packfile '%s'"),
 				full_path);
 			return;
 		}
 
-		if (open_pack_index(p)) {
+		if (packfile_open_index(p)) {
 			warning(_("failed to open pack-index '%s'"),
 				full_path);
-			close_pack(p);
+			packfile_close(p);
 			free(p);
 			return;
 		}
@@ -1464,7 +1464,7 @@ static int write_midx_internal(struct write_midx_opts *opts)
 		 * Ignore failures as they can be expected and are not
 		 * fatal during this selection time.
 		 */
-		open_pack_index(oldest);
+		packfile_open_index(oldest);
 
 		if (opts->packs_to_drop && opts->packs_to_drop->nr)
 			BUG("cannot write a MIDX bitmap during expiration");
@@ -1480,7 +1480,7 @@ static int write_midx_internal(struct write_midx_opts *opts)
 
 			if (!oldest->num_objects || p->mtime < oldest->mtime) {
 				oldest = p;
-				open_pack_index(oldest);
+				packfile_open_index(oldest);
 				ctx.preferred_pack_idx = i;
 			}
 		}
@@ -1505,7 +1505,7 @@ static int write_midx_internal(struct write_midx_opts *opts)
 	if (ctx.preferred_pack_idx != NO_PREFERRED_PACK) {
 		struct packed_git *preferred = ctx.info[ctx.preferred_pack_idx].p;
 
-		if (open_pack_index(preferred))
+		if (packfile_open_index(preferred))
 			die(_("failed to open preferred pack %s"),
 			    ctx.info[ctx.preferred_pack_idx].pack_name);
 
@@ -1828,7 +1828,7 @@ static int write_midx_internal(struct write_midx_opts *opts)
 cleanup:
 	for (size_t i = 0; i < ctx.nr; i++) {
 		if (ctx.info[i].p) {
-			close_pack(ctx.info[i].p);
+			packfile_close(ctx.info[i].p);
 			free(ctx.info[i].p);
 		}
 		free(ctx.info[i].pack_name);
@@ -1944,7 +1944,7 @@ int expire_midx_packs(struct odb_source_packed *source, unsigned flags)
 			continue;
 
 		pack_name = xstrdup(m->packs[i]->pack_name);
-		close_pack(m->packs[i]);
+		packfile_close(m->packs[i]);
 
 		string_list_insert(&packs_to_drop, m->pack_names[i]);
 		unlink_pack_path(pack_name, 0);
@@ -2000,7 +2000,7 @@ static int want_included_pack(struct multi_pack_index *m,
 		return 0;
 	if (p->is_cruft)
 		return 0;
-	if (open_pack_index(p) || !p->num_objects)
+	if (packfile_open_index(p) || !p->num_objects)
 		return 0;
 	return 1;
 }
diff --git a/midx.c b/midx.c
index 2d4feeeadeb..d2b8a3a4a5e 100644
--- a/midx.c
+++ b/midx.c
@@ -1023,8 +1023,8 @@ int verify_midx_file(struct odb_source_packed *source, unsigned flags)
 			uint32_t pack_int_id = pairs[i-1].pack_int_id;
 			struct packed_git *p = nth_midxed_pack(m, pack_int_id);
 
-			close_pack_fd(p);
-			close_pack_index(p);
+			packfile_close_fd(p);
+			packfile_close_index(p);
 		}
 
 		nth_midxed_object_oid(&oid, m, pairs[i].pos);
@@ -1035,7 +1035,7 @@ int verify_midx_file(struct odb_source_packed *source, unsigned flags)
 			continue;
 		}
 
-		if (open_pack_index(e.p)) {
+		if (packfile_open_index(e.p)) {
 			midx_report(_("failed to load pack-index for packfile %s"),
 				    e.p->pack_name);
 			break;
diff --git a/odb/source-packed.c b/odb/source-packed.c
index 7fcbbb4fe9e..87e2dcdf52e 100644
--- a/odb/source-packed.c
+++ b/odb/source-packed.c
@@ -351,7 +351,7 @@ static int odb_source_packed_for_each_prefixed_object(
 		if (should_exclude_pack(e->pack, opts->flags))
 			continue;
 
-		if (open_pack_index(e->pack)) {
+		if (packfile_open_index(e->pack)) {
 			pack_errors = true;
 			continue;
 		}
@@ -459,7 +459,7 @@ static int odb_source_packed_for_each_object(struct odb_source *source,
 		if (bitmap && bitmap_index_contains_pack(bitmap, p))
 			continue;
 
-		if (open_pack_index(p)) {
+		if (packfile_open_index(p)) {
 			pack_errors = 1;
 			continue;
 		}
@@ -498,7 +498,7 @@ static int odb_source_packed_count_objects(struct odb_source *source,
 	for (e = packfile_store_get_packs(packed); e; e = e->next) {
 		if (e->pack->multi_pack_index)
 			continue;
-		if (open_pack_index(e->pack)) {
+		if (packfile_open_index(e->pack)) {
 			ret = -1;
 			goto out;
 		}
@@ -615,7 +615,7 @@ static int odb_source_packed_find_abbrev_len(struct odb_source *source,
 	for (e = packfile_store_get_packs(packed); e; e = e->next) {
 		if (e->pack->multi_pack_index)
 			continue;
-		if (open_pack_index(e->pack) || !e->pack->num_objects)
+		if (packfile_open_index(e->pack) || !e->pack->num_objects)
 			continue;
 
 		find_abbrev_len_for_pack(e->pack, oid, min_len, &min_len);
@@ -847,7 +847,7 @@ static void odb_source_packed_close(struct odb_source *source)
 	for (struct packfile_list_entry *e = packed->packs.head; e; e = e->next) {
 		if (e->pack->do_not_close)
 			BUG("want to close pack marked 'do-not-close'");
-		close_pack(e->pack);
+		packfile_close(e->pack);
 	}
 	if (packed->midx)
 		close_midx(packed->midx);
@@ -880,7 +880,7 @@ static int verify_packs(struct odb_source_packed *source,
 
 	if (opts->flags & ODB_FSCK_PROGRESS) {
 		for (e = packfile_store_get_packs(source); e; e = e->next) {
-			if (open_pack_index(e->pack))
+			if (packfile_open_index(e->pack))
 				continue;
 			total += e->pack->num_objects;
 		}
diff --git a/pack-check.c b/pack-check.c
index 605811b7b09..e6663bb8e16 100644
--- a/pack-check.c
+++ b/pack-check.c
@@ -184,7 +184,7 @@ int verify_pack_index(struct packed_git *p)
 {
 	int err = 0;
 
-	if (open_pack_index(p))
+	if (packfile_open_index(p))
 		return error("packfile %s index not opened", p->pack_name);
 
 	/* Verify SHA1 sum of the index file */
diff --git a/pack-mtimes.c b/pack-mtimes.c
index 8e1f2dec0ef..8ce9c808e7f 100644
--- a/pack-mtimes.c
+++ b/pack-mtimes.c
@@ -106,7 +106,7 @@ int load_pack_mtimes(struct packed_git *p)
 	if (p->mtimes_map)
 		return ret; /* already loaded */
 
-	ret = open_pack_index(p);
+	ret = packfile_open_index(p);
 	if (ret < 0)
 		goto cleanup;
 
diff --git a/pack-revindex.c b/pack-revindex.c
index 62387ae6320..5de5e4064ba 100644
--- a/pack-revindex.c
+++ b/pack-revindex.c
@@ -176,7 +176,7 @@ static int create_pack_revindex_in_memory(struct packed_git *p)
 	if (git_env_bool(GIT_TEST_REV_INDEX_DIE_IN_MEMORY, 0))
 		die("dying as requested by '%s'",
 		    GIT_TEST_REV_INDEX_DIE_IN_MEMORY);
-	if (open_pack_index(p))
+	if (packfile_open_index(p))
 		return -1;
 	create_pack_revindex(p);
 	return 0;
@@ -281,7 +281,7 @@ int load_pack_revindex_from_disk(struct packed_git *p)
 	if (p->revindex_data)
 		return 0;
 
-	if (open_pack_index(p))
+	if (packfile_open_index(p))
 		return -1;
 
 	revindex_name = pack_revindex_filename(p);
diff --git a/packfile.c b/packfile.c
index 5aa9b009174..95a97c302c7 100644
--- a/packfile.c
+++ b/packfile.c
@@ -155,7 +155,7 @@ int load_idx(const char *path, const unsigned int hashsz, void *idx_map,
 	return 0;
 }
 
-int open_pack_index(struct packed_git *p)
+int packfile_open_index(struct packed_git *p)
 {
 	char *idx_name;
 	size_t len;
@@ -177,7 +177,7 @@ uint32_t get_pack_fanout(struct packed_git *p, uint32_t value)
 	const uint32_t *level1_ofs = p->index_data;
 
 	if (!level1_ofs) {
-		if (open_pack_index(p))
+		if (packfile_open_index(p))
 			return 0;
 		level1_ofs = p->index_data;
 	}
@@ -189,7 +189,7 @@ uint32_t get_pack_fanout(struct packed_git *p, uint32_t value)
 	return ntohl(level1_ofs[value]);
 }
 
-static struct packed_git *alloc_packed_git(struct repository *r, int extra)
+static struct packed_git *packfile_new(struct repository *r, int extra)
 {
 	struct packed_git *p = xmalloc(st_add(sizeof(*p), extra));
 	memset(p, 0, sizeof(*p));
@@ -211,7 +211,7 @@ struct packed_git *parse_pack_index(struct repository *r, unsigned char *sha1,
 {
 	char *path = pack_path_from_idx(idx_path);
 	size_t alloc = st_add(strlen(path), 1);
-	struct packed_git *p = alloc_packed_git(r, alloc);
+	struct packed_git *p = packfile_new(r, alloc);
 
 	memcpy(p->pack_name, path, alloc); /* includes NUL */
 	free(path);
@@ -268,7 +268,7 @@ static int unuse_one_window(struct object_database *odb)
 	return 0;
 }
 
-void close_pack_windows(struct packed_git *p)
+void packfile_close_windows(struct packed_git *p)
 {
 	while (p->windows) {
 		struct pack_window *w = p->windows;
@@ -283,7 +283,7 @@ void close_pack_windows(struct packed_git *p)
 	}
 }
 
-int close_pack_fd(struct packed_git *p)
+int packfile_close_fd(struct packed_git *p)
 {
 	if (p->pack_fd < 0)
 		return 0;
@@ -295,7 +295,7 @@ int close_pack_fd(struct packed_git *p)
 	return 1;
 }
 
-void close_pack_index(struct packed_git *p)
+void packfile_close_index(struct packed_git *p)
 {
 	if (p->index_data) {
 		munmap((void *)p->index_data, p->index_size);
@@ -424,7 +424,7 @@ static int close_one_pack(struct repository *r)
 	}
 
 	if (lru_p)
-		return close_pack_fd(lru_p);
+		return packfile_close_fd(lru_p);
 
 	return 0;
 }
@@ -493,7 +493,7 @@ static int open_packed_git_1(struct packed_git *p)
 	ssize_t read_result;
 	const unsigned hashsz = p->repo->hash_algo->rawsz;
 
-	if (open_pack_index(p))
+	if (packfile_open_index(p))
 		return error("packfile %s index unavailable", p->pack_name);
 
 	if (!pack_max_fds) {
@@ -557,7 +557,7 @@ static int open_packed_git(struct packed_git *p)
 {
 	if (!open_packed_git_1(p))
 		return 0;
-	close_pack_fd(p);
+	packfile_close_fd(p);
 	return -1;
 }
 
@@ -634,7 +634,7 @@ unsigned char *use_pack(struct packed_git *p,
 					  p->pack_name, mmap_os_err());
 			if (!win->offset && win->len == p->pack_size
 				&& !p->do_not_close)
-				close_pack_fd(p);
+				packfile_close_fd(p);
 			win->next = p->windows;
 			p->windows = win;
 		}
@@ -659,8 +659,8 @@ void unuse_pack(struct pack_window **w_cursor)
 	}
 }
 
-struct packed_git *add_packed_git(struct repository *r, const char *path,
-				  size_t path_len, int local)
+struct packed_git *packfile_from_idx(struct repository *r, const char *path,
+				     size_t path_len, int local)
 {
 	struct stat st;
 	size_t alloc;
@@ -679,7 +679,7 @@ struct packed_git *add_packed_git(struct repository *r, const char *path,
 	 * the use xsnprintf double-checks that)
 	 */
 	alloc = st_add3(path_len, strlen(".promisor"), 1);
-	p = alloc_packed_git(r, alloc);
+	p = packfile_new(r, alloc);
 	memcpy(p->pack_name, path, path_len);
 
 	/*
@@ -753,8 +753,8 @@ struct packed_git *packfile_store_load_pack(struct odb_source_packed *store,
 
 	p = strmap_get(&store->packs_by_path, key.buf);
 	if (!p) {
-		p = add_packed_git(store->base.odb->repo, idx_path,
-				   strlen(idx_path), local);
+		p = packfile_from_idx(store->base.odb->repo, idx_path,
+				      strlen(idx_path), local);
 		if (p)
 			packfile_store_add_pack(store, p);
 	}
@@ -875,7 +875,7 @@ size_t get_size_from_delta(struct packed_git *p,
 		 * counter is incremented before window reading and checked
 		 * before window disposal.
 		 *
-		 * Other worrying sections could be the call to close_pack_fd(),
+		 * Other worrying sections could be the call to packfile_close_fd(),
 		 * which can close packs even with in-use windows, and to
 		 * odb_reprepare(). Regarding the former, mmap doc says:
 		 * "closing the file descriptor does not unmap the region". And
@@ -1227,11 +1227,11 @@ static void delta_base_cache_evict_entry(struct packed_git *p)
 	}
 }
 
-void close_pack(struct packed_git *p)
+void packfile_close(struct packed_git *p)
 {
-	close_pack_windows(p);
-	close_pack_fd(p);
-	close_pack_index(p);
+	packfile_close_windows(p);
+	packfile_close_fd(p);
+	packfile_close_index(p);
 	close_pack_revindex(p);
 	close_pack_mtimes(p);
 	oidset_clear(&p->bad_objects);
@@ -1725,7 +1725,7 @@ int nth_packed_object_id(struct object_id *oid,
 	const unsigned char *index = p->index_data;
 	const unsigned int hashsz = p->repo->hash_algo->rawsz;
 	if (!index) {
-		if (open_pack_index(p))
+		if (packfile_open_index(p))
 			return -1;
 		index = p->index_data;
 	}
@@ -1783,7 +1783,7 @@ off_t find_pack_entry_one(const struct object_id *oid,
 	uint32_t result;
 
 	if (!index) {
-		if (open_pack_index(p))
+		if (packfile_open_index(p))
 			return 0;
 	}
 
diff --git a/packfile.h b/packfile.h
index bab9931d00e..f817bc69e29 100644
--- a/packfile.h
+++ b/packfile.h
@@ -180,7 +180,7 @@ const char *pack_basename(struct packed_git *p);
  * You probably don't want to use this function! It skips most of the normal
  * sanity checks (including whether we even have the matching .pack file),
  * and does not add the resulting packed_git struct to the internal list of
- * packs. You probably want add_packed_git() instead.
+ * packs. You probably want packfile_from_idx() instead.
  */
 struct packed_git *parse_pack_index(struct repository *r, unsigned char *sha1,
 				    const char *idx_path);
@@ -217,32 +217,33 @@ int for_each_object_in_pack(struct packed_git *p,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
+uint32_t get_pack_fanout(struct packed_git *p, uint32_t value);
+
+struct object_database;
+
+struct packed_git *packfile_from_idx(struct repository *r, const char *path,
+				     size_t path_len, int local);
+
 /*
  * mmap the index file for the specified packfile (if it is not
  * already mmapped).  Return 0 on success.
  */
-int open_pack_index(struct packed_git *);
+int packfile_open_index(struct packed_git *);
+
+void packfile_close(struct packed_git *);
+int packfile_close_fd(struct packed_git *p);
+void packfile_close_windows(struct packed_git *);
 
 /*
  * munmap the index file for the specified packfile (if it is
  * currently mmapped).
  */
-void close_pack_index(struct packed_git *);
-
-int close_pack_fd(struct packed_git *p);
-
-uint32_t get_pack_fanout(struct packed_git *p, uint32_t value);
-
-struct object_database;
+void packfile_close_index(struct packed_git *);
 
 unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t,
 			size_t *);
-void close_pack_windows(struct packed_git *);
-void close_pack(struct packed_git *);
 void unuse_pack(struct pack_window **);
 void clear_delta_base_cache(void);
-struct packed_git *add_packed_git(struct repository *r, const char *path,
-				  size_t path_len, int local);
 
 /*
  * Unlink the .pack and associated extension files.
@@ -345,7 +346,7 @@ int is_promisor_object(struct repository *r, const struct object_id *oid);
  *
  * This function should not be used directly. It is exposed here only so that we
  * have a convenient entry-point for fuzz testing. For real uses, you should
- * probably use open_pack_index() instead.
+ * probably use packfile_open_index() instead.
  */
 int load_idx(const char *path, const unsigned int hashsz, void *idx_map,
 	     size_t idx_size, struct packed_git *p);
diff --git a/repack-geometry.c b/repack-geometry.c
index b541c34af5f..671a3dfe6a3 100644
--- a/repack-geometry.c
+++ b/repack-geometry.c
@@ -9,7 +9,7 @@
 
 static uint32_t pack_geometry_weight(struct packed_git *p)
 {
-	if (open_pack_index(p))
+	if (packfile_open_index(p))
 		die(_("cannot open index for %s"), p->pack_name);
 	return p->num_objects;
 }
diff --git a/repack-midx.c b/repack-midx.c
index 72810034733..7051a0d0637 100644
--- a/repack-midx.c
+++ b/repack-midx.c
@@ -576,7 +576,7 @@ static int midx_compaction_step_include_packs(struct midx_compaction_step *step,
 		strbuf_reset(&path);
 		strbuf_addf(&path, "%s/%s", opts->packdir, item->string);
 		p = packfile_store_load_pack(files->dirs->packed, path.buf, 1);
-		if (!p || open_pack_index(p)) {
+		if (!p || packfile_open_index(p)) {
 			ret = error(_("cannot open index for %s"), path.buf);
 			goto out;
 		}
diff --git a/t/helper/test-bitmap.c b/t/helper/test-bitmap.c
index 8547ef67e24..e3a3c8ff5cc 100644
--- a/t/helper/test-bitmap.c
+++ b/t/helper/test-bitmap.c
@@ -87,7 +87,7 @@ static int bitmap_write(const char *basename)
 	if (!p)
 		die("could not find pack '%s'", basename);
 
-	if (open_pack_index(p))
+	if (packfile_open_index(p))
 		die("cannot open pack index for '%s'", p->pack_name);
 
 	prepare_packing_data(the_repository, &packed);

-- 
2.56.1.831.g720f3f5937.dirty



```

## Patrick Steinhardt, 2026-11-20 12:02

Subject: [PATCH 3/6] packfile: inline `open_packed_git_1()`
Message-ID: <20261120-pks-packfile-global-statistics-v1-3-19beb83b145f@pks.im>
URL: https://gitlist.dev/e/20261120-pks-packfile-global-statistics-v1-3-19beb83b145f%40pks.im
In-Reply-To: <20261120-pks-packfile-global-statistics-v1-0-19beb83b145f@pks.im>

```
The function `open_packed_git()` is responsible for opening a packfile.
The majority of work is not performed by it though, but instead by its
`_1()` sibling. But this sibling really only exists because we thought
it too hard to handle errors properly and close the packfile's file
descriptor on an error path. So `_1()` doesn't handle those, and instead
defers to its only caller to close the file descriptor in the error path
again.

Fix this design smell by introducing a common exit path that knows to
close the file descriptor on error. This allows us to inline the call to
`open_packed_git_1()`.

While at it, rename the function to `packfile_open_pack()` to match
modern best practices. Besides that, it also gives the caller a better
idea of _what_ is being opened, as we also have functions that open
other files that belong to a packfile.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 packfile.c | 115 +++++++++++++++++++++++++++++++++++++------------------------
 1 file changed, 70 insertions(+), 45 deletions(-)

diff --git a/packfile.c b/packfile.c
index 95a97c302c7..9b03258311f 100644
--- a/packfile.c
+++ b/packfile.c
@@ -480,11 +480,7 @@ const char *pack_basename(struct packed_git *p)
 	return ret;
 }
 
-/*
- * Do not call this directly as this leaks p->pack_fd on error return;
- * call open_packed_git() instead.
- */
-static int open_packed_git_1(struct packed_git *p)
+static int packfile_open_pack(struct packed_git *p)
 {
 	struct stat st;
 	struct pack_header hdr;
@@ -492,9 +488,12 @@ static int open_packed_git_1(struct packed_git *p)
 	unsigned char *idx_hash;
 	ssize_t read_result;
 	const unsigned hashsz = p->repo->hash_algo->rawsz;
+	int ret;
 
-	if (packfile_open_index(p))
-		return error("packfile %s index unavailable", p->pack_name);
+	if (packfile_open_index(p)) {
+		ret = error("packfile %s index unavailable", p->pack_name);
+		goto out;
+	}
 
 	if (!pack_max_fds) {
 		unsigned int max_fds = get_max_fd_limit();
@@ -510,55 +509,81 @@ static int open_packed_git_1(struct packed_git *p)
 		; /* nothing */
 
 	p->pack_fd = git_open(p->pack_name);
-	if (p->pack_fd < 0 || fstat(p->pack_fd, &st))
-		return -1;
+	if (p->pack_fd < 0 || fstat(p->pack_fd, &st)) {
+		ret = -1;
+		goto out;
+	}
 	pack_open_fds++;
 
 	/* If we created the struct before we had the pack we lack size. */
 	if (!p->pack_size) {
-		if (!S_ISREG(st.st_mode))
-			return error("packfile %s not a regular file", p->pack_name);
+		if (!S_ISREG(st.st_mode)) {
+			ret = error("packfile %s not a regular file", p->pack_name);
+			goto out;
+		}
+
 		p->pack_size = st.st_size;
-	} else if (p->pack_size != st.st_size)
-		return error("packfile %s size changed", p->pack_name);
+	} else if (p->pack_size != st.st_size) {
+		ret = error("packfile %s size changed", p->pack_name);
+		goto out;
+	}
 
 	/* Verify we recognize this pack file format. */
 	read_result = read_in_full(p->pack_fd, &hdr, sizeof(hdr));
-	if (read_result < 0)
-		return error_errno("error reading from %s", p->pack_name);
-	if (read_result != sizeof(hdr))
-		return error("file %s is far too short to be a packfile", p->pack_name);
-	if (hdr.hdr_signature != htonl(PACK_SIGNATURE))
-		return error("file %s is not a GIT packfile", p->pack_name);
-	if (!pack_version_ok(hdr.hdr_version))
-		return error("packfile %s is version %"PRIu32" and not"
-			" supported (try upgrading GIT to a newer version)",
-			p->pack_name, ntohl(hdr.hdr_version));
+	if (read_result < 0) {
+		ret = error_errno("error reading from %s", p->pack_name);
+		goto out;
+	}
+
+	if (read_result != sizeof(hdr)) {
+		ret = error("file %s is far too short to be a packfile", p->pack_name);
+		goto out;
+	}
+
+	if (hdr.hdr_signature != htonl(PACK_SIGNATURE)) {
+		ret = error("file %s is not a GIT packfile", p->pack_name);
+		goto out;
+	}
+
+	if (!pack_version_ok(hdr.hdr_version)) {
+		ret = error("packfile %s is version %"PRIu32" and not"
+			    " supported (try upgrading GIT to a newer version)",
+			    p->pack_name, ntohl(hdr.hdr_version));
+		goto out;
+	}
 
 	/* Verify the pack matches its index. */
-	if (p->num_objects != ntohl(hdr.hdr_entries))
-		return error("packfile %s claims to have %"PRIu32" objects"
-			     " while index indicates %"PRIu32" objects",
-			     p->pack_name, ntohl(hdr.hdr_entries),
-			     p->num_objects);
+	if (p->num_objects != ntohl(hdr.hdr_entries)) {
+		ret = error("packfile %s claims to have %"PRIu32" objects"
+			    " while index indicates %"PRIu32" objects",
+			    p->pack_name, ntohl(hdr.hdr_entries),
+			    p->num_objects);
+		goto out;
+	}
+
 	read_result = pread_in_full(p->pack_fd, hash, hashsz,
 					p->pack_size - hashsz);
-	if (read_result < 0)
-		return error_errno("error reading from %s", p->pack_name);
-	if (read_result != hashsz)
-		return error("packfile %s signature is unavailable", p->pack_name);
+	if (read_result < 0) {
+		ret = error_errno("error reading from %s", p->pack_name);
+		goto out;
+	}
+	if (read_result != hashsz) {
+		ret = error("packfile %s signature is unavailable", p->pack_name);
+		goto out;
+	}
+
 	idx_hash = ((unsigned char *)p->index_data) + p->index_size - hashsz * 2;
-	if (!hasheq(hash, idx_hash, p->repo->hash_algo))
-		return error("packfile %s does not match index", p->pack_name);
-	return 0;
-}
+	if (!hasheq(hash, idx_hash, p->repo->hash_algo)) {
+		ret = error("packfile %s does not match index", p->pack_name);
+		goto out;
+	}
 
-static int open_packed_git(struct packed_git *p)
-{
-	if (!open_packed_git_1(p))
-		return 0;
-	packfile_close_fd(p);
-	return -1;
+	ret = 0;
+
+out:
+	if (ret < 0)
+		packfile_close_fd(p);
+	return ret;
 }
 
 static int in_window(struct repository *r, struct pack_window *win,
@@ -587,7 +612,7 @@ unsigned char *use_pack(struct packed_git *p,
 	 * hash, and the in_window function above wouldn't match
 	 * don't allow an offset too close to the end of the file.
 	 */
-	if (!p->pack_size && p->pack_fd == -1 && open_packed_git(p))
+	if (!p->pack_size && p->pack_fd == -1 && packfile_open_pack(p))
 		die("packfile %s cannot be accessed", p->pack_name);
 	if (offset > (p->pack_size - p->repo->hash_algo->rawsz))
 		die("offset beyond end of packfile (truncated pack?)");
@@ -612,7 +637,7 @@ unsigned char *use_pack(struct packed_git *p,
 
 			window_align = settings->packed_git_window_size / 2;
 
-			if (p->pack_fd == -1 && open_packed_git(p))
+			if (p->pack_fd == -1 && packfile_open_pack(p))
 				die("packfile %s cannot be accessed", p->pack_name);
 
 			CALLOC_ARRAY(win, 1);
@@ -1810,7 +1835,7 @@ int is_pack_valid(struct packed_git *p)
 	}
 
 	/* Force the pack to open to prove its valid. */
-	return !open_packed_git(p);
+	return !packfile_open_pack(p);
 }
 
 int packfile_fill_entry(struct packed_git *p,

-- 
2.56.1.831.g720f3f5937.dirty



```

## Patrick Steinhardt, 2026-11-20 12:02

Subject: [PATCH 4/6] packfile: introduce function to free a packfile
Message-ID: <20261120-pks-packfile-global-statistics-v1-4-19beb83b145f@pks.im>
URL: https://gitlist.dev/e/20261120-pks-packfile-global-statistics-v1-4-19beb83b145f%40pks.im
In-Reply-To: <20261120-pks-packfile-global-statistics-v1-0-19beb83b145f@pks.im>

```
We have a bunch of callsites that want to free a packfile, which
requires them to first close it followed by a call to free(3p). This
feels rather leaky as an abstraction.

Introduce `packfile_free()` to handle this properly. This function will
also be extended in a subsequent patch to maintain a list of all packs.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/index-pack.c | 10 +---------
 commit-graph.c       |  6 ++----
 connected.c          |  6 ++----
 http-walker.c        |  6 ++----
 http.c               |  2 +-
 midx-write.c         |  9 +++------
 odb/source-packed.c  |  2 +-
 packfile.c           | 10 ++++++++--
 packfile.h           |  1 +
 9 files changed, 21 insertions(+), 31 deletions(-)

diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 71048a5a8af..663375fbb36 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1755,15 +1755,7 @@ static void read_idx_option(struct pack_idx_option *opts, const char *pack_name)
 	if (opts->version == 2)
 		read_v2_anomalous_offsets(p, opts);
 
-	/*
-	 * Get rid of the idx file as we do not need it anymore.
-	 * NEEDSWORK: extract this bit from free_pack_by_name() in
-	 * object-file.c, perhaps?  It shouldn't matter very much as we
-	 * know we haven't installed this pack (hence we never have
-	 * read anything from it).
-	 */
-	packfile_close_index(p);
-	free(p);
+	packfile_free(p);
 }
 
 static void show_pack_info(int stat_only)
diff --git a/commit-graph.c b/commit-graph.c
index c3b04e8c1f4..03df5fb7c1f 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -1958,14 +1958,12 @@ static int fill_oids_from_packs(struct write_commit_graph_context *ctx,
 		}
 		if (packfile_open_index(p)) {
 			ret = error(_("error opening index for %s"), pack_indexes->items[i].string);
-			packfile_close(p);
-			free(p);
+			packfile_free(p);
 			goto cleanup;
 		}
 		for_each_object_in_pack(p, add_packed_commits, ctx,
 					ODB_FOR_EACH_OBJECT_PACK_ORDER);
-		packfile_close(p);
-		free(p);
+		packfile_free(p);
 	}
 
 cleanup:
diff --git a/connected.c b/connected.c
index fbacff734b6..78f08b41383 100644
--- a/connected.c
+++ b/connected.c
@@ -238,9 +238,7 @@ int check_connected(struct repository *repo,
 		err = error_errno(_("failed to close rev-list's stdin"));
 
 	sigchain_pop(SIGPIPE);
-	if (new_pack) {
-		packfile_close(new_pack);
-		free(new_pack);
-	}
+	if (new_pack)
+		packfile_free(new_pack);
 	return finish_command(&rev_list) || err;
 }
diff --git a/http-walker.c b/http-walker.c
index de43d9cf673..616a5fd234d 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -587,10 +587,8 @@ static void cleanup(struct walker *walker)
 
 			alt_next = alt->next;
 
-			for (e = alt->packs.head; e; e = e->next) {
-				packfile_close(e->pack);
-				free(e->pack);
-			}
+			for (e = alt->packs.head; e; e = e->next)
+				packfile_free(e->pack);
 			packfile_list_clear(&alt->packs);
 
 			free(alt->base);
diff --git a/http.c b/http.c
index e86bebdffb2..d518683e15e 100644
--- a/http.c
+++ b/http.c
@@ -2661,7 +2661,7 @@ static int fetch_and_setup_pack_index(struct packfile_list *packs,
 	packfile_close_index(new_pack);
 	free(tmp_idx);
 	if (ret) {
-		free(new_pack);
+		packfile_free(new_pack);
 		return -1;
 	}
 
diff --git a/midx-write.c b/midx-write.c
index d5be4e40843..3e227bb5e57 100644
--- a/midx-write.c
+++ b/midx-write.c
@@ -182,8 +182,7 @@ static void add_pack_to_midx(const char *full_path, size_t full_path_len,
 		if (packfile_open_index(p)) {
 			warning(_("failed to open pack-index '%s'"),
 				full_path);
-			packfile_close(p);
-			free(p);
+			packfile_free(p);
 			return;
 		}
 
@@ -1827,10 +1826,8 @@ static int write_midx_internal(struct write_midx_opts *opts)
 
 cleanup:
 	for (size_t i = 0; i < ctx.nr; i++) {
-		if (ctx.info[i].p) {
-			packfile_close(ctx.info[i].p);
-			free(ctx.info[i].p);
-		}
+		if (ctx.info[i].p)
+			packfile_free(ctx.info[i].p);
 		free(ctx.info[i].pack_name);
 	}
 
diff --git a/odb/source-packed.c b/odb/source-packed.c
index 87e2dcdf52e..866b4e890b6 100644
--- a/odb/source-packed.c
+++ b/odb/source-packed.c
@@ -862,7 +862,7 @@ static void odb_source_packed_free(struct odb_source *source)
 
 	odb_source_close(source);
 	for (struct packfile_list_entry *e = packed->packs.head; e; e = e->next)
-		free(e->pack);
+		packfile_free(e->pack);
 	packfile_list_clear(&packed->packs);
 
 	strmap_clear(&packed->packs_by_path, 0);
diff --git a/packfile.c b/packfile.c
index 9b03258311f..077b0395a79 100644
--- a/packfile.c
+++ b/packfile.c
@@ -217,7 +217,7 @@ struct packed_git *parse_pack_index(struct repository *r, unsigned char *sha1,
 	free(path);
 	hashcpy(p->hash, sha1, p->repo->hash_algo);
 	if (check_packed_git_idx(idx_path, p)) {
-		free(p);
+		packfile_free(p);
 		return NULL;
 	}
 
@@ -732,7 +732,7 @@ struct packed_git *packfile_from_idx(struct repository *r, const char *path,
 
 	xsnprintf(p->pack_name + path_len, alloc - path_len, ".pack");
 	if (stat(p->pack_name, &st) || !S_ISREG(st.st_mode)) {
-		free(p);
+		packfile_free(p);
 		return NULL;
 	}
 
@@ -752,6 +752,12 @@ struct packed_git *packfile_from_idx(struct repository *r, const char *path,
 	return p;
 }
 
+void packfile_free(struct packed_git *p)
+{
+	packfile_close(p);
+	free(p);
+}
+
 void packfile_store_add_pack(struct odb_source_packed *store,
 			     struct packed_git *pack)
 {
diff --git a/packfile.h b/packfile.h
index f817bc69e29..f314a55a904 100644
--- a/packfile.h
+++ b/packfile.h
@@ -223,6 +223,7 @@ struct object_database;
 
 struct packed_git *packfile_from_idx(struct repository *r, const char *path,
 				     size_t path_len, int local);
+void packfile_free(struct packed_git *p);
 
 /*
  * mmap the index file for the specified packfile (if it is not

-- 
2.56.1.831.g720f3f5937.dirty



```

## Patrick Steinhardt, 2026-11-20 12:02

Subject: [PATCH 5/6] packfile: maintain process-wide limits globally
Message-ID: <20261120-pks-packfile-global-statistics-v1-5-19beb83b145f@pks.im>
URL: https://gitlist.dev/e/20261120-pks-packfile-global-statistics-v1-5-19beb83b145f%40pks.im
In-Reply-To: <20261120-pks-packfile-global-statistics-v1-0-19beb83b145f@pks.im>

```
Our packfile subsystem has two limits that are supposed to apply
globally across all open packfiles:

  - `pack_max_fds` is the maximum number of open file descriptors for
    packfiles. When we exceed that limit, we close least-recently-used
    packs until we're below it again.

  - "core.packedGitLimit" is the maximum number of packfile data that
    may be mapped into memory. When we exceed that limit, we close the
    least-recently-used windows into our packfiles.

Those limits are supposed to be process-wide, so even across multiple
repositories. But our current implementation will only consider
packfiles from a given repository or object database, which may not even
have any packfiles open when the logic kicks in. Furthermore, the logic
assumes that we're using the "files" backend and thus unconditionally
downcasts the object database's backend to it.

Introduce a global list of all packfiles. This list is completely
independent of the owning repository or object database, and it is
agnostic of the actual object database backend that's in use. Use it to
maintain the above two limits.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 packfile.c | 29 +++++++++++++----------------
 1 file changed, 13 insertions(+), 16 deletions(-)

diff --git a/packfile.c b/packfile.c
index 077b0395a79..0d5647c74c8 100644
--- a/packfile.c
+++ b/packfile.c
@@ -35,6 +35,7 @@ char *odb_pack_name(struct repository *r, struct strbuf *buf,
 	return buf->buf;
 }
 
+static struct packfile_list all_packfiles;
 static unsigned int pack_used_ctr;
 static unsigned int pack_open_fds;
 static unsigned int pack_max_fds;
@@ -195,6 +196,7 @@ static struct packed_git *packfile_new(struct repository *r, int extra)
 	memset(p, 0, sizeof(*p));
 	p->pack_fd = -1;
 	p->repo = r;
+	packfile_list_append(&all_packfiles, p, 1);
 	return p;
 }
 
@@ -243,17 +245,14 @@ static void scan_windows(struct packed_git *p,
 	}
 }
 
-static int unuse_one_window(struct object_database *odb)
+static int unuse_one_window(void)
 {
-	struct odb_source_files *files = odb_source_files_downcast(odb->source);
 	struct packfile_list_entry *e;
 	struct packed_git *lru_p = NULL;
 	struct pack_window *lru_w = NULL, *lru_l = NULL;
 
-	for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
-		for (e = dir->packed->packs.head; e; e = e->next)
-			scan_windows(e->pack, &lru_p, &lru_w, &lru_l);
-	}
+	for (e = all_packfiles.head; e; e = e->next)
+		scan_windows(e->pack, &lru_p, &lru_w, &lru_l);
 
 	if (lru_p) {
 		munmap(lru_w->base, lru_w->len);
@@ -407,20 +406,17 @@ static void find_lru_pack(struct packed_git *p, struct packed_git **lru_p, struc
 	*accept_windows_inuse = has_windows_inuse;
 }
 
-static int close_one_pack(struct repository *r)
+static int close_one_pack(void)
 {
-	struct odb_source_files *files = odb_source_files_downcast(r->objects->source);
 	struct packfile_list_entry *e;
 	struct packed_git *lru_p = NULL;
 	struct pack_window *mru_w = NULL;
 	int accept_windows_inuse = 1;
 
-	for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
-		for (e = dir->packed->packs.head; e; e = e->next) {
-			if (e->pack->pack_fd == -1)
-				continue;
-			find_lru_pack(e->pack, &lru_p, &mru_w, &accept_windows_inuse);
-		}
+	for (e = all_packfiles.head; e; e = e->next) {
+		if (e->pack->pack_fd == -1)
+			continue;
+		find_lru_pack(e->pack, &lru_p, &mru_w, &accept_windows_inuse);
 	}
 
 	if (lru_p)
@@ -505,7 +501,7 @@ static int packfile_open_pack(struct packed_git *p)
 			pack_max_fds = 1;
 	}
 
-	while (pack_max_fds <= pack_open_fds && close_one_pack(p->repo))
+	while (pack_max_fds <= pack_open_fds && close_one_pack())
 		; /* nothing */
 
 	p->pack_fd = git_open(p->pack_name);
@@ -649,7 +645,7 @@ unsigned char *use_pack(struct packed_git *p,
 			pack_mapped += win->len;
 
 			while (settings->packed_git_limit < pack_mapped &&
-			       unuse_one_window(p->repo->objects))
+			       unuse_one_window())
 				; /* nothing */
 			win->base = xmmap_gently(NULL, win->len,
 				PROT_READ, MAP_PRIVATE,
@@ -754,6 +750,7 @@ struct packed_git *packfile_from_idx(struct repository *r, const char *path,
 
 void packfile_free(struct packed_git *p)
 {
+	packfile_list_remove(&all_packfiles, p);
 	packfile_close(p);
 	free(p);
 }

-- 
2.56.1.831.g720f3f5937.dirty



```

## Patrick Steinhardt, 2026-11-20 12:02

Subject: [PATCH 6/6] packfile: skip closing packs marked as do-not-close
Message-ID: <20261120-pks-packfile-global-statistics-v1-6-19beb83b145f@pks.im>
URL: https://gitlist.dev/e/20261120-pks-packfile-global-statistics-v1-6-19beb83b145f%40pks.im
In-Reply-To: <20261120-pks-packfile-global-statistics-v1-0-19beb83b145f@pks.im>

```
When closing a packfile to maintain the maximum number of open packs we
completely ignore the `do_not_close` bit. That bit is only set by
git-fast-import(1), and as that command allocates its packs manually it
wouldn't add those packs to the list of all packfiles, and consequently
we won't ever try to close such a packfile via `close_one_pack()`. But
that feels like coincidence rather than proper design, and the result is
rather fragile.

Skip over packfiles marked as do-not-close to reduce the fragility. This
should not result in a user-visible change in behaviour.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 packfile.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/packfile.c b/packfile.c
index 0d5647c74c8..86ef3a79d9c 100644
--- a/packfile.c
+++ b/packfile.c
@@ -414,7 +414,8 @@ static int close_one_pack(void)
 	int accept_windows_inuse = 1;
 
 	for (e = all_packfiles.head; e; e = e->next) {
-		if (e->pack->pack_fd == -1)
+		if (e->pack->pack_fd == -1 ||
+		    e->pack->do_not_close)
 			continue;
 		find_lru_pack(e->pack, &lru_p, &mru_w, &accept_windows_inuse);
 	}

-- 
2.56.1.831.g720f3f5937.dirty



```
