threads / patch / 66896

patch, 6 partspackfile: some lifecycle improvements

Subject: [PATCH 0/6] packfile: some lifecycle improvements

## tl;dr

Patrick Steinhardt's six patches fix how the packfile subsystem enforces its limits across repositories, and tidy function names and ownership along the way. Read the story.

replies: 6people: 1as markdown or json

Patrick Steinhardt· Nov 20, 2026, 12:02 UTC · lore
Hi,

this small patch series polishes our lifecycle handling of packfiles a bit. The original motivation of this patch series is really only the last patch, which fixes how we handle process-wide limitations. But I took the chance to also polish this subsystem a bit, as a lot of its parts feel rather dated nowadays.

There's a minor semantic merge conflict with "seen" that can be sovled like this:

Show changes to odb/source-packed.c +1 −1
diff --git a/odb/source-packed.c b/odb/source-packed.c
index 77f8227ae52..e24f6cc7ef9 100644
--- a/odb/source-packed.c
+++ b/odb/source-packed.c
@@ -898,7 +898,7 @@ static int odb_source_packed_map_oid(struct odb_source *source,
 						       &hash_algos[src->algo] :
 						       repo->hash_algo;
 
-		if (open_pack_index(p) || !p->num_objects)
+		if (packfile_open_index(p) || !p->num_objects)
 			continue;
 		/* Only v3 indices store object IDs in both algorithms. */
 		if (p->index_version < 3)

Thanks!

Patrick

---
Patrick Steinhardt (6):
      builtin/fast-import: stop reporting packfile statistics
      packfile: rename lifecycle functions
      packfile: inline `open_packed_git_1()`
      packfile: introduce function to free a packfile
      packfile: maintain process-wide limits globally
      packfile: skip closing packs marked as do-not-close

 Documentation/git-fast-import.adoc |   5 +-
 builtin/count-objects.c            |   2 +-
 builtin/fast-import.c              |   8 +-
 builtin/index-pack.c               |  16 +--
 builtin/pack-objects.c             |   2 +-
 builtin/pack-redundant.c           |   2 +-
 commit-graph.c                     |  12 +-
 connected.c                        |  10 +-
 http-push.c                        |   2 +-
 http-walker.c                      |   8 +-
 http.c                             |   4 +-
 midx-write.c                       |  23 ++--
 midx.c                             |   6 +-
 odb/source-packed.c                |  14 +--
 pack-check.c                       |   2 +-
 pack-mtimes.c                      |   2 +-
 pack-revindex.c                    |   4 +-
 packfile.c                         | 233 ++++++++++++++++++-------------------
 packfile.h                         |  30 ++---
 repack-geometry.c                  |   2 +-
 repack-midx.c                      |   2 +-
 t/helper/test-bitmap.c             |   2 +-
 22 files changed, 182 insertions(+), 209 deletions(-)


---
base-commit: f7de81456940dd6f3274e461149bba12965de609
change-id: 20261120-pks-packfile-global-statistics-92a9be97a11f
Patrick Steinhardt· Nov 20, 2026, 12:02 UTC · re: Patrick Steinhardt · lore

[PATCH 1/6] builtin/fast-import: stop reporting packfile statistics

By default, git-fast-import(1) prints a bunch of statistics about all kinds of internal details of Git. This includes for example how many objects have been allocated, the number of refs processed, or the amount of memory that was allocated.

Reporting this kind information is highly questionable, doubly so as it is done by default, as a user really shoudln't have to worry about such internal implementation details. For some of the above metrics it could be argued though that it's indeed interesting to learn about the number of objects processed to get a rough sense of what git-fast-import(1) has done. But for other kind of information that's not the case, as it is without a doubt purely internal implementation details.

One of these reported implementation details is our packfile statistics. These include information on the system's page size, the number of packs that were opened, the number of calls to mmap(3) and how many pack windows were created. Reporting that kind of information, and that even by default, is just completely uncalled for. This information may be useful to a developer, but a user really shouldn't have to care about any of this.

More importantly though, with pluggable object databases we shouldn't assume that we're using packfiles in the first place anymore. The current implementation of git-fast-import(1) still does, but we will convert it to use transactions sooner rather than later.

Let's just unceremoniously drop reporting packfile information to the user. If anyone complains (which I highly doubt), then we may consider reintroducing this as trace2 data. The same is arguably true for many of the other reported statistics. But for now I care about the packfile subsystem, only, and hence those are left as-is.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/git-fast-import.adoc |  5 ++---
 builtin/fast-import.c              |  4 +---
 packfile.c                         | 36 ------------------------------------
 packfile.h                         |  2 --
 4 files changed, 3 insertions(+), 44 deletions(-)
Show changes to 4 files +3 −44

Documentation/git-fast-import.adoc, builtin/fast-import.c, packfile.c, packfile.h

diff --git a/Documentation/git-fast-import.adoc b/Documentation/git-fast-import.adoc
index 5201e493345..6afe14b650c 100644
--- a/Documentation/git-fast-import.adoc
+++ b/Documentation/git-fast-import.adoc
@@ -48,9 +48,8 @@ OPTIONS
 
 --stats::
 	Display some basic statistics about the objects fast-import has
-	created, the packfiles they were stored into, and the
-	memory used by fast-import during this run.  Showing this output
-	is currently the default, but can be disabled with --quiet.
+	created, and the memory used by fast-import during this run.  Showing
+	this output is currently the default, but can be disabled with --quiet.
 
 --allow-unsafe-features::
 	Many command-line options can be provided as part of the
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index ab5d2a23448..14e75149415 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -4153,7 +4153,7 @@ static int option_parse_early_allow_unsafe(const struct option *option,
 int cmd_fast_import(int argc,
 		    const char **argv,
 		    const char *prefix,
-		    struct repository *repo)
+		    struct repository *repo UNUSED)
 {
 	struct fast_import_state state;
 
@@ -4346,8 +4346,6 @@ int cmd_fast_import(int argc,
 		fprintf(stderr, "       pools:    %10lu KiB\n", (unsigned long)((tree_entry_allocd + fi_mem_pool.pool_alloc) /1024));
 		fprintf(stderr, "     objects:    %10" PRIuMAX " KiB\n", (alloc_count*sizeof(struct object_entry))/1024);
 		fprintf(stderr, "---------------------------------------------------------------------\n");
-		pack_report(repo);
-		fprintf(stderr, "---------------------------------------------------------------------\n");
 		fprintf(stderr, "\n");
 	}
 
diff --git a/packfile.c b/packfile.c
index b32b4cd75da..5aa9b009174 100644
--- a/packfile.c
+++ b/packfile.c
@@ -36,38 +36,10 @@ char *odb_pack_name(struct repository *r, struct strbuf *buf,
 }
 
 static unsigned int pack_used_ctr;
-static unsigned int pack_mmap_calls;
-static unsigned int peak_pack_open_windows;
-static unsigned int pack_open_windows;
 static unsigned int pack_open_fds;
 static unsigned int pack_max_fds;
-static size_t peak_pack_mapped;
 static size_t pack_mapped;
 
-#define SZ_FMT PRIuMAX
-static inline uintmax_t sz_fmt(size_t s) { return s; }
-
-void pack_report(struct repository *repo)
-{
-	fprintf(stderr,
-		"pack_report: getpagesize()            = %10" SZ_FMT "\n"
-		"pack_report: core.packedGitWindowSize = %10" SZ_FMT "\n"
-		"pack_report: core.packedGitLimit      = %10" SZ_FMT "\n",
-		sz_fmt(getpagesize()),
-		sz_fmt(repo->settings.packed_git_window_size),
-		sz_fmt(repo->settings.packed_git_limit));
-	fprintf(stderr,
-		"pack_report: pack_used_ctr            = %10u\n"
-		"pack_report: pack_mmap_calls          = %10u\n"
-		"pack_report: pack_open_windows        = %10u / %10u\n"
-		"pack_report: pack_mapped              = "
-			"%10" SZ_FMT " / %10" SZ_FMT "\n",
-		pack_used_ctr,
-		pack_mmap_calls,
-		pack_open_windows, peak_pack_open_windows,
-		sz_fmt(pack_mapped), sz_fmt(peak_pack_mapped));
-}
-
 /*
  * Open and mmap the index file at path, perform a couple of
  * consistency checks, then record its information to p.  Return 0 on
@@ -291,7 +263,6 @@ static int unuse_one_window(struct object_database *odb)
 		else
 			lru_p->windows = lru_w->next;
 		free(lru_w);
-		pack_open_windows--;
 		return 1;
 	}
 	return 0;
@@ -307,7 +278,6 @@ void close_pack_windows(struct packed_git *p)
 			    p->pack_name);
 		munmap(w->base, w->len);
 		pack_mapped -= w->len;
-		pack_open_windows--;
 		p->windows = w->next;
 		free(w);
 	}
@@ -665,12 +635,6 @@ unsigned char *use_pack(struct packed_git *p,
 			if (!win->offset && win->len == p->pack_size
 				&& !p->do_not_close)
 				close_pack_fd(p);
-			pack_mmap_calls++;
-			pack_open_windows++;
-			if (pack_mapped > peak_pack_mapped)
-				peak_pack_mapped = pack_mapped;
-			if (pack_open_windows > peak_pack_open_windows)
-				peak_pack_open_windows = pack_open_windows;
 			win->next = p->windows;
 			p->windows = win;
 		}
diff --git a/packfile.h b/packfile.h
index ed3378c4576..bab9931d00e 100644
--- a/packfile.h
+++ b/packfile.h
@@ -217,8 +217,6 @@ int for_each_object_in_pack(struct packed_git *p,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-void pack_report(struct repository *repo);
-
 /*
  * mmap the index file for the specified packfile (if it is not
  * already mmapped).  Return 0 on success.
-- 
2.56.1.831.g720f3f5937.dirty
Patrick Steinhardt· Nov 20, 2026, 12:02 UTC · re: Patrick Steinhardt · lore

[PATCH 2/6] packfile: rename lifecycle functions

The packfile subsystem is kind of the wild west when it comes to function naming. None of the functions have proper subsystem prefixes and the naming of `struct packed_git` feels like a relict from the past.

Ideally we'd modernize all of that, but that would lead to a huge amount of churn. Instead, let's improve the subsystem one step at a time and rename lifecycle-related functions to match modern best practices.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/count-objects.c  |  2 +-
 builtin/fast-import.c    |  4 ++--
 builtin/index-pack.c     |  8 ++++----
 builtin/pack-objects.c   |  2 +-
 builtin/pack-redundant.c |  2 +-
 commit-graph.c           | 10 +++++-----
 connected.c              |  6 +++---
 http-push.c              |  2 +-
 http-walker.c            |  4 ++--
 http.c                   |  2 +-
 midx-write.c             | 18 +++++++++---------
 midx.c                   |  6 +++---
 odb/source-packed.c      | 12 ++++++------
 pack-check.c             |  2 +-
 pack-mtimes.c            |  2 +-
 pack-revindex.c          |  4 ++--
 packfile.c               | 46 +++++++++++++++++++++++-----------------------
 packfile.h               | 29 +++++++++++++++--------------
 repack-geometry.c        |  2 +-
 repack-midx.c            |  2 +-
 t/helper/test-bitmap.c   |  2 +-
 21 files changed, 84 insertions(+), 83 deletions(-)
Show changes to 21 files +84 −83

builtin/count-objects.c, builtin/fast-import.c, builtin/index-pack.c, builtin/pack-objects.c, builtin/pack-redundant.c, commit-graph.c, connected.c, http-push.c, http-walker.c, http.c, midx-write.c, midx.c, odb/source-packed.c, pack-check.c, pack-mtimes.c, pack-revindex.c, packfile.c, packfile.h, repack-geometry.c, repack-midx.c, t/helper/test-bitmap.c

diff --git a/builtin/count-objects.c b/builtin/count-objects.c
index 3f7b20fde53..09922ae4a70 100644
--- a/builtin/count-objects.c
+++ b/builtin/count-objects.c
@@ -132,7 +132,7 @@ int cmd_count_objects(int argc,
 		repo_for_each_pack(the_repository, p) {
 			if (!p->pack_local)
 				continue;
-			if (open_pack_index(p))
+			if (packfile_open_index(p))
 				continue;
 			packed += p->num_objects;
 			size_pack += p->pack_size + p->index_size;
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index 14e75149415..5f034d39a71 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -906,7 +906,7 @@ static void end_packfile(void)
 		struct branch *b;
 		struct tag *t;
 
-		close_pack_windows(pack_data);
+		packfile_close_windows(pack_data);
 		finalize_hashfile(pack_file, cur_pack_oid.hash, FSYNC_COMPONENT_PACK, 0);
 		fixup_pack_header_footer(the_hash_algo, pack_data->pack_fd,
 					 pack_data->hash, pack_data->pack_name,
@@ -1273,7 +1273,7 @@ static void *gfi_unpack_entry(
 		 * and updating the packfile length ensures we can read
 		 * the newly written data.
 		 */
-		close_pack_windows(p);
+		packfile_close_windows(p);
 		hashflush(pack_file);
 
 		/* We have to offer rawsz bytes additional on the end of
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 0264c18c5f8..71048a5a8af 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1741,12 +1741,12 @@ static void read_v2_anomalous_offsets(struct packed_git *p,
 
 static void read_idx_option(struct pack_idx_option *opts, const char *pack_name)
 {
-	struct packed_git *p = add_packed_git(the_repository, pack_name,
-					      strlen(pack_name), 1);
+	struct packed_git *p = packfile_from_idx(the_repository, pack_name,
+						 strlen(pack_name), 1);
 
 	if (!p)
 		die(_("Cannot open existing pack file '%s'"), pack_name);
-	if (open_pack_index(p))
+	if (packfile_open_index(p))
 		die(_("Cannot open existing pack idx file for '%s'"), pack_name);
 
 	/* Read the attributes from the existing idx file */
@@ -1762,7 +1762,7 @@ static void read_idx_option(struct pack_idx_option *opts, const char *pack_name)
 	 * know we haven't installed this pack (hence we never have
 	 * read anything from it).
 	 */
-	close_pack_index(p);
+	packfile_close_index(p);
 	free(p);
 }
 
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 87c3102e622..9c375811937 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -4724,7 +4724,7 @@ static void loosen_unused_packed_objects(void)
 		if (!p->pack_local || p->pack_keep || p->pack_keep_in_core)
 			continue;
 
-		if (open_pack_index(p))
+		if (packfile_open_index(p))
 			die(_("cannot open pack index"));
 
 		for (i = 0; i < p->num_objects; i++) {
diff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c
index 86749bb7e77..1b437bb6565 100644
--- a/builtin/pack-redundant.c
+++ b/builtin/pack-redundant.c
@@ -546,7 +546,7 @@ static struct pack_list * add_pack(struct packed_git *p)
 	l.pack = p;
 	llist_init(&l.remaining_objects);
 
-	if (open_pack_index(p)) {
+	if (packfile_open_index(p)) {
 		llist_free(l.remaining_objects);
 		return NULL;
 	}
diff --git a/commit-graph.c b/commit-graph.c
index da65cd3deb9..c3b04e8c1f4 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -1950,21 +1950,21 @@ static int fill_oids_from_packs(struct write_commit_graph_context *ctx,
 	for (i = 0; i < pack_indexes->nr; i++) {
 		struct packed_git *p;
 
-		p = add_packed_git(ctx->r, pack_indexes->items[i].string,
-				   strlen(pack_indexes->items[i].string), 1);
+		p = packfile_from_idx(ctx->r, pack_indexes->items[i].string,
+				      strlen(pack_indexes->items[i].string), 1);
 		if (!p) {
 			ret = error(_("error adding pack %s"), pack_indexes->items[i].string);
 			goto cleanup;
 		}
-		if (open_pack_index(p)) {
+		if (packfile_open_index(p)) {
 			ret = error(_("error opening index for %s"), pack_indexes->items[i].string);
-			close_pack(p);
+			packfile_close(p);
 			free(p);
 			goto cleanup;
 		}
 		for_each_object_in_pack(p, add_packed_commits, ctx,
 					ODB_FOR_EACH_OBJECT_PACK_ORDER);
-		close_pack(p);
+		packfile_close(p);
 		free(p);
 	}
 
diff --git a/connected.c b/connected.c
index bfb771e4695..fbacff734b6 100644
--- a/connected.c
+++ b/connected.c
@@ -207,8 +207,8 @@ int check_connected(struct repository *repo,
 		strbuf_add(&idx_file, transport->pack_lockfiles.items[0].string,
 			   base_len);
 		strbuf_addstr(&idx_file, ".idx");
-		new_pack = add_packed_git(repo, idx_file.buf,
-					  idx_file.len, 1);
+		new_pack = packfile_from_idx(repo, idx_file.buf,
+					     idx_file.len, 1);
 		strbuf_release(&idx_file);
 	}
 
@@ -239,7 +239,7 @@ int check_connected(struct repository *repo,
 
 	sigchain_pop(SIGPIPE);
 	if (new_pack) {
-		close_pack(new_pack);
+		packfile_close(new_pack);
 		free(new_pack);
 	}
 	return finish_command(&rev_list) || err;
diff --git a/http-push.c b/http-push.c
index b8f3faaed95..d5335c023c8 100644
--- a/http-push.c
+++ b/http-push.c
@@ -318,7 +318,7 @@ static void start_fetch_packed(struct transfer_request *request)
 		release_request(request);
 		return;
 	}
-	close_pack_index(target);
+	packfile_close_index(target);
 	request->target = target;
 
 	fprintf(stderr,	"Fetching pack %s\n",
diff --git a/http-walker.c b/http-walker.c
index 4f50dd9b8d0..de43d9cf673 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -435,7 +435,7 @@ static int http_fetch_pack(struct walker *walker, struct alt_base *repo,
 	target = packfile_list_find_oid(repo->packs.head, oid);
 	if (!target)
 		return -1;
-	close_pack_index(target);
+	packfile_close_index(target);
 
 	if (walker->get_verbosely) {
 		fprintf(stderr, "Getting pack %s\n",
@@ -588,7 +588,7 @@ static void cleanup(struct walker *walker)
 			alt_next = alt->next;
 
 			for (e = alt->packs.head; e; e = e->next) {
-				close_pack(e->pack);
+				packfile_close(e->pack);
 				free(e->pack);
 			}
 			packfile_list_clear(&alt->packs);
diff --git a/http.c b/http.c
index d79678d01f0..e86bebdffb2 100644
--- a/http.c
+++ b/http.c
@@ -2658,7 +2658,7 @@ static int fetch_and_setup_pack_index(struct packfile_list *packs,
 
 	ret = verify_pack_index(new_pack);
 
-	close_pack_index(new_pack);
+	packfile_close_index(new_pack);
 	free(tmp_idx);
 	if (ret) {
 		free(new_pack);
diff --git a/midx-write.c b/midx-write.c
index 3038bbfad26..d5be4e40843 100644
--- a/midx-write.c
+++ b/midx-write.c
@@ -172,17 +172,17 @@ static void add_pack_to_midx(const char *full_path, size_t full_path_len,
 			return;
 
 		ALLOC_GROW(ctx->info, ctx->nr + 1, ctx->alloc);
-		p = add_packed_git(ctx->repo, full_path, full_path_len, 0);
+		p = packfile_from_idx(ctx->repo, full_path, full_path_len, 0);
 		if (!p) {
 			warning(_("failed to add packfile '%s'"),
 				full_path);
 			return;
 		}
 
-		if (open_pack_index(p)) {
+		if (packfile_open_index(p)) {
 			warning(_("failed to open pack-index '%s'"),
 				full_path);
-			close_pack(p);
+			packfile_close(p);
 			free(p);
 			return;
 		}
@@ -1464,7 +1464,7 @@ static int write_midx_internal(struct write_midx_opts *opts)
 		 * Ignore failures as they can be expected and are not
 		 * fatal during this selection time.
 		 */
-		open_pack_index(oldest);
+		packfile_open_index(oldest);
 
 		if (opts->packs_to_drop && opts->packs_to_drop->nr)
 			BUG("cannot write a MIDX bitmap during expiration");
@@ -1480,7 +1480,7 @@ static int write_midx_internal(struct write_midx_opts *opts)
 
 			if (!oldest->num_objects || p->mtime < oldest->mtime) {
 				oldest = p;
-				open_pack_index(oldest);
+				packfile_open_index(oldest);
 				ctx.preferred_pack_idx = i;
 			}
 		}
@@ -1505,7 +1505,7 @@ static int write_midx_internal(struct write_midx_opts *opts)
 	if (ctx.preferred_pack_idx != NO_PREFERRED_PACK) {
 		struct packed_git *preferred = ctx.info[ctx.preferred_pack_idx].p;
 
-		if (open_pack_index(preferred))
+		if (packfile_open_index(preferred))
 			die(_("failed to open preferred pack %s"),
 			    ctx.info[ctx.preferred_pack_idx].pack_name);
 
@@ -1828,7 +1828,7 @@ static int write_midx_internal(struct write_midx_opts *opts)
 cleanup:
 	for (size_t i = 0; i < ctx.nr; i++) {
 		if (ctx.info[i].p) {
-			close_pack(ctx.info[i].p);
+			packfile_close(ctx.info[i].p);
 			free(ctx.info[i].p);
 		}
 		free(ctx.info[i].pack_name);
@@ -1944,7 +1944,7 @@ int expire_midx_packs(struct odb_source_packed *source, unsigned flags)
 			continue;
 
 		pack_name = xstrdup(m->packs[i]->pack_name);
-		close_pack(m->packs[i]);
+		packfile_close(m->packs[i]);
 
 		string_list_insert(&packs_to_drop, m->pack_names[i]);
 		unlink_pack_path(pack_name, 0);
@@ -2000,7 +2000,7 @@ static int want_included_pack(struct multi_pack_index *m,
 		return 0;
 	if (p->is_cruft)
 		return 0;
-	if (open_pack_index(p) || !p->num_objects)
+	if (packfile_open_index(p) || !p->num_objects)
 		return 0;
 	return 1;
 }
diff --git a/midx.c b/midx.c
index 2d4feeeadeb..d2b8a3a4a5e 100644
--- a/midx.c
+++ b/midx.c
@@ -1023,8 +1023,8 @@ int verify_midx_file(struct odb_source_packed *source, unsigned flags)
 			uint32_t pack_int_id = pairs[i-1].pack_int_id;
 			struct packed_git *p = nth_midxed_pack(m, pack_int_id);
 
-			close_pack_fd(p);
-			close_pack_index(p);
+			packfile_close_fd(p);
+			packfile_close_index(p);
 		}
 
 		nth_midxed_object_oid(&oid, m, pairs[i].pos);
@@ -1035,7 +1035,7 @@ int verify_midx_file(struct odb_source_packed *source, unsigned flags)
 			continue;
 		}
 
-		if (open_pack_index(e.p)) {
+		if (packfile_open_index(e.p)) {
 			midx_report(_("failed to load pack-index for packfile %s"),
 				    e.p->pack_name);
 			break;
diff --git a/odb/source-packed.c b/odb/source-packed.c
index 7fcbbb4fe9e..87e2dcdf52e 100644
--- a/odb/source-packed.c
+++ b/odb/source-packed.c
@@ -351,7 +351,7 @@ static int odb_source_packed_for_each_prefixed_object(
 		if (should_exclude_pack(e->pack, opts->flags))
 			continue;
 
-		if (open_pack_index(e->pack)) {
+		if (packfile_open_index(e->pack)) {
 			pack_errors = true;
 			continue;
 		}
@@ -459,7 +459,7 @@ static int odb_source_packed_for_each_object(struct odb_source *source,
 		if (bitmap && bitmap_index_contains_pack(bitmap, p))
 			continue;
 
-		if (open_pack_index(p)) {
+		if (packfile_open_index(p)) {
 			pack_errors = 1;
 			continue;
 		}
@@ -498,7 +498,7 @@ static int odb_source_packed_count_objects(struct odb_source *source,
 	for (e = packfile_store_get_packs(packed); e; e = e->next) {
 		if (e->pack->multi_pack_index)
 			continue;
-		if (open_pack_index(e->pack)) {
+		if (packfile_open_index(e->pack)) {
 			ret = -1;
 			goto out;
 		}
@@ -615,7 +615,7 @@ static int odb_source_packed_find_abbrev_len(struct odb_source *source,
 	for (e = packfile_store_get_packs(packed); e; e = e->next) {
 		if (e->pack->multi_pack_index)
 			continue;
-		if (open_pack_index(e->pack) || !e->pack->num_objects)
+		if (packfile_open_index(e->pack) || !e->pack->num_objects)
 			continue;
 
 		find_abbrev_len_for_pack(e->pack, oid, min_len, &min_len);
@@ -847,7 +847,7 @@ static void odb_source_packed_close(struct odb_source *source)
 	for (struct packfile_list_entry *e = packed->packs.head; e; e = e->next) {
 		if (e->pack->do_not_close)
 			BUG("want to close pack marked 'do-not-close'");
-		close_pack(e->pack);
+		packfile_close(e->pack);
 	}
 	if (packed->midx)
 		close_midx(packed->midx);
@@ -880,7 +880,7 @@ static int verify_packs(struct odb_source_packed *source,
 
 	if (opts->flags & ODB_FSCK_PROGRESS) {
 		for (e = packfile_store_get_packs(source); e; e = e->next) {
-			if (open_pack_index(e->pack))
+			if (packfile_open_index(e->pack))
 				continue;
 			total += e->pack->num_objects;
 		}
diff --git a/pack-check.c b/pack-check.c
index 605811b7b09..e6663bb8e16 100644
--- a/pack-check.c
+++ b/pack-check.c
@@ -184,7 +184,7 @@ int verify_pack_index(struct packed_git *p)
 {
 	int err = 0;
 
-	if (open_pack_index(p))
+	if (packfile_open_index(p))
 		return error("packfile %s index not opened", p->pack_name);
 
 	/* Verify SHA1 sum of the index file */
diff --git a/pack-mtimes.c b/pack-mtimes.c
index 8e1f2dec0ef..8ce9c808e7f 100644
--- a/pack-mtimes.c
+++ b/pack-mtimes.c
@@ -106,7 +106,7 @@ int load_pack_mtimes(struct packed_git *p)
 	if (p->mtimes_map)
 		return ret; /* already loaded */
 
-	ret = open_pack_index(p);
+	ret = packfile_open_index(p);
 	if (ret < 0)
 		goto cleanup;
 
diff --git a/pack-revindex.c b/pack-revindex.c
index 62387ae6320..5de5e4064ba 100644
--- a/pack-revindex.c
+++ b/pack-revindex.c
@@ -176,7 +176,7 @@ static int create_pack_revindex_in_memory(struct packed_git *p)
 	if (git_env_bool(GIT_TEST_REV_INDEX_DIE_IN_MEMORY, 0))
 		die("dying as requested by '%s'",
 		    GIT_TEST_REV_INDEX_DIE_IN_MEMORY);
-	if (open_pack_index(p))
+	if (packfile_open_index(p))
 		return -1;
 	create_pack_revindex(p);
 	return 0;
@@ -281,7 +281,7 @@ int load_pack_revindex_from_disk(struct packed_git *p)
 	if (p->revindex_data)
 		return 0;
 
-	if (open_pack_index(p))
+	if (packfile_open_index(p))
 		return -1;
 
 	revindex_name = pack_revindex_filename(p);
diff --git a/packfile.c b/packfile.c
index 5aa9b009174..95a97c302c7 100644
--- a/packfile.c
+++ b/packfile.c
@@ -155,7 +155,7 @@ int load_idx(const char *path, const unsigned int hashsz, void *idx_map,
 	return 0;
 }
 
-int open_pack_index(struct packed_git *p)
+int packfile_open_index(struct packed_git *p)
 {
 	char *idx_name;
 	size_t len;
@@ -177,7 +177,7 @@ uint32_t get_pack_fanout(struct packed_git *p, uint32_t value)
 	const uint32_t *level1_ofs = p->index_data;
 
 	if (!level1_ofs) {
-		if (open_pack_index(p))
+		if (packfile_open_index(p))
 			return 0;
 		level1_ofs = p->index_data;
 	}
@@ -189,7 +189,7 @@ uint32_t get_pack_fanout(struct packed_git *p, uint32_t value)
 	return ntohl(level1_ofs[value]);
 }
 
-static struct packed_git *alloc_packed_git(struct repository *r, int extra)
+static struct packed_git *packfile_new(struct repository *r, int extra)
 {
 	struct packed_git *p = xmalloc(st_add(sizeof(*p), extra));
 	memset(p, 0, sizeof(*p));
@@ -211,7 +211,7 @@ struct packed_git *parse_pack_index(struct repository *r, unsigned char *sha1,
 {
 	char *path = pack_path_from_idx(idx_path);
 	size_t alloc = st_add(strlen(path), 1);
-	struct packed_git *p = alloc_packed_git(r, alloc);
+	struct packed_git *p = packfile_new(r, alloc);
 
 	memcpy(p->pack_name, path, alloc); /* includes NUL */
 	free(path);
@@ -268,7 +268,7 @@ static int unuse_one_window(struct object_database *odb)
 	return 0;
 }
 
-void close_pack_windows(struct packed_git *p)
+void packfile_close_windows(struct packed_git *p)
 {
 	while (p->windows) {
 		struct pack_window *w = p->windows;
@@ -283,7 +283,7 @@ void close_pack_windows(struct packed_git *p)
 	}
 }
 
-int close_pack_fd(struct packed_git *p)
+int packfile_close_fd(struct packed_git *p)
 {
 	if (p->pack_fd < 0)
 		return 0;
@@ -295,7 +295,7 @@ int close_pack_fd(struct packed_git *p)
 	return 1;
 }
 
-void close_pack_index(struct packed_git *p)
+void packfile_close_index(struct packed_git *p)
 {
 	if (p->index_data) {
 		munmap((void *)p->index_data, p->index_size);
@@ -424,7 +424,7 @@ static int close_one_pack(struct repository *r)
 	}
 
 	if (lru_p)
-		return close_pack_fd(lru_p);
+		return packfile_close_fd(lru_p);
 
 	return 0;
 }
@@ -493,7 +493,7 @@ static int open_packed_git_1(struct packed_git *p)
 	ssize_t read_result;
 	const unsigned hashsz = p->repo->hash_algo->rawsz;
 
-	if (open_pack_index(p))
+	if (packfile_open_index(p))
 		return error("packfile %s index unavailable", p->pack_name);
 
 	if (!pack_max_fds) {
@@ -557,7 +557,7 @@ static int open_packed_git(struct packed_git *p)
 {
 	if (!open_packed_git_1(p))
 		return 0;
-	close_pack_fd(p);
+	packfile_close_fd(p);
 	return -1;
 }
 
@@ -634,7 +634,7 @@ unsigned char *use_pack(struct packed_git *p,
 					  p->pack_name, mmap_os_err());
 			if (!win->offset && win->len == p->pack_size
 				&& !p->do_not_close)
-				close_pack_fd(p);
+				packfile_close_fd(p);
 			win->next = p->windows;
 			p->windows = win;
 		}
@@ -659,8 +659,8 @@ void unuse_pack(struct pack_window **w_cursor)
 	}
 }
 
-struct packed_git *add_packed_git(struct repository *r, const char *path,
-				  size_t path_len, int local)
+struct packed_git *packfile_from_idx(struct repository *r, const char *path,
+				     size_t path_len, int local)
 {
 	struct stat st;
 	size_t alloc;
@@ -679,7 +679,7 @@ struct packed_git *add_packed_git(struct repository *r, const char *path,
 	 * the use xsnprintf double-checks that)
 	 */
 	alloc = st_add3(path_len, strlen(".promisor"), 1);
-	p = alloc_packed_git(r, alloc);
+	p = packfile_new(r, alloc);
 	memcpy(p->pack_name, path, path_len);
 
 	/*
@@ -753,8 +753,8 @@ struct packed_git *packfile_store_load_pack(struct odb_source_packed *store,
 
 	p = strmap_get(&store->packs_by_path, key.buf);
 	if (!p) {
-		p = add_packed_git(store->base.odb->repo, idx_path,
-				   strlen(idx_path), local);
+		p = packfile_from_idx(store->base.odb->repo, idx_path,
+				      strlen(idx_path), local);
 		if (p)
 			packfile_store_add_pack(store, p);
 	}
@@ -875,7 +875,7 @@ size_t get_size_from_delta(struct packed_git *p,
 		 * counter is incremented before window reading and checked
 		 * before window disposal.
 		 *
-		 * Other worrying sections could be the call to close_pack_fd(),
+		 * Other worrying sections could be the call to packfile_close_fd(),
 		 * which can close packs even with in-use windows, and to
 		 * odb_reprepare(). Regarding the former, mmap doc says:
 		 * "closing the file descriptor does not unmap the region". And
@@ -1227,11 +1227,11 @@ static void delta_base_cache_evict_entry(struct packed_git *p)
 	}
 }
 
-void close_pack(struct packed_git *p)
+void packfile_close(struct packed_git *p)
 {
-	close_pack_windows(p);
-	close_pack_fd(p);
-	close_pack_index(p);
+	packfile_close_windows(p);
+	packfile_close_fd(p);
+	packfile_close_index(p);
 	close_pack_revindex(p);
 	close_pack_mtimes(p);
 	oidset_clear(&p->bad_objects);
@@ -1725,7 +1725,7 @@ int nth_packed_object_id(struct object_id *oid,
 	const unsigned char *index = p->index_data;
 	const unsigned int hashsz = p->repo->hash_algo->rawsz;
 	if (!index) {
-		if (open_pack_index(p))
+		if (packfile_open_index(p))
 			return -1;
 		index = p->index_data;
 	}
@@ -1783,7 +1783,7 @@ off_t find_pack_entry_one(const struct object_id *oid,
 	uint32_t result;
 
 	if (!index) {
-		if (open_pack_index(p))
+		if (packfile_open_index(p))
 			return 0;
 	}
 
diff --git a/packfile.h b/packfile.h
index bab9931d00e..f817bc69e29 100644
--- a/packfile.h
+++ b/packfile.h
@@ -180,7 +180,7 @@ const char *pack_basename(struct packed_git *p);
  * You probably don't want to use this function! It skips most of the normal
  * sanity checks (including whether we even have the matching .pack file),
  * and does not add the resulting packed_git struct to the internal list of
- * packs. You probably want add_packed_git() instead.
+ * packs. You probably want packfile_from_idx() instead.
  */
 struct packed_git *parse_pack_index(struct repository *r, unsigned char *sha1,
 				    const char *idx_path);
@@ -217,32 +217,33 @@ int for_each_object_in_pack(struct packed_git *p,
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
+uint32_t get_pack_fanout(struct packed_git *p, uint32_t value);
+
+struct object_database;
+
+struct packed_git *packfile_from_idx(struct repository *r, const char *path,
+				     size_t path_len, int local);
+
 /*
  * mmap the index file for the specified packfile (if it is not
  * already mmapped).  Return 0 on success.
  */
-int open_pack_index(struct packed_git *);
+int packfile_open_index(struct packed_git *);
+
+void packfile_close(struct packed_git *);
+int packfile_close_fd(struct packed_git *p);
+void packfile_close_windows(struct packed_git *);
 
 /*
  * munmap the index file for the specified packfile (if it is
  * currently mmapped).
  */
-void close_pack_index(struct packed_git *);
-
-int close_pack_fd(struct packed_git *p);
-
-uint32_t get_pack_fanout(struct packed_git *p, uint32_t value);
-
-struct object_database;
+void packfile_close_index(struct packed_git *);
 
 unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t,
 			size_t *);
-void close_pack_windows(struct packed_git *);
-void close_pack(struct packed_git *);
 void unuse_pack(struct pack_window **);
 void clear_delta_base_cache(void);
-struct packed_git *add_packed_git(struct repository *r, const char *path,
-				  size_t path_len, int local);
 
 /*
  * Unlink the .pack and associated extension files.
@@ -345,7 +346,7 @@ int is_promisor_object(struct repository *r, const struct object_id *oid);
  *
  * This function should not be used directly. It is exposed here only so that we
  * have a convenient entry-point for fuzz testing. For real uses, you should
- * probably use open_pack_index() instead.
+ * probably use packfile_open_index() instead.
  */
 int load_idx(const char *path, const unsigned int hashsz, void *idx_map,
 	     size_t idx_size, struct packed_git *p);
diff --git a/repack-geometry.c b/repack-geometry.c
index b541c34af5f..671a3dfe6a3 100644
--- a/repack-geometry.c
+++ b/repack-geometry.c
@@ -9,7 +9,7 @@
 
 static uint32_t pack_geometry_weight(struct packed_git *p)
 {
-	if (open_pack_index(p))
+	if (packfile_open_index(p))
 		die(_("cannot open index for %s"), p->pack_name);
 	return p->num_objects;
 }
diff --git a/repack-midx.c b/repack-midx.c
index 72810034733..7051a0d0637 100644
--- a/repack-midx.c
+++ b/repack-midx.c
@@ -576,7 +576,7 @@ static int midx_compaction_step_include_packs(struct midx_compaction_step *step,
 		strbuf_reset(&path);
 		strbuf_addf(&path, "%s/%s", opts->packdir, item->string);
 		p = packfile_store_load_pack(files->dirs->packed, path.buf, 1);
-		if (!p || open_pack_index(p)) {
+		if (!p || packfile_open_index(p)) {
 			ret = error(_("cannot open index for %s"), path.buf);
 			goto out;
 		}
diff --git a/t/helper/test-bitmap.c b/t/helper/test-bitmap.c
index 8547ef67e24..e3a3c8ff5cc 100644
--- a/t/helper/test-bitmap.c
+++ b/t/helper/test-bitmap.c
@@ -87,7 +87,7 @@ static int bitmap_write(const char *basename)
 	if (!p)
 		die("could not find pack '%s'", basename);
 
-	if (open_pack_index(p))
+	if (packfile_open_index(p))
 		die("cannot open pack index for '%s'", p->pack_name);
 
 	prepare_packing_data(the_repository, &packed);
-- 
2.56.1.831.g720f3f5937.dirty
Patrick Steinhardt· Nov 20, 2026, 12:02 UTC · re: Patrick Steinhardt · lore

[PATCH 3/6] packfile: inline `open_packed_git_1()`

The function `open_packed_git()` is responsible for opening a packfile. The majority of work is not performed by it though, but instead by its `_1()` sibling. But this sibling really only exists because we thought it too hard to handle errors properly and close the packfile's file descriptor on an error path. So `_1()` doesn't handle those, and instead defers to its only caller to close the file descriptor in the error path again.

Fix this design smell by introducing a common exit path that knows to close the file descriptor on error. This allows us to inline the call to `open_packed_git_1()`.

While at it, rename the function to `packfile_open_pack()` to match modern best practices. Besides that, it also gives the caller a better idea of _what_ is being opened, as we also have functions that open other files that belong to a packfile.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 packfile.c | 115 +++++++++++++++++++++++++++++++++++++------------------------
 1 file changed, 70 insertions(+), 45 deletions(-)
Show changes to packfile.c +70 −45
diff --git a/packfile.c b/packfile.c
index 95a97c302c7..9b03258311f 100644
--- a/packfile.c
+++ b/packfile.c
@@ -480,11 +480,7 @@ const char *pack_basename(struct packed_git *p)
 	return ret;
 }
 
-/*
- * Do not call this directly as this leaks p->pack_fd on error return;
- * call open_packed_git() instead.
- */
-static int open_packed_git_1(struct packed_git *p)
+static int packfile_open_pack(struct packed_git *p)
 {
 	struct stat st;
 	struct pack_header hdr;
@@ -492,9 +488,12 @@ static int open_packed_git_1(struct packed_git *p)
 	unsigned char *idx_hash;
 	ssize_t read_result;
 	const unsigned hashsz = p->repo->hash_algo->rawsz;
+	int ret;
 
-	if (packfile_open_index(p))
-		return error("packfile %s index unavailable", p->pack_name);
+	if (packfile_open_index(p)) {
+		ret = error("packfile %s index unavailable", p->pack_name);
+		goto out;
+	}
 
 	if (!pack_max_fds) {
 		unsigned int max_fds = get_max_fd_limit();
@@ -510,55 +509,81 @@ static int open_packed_git_1(struct packed_git *p)
 		; /* nothing */
 
 	p->pack_fd = git_open(p->pack_name);
-	if (p->pack_fd < 0 || fstat(p->pack_fd, &st))
-		return -1;
+	if (p->pack_fd < 0 || fstat(p->pack_fd, &st)) {
+		ret = -1;
+		goto out;
+	}
 	pack_open_fds++;
 
 	/* If we created the struct before we had the pack we lack size. */
 	if (!p->pack_size) {
-		if (!S_ISREG(st.st_mode))
-			return error("packfile %s not a regular file", p->pack_name);
+		if (!S_ISREG(st.st_mode)) {
+			ret = error("packfile %s not a regular file", p->pack_name);
+			goto out;
+		}
+
 		p->pack_size = st.st_size;
-	} else if (p->pack_size != st.st_size)
-		return error("packfile %s size changed", p->pack_name);
+	} else if (p->pack_size != st.st_size) {
+		ret = error("packfile %s size changed", p->pack_name);
+		goto out;
+	}
 
 	/* Verify we recognize this pack file format. */
 	read_result = read_in_full(p->pack_fd, &hdr, sizeof(hdr));
-	if (read_result < 0)
-		return error_errno("error reading from %s", p->pack_name);
-	if (read_result != sizeof(hdr))
-		return error("file %s is far too short to be a packfile", p->pack_name);
-	if (hdr.hdr_signature != htonl(PACK_SIGNATURE))
-		return error("file %s is not a GIT packfile", p->pack_name);
-	if (!pack_version_ok(hdr.hdr_version))
-		return error("packfile %s is version %"PRIu32" and not"
-			" supported (try upgrading GIT to a newer version)",
-			p->pack_name, ntohl(hdr.hdr_version));
+	if (read_result < 0) {
+		ret = error_errno("error reading from %s", p->pack_name);
+		goto out;
+	}
+
+	if (read_result != sizeof(hdr)) {
+		ret = error("file %s is far too short to be a packfile", p->pack_name);
+		goto out;
+	}
+
+	if (hdr.hdr_signature != htonl(PACK_SIGNATURE)) {
+		ret = error("file %s is not a GIT packfile", p->pack_name);
+		goto out;
+	}
+
+	if (!pack_version_ok(hdr.hdr_version)) {
+		ret = error("packfile %s is version %"PRIu32" and not"
+			    " supported (try upgrading GIT to a newer version)",
+			    p->pack_name, ntohl(hdr.hdr_version));
+		goto out;
+	}
 
 	/* Verify the pack matches its index. */
-	if (p->num_objects != ntohl(hdr.hdr_entries))
-		return error("packfile %s claims to have %"PRIu32" objects"
-			     " while index indicates %"PRIu32" objects",
-			     p->pack_name, ntohl(hdr.hdr_entries),
-			     p->num_objects);
+	if (p->num_objects != ntohl(hdr.hdr_entries)) {
+		ret = error("packfile %s claims to have %"PRIu32" objects"
+			    " while index indicates %"PRIu32" objects",
+			    p->pack_name, ntohl(hdr.hdr_entries),
+			    p->num_objects);
+		goto out;
+	}
+
 	read_result = pread_in_full(p->pack_fd, hash, hashsz,
 					p->pack_size - hashsz);
-	if (read_result < 0)
-		return error_errno("error reading from %s", p->pack_name);
-	if (read_result != hashsz)
-		return error("packfile %s signature is unavailable", p->pack_name);
+	if (read_result < 0) {
+		ret = error_errno("error reading from %s", p->pack_name);
+		goto out;
+	}
+	if (read_result != hashsz) {
+		ret = error("packfile %s signature is unavailable", p->pack_name);
+		goto out;
+	}
+
 	idx_hash = ((unsigned char *)p->index_data) + p->index_size - hashsz * 2;
-	if (!hasheq(hash, idx_hash, p->repo->hash_algo))
-		return error("packfile %s does not match index", p->pack_name);
-	return 0;
-}
+	if (!hasheq(hash, idx_hash, p->repo->hash_algo)) {
+		ret = error("packfile %s does not match index", p->pack_name);
+		goto out;
+	}
 
-static int open_packed_git(struct packed_git *p)
-{
-	if (!open_packed_git_1(p))
-		return 0;
-	packfile_close_fd(p);
-	return -1;
+	ret = 0;
+
+out:
+	if (ret < 0)
+		packfile_close_fd(p);
+	return ret;
 }
 
 static int in_window(struct repository *r, struct pack_window *win,
@@ -587,7 +612,7 @@ unsigned char *use_pack(struct packed_git *p,
 	 * hash, and the in_window function above wouldn't match
 	 * don't allow an offset too close to the end of the file.
 	 */
-	if (!p->pack_size && p->pack_fd == -1 && open_packed_git(p))
+	if (!p->pack_size && p->pack_fd == -1 && packfile_open_pack(p))
 		die("packfile %s cannot be accessed", p->pack_name);
 	if (offset > (p->pack_size - p->repo->hash_algo->rawsz))
 		die("offset beyond end of packfile (truncated pack?)");
@@ -612,7 +637,7 @@ unsigned char *use_pack(struct packed_git *p,
 
 			window_align = settings->packed_git_window_size / 2;
 
-			if (p->pack_fd == -1 && open_packed_git(p))
+			if (p->pack_fd == -1 && packfile_open_pack(p))
 				die("packfile %s cannot be accessed", p->pack_name);
 
 			CALLOC_ARRAY(win, 1);
@@ -1810,7 +1835,7 @@ int is_pack_valid(struct packed_git *p)
 	}
 
 	/* Force the pack to open to prove its valid. */
-	return !open_packed_git(p);
+	return !packfile_open_pack(p);
 }
 
 int packfile_fill_entry(struct packed_git *p,
-- 
2.56.1.831.g720f3f5937.dirty
Patrick Steinhardt· Nov 20, 2026, 12:02 UTC · re: Patrick Steinhardt · lore

[PATCH 4/6] packfile: introduce function to free a packfile

We have a bunch of callsites that want to free a packfile, which requires them to first close it followed by a call to free(3p). This feels rather leaky as an abstraction.

Introduce `packfile_free()` to handle this properly. This function will also be extended in a subsequent patch to maintain a list of all packs.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/index-pack.c | 10 +---------
 commit-graph.c       |  6 ++----
 connected.c          |  6 ++----
 http-walker.c        |  6 ++----
 http.c               |  2 +-
 midx-write.c         |  9 +++------
 odb/source-packed.c  |  2 +-
 packfile.c           | 10 ++++++++--
 packfile.h           |  1 +
 9 files changed, 21 insertions(+), 31 deletions(-)
Show changes to 9 files +21 −31

builtin/index-pack.c, commit-graph.c, connected.c, http-walker.c, http.c, midx-write.c, odb/source-packed.c, packfile.c, packfile.h

diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 71048a5a8af..663375fbb36 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1755,15 +1755,7 @@ static void read_idx_option(struct pack_idx_option *opts, const char *pack_name)
 	if (opts->version == 2)
 		read_v2_anomalous_offsets(p, opts);
 
-	/*
-	 * Get rid of the idx file as we do not need it anymore.
-	 * NEEDSWORK: extract this bit from free_pack_by_name() in
-	 * object-file.c, perhaps?  It shouldn't matter very much as we
-	 * know we haven't installed this pack (hence we never have
-	 * read anything from it).
-	 */
-	packfile_close_index(p);
-	free(p);
+	packfile_free(p);
 }
 
 static void show_pack_info(int stat_only)
diff --git a/commit-graph.c b/commit-graph.c
index c3b04e8c1f4..03df5fb7c1f 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -1958,14 +1958,12 @@ static int fill_oids_from_packs(struct write_commit_graph_context *ctx,
 		}
 		if (packfile_open_index(p)) {
 			ret = error(_("error opening index for %s"), pack_indexes->items[i].string);
-			packfile_close(p);
-			free(p);
+			packfile_free(p);
 			goto cleanup;
 		}
 		for_each_object_in_pack(p, add_packed_commits, ctx,
 					ODB_FOR_EACH_OBJECT_PACK_ORDER);
-		packfile_close(p);
-		free(p);
+		packfile_free(p);
 	}
 
 cleanup:
diff --git a/connected.c b/connected.c
index fbacff734b6..78f08b41383 100644
--- a/connected.c
+++ b/connected.c
@@ -238,9 +238,7 @@ int check_connected(struct repository *repo,
 		err = error_errno(_("failed to close rev-list's stdin"));
 
 	sigchain_pop(SIGPIPE);
-	if (new_pack) {
-		packfile_close(new_pack);
-		free(new_pack);
-	}
+	if (new_pack)
+		packfile_free(new_pack);
 	return finish_command(&rev_list) || err;
 }
diff --git a/http-walker.c b/http-walker.c
index de43d9cf673..616a5fd234d 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -587,10 +587,8 @@ static void cleanup(struct walker *walker)
 
 			alt_next = alt->next;
 
-			for (e = alt->packs.head; e; e = e->next) {
-				packfile_close(e->pack);
-				free(e->pack);
-			}
+			for (e = alt->packs.head; e; e = e->next)
+				packfile_free(e->pack);
 			packfile_list_clear(&alt->packs);
 
 			free(alt->base);
diff --git a/http.c b/http.c
index e86bebdffb2..d518683e15e 100644
--- a/http.c
+++ b/http.c
@@ -2661,7 +2661,7 @@ static int fetch_and_setup_pack_index(struct packfile_list *packs,
 	packfile_close_index(new_pack);
 	free(tmp_idx);
 	if (ret) {
-		free(new_pack);
+		packfile_free(new_pack);
 		return -1;
 	}
 
diff --git a/midx-write.c b/midx-write.c
index d5be4e40843..3e227bb5e57 100644
--- a/midx-write.c
+++ b/midx-write.c
@@ -182,8 +182,7 @@ static void add_pack_to_midx(const char *full_path, size_t full_path_len,
 		if (packfile_open_index(p)) {
 			warning(_("failed to open pack-index '%s'"),
 				full_path);
-			packfile_close(p);
-			free(p);
+			packfile_free(p);
 			return;
 		}
 
@@ -1827,10 +1826,8 @@ static int write_midx_internal(struct write_midx_opts *opts)
 
 cleanup:
 	for (size_t i = 0; i < ctx.nr; i++) {
-		if (ctx.info[i].p) {
-			packfile_close(ctx.info[i].p);
-			free(ctx.info[i].p);
-		}
+		if (ctx.info[i].p)
+			packfile_free(ctx.info[i].p);
 		free(ctx.info[i].pack_name);
 	}
 
diff --git a/odb/source-packed.c b/odb/source-packed.c
index 87e2dcdf52e..866b4e890b6 100644
--- a/odb/source-packed.c
+++ b/odb/source-packed.c
@@ -862,7 +862,7 @@ static void odb_source_packed_free(struct odb_source *source)
 
 	odb_source_close(source);
 	for (struct packfile_list_entry *e = packed->packs.head; e; e = e->next)
-		free(e->pack);
+		packfile_free(e->pack);
 	packfile_list_clear(&packed->packs);
 
 	strmap_clear(&packed->packs_by_path, 0);
diff --git a/packfile.c b/packfile.c
index 9b03258311f..077b0395a79 100644
--- a/packfile.c
+++ b/packfile.c
@@ -217,7 +217,7 @@ struct packed_git *parse_pack_index(struct repository *r, unsigned char *sha1,
 	free(path);
 	hashcpy(p->hash, sha1, p->repo->hash_algo);
 	if (check_packed_git_idx(idx_path, p)) {
-		free(p);
+		packfile_free(p);
 		return NULL;
 	}
 
@@ -732,7 +732,7 @@ struct packed_git *packfile_from_idx(struct repository *r, const char *path,
 
 	xsnprintf(p->pack_name + path_len, alloc - path_len, ".pack");
 	if (stat(p->pack_name, &st) || !S_ISREG(st.st_mode)) {
-		free(p);
+		packfile_free(p);
 		return NULL;
 	}
 
@@ -752,6 +752,12 @@ struct packed_git *packfile_from_idx(struct repository *r, const char *path,
 	return p;
 }
 
+void packfile_free(struct packed_git *p)
+{
+	packfile_close(p);
+	free(p);
+}
+
 void packfile_store_add_pack(struct odb_source_packed *store,
 			     struct packed_git *pack)
 {
diff --git a/packfile.h b/packfile.h
index f817bc69e29..f314a55a904 100644
--- a/packfile.h
+++ b/packfile.h
@@ -223,6 +223,7 @@ struct object_database;
 
 struct packed_git *packfile_from_idx(struct repository *r, const char *path,
 				     size_t path_len, int local);
+void packfile_free(struct packed_git *p);
 
 /*
  * mmap the index file for the specified packfile (if it is not
-- 
2.56.1.831.g720f3f5937.dirty
Patrick Steinhardt· Nov 20, 2026, 12:02 UTC · re: Patrick Steinhardt · lore

[PATCH 5/6] packfile: maintain process-wide limits globally

Our packfile subsystem has two limits that are supposed to apply globally across all open packfiles:

  - `pack_max_fds` is the maximum number of open file descriptors for
    packfiles. When we exceed that limit, we close least-recently-used
    packs until we're below it again.
  - "core.packedGitLimit" is the maximum number of packfile data that
    may be mapped into memory. When we exceed that limit, we close the
    least-recently-used windows into our packfiles.

Those limits are supposed to be process-wide, so even across multiple repositories. But our current implementation will only consider packfiles from a given repository or object database, which may not even have any packfiles open when the logic kicks in. Furthermore, the logic assumes that we're using the "files" backend and thus unconditionally downcasts the object database's backend to it.

Introduce a global list of all packfiles. This list is completely independent of the owning repository or object database, and it is agnostic of the actual object database backend that's in use. Use it to maintain the above two limits.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 packfile.c | 29 +++++++++++++----------------
 1 file changed, 13 insertions(+), 16 deletions(-)
Show changes to packfile.c +13 −16
diff --git a/packfile.c b/packfile.c
index 077b0395a79..0d5647c74c8 100644
--- a/packfile.c
+++ b/packfile.c
@@ -35,6 +35,7 @@ char *odb_pack_name(struct repository *r, struct strbuf *buf,
 	return buf->buf;
 }
 
+static struct packfile_list all_packfiles;
 static unsigned int pack_used_ctr;
 static unsigned int pack_open_fds;
 static unsigned int pack_max_fds;
@@ -195,6 +196,7 @@ static struct packed_git *packfile_new(struct repository *r, int extra)
 	memset(p, 0, sizeof(*p));
 	p->pack_fd = -1;
 	p->repo = r;
+	packfile_list_append(&all_packfiles, p, 1);
 	return p;
 }
 
@@ -243,17 +245,14 @@ static void scan_windows(struct packed_git *p,
 	}
 }
 
-static int unuse_one_window(struct object_database *odb)
+static int unuse_one_window(void)
 {
-	struct odb_source_files *files = odb_source_files_downcast(odb->source);
 	struct packfile_list_entry *e;
 	struct packed_git *lru_p = NULL;
 	struct pack_window *lru_w = NULL, *lru_l = NULL;
 
-	for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
-		for (e = dir->packed->packs.head; e; e = e->next)
-			scan_windows(e->pack, &lru_p, &lru_w, &lru_l);
-	}
+	for (e = all_packfiles.head; e; e = e->next)
+		scan_windows(e->pack, &lru_p, &lru_w, &lru_l);
 
 	if (lru_p) {
 		munmap(lru_w->base, lru_w->len);
@@ -407,20 +406,17 @@ static void find_lru_pack(struct packed_git *p, struct packed_git **lru_p, struc
 	*accept_windows_inuse = has_windows_inuse;
 }
 
-static int close_one_pack(struct repository *r)
+static int close_one_pack(void)
 {
-	struct odb_source_files *files = odb_source_files_downcast(r->objects->source);
 	struct packfile_list_entry *e;
 	struct packed_git *lru_p = NULL;
 	struct pack_window *mru_w = NULL;
 	int accept_windows_inuse = 1;
 
-	for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
-		for (e = dir->packed->packs.head; e; e = e->next) {
-			if (e->pack->pack_fd == -1)
-				continue;
-			find_lru_pack(e->pack, &lru_p, &mru_w, &accept_windows_inuse);
-		}
+	for (e = all_packfiles.head; e; e = e->next) {
+		if (e->pack->pack_fd == -1)
+			continue;
+		find_lru_pack(e->pack, &lru_p, &mru_w, &accept_windows_inuse);
 	}
 
 	if (lru_p)
@@ -505,7 +501,7 @@ static int packfile_open_pack(struct packed_git *p)
 			pack_max_fds = 1;
 	}
 
-	while (pack_max_fds <= pack_open_fds && close_one_pack(p->repo))
+	while (pack_max_fds <= pack_open_fds && close_one_pack())
 		; /* nothing */
 
 	p->pack_fd = git_open(p->pack_name);
@@ -649,7 +645,7 @@ unsigned char *use_pack(struct packed_git *p,
 			pack_mapped += win->len;
 
 			while (settings->packed_git_limit < pack_mapped &&
-			       unuse_one_window(p->repo->objects))
+			       unuse_one_window())
 				; /* nothing */
 			win->base = xmmap_gently(NULL, win->len,
 				PROT_READ, MAP_PRIVATE,
@@ -754,6 +750,7 @@ struct packed_git *packfile_from_idx(struct repository *r, const char *path,
 
 void packfile_free(struct packed_git *p)
 {
+	packfile_list_remove(&all_packfiles, p);
 	packfile_close(p);
 	free(p);
 }
-- 
2.56.1.831.g720f3f5937.dirty
Patrick Steinhardt· Nov 20, 2026, 12:02 UTC · re: Patrick Steinhardt · lore

[PATCH 6/6] packfile: skip closing packs marked as do-not-close

When closing a packfile to maintain the maximum number of open packs we completely ignore the `do_not_close` bit. That bit is only set by git-fast-import(1), and as that command allocates its packs manually it wouldn't add those packs to the list of all packfiles, and consequently we won't ever try to close such a packfile via `close_one_pack()`. But that feels like coincidence rather than proper design, and the result is rather fragile.

Skip over packfiles marked as do-not-close to reduce the fragility. This should not result in a user-visible change in behaviour.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 packfile.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
Show changes to packfile.c +2 −1
diff --git a/packfile.c b/packfile.c
index 0d5647c74c8..86ef3a79d9c 100644
--- a/packfile.c
+++ b/packfile.c
@@ -414,7 +414,8 @@ static int close_one_pack(void)
 	int accept_windows_inuse = 1;
 
 	for (e = all_packfiles.head; e; e = e->next) {
-		if (e->pack->pack_fd == -1)
+		if (e->pack->pack_fd == -1 ||
+		    e->pack->do_not_close)
 			continue;
 		find_lru_pack(e->pack, &lru_p, &mru_w, &accept_windows_inuse);
 	}
-- 
2.56.1.831.g720f3f5937.dirty

← back to recent threads