{"thread":{"id":"66896","subject":"[PATCH 0/6] packfile: some lifecycle improvements","startedAt":"2026-11-20T12:02:27Z","lastAt":"2026-11-20T12:02:33Z","messageCount":7,"participants":["Patrick Steinhardt"],"isPatch":true,"patchVersion":1,"patchTotal":6},"messages":[{"id":"558921","messageId":"20261120-pks-packfile-global-statistics-v1-0-19beb83b145f@pks.im","threadId":"66896","inReplyTo":null,"subject":"[PATCH 0/6] packfile: some lifecycle improvements","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-20T12:02:27Z","receivedAt":"2026-11-20T12:02:27Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Hi,\n\nthis small patch series polishes our lifecycle handling of packfiles a\nbit. The original motivation of this patch series is really only the\nlast patch, which fixes how we handle process-wide limitations. But I\ntook the chance to also polish this subsystem a bit, as a lot of its\nparts feel rather dated nowadays.\n\nThere's a minor semantic merge conflict with \"seen\" that can be sovled\nlike this:\n\ndiff --git a/odb/source-packed.c b/odb/source-packed.c\nindex 77f8227ae52..e24f6cc7ef9 100644\n--- a/odb/source-packed.c\n+++ b/odb/source-packed.c\n@@ -898,7 +898,7 @@ static int odb_source_packed_map_oid(struct odb_source *source,\n \t\t\t\t\t\t       &hash_algos[src->algo] :\n \t\t\t\t\t\t       repo->hash_algo;\n \n-\t\tif (open_pack_index(p) || !p->num_objects)\n+\t\tif (packfile_open_index(p) || !p->num_objects)\n \t\t\tcontinue;\n \t\t/* Only v3 indices store object IDs in both algorithms. */\n \t\tif (p->index_version < 3)\n\nThanks!\n\nPatrick\n\n---\nPatrick Steinhardt (6):\n      builtin/fast-import: stop reporting packfile statistics\n      packfile: rename lifecycle functions\n      packfile: inline `open_packed_git_1()`\n      packfile: introduce function to free a packfile\n      packfile: maintain process-wide limits globally\n      packfile: skip closing packs marked as do-not-close\n\n Documentation/git-fast-import.adoc |   5 +-\n builtin/count-objects.c            |   2 +-\n builtin/fast-import.c              |   8 +-\n builtin/index-pack.c               |  16 +--\n builtin/pack-objects.c             |   2 +-\n builtin/pack-redundant.c           |   2 +-\n commit-graph.c                     |  12 +-\n connected.c                        |  10 +-\n http-push.c                        |   2 +-\n http-walker.c                      |   8 +-\n http.c                             |   4 +-\n midx-write.c                       |  23 ++--\n midx.c                             |   6 +-\n odb/source-packed.c                |  14 +--\n pack-check.c                       |   2 +-\n pack-mtimes.c                      |   2 +-\n pack-revindex.c                    |   4 +-\n packfile.c                         | 233 ++++++++++++++++++-------------------\n packfile.h                         |  30 ++---\n repack-geometry.c                  |   2 +-\n repack-midx.c                      |   2 +-\n t/helper/test-bitmap.c             |   2 +-\n 22 files changed, 182 insertions(+), 209 deletions(-)\n\n\n---\nbase-commit: f7de81456940dd6f3274e461149bba12965de609\nchange-id: 20261120-pks-packfile-global-statistics-92a9be97a11f\n\n\n"},{"id":"558922","messageId":"20261120-pks-packfile-global-statistics-v1-1-19beb83b145f@pks.im","threadId":"66896","inReplyTo":"20261120-pks-packfile-global-statistics-v1-0-19beb83b145f@pks.im","subject":"[PATCH 1/6] builtin/fast-import: stop reporting packfile statistics","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-20T12:02:28Z","receivedAt":"2026-11-20T12:02:28Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"By default, git-fast-import(1) prints a bunch of statistics about all\nkinds of internal details of Git. This includes for example how many\nobjects have been allocated, the number of refs processed, or the amount\nof memory that was allocated.\n\nReporting this kind information is highly questionable, doubly so as it\nis done by default, as a user really shoudln't have to worry about such\ninternal implementation details. For some of the above metrics it could\nbe argued though that it's indeed interesting to learn about the number\nof objects processed to get a rough sense of what git-fast-import(1) has\ndone. But for other kind of information that's not the case, as it is\nwithout a doubt purely internal implementation details.\n\nOne of these reported implementation details is our packfile statistics.\nThese include information on the system's page size, the number of packs\nthat were opened, the number of calls to mmap(3) and how many pack\nwindows were created. Reporting that kind of information, and that even\nby default, is just completely uncalled for. This information may be\nuseful to a developer, but a user really shouldn't have to care about\nany of this.\n\nMore importantly though, with pluggable object databases we shouldn't\nassume that we're using packfiles in the first place anymore. The\ncurrent implementation of git-fast-import(1) still does, but we will\nconvert it to use transactions sooner rather than later.\n\nLet's just unceremoniously drop reporting packfile information to the\nuser. If anyone complains (which I highly doubt), then we may consider\nreintroducing this as trace2 data. The same is arguably true for many of\nthe other reported statistics. But for now I care about the packfile\nsubsystem, only, and hence those are left as-is.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n Documentation/git-fast-import.adoc |  5 ++---\n builtin/fast-import.c              |  4 +---\n packfile.c                         | 36 ------------------------------------\n packfile.h                         |  2 --\n 4 files changed, 3 insertions(+), 44 deletions(-)\n\ndiff --git a/Documentation/git-fast-import.adoc b/Documentation/git-fast-import.adoc\nindex 5201e493345..6afe14b650c 100644\n--- a/Documentation/git-fast-import.adoc\n+++ b/Documentation/git-fast-import.adoc\n@@ -48,9 +48,8 @@ OPTIONS\n \n --stats::\n \tDisplay some basic statistics about the objects fast-import has\n-\tcreated, the packfiles they were stored into, and the\n-\tmemory used by fast-import during this run.  Showing this output\n-\tis currently the default, but can be disabled with --quiet.\n+\tcreated, and the memory used by fast-import during this run.  Showing\n+\tthis output is currently the default, but can be disabled with --quiet.\n \n --allow-unsafe-features::\n \tMany command-line options can be provided as part of the\ndiff --git a/builtin/fast-import.c b/builtin/fast-import.c\nindex ab5d2a23448..14e75149415 100644\n--- a/builtin/fast-import.c\n+++ b/builtin/fast-import.c\n@@ -4153,7 +4153,7 @@ static int option_parse_early_allow_unsafe(const struct option *option,\n int cmd_fast_import(int argc,\n \t\t    const char **argv,\n \t\t    const char *prefix,\n-\t\t    struct repository *repo)\n+\t\t    struct repository *repo UNUSED)\n {\n \tstruct fast_import_state state;\n \n@@ -4346,8 +4346,6 @@ int cmd_fast_import(int argc,\n \t\tfprintf(stderr, \"       pools:    %10lu KiB\\n\", (unsigned long)((tree_entry_allocd + fi_mem_pool.pool_alloc) /1024));\n \t\tfprintf(stderr, \"     objects:    %10\" PRIuMAX \" KiB\\n\", (alloc_count*sizeof(struct object_entry))/1024);\n \t\tfprintf(stderr, \"---------------------------------------------------------------------\\n\");\n-\t\tpack_report(repo);\n-\t\tfprintf(stderr, \"---------------------------------------------------------------------\\n\");\n \t\tfprintf(stderr, \"\\n\");\n \t}\n \ndiff --git a/packfile.c b/packfile.c\nindex b32b4cd75da..5aa9b009174 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -36,38 +36,10 @@ char *odb_pack_name(struct repository *r, struct strbuf *buf,\n }\n \n static unsigned int pack_used_ctr;\n-static unsigned int pack_mmap_calls;\n-static unsigned int peak_pack_open_windows;\n-static unsigned int pack_open_windows;\n static unsigned int pack_open_fds;\n static unsigned int pack_max_fds;\n-static size_t peak_pack_mapped;\n static size_t pack_mapped;\n \n-#define SZ_FMT PRIuMAX\n-static inline uintmax_t sz_fmt(size_t s) { return s; }\n-\n-void pack_report(struct repository *repo)\n-{\n-\tfprintf(stderr,\n-\t\t\"pack_report: getpagesize()            = %10\" SZ_FMT \"\\n\"\n-\t\t\"pack_report: core.packedGitWindowSize = %10\" SZ_FMT \"\\n\"\n-\t\t\"pack_report: core.packedGitLimit      = %10\" SZ_FMT \"\\n\",\n-\t\tsz_fmt(getpagesize()),\n-\t\tsz_fmt(repo->settings.packed_git_window_size),\n-\t\tsz_fmt(repo->settings.packed_git_limit));\n-\tfprintf(stderr,\n-\t\t\"pack_report: pack_used_ctr            = %10u\\n\"\n-\t\t\"pack_report: pack_mmap_calls          = %10u\\n\"\n-\t\t\"pack_report: pack_open_windows        = %10u / %10u\\n\"\n-\t\t\"pack_report: pack_mapped              = \"\n-\t\t\t\"%10\" SZ_FMT \" / %10\" SZ_FMT \"\\n\",\n-\t\tpack_used_ctr,\n-\t\tpack_mmap_calls,\n-\t\tpack_open_windows, peak_pack_open_windows,\n-\t\tsz_fmt(pack_mapped), sz_fmt(peak_pack_mapped));\n-}\n-\n /*\n  * Open and mmap the index file at path, perform a couple of\n  * consistency checks, then record its information to p.  Return 0 on\n@@ -291,7 +263,6 @@ static int unuse_one_window(struct object_database *odb)\n \t\telse\n \t\t\tlru_p->windows = lru_w->next;\n \t\tfree(lru_w);\n-\t\tpack_open_windows--;\n \t\treturn 1;\n \t}\n \treturn 0;\n@@ -307,7 +278,6 @@ void close_pack_windows(struct packed_git *p)\n \t\t\t    p->pack_name);\n \t\tmunmap(w->base, w->len);\n \t\tpack_mapped -= w->len;\n-\t\tpack_open_windows--;\n \t\tp->windows = w->next;\n \t\tfree(w);\n \t}\n@@ -665,12 +635,6 @@ unsigned char *use_pack(struct packed_git *p,\n \t\t\tif (!win->offset && win->len == p->pack_size\n \t\t\t\t&& !p->do_not_close)\n \t\t\t\tclose_pack_fd(p);\n-\t\t\tpack_mmap_calls++;\n-\t\t\tpack_open_windows++;\n-\t\t\tif (pack_mapped > peak_pack_mapped)\n-\t\t\t\tpeak_pack_mapped = pack_mapped;\n-\t\t\tif (pack_open_windows > peak_pack_open_windows)\n-\t\t\t\tpeak_pack_open_windows = pack_open_windows;\n \t\t\twin->next = p->windows;\n \t\t\tp->windows = win;\n \t\t}\ndiff --git a/packfile.h b/packfile.h\nindex ed3378c4576..bab9931d00e 100644\n--- a/packfile.h\n+++ b/packfile.h\n@@ -217,8 +217,6 @@ int for_each_object_in_pack(struct packed_git *p,\n #define PACKDIR_FILE_GARBAGE 4\n extern void (*report_garbage)(unsigned seen_bits, const char *path);\n \n-void pack_report(struct repository *repo);\n-\n /*\n  * mmap the index file for the specified packfile (if it is not\n  * already mmapped).  Return 0 on success.\n\n-- \n2.56.1.831.g720f3f5937.dirty\n\n\n"},{"id":"558923","messageId":"20261120-pks-packfile-global-statistics-v1-2-19beb83b145f@pks.im","threadId":"66896","inReplyTo":"20261120-pks-packfile-global-statistics-v1-0-19beb83b145f@pks.im","subject":"[PATCH 2/6] packfile: rename lifecycle functions","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-20T12:02:29Z","receivedAt":"2026-11-20T12:02:29Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The packfile subsystem is kind of the wild west when it comes to\nfunction naming. None of the functions have proper subsystem prefixes\nand the naming of `struct packed_git` feels like a relict from the past.\n\nIdeally we'd modernize all of that, but that would lead to a huge amount\nof churn. Instead, let's improve the subsystem one step at a time and\nrename lifecycle-related functions to match modern best practices.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n builtin/count-objects.c  |  2 +-\n builtin/fast-import.c    |  4 ++--\n builtin/index-pack.c     |  8 ++++----\n builtin/pack-objects.c   |  2 +-\n builtin/pack-redundant.c |  2 +-\n commit-graph.c           | 10 +++++-----\n connected.c              |  6 +++---\n http-push.c              |  2 +-\n http-walker.c            |  4 ++--\n http.c                   |  2 +-\n midx-write.c             | 18 +++++++++---------\n midx.c                   |  6 +++---\n odb/source-packed.c      | 12 ++++++------\n pack-check.c             |  2 +-\n pack-mtimes.c            |  2 +-\n pack-revindex.c          |  4 ++--\n packfile.c               | 46 +++++++++++++++++++++++-----------------------\n packfile.h               | 29 +++++++++++++++--------------\n repack-geometry.c        |  2 +-\n repack-midx.c            |  2 +-\n t/helper/test-bitmap.c   |  2 +-\n 21 files changed, 84 insertions(+), 83 deletions(-)\n\ndiff --git a/builtin/count-objects.c b/builtin/count-objects.c\nindex 3f7b20fde53..09922ae4a70 100644\n--- a/builtin/count-objects.c\n+++ b/builtin/count-objects.c\n@@ -132,7 +132,7 @@ int cmd_count_objects(int argc,\n \t\trepo_for_each_pack(the_repository, p) {\n \t\t\tif (!p->pack_local)\n \t\t\t\tcontinue;\n-\t\t\tif (open_pack_index(p))\n+\t\t\tif (packfile_open_index(p))\n \t\t\t\tcontinue;\n \t\t\tpacked += p->num_objects;\n \t\t\tsize_pack += p->pack_size + p->index_size;\ndiff --git a/builtin/fast-import.c b/builtin/fast-import.c\nindex 14e75149415..5f034d39a71 100644\n--- a/builtin/fast-import.c\n+++ b/builtin/fast-import.c\n@@ -906,7 +906,7 @@ static void end_packfile(void)\n \t\tstruct branch *b;\n \t\tstruct tag *t;\n \n-\t\tclose_pack_windows(pack_data);\n+\t\tpackfile_close_windows(pack_data);\n \t\tfinalize_hashfile(pack_file, cur_pack_oid.hash, FSYNC_COMPONENT_PACK, 0);\n \t\tfixup_pack_header_footer(the_hash_algo, pack_data->pack_fd,\n \t\t\t\t\t pack_data->hash, pack_data->pack_name,\n@@ -1273,7 +1273,7 @@ static void *gfi_unpack_entry(\n \t\t * and updating the packfile length ensures we can read\n \t\t * the newly written data.\n \t\t */\n-\t\tclose_pack_windows(p);\n+\t\tpackfile_close_windows(p);\n \t\thashflush(pack_file);\n \n \t\t/* We have to offer rawsz bytes additional on the end of\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 0264c18c5f8..71048a5a8af 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1741,12 +1741,12 @@ static void read_v2_anomalous_offsets(struct packed_git *p,\n \n static void read_idx_option(struct pack_idx_option *opts, const char *pack_name)\n {\n-\tstruct packed_git *p = add_packed_git(the_repository, pack_name,\n-\t\t\t\t\t      strlen(pack_name), 1);\n+\tstruct packed_git *p = packfile_from_idx(the_repository, pack_name,\n+\t\t\t\t\t\t strlen(pack_name), 1);\n \n \tif (!p)\n \t\tdie(_(\"Cannot open existing pack file '%s'\"), pack_name);\n-\tif (open_pack_index(p))\n+\tif (packfile_open_index(p))\n \t\tdie(_(\"Cannot open existing pack idx file for '%s'\"), pack_name);\n \n \t/* Read the attributes from the existing idx file */\n@@ -1762,7 +1762,7 @@ static void read_idx_option(struct pack_idx_option *opts, const char *pack_name)\n \t * know we haven't installed this pack (hence we never have\n \t * read anything from it).\n \t */\n-\tclose_pack_index(p);\n+\tpackfile_close_index(p);\n \tfree(p);\n }\n \ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex 87c3102e622..9c375811937 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -4724,7 +4724,7 @@ static void loosen_unused_packed_objects(void)\n \t\tif (!p->pack_local || p->pack_keep || p->pack_keep_in_core)\n \t\t\tcontinue;\n \n-\t\tif (open_pack_index(p))\n+\t\tif (packfile_open_index(p))\n \t\t\tdie(_(\"cannot open pack index\"));\n \n \t\tfor (i = 0; i < p->num_objects; i++) {\ndiff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c\nindex 86749bb7e77..1b437bb6565 100644\n--- a/builtin/pack-redundant.c\n+++ b/builtin/pack-redundant.c\n@@ -546,7 +546,7 @@ static struct pack_list * add_pack(struct packed_git *p)\n \tl.pack = p;\n \tllist_init(&l.remaining_objects);\n \n-\tif (open_pack_index(p)) {\n+\tif (packfile_open_index(p)) {\n \t\tllist_free(l.remaining_objects);\n \t\treturn NULL;\n \t}\ndiff --git a/commit-graph.c b/commit-graph.c\nindex da65cd3deb9..c3b04e8c1f4 100644\n--- a/commit-graph.c\n+++ b/commit-graph.c\n@@ -1950,21 +1950,21 @@ static int fill_oids_from_packs(struct write_commit_graph_context *ctx,\n \tfor (i = 0; i < pack_indexes->nr; i++) {\n \t\tstruct packed_git *p;\n \n-\t\tp = add_packed_git(ctx->r, pack_indexes->items[i].string,\n-\t\t\t\t   strlen(pack_indexes->items[i].string), 1);\n+\t\tp = packfile_from_idx(ctx->r, pack_indexes->items[i].string,\n+\t\t\t\t      strlen(pack_indexes->items[i].string), 1);\n \t\tif (!p) {\n \t\t\tret = error(_(\"error adding pack %s\"), pack_indexes->items[i].string);\n \t\t\tgoto cleanup;\n \t\t}\n-\t\tif (open_pack_index(p)) {\n+\t\tif (packfile_open_index(p)) {\n \t\t\tret = error(_(\"error opening index for %s\"), pack_indexes->items[i].string);\n-\t\t\tclose_pack(p);\n+\t\t\tpackfile_close(p);\n \t\t\tfree(p);\n \t\t\tgoto cleanup;\n \t\t}\n \t\tfor_each_object_in_pack(p, add_packed_commits, ctx,\n \t\t\t\t\tODB_FOR_EACH_OBJECT_PACK_ORDER);\n-\t\tclose_pack(p);\n+\t\tpackfile_close(p);\n \t\tfree(p);\n \t}\n \ndiff --git a/connected.c b/connected.c\nindex bfb771e4695..fbacff734b6 100644\n--- a/connected.c\n+++ b/connected.c\n@@ -207,8 +207,8 @@ int check_connected(struct repository *repo,\n \t\tstrbuf_add(&idx_file, transport->pack_lockfiles.items[0].string,\n \t\t\t   base_len);\n \t\tstrbuf_addstr(&idx_file, \".idx\");\n-\t\tnew_pack = add_packed_git(repo, idx_file.buf,\n-\t\t\t\t\t  idx_file.len, 1);\n+\t\tnew_pack = packfile_from_idx(repo, idx_file.buf,\n+\t\t\t\t\t     idx_file.len, 1);\n \t\tstrbuf_release(&idx_file);\n \t}\n \n@@ -239,7 +239,7 @@ int check_connected(struct repository *repo,\n \n \tsigchain_pop(SIGPIPE);\n \tif (new_pack) {\n-\t\tclose_pack(new_pack);\n+\t\tpackfile_close(new_pack);\n \t\tfree(new_pack);\n \t}\n \treturn finish_command(&rev_list) || err;\ndiff --git a/http-push.c b/http-push.c\nindex b8f3faaed95..d5335c023c8 100644\n--- a/http-push.c\n+++ b/http-push.c\n@@ -318,7 +318,7 @@ static void start_fetch_packed(struct transfer_request *request)\n \t\trelease_request(request);\n \t\treturn;\n \t}\n-\tclose_pack_index(target);\n+\tpackfile_close_index(target);\n \trequest->target = target;\n \n \tfprintf(stderr,\t\"Fetching pack %s\\n\",\ndiff --git a/http-walker.c b/http-walker.c\nindex 4f50dd9b8d0..de43d9cf673 100644\n--- a/http-walker.c\n+++ b/http-walker.c\n@@ -435,7 +435,7 @@ static int http_fetch_pack(struct walker *walker, struct alt_base *repo,\n \ttarget = packfile_list_find_oid(repo->packs.head, oid);\n \tif (!target)\n \t\treturn -1;\n-\tclose_pack_index(target);\n+\tpackfile_close_index(target);\n \n \tif (walker->get_verbosely) {\n \t\tfprintf(stderr, \"Getting pack %s\\n\",\n@@ -588,7 +588,7 @@ static void cleanup(struct walker *walker)\n \t\t\talt_next = alt->next;\n \n \t\t\tfor (e = alt->packs.head; e; e = e->next) {\n-\t\t\t\tclose_pack(e->pack);\n+\t\t\t\tpackfile_close(e->pack);\n \t\t\t\tfree(e->pack);\n \t\t\t}\n \t\t\tpackfile_list_clear(&alt->packs);\ndiff --git a/http.c b/http.c\nindex d79678d01f0..e86bebdffb2 100644\n--- a/http.c\n+++ b/http.c\n@@ -2658,7 +2658,7 @@ static int fetch_and_setup_pack_index(struct packfile_list *packs,\n \n \tret = verify_pack_index(new_pack);\n \n-\tclose_pack_index(new_pack);\n+\tpackfile_close_index(new_pack);\n \tfree(tmp_idx);\n \tif (ret) {\n \t\tfree(new_pack);\ndiff --git a/midx-write.c b/midx-write.c\nindex 3038bbfad26..d5be4e40843 100644\n--- a/midx-write.c\n+++ b/midx-write.c\n@@ -172,17 +172,17 @@ static void add_pack_to_midx(const char *full_path, size_t full_path_len,\n \t\t\treturn;\n \n \t\tALLOC_GROW(ctx->info, ctx->nr + 1, ctx->alloc);\n-\t\tp = add_packed_git(ctx->repo, full_path, full_path_len, 0);\n+\t\tp = packfile_from_idx(ctx->repo, full_path, full_path_len, 0);\n \t\tif (!p) {\n \t\t\twarning(_(\"failed to add packfile '%s'\"),\n \t\t\t\tfull_path);\n \t\t\treturn;\n \t\t}\n \n-\t\tif (open_pack_index(p)) {\n+\t\tif (packfile_open_index(p)) {\n \t\t\twarning(_(\"failed to open pack-index '%s'\"),\n \t\t\t\tfull_path);\n-\t\t\tclose_pack(p);\n+\t\t\tpackfile_close(p);\n \t\t\tfree(p);\n \t\t\treturn;\n \t\t}\n@@ -1464,7 +1464,7 @@ static int write_midx_internal(struct write_midx_opts *opts)\n \t\t * Ignore failures as they can be expected and are not\n \t\t * fatal during this selection time.\n \t\t */\n-\t\topen_pack_index(oldest);\n+\t\tpackfile_open_index(oldest);\n \n \t\tif (opts->packs_to_drop && opts->packs_to_drop->nr)\n \t\t\tBUG(\"cannot write a MIDX bitmap during expiration\");\n@@ -1480,7 +1480,7 @@ static int write_midx_internal(struct write_midx_opts *opts)\n \n \t\t\tif (!oldest->num_objects || p->mtime < oldest->mtime) {\n \t\t\t\toldest = p;\n-\t\t\t\topen_pack_index(oldest);\n+\t\t\t\tpackfile_open_index(oldest);\n \t\t\t\tctx.preferred_pack_idx = i;\n \t\t\t}\n \t\t}\n@@ -1505,7 +1505,7 @@ static int write_midx_internal(struct write_midx_opts *opts)\n \tif (ctx.preferred_pack_idx != NO_PREFERRED_PACK) {\n \t\tstruct packed_git *preferred = ctx.info[ctx.preferred_pack_idx].p;\n \n-\t\tif (open_pack_index(preferred))\n+\t\tif (packfile_open_index(preferred))\n \t\t\tdie(_(\"failed to open preferred pack %s\"),\n \t\t\t    ctx.info[ctx.preferred_pack_idx].pack_name);\n \n@@ -1828,7 +1828,7 @@ static int write_midx_internal(struct write_midx_opts *opts)\n cleanup:\n \tfor (size_t i = 0; i < ctx.nr; i++) {\n \t\tif (ctx.info[i].p) {\n-\t\t\tclose_pack(ctx.info[i].p);\n+\t\t\tpackfile_close(ctx.info[i].p);\n \t\t\tfree(ctx.info[i].p);\n \t\t}\n \t\tfree(ctx.info[i].pack_name);\n@@ -1944,7 +1944,7 @@ int expire_midx_packs(struct odb_source_packed *source, unsigned flags)\n \t\t\tcontinue;\n \n \t\tpack_name = xstrdup(m->packs[i]->pack_name);\n-\t\tclose_pack(m->packs[i]);\n+\t\tpackfile_close(m->packs[i]);\n \n \t\tstring_list_insert(&packs_to_drop, m->pack_names[i]);\n \t\tunlink_pack_path(pack_name, 0);\n@@ -2000,7 +2000,7 @@ static int want_included_pack(struct multi_pack_index *m,\n \t\treturn 0;\n \tif (p->is_cruft)\n \t\treturn 0;\n-\tif (open_pack_index(p) || !p->num_objects)\n+\tif (packfile_open_index(p) || !p->num_objects)\n \t\treturn 0;\n \treturn 1;\n }\ndiff --git a/midx.c b/midx.c\nindex 2d4feeeadeb..d2b8a3a4a5e 100644\n--- a/midx.c\n+++ b/midx.c\n@@ -1023,8 +1023,8 @@ int verify_midx_file(struct odb_source_packed *source, unsigned flags)\n \t\t\tuint32_t pack_int_id = pairs[i-1].pack_int_id;\n \t\t\tstruct packed_git *p = nth_midxed_pack(m, pack_int_id);\n \n-\t\t\tclose_pack_fd(p);\n-\t\t\tclose_pack_index(p);\n+\t\t\tpackfile_close_fd(p);\n+\t\t\tpackfile_close_index(p);\n \t\t}\n \n \t\tnth_midxed_object_oid(&oid, m, pairs[i].pos);\n@@ -1035,7 +1035,7 @@ int verify_midx_file(struct odb_source_packed *source, unsigned flags)\n \t\t\tcontinue;\n \t\t}\n \n-\t\tif (open_pack_index(e.p)) {\n+\t\tif (packfile_open_index(e.p)) {\n \t\t\tmidx_report(_(\"failed to load pack-index for packfile %s\"),\n \t\t\t\t    e.p->pack_name);\n \t\t\tbreak;\ndiff --git a/odb/source-packed.c b/odb/source-packed.c\nindex 7fcbbb4fe9e..87e2dcdf52e 100644\n--- a/odb/source-packed.c\n+++ b/odb/source-packed.c\n@@ -351,7 +351,7 @@ static int odb_source_packed_for_each_prefixed_object(\n \t\tif (should_exclude_pack(e->pack, opts->flags))\n \t\t\tcontinue;\n \n-\t\tif (open_pack_index(e->pack)) {\n+\t\tif (packfile_open_index(e->pack)) {\n \t\t\tpack_errors = true;\n \t\t\tcontinue;\n \t\t}\n@@ -459,7 +459,7 @@ static int odb_source_packed_for_each_object(struct odb_source *source,\n \t\tif (bitmap && bitmap_index_contains_pack(bitmap, p))\n \t\t\tcontinue;\n \n-\t\tif (open_pack_index(p)) {\n+\t\tif (packfile_open_index(p)) {\n \t\t\tpack_errors = 1;\n \t\t\tcontinue;\n \t\t}\n@@ -498,7 +498,7 @@ static int odb_source_packed_count_objects(struct odb_source *source,\n \tfor (e = packfile_store_get_packs(packed); e; e = e->next) {\n \t\tif (e->pack->multi_pack_index)\n \t\t\tcontinue;\n-\t\tif (open_pack_index(e->pack)) {\n+\t\tif (packfile_open_index(e->pack)) {\n \t\t\tret = -1;\n \t\t\tgoto out;\n \t\t}\n@@ -615,7 +615,7 @@ static int odb_source_packed_find_abbrev_len(struct odb_source *source,\n \tfor (e = packfile_store_get_packs(packed); e; e = e->next) {\n \t\tif (e->pack->multi_pack_index)\n \t\t\tcontinue;\n-\t\tif (open_pack_index(e->pack) || !e->pack->num_objects)\n+\t\tif (packfile_open_index(e->pack) || !e->pack->num_objects)\n \t\t\tcontinue;\n \n \t\tfind_abbrev_len_for_pack(e->pack, oid, min_len, &min_len);\n@@ -847,7 +847,7 @@ static void odb_source_packed_close(struct odb_source *source)\n \tfor (struct packfile_list_entry *e = packed->packs.head; e; e = e->next) {\n \t\tif (e->pack->do_not_close)\n \t\t\tBUG(\"want to close pack marked 'do-not-close'\");\n-\t\tclose_pack(e->pack);\n+\t\tpackfile_close(e->pack);\n \t}\n \tif (packed->midx)\n \t\tclose_midx(packed->midx);\n@@ -880,7 +880,7 @@ static int verify_packs(struct odb_source_packed *source,\n \n \tif (opts->flags & ODB_FSCK_PROGRESS) {\n \t\tfor (e = packfile_store_get_packs(source); e; e = e->next) {\n-\t\t\tif (open_pack_index(e->pack))\n+\t\t\tif (packfile_open_index(e->pack))\n \t\t\t\tcontinue;\n \t\t\ttotal += e->pack->num_objects;\n \t\t}\ndiff --git a/pack-check.c b/pack-check.c\nindex 605811b7b09..e6663bb8e16 100644\n--- a/pack-check.c\n+++ b/pack-check.c\n@@ -184,7 +184,7 @@ int verify_pack_index(struct packed_git *p)\n {\n \tint err = 0;\n \n-\tif (open_pack_index(p))\n+\tif (packfile_open_index(p))\n \t\treturn error(\"packfile %s index not opened\", p->pack_name);\n \n \t/* Verify SHA1 sum of the index file */\ndiff --git a/pack-mtimes.c b/pack-mtimes.c\nindex 8e1f2dec0ef..8ce9c808e7f 100644\n--- a/pack-mtimes.c\n+++ b/pack-mtimes.c\n@@ -106,7 +106,7 @@ int load_pack_mtimes(struct packed_git *p)\n \tif (p->mtimes_map)\n \t\treturn ret; /* already loaded */\n \n-\tret = open_pack_index(p);\n+\tret = packfile_open_index(p);\n \tif (ret < 0)\n \t\tgoto cleanup;\n \ndiff --git a/pack-revindex.c b/pack-revindex.c\nindex 62387ae6320..5de5e4064ba 100644\n--- a/pack-revindex.c\n+++ b/pack-revindex.c\n@@ -176,7 +176,7 @@ static int create_pack_revindex_in_memory(struct packed_git *p)\n \tif (git_env_bool(GIT_TEST_REV_INDEX_DIE_IN_MEMORY, 0))\n \t\tdie(\"dying as requested by '%s'\",\n \t\t    GIT_TEST_REV_INDEX_DIE_IN_MEMORY);\n-\tif (open_pack_index(p))\n+\tif (packfile_open_index(p))\n \t\treturn -1;\n \tcreate_pack_revindex(p);\n \treturn 0;\n@@ -281,7 +281,7 @@ int load_pack_revindex_from_disk(struct packed_git *p)\n \tif (p->revindex_data)\n \t\treturn 0;\n \n-\tif (open_pack_index(p))\n+\tif (packfile_open_index(p))\n \t\treturn -1;\n \n \trevindex_name = pack_revindex_filename(p);\ndiff --git a/packfile.c b/packfile.c\nindex 5aa9b009174..95a97c302c7 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -155,7 +155,7 @@ int load_idx(const char *path, const unsigned int hashsz, void *idx_map,\n \treturn 0;\n }\n \n-int open_pack_index(struct packed_git *p)\n+int packfile_open_index(struct packed_git *p)\n {\n \tchar *idx_name;\n \tsize_t len;\n@@ -177,7 +177,7 @@ uint32_t get_pack_fanout(struct packed_git *p, uint32_t value)\n \tconst uint32_t *level1_ofs = p->index_data;\n \n \tif (!level1_ofs) {\n-\t\tif (open_pack_index(p))\n+\t\tif (packfile_open_index(p))\n \t\t\treturn 0;\n \t\tlevel1_ofs = p->index_data;\n \t}\n@@ -189,7 +189,7 @@ uint32_t get_pack_fanout(struct packed_git *p, uint32_t value)\n \treturn ntohl(level1_ofs[value]);\n }\n \n-static struct packed_git *alloc_packed_git(struct repository *r, int extra)\n+static struct packed_git *packfile_new(struct repository *r, int extra)\n {\n \tstruct packed_git *p = xmalloc(st_add(sizeof(*p), extra));\n \tmemset(p, 0, sizeof(*p));\n@@ -211,7 +211,7 @@ struct packed_git *parse_pack_index(struct repository *r, unsigned char *sha1,\n {\n \tchar *path = pack_path_from_idx(idx_path);\n \tsize_t alloc = st_add(strlen(path), 1);\n-\tstruct packed_git *p = alloc_packed_git(r, alloc);\n+\tstruct packed_git *p = packfile_new(r, alloc);\n \n \tmemcpy(p->pack_name, path, alloc); /* includes NUL */\n \tfree(path);\n@@ -268,7 +268,7 @@ static int unuse_one_window(struct object_database *odb)\n \treturn 0;\n }\n \n-void close_pack_windows(struct packed_git *p)\n+void packfile_close_windows(struct packed_git *p)\n {\n \twhile (p->windows) {\n \t\tstruct pack_window *w = p->windows;\n@@ -283,7 +283,7 @@ void close_pack_windows(struct packed_git *p)\n \t}\n }\n \n-int close_pack_fd(struct packed_git *p)\n+int packfile_close_fd(struct packed_git *p)\n {\n \tif (p->pack_fd < 0)\n \t\treturn 0;\n@@ -295,7 +295,7 @@ int close_pack_fd(struct packed_git *p)\n \treturn 1;\n }\n \n-void close_pack_index(struct packed_git *p)\n+void packfile_close_index(struct packed_git *p)\n {\n \tif (p->index_data) {\n \t\tmunmap((void *)p->index_data, p->index_size);\n@@ -424,7 +424,7 @@ static int close_one_pack(struct repository *r)\n \t}\n \n \tif (lru_p)\n-\t\treturn close_pack_fd(lru_p);\n+\t\treturn packfile_close_fd(lru_p);\n \n \treturn 0;\n }\n@@ -493,7 +493,7 @@ static int open_packed_git_1(struct packed_git *p)\n \tssize_t read_result;\n \tconst unsigned hashsz = p->repo->hash_algo->rawsz;\n \n-\tif (open_pack_index(p))\n+\tif (packfile_open_index(p))\n \t\treturn error(\"packfile %s index unavailable\", p->pack_name);\n \n \tif (!pack_max_fds) {\n@@ -557,7 +557,7 @@ static int open_packed_git(struct packed_git *p)\n {\n \tif (!open_packed_git_1(p))\n \t\treturn 0;\n-\tclose_pack_fd(p);\n+\tpackfile_close_fd(p);\n \treturn -1;\n }\n \n@@ -634,7 +634,7 @@ unsigned char *use_pack(struct packed_git *p,\n \t\t\t\t\t  p->pack_name, mmap_os_err());\n \t\t\tif (!win->offset && win->len == p->pack_size\n \t\t\t\t&& !p->do_not_close)\n-\t\t\t\tclose_pack_fd(p);\n+\t\t\t\tpackfile_close_fd(p);\n \t\t\twin->next = p->windows;\n \t\t\tp->windows = win;\n \t\t}\n@@ -659,8 +659,8 @@ void unuse_pack(struct pack_window **w_cursor)\n \t}\n }\n \n-struct packed_git *add_packed_git(struct repository *r, const char *path,\n-\t\t\t\t  size_t path_len, int local)\n+struct packed_git *packfile_from_idx(struct repository *r, const char *path,\n+\t\t\t\t     size_t path_len, int local)\n {\n \tstruct stat st;\n \tsize_t alloc;\n@@ -679,7 +679,7 @@ struct packed_git *add_packed_git(struct repository *r, const char *path,\n \t * the use xsnprintf double-checks that)\n \t */\n \talloc = st_add3(path_len, strlen(\".promisor\"), 1);\n-\tp = alloc_packed_git(r, alloc);\n+\tp = packfile_new(r, alloc);\n \tmemcpy(p->pack_name, path, path_len);\n \n \t/*\n@@ -753,8 +753,8 @@ struct packed_git *packfile_store_load_pack(struct odb_source_packed *store,\n \n \tp = strmap_get(&store->packs_by_path, key.buf);\n \tif (!p) {\n-\t\tp = add_packed_git(store->base.odb->repo, idx_path,\n-\t\t\t\t   strlen(idx_path), local);\n+\t\tp = packfile_from_idx(store->base.odb->repo, idx_path,\n+\t\t\t\t      strlen(idx_path), local);\n \t\tif (p)\n \t\t\tpackfile_store_add_pack(store, p);\n \t}\n@@ -875,7 +875,7 @@ size_t get_size_from_delta(struct packed_git *p,\n \t\t * counter is incremented before window reading and checked\n \t\t * before window disposal.\n \t\t *\n-\t\t * Other worrying sections could be the call to close_pack_fd(),\n+\t\t * Other worrying sections could be the call to packfile_close_fd(),\n \t\t * which can close packs even with in-use windows, and to\n \t\t * odb_reprepare(). Regarding the former, mmap doc says:\n \t\t * \"closing the file descriptor does not unmap the region\". And\n@@ -1227,11 +1227,11 @@ static void delta_base_cache_evict_entry(struct packed_git *p)\n \t}\n }\n \n-void close_pack(struct packed_git *p)\n+void packfile_close(struct packed_git *p)\n {\n-\tclose_pack_windows(p);\n-\tclose_pack_fd(p);\n-\tclose_pack_index(p);\n+\tpackfile_close_windows(p);\n+\tpackfile_close_fd(p);\n+\tpackfile_close_index(p);\n \tclose_pack_revindex(p);\n \tclose_pack_mtimes(p);\n \toidset_clear(&p->bad_objects);\n@@ -1725,7 +1725,7 @@ int nth_packed_object_id(struct object_id *oid,\n \tconst unsigned char *index = p->index_data;\n \tconst unsigned int hashsz = p->repo->hash_algo->rawsz;\n \tif (!index) {\n-\t\tif (open_pack_index(p))\n+\t\tif (packfile_open_index(p))\n \t\t\treturn -1;\n \t\tindex = p->index_data;\n \t}\n@@ -1783,7 +1783,7 @@ off_t find_pack_entry_one(const struct object_id *oid,\n \tuint32_t result;\n \n \tif (!index) {\n-\t\tif (open_pack_index(p))\n+\t\tif (packfile_open_index(p))\n \t\t\treturn 0;\n \t}\n \ndiff --git a/packfile.h b/packfile.h\nindex bab9931d00e..f817bc69e29 100644\n--- a/packfile.h\n+++ b/packfile.h\n@@ -180,7 +180,7 @@ const char *pack_basename(struct packed_git *p);\n  * You probably don't want to use this function! It skips most of the normal\n  * sanity checks (including whether we even have the matching .pack file),\n  * and does not add the resulting packed_git struct to the internal list of\n- * packs. You probably want add_packed_git() instead.\n+ * packs. You probably want packfile_from_idx() instead.\n  */\n struct packed_git *parse_pack_index(struct repository *r, unsigned char *sha1,\n \t\t\t\t    const char *idx_path);\n@@ -217,32 +217,33 @@ int for_each_object_in_pack(struct packed_git *p,\n #define PACKDIR_FILE_GARBAGE 4\n extern void (*report_garbage)(unsigned seen_bits, const char *path);\n \n+uint32_t get_pack_fanout(struct packed_git *p, uint32_t value);\n+\n+struct object_database;\n+\n+struct packed_git *packfile_from_idx(struct repository *r, const char *path,\n+\t\t\t\t     size_t path_len, int local);\n+\n /*\n  * mmap the index file for the specified packfile (if it is not\n  * already mmapped).  Return 0 on success.\n  */\n-int open_pack_index(struct packed_git *);\n+int packfile_open_index(struct packed_git *);\n+\n+void packfile_close(struct packed_git *);\n+int packfile_close_fd(struct packed_git *p);\n+void packfile_close_windows(struct packed_git *);\n \n /*\n  * munmap the index file for the specified packfile (if it is\n  * currently mmapped).\n  */\n-void close_pack_index(struct packed_git *);\n-\n-int close_pack_fd(struct packed_git *p);\n-\n-uint32_t get_pack_fanout(struct packed_git *p, uint32_t value);\n-\n-struct object_database;\n+void packfile_close_index(struct packed_git *);\n \n unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t,\n \t\t\tsize_t *);\n-void close_pack_windows(struct packed_git *);\n-void close_pack(struct packed_git *);\n void unuse_pack(struct pack_window **);\n void clear_delta_base_cache(void);\n-struct packed_git *add_packed_git(struct repository *r, const char *path,\n-\t\t\t\t  size_t path_len, int local);\n \n /*\n  * Unlink the .pack and associated extension files.\n@@ -345,7 +346,7 @@ int is_promisor_object(struct repository *r, const struct object_id *oid);\n  *\n  * This function should not be used directly. It is exposed here only so that we\n  * have a convenient entry-point for fuzz testing. For real uses, you should\n- * probably use open_pack_index() instead.\n+ * probably use packfile_open_index() instead.\n  */\n int load_idx(const char *path, const unsigned int hashsz, void *idx_map,\n \t     size_t idx_size, struct packed_git *p);\ndiff --git a/repack-geometry.c b/repack-geometry.c\nindex b541c34af5f..671a3dfe6a3 100644\n--- a/repack-geometry.c\n+++ b/repack-geometry.c\n@@ -9,7 +9,7 @@\n \n static uint32_t pack_geometry_weight(struct packed_git *p)\n {\n-\tif (open_pack_index(p))\n+\tif (packfile_open_index(p))\n \t\tdie(_(\"cannot open index for %s\"), p->pack_name);\n \treturn p->num_objects;\n }\ndiff --git a/repack-midx.c b/repack-midx.c\nindex 72810034733..7051a0d0637 100644\n--- a/repack-midx.c\n+++ b/repack-midx.c\n@@ -576,7 +576,7 @@ static int midx_compaction_step_include_packs(struct midx_compaction_step *step,\n \t\tstrbuf_reset(&path);\n \t\tstrbuf_addf(&path, \"%s/%s\", opts->packdir, item->string);\n \t\tp = packfile_store_load_pack(files->dirs->packed, path.buf, 1);\n-\t\tif (!p || open_pack_index(p)) {\n+\t\tif (!p || packfile_open_index(p)) {\n \t\t\tret = error(_(\"cannot open index for %s\"), path.buf);\n \t\t\tgoto out;\n \t\t}\ndiff --git a/t/helper/test-bitmap.c b/t/helper/test-bitmap.c\nindex 8547ef67e24..e3a3c8ff5cc 100644\n--- a/t/helper/test-bitmap.c\n+++ b/t/helper/test-bitmap.c\n@@ -87,7 +87,7 @@ static int bitmap_write(const char *basename)\n \tif (!p)\n \t\tdie(\"could not find pack '%s'\", basename);\n \n-\tif (open_pack_index(p))\n+\tif (packfile_open_index(p))\n \t\tdie(\"cannot open pack index for '%s'\", p->pack_name);\n \n \tprepare_packing_data(the_repository, &packed);\n\n-- \n2.56.1.831.g720f3f5937.dirty\n\n\n"},{"id":"558924","messageId":"20261120-pks-packfile-global-statistics-v1-3-19beb83b145f@pks.im","threadId":"66896","inReplyTo":"20261120-pks-packfile-global-statistics-v1-0-19beb83b145f@pks.im","subject":"[PATCH 3/6] packfile: inline `open_packed_git_1()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-20T12:02:30Z","receivedAt":"2026-11-20T12:02:30Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The function `open_packed_git()` is responsible for opening a packfile.\nThe majority of work is not performed by it though, but instead by its\n`_1()` sibling. But this sibling really only exists because we thought\nit too hard to handle errors properly and close the packfile's file\ndescriptor on an error path. So `_1()` doesn't handle those, and instead\ndefers to its only caller to close the file descriptor in the error path\nagain.\n\nFix this design smell by introducing a common exit path that knows to\nclose the file descriptor on error. This allows us to inline the call to\n`open_packed_git_1()`.\n\nWhile at it, rename the function to `packfile_open_pack()` to match\nmodern best practices. Besides that, it also gives the caller a better\nidea of _what_ is being opened, as we also have functions that open\nother files that belong to a packfile.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n packfile.c | 115 +++++++++++++++++++++++++++++++++++++------------------------\n 1 file changed, 70 insertions(+), 45 deletions(-)\n\ndiff --git a/packfile.c b/packfile.c\nindex 95a97c302c7..9b03258311f 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -480,11 +480,7 @@ const char *pack_basename(struct packed_git *p)\n \treturn ret;\n }\n \n-/*\n- * Do not call this directly as this leaks p->pack_fd on error return;\n- * call open_packed_git() instead.\n- */\n-static int open_packed_git_1(struct packed_git *p)\n+static int packfile_open_pack(struct packed_git *p)\n {\n \tstruct stat st;\n \tstruct pack_header hdr;\n@@ -492,9 +488,12 @@ static int open_packed_git_1(struct packed_git *p)\n \tunsigned char *idx_hash;\n \tssize_t read_result;\n \tconst unsigned hashsz = p->repo->hash_algo->rawsz;\n+\tint ret;\n \n-\tif (packfile_open_index(p))\n-\t\treturn error(\"packfile %s index unavailable\", p->pack_name);\n+\tif (packfile_open_index(p)) {\n+\t\tret = error(\"packfile %s index unavailable\", p->pack_name);\n+\t\tgoto out;\n+\t}\n \n \tif (!pack_max_fds) {\n \t\tunsigned int max_fds = get_max_fd_limit();\n@@ -510,55 +509,81 @@ static int open_packed_git_1(struct packed_git *p)\n \t\t; /* nothing */\n \n \tp->pack_fd = git_open(p->pack_name);\n-\tif (p->pack_fd < 0 || fstat(p->pack_fd, &st))\n-\t\treturn -1;\n+\tif (p->pack_fd < 0 || fstat(p->pack_fd, &st)) {\n+\t\tret = -1;\n+\t\tgoto out;\n+\t}\n \tpack_open_fds++;\n \n \t/* If we created the struct before we had the pack we lack size. */\n \tif (!p->pack_size) {\n-\t\tif (!S_ISREG(st.st_mode))\n-\t\t\treturn error(\"packfile %s not a regular file\", p->pack_name);\n+\t\tif (!S_ISREG(st.st_mode)) {\n+\t\t\tret = error(\"packfile %s not a regular file\", p->pack_name);\n+\t\t\tgoto out;\n+\t\t}\n+\n \t\tp->pack_size = st.st_size;\n-\t} else if (p->pack_size != st.st_size)\n-\t\treturn error(\"packfile %s size changed\", p->pack_name);\n+\t} else if (p->pack_size != st.st_size) {\n+\t\tret = error(\"packfile %s size changed\", p->pack_name);\n+\t\tgoto out;\n+\t}\n \n \t/* Verify we recognize this pack file format. */\n \tread_result = read_in_full(p->pack_fd, &hdr, sizeof(hdr));\n-\tif (read_result < 0)\n-\t\treturn error_errno(\"error reading from %s\", p->pack_name);\n-\tif (read_result != sizeof(hdr))\n-\t\treturn error(\"file %s is far too short to be a packfile\", p->pack_name);\n-\tif (hdr.hdr_signature != htonl(PACK_SIGNATURE))\n-\t\treturn error(\"file %s is not a GIT packfile\", p->pack_name);\n-\tif (!pack_version_ok(hdr.hdr_version))\n-\t\treturn error(\"packfile %s is version %\"PRIu32\" and not\"\n-\t\t\t\" supported (try upgrading GIT to a newer version)\",\n-\t\t\tp->pack_name, ntohl(hdr.hdr_version));\n+\tif (read_result < 0) {\n+\t\tret = error_errno(\"error reading from %s\", p->pack_name);\n+\t\tgoto out;\n+\t}\n+\n+\tif (read_result != sizeof(hdr)) {\n+\t\tret = error(\"file %s is far too short to be a packfile\", p->pack_name);\n+\t\tgoto out;\n+\t}\n+\n+\tif (hdr.hdr_signature != htonl(PACK_SIGNATURE)) {\n+\t\tret = error(\"file %s is not a GIT packfile\", p->pack_name);\n+\t\tgoto out;\n+\t}\n+\n+\tif (!pack_version_ok(hdr.hdr_version)) {\n+\t\tret = error(\"packfile %s is version %\"PRIu32\" and not\"\n+\t\t\t    \" supported (try upgrading GIT to a newer version)\",\n+\t\t\t    p->pack_name, ntohl(hdr.hdr_version));\n+\t\tgoto out;\n+\t}\n \n \t/* Verify the pack matches its index. */\n-\tif (p->num_objects != ntohl(hdr.hdr_entries))\n-\t\treturn error(\"packfile %s claims to have %\"PRIu32\" objects\"\n-\t\t\t     \" while index indicates %\"PRIu32\" objects\",\n-\t\t\t     p->pack_name, ntohl(hdr.hdr_entries),\n-\t\t\t     p->num_objects);\n+\tif (p->num_objects != ntohl(hdr.hdr_entries)) {\n+\t\tret = error(\"packfile %s claims to have %\"PRIu32\" objects\"\n+\t\t\t    \" while index indicates %\"PRIu32\" objects\",\n+\t\t\t    p->pack_name, ntohl(hdr.hdr_entries),\n+\t\t\t    p->num_objects);\n+\t\tgoto out;\n+\t}\n+\n \tread_result = pread_in_full(p->pack_fd, hash, hashsz,\n \t\t\t\t\tp->pack_size - hashsz);\n-\tif (read_result < 0)\n-\t\treturn error_errno(\"error reading from %s\", p->pack_name);\n-\tif (read_result != hashsz)\n-\t\treturn error(\"packfile %s signature is unavailable\", p->pack_name);\n+\tif (read_result < 0) {\n+\t\tret = error_errno(\"error reading from %s\", p->pack_name);\n+\t\tgoto out;\n+\t}\n+\tif (read_result != hashsz) {\n+\t\tret = error(\"packfile %s signature is unavailable\", p->pack_name);\n+\t\tgoto out;\n+\t}\n+\n \tidx_hash = ((unsigned char *)p->index_data) + p->index_size - hashsz * 2;\n-\tif (!hasheq(hash, idx_hash, p->repo->hash_algo))\n-\t\treturn error(\"packfile %s does not match index\", p->pack_name);\n-\treturn 0;\n-}\n+\tif (!hasheq(hash, idx_hash, p->repo->hash_algo)) {\n+\t\tret = error(\"packfile %s does not match index\", p->pack_name);\n+\t\tgoto out;\n+\t}\n \n-static int open_packed_git(struct packed_git *p)\n-{\n-\tif (!open_packed_git_1(p))\n-\t\treturn 0;\n-\tpackfile_close_fd(p);\n-\treturn -1;\n+\tret = 0;\n+\n+out:\n+\tif (ret < 0)\n+\t\tpackfile_close_fd(p);\n+\treturn ret;\n }\n \n static int in_window(struct repository *r, struct pack_window *win,\n@@ -587,7 +612,7 @@ unsigned char *use_pack(struct packed_git *p,\n \t * hash, and the in_window function above wouldn't match\n \t * don't allow an offset too close to the end of the file.\n \t */\n-\tif (!p->pack_size && p->pack_fd == -1 && open_packed_git(p))\n+\tif (!p->pack_size && p->pack_fd == -1 && packfile_open_pack(p))\n \t\tdie(\"packfile %s cannot be accessed\", p->pack_name);\n \tif (offset > (p->pack_size - p->repo->hash_algo->rawsz))\n \t\tdie(\"offset beyond end of packfile (truncated pack?)\");\n@@ -612,7 +637,7 @@ unsigned char *use_pack(struct packed_git *p,\n \n \t\t\twindow_align = settings->packed_git_window_size / 2;\n \n-\t\t\tif (p->pack_fd == -1 && open_packed_git(p))\n+\t\t\tif (p->pack_fd == -1 && packfile_open_pack(p))\n \t\t\t\tdie(\"packfile %s cannot be accessed\", p->pack_name);\n \n \t\t\tCALLOC_ARRAY(win, 1);\n@@ -1810,7 +1835,7 @@ int is_pack_valid(struct packed_git *p)\n \t}\n \n \t/* Force the pack to open to prove its valid. */\n-\treturn !open_packed_git(p);\n+\treturn !packfile_open_pack(p);\n }\n \n int packfile_fill_entry(struct packed_git *p,\n\n-- \n2.56.1.831.g720f3f5937.dirty\n\n\n"},{"id":"558925","messageId":"20261120-pks-packfile-global-statistics-v1-4-19beb83b145f@pks.im","threadId":"66896","inReplyTo":"20261120-pks-packfile-global-statistics-v1-0-19beb83b145f@pks.im","subject":"[PATCH 4/6] packfile: introduce function to free a packfile","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-20T12:02:31Z","receivedAt":"2026-11-20T12:02:31Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"We have a bunch of callsites that want to free a packfile, which\nrequires them to first close it followed by a call to free(3p). This\nfeels rather leaky as an abstraction.\n\nIntroduce `packfile_free()` to handle this properly. This function will\nalso be extended in a subsequent patch to maintain a list of all packs.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n builtin/index-pack.c | 10 +---------\n commit-graph.c       |  6 ++----\n connected.c          |  6 ++----\n http-walker.c        |  6 ++----\n http.c               |  2 +-\n midx-write.c         |  9 +++------\n odb/source-packed.c  |  2 +-\n packfile.c           | 10 ++++++++--\n packfile.h           |  1 +\n 9 files changed, 21 insertions(+), 31 deletions(-)\n\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 71048a5a8af..663375fbb36 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1755,15 +1755,7 @@ static void read_idx_option(struct pack_idx_option *opts, const char *pack_name)\n \tif (opts->version == 2)\n \t\tread_v2_anomalous_offsets(p, opts);\n \n-\t/*\n-\t * Get rid of the idx file as we do not need it anymore.\n-\t * NEEDSWORK: extract this bit from free_pack_by_name() in\n-\t * object-file.c, perhaps?  It shouldn't matter very much as we\n-\t * know we haven't installed this pack (hence we never have\n-\t * read anything from it).\n-\t */\n-\tpackfile_close_index(p);\n-\tfree(p);\n+\tpackfile_free(p);\n }\n \n static void show_pack_info(int stat_only)\ndiff --git a/commit-graph.c b/commit-graph.c\nindex c3b04e8c1f4..03df5fb7c1f 100644\n--- a/commit-graph.c\n+++ b/commit-graph.c\n@@ -1958,14 +1958,12 @@ static int fill_oids_from_packs(struct write_commit_graph_context *ctx,\n \t\t}\n \t\tif (packfile_open_index(p)) {\n \t\t\tret = error(_(\"error opening index for %s\"), pack_indexes->items[i].string);\n-\t\t\tpackfile_close(p);\n-\t\t\tfree(p);\n+\t\t\tpackfile_free(p);\n \t\t\tgoto cleanup;\n \t\t}\n \t\tfor_each_object_in_pack(p, add_packed_commits, ctx,\n \t\t\t\t\tODB_FOR_EACH_OBJECT_PACK_ORDER);\n-\t\tpackfile_close(p);\n-\t\tfree(p);\n+\t\tpackfile_free(p);\n \t}\n \n cleanup:\ndiff --git a/connected.c b/connected.c\nindex fbacff734b6..78f08b41383 100644\n--- a/connected.c\n+++ b/connected.c\n@@ -238,9 +238,7 @@ int check_connected(struct repository *repo,\n \t\terr = error_errno(_(\"failed to close rev-list's stdin\"));\n \n \tsigchain_pop(SIGPIPE);\n-\tif (new_pack) {\n-\t\tpackfile_close(new_pack);\n-\t\tfree(new_pack);\n-\t}\n+\tif (new_pack)\n+\t\tpackfile_free(new_pack);\n \treturn finish_command(&rev_list) || err;\n }\ndiff --git a/http-walker.c b/http-walker.c\nindex de43d9cf673..616a5fd234d 100644\n--- a/http-walker.c\n+++ b/http-walker.c\n@@ -587,10 +587,8 @@ static void cleanup(struct walker *walker)\n \n \t\t\talt_next = alt->next;\n \n-\t\t\tfor (e = alt->packs.head; e; e = e->next) {\n-\t\t\t\tpackfile_close(e->pack);\n-\t\t\t\tfree(e->pack);\n-\t\t\t}\n+\t\t\tfor (e = alt->packs.head; e; e = e->next)\n+\t\t\t\tpackfile_free(e->pack);\n \t\t\tpackfile_list_clear(&alt->packs);\n \n \t\t\tfree(alt->base);\ndiff --git a/http.c b/http.c\nindex e86bebdffb2..d518683e15e 100644\n--- a/http.c\n+++ b/http.c\n@@ -2661,7 +2661,7 @@ static int fetch_and_setup_pack_index(struct packfile_list *packs,\n \tpackfile_close_index(new_pack);\n \tfree(tmp_idx);\n \tif (ret) {\n-\t\tfree(new_pack);\n+\t\tpackfile_free(new_pack);\n \t\treturn -1;\n \t}\n \ndiff --git a/midx-write.c b/midx-write.c\nindex d5be4e40843..3e227bb5e57 100644\n--- a/midx-write.c\n+++ b/midx-write.c\n@@ -182,8 +182,7 @@ static void add_pack_to_midx(const char *full_path, size_t full_path_len,\n \t\tif (packfile_open_index(p)) {\n \t\t\twarning(_(\"failed to open pack-index '%s'\"),\n \t\t\t\tfull_path);\n-\t\t\tpackfile_close(p);\n-\t\t\tfree(p);\n+\t\t\tpackfile_free(p);\n \t\t\treturn;\n \t\t}\n \n@@ -1827,10 +1826,8 @@ static int write_midx_internal(struct write_midx_opts *opts)\n \n cleanup:\n \tfor (size_t i = 0; i < ctx.nr; i++) {\n-\t\tif (ctx.info[i].p) {\n-\t\t\tpackfile_close(ctx.info[i].p);\n-\t\t\tfree(ctx.info[i].p);\n-\t\t}\n+\t\tif (ctx.info[i].p)\n+\t\t\tpackfile_free(ctx.info[i].p);\n \t\tfree(ctx.info[i].pack_name);\n \t}\n \ndiff --git a/odb/source-packed.c b/odb/source-packed.c\nindex 87e2dcdf52e..866b4e890b6 100644\n--- a/odb/source-packed.c\n+++ b/odb/source-packed.c\n@@ -862,7 +862,7 @@ static void odb_source_packed_free(struct odb_source *source)\n \n \todb_source_close(source);\n \tfor (struct packfile_list_entry *e = packed->packs.head; e; e = e->next)\n-\t\tfree(e->pack);\n+\t\tpackfile_free(e->pack);\n \tpackfile_list_clear(&packed->packs);\n \n \tstrmap_clear(&packed->packs_by_path, 0);\ndiff --git a/packfile.c b/packfile.c\nindex 9b03258311f..077b0395a79 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -217,7 +217,7 @@ struct packed_git *parse_pack_index(struct repository *r, unsigned char *sha1,\n \tfree(path);\n \thashcpy(p->hash, sha1, p->repo->hash_algo);\n \tif (check_packed_git_idx(idx_path, p)) {\n-\t\tfree(p);\n+\t\tpackfile_free(p);\n \t\treturn NULL;\n \t}\n \n@@ -732,7 +732,7 @@ struct packed_git *packfile_from_idx(struct repository *r, const char *path,\n \n \txsnprintf(p->pack_name + path_len, alloc - path_len, \".pack\");\n \tif (stat(p->pack_name, &st) || !S_ISREG(st.st_mode)) {\n-\t\tfree(p);\n+\t\tpackfile_free(p);\n \t\treturn NULL;\n \t}\n \n@@ -752,6 +752,12 @@ struct packed_git *packfile_from_idx(struct repository *r, const char *path,\n \treturn p;\n }\n \n+void packfile_free(struct packed_git *p)\n+{\n+\tpackfile_close(p);\n+\tfree(p);\n+}\n+\n void packfile_store_add_pack(struct odb_source_packed *store,\n \t\t\t     struct packed_git *pack)\n {\ndiff --git a/packfile.h b/packfile.h\nindex f817bc69e29..f314a55a904 100644\n--- a/packfile.h\n+++ b/packfile.h\n@@ -223,6 +223,7 @@ struct object_database;\n \n struct packed_git *packfile_from_idx(struct repository *r, const char *path,\n \t\t\t\t     size_t path_len, int local);\n+void packfile_free(struct packed_git *p);\n \n /*\n  * mmap the index file for the specified packfile (if it is not\n\n-- \n2.56.1.831.g720f3f5937.dirty\n\n\n"},{"id":"558926","messageId":"20261120-pks-packfile-global-statistics-v1-5-19beb83b145f@pks.im","threadId":"66896","inReplyTo":"20261120-pks-packfile-global-statistics-v1-0-19beb83b145f@pks.im","subject":"[PATCH 5/6] packfile: maintain process-wide limits globally","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-20T12:02:32Z","receivedAt":"2026-11-20T12:02:32Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Our packfile subsystem has two limits that are supposed to apply\nglobally across all open packfiles:\n\n  - `pack_max_fds` is the maximum number of open file descriptors for\n    packfiles. When we exceed that limit, we close least-recently-used\n    packs until we're below it again.\n\n  - \"core.packedGitLimit\" is the maximum number of packfile data that\n    may be mapped into memory. When we exceed that limit, we close the\n    least-recently-used windows into our packfiles.\n\nThose limits are supposed to be process-wide, so even across multiple\nrepositories. But our current implementation will only consider\npackfiles from a given repository or object database, which may not even\nhave any packfiles open when the logic kicks in. Furthermore, the logic\nassumes that we're using the \"files\" backend and thus unconditionally\ndowncasts the object database's backend to it.\n\nIntroduce a global list of all packfiles. This list is completely\nindependent of the owning repository or object database, and it is\nagnostic of the actual object database backend that's in use. Use it to\nmaintain the above two limits.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n packfile.c | 29 +++++++++++++----------------\n 1 file changed, 13 insertions(+), 16 deletions(-)\n\ndiff --git a/packfile.c b/packfile.c\nindex 077b0395a79..0d5647c74c8 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -35,6 +35,7 @@ char *odb_pack_name(struct repository *r, struct strbuf *buf,\n \treturn buf->buf;\n }\n \n+static struct packfile_list all_packfiles;\n static unsigned int pack_used_ctr;\n static unsigned int pack_open_fds;\n static unsigned int pack_max_fds;\n@@ -195,6 +196,7 @@ static struct packed_git *packfile_new(struct repository *r, int extra)\n \tmemset(p, 0, sizeof(*p));\n \tp->pack_fd = -1;\n \tp->repo = r;\n+\tpackfile_list_append(&all_packfiles, p, 1);\n \treturn p;\n }\n \n@@ -243,17 +245,14 @@ static void scan_windows(struct packed_git *p,\n \t}\n }\n \n-static int unuse_one_window(struct object_database *odb)\n+static int unuse_one_window(void)\n {\n-\tstruct odb_source_files *files = odb_source_files_downcast(odb->source);\n \tstruct packfile_list_entry *e;\n \tstruct packed_git *lru_p = NULL;\n \tstruct pack_window *lru_w = NULL, *lru_l = NULL;\n \n-\tfor (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {\n-\t\tfor (e = dir->packed->packs.head; e; e = e->next)\n-\t\t\tscan_windows(e->pack, &lru_p, &lru_w, &lru_l);\n-\t}\n+\tfor (e = all_packfiles.head; e; e = e->next)\n+\t\tscan_windows(e->pack, &lru_p, &lru_w, &lru_l);\n \n \tif (lru_p) {\n \t\tmunmap(lru_w->base, lru_w->len);\n@@ -407,20 +406,17 @@ static void find_lru_pack(struct packed_git *p, struct packed_git **lru_p, struc\n \t*accept_windows_inuse = has_windows_inuse;\n }\n \n-static int close_one_pack(struct repository *r)\n+static int close_one_pack(void)\n {\n-\tstruct odb_source_files *files = odb_source_files_downcast(r->objects->source);\n \tstruct packfile_list_entry *e;\n \tstruct packed_git *lru_p = NULL;\n \tstruct pack_window *mru_w = NULL;\n \tint accept_windows_inuse = 1;\n \n-\tfor (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {\n-\t\tfor (e = dir->packed->packs.head; e; e = e->next) {\n-\t\t\tif (e->pack->pack_fd == -1)\n-\t\t\t\tcontinue;\n-\t\t\tfind_lru_pack(e->pack, &lru_p, &mru_w, &accept_windows_inuse);\n-\t\t}\n+\tfor (e = all_packfiles.head; e; e = e->next) {\n+\t\tif (e->pack->pack_fd == -1)\n+\t\t\tcontinue;\n+\t\tfind_lru_pack(e->pack, &lru_p, &mru_w, &accept_windows_inuse);\n \t}\n \n \tif (lru_p)\n@@ -505,7 +501,7 @@ static int packfile_open_pack(struct packed_git *p)\n \t\t\tpack_max_fds = 1;\n \t}\n \n-\twhile (pack_max_fds <= pack_open_fds && close_one_pack(p->repo))\n+\twhile (pack_max_fds <= pack_open_fds && close_one_pack())\n \t\t; /* nothing */\n \n \tp->pack_fd = git_open(p->pack_name);\n@@ -649,7 +645,7 @@ unsigned char *use_pack(struct packed_git *p,\n \t\t\tpack_mapped += win->len;\n \n \t\t\twhile (settings->packed_git_limit < pack_mapped &&\n-\t\t\t       unuse_one_window(p->repo->objects))\n+\t\t\t       unuse_one_window())\n \t\t\t\t; /* nothing */\n \t\t\twin->base = xmmap_gently(NULL, win->len,\n \t\t\t\tPROT_READ, MAP_PRIVATE,\n@@ -754,6 +750,7 @@ struct packed_git *packfile_from_idx(struct repository *r, const char *path,\n \n void packfile_free(struct packed_git *p)\n {\n+\tpackfile_list_remove(&all_packfiles, p);\n \tpackfile_close(p);\n \tfree(p);\n }\n\n-- \n2.56.1.831.g720f3f5937.dirty\n\n\n"},{"id":"558927","messageId":"20261120-pks-packfile-global-statistics-v1-6-19beb83b145f@pks.im","threadId":"66896","inReplyTo":"20261120-pks-packfile-global-statistics-v1-0-19beb83b145f@pks.im","subject":"[PATCH 6/6] packfile: skip closing packs marked as do-not-close","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-20T12:02:33Z","receivedAt":"2026-11-20T12:02:33Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"When closing a packfile to maintain the maximum number of open packs we\ncompletely ignore the `do_not_close` bit. That bit is only set by\ngit-fast-import(1), and as that command allocates its packs manually it\nwouldn't add those packs to the list of all packfiles, and consequently\nwe won't ever try to close such a packfile via `close_one_pack()`. But\nthat feels like coincidence rather than proper design, and the result is\nrather fragile.\n\nSkip over packfiles marked as do-not-close to reduce the fragility. This\nshould not result in a user-visible change in behaviour.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n packfile.c | 3 ++-\n 1 file changed, 2 insertions(+), 1 deletion(-)\n\ndiff --git a/packfile.c b/packfile.c\nindex 0d5647c74c8..86ef3a79d9c 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -414,7 +414,8 @@ static int close_one_pack(void)\n \tint accept_windows_inuse = 1;\n \n \tfor (e = all_packfiles.head; e; e = e->next) {\n-\t\tif (e->pack->pack_fd == -1)\n+\t\tif (e->pack->pack_fd == -1 ||\n+\t\t    e->pack->do_not_close)\n \t\t\tcontinue;\n \t\tfind_lru_pack(e->pack, &lru_p, &mru_w, &accept_windows_inuse);\n \t}\n\n-- \n2.56.1.831.g720f3f5937.dirty\n\n\n"}]}