{"thread":{"id":"66019","subject":"[RFC PATCH 0/7] repack: add --drop-filtered to reclaim space in partial clones","startedAt":"2026-07-16T13:29:09Z","lastAt":"2026-09-04T10:52:12Z","messageCount":76,"participants":["Siddharth Shrimali","Junio C Hamano","Christian Couder","Siddharth Asthana","Samuel Bronson"],"isPatch":true,"patchVersion":1,"patchTotal":7},"messages":[{"id":"548404","messageId":"20260716132848.95982-1-r.siddharth.shrimali@gmail.com","threadId":"66019","inReplyTo":null,"subject":"[RFC PATCH 0/7] repack: add --drop-filtered to reclaim space in partial clones","fromName":"Siddharth Shrimali","fromEmail":"r.siddharth.shrimali@gmail.com","sentAt":"2026-07-16T13:28:41Z","receivedAt":"2026-07-16T13:29:09Z","isPatch":true,"body":"This is an RFC series seeking feedback on the design and approach.\nSeveral pieces are still missing (noted below) and the commit\norganization needs cleanup.\n\nPartial clones let you work with large repositories without downloading\nevery blob up front and the missing blobs are lazily fetched from the promisor\nremote on demand. Over time, though, these lazily-fetched blobs\naccumulate locally and there is currently no safe, built-in way to\nreclaim that disk space instead of re-cloning.\n\nThis series adds a \"git repack --drop-filtered --filter=\" command\nthat removes large, locally-held promisor blobs that are recoverable\nfrom the promisor remote. The dropped blobs become absent locally but\nremain lazily re-fetchable, making the partial-clone still reversible.\n\nHow it works:\n * Enumerate promisor objects directly (ODB_FOR_EACH_OBJECT_PROMISOR_ONLY)\n and select the blobs exceeding the filter threshold. Because every\n enumerated object is a promisor object, it is guaranteed recoverable and\n locally-created objects are never candidates.\n\n * Rebuild the promisor pack without the selected blobs, reusing the\n existing repack machinery, so the drop is crash-safe.\n\n * Record each dropped object in a drop log\n ($GIT_DIR/objects/info/promisor-dropped) so a later change can\n explain a failed lazy fetch (when it was dropped, which filter\n matched, which remotes) instead of a bare \"could not fetch\" error.\n\n * --dry-run lists the candidates and changes nothing.\n\nPlanned follow-ups:\n * Safety guards: refuse to run while a merge/rebase/cherry-pick is in\n progress, and refuse to drop blobs referenced by the current index.\n\n * Authoritative remote verification: the drop log currently lists all\n configured promisor remotes rather than the exact remote each object\n is recoverable from, because there is no client-side way to query a\n remote for object availability yet. A \"remote-object-info\" command\n is being added to the \"git cat-file --batch\" protocol for this. Once\n available, the exact remote can be recorded.\n\nKnown issues to address in v2:\n * There is churn between \"enumerate promisor blobs\" and \"actually drop\n filtered promisor blobs\". The former introduces\n enumerate_promisor_blobs() with an interim signature that the latter\n rewrites. These will be reorganized so the function is introduced\n in its final form.\n\n * The tests are in a standalone commit. They will instead be\n distributed into the commits that introduce the behavior they test.\n\nSiddharth Shrimali (7):\n builtin/repack.c: add --drop-filtered and --dry-run options\n list-objects-filter: add list_objects_filter__filter_oidset()\n repack-promisor: allow excluding objects from the rebuilt promisor\n pack\n builtin/repack: enumerate promisor blobs for --drop-filtered\n t7706: test --drop-filtered enumeration and validation\n builtin/repack: actually drop filtered promisor blobs\n repack-promisor: record dropped objects in a drop log\n\n builtin/repack.c | 76 ++++++++++++++++-\n list-objects-filter.c | 45 ++++++++++\n list-objects-filter.h | 16 ++++\n repack-filtered.c | 81 ++++++++++++++++++\n repack-promisor.c | 106 ++++++++++++++++++++++-\n repack.h | 12 ++-\n t/meson.build | 1 +\n t/t7706-repack-drop-filtered.sh | 145 ++++++++++++++++++++++++++++++++\n 8 files changed, 478 insertions(+), 4 deletions(-)\n create mode 100755 t/t7706-repack-drop-filtered.sh\n\n-- \n2.54.0\n\n"},{"id":"548405","messageId":"20260716132848.95982-2-r.siddharth.shrimali@gmail.com","threadId":"66019","inReplyTo":"20260716132848.95982-1-r.siddharth.shrimali@gmail.com","subject":"[RFC PATCH 1/7] builtin/repack.c: add --drop-filtered and --dry-run options","fromName":"Siddharth Shrimali","fromEmail":"r.siddharth.shrimali@gmail.com","sentAt":"2026-07-16T13:28:42Z","receivedAt":"2026-07-16T13:29:15Z","isPatch":true,"body":"Add two new command-line options to 'git-repack':\n\n --drop-filtered: intended to eventually delete objects that match\n the filter specification. Requires --filter and -a,\n and is incompatible with --filter-to.\n --dry-run: show which objects would be dropped without making any\n changes. Only meaningful with --drop-filtered.\n\n--drop-filtered also requires a promisor remote to be configured,\nsince dropping objects without a remote to fetch them back from would\nbe permanent data loss.\n\n--drop-filtered is incompatible with bitmap writing: filtering breaks\nthe \"all objects in one pack\" closure that bitmaps require. An explicit\n-b is rejected with a clear error and a default-on bitmap configuration is\nsilently disabled for the duration of the command.\n\nThese options currently only perform validation. The actual enumeration\nand deletion will be added in follow-up commits.\n\nMentored-by: Christian Couder \nMentored-by: Siddharth Asthana \nSigned-off-by: Siddharth Shrimali \n---\n builtin/repack.c | 44 ++++++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 44 insertions(+)\n\ndiff --git a/builtin/repack.c b/builtin/repack.c\nindex db504d673f..f4db0fc535 100644\n--- a/builtin/repack.c\n+++ b/builtin/repack.c\n@@ -14,6 +14,7 @@\n #include \"promisor-remote.h\"\n #include \"repack.h\"\n #include \"shallow.h\"\n+#include \"list-objects-filter-options.h\"\n \n #define ALL_INTO_ONE 1\n #define LOOSEN_UNREACHABLE 2\n@@ -28,6 +29,8 @@ static int use_delta_islands;\n static int run_update_server_info = 1;\n static char *packdir, *packtmp_name, *packtmp;\n static int midx_must_contain_cruft = 1;\n+static int drop_filtered;\n+static int dry_run;\n \n static const char *const git_repack_usage[] = {\n \tN_(\"git repack [-a] [-A] [-d] [-f] [-F] [-l] [-n] [-q] [-b] [-m]\\n\"\n@@ -231,6 +234,10 @@ int cmd_repack(int argc,\n \t\t\t N_(\"pack prefix to store a pack containing pruned objects\")),\n \t\tOPT_STRING(0, \"filter-to\", &filter_to, N_(\"dir\"),\n \t\t\t N_(\"pack prefix to store a pack containing filtered out objects\")),\n+\t\tOPT_BOOL(0, \"drop-filtered\", &drop_filtered,\n+\t\t\t\tN_(\"delete filtered out objects (requires --filter)\")),\n+\t\tOPT_BOOL(0, \"dry-run\", &dry_run,\n+\t\t\t\tN_(\"only show which objects would be dropped\")),\n \t\tOPT_END()\n \t};\n \n@@ -252,6 +259,43 @@ int cmd_repack(int argc,\n \tpo_args.depth = xstrdup_or_null(opt_depth);\n \tpo_args.threads = xstrdup_or_null(opt_threads);\n \n+\tdie_for_incompatible_opt2(drop_filtered, \"--drop-filtered\",\n+\t\t!!filter_to, \"--filter-to\");\n+\n+\tdie_for_incompatible_opt2(drop_filtered, \"--drop-filtered\",\n+\t\twrite_bitmaps > 0, \"--write-bitmap-index\");\n+\n+\tif (dry_run && !drop_filtered)\n+\t\tdie(_(\"--dry-run only takes effect with --drop-filtered\"));\n+\n+\tif (drop_filtered) {\n+\t\tif (!dry_run)\n+\t\t\tdie(_(\"--drop-filtered doesn't work without --dry-run yet\"));\n+\n+\t\tif (!po_args.filter_options.choice)\n+\t\t\tdie(_(\"--drop-filtered requires --filter\"));\n+\n+\t\tif (!(pack_everything & ALL_INTO_ONE))\n+\t\t\tdie(_(\"--drop-filtered requires -a\"));\n+\n+\t\t/*\n+\t\t * Only blob:limit= is supported for now. Reject other\n+\t\t * filter choices early, before walking the object database.\n+\t\t */\n+\t\tif (po_args.filter_options.choice != LOFC_BLOB_LIMIT)\n+\t\t\tdie(_(\"--drop-filtered only supports --filter=blob:limit= for now\"));\n+\n+\t\t/*\n+\t\t * Without a promisor remote there is nowhere to re-fetch the\n+\t\t * dropped objects from, so dropping them would be permanent\n+\t\t * data loss.\n+\t\t */\n+\t\tif (!repo_has_promisor_remote(repo))\n+\t\t\tdie(_(\"--drop-filtered requires a promisor remote\"));\n+\n+\t\twrite_bitmaps = 0;\n+\t}\n+\n \tif (delete_redundant && repo->repository_format_precious_objects)\n \t\tdie(_(\"cannot delete packs in a precious-objects repo\"));\n \n-- \n2.54.0\n\n"},{"id":"548406","messageId":"20260716132848.95982-3-r.siddharth.shrimali@gmail.com","threadId":"66019","inReplyTo":"20260716132848.95982-1-r.siddharth.shrimali@gmail.com","subject":"[RFC PATCH 2/7] list-objects-filter: add list_objects_filter__filter_oidset()","fromName":"Siddharth Shrimali","fromEmail":"r.siddharth.shrimali@gmail.com","sentAt":"2026-07-16T13:28:43Z","receivedAt":"2026-07-16T13:29:24Z","isPatch":true,"body":"The existing filter entry point, list_objects_filter__filter_object(),\nis built around the object-walk path: it expects traversal context and\nprovisional omit sets, and is meant to be called as objects are\nvisited during a walk. A caller that already has a set of OIDs in hand\nand only wants to know which ones a filter would select has no usable\nentry point into the filter API.\n\n--drop-filtered is exactly such a caller: it collects promisor blobs\ninto an oidset and needs to know which of them exceed the filter\nthreshold, without performing an object walk.\n\nAdd a helper, list_objects_filter__filter_oidset(), that takes a set\nof OIDs and populates an \"omitted\" set with those that would be\nfiltered out by the given filter options. Only blob:limit=N filters\nare supported for now.\n\nThis helper does not actually reuse the existing filter machinery.\nIt reimplements the blob:limit size check directly. That machinery\nis tied to the object-walk path and cannot easily be driven\nfrom a plain oidset. A NEEDSWORK comment marks this so the helper can\nlater be refactored to reuse the real filter logic instead of\nduplicating it.\n\nOBJECT_INFO_SKIP_FETCH_OBJECT is passed when reading object info so\nthe helper never triggers a lazy fetch.\n\nMentored-by: Christian Couder \nMentored-by: Siddharth Asthana \nSigned-off-by: Siddharth Shrimali \n---\n list-objects-filter.c | 45 +++++++++++++++++++++++++++++++++++++++++++\n list-objects-filter.h | 16 +++++++++++++++\n 2 files changed, 61 insertions(+)\n\ndiff --git a/list-objects-filter.c b/list-objects-filter.c\nindex c912ff3079..6a2e9d5b24 100644\n--- a/list-objects-filter.c\n+++ b/list-objects-filter.c\n@@ -828,3 +828,48 @@ void list_objects_filter__free(struct filter *filter)\n \tfilter->free_fn(filter->filter_data);\n \tfree(filter);\n }\n+\n+/*\n+ * NEEDSWORK: this reimplements the blob:limit size check rather than\n+ * reusing the existing filter machinery in\n+ * list_objects_filter__filter_object(). That machinery is currently\n+ * tied to the object-walk path and cannot easily be driven from a\n+ * plain oidset. It would be nice to refactor the filter code so this\n+ * helper can reuse it instead of duplicating the size check.\n+ */\n+int list_objects_filter__filter_oidset(struct repository *r,\n+\tstruct list_objects_filter_options *opts,\n+\tconst struct oidset *in,\n+\tstruct oidset *omitted)\n+{\n+\tstruct oidset_iter iter;\n+\tconst struct object_id *oid;\n+\n+\tif (opts->choice != LOFC_BLOB_LIMIT)\n+\t\treturn error(_(\"filter_oidset: only blob:limit filters are supported\"));\n+\n+\toidset_iter_init(in, &iter);\n+\twhile ((oid = oidset_iter_next(&iter))) {\n+\t\tstruct object_info info = OBJECT_INFO_INIT;\n+\t\tenum object_type type;\n+\t\tunsigned long size;\n+\n+\t\tinfo.typep = &type;\n+\t\tinfo.sizep = &size;\n+\n+\t\t/*\n+\t\t * Use OBJECT_INFO_SKIP_FETCH_OBJECT to avoid triggering\n+\t\t * a lazy fetch while inspecting candidates for removal.\n+\t\t */\n+\t\tif (odb_read_object_info_extended(r->objects, oid, &info,\n+\t\t\t\tOBJECT_INFO_SKIP_FETCH_OBJECT) < 0)\n+\t\t\tcontinue;\n+\n+\t\tif (type != OBJ_BLOB)\n+\t\t\tcontinue;\n+\n+\t\tif (size >= opts->blob_limit_value)\n+\t\t\toidset_insert(omitted, oid);\n+\t}\n+\treturn 0;\n+}\ndiff --git a/list-objects-filter.h b/list-objects-filter.h\nindex 9e98814111..56a2d87aa0 100644\n--- a/list-objects-filter.h\n+++ b/list-objects-filter.h\n@@ -94,4 +94,20 @@ enum list_objects_filter_result list_objects_filter__filter_object(\n */\n void list_objects_filter__free(struct filter *filter);\n \n+/*\n+ * Given a set of OIDs in 'in', populate 'omitted' with those that\n+ * would be filtered by 'opts'. Currently only blob:limit=N is\n+ * supported. Objects that cannot be read are silently skipped.\n+ *\n+ * NEEDSWORK: this reimplements the blob:limit size check rather than\n+ * reusing the existing filter machinery. See the matching comment in\n+ * list-objects-filter.c.\n+ *\n+ * Return 0 on success, -1 if the filter is not supported.\n+ */\n+int list_objects_filter__filter_oidset(struct repository *r,\n+\tstruct list_objects_filter_options *opts,\n+\tconst struct oidset *in,\n+\tstruct oidset *omitted);\n+\n #endif /* LIST_OBJECTS_FILTER_H */\n-- \n2.54.0\n\n"},{"id":"548407","messageId":"20260716132848.95982-4-r.siddharth.shrimali@gmail.com","threadId":"66019","inReplyTo":"20260716132848.95982-1-r.siddharth.shrimali@gmail.com","subject":"[RFC PATCH 3/7] repack-promisor: allow excluding objects from the rebuilt promisor pack","fromName":"Siddharth Shrimali","fromEmail":"r.siddharth.shrimali@gmail.com","sentAt":"2026-07-16T13:28:44Z","receivedAt":"2026-07-16T13:29:30Z","isPatch":true,"body":"Add a to_drop oidset parameter to repack_promisor_objects(). When it is\nnon-NULL, write_oid() omits those objects from the rebuilt promisor\npack. This is the mechanism --drop-filtered will use to remove promisor\nblobs, i.e. rebuild the promisor pack without them.\n\nAll existing callers pass NULL, so behavior is unchanged.\n\nMentored-by: Christian Couder \nMentored-by: Siddharth Asthana \nSigned-off-by: Siddharth Shrimali \n---\n builtin/repack.c | 2 +-\n repack-promisor.c | 15 ++++++++++++++-\n repack.h | 4 +++-\n 3 files changed, 18 insertions(+), 3 deletions(-)\n\ndiff --git a/builtin/repack.c b/builtin/repack.c\nindex f4db0fc535..433b2c8205 100644\n--- a/builtin/repack.c\n+++ b/builtin/repack.c\n@@ -406,7 +406,7 @@ int cmd_repack(int argc,\n \t\tstrvec_push(&cmd.args, \"--delta-islands\");\n \n \tif (pack_everything & ALL_INTO_ONE) {\n-\t\trepack_promisor_objects(repo, &po_args, &names, packtmp);\n+\t\trepack_promisor_objects(repo, &po_args, &names, packtmp, NULL);\n \n \t\tif (existing_packs_has_non_kept(&existing) &&\n \t\t delete_redundant &&\ndiff --git a/repack-promisor.c b/repack-promisor.c\nindex 90318ce150..fabfdc168a 100644\n--- a/repack-promisor.c\n+++ b/repack-promisor.c\n@@ -6,10 +6,12 @@\n #include \"path.h\"\n #include \"repository.h\"\n #include \"run-command.h\"\n+#include \"oidset.h\"\n \n struct write_oid_context {\n \tstruct child_process *cmd;\n \tconst struct git_hash_algo *algop;\n+\tconst struct oidset *to_drop;\n };\n \n /*\n@@ -23,6 +25,15 @@ static int write_oid(const struct object_id *oid,\n \tstruct write_oid_context *ctx = data;\n \tstruct child_process *cmd = ctx->cmd;\n \n+\t/*\n+\t * Objects in to_drop are being removed from the repository, so\n+\t * omit them from the rebuilt promisor pack. Each such object is a\n+\t * promisor object and therefore remains recoverable from the\n+\t * promisor remote.\n+\t */\n+\tif (ctx->to_drop && oidset_contains(ctx->to_drop, oid))\n+\t\treturn 0;\n+\n \tif (cmd->in == -1) {\n \t\tif (start_command(cmd))\n \t\t\tdie(_(\"could not start pack-objects to repack promisor objects\"));\n@@ -81,7 +92,8 @@ static void finish_repacking_promisor_objects(struct repository *repo,\n \n void repack_promisor_objects(struct repository *repo,\n \t\t\t const struct pack_objects_args *args,\n-\t\t\t struct string_list *names, const char *packtmp)\n+\t\t\t struct string_list *names, const char *packtmp,\n+\t\t\t const struct oidset *to_drop)\n {\n \tstruct write_oid_context ctx;\n \tstruct child_process cmd = CHILD_PROCESS_INIT;\n@@ -98,6 +110,7 @@ void repack_promisor_objects(struct repository *repo,\n \t */\n \tctx.cmd = &cmd;\n \tctx.algop = repo->hash_algo;\n+\tctx.to_drop = to_drop;\n \todb_for_each_object(repo->objects, NULL, write_oid, &ctx,\n \t\t\t ODB_FOR_EACH_OBJECT_PROMISOR_ONLY);\n \ndiff --git a/repack.h b/repack.h\nindex f9fbc895f0..a5a3f7c6ba 100644\n--- a/repack.h\n+++ b/repack.h\n@@ -3,6 +3,7 @@\n \n #include \"list-objects-filter-options.h\"\n #include \"string-list.h\"\n+#include \"oidset.h\"\n \n struct pack_objects_args {\n \tchar *window;\n@@ -100,7 +101,8 @@ void generated_pack_install(struct generated_pack *pack, const char *name,\n \n void repack_promisor_objects(struct repository *repo,\n \t\t\t const struct pack_objects_args *args,\n-\t\t\t struct string_list *names, const char *packtmp);\n+\t\t\t struct string_list *names, const char *packtmp,\n+\t\t\t const struct oidset *to_drop);\n \n struct pack_geometry {\n \tstruct packed_git **pack;\n-- \n2.54.0\n\n"},{"id":"548408","messageId":"20260716132848.95982-5-r.siddharth.shrimali@gmail.com","threadId":"66019","inReplyTo":"20260716132848.95982-1-r.siddharth.shrimali@gmail.com","subject":"[RFC PATCH 4/7] builtin/repack: enumerate promisor blobs for --drop-filtered","fromName":"Siddharth Shrimali","fromEmail":"r.siddharth.shrimali@gmail.com","sentAt":"2026-07-16T13:28:45Z","receivedAt":"2026-07-16T13:29:38Z","isPatch":true,"body":"Add enumeration logic for --drop-filtered. In --dry-run mode, print\nthe OIDs of locally-held promisor blobs that exceed the filter\nthreshold, as candidates for removal.\n\nReading from write_filtered_pack() cannot work for partial clones.\ngit repack routes promisor objects through a separate path:\nrepack_promisor_objects() repacks them first, and the main\npack-objects run uses --exclude-promisor-objects. By the time\nwrite_filtered_pack() runs, the promisor blobs are already consumed by\nthe main pack. The filtered pack is always empty on a partial clone.\n\nInstead, walk promisor objects directly via odb_for_each_object() with\nODB_FOR_EACH_OBJECT_PROMISOR_ONLY, collecting all promisor blobs into\nan oidset. The blobs exceeding the filter threshold are then selected\nusing list_objects_filter__filter_oidset().\n\nEvery object enumerated this way is a promisor object by construction,\nso it is guaranteed to be recoverable from the promisor remote and is\nsafe to drop. No separate is_promisor_object() check is needed.\n\nOBJECT_INFO_SKIP_FETCH_OBJECT is passed to every object info query so\nenumeration never triggers a lazy fetch.\n\nDeletion of the enumerated objects, together with the required\npromisor-remote verification, will be added separately.\n\nMentored-by: Christian Couder \nMentored-by: Siddharth Asthana \nSigned-off-by: Siddharth Shrimali \n---\n builtin/repack.c | 41 ++++++++++++++-------\n repack-filtered.c | 92 +++++++++++++++++++++++++++++++++++++++++++++++\n repack.h | 4 +++\n 3 files changed, 124 insertions(+), 13 deletions(-)\n\ndiff --git a/builtin/repack.c b/builtin/repack.c\nindex 433b2c8205..c2b07477d2 100644\n--- a/builtin/repack.c\n+++ b/builtin/repack.c\n@@ -590,19 +590,34 @@ int cmd_repack(int argc,\n \t}\n \n \tif (po_args.filter_options.choice) {\n-\t\tstruct write_pack_opts opts = {\n-\t\t\t.po_args = &po_args,\n-\t\t\t.destination = filter_to,\n-\t\t\t.packdir = packdir,\n-\t\t\t.packtmp = packtmp,\n-\t\t};\n-\n-\t\tif (!opts.destination)\n-\t\t\topts.destination = packtmp;\n-\n-\t\tret = write_filtered_pack(&opts, &existing, &names);\n-\t\tif (ret)\n-\t\t\tgoto cleanup;\n+\t\tif (drop_filtered) {\n+\t\t\t/*\n+\t\t\t * Enumerate promisor objects directly rather than\n+\t\t\t * going through write_filtered_pack(). The filter\n+\t\t\t * machinery cannot see promisor objects because\n+\t\t\t * repack_promisor_objects() handles them separately\n+\t\t\t * before the filter runs.\n+\t\t\t */\n+\t\t\tret = enumerate_promisor_blobs(repo,\n+\t\t\t\t\t&po_args.filter_options,\n+\t\t\t\t\tdry_run);\n+\t\t\tif (ret)\n+\t\t\t\tgoto cleanup;\n+\t\t} else {\n+\t\t\tstruct write_pack_opts opts = {\n+\t\t\t\t.po_args = &po_args,\n+\t\t\t\t.destination = filter_to,\n+\t\t\t\t.packdir = packdir,\n+\t\t\t\t.packtmp = packtmp,\n+\t\t\t};\n+\n+\t\t\tif (!opts.destination)\n+\t\t\t\topts.destination = packtmp;\n+\n+\t\t\tret = write_filtered_pack(&opts, &existing, &names);\n+\t\t\tif (ret)\n+\t\t\t\tgoto cleanup;\n+\t\t}\n \t}\n \n \tstring_list_sort(&names);\ndiff --git a/repack-filtered.c b/repack-filtered.c\nindex edcf7667c5..f5a1dae5b1 100644\n--- a/repack-filtered.c\n+++ b/repack-filtered.c\n@@ -3,6 +3,12 @@\n #include \"repository.h\"\n #include \"run-command.h\"\n #include \"string-list.h\"\n+#include \"hex.h\"\n+#include \"packfile.h\"\n+#include \"list-objects-filter-options.h\"\n+#include \"list-objects-filter.h\"\n+#include \"odb.h\"\n+#include \"promisor-remote.h\"\n \n int write_filtered_pack(const struct write_pack_opts *opts,\n \t\t\tstruct existing_packs *existing,\n@@ -49,3 +55,89 @@ int write_filtered_pack(const struct write_pack_opts *opts,\n \treturn finish_pack_objects_cmd(existing->repo->hash_algo, opts, &cmd,\n \t\t\t\t names);\n }\n+\n+struct collect_cb_data {\n+\tstruct repository *repo;\n+\tstruct oidset *set;\n+};\n+\n+static int collect_promisor_blob(const struct object_id *oid,\n+\t\tstruct object_info *oi UNUSED,\n+\t\tvoid *cb_data)\n+{\n+\tstruct collect_cb_data *data = cb_data;\n+\tstruct object_info info = OBJECT_INFO_INIT;\n+\tenum object_type type;\n+\n+\tinfo.typep = &type;\n+\n+\t/*\n+\t * Use OBJECT_INFO_SKIP_FETCH_OBJECT to avoid triggering a\n+\t * lazy fetch while collecting promisor blobs.\n+\t */\n+\tif (odb_read_object_info_extended(data->repo->objects, oid, &info,\n+\t\t\tOBJECT_INFO_SKIP_FETCH_OBJECT) < 0)\n+\t\treturn 0;\n+\n+\tif (type == OBJ_BLOB)\n+\t\toidset_insert(data->set, oid);\n+\n+\treturn 0;\n+}\n+\n+int enumerate_promisor_blobs(struct repository *repo,\n+\t\t\tconst struct list_objects_filter_options *filter,\n+\t\t\tint dry_run)\n+{\n+\tstruct oidset all_promisor_blobs = OIDSET_INIT;\n+\tstruct oidset to_drop = OIDSET_INIT;\n+\tstruct collect_cb_data cb = {\n+\t\t.repo = repo,\n+\t\t.set = &all_promisor_blobs\n+\t};\n+\tstruct oidset_iter iter;\n+\tconst struct object_id *oid;\n+\tint ret = 0;\n+\n+\t/*\n+\t * The caller (cmd_repack) is responsible for validating that a\n+\t * blob:limit filter and a promisor remote are present before\n+\t * calling this function.\n+\t *\n+\t * Walk only promisor objects. Every object visited here is\n+\t * guaranteed to be recoverable from the promisor remote, so\n+\t * it is safe to drop.\n+\t *\n+\t * We do not use write_filtered_pack() here because git repack\n+\t * routes promisor objects through repack_promisor_objects()\n+\t * before the filter machinery runs, so the filtered pack never\n+\t * contains promisor blobs. Direct enumeration via\n+\t * ODB_FOR_EACH_OBJECT_PROMISOR_ONLY is the correct approach.\n+\t */\n+\tret = odb_for_each_object(repo->objects, NULL,\n+\t\t\tcollect_promisor_blob, &cb,\n+\t\t\tODB_FOR_EACH_OBJECT_PROMISOR_ONLY);\n+\tif (ret)\n+\t\tgoto cleanup;\n+\n+\t/*\n+\t * Apply the filter to find which blobs exceed the threshold.\n+\t */\n+\tret = list_objects_filter__filter_oidset(repo,\n+\t\t(struct list_objects_filter_options *)filter,\n+\t\t&all_promisor_blobs,\n+\t\t&to_drop);\n+\tif (ret)\n+\t\tgoto cleanup;\n+\n+\tif (dry_run) {\n+\t\toidset_iter_init(&to_drop, &iter);\n+\t\twhile ((oid = oidset_iter_next(&iter)))\n+\t\t\tprintf(\"%s\\n\", oid_to_hex(oid));\n+\t}\n+\n+cleanup:\n+\toidset_clear(&all_promisor_blobs);\n+\toidset_clear(&to_drop);\n+\treturn ret;\n+}\ndiff --git a/repack.h b/repack.h\nindex a5a3f7c6ba..d08e25b852 100644\n--- a/repack.h\n+++ b/repack.h\n@@ -167,6 +167,10 @@ int write_filtered_pack(const struct write_pack_opts *opts,\n \t\t\tstruct existing_packs *existing,\n \t\t\tstruct string_list *names);\n \n+int enumerate_promisor_blobs(struct repository *repo,\n+\t\t\t const struct list_objects_filter_options *filter,\n+\t\t\t int dry_run);\n+\n int write_cruft_pack(const struct write_pack_opts *opts,\n \t\t const char *cruft_expiration,\n \t\t unsigned long combine_cruft_below_size,\n-- \n2.54.0\n\n"},{"id":"548409","messageId":"20260716132848.95982-6-r.siddharth.shrimali@gmail.com","threadId":"66019","inReplyTo":"20260716132848.95982-1-r.siddharth.shrimali@gmail.com","subject":"[RFC PATCH 5/7] t7706: test --drop-filtered enumeration and validation","fromName":"Siddharth Shrimali","fromEmail":"r.siddharth.shrimali@gmail.com","sentAt":"2026-07-16T13:28:46Z","receivedAt":"2026-07-16T13:29:43Z","isPatch":true,"body":"Add tests for the --drop-filtered option.\n\n * Validation: --drop-filtered requires --filter and -a, is\n incompatible with --filter-to and --write-bitmap-index, --dry-run\n only takes effect with --drop-filtered, and --drop-filtered\n requires a promisor remote.\n\n * Enumeration: in a repository with a promisor remote, --dry-run\n lists promisor blobs above the filter threshold and excludes\n smaller ones. Promisor blobs are created with a synthetic promisor\n pack, following the helper pattern used in t0410.\n\n * Safety: a locally created large blob, which is not a promisor\n object and therefore not recoverable from the remote, is never\n listed as a drop candidate.\n\n * Non-destructiveness: --dry-run leaves the filtered objects intact\n in the repository.\n\nMentored-by: Christian Couder \nMentored-by: Siddharth Asthana \nSigned-off-by: Siddharth Shrimali \n---\n t/meson.build | 1 +\n t/t7706-repack-drop-filtered.sh | 139 ++++++++++++++++++++++++++++++++\n 2 files changed, 140 insertions(+)\n create mode 100755 t/t7706-repack-drop-filtered.sh\n\ndiff --git a/t/meson.build b/t/meson.build\nindex 8ae6ab6c5f..37f272d7f4 100644\n--- a/t/meson.build\n+++ b/t/meson.build\n@@ -962,6 +962,7 @@ integration_tests = [\n 't7703-repack-geometric.sh',\n 't7704-repack-cruft.sh',\n 't7705-repack-incremental-midx.sh',\n+ 't7706-repack-drop-filtered.sh',\n 't7800-difftool.sh',\n 't7810-grep.sh',\n 't7811-grep-open.sh',\ndiff --git a/t/t7706-repack-drop-filtered.sh b/t/t7706-repack-drop-filtered.sh\nnew file mode 100755\nindex 0000000000..b558807847\n--- /dev/null\n+++ b/t/t7706-repack-drop-filtered.sh\n@@ -0,0 +1,139 @@\n+#!/bin/sh\n+\n+test_description='git repack --drop-filtered enumerates\n+filtered promisor blobs'\n+\n+. ./test-lib.sh\n+\n+delete_object () {\n+\tlocal repo=\"$1\" &&\n+\tlocal obj=\"$2\" &&\n+\tlocal path=\"$repo/.git/objects/$(test_oid_to_path \"$obj\")\" &&\n+\trm \"$path\"\n+}\n+\n+# pack the objects into a promisor pack inside \"repo\",\n+# it is a pack accompanied by an empty \".promisor\" marker file. objects\n+# in such a pack are treated as recoverable from the promisor remote.\n+pack_as_from_promisor () {\n+\tHASH=$(git -C repo pack-objects .git/objects/pack/pack) &&\n+\t>repo/.git/objects/pack/pack-$HASH.promisor &&\n+\techo $HASH\n+}\n+\n+# write a blob of $1 bytes into \"repo\", record it as coming from the\n+# promisor remote (promisor pack), and remove the loose copy so the\n+# object is only present in the promisor pack.\n+promisor_blob () {\n+\ttest-tool genrandom \"$1\" \"$2\" >blob_content &&\n+\tOID=$(git -C repo hash-object -w --stdin /dev/null &&\n+\tdelete_object repo \"$OID\" &&\n+\techo \"$OID\"\n+}\n+\n+# checks for options validations before any promisor walk\n+test_expect_success 'setup plain repo for validation' '\n+\tgit init plain &&\n+\ttest_commit -C plain initial &&\n+\tgit clone --bare plain plain.git &&\n+\tgit -C plain.git repack -a -d\n+'\n+\n+test_expect_success '--drop-filtered requires --filter' '\n+\ttest_must_fail git -C plain.git repack --drop-filtered --dry-run -a 2>err &&\n+\ttest_grep \"drop-filtered requires --filter\" err\n+'\n+\n+test_expect_success '--drop-filtered cannot be used with --filter-to' '\n+\ttest_must_fail git -C plain.git repack --drop-filtered \\\n+\t\t--filter=blob:limit=1k --filter-to=./filter-out 2>err &&\n+\ttest_grep \"options .--drop-filtered. and .--filter-to. cannot be used together\" err\n+'\n+\n+test_expect_success '--dry-run only takes effect with --drop-filtered' '\n+\ttest_must_fail git -C plain.git repack --dry-run 2>err &&\n+\ttest_grep \"dry-run only takes effect with --drop-filtered\" err\n+'\n+\n+test_expect_success '--drop-filtered without --dry-run is rejected' '\n+\ttest_must_fail git -C plain.git repack --drop-filtered \\\n+\t\t--filter=blob:limit=1k -a 2>err &&\n+\ttest_grep \"drop-filtered doesn.t work without --dry-run yet\" err\n+'\n+\n+test_expect_success '--drop-filtered requires -a' '\n+\ttest_must_fail git -C plain.git repack --drop-filtered \\\n+\t\t--filter=blob:limit=1k --dry-run 2>err &&\n+\ttest_grep \"drop-filtered requires -a\" err\n+'\n+\n+test_expect_success '--drop-filtered fails with --write-bitmap-index' '\n+\ttest_must_fail git -C plain.git repack --drop-filtered \\\n+\t\t--filter=blob:limit=1k --dry-run -a -b 2>err &&\n+\ttest_grep \"options .--drop-filtered. and .--write-bitmap-index. cannot be used together\" err\n+'\n+\n+test_expect_success '--drop-filtered fails without a promisor remote' '\n+\ttest_must_fail git -C plain.git repack --drop-filtered \\\n+\t\t--filter=blob:limit=1k --dry-run -a 2>err &&\n+\ttest_grep \"drop-filtered requires a promisor remote\" err\n+'\n+\n+# enumeration and safety tests using promisor packs\n+test_expect_success 'setup repo with a promisor remote' '\n+\trm -rf repo &&\n+\ttest_create_repo repo &&\n+\ttest_commit -C repo base &&\n+\n+\t# mark the repo as a partial clone with a promisor remote so the\n+\t# promisor walk and the safety guard are satisfied.\n+\tgit -C repo config core.repositoryformatversion 1 &&\n+\tgit -C repo config extensions.partialclone origin &&\n+\tgit -C repo config remote.origin.promisor true &&\n+\tgit -C repo config remote.origin.url \".\" &&\n+\n+\tBIG=$(promisor_blob big 3072) &&\n+\tSMALL=$(promisor_blob small 512) &&\n+\techo \"$BIG\" >big_oid &&\n+\techo \"$SMALL\" >small_oid\n+'\n+\n+test_expect_success 'promisor blob over the threshold is listed' '\n+\tBIG=$(cat big_oid) &&\n+\tSMALL=$(cat small_oid) &&\n+\n+\tgit -C repo -c repack.writeBitmaps=false \\\n+\t\trepack --drop-filtered --filter=blob:limit=1k --dry-run -a >out &&\n+\n+\ttest_grep \"$BIG\" out &&\n+\ttest_grep ! \"$SMALL\" out\n+'\n+\n+test_expect_success 'locally created blob is never listed' '\n+\tBIG=$(cat big_oid) &&\n+\n+\t# large blob that exists only locally (no promisor pack) must\n+\t# never be a drop candidate: dropping it would be unrecoverable\n+\t# data loss.\n+\ttest-tool genrandom local 4096 >local_content &&\n+\tLOCAL=$(git -C repo hash-object -w --stdin out &&\n+\n+\ttest_grep \"$BIG\" out &&\n+\ttest_grep ! \"$LOCAL\" out\n+'\n+\n+test_expect_success '--dry-run does not remove the filtered objects' '\n+\tBIG=$(cat big_oid) &&\n+\n+\tgit -C repo -c repack.writeBitmaps=false \\\n+\t\trepack --drop-filtered --filter=blob:limit=1k --dry-run -a >out &&\n+\n+\t# candidate blob must still be present after a dry run.\n+\tgit -C repo cat-file -e \"$BIG\"\n+'\n+\n+test_done\n-- \n2.54.0\n\n"},{"id":"548410","messageId":"20260716132848.95982-7-r.siddharth.shrimali@gmail.com","threadId":"66019","inReplyTo":"20260716132848.95982-1-r.siddharth.shrimali@gmail.com","subject":"[RFC PATCH 6/7] builtin/repack: actually drop filtered promisor blobs","fromName":"Siddharth Shrimali","fromEmail":"r.siddharth.shrimali@gmail.com","sentAt":"2026-07-16T13:28:47Z","receivedAt":"2026-07-16T13:29:48Z","isPatch":true,"body":"Make --drop-filtered remove the enumerated promisor blobs instead of\nonly listing them.\n\nThe drop set is computed before repack_promisor_objects() runs, and on\na real run it is passed in so the rebuilt promisor pack omits those\nblobs. --drop-filtered implies -d so the old promisor packs, which\nstill contain the dropped blobs, are removed. Without this the blobs\nwould survive in the redundant packs. The existing repack machinery\nperforms the write-before-delete and fsync, so the drop is crash-safe.\n\nThe dropped blobs become absent locally but remain recoverable from the\npromisor remote, so a later access lazy-fetches them back\ntransparently. --dry-run keeps its previous behavior, i.e. it lists the\ncandidates and changes nothing.\n\nMentored-by: Christian Couder \nMentored-by: Siddharth Asthana \nSigned-off-by: Siddharth Shrimali \n---\n builtin/repack.c | 75 ++++++++++++++++++---------------\n repack-filtered.c | 17 ++------\n repack.h | 4 +-\n t/t7706-repack-drop-filtered.sh | 18 +++++---\n 4 files changed, 59 insertions(+), 55 deletions(-)\n\ndiff --git a/builtin/repack.c b/builtin/repack.c\nindex c2b07477d2..aa3257a98a 100644\n--- a/builtin/repack.c\n+++ b/builtin/repack.c\n@@ -15,6 +15,8 @@\n #include \"repack.h\"\n #include \"shallow.h\"\n #include \"list-objects-filter-options.h\"\n+#include \"oidset.h\"\n+#include \"hex.h\"\n \n #define ALL_INTO_ONE 1\n #define LOOSEN_UNREACHABLE 2\n@@ -143,6 +145,7 @@ int cmd_repack(int argc,\n \tstruct string_list_item *item;\n \tstruct string_list names = STRING_LIST_INIT_DUP;\n \tstruct existing_packs existing = EXISTING_PACKS_INIT;\n+\tstruct oidset drop_oids = OIDSET_INIT;\n \tstruct pack_geometry geometry = { 0 };\n \tstruct tempfile *refs_snapshot = NULL;\n \tint i, ret;\n@@ -269,9 +272,6 @@ int cmd_repack(int argc,\n \t\tdie(_(\"--dry-run only takes effect with --drop-filtered\"));\n \n \tif (drop_filtered) {\n-\t\tif (!dry_run)\n-\t\t\tdie(_(\"--drop-filtered doesn't work without --dry-run yet\"));\n-\n \t\tif (!po_args.filter_options.choice)\n \t\t\tdie(_(\"--drop-filtered requires --filter\"));\n \n@@ -294,6 +294,28 @@ int cmd_repack(int argc,\n \t\t\tdie(_(\"--drop-filtered requires a promisor remote\"));\n \n \t\twrite_bitmaps = 0;\n+\n+\t\t/*\n+\t\t * Dropping objects means rebuilding the promisor packs\n+\t\t * without them and then removing the old packs, so the\n+\t\t * redundant packs must be deleted. Imply -d on a real run.\n+\t\t */\n+\t\tif (!dry_run)\n+\t\t\tdelete_redundant = 1;\n+\n+\t\tret = enumerate_promisor_blobs(repo, &po_args.filter_options, &drop_oids);\n+\n+\t\tif (ret)\n+\t\t\tgoto cleanup;\n+\n+\t\tif (dry_run) {\n+\t\t\tstruct oidset_iter iter;\n+\t\t\tconst struct object_id *oid;\n+\n+\t\t\toidset_iter_init(&drop_oids, &iter);\n+\t\t\twhile ((oid = oidset_iter_next(&iter)))\n+\t\t\t\tprintf(\"%s\\n\", oid_to_hex(oid));\n+\t\t}\n \t}\n \n \tif (delete_redundant && repo->repository_format_precious_objects)\n@@ -406,7 +428,8 @@ int cmd_repack(int argc,\n \t\tstrvec_push(&cmd.args, \"--delta-islands\");\n \n \tif (pack_everything & ALL_INTO_ONE) {\n-\t\trepack_promisor_objects(repo, &po_args, &names, packtmp, NULL);\n+\t\trepack_promisor_objects(repo, &po_args, &names, packtmp,\n+\t\t\t(drop_filtered && !dry_run) ? &drop_oids : NULL);\n \n \t\tif (existing_packs_has_non_kept(&existing) &&\n \t\t delete_redundant &&\n@@ -589,35 +612,20 @@ int cmd_repack(int argc,\n \t\t}\n \t}\n \n-\tif (po_args.filter_options.choice) {\n-\t\tif (drop_filtered) {\n-\t\t\t/*\n-\t\t\t * Enumerate promisor objects directly rather than\n-\t\t\t * going through write_filtered_pack(). The filter\n-\t\t\t * machinery cannot see promisor objects because\n-\t\t\t * repack_promisor_objects() handles them separately\n-\t\t\t * before the filter runs.\n-\t\t\t */\n-\t\t\tret = enumerate_promisor_blobs(repo,\n-\t\t\t\t\t&po_args.filter_options,\n-\t\t\t\t\tdry_run);\n-\t\t\tif (ret)\n-\t\t\t\tgoto cleanup;\n-\t\t} else {\n-\t\t\tstruct write_pack_opts opts = {\n-\t\t\t\t.po_args = &po_args,\n-\t\t\t\t.destination = filter_to,\n-\t\t\t\t.packdir = packdir,\n-\t\t\t\t.packtmp = packtmp,\n-\t\t\t};\n-\n-\t\t\tif (!opts.destination)\n-\t\t\t\topts.destination = packtmp;\n-\n-\t\t\tret = write_filtered_pack(&opts, &existing, &names);\n-\t\t\tif (ret)\n-\t\t\t\tgoto cleanup;\n-\t\t}\n+\tif (po_args.filter_options.choice && !drop_filtered) {\n+\t\tstruct write_pack_opts opts = {\n+\t\t\t.po_args = &po_args,\n+\t\t\t.destination = filter_to,\n+\t\t\t.packdir = packdir,\n+\t\t\t.packtmp = packtmp,\n+\t\t};\n+\n+\t\tif (!opts.destination)\n+\t\t\topts.destination = packtmp;\n+\n+\t\tret = write_filtered_pack(&opts, &existing, &names);\n+\t\tif (ret)\n+\t\t\tgoto cleanup;\n \t}\n \n \tstring_list_sort(&names);\n@@ -697,6 +705,7 @@ int cmd_repack(int argc,\n cleanup:\n \tstring_list_clear(&keep_pack_list, 0);\n \tstring_list_clear(&names, 1);\n+\toidset_clear(&drop_oids);\n \texisting_packs_release(&existing);\n \tpack_geometry_release(&geometry);\n \tpack_objects_args_release(&po_args);\ndiff --git a/repack-filtered.c b/repack-filtered.c\nindex f5a1dae5b1..6f0cecca9b 100644\n--- a/repack-filtered.c\n+++ b/repack-filtered.c\n@@ -87,16 +87,13 @@ static int collect_promisor_blob(const struct object_id *oid,\n \n int enumerate_promisor_blobs(struct repository *repo,\n \t\t\tconst struct list_objects_filter_options *filter,\n-\t\t\tint dry_run)\n+\t\t\tstruct oidset *to_drop)\n {\n \tstruct oidset all_promisor_blobs = OIDSET_INIT;\n-\tstruct oidset to_drop = OIDSET_INIT;\n \tstruct collect_cb_data cb = {\n \t\t.repo = repo,\n \t\t.set = &all_promisor_blobs\n \t};\n-\tstruct oidset_iter iter;\n-\tconst struct object_id *oid;\n \tint ret = 0;\n \n \t/*\n@@ -122,22 +119,14 @@ int enumerate_promisor_blobs(struct repository *repo,\n \n \t/*\n \t * Apply the filter to find which blobs exceed the threshold.\n+\t * The caller has to_drop and is responsible for clearing it.\n \t */\n \tret = list_objects_filter__filter_oidset(repo,\n \t\t(struct list_objects_filter_options *)filter,\n \t\t&all_promisor_blobs,\n-\t\t&to_drop);\n-\tif (ret)\n-\t\tgoto cleanup;\n-\n-\tif (dry_run) {\n-\t\toidset_iter_init(&to_drop, &iter);\n-\t\twhile ((oid = oidset_iter_next(&iter)))\n-\t\t\tprintf(\"%s\\n\", oid_to_hex(oid));\n-\t}\n+\t\tto_drop);\n \n cleanup:\n \toidset_clear(&all_promisor_blobs);\n-\toidset_clear(&to_drop);\n \treturn ret;\n }\ndiff --git a/repack.h b/repack.h\nindex d08e25b852..61e554e4ed 100644\n--- a/repack.h\n+++ b/repack.h\n@@ -168,8 +168,8 @@ int write_filtered_pack(const struct write_pack_opts *opts,\n \t\t\tstruct string_list *names);\n \n int enumerate_promisor_blobs(struct repository *repo,\n-\t\t\t const struct list_objects_filter_options *filter,\n-\t\t\t int dry_run);\n+\t\t\t const struct list_objects_filter_options *filter,\n+\t\t\t struct oidset *to_drop);\n \n int write_cruft_pack(const struct write_pack_opts *opts,\n \t\t const char *cruft_expiration,\ndiff --git a/t/t7706-repack-drop-filtered.sh b/t/t7706-repack-drop-filtered.sh\nindex b558807847..41e7941799 100755\n--- a/t/t7706-repack-drop-filtered.sh\n+++ b/t/t7706-repack-drop-filtered.sh\n@@ -56,12 +56,6 @@ test_expect_success '--dry-run only takes effect with --drop-filtered' '\n \ttest_grep \"dry-run only takes effect with --drop-filtered\" err\n '\n \n-test_expect_success '--drop-filtered without --dry-run is rejected' '\n-\ttest_must_fail git -C plain.git repack --drop-filtered \\\n-\t\t--filter=blob:limit=1k -a 2>err &&\n-\ttest_grep \"drop-filtered doesn.t work without --dry-run yet\" err\n-'\n-\n test_expect_success '--drop-filtered requires -a' '\n \ttest_must_fail git -C plain.git repack --drop-filtered \\\n \t\t--filter=blob:limit=1k --dry-run 2>err &&\n@@ -136,4 +130,16 @@ test_expect_success '--dry-run does not remove the filtered objects' '\n \tgit -C repo cat-file -e \"$BIG\"\n '\n \n+test_expect_success '--drop-filtered removes the promisor blob locally' '\n+\tBIG=$(cat big_oid) &&\n+\tSMALL=$(cat small_oid) &&\n+\n+\tgit -C repo -c repack.writeBitmaps=false \\\n+\t\trepack --drop-filtered --filter=blob:limit=1k -a &&\n+\n+\tgit -C repo cat-file --batch-all-objects --batch-check=\"%(objectname)\" >present &&\n+\t! grep -q \"$BIG\" present &&\n+\tgrep -q \"$SMALL\" present\n+'\n+\n test_done\n-- \n2.54.0\n\n"},{"id":"548411","messageId":"20260716132848.95982-8-r.siddharth.shrimali@gmail.com","threadId":"66019","inReplyTo":"20260716132848.95982-1-r.siddharth.shrimali@gmail.com","subject":"[RFC PATCH 7/7] repack-promisor: record dropped objects in a drop log","fromName":"Siddharth Shrimali","fromEmail":"r.siddharth.shrimali@gmail.com","sentAt":"2026-07-16T13:28:48Z","receivedAt":"2026-07-16T13:29:53Z","isPatch":true,"body":"After --drop-filtered removes promisor blobs, append a record of each\ndropped object to $GIT_DIR/objects/info/promisor-dropped. Each line\nrecords the object ID, a reflog-style timestamp (Unix seconds and\ntimezone), the filter spec, and the promisor remote it was attested\nrecoverable from like the following:\n\n