git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[RFC PATCH 4/7] builtin/repack: enumerate promisor blobs for --drop-filtered

From
Siddharth Shrimali <r.siddharth.shrimali@gmail.com>
Date
Jul 16, 2026, 13:28 UTC
Message-ID
<20260716132848.95982-5-r.siddharth.shrimali@gmail.com>
In-Reply-To
<20260716132848.95982-1-r.siddharth.shrimali@gmail.com>

Add enumeration logic for --drop-filtered. In --dry-run mode, print the OIDs of locally-held promisor blobs that exceed the filter threshold, as candidates for removal.

Reading from write_filtered_pack() cannot work for partial clones. git repack routes promisor objects through a separate path: repack_promisor_objects() repacks them first, and the main pack-objects run uses --exclude-promisor-objects. By the time write_filtered_pack() runs, the promisor blobs are already consumed by the main pack. The filtered pack is always empty on a partial clone.

Instead, walk promisor objects directly via odb_for_each_object() with ODB_FOR_EACH_OBJECT_PROMISOR_ONLY, collecting all promisor blobs into an oidset. The blobs exceeding the filter threshold are then selected using list_objects_filter__filter_oidset().

Every object enumerated this way is a promisor object by construction, so it is guaranteed to be recoverable from the promisor remote and is safe to drop. No separate is_promisor_object() check is needed.

OBJECT_INFO_SKIP_FETCH_OBJECT is passed to every object info query so enumeration never triggers a lazy fetch.

Deletion of the enumerated objects, together with the required promisor-remote verification, will be added separately.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Siddharth Asthana <siddharthasthana31@gmail.com>
Signed-off-by: Siddharth Shrimali <r.siddharth.shrimali@gmail.com>
---
 builtin/repack.c  | 41 ++++++++++++++-------
 repack-filtered.c | 92 +++++++++++++++++++++++++++++++++++++++++++++++
 repack.h          |  4 +++
 3 files changed, 124 insertions(+), 13 deletions(-)
diff --git a/builtin/repack.c b/builtin/repack.c
index 433b2c8205..c2b07477d2 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -590,19 +590,34 @@ int cmd_repack(int argc,
 	}
 
 	if (po_args.filter_options.choice) {
-		struct write_pack_opts opts = {
-			.po_args = &po_args,
-			.destination = filter_to,
-			.packdir = packdir,
-			.packtmp = packtmp,
-		};
-
-		if (!opts.destination)
-			opts.destination = packtmp;
-
-		ret = write_filtered_pack(&opts, &existing, &names);
-		if (ret)
-			goto cleanup;
+		if (drop_filtered) {
+			/*
+			 * Enumerate promisor objects directly rather than
+			 * going through write_filtered_pack(). The filter
+			 * machinery cannot see promisor objects because
+			 * repack_promisor_objects() handles them separately
+			 * before the filter runs.
+			 */
+			ret = enumerate_promisor_blobs(repo,
+					&po_args.filter_options,
+					dry_run);
+			if (ret)
+				goto cleanup;
+		} else {
+			struct write_pack_opts opts = {
+				.po_args = &po_args,
+				.destination = filter_to,
+				.packdir = packdir,
+				.packtmp = packtmp,
+			};
+
+			if (!opts.destination)
+				opts.destination = packtmp;
+
+			ret = write_filtered_pack(&opts, &existing, &names);
+			if (ret)
+				goto cleanup;
+		}
 	}
 
 	string_list_sort(&names);
diff --git a/repack-filtered.c b/repack-filtered.c
index edcf7667c5..f5a1dae5b1 100644
--- a/repack-filtered.c
+++ b/repack-filtered.c
@@ -3,6 +3,12 @@
 #include "repository.h"
 #include "run-command.h"
 #include "string-list.h"
+#include "hex.h"
+#include "packfile.h"
+#include "list-objects-filter-options.h"
+#include "list-objects-filter.h"
+#include "odb.h"
+#include "promisor-remote.h"
 
 int write_filtered_pack(const struct write_pack_opts *opts,
 			struct existing_packs *existing,
@@ -49,3 +55,89 @@ int write_filtered_pack(const struct write_pack_opts *opts,
 	return finish_pack_objects_cmd(existing->repo->hash_algo, opts, &cmd,
 				       names);
 }
+
+struct collect_cb_data {
+	struct repository *repo;
+	struct oidset *set;
+};
+
+static int collect_promisor_blob(const struct object_id *oid,
+		struct object_info *oi UNUSED,
+		void *cb_data)
+{
+	struct collect_cb_data *data = cb_data;
+	struct object_info info = OBJECT_INFO_INIT;
+	enum object_type type;
+
+	info.typep = &type;
+
+	/*
+	 * Use OBJECT_INFO_SKIP_FETCH_OBJECT to avoid triggering a
+	 * lazy fetch while collecting promisor blobs.
+	 */
+	if (odb_read_object_info_extended(data->repo->objects, oid, &info,
+			OBJECT_INFO_SKIP_FETCH_OBJECT) < 0)
+		return 0;
+
+	if (type == OBJ_BLOB)
+		oidset_insert(data->set, oid);
+
+	return 0;
+}
+
+int enumerate_promisor_blobs(struct repository *repo,
+			const struct list_objects_filter_options *filter,
+			int dry_run)
+{
+	struct oidset all_promisor_blobs = OIDSET_INIT;
+	struct oidset to_drop = OIDSET_INIT;
+	struct collect_cb_data cb = {
+		.repo = repo,
+		.set = &all_promisor_blobs
+	};
+	struct oidset_iter iter;
+	const struct object_id *oid;
+	int ret = 0;
+
+	/*
+	 * The caller (cmd_repack) is responsible for validating that a
+	 * blob:limit filter and a promisor remote are present before
+	 * calling this function.
+	 *
+	 * Walk only promisor objects. Every object visited here is
+	 * guaranteed to be recoverable from the promisor remote, so
+	 * it is safe to drop.
+	 *
+	 * We do not use write_filtered_pack() here because git repack
+	 * routes promisor objects through repack_promisor_objects()
+	 * before the filter machinery runs, so the filtered pack never
+	 * contains promisor blobs. Direct enumeration via
+	 * ODB_FOR_EACH_OBJECT_PROMISOR_ONLY is the correct approach.
+	 */
+	ret = odb_for_each_object(repo->objects, NULL,
+			collect_promisor_blob, &cb,
+			ODB_FOR_EACH_OBJECT_PROMISOR_ONLY);
+	if (ret)
+		goto cleanup;
+
+	/*
+	 * Apply the filter to find which blobs exceed the threshold.
+	 */
+	ret = list_objects_filter__filter_oidset(repo,
+		(struct list_objects_filter_options *)filter,
+		&all_promisor_blobs,
+		&to_drop);
+	if (ret)
+		goto cleanup;
+
+	if (dry_run) {
+		oidset_iter_init(&to_drop, &iter);
+		while ((oid = oidset_iter_next(&iter)))
+			printf("%s\n", oid_to_hex(oid));
+	}
+
+cleanup:
+	oidset_clear(&all_promisor_blobs);
+	oidset_clear(&to_drop);
+	return ret;
+}
diff --git a/repack.h b/repack.h
index a5a3f7c6ba..d08e25b852 100644
--- a/repack.h
+++ b/repack.h
@@ -167,6 +167,10 @@ int write_filtered_pack(const struct write_pack_opts *opts,
 			struct existing_packs *existing,
 			struct string_list *names);
 
+int enumerate_promisor_blobs(struct repository *repo,
+			       const struct list_objects_filter_options *filter,
+			       int dry_run);
+
 int write_cruft_pack(const struct write_pack_opts *opts,
 		     const char *cruft_expiration,
 		     unsigned long combine_cruft_below_size,
-- 
2.54.0
Previous: Siddharth ShrimaliNext: Siddharth Shrimali
Message 10 of 76 in “repack: add --drop-filtered to reclaim space in partial clones”
  1. 0/7 repack: add --drop-filtered to reclaim space in partial clonesSiddharth Shrimali, Jul 16, 2026
  2. 1/7 builtin/repack.c: add --drop-filtered and --dry-run optionsSiddharth Shrimali, Jul 16, 2026
  3. Junio C HamanoJul 16, 2026
  4. Siddharth ShrimaliJul 17, 2026
  5. Siddharth AsthanaJul 23, 2026
  6. Christian CouderJul 18, 2026
  7. Siddharth ShrimaliJul 20, 2026
  8. 2/7 list-objects-filter: add list_objects_filter__filter_oidset()Siddharth Shrimali, Jul 16, 2026
  9. 3/7 repack-promisor: allow excluding objects from the rebuilt promisor packSiddharth Shrimali, Jul 16, 2026
  10. 4/7 builtin/repack: enumerate promisor blobs for --drop-filteredSiddharth Shrimali, Jul 16, 2026
  11. 5/7 t7706: test --drop-filtered enumeration and validationSiddharth Shrimali, Jul 16, 2026
  12. 6/7 builtin/repack: actually drop filtered promisor blobsSiddharth Shrimali, Jul 16, 2026
  13. Siddharth AsthanaJul 23, 2026
  14. Siddharth ShrimaliJul 25, 2026
  15. 7/7 repack-promisor: record dropped objects in a drop logSiddharth Shrimali, Jul 16, 2026
  16. Siddharth AsthanaJul 23, 2026
  17. Siddharth ShrimaliJul 25, 2026
  18. Siddharth AsthanaJul 23, 2026
  19. Siddharth ShrimaliJul 25, 2026
  20. 0/7 repack: add --drop-filtered to reclaim space in partial clonesSiddharth Shrimali, Jul 30, 2026
  21. 1/7 builtin/repack.c: add --drop-filtered and --dry-run optionsSiddharth Shrimali, Jul 30, 2026
  22. Siddharth AsthanaAug 4, 2026
  23. 2/7 list-objects-filter: add list_objects_filter__filter_oidset()Siddharth Shrimali, Jul 30, 2026
  24. 3/7 repack-promisor: allow excluding objects from the rebuilt promisor packSiddharth Shrimali, Jul 30, 2026
  25. 4/7 builtin/repack: enumerate promisor blobs for --drop-filteredSiddharth Shrimali, Jul 30, 2026
  26. 5/7 builtin/repack: actually drop filtered promisor blobsSiddharth Shrimali, Jul 30, 2026
  27. 6/7 builtin/repack: add safety guards for --drop-filteredSiddharth Shrimali, Jul 30, 2026
  28. Siddharth AsthanaAug 4, 2026
  29. 7/7 Documentation/git-repack: document --drop-filtered and --dry-runSiddharth Shrimali, Jul 30, 2026
  30. Junio C HamanoJul 31, 2026
  31. Siddharth ShrimaliAug 1, 2026
  32. Junio C HamanoAug 2, 2026
  33. Siddharth ShrimaliAug 2, 2026
  34. Siddharth AsthanaAug 4, 2026
  35. 0/7 repack: add --drop-filtered to reclaim space in partial clonesSiddharth Shrimali, Aug 6, 2026
  36. 1/7 builtin/repack.c: add --drop-filtered and --dry-run optionsSiddharth Shrimali, Aug 6, 2026
  37. 2/7 list-objects-filter: add list_objects_filter__filter_oidset()Siddharth Shrimali, Aug 6, 2026
  38. 3/7 repack-promisor: allow excluding objects from the rebuilt promisor packSiddharth Shrimali, Aug 6, 2026
  39. 4/7 builtin/repack: enumerate promisor blobs for --drop-filteredSiddharth Shrimali, Aug 6, 2026
  40. 5/7 builtin/repack: actually drop filtered promisor blobsSiddharth Shrimali, Aug 6, 2026
  41. 6/7 builtin/repack: add guards for --drop-filteredSiddharth Shrimali, Aug 6, 2026
  42. 7/7 Documentation/git-repack: document --drop-filtered and --dry-runSiddharth Shrimali, Aug 6, 2026
  43. Junio C HamanoAug 6, 2026
  44. Junio C HamanoAug 6, 2026
  45. Siddharth ShrimaliAug 7, 2026
  46. Junio C HamanoAug 7, 2026
  47. Siddharth ShrimaliAug 8, 2026
  48. 0/7 repack: add --drop-filtered to reclaim space in partial clonesSiddharth Shrimali, Aug 10, 2026
  49. 1/7 builtin/repack.c: add --drop-filtered and --dry-run optionsSiddharth Shrimali, Aug 10, 2026
  50. 2/7 list-objects-filter: add list_objects_filter__filter_oidset()Siddharth Shrimali, Aug 10, 2026
  51. Christian CouderAug 12, 2026
  52. Junio C HamanoAug 12, 2026
  53. Siddharth ShrimaliAug 12, 2026
  54. 3/7 repack-promisor: allow excluding objects from the rebuilt promisor packSiddharth Shrimali, Aug 10, 2026
  55. 4/7 builtin/repack: enumerate promisor blobs for --drop-filteredSiddharth Shrimali, Aug 10, 2026
  56. Christian CouderAug 12, 2026
  57. 5/7 builtin/repack: actually drop filtered promisor blobsSiddharth Shrimali, Aug 10, 2026
  58. 6/7 builtin/repack: add guards for --drop-filteredSiddharth Shrimali, Aug 10, 2026
  59. Christian CouderAug 12, 2026
  60. Siddharth ShrimaliAug 12, 2026
  61. 7/7 Documentation/git-repack: document --drop-filtered and --dry-runSiddharth Shrimali, Aug 10, 2026
  62. Christian CouderAug 12, 2026
  63. Junio C HamanoAug 11, 2026
  64. Siddharth ShrimaliAug 12, 2026
  65. 0/6 repack: add --drop-filtered to reclaim space in partial clonesSiddharth Shrimali, Aug 13, 2026
  66. 1/6 builtin/repack: add --drop-filtered and --dry-run optionsSiddharth Shrimali, Aug 13, 2026
  67. 2/6 list-objects-filter: add list_objects_filter__filter_oidset()Siddharth Shrimali, Aug 13, 2026
  68. 3/6 repack-promisor: allow excluding objects from the rebuilt promisor packSiddharth Shrimali, Aug 13, 2026
  69. 4/6 builtin/repack: enumerate promisor blobs for --drop-filteredSiddharth Shrimali, Aug 13, 2026
  70. 5/6 builtin/repack: actually drop filtered promisor blobsSiddharth Shrimali, Aug 13, 2026
  71. 6/6 builtin/repack: add guards for --drop-filteredSiddharth Shrimali, Aug 13, 2026
  72. Samuel BronsonSep 3, 2026
  73. Siddharth ShrimaliSep 4, 2026
  74. Christian CouderAug 14, 2026
  75. Junio C HamanoAug 14, 2026
  76. Siddharth ShrimaliAug 14, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.