git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 04/10] builtin/cat-file: support "blob:limit=" objects filter

From
Patrick Steinhardt <ps@pks.im>
Date
Mar 27, 2025, 09:43 UTC
Message-ID
<20250327-pks-cat-file-object-type-filter-v2-4-4bbc7085d7c5@pks.im>
In-Reply-To
<20250327-pks-cat-file-object-type-filter-v2-0-4bbc7085d7c5@pks.im>

Implement support for the "blob:limit=" filter in git-cat-file(1), which causes us to omit all blobs that are bigger than a certain size.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/git-cat-file.adoc |  5 +++++
 builtin/cat-file.c              | 11 ++++++++++-
 t/t1006-cat-file.sh             | 18 +++++++++++++++---
 3 files changed, 30 insertions(+), 4 deletions(-)
diff --git a/Documentation/git-cat-file.adoc b/Documentation/git-cat-file.adoc
index bb32f715944..62bfb00f4b1 100644
--- a/Documentation/git-cat-file.adoc
+++ b/Documentation/git-cat-file.adoc
@@ -88,6 +88,11 @@ OPTIONS
 	one of the following:
 +
 The form '--filter=blob:none' omits all blobs.
++
+The form '--filter=blob:limit=<n>[kmg]' omits blobs of size at least n
+bytes or units.  n may be zero.  The suffixes k, m, and g can be used to name
+units in KiB, MiB, or GiB.  For example, 'blob:limit=1k' is the same as
+'blob:limit=1024'.
 
 --path=<path>::
 	For use with `--textconv` or `--filters`, to allow specifying an object
diff --git a/builtin/cat-file.c b/builtin/cat-file.c
index e783dbbad58..55755a461bc 100644
--- a/builtin/cat-file.c
+++ b/builtin/cat-file.c
@@ -473,8 +473,11 @@ static void batch_object_write(const char *obj_name,
 		int ret;
 
 		if (use_mailmap ||
-		    opt->objects_filter.choice == LOFC_BLOB_NONE)
+		    opt->objects_filter.choice == LOFC_BLOB_NONE ||
+		    opt->objects_filter.choice == LOFC_BLOB_LIMIT)
 			data->info.typep = &data->type;
+		if (opt->objects_filter.choice == LOFC_BLOB_LIMIT)
+			data->info.sizep = &data->size;
 
 		if (pack)
 			ret = packed_object_info(the_repository, pack, offset,
@@ -497,6 +500,11 @@ static void batch_object_write(const char *obj_name,
 			if (data->type == OBJ_BLOB)
 				return;
 			break;
+		case LOFC_BLOB_LIMIT:
+			if (data->type == OBJ_BLOB &&
+			    data->size >= opt->objects_filter.blob_limit_value)
+				return;
+			break;
 		default:
 			BUG("unsupported objects filter");
 		}
@@ -1038,6 +1046,7 @@ int cmd_cat_file(int argc,
 	case LOFC_DISABLED:
 		break;
 	case LOFC_BLOB_NONE:
+	case LOFC_BLOB_LIMIT:
 		if (!batch.enabled)
 			usage(_("objects filter only supported in batch mode"));
 		break;
diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh
index d00073f8add..1a0931bd2ca 100755
--- a/t/t1006-cat-file.sh
+++ b/t/t1006-cat-file.sh
@@ -1356,11 +1356,12 @@ test_expect_success PERL '--batch-command info is unbuffered by default' '
 test_expect_success 'setup for objects filter' '
 	git init repo &&
 	(
-		# Seed the repository with three different sets of objects:
+		# Seed the repository with four different sets of objects:
 		#
 		#   - The first set is fully packed and has a bitmap.
 		#   - The second set is packed, but has no bitmap.
 		#   - The third set is loose.
+		#   - The fourth set is loose and contains big objects.
 		#
 		# This ensures that we cover all these types as expected.
 		cd repo &&
@@ -1368,7 +1369,14 @@ test_expect_success 'setup for objects filter' '
 		git repack -Adb &&
 		test_commit second &&
 		git repack -d &&
-		test_commit third
+		test_commit third &&
+
+		for n in 1000 10000
+		do
+			printf "%"$n"s" X >large.$n || return 1
+		done &&
+		git add large.* &&
+		git commit -m fourth
 	)
 '
 
@@ -1380,7 +1388,7 @@ test_expect_success 'objects filter with unknown option' '
 	test_cmp expect err
 '
 
-for option in blob:limit=1 object:type=tag sparse:oid=1234 tree:1 sparse:path=x
+for option in object:type=tag sparse:oid=1234 tree:1 sparse:path=x
 do
 	test_expect_success "objects filter with unsupported option $option" '
 		case "$option" in
@@ -1413,5 +1421,9 @@ test_objects_filter () {
 }
 
 test_objects_filter "blob:none"
+test_objects_filter "blob:limit=1"
+test_objects_filter "blob:limit=500"
+test_objects_filter "blob:limit=1000"
+test_objects_filter "blob:limit=1g"
 
 test_done
-- 
2.49.0.472.ge94155a9ec.dirty
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 11 of 28 in “builtin/cat-file: allow filtering objects in batch mode”
  1. 00/10 builtin/cat-file: allow filtering objects in batch modePatrick Steinhardt, Mar 27, 2025
  2. 01/10 builtin/cat-file: rename variable that tracks usagePatrick Steinhardt, Mar 27, 2025
  3. Karthik NayakApr 1, 2025
  4. Patrick SteinhardtApr 2, 2025
  5. Junio C HamanoApr 7, 2025
  6. 02/10 builtin/cat-file: wire up an option to filter objectsPatrick Steinhardt, Mar 27, 2025
  7. Toon ClaesApr 1, 2025
  8. Patrick SteinhardtApr 2, 2025
  9. Karthik NayakApr 1, 2025
  10. Patrick SteinhardtApr 2, 2025
  11. 04/10 builtin/cat-file: support "blob:limit=" objects filterPatrick Steinhardt, Mar 27, 2025
  12. 03/10 builtin/cat-file: support "blob:none" objects filterPatrick Steinhardt, Mar 27, 2025
  13. Karthik NayakApr 1, 2025
  14. Karthik NayakApr 1, 2025
  15. Patrick SteinhardtApr 2, 2025
  16. 05/10 builtin/cat-file: support "object:type=" objects filterPatrick Steinhardt, Mar 27, 2025
  17. 06/10 pack-bitmap: allow passing payloads to `show_reachable_fn()`Patrick Steinhardt, Mar 27, 2025
  18. Toon ClaesApr 1, 2025
  19. Patrick SteinhardtApr 2, 2025
  20. 07/10 pack-bitmap: add function to iterate over filtered bitmapped objectsPatrick Steinhardt, Mar 27, 2025
  21. 08/10 pack-bitmap: introduce function to check whether a pack is bitmappedPatrick Steinhardt, Mar 27, 2025
  22. Toon ClaesApr 1, 2025
  23. Patrick SteinhardtApr 2, 2025
  24. 10/10 builtin/cat-file: use bitmaps to efficiently filter by object typePatrick Steinhardt, Mar 27, 2025
  25. 09/10 builtin/cat-file: deduplicate logic to iterate over all objectsPatrick Steinhardt, Mar 27, 2025
  26. Toon ClaesApr 1, 2025
  27. Patrick SteinhardtApr 2, 2025
  28. Toon ClaesApr 3, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.