git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 12/13] pack-objects: add list-objects filtering

From
JHJeff Hostetler <git@jeffhostetler.com>
Date
Oct 24, 2017, 18:53 UTC
Message-ID
<20171024185332.57261-13-git@jeffhostetler.com>
In-Reply-To
<20171024185332.57261-1-git@jeffhostetler.com>
From: Jeff Hostetler <jeffhost@microsoft.com>

Teach pack-objects to use the filtering provided by the traverse_commit_list_filtered() interface to omit unwanted objects from the resulting packfile.

This feature is intended for partial clone/fetch.
Filtering requires the use of the "--stdout" option.
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
---
 Documentation/git-pack-objects.txt |  8 +++++++-
 builtin/pack-objects.c             | 18 +++++++++++++++++-
 2 files changed, 24 insertions(+), 2 deletions(-)
diff --git a/Documentation/git-pack-objects.txt b/Documentation/git-pack-objects.txt
index 473a161..8b4a223 100644
--- a/Documentation/git-pack-objects.txt
+++ b/Documentation/git-pack-objects.txt
@@ -12,7 +12,8 @@ SYNOPSIS
 'git pack-objects' [-q | --progress | --all-progress] [--all-progress-implied]
 	[--no-reuse-delta] [--delta-base-offset] [--non-empty]
 	[--local] [--incremental] [--window=<n>] [--depth=<n>]
-	[--revs [--unpacked | --all]] [--stdout | base-name]
+	[--revs [--unpacked | --all]]
+	[--stdout [--filter=<filter-spec>] | base-name]
 	[--shallow] [--keep-true-parents] < object-list
 
 
@@ -236,6 +237,11 @@ So does `git bundle` (see linkgit:git-bundle[1]) when it creates a bundle.
 	With this option, parents that are hidden by grafts are packed
 	nevertheless.
 
+--filter=<filter-spec>::
+	Requires `--stdout`.  Omits certain objects (usually blobs) from
+	the resulting packfile.  See linkgit:git-rev-list[1] for valid
+	`<filter-spec>` forms.
+
 SEE ALSO
 --------
 linkgit:git-rev-list[1]
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 6e77dfd..a251850 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -79,6 +79,8 @@ static unsigned long cache_max_small_delta_size = 1000;
 
 static unsigned long window_memory_limit = 0;
 
+static struct list_objects_filter_options filter_options;
+
 /*
  * stats
  */
@@ -2816,7 +2818,12 @@ static void get_object_list(int ac, const char **av)
 	if (prepare_revision_walk(&revs))
 		die("revision walk setup failed");
 	mark_edges_uninteresting(&revs, show_edge);
-	traverse_commit_list(&revs, show_commit, show_object, NULL);
+	if (filter_options.choice)
+		traverse_commit_list_filtered(&filter_options, &revs,
+					      show_commit, show_object,
+					      NULL, NULL);
+	else
+		traverse_commit_list(&revs, show_commit, show_object, NULL);
 
 	if (unpack_unreachable_expiration) {
 		revs.ignore_missing_links = 1;
@@ -2952,6 +2959,9 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 			 N_("use a bitmap index if available to speed up counting objects")),
 		OPT_BOOL(0, "write-bitmap-index", &write_bitmap_index,
 			 N_("write a bitmap index together with the pack index")),
+
+		OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
+
 		OPT_END(),
 	};
 
@@ -3028,6 +3038,12 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 	if (!rev_list_all || !rev_list_reflog || !rev_list_index)
 		unpack_unreachable_expiration = 0;
 
+	if (filter_options.choice) {
+		if (!pack_to_stdout)
+			die("cannot use filtering with an indexable pack.");
+		use_bitmap_index = 0;
+	}
+
 	/*
 	 * "soft" reasons not to use bitmaps - for on-disk repack by default we want
 	 *
-- 
2.9.3
Previous: Jeff HostetlerNext: Jeff Hostetler
Message 13 of 37 in “WIP Partial clone part 1: object filtering”
  1. 00/13 WIP Partial clone part 1: object filteringJeff Hostetler, Oct 24, 2017
  2. 02/13 list-objects-filter-map: extend oidmap to collect omitted objectsJeff Hostetler, Oct 24, 2017
  3. Junio C HamanoOct 25, 2017
  4. Jeff HostetlerOct 25, 2017
  5. Junio C HamanoOct 26, 2017
  6. 06/13 list-objects-filter-sparse: add sparse filterJeff Hostetler, Oct 24, 2017
  7. 08/13 list-objects: add traverse_commit_list_filtered methodJeff Hostetler, Oct 24, 2017
  8. Jonathan TanOct 25, 2017
  9. Jeff HostetlerOct 25, 2017
  10. 11/13 t6112: rev-list object filtering testJeff Hostetler, Oct 24, 2017
  11. 05/13 list-objects-filter-blobs-limit: add large blob filteringJeff Hostetler, Oct 24, 2017
  12. 13/13 t5317: pack-objects object filtering testJeff Hostetler, Oct 24, 2017
  13. 12/13 pack-objects: add list-objects filteringJeff Hostetler, Oct 24, 2017
  14. 10/13 rev-list: add list-objects filtering supportJeff Hostetler, Oct 24, 2017
  15. Jonathan TanOct 25, 2017
  16. Jeff HostetlerOct 25, 2017
  17. 04/13 list-objects-filter-blobs-none: add filter to omit all blobsJeff Hostetler, Oct 24, 2017
  18. 09/13 extension.partialclone: introduce partial clone extensionJeff Hostetler, Oct 24, 2017
  19. 01/13 dir: allow exclusions from blob in addition to fileJeff Hostetler, Oct 24, 2017
  20. Eric SunshineOct 25, 2017
  21. Junio C HamanoOct 25, 2017
  22. Jeff HostetlerOct 25, 2017
  23. Junio C HamanoOct 26, 2017
  24. Jeff HostetlerOct 26, 2017
  25. 07/13 list-objects-filter-options: common argument parsingJeff Hostetler, Oct 24, 2017
  26. Jonathan TanOct 25, 2017
  27. Jeff HostetlerOct 25, 2017
  28. 03/13 list-objects: filter objects in traverse_commit_listJeff Hostetler, Oct 24, 2017
  29. Jonathan TanOct 25, 2017
  30. Jeff HostetlerOct 25, 2017
  31. Jonathan TanOct 25, 2017
  32. Junio C HamanoOct 25, 2017
  33. Jonathan TanOct 25, 2017
  34. Jeff HostetlerOct 25, 2017
  35. Junio C HamanoOct 26, 2017
  36. Junio C HamanoOct 26, 2017
  37. Jonathan NiederOct 30, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.