git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 19/19] fetch: add object filtering to fetch

From
JHJeff Hostetler <git@jeffhostetler.com>
Date
Jul 13, 2017, 17:34 UTC
Message-ID
<20170713173459.3559-20-git@jeffhostetler.com>
In-Reply-To
<20170713173459.3559-1-git@jeffhostetler.com>
From: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
---
 builtin/fetch.c | 27 ++++++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 5f2c2ab..306c165 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -16,6 +16,7 @@
 #include "connected.h"
 #include "argv-array.h"
 #include "utf8.h"
+#include "object-filter.h"
 
 static const char * const builtin_fetch_usage[] = {
 	N_("git fetch [<options>] [<repository> [<refspec>...]]"),
@@ -52,6 +53,7 @@ static const char *recurse_submodules_default;
 static int shown_url = 0;
 static int refmap_alloc, refmap_nr;
 static const char **refmap_array;
+static struct object_filter_options filter_options;
 
 static int option_parse_recurse_submodules(const struct option *opt,
 				   const char *arg, int unset)
@@ -141,6 +143,14 @@ static struct option builtin_fetch_options[] = {
 			TRANSPORT_FAMILY_IPV4),
 	OPT_SET_INT('6', "ipv6", &family, N_("use IPv6 addresses only"),
 			TRANSPORT_FAMILY_IPV6),
+
+	OPT_PARSE_FILTER_OMIT_ALL_BLOBS(&filter_options),
+	OPT_PARSE_FILTER_OMIT_LARGE_BLOBS(&filter_options),
+	OPT_PARSE_FILTER_USE_SPARSE(&filter_options),
+
+	/* OPT_PARSE_FILTER_PRINT_MANIFEST(&filter_options), */
+	/* OPT_PARSE_FILTER_RELAX(&filter_options), */
+
 	OPT_END()
 };
 
@@ -733,6 +743,14 @@ static int store_updated_refs(const char *raw_url, const char *remote_name,
 	const char *filename = dry_run ? "/dev/null" : git_path_fetch_head();
 	int want_status;
 	int summary_width = transport_summary_width(ref_map);
+	struct check_connected_options opt = CHECK_CONNECTED_INIT;
+
+	/*
+	 * Relax consistency check to allow missing blobs (presumably
+	 * because they are exactly the set that we requested be
+	 * omitted.
+	 */
+	opt.filter_relax = object_filter_enabled(&filter_options);
 
 	fp = fopen(filename, "a");
 	if (!fp)
@@ -744,7 +762,7 @@ static int store_updated_refs(const char *raw_url, const char *remote_name,
 		url = xstrdup("foreign");
 
 	rm = ref_map;
-	if (check_connected(iterate_ref_map, &rm, NULL)) {
+	if (check_connected(iterate_ref_map, &rm, &opt)) {
 		rc = error(_("%s did not send all necessary objects\n"), url);
 		goto abort;
 	}
@@ -885,6 +903,13 @@ static int quickfetch(struct ref *ref_map)
 	struct check_connected_options opt = CHECK_CONNECTED_INIT;
 
 	/*
+	 * Relax consistency check to allow missing blobs (presumably
+	 * because they are exactly the set that we requested be
+	 * omitted.
+	 */
+	opt.filter_relax = object_filter_enabled(&filter_options);
+
+	/*
 	 * If we are deepening a shallow clone we already have these
 	 * objects reachable.  Running rev-list here will return with
 	 * a good (0) exit status and we'll bypass the fetch that we
-- 
2.9.3
Previous: Jeff HostetlerNext: Jeff Hostetler
Message 15 of 20 in “WIP object filtering for partial clone”
  1. 00/19 WIP object filtering for partial cloneJeff Hostetler, Jul 13, 2017
  2. 01/19 dir: refactor add_excludes()Jeff Hostetler, Jul 13, 2017
  3. 03/19 list-objects: filter objects in traverse_commit_listJeff Hostetler, Jul 13, 2017
  4. 06/19 list-objects-filters: add use-sparse-checkout filterJeff Hostetler, Jul 13, 2017
  5. 07/19 object-filter: common declarations for object filteringJeff Hostetler, Jul 13, 2017
  6. 11/19 pack-objects: add object filtering supportJeff Hostetler, Jul 13, 2017
  7. 10/19 t6112: rev-list object filtering testJeff Hostetler, Jul 13, 2017
  8. 13/19 upload-pack: add filter-objects to protocol documentationJeff Hostetler, Jul 13, 2017
  9. 17/19 clone: add filter argumentsJeff Hostetler, Jul 13, 2017
  10. 18/19 index-pack: relax consistency checks for omitted objectsJeff Hostetler, Jul 13, 2017
  11. 12/19 pack-objects: add filtering help textJeff Hostetler, Jul 13, 2017
  12. 14/19 upload-pack: add object filteringJeff Hostetler, Jul 13, 2017
  13. 15/19 fetch-pack: add object filtering supportJeff Hostetler, Jul 13, 2017
  14. 16/19 connected: add filter_allow_omitted option to APIJeff Hostetler, Jul 13, 2017
  15. 19/19 fetch: add object filtering to fetchJeff Hostetler, Jul 13, 2017
  16. 08/19 rev-list: add object filtering supportJeff Hostetler, Jul 13, 2017
  17. 09/19 rev-list: add filtering help textJeff Hostetler, Jul 13, 2017
  18. 02/19 oidset2: create oidset subclass with object length and pathnameJeff Hostetler, Jul 13, 2017
  19. 05/19 list-objects-filters: add omit-large-blobs filterJeff Hostetler, Jul 13, 2017
  20. 04/19 list-objects-filters: add omit-all-blobs filterJeff Hostetler, Jul 13, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.