[RFC] fetch: support hideRefs to speed up connectivity checks

From: Eric Wong <e@80x24.org>
Date: Feb 9, 2023, 12:28 UTC
Message-ID: <20230209122857.M669733@dcvr>

Not sure if this is the right way to go about this... If it's close, maybe --exclude-hidden=fetch can be supported. I'm using `receive' for now to minimize the change.

With roughly 800 remotes all fetching to their own refs/remotes/$REMOTE/* island, the connectivity check[1] gets expensive for each fetch.

To do a no-op fetch on one $REMOTE out of hundreds, hideRefs now allows the no-op fetch to take ~30 seconds instead of ~20 minutes on a noisy, RAM-constrained machine (localhost, so no network latency):

   git -c transfer.hideRefs=refs \
	-c transfer.hideRefs='!refs/remotes/$REMOTE/' \
	fetch $REMOTE

I initially considered passing --negotiation-tip OIDs, but this seems like an easier solution as I'm not yet familiar with this code and prefer to avoid writing too much C.

[1] `git rev-list --objects --stdin --not --all --quiet --alternate-refs'
    gets painful w/o enough RAM to cache the repo, even on a SATA-2 SSD.
---
 builtin/fetch.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index 12978622d5..473d99fd26 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1131,6 +1131,7 @@ static int store_updated_refs(const char *raw_url, const char *remote_name,
 	if (!connectivity_checked) {
 		struct check_connected_options opt = CHECK_CONNECTED_INIT;
 
+		opt.exclude_hidden_refs_section = "receive";
 		rm = ref_map;
 		if (check_connected(iterate_ref_map, &rm, &opt)) {
 			rc = error(_("%s did not send all necessary objects\n"), url);
@@ -1324,6 +1325,7 @@ static int check_exist_and_connected(struct ref *ref_map)
 	}
 
 	opt.quiet = 1;
+	opt.exclude_hidden_refs_section = "receive";
 	return check_connected(iterate_ref_map, &rm, &opt);
 }