git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 05/24] fetch: add --bundle-uri option

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
May 20, 2022, 18:40 UTC
Message-ID
<c7ec97011e0b2b6a347a144549be93bd78526476.1653072042.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.1234.git.1653072042.gitgitgadget@gmail.com>
From: Derrick Stolee <derrickstolee@github.com>

Teach 'git fetch' a new --bundle-uri=<uri> option which changes the mode from fetching from a remote using the Git protocol to fetching a bundle from the given <uri>. See Documentation/technical/bundle-uri.txt for more information on the design of this feature.

This implementation is limited to the most basic version of the feature. We expect the content at that URI to be a bundle file, not a bundle list. Bundle lists will be implemented later.

This implementation is sufficient for a bundle provider to create a single bootstrap bundle for a large repository. The user would bootstrap a repository using a sequence of Git commands, such as:

 1. git init <repo> && cd <repo>
 2. git fetch --bundle-uri=<uri>
 3. git remote add origin <url>
 4. git fetch origin
 5. git checkout FETCH_HEAD

Later changes will make this seamless within a 'git clone' command, but this implementation is large enough to delay that integration.

Currently, this option supports URIs using "http(s)://", "file://" or simply specifying a filename. Other protocols could be added in the future.

RFC TODO: add end-to-end tests of this workflow.
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
---
 Documentation/fetch-options.txt |  5 +++++
 Documentation/git-fetch.txt     |  1 +
 builtin/fetch.c                 | 10 ++++++++++
 3 files changed, 16 insertions(+)
diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index 622bd84768b..09bd1feeed8 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -317,3 +317,8 @@ endif::git-pull[]
 -6::
 --ipv6::
 	Use IPv6 addresses only, ignoring IPv4 addresses.
+
+--bundle-uri=<uri>::
+	Instead of fetching from a remote, fetch a bundle from the given
+	`<uri>` and unbundle the data into the local repository. The refs
+	in the bundle will be stored under the `refs/bundle/*` namespace.
diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
index e9d364669af..4fd8911b336 100644
--- a/Documentation/git-fetch.txt
+++ b/Documentation/git-fetch.txt
@@ -13,6 +13,7 @@ SYNOPSIS
 'git fetch' [<options>] <group>
 'git fetch' --multiple [<options>] [(<repository> | <group>)...]
 'git fetch' --all [<options>]
+'git fetch' --bundle-uri=<uri> [<options>]
 
 
 DESCRIPTION
diff --git a/builtin/fetch.c b/builtin/fetch.c
index e3791f09ed5..cb0d2fbe82c 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -29,6 +29,7 @@
 #include "commit-graph.h"
 #include "shallow.h"
 #include "worktree.h"
+#include "bundle-uri.h"
 
 #define FORCED_UPDATES_DELAY_WARNING_IN_MS (10 * 1000)
 
@@ -37,6 +38,7 @@ static const char * const builtin_fetch_usage[] = {
 	N_("git fetch [<options>] <group>"),
 	N_("git fetch --multiple [<options>] [(<repository> | <group>)...]"),
 	N_("git fetch --all [<options>]"),
+	N_("git fetch --bundle-uri=<uri> [<options>]"),
 	NULL
 };
 
@@ -86,6 +88,7 @@ static struct string_list negotiation_tip = STRING_LIST_INIT_NODUP;
 static int fetch_write_commit_graph = -1;
 static int stdin_refspecs = 0;
 static int negotiate_only;
+static const char *bundle_uri;
 
 static int git_fetch_config(const char *k, const char *v, void *cb)
 {
@@ -224,6 +227,8 @@ static struct option builtin_fetch_options[] = {
 		 N_("write the commit-graph after fetching")),
 	OPT_BOOL(0, "stdin", &stdin_refspecs,
 		 N_("accept refspecs from stdin")),
+	OPT_STRING(0, "bundle-uri", &bundle_uri, N_("uri"),
+		   N_("download bundle data from the given URI instead of from a remote")),
 	OPT_END()
 };
 
@@ -2181,6 +2186,11 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
 	if (dry_run)
 		write_fetch_head = 0;
 
+	if (bundle_uri) {
+		result = fetch_bundle_uri(the_repository, bundle_uri);
+		goto cleanup;
+	}
+
 	if (all) {
 		if (argc == 1)
 			die(_("fetch --all does not take a repository argument"));
-- 
gitgitgadget
Previous: Derrick Stolee via GitGitGadgetNext: Derrick Stolee via GitGitGadget
Message 5 of 25 in “[RFC] Bundle URIs Combined RFC”
  1. 00/24 [RFC] Bundle URIs Combined RFCDerrick Stolee via GitGitGadget, May 20, 2022
  2. 01/24 docs: document bundle URI standardDerrick Stolee via GitGitGadget, May 20, 2022
  3. 02/24 remote-curl: add 'get' capabilityDerrick Stolee via GitGitGadget, May 20, 2022
  4. 06/24 fetch: add 'refs/bundle/' to log.excludeDecorationDerrick Stolee via GitGitGadget, May 20, 2022
  5. 05/24 fetch: add --bundle-uri optionDerrick Stolee via GitGitGadget, May 20, 2022
  6. 04/24 bundle-uri: add support for http(s):// and file://Derrick Stolee via GitGitGadget, May 20, 2022
  7. 03/24 bundle-uri: create basic file-copy logicDerrick Stolee via GitGitGadget, May 20, 2022
  8. 12/24 bundle-uri: unit test "key=value" parsingÆvar Arnfjörð Bjarmason via GitGitGadget, May 20, 2022
  9. 10/24 bundle-uri: create base key-value pair parsingDerrick Stolee via GitGitGadget, May 20, 2022
  10. 07/24 clone: add --bundle-uri optionDerrick Stolee via GitGitGadget, May 20, 2022
  11. 11/24 bundle-uri: create "key=value" line parsingÆvar Arnfjörð Bjarmason via GitGitGadget, May 20, 2022
  12. 14/24 bundle-uri: parse bundle list in config formatDerrick Stolee via GitGitGadget, May 20, 2022
  13. 09/24 bundle-uri: create bundle_list struct and helpersDerrick Stolee via GitGitGadget, May 20, 2022
  14. 15/24 bundle-uri: fetch a list of bundlesDerrick Stolee via GitGitGadget, May 20, 2022
  15. 16/24 protocol v2: add server-side "bundle-uri" skeletonÆvar Arnfjörð Bjarmason via GitGitGadget, May 20, 2022
  16. 08/24 clone: --bundle-uri cannot be combined with --depthDerrick Stolee via GitGitGadget, May 20, 2022
  17. 13/24 bundle-uri: limit recursion depth for bundle listsDerrick Stolee via GitGitGadget, May 20, 2022
  18. 18/24 bundle-uri client: add "git ls-remote-bundle-uri"Ævar Arnfjörð Bjarmason via GitGitGadget, May 20, 2022
  19. 17/24 bundle-uri client: add minimal NOOP clientÆvar Arnfjörð Bjarmason via GitGitGadget, May 20, 2022
  20. 19/24 bundle-uri: serve URI advertisement from bundle.* configDerrick Stolee via GitGitGadget, May 20, 2022
  21. 22/24 bundle-uri: download bundles from an advertised listDerrick Stolee via GitGitGadget, May 20, 2022
  22. 24/24 t5601: basic bundle URI testsDerrick Stolee via GitGitGadget, May 20, 2022
  23. 20/24 bundle-uri client: add boolean transfer.bundleURI settingÆvar Arnfjörð Bjarmason via GitGitGadget, May 20, 2022
  24. 21/24 bundle-uri: allow relative URLs in bundle listsDerrick Stolee via GitGitGadget, May 20, 2022
  25. 23/24 clone: unbundle the advertised bundlesDerrick Stolee via GitGitGadget, May 20, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.