git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[RFC PATCH 03/15] Read sparse limiting args from $GIT_DIR/sparse-limit

From
Elijah Newren <newren@gmail.com>
Date
Sep 5, 2010, 00:13 UTC
Message-ID
<1283645647-1891-4-git-send-email-newren@gmail.com>
In-Reply-To
<1283645647-1891-1-git-send-email-newren@gmail.com>

Sparse clones will be created by specifying rev-list args to clone in order to limit what is downloaded. In order to avoid accessing unavailable objects, these limiting rev-list args are recorded in $GIT_DIR/sparse-limit so that other operations can make use of them.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Makefile      |    2 +
 cache.h       |    3 ++
 environment.c |    2 +
 setup.c       |    2 +
 sparse-repo.c |   64 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 sparse-repo.h |    3 ++
 tree-diff.c   |    2 +
 7 files changed, 78 insertions(+), 0 deletions(-)
 create mode 100644 sparse-repo.c
 create mode 100644 sparse-repo.h
diff --git a/Makefile b/Makefile
index d3db459..ba4b9bf 100644
--- a/Makefile
+++ b/Makefile
@@ -548,6 +548,7 @@ LIB_H += run-command.h
 LIB_H += sha1-lookup.h
 LIB_H += sideband.h
 LIB_H += sigchain.h
+LIB_H += sparse-repo.h
 LIB_H += strbuf.h
 LIB_H += string-list.h
 LIB_H += submodule.h
@@ -657,6 +658,7 @@ LIB_OBJS += sha1_name.o
 LIB_OBJS += shallow.o
 LIB_OBJS += sideband.o
 LIB_OBJS += sigchain.o
+LIB_OBJS += sparse-repo.o
 LIB_OBJS += strbuf.o
 LIB_OBJS += string-list.o
 LIB_OBJS += submodule.o
diff --git a/cache.h b/cache.h
index ce7bdd1..6f88dbb 100644
--- a/cache.h
+++ b/cache.h
@@ -403,6 +403,9 @@ extern const char *const local_repo_env[LOCAL_REPO_ENV_SIZE + 1];
 extern int is_bare_repository_cfg;
 extern int is_bare_repository(void);
 extern int is_inside_git_dir(void);
+extern const char **git_sparse_limits;
+extern const char **git_sparse_pathspecs;
+extern struct diff_options git_sparse_diffopts;
 extern char *git_work_tree_cfg;
 extern int is_inside_work_tree(void);
 extern int have_git_dir(void);
diff --git a/environment.c b/environment.c
index cc097b7..fc9d21d 100644
--- a/environment.c
+++ b/environment.c
@@ -61,6 +61,8 @@ struct startup_info *startup_info;
 int core_preload_index = 0;
 
 /* This is set by setup_git_dir_gently() and/or git_default_config() */
+const char **git_sparse_limits = NULL;
+const char **git_sparse_pathspecs = NULL;
 char *git_work_tree_cfg;
 static char *work_tree;
 
diff --git a/setup.c b/setup.c
index a3b76de..92f4385 100644
--- a/setup.c
+++ b/setup.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "dir.h"
+#include "sparse-repo.h"
 
 static int inside_git_dir = -1;
 static int inside_work_tree = -1;
@@ -256,6 +257,7 @@ static int check_repository_format_gently(int *nongit_ok)
 		*nongit_ok = -1;
 		return -1;
 	}
+	check_sparse_repo();
 	return 0;
 }
 
diff --git a/sparse-repo.c b/sparse-repo.c
new file mode 100644
index 0000000..18ea976
--- /dev/null
+++ b/sparse-repo.c
@@ -0,0 +1,64 @@
+#include "sparse-repo.h"
+#include "cache.h"
+#include "diff.h"
+#include "quote.h"
+
+#define SPARSE_FILE "sparse-limits"
+
+void register_sparse_limits(const char **revlist_argv)
+{
+	if (revlist_argv != git_sparse_limits) {
+		free(git_sparse_limits);
+		git_sparse_limits = revlist_argv;
+	}
+
+	git_sparse_pathspecs = git_sparse_limits;
+	for (; *git_sparse_pathspecs; git_sparse_pathspecs++)
+		/*
+		 * Sparse limiting args must always have a double dash before
+		 * any paths, since users will likely fetch/pull from bare
+		 * repositories.  That makes it easy to find the pathspec
+		 * portion of the sparse limits.
+		 */
+		if (strcmp(*git_sparse_pathspecs, "--") == 0) {
+			/*
+			 * Make git_sparse_limits provide quick access
+			 * to non-pathspec rev-list arguments, and
+			 * git_sparse_pathspec provide access to the
+			 * rest.
+			 */
+			*git_sparse_pathspecs = NULL;
+			git_sparse_pathspecs++;
+			break;
+		}
+	diff_tree_setup_paths(git_sparse_pathspecs, &git_sparse_diffopts);
+}
+
+void register_quoted_sparse_limits(char *quoted_argv)
+{
+	int nr = 0, alloc = 0;
+
+	sq_dequote_to_argv(strdup(quoted_argv), &git_sparse_limits, &nr, &alloc);
+	ALLOC_GROW(git_sparse_limits, nr+1, alloc);
+	git_sparse_limits[nr] = NULL;
+
+	register_sparse_limits(git_sparse_limits);
+}
+
+void check_sparse_repo(void)
+{
+	struct stat statbuf;
+	struct strbuf buffer = STRBUF_INIT;
+
+	char *sparse_file = git_path(SPARSE_FILE);
+	if (stat(sparse_file, &statbuf))
+		return;
+
+	if (git_sparse_limits)
+		die("tried to initialize sparse limits twice!\n");
+
+	if (strbuf_read_file(&buffer, sparse_file, statbuf.st_size) < 0)
+		die_errno("could not read %s", sparse_file);
+	register_quoted_sparse_limits(buffer.buf);
+	strbuf_release(&buffer);
+}
diff --git a/sparse-repo.h b/sparse-repo.h
new file mode 100644
index 0000000..ed9018a
--- /dev/null
+++ b/sparse-repo.h
@@ -0,0 +1,3 @@
+void register_sparse_limits(const char **revlist_argv);
+void register_quoted_sparse_limits(char *quoted_argv);
+void check_sparse_repo(void);
diff --git a/tree-diff.c b/tree-diff.c
index d976bdf..951b53b 100644
--- a/tree-diff.c
+++ b/tree-diff.c
@@ -6,6 +6,8 @@
 #include "diffcore.h"
 #include "tree.h"
 
+struct diff_options git_sparse_diffopts;
+
 static char *malloc_base(const char *base, int baselen, const char *path, int pathlen)
 {
 	char *newbase = xmalloc(baselen + pathlen + 2);
-- 
1.7.2.2.140.gd06af
Previous: Elijah NewrenNext: Elijah Newren
Message 7 of 41 in “Sparse clones”
  1. 00/15 Sparse clonesElijah Newren, Sep 5, 2010
  2. 01/15 README-sparse-clone: Add a basic writeup of my ideas for sparse clonesElijah Newren, Sep 5, 2010
  3. Nguyen Thai Ngoc DuySep 5, 2010
  4. Elijah NewrenSep 5, 2010
  5. Nguyen Thai Ngoc DuySep 6, 2010
  6. 02/15 Add tests for client handling in a sparse repositoryElijah Newren, Sep 5, 2010
  7. 03/15 Read sparse limiting args from $GIT_DIR/sparse-limitElijah Newren, Sep 5, 2010
  8. 04/15 When unpacking in a sparse repository, avoid traversing missing trees/blobsElijah Newren, Sep 5, 2010
  9. 05/15 read_tree_recursive: Avoid missing blobs and trees in a sparse repositoryElijah Newren, Sep 5, 2010
  10. Nguyen Thai Ngoc DuySep 5, 2010
  11. Elijah NewrenSep 5, 2010
  12. Elijah NewrenSep 5, 2010
  13. 06/15 Automatically reuse sparse limiting arguments in revision walkingElijah Newren, Sep 5, 2010
  14. Nguyen Thai Ngoc DuySep 5, 2010
  15. Elijah NewrenSep 5, 2010
  16. Nguyen Thai Ngoc DuySep 5, 2010
  17. 07/15 cache_tree_update(): Capability to handle tree entries missing from indexElijah Newren, Sep 5, 2010
  18. Nguyen Thai Ngoc DuySep 5, 2010
  19. Elijah NewrenSep 5, 2010
  20. Elijah NewrenSep 6, 2010
  21. Nguyen Thai Ngoc DuySep 6, 2010
  22. 0/4 en/object-list-with-pathspec updateNguyễn Thái Ngọc Duy, Sep 6, 2010
  23. 1/4 Add testcases showing how pathspecs are ignored with rev-list --objectsNguyễn Thái Ngọc Duy, Sep 6, 2010
  24. 2/4 tree-walk: copy tree_entry_interesting() as is from tree-diff.cNguyễn Thái Ngọc Duy, Sep 6, 2010
  25. Elijah NewrenSep 6, 2010
  26. Nguyen Thai Ngoc DuySep 6, 2010
  27. 3/4 tree-walk: actually move tree_entry_interesting() to tree-walk.cNguyễn Thái Ngọc Duy, Sep 6, 2010
  28. Elijah NewrenSep 6, 2010
  29. Nguyen Thai Ngoc DuySep 6, 2010
  30. Junio C HamanoSep 6, 2010
  31. 4/4 Make rev-list --objects work together with pathspecsNguyễn Thái Ngọc Duy, Sep 6, 2010
  32. Nguyen Thai Ngoc DuySep 7, 2010
  33. Elijah NewrenSep 7, 2010
  34. 08/15 cache_tree_update(): Require relevant tree to be passedElijah Newren, Sep 5, 2010
  35. 09/15 Add tests for communication dealing with sparse repositoriesElijah Newren, Sep 5, 2010
  36. 10/15 sparse-repo: Provide a function to record sparse limiting argumentsElijah Newren, Sep 5, 2010
  37. 11/15 builtin-clone: Accept paths for sparse cloneElijah Newren, Sep 5, 2010
  38. 12/15 Pass extra (rev-list) args on, at least in some casesElijah Newren, Sep 5, 2010
  39. 13/15 upload-pack: Handle extra rev-list arguments being passedElijah Newren, Sep 5, 2010
  40. 14/15 EVIL COMMIT: Include all commitsElijah Newren, Sep 5, 2010
  41. 15/15 clone: Ensure sparse limiting arguments are used in subsequent operationsElijah Newren, Sep 5, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.