git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 3/3] repo-settings: pack.useSparse=true

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
Jul 1, 2019, 14:29 UTC
Message-ID
<d080065a9208852a7e551cc8bef7d326576c076d.1561991348.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.254.v3.git.gitgitgadget@gmail.com>
From: Derrick Stolee <dstolee@microsoft.com>

If a repo is large, then it probably has a very large working directory. In this case, a typical developer's edits usually impact many fewer paths than the full path set. The sparse treewalk algorithm is optimized for this case, speeding up 'git push' calls.

Use pack.useSparse=true when core.featureAdoptionRate is at least five. This is the first setting where the feature has only been out for a single major version. This could be moved to the "at least three" category after another major version.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 Documentation/config/core.txt | 9 +++++++++
 Documentation/config/pack.txt | 3 ++-
 builtin/pack-objects.c        | 9 +++++----
 repo-settings.c               | 8 ++++++++
 repo-settings.h               | 1 +
 5 files changed, 25 insertions(+), 5 deletions(-)
diff --git a/Documentation/config/core.txt b/Documentation/config/core.txt
index 865252aba9..60356102a8 100644
--- a/Documentation/config/core.txt
+++ b/Documentation/config/core.txt
@@ -624,3 +624,12 @@ The settings do not modify the user-facing output of porcelain commands.
 +
 * `index.version=4` uses prefix-compression to reduce the size of the
 .git/index file.
++
+If the value is at least 5, then all of the defaults above are included,
+plus the defaults below. These represent new features that present
+significant performance benefits, but may not have been released for
+multiple major versions.
++
+* `pack.useSparse=true` uses the sparse tree-walk algorithm, which is
+optimized for enumerating objects during linkgit:git-push[1] from a
+client machine.
diff --git a/Documentation/config/pack.txt b/Documentation/config/pack.txt
index 9cdcfa7324..9c4f8ea9ff 100644
--- a/Documentation/config/pack.txt
+++ b/Documentation/config/pack.txt
@@ -112,7 +112,8 @@ pack.useSparse::
 	objects. This can have significant performance benefits when
 	computing a pack to send a small change. However, it is possible
 	that extra objects are added to the pack-file if the included
-	commits contain certain types of direct renames.
+	commits contain certain types of direct renames. Defaults to
+	false, unless `core.featureAdoptionRate` is at least five.
 
 pack.writeBitmaps (deprecated)::
 	This is a deprecated synonym for `repack.writeBitmaps`.
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 41d7fc5983..f26b3f2892 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -34,6 +34,7 @@
 #include "dir.h"
 #include "midx.h"
 #include "trace2.h"
+#include "repo-settings.h"
 
 #define IN_PACK(obj) oe_in_pack(&to_pack, obj)
 #define SIZE(obj) oe_size(&to_pack, obj)
@@ -2707,10 +2708,6 @@ static int git_pack_config(const char *k, const char *v, void *cb)
 		use_bitmap_index_default = git_config_bool(k, v);
 		return 0;
 	}
-	if (!strcmp(k, "pack.usesparse")) {
-		sparse = git_config_bool(k, v);
-		return 0;
-	}
 	if (!strcmp(k, "pack.threads")) {
 		delta_search_threads = git_config_int(k, v);
 		if (delta_search_threads < 0)
@@ -3330,6 +3327,10 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 	read_replace_refs = 0;
 
 	sparse = git_env_bool("GIT_TEST_PACK_SPARSE", 0);
+	prepare_repo_settings(the_repository);
+	if (!sparse && the_repository->settings->pack_use_sparse != -1)
+		sparse = the_repository->settings->pack_use_sparse;
+
 	reset_pack_idx_option(&pack_idx_opts);
 	git_config(git_pack_config, NULL);
 
diff --git a/repo-settings.c b/repo-settings.c
index 5753153a84..c700edc286 100644
--- a/repo-settings.c
+++ b/repo-settings.c
@@ -16,6 +16,9 @@ static int git_repo_config(const char *key, const char *value, void *cb)
 			UPDATE_DEFAULT(rs->gc_write_commit_graph, 1);
 			UPDATE_DEFAULT(rs->index_version, 4);
 		}
+		if (rate >= 5) {
+			UPDATE_DEFAULT(rs->pack_use_sparse, 1);
+		}
 		return 0;
 	}
 	if (!strcmp(key, "core.commitgraph")) {
@@ -26,6 +29,10 @@ static int git_repo_config(const char *key, const char *value, void *cb)
 		rs->gc_write_commit_graph = git_config_bool(key, value);
 		return 0;
 	}
+	if (!strcmp(key, "pack.usesparse")) {
+		rs->pack_use_sparse = git_config_bool(key, value);
+		return 0;
+	}
 	if (!strcmp(key, "index.version")) {
 		rs->index_version = git_config_int(key, value);
 		return 0;
@@ -44,6 +51,7 @@ void prepare_repo_settings(struct repository *r)
 	/* Defaults */
 	r->settings->core_commit_graph = -1;
 	r->settings->gc_write_commit_graph = -1;
+	r->settings->pack_use_sparse = -1;
 	r->settings->index_version = -1;
 
 	repo_config(r, git_repo_config, r->settings);
diff --git a/repo-settings.h b/repo-settings.h
index b752dfe8b4..1151c2193a 100644
--- a/repo-settings.h
+++ b/repo-settings.h
@@ -4,6 +4,7 @@
 struct repo_settings {
 	int core_commit_graph;
 	int gc_write_commit_graph;
+	int pack_use_sparse;
 	int index_version;
 };
 
-- 
gitgitgadget
Previous: Jakub NarebskiNext: Derrick Stolee
Message 41 of 48 in “[RFC] Create 'core.size=large' setting to update config defaults”
  1. 00/11 [RFC] Create 'core.size=large' setting to update config defaultsDerrick Stolee via GitGitGadget, Jun 3, 2019
  2. 01/11 repo-settings: create repo.size=large settingDerrick Stolee via GitGitGadget, Jun 3, 2019
  3. Jeff HostetlerJun 3, 2019
  4. 02/11 repo-settings: use index.version=4 by defaultDerrick Stolee via GitGitGadget, Jun 3, 2019
  5. 05/11 status: add warning when a/b calculation takes too long for long/normal formatJeff Hostetler via GitGitGadget, Jun 3, 2019
  6. 09/11 fetch: warn about forced updates after branch listDerrick Stolee via GitGitGadget, Jun 3, 2019
  7. 07/11 repo-settings: status.aheadBehind=falseDerrick Stolee via GitGitGadget, Jun 3, 2019
  8. 11/11 repo-settings: fetch.showForcedUpdates=falseDerrick Stolee via GitGitGadget, Jun 3, 2019
  9. 10/11 pull: add --[no-]show-forced-updates passthrough to fetchDerrick Stolee via GitGitGadget, Jun 3, 2019
  10. 08/11 fetch: add --[no-]show-forced-updates argumentDerrick Stolee via GitGitGadget, Jun 3, 2019
  11. 03/11 repo-settings: pack.useSparse=trueDerrick Stolee via GitGitGadget, Jun 3, 2019
  12. 06/11 status: ignore status.aheadbehind in porcelain formatsJeff Hostetler via GitGitGadget, Jun 3, 2019
  13. 04/11 status: add status.aheadbehind settingJeff Hostetler via GitGitGadget, Jun 3, 2019
  14. Derrick StoleeJun 3, 2019
  15. Johannes SchindelinJun 4, 2019
  16. Derrick StoleeJun 4, 2019
  17. Junio C HamanoJun 5, 2019
  18. Derrick StoleeJun 6, 2019
  19. Junio C HamanoJun 6, 2019
  20. 0/3 [RFC] Create 'core.featureAdoptionRate' setting to update config defaultsDerrick Stolee via GitGitGadget, Jun 19, 2019
  21. 2/3 repo-settings: use index.version=4 by defaultDerrick Stolee via GitGitGadget, Jun 19, 2019
  22. 3/3 repo-settings: pack.useSparse=trueDerrick Stolee via GitGitGadget, Jun 19, 2019
  23. 1/3 repo-settings: create core.featureAdoptionRate settingDerrick Stolee via GitGitGadget, Jun 19, 2019
  24. Junio C HamanoJun 28, 2019
  25. Derrick StoleeJun 28, 2019
  26. Junio C HamanoJun 28, 2019
  27. Derrick StoleeJun 29, 2019
  28. Carlo ArenasJun 30, 2019
  29. Derrick StoleeJul 1, 2019
  30. Ævar Arnfjörð BjarmasonJul 2, 2019
  31. Duy NguyenJul 2, 2019
  32. Derrick StoleeJul 2, 2019
  33. Junio C HamanoJul 2, 2019
  34. 0/3 [RFC] Create 'core.featureAdoptionRate' setting to update config defaultsDerrick Stolee via GitGitGadget, Jul 1, 2019
  35. 2/3 repo-settings: use index.version=4 by defaultDerrick Stolee via GitGitGadget, Jul 1, 2019
  36. 1/3 repo-settings: create core.featureAdoptionRate settingDerrick Stolee via GitGitGadget, Jul 1, 2019
  37. Carlo ArenasJul 1, 2019
  38. Duy NguyenJul 2, 2019
  39. Ævar Arnfjörð BjarmasonJul 2, 2019
  40. Jakub NarebskiJul 4, 2019
  41. 3/3 repo-settings: pack.useSparse=trueDerrick Stolee via GitGitGadget, Jul 1, 2019
  42. Derrick StoleeJul 8, 2019
  43. Taylor BlauJul 9, 2019
  44. Junio C HamanoJul 9, 2019
  45. Derrick StoleeJul 9, 2019
  46. Junio C HamanoJul 9, 2019
  47. Derrick StoleeJul 22, 2019
  48. Jakub NarebskiJul 11, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.