git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 5/6] pack-objects: create pack.useSparse setting

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
Dec 10, 2018, 16:42 UTC
Message-ID
<33d2c04dd696d4a7f648343f3da5e48a9e1f1b5b.1544460159.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.89.v3.git.gitgitgadget@gmail.com>
From: Derrick Stolee <dstolee@microsoft.com>

The '--sparse' flag in 'git pack-objects' changes the algorithm used to enumerate objects to one that is faster for individual users pushing new objects that change only a small cone of the working directory. The sparse algorithm is not recommended for a server, which likely sends new objects that appear across the entire working directory.

Create a 'pack.useSparse' setting that enables this new algorithm. This allows 'git push' to use this algorithm without passing a '--sparse' flag all the way through four levels of run_command() calls.

If the '--no-sparse' flag is set, then this config setting is overridden.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 Documentation/config/pack.txt  |  9 +++++++++
 builtin/pack-objects.c         |  4 ++++
 t/t5322-pack-objects-sparse.sh | 15 +++++++++++++++
 3 files changed, 28 insertions(+)
diff --git a/Documentation/config/pack.txt b/Documentation/config/pack.txt
index edac75c83f..425c73aa52 100644
--- a/Documentation/config/pack.txt
+++ b/Documentation/config/pack.txt
@@ -105,6 +105,15 @@ pack.useBitmaps::
 	true. You should not generally need to turn this off unless
 	you are debugging pack bitmaps.
 
+pack.useSparse::
+	When true, git will default to using the '--sparse' option in
+	'git pack-objects' when the '--revs' option is present. This
+	algorithm only walks trees that appear in paths that introduce new
+	objects. This can have significant performance benefits when
+	computing a pack to send a small change. However, it is possible
+	that extra objects are added to the pack-file if the included
+	commits contain certain types of direct renames.
+
 pack.writeBitmaps (deprecated)::
 	This is a deprecated synonym for `repack.writeBitmaps`.
 
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 7d5b0735e3..124b1bafc4 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -2711,6 +2711,10 @@ static int git_pack_config(const char *k, const char *v, void *cb)
 		use_bitmap_index_default = git_config_bool(k, v);
 		return 0;
 	}
+	if (!strcmp(k, "pack.usesparse")) {
+		sparse = git_config_bool(k, v);
+		return 0;
+	}
 	if (!strcmp(k, "pack.threads")) {
 		delta_search_threads = git_config_int(k, v);
 		if (delta_search_threads < 0)
diff --git a/t/t5322-pack-objects-sparse.sh b/t/t5322-pack-objects-sparse.sh
index 45dba6e014..8f5699bd91 100755
--- a/t/t5322-pack-objects-sparse.sh
+++ b/t/t5322-pack-objects-sparse.sh
@@ -121,4 +121,19 @@ test_expect_success 'sparse pack-objects' '
 	test_cmp expect_sparse_objects.txt sparse_objects.txt
 '
 
+test_expect_success 'pack.useSparse enables algorithm' '
+	git config pack.useSparse true &&
+	git pack-objects --stdout --revs <packinput.txt >sparse.pack &&
+	git index-pack -o sparse.idx sparse.pack &&
+	git show-index <sparse.idx | awk "{print \$2}" >sparse_objects.txt &&
+	test_cmp expect_sparse_objects.txt sparse_objects.txt
+'
+
+test_expect_success 'pack.useSparse overridden' '
+	git pack-objects --stdout --revs --no-sparse <packinput.txt >sparse.pack &&
+	git index-pack -o sparse.idx sparse.pack &&
+	git show-index <sparse.idx | awk "{print \$2}" >sparse_objects.txt &&
+	test_cmp expect_objects.txt sparse_objects.txt
+'
+
 test_done
-- 
gitgitgadget
Previous: Derrick Stolee via GitGitGadgetNext: Derrick Stolee via GitGitGadget
Message 26 of 51 in “Add a new "sparse" tree walk algorithm”
  1. 0/5 Add a new "sparse" tree walk algorithmDerrick Stolee via GitGitGadget, Nov 28, 2018
  2. 1/5 revision: add mark_tree_uninteresting_sparseDerrick Stolee via GitGitGadget, Nov 28, 2018
  3. 2/5 list-objects: consume sparse tree walkDerrick Stolee via GitGitGadget, Nov 28, 2018
  4. 3/5 pack-objects: add --sparse optionDerrick Stolee via GitGitGadget, Nov 28, 2018
  5. Stefan BellerNov 28, 2018
  6. Derrick StoleeNov 29, 2018
  7. Junio C HamanoNov 30, 2018
  8. Derrick StoleeNov 30, 2018
  9. 5/5 pack-objects: create pack.useSparse settingDerrick Stolee via GitGitGadget, Nov 28, 2018
  10. 4/5 revision: implement sparse algorithmDerrick Stolee via GitGitGadget, Nov 28, 2018
  11. Ævar Arnfjörð BjarmasonNov 28, 2018
  12. Derrick StoleeNov 29, 2018
  13. 0/6 Add a new "sparse" tree walk algorithmDerrick Stolee via GitGitGadget, Nov 29, 2018
  14. 3/6 pack-objects: add --sparse optionDerrick Stolee via GitGitGadget, Nov 29, 2018
  15. 4/6 revision: implement sparse algorithmDerrick Stolee via GitGitGadget, Nov 29, 2018
  16. 6/6 pack-objects: create GIT_TEST_PACK_SPARSEDerrick Stolee via GitGitGadget, Nov 29, 2018
  17. 5/6 pack-objects: create pack.useSparse settingDerrick Stolee via GitGitGadget, Nov 29, 2018
  18. 2/6 list-objects: consume sparse tree walkDerrick Stolee via GitGitGadget, Nov 29, 2018
  19. 1/6 revision: add mark_tree_uninteresting_sparseDerrick Stolee via GitGitGadget, Nov 29, 2018
  20. 0/6 Add a new "sparse" tree walk algorithmDerrick Stolee via GitGitGadget, Dec 10, 2018
  21. 1/6 revision: add mark_tree_uninteresting_sparseDerrick Stolee via GitGitGadget, Dec 10, 2018
  22. 3/6 pack-objects: add --sparse optionDerrick Stolee via GitGitGadget, Dec 10, 2018
  23. 2/6 list-objects: consume sparse tree walkDerrick Stolee via GitGitGadget, Dec 10, 2018
  24. 4/6 revision: implement sparse algorithmDerrick Stolee via GitGitGadget, Dec 10, 2018
  25. 6/6 pack-objects: create GIT_TEST_PACK_SPARSEDerrick Stolee via GitGitGadget, Dec 10, 2018
  26. 5/6 pack-objects: create pack.useSparse settingDerrick Stolee via GitGitGadget, Dec 10, 2018
  27. 0/6 Add a new "sparse" tree walk algorithmDerrick Stolee via GitGitGadget, Dec 14, 2018
  28. 1/6 revision: add mark_tree_uninteresting_sparseDerrick Stolee via GitGitGadget, Dec 14, 2018
  29. Junio C HamanoJan 11, 2019
  30. Junio C HamanoJan 11, 2019
  31. Derrick StoleeJan 11, 2019
  32. 2/6 list-objects: consume sparse tree walkDerrick Stolee via GitGitGadget, Dec 14, 2018
  33. Junio C HamanoJan 11, 2019
  34. 3/6 pack-objects: add --sparse optionDerrick Stolee via GitGitGadget, Dec 14, 2018
  35. Junio C HamanoJan 11, 2019
  36. Derrick StoleeJan 15, 2019
  37. Junio C HamanoJan 15, 2019
  38. 4/6 revision: implement sparse algorithmDerrick Stolee via GitGitGadget, Dec 14, 2018
  39. Ævar Arnfjörð BjarmasonDec 14, 2018
  40. Derrick StoleeDec 17, 2018
  41. Ævar Arnfjörð BjarmasonDec 17, 2018
  42. Derrick StoleeDec 17, 2018
  43. Junio C HamanoJan 11, 2019
  44. 5/6 pack-objects: create pack.useSparse settingDerrick Stolee via GitGitGadget, Dec 14, 2018
  45. 6/6 pack-objects: create GIT_TEST_PACK_SPARSEDerrick Stolee via GitGitGadget, Dec 14, 2018
  46. 0/5 Add a new "sparse" tree walk algorithmDerrick Stolee via GitGitGadget, Jan 16, 2019
  47. 2/5 list-objects: consume sparse tree walkDerrick Stolee via GitGitGadget, Jan 16, 2019
  48. 3/5 revision: implement sparse algorithmDerrick Stolee via GitGitGadget, Jan 16, 2019
  49. 4/5 pack-objects: create pack.useSparse settingDerrick Stolee via GitGitGadget, Jan 16, 2019
  50. 5/5 pack-objects: create GIT_TEST_PACK_SPARSEDerrick Stolee via GitGitGadget, Jan 16, 2019
  51. 1/5 revision: add mark_tree_uninteresting_sparseDerrick Stolee via GitGitGadget, Jan 16, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.