git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 25/30] pack-objects: enable --path-walk via config

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
Sep 10, 2024, 02:28 UTC
Message-ID
<1942f7d03622f2740d83e766fca65938cb590f6a.1725935335.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.1786.git.1725935335.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>

Users may want to enable the --path-walk option for 'git pack-objects' by default, especially underneath commands like 'git push' or 'git repack'.

This should be limited to client repositories, since the --path-walk option disables bitmap walks, so would be bad to include in Git servers when serving fetches and clones. There is potential that it may be helpful to consider when repacking the repository, to take advantage of improved deltas across historical versions of the same files.

Much like how "pack.useSparse" was introduced and included in "feature.experimental" before being enabled by default, use the repository settings infrastructure to make the new "pack.usePathWalk" config enabled by "feature.experimental" and "feature.manyFiles".

Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
 Documentation/config/pack.txt | 8 ++++++++
 builtin/pack-objects.c        | 4 +++-
 repo-settings.c               | 3 +++
 repository.h                  | 1 +
 4 files changed, 15 insertions(+), 1 deletion(-)
diff --git a/Documentation/config/pack.txt b/Documentation/config/pack.txt
index da527377faf..08d06271177 100644
--- a/Documentation/config/pack.txt
+++ b/Documentation/config/pack.txt
@@ -155,6 +155,14 @@ pack.useSparse::
 	commits contain certain types of direct renames. Default is
 	`true`.
 
+pack.usePathWalk::
+	When true, git will default to using the '--path-walk' option in
+	'git pack-objects' when the '--revs' option is present. This
+	algorithm groups objects by path to maximize the ability to
+	compute delta chains across historical versions of the same
+	object. This may disable other options, such as using bitmaps to
+	enumerate objects.
+
 pack.preferBitmapTips::
 	When selecting which commits will receive bitmaps, prefer a
 	commit at the tip of any reference that is a suffix of any value
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index b9fe1b1fbd5..e7a9d0349c3 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -4534,12 +4534,14 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 
 	disable_replace_refs();
 
-	path_walk = git_env_bool("GIT_TEST_PACK_PATH_WALK", 0);
+	path_walk = git_env_bool("GIT_TEST_PACK_PATH_WALK", -1);
 	sparse = git_env_bool("GIT_TEST_PACK_SPARSE", -1);
 	if (the_repository->gitdir) {
 		prepare_repo_settings(the_repository);
 		if (sparse < 0)
 			sparse = the_repository->settings.pack_use_sparse;
+		if (path_walk < 0)
+			path_walk = the_repository->settings.pack_use_path_walk;
 		if (the_repository->settings.pack_use_multi_pack_reuse)
 			allow_pack_reuse = MULTI_PACK_REUSE;
 	}
diff --git a/repo-settings.c b/repo-settings.c
index 2b4e68731be..d9597d84556 100644
--- a/repo-settings.c
+++ b/repo-settings.c
@@ -45,11 +45,13 @@ void prepare_repo_settings(struct repository *r)
 		r->settings.fetch_negotiation_algorithm = FETCH_NEGOTIATION_SKIPPING;
 		r->settings.pack_use_bitmap_boundary_traversal = 1;
 		r->settings.pack_use_multi_pack_reuse = 1;
+		r->settings.pack_use_path_walk = 1;
 	}
 	if (manyfiles) {
 		r->settings.index_version = 4;
 		r->settings.index_skip_hash = 1;
 		r->settings.core_untracked_cache = UNTRACKED_CACHE_WRITE;
+		r->settings.pack_use_path_walk = 1;
 	}
 
 	/* Commit graph config or default, does not cascade (simple) */
@@ -64,6 +66,7 @@ void prepare_repo_settings(struct repository *r)
 
 	/* Boolean config or default, does not cascade (simple)  */
 	repo_cfg_bool(r, "pack.usesparse", &r->settings.pack_use_sparse, 1);
+	repo_cfg_bool(r, "pack.usepathwalk", &r->settings.pack_use_path_walk, 0);
 	repo_cfg_bool(r, "core.multipackindex", &r->settings.core_multi_pack_index, 1);
 	repo_cfg_bool(r, "index.sparse", &r->settings.sparse_index, 0);
 	repo_cfg_bool(r, "index.skiphash", &r->settings.index_skip_hash, r->settings.index_skip_hash);
diff --git a/repository.h b/repository.h
index af6ea0a62cd..2ae9c2b1741 100644
--- a/repository.h
+++ b/repository.h
@@ -62,6 +62,7 @@ struct repo_settings {
 	enum untracked_cache_setting core_untracked_cache;
 
 	int pack_use_sparse;
+	int pack_use_path_walk;
 	enum fetch_negotiation_setting fetch_negotiation_algorithm;
 
 	int core_multi_pack_index;
-- 
gitgitgadget
Previous: Derrick Stolee via GitGitGadgetNext: Derrick Stolee via GitGitGadget
Message 26 of 38 in “[RFC] Path-walk API and applications”
  1. 00/30 [RFC] Path-walk API and applicationsDerrick Stolee via GitGitGadget, Sep 10, 2024
  2. 01/30 path-walk: introduce an object walk by pathDerrick Stolee via GitGitGadget, Sep 10, 2024
  3. 02/30 backfill: add builtin boilerplateDerrick Stolee via GitGitGadget, Sep 10, 2024
  4. 03/30 backfill: basic functionality and testsDerrick Stolee via GitGitGadget, Sep 10, 2024
  5. 04/30 backfill: add --batch-size=<n> optionDerrick Stolee via GitGitGadget, Sep 10, 2024
  6. 06/30 backfill: assume --sparse when sparse-checkout is enabledDerrick Stolee via GitGitGadget, Sep 10, 2024
  7. 05/30 backfill: add --sparse optionDerrick Stolee via GitGitGadget, Sep 10, 2024
  8. 07/30 path-walk: allow consumer to specify object typesDerrick Stolee via GitGitGadget, Sep 10, 2024
  9. 08/30 path-walk: allow visiting tagsDerrick Stolee via GitGitGadget, Sep 10, 2024
  10. 09/30 survey: stub in new experimental `git-survey` commandJeff Hostetler via GitGitGadget, Sep 10, 2024
  11. 10/30 survey: add command line opts to select referencesJeff Hostetler via GitGitGadget, Sep 10, 2024
  12. 11/30 survey: collect the set of requested refsJeff Hostetler via GitGitGadget, Sep 10, 2024
  13. 12/30 survey: start pretty printing data in table formDerrick Stolee via GitGitGadget, Sep 10, 2024
  14. 13/30 survey: add object count summaryDerrick Stolee via GitGitGadget, Sep 10, 2024
  15. 14/30 survey: summarize total sizes by object typeDerrick Stolee via GitGitGadget, Sep 10, 2024
  16. 15/30 survey: show progress during object walkDerrick Stolee via GitGitGadget, Sep 10, 2024
  17. 16/30 survey: add ability to track prioritized listsDerrick Stolee via GitGitGadget, Sep 10, 2024
  18. 17/30 survey: add report of "largest" pathsDerrick Stolee via GitGitGadget, Sep 10, 2024
  19. 18/30 revision: create mark_trees_uninteresting_dense()Derrick Stolee via GitGitGadget, Sep 10, 2024
  20. 19/30 path-walk: add prune_all_uninteresting optionDerrick Stolee via GitGitGadget, Sep 10, 2024
  21. 20/30 pack-objects: add --path-walk optionDerrick Stolee via GitGitGadget, Sep 10, 2024
  22. 21/30 pack-objects: extract should_attempt_deltas()Derrick Stolee via GitGitGadget, Sep 10, 2024
  23. 22/30 pack-objects: introduce GIT_TEST_PACK_PATH_WALKDerrick Stolee via GitGitGadget, Sep 10, 2024
  24. 23/30 p5313: add size comparison testDerrick Stolee via GitGitGadget, Sep 10, 2024
  25. 24/30 repack: add --path-walk optionDerrick Stolee via GitGitGadget, Sep 10, 2024
  26. 25/30 pack-objects: enable --path-walk via configDerrick Stolee via GitGitGadget, Sep 10, 2024
  27. 26/30 scalar: enable path-walk during push via configDerrick Stolee via GitGitGadget, Sep 10, 2024
  28. 27/30 pack-objects: add --full-name-hash optionDerrick Stolee via GitGitGadget, Sep 10, 2024
  29. 28/30 test-name-hash: add helper to compute name-hash functionsDerrick Stolee via GitGitGadget, Sep 10, 2024
  30. 29/30 p5314: add a size test for name-hash collisionsDerrick Stolee via GitGitGadget, Sep 10, 2024
  31. 30/30 pack-objects: output debug info about deltasDerrick Stolee via GitGitGadget, Sep 10, 2024
  32. Junio C HamanoSep 11, 2024
  33. Christian CouderSep 17, 2024
  34. Derrick StoleeSep 18, 2024
  35. Junio C HamanoSep 22, 2024
  36. Derrick StoleeSep 23, 2024
  37. Junio C HamanoSep 23, 2024
  38. Kristoffer HaugsbakkSep 22, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.