git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3] send-pack.c: add config push.useBitmaps

From
Kyle Zhao via GitGitGadget <gitgitgadget@gmail.com>
Date
Jun 17, 2022, 03:59 UTC
Message-ID
<pull.1263.v3.git.1655438361228.gitgitgadget@gmail.com>
In-Reply-To
<pull.1263.v2.git.1655350617442.gitgitgadget@gmail.com>
From: Kyle Zhao <kylezhao@tencent.com>
This allows you to disable bitmaps for "git push". Default is true.

Reachability bitmaps are designed to speed up the "counting objects" phase of generating a pack during a clone or fetch. They are not optimized for Git clients sending a small topic branch via "git push". In some cases (see [1]), using reachability bitmaps during "git push" can cause significant performance regressions.

Add a new "push.useBitmaps" config option to disable reachability bitmaps during "git push". This allows reachability bitmaps to still be used in other areas, such as "git rev-list --use-bitmap-index".

[1]: https://lore.kernel.org/git/87zhoz8b9o.fsf@evledraar.gmail.com/
Signed-off-by: Kyle Zhao <kylezhao@tencent.com>
---
    send-pack.c: add config push.useBitmaps
    
    This patch add config push.useBitmaps to prevent git push using bitmap.
    
    The origin discussion is here:
    https://lore.kernel.org/git/b940e705fbe9454685757f2e3055e2ce@tencent.com/
    
    Thanks, -Kyle
    
    Changes since v1:
    
     * changed the commit message
     * modified and added missing \n to push.txt
     * used test_subcommand for test
     * modified "if" statement for "git_config_get_bool()" in send-pack.c
    
    Changes since v2:
    
     * enable 'push.useBitmaps" by default
     * fix nit in t/t5516-fetch-push.sh
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1263%2Fkeyu98%2Fkz%2Fpush-usebitmps-config-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1263/keyu98/kz/push-usebitmps-config-v3
Pull-Request: https://github.com/gitgitgadget/git/pull/1263
Range-diff vs v2:
 1:  42e0b4845b2 ! 1:  a523cb52542 send-pack.c: add config push.useBitmaps
     @@ Metadata
       ## Commit message ##
          send-pack.c: add config push.useBitmaps
      
     -    This allows you to disable bitmaps for "git push". Default is false.
     +    This allows you to disable bitmaps for "git push". Default is true.
      
          Reachability bitmaps are designed to speed up the "counting objects"
          phase of generating a pack during a clone or fetch. They are not
     @@ Documentation/config/push.txt: push.negotiate::
       	in common.
      +
      +push.useBitmaps::
     -+    If this config and `pack.useBitmaps` are both `true`, then Git will
     -+    use reachability bitmaps during `git push`, if available (disabled
     -+    by default).
     ++	If this config and `pack.useBitmaps` are both `true`, then Git will
     ++	use reachability bitmaps during `git push`, if available. If set to
     ++	`false`, may improve push performance without affecting use of the
     ++	reachability bitmaps for other operations. Default is true.
      
       ## send-pack.c ##
      @@ send-pack.c: static int pack_objects(int fd, struct ref *refs, struct oid_array *advertised,
       		strvec_push(&po.args, "--progress");
       	if (is_repository_shallow(the_repository))
       		strvec_push(&po.args, "--shallow");
     -+	if (!args->use_bitmaps)
     ++	if (args->no_use_bitmaps)
      +		strvec_push(&po.args, "--no-use-bitmap-index");
       	po.in = -1;
       	po.out = args->stateless_rpc ? -1 : fd;
     @@ send-pack.c: int send_pack(struct send_pack_args *args,
       		get_commons_through_negotiation(args->url, remote_refs, &commons);
       
      +	if (!git_config_get_bool("push.usebitmaps", &use_bitmaps))
     -+		args->use_bitmaps = use_bitmaps;
     ++		args->no_use_bitmaps = !use_bitmaps;
      +
       	git_config_get_bool("transfer.advertisesid", &advertise_sid);
       
     @@ send-pack.h: struct send_pack_args {
       		stateless_rpc:1,
      -		atomic:1;
      +		atomic:1,
     -+		use_bitmaps:1;
     ++		no_use_bitmaps:1;
       	const struct string_list *push_options;
       };
       
     @@ t/t5516-fetch-push.sh: test_expect_success 'push warns or fails when using usern
      +	GIT_TRACE2_EVENT="$PWD/default" \
      +	git push testrepo main:test &&
      +	test_subcommand git pack-objects --all-progress-implied --revs --stdout \
     -+	--thin --delta-base-offset -q --no-use-bitmap-index <default &&
     ++		--thin --delta-base-offset -q <default &&
      +
      +	git config push.useBitmaps true &&
      +	GIT_TRACE2_EVENT="$PWD/true" \
      +	git push testrepo main:test2 &&
      +	test_subcommand git pack-objects --all-progress-implied --revs --stdout \
     -+	--thin --delta-base-offset -q <true &&
     ++		--thin --delta-base-offset -q <true &&
      +
      +	git config push.useBitmaps false &&
      +	GIT_TRACE2_EVENT="$PWD/false" \
      +	git push testrepo main:test3 &&
      +	test_subcommand git pack-objects --all-progress-implied --revs --stdout \
     -+	--thin --delta-base-offset -q --no-use-bitmap-index <false
     ++		--thin --delta-base-offset -q --no-use-bitmap-index <false
      +'
      +
       test_done
 Documentation/config/push.txt |  6 ++++++
 send-pack.c                   |  6 ++++++
 send-pack.h                   |  3 ++-
 t/t5516-fetch-push.sh         | 21 +++++++++++++++++++++
 4 files changed, 35 insertions(+), 1 deletion(-)
diff --git a/Documentation/config/push.txt b/Documentation/config/push.txt
index e32801e6c91..f16ac9311db 100644
--- a/Documentation/config/push.txt
+++ b/Documentation/config/push.txt
@@ -137,3 +137,9 @@ push.negotiate::
 	server attempt to find commits in common. If "false", Git will
 	rely solely on the server's ref advertisement to find commits
 	in common.
+
+push.useBitmaps::
+	If this config and `pack.useBitmaps` are both `true`, then Git will
+	use reachability bitmaps during `git push`, if available. If set to
+	`false`, may improve push performance without affecting use of the
+	reachability bitmaps for other operations. Default is true.
diff --git a/send-pack.c b/send-pack.c
index bc0fcdbb000..d18cde850ef 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -84,6 +84,8 @@ static int pack_objects(int fd, struct ref *refs, struct oid_array *advertised,
 		strvec_push(&po.args, "--progress");
 	if (is_repository_shallow(the_repository))
 		strvec_push(&po.args, "--shallow");
+	if (args->no_use_bitmaps)
+		strvec_push(&po.args, "--no-use-bitmap-index");
 	po.in = -1;
 	po.out = args->stateless_rpc ? -1 : fd;
 	po.git_cmd = 1;
@@ -487,6 +489,7 @@ int send_pack(struct send_pack_args *args,
 	struct async demux;
 	const char *push_cert_nonce = NULL;
 	struct packet_reader reader;
+	int use_bitmaps;
 
 	if (!remote_refs) {
 		fprintf(stderr, "No refs in common and none specified; doing nothing.\n"
@@ -498,6 +501,9 @@ int send_pack(struct send_pack_args *args,
 	if (push_negotiate)
 		get_commons_through_negotiation(args->url, remote_refs, &commons);
 
+	if (!git_config_get_bool("push.usebitmaps", &use_bitmaps))
+		args->no_use_bitmaps = !use_bitmaps;
+
 	git_config_get_bool("transfer.advertisesid", &advertise_sid);
 
 	/* Does the other end support the reporting? */
diff --git a/send-pack.h b/send-pack.h
index e148fcd9609..d5d6ff589d9 100644
--- a/send-pack.h
+++ b/send-pack.h
@@ -26,7 +26,8 @@ struct send_pack_args {
 		/* One of the SEND_PACK_PUSH_CERT_* constants. */
 		push_cert:2,
 		stateless_rpc:1,
-		atomic:1;
+		atomic:1,
+		no_use_bitmaps:1;
 	const struct string_list *push_options;
 };
 
diff --git a/t/t5516-fetch-push.sh b/t/t5516-fetch-push.sh
index dedca106a7a..ffda830ba22 100755
--- a/t/t5516-fetch-push.sh
+++ b/t/t5516-fetch-push.sh
@@ -1865,4 +1865,25 @@ test_expect_success 'push warns or fails when using username:password' '
 	test_line_count = 1 warnings
 '
 
+test_expect_success 'push with config push.useBitmaps' '
+	mk_test testrepo heads/main &&
+	git checkout main &&
+	GIT_TRACE2_EVENT="$PWD/default" \
+	git push testrepo main:test &&
+	test_subcommand git pack-objects --all-progress-implied --revs --stdout \
+		--thin --delta-base-offset -q <default &&
+
+	git config push.useBitmaps true &&
+	GIT_TRACE2_EVENT="$PWD/true" \
+	git push testrepo main:test2 &&
+	test_subcommand git pack-objects --all-progress-implied --revs --stdout \
+		--thin --delta-base-offset -q <true &&
+
+	git config push.useBitmaps false &&
+	GIT_TRACE2_EVENT="$PWD/false" \
+	git push testrepo main:test3 &&
+	test_subcommand git pack-objects --all-progress-implied --revs --stdout \
+		--thin --delta-base-offset -q --no-use-bitmap-index <false
+'
+
 test_done

base-commit: 8168d5e9c23ed44ae3d604f392320d66556453c9
-- 
gitgitgadget
Previous: Jeff KingNext: Junio C Hamano
Message 14 of 17 in “send-pack.c: add config push.useBitmaps”
  1. send-pack.c: add config push.useBitmapsKyle Zhao via GitGitGadget, Jun 15, 2022
  2. Derrick StoleeJun 15, 2022
  3. Taylor BlauJun 15, 2022
  4. Ævar Arnfjörð BjarmasonJun 15, 2022
  5. Taylor BlauJun 15, 2022
  6. kylezhao (赵柯宇)Jun 16, 2022
  7. send-pack.c: add config push.useBitmapsKyle Zhao via GitGitGadget, Jun 16, 2022
  8. Derrick StoleeJun 16, 2022
  9. Ævar Arnfjörð BjarmasonJun 16, 2022
  10. kylezhao (赵柯宇)Jun 16, 2022
  11. Taylor BlauJun 16, 2022
  12. Junio C HamanoJun 16, 2022
  13. Jeff KingJun 16, 2022
  14. send-pack.c: add config push.useBitmapsKyle Zhao via GitGitGadget, Jun 17, 2022
  15. Junio C HamanoJun 17, 2022
  16. send-pack.c: add config push.useBitmapsKyle Zhao via GitGitGadget, Jun 17, 2022
  17. Junio C HamanoJun 17, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.