{"thread":{"id":"55340","subject":"[PATCH] fetch: show progress for packfile uri downloads","startedAt":"2021-03-17T19:04:12Z","lastAt":"2021-06-30T23:09:10Z","messageCount":7,"participants":["Albert Cui via GitGitGadget","Jeff King","Junio C Hamano","Ævar Arnfjörð Bjarmason","Jonathan Nieder"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"419534","messageId":"pull.907.git.1616007794513.gitgitgadget@gmail.com","threadId":"55340","inReplyTo":null,"subject":"[PATCH] fetch: show progress for packfile uri downloads","fromName":"Albert Cui via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-03-17T19:03:13Z","receivedAt":"2021-03-17T19:04:12Z","isPatch":true,"sender":{"key":"albertqcui@gmail.com","avatar":"https://gravatar.com/avatar/8cff5e385bd4fdd4895efb8654e2ce82c8e680820acb817daaa1cd80e2fdb707?d=mp&s=160"},"body":"From: Albert Cui <albertqcui@gmail.com>\n\nGit appears to hang when downloading packfiles as this part of the\nfetch is silent, causing user confusion. This change implements\nprogress for the number of packfiles downloaded; a progress display\nfor bytes would involve deeper changes at the http-fetch layer\ninstead of fetch-pack, the caller.\n\nSigned-off-by: Albert Cui <albertqcui@gmail.com>\n---\n    fetch: show progress for packfile uri downloads\n    \n    Git appears to hang when downloading packfiles as this part of the fetch\n    is silent, causing user confusion. This change implements progress for\n    the number of packfiles downloaded; a progress display for bytes would\n    involve deeper changes at the http-fetch layer instead of fetch-pack,\n    the caller.\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-907%2Falbertcui%2Fprogress-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-907/albertcui/progress-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/907\n\n fetch-pack.c           | 8 ++++++++\n t/t5702-protocol-v2.sh | 5 ++++-\n 2 files changed, 12 insertions(+), 1 deletion(-)\n\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 6a61a464283e..5646d151293d 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -23,6 +23,7 @@\n #include \"fetch-negotiator.h\"\n #include \"fsck.h\"\n #include \"shallow.h\"\n+#include \"progress.h\"\n \n static int transfer_unpack_limit = -1;\n static int fetch_unpack_limit = -1;\n@@ -1585,6 +1586,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \tstruct fetch_negotiator *negotiator;\n \tint seen_ack = 0;\n \tstruct string_list packfile_uris = STRING_LIST_INIT_DUP;\n+\tstruct progress *packfile_uri_progress;\n \tint i;\n \tstruct strvec index_pack_args = STRVEC_INIT;\n \tstruct oidset gitmodules_oids = OIDSET_INIT;\n@@ -1689,6 +1691,8 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \t\t}\n \t}\n \n+\tpackfile_uri_progress = start_progress(_(\"Downloading packs\"), packfile_uris.nr);\n+\n \tfor (i = 0; i < packfile_uris.nr; i++) {\n \t\tint j;\n \t\tstruct child_process cmd = CHILD_PROCESS_INIT;\n@@ -1696,6 +1700,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \t\tconst char *uri = packfile_uris.items[i].string +\n \t\t\tthe_hash_algo->hexsz + 1;\n \n+\t\tdisplay_progress(packfile_uri_progress, i+1);\n \t\tstrvec_push(&cmd.args, \"http-fetch\");\n \t\tstrvec_pushf(&cmd.args, \"--packfile=%.*s\",\n \t\t\t     (int) the_hash_algo->hexsz,\n@@ -1739,6 +1744,9 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \t\t\t\t\t\t get_object_directory(),\n \t\t\t\t\t\t packname));\n \t}\n+\n+\tstop_progress(&packfile_uri_progress);\n+\n \tstring_list_clear(&packfile_uris, 0);\n \tstrvec_clear(&index_pack_args);\n \ndiff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh\nindex 2e1243ca40b0..8964a4003678 100755\n--- a/t/t5702-protocol-v2.sh\n+++ b/t/t5702-protocol-v2.sh\n@@ -851,7 +851,8 @@ test_expect_success 'part of packfile response provided as URI' '\n \tGIT_TRACE=1 GIT_TRACE_PACKET=\"$(pwd)/log\" GIT_TEST_SIDEBAND_ALL=1 \\\n \tgit -c protocol.version=2 \\\n \t\t-c fetch.uriprotocols=http,https \\\n-\t\tclone \"$HTTPD_URL/smart/http_parent\" http_child &&\n+\t\tclone \"$HTTPD_URL/smart/http_parent\" http_child \\\n+\t\t--progress 2>progress &&\n \n \t# Ensure that my-blob and other-blob are in separate packfiles.\n \tfor idx in http_child/.git/objects/pack/*.idx\n@@ -875,6 +876,8 @@ test_expect_success 'part of packfile response provided as URI' '\n \ttest -f hfound &&\n \ttest -f h2found &&\n \n+\ttest_i18ngrep \"Downloading packs\" progress &&\n+\n \t# Ensure that there are exactly 3 packfiles with associated .idx\n \tls http_child/.git/objects/pack/*.pack \\\n \t    http_child/.git/objects/pack/*.idx >filelist &&\n\nbase-commit: a5828ae6b52137b913b978e16cd2334482eb4c1f\n-- \ngitgitgadget\n"},{"id":"419539","messageId":"YFJWz5yIGng+a16k@coredump.intra.peff.net","threadId":"55340","inReplyTo":"pull.907.git.1616007794513.gitgitgadget@gmail.com","subject":"Re: [PATCH] fetch: show progress for packfile uri downloads","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-03-17T19:21:51Z","receivedAt":"2021-03-17T19:22:46Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Mar 17, 2021 at 07:03:13PM +0000, Albert Cui via GitGitGadget wrote:\n\n> From: Albert Cui <albertqcui@gmail.com>\n> \n> Git appears to hang when downloading packfiles as this part of the\n> fetch is silent, causing user confusion. This change implements\n> progress for the number of packfiles downloaded; a progress display\n> for bytes would involve deeper changes at the http-fetch layer\n> instead of fetch-pack, the caller.\n\nI think this is an improvement, but I agree that a real byte display\nwould be much better. I actually worked on this a long time ago for a\nvery similar feature that we never quite pushed over the finish line.\nSee patches 11 and 12 here:\n\n  https://lore.kernel.org/git/20111110074330.GA27925@sigill.intra.peff.net/\n\n(it might need some of the earlier refactoring, too, I'm not sure; and\nquite likely will need forward-porting as it has been 10 years).\n\n-Peff\n"},{"id":"419540","messageId":"xmqqsg4ttw5g.fsf@gitster.g","threadId":"55340","inReplyTo":"pull.907.git.1616007794513.gitgitgadget@gmail.com","subject":"Re: [PATCH] fetch: show progress for packfile uri downloads","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-17T19:31:07Z","receivedAt":"2021-03-17T19:32:13Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Albert Cui via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n\n> From: Albert Cui <albertqcui@gmail.com>\n>\n> Git appears to hang when downloading packfiles as this part of the\n> fetch is silent, causing user confusion. This change implements\n> progress for the number of packfiles downloaded; a progress display\n> for bytes would involve deeper changes at the http-fetch layer\n> instead of fetch-pack, the caller.\n\n... \"hence we do not do so in this patch\"?  \n\nThat's probably a very sensible way to go.\n\nI expect that http-fetch will in the longer term become a mere\nfallback default used by those who do not have anything better.\nBecause we are not in the business of writing a performant HTTP\ndownloader, we would be better off if we make it easy to plug an\nexternal HTTP downloader other people write in to this codepath.\n\n> +\tpackfile_uri_progress = start_progress(_(\"Downloading packs\"), packfile_uris.nr);\n\nOK, so we plan to count from 0 up to .nr; and the message is made\nlocalizable.  Good.\n\n> @@ -1696,6 +1700,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n>  \t\tconst char *uri = packfile_uris.items[i].string +\n>  \t\t\tthe_hash_algo->hexsz + 1;\n>  \n> +\t\tdisplay_progress(packfile_uri_progress, i+1);\n\n\t\tdisplay_progress(packfile_uri_progress, i + 1);\n\n> diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh\n> index 2e1243ca40b0..8964a4003678 100755\n> --- a/t/t5702-protocol-v2.sh\n> +++ b/t/t5702-protocol-v2.sh\n> @@ -851,7 +851,8 @@ test_expect_success 'part of packfile response provided as URI' '\n>  \tGIT_TRACE=1 GIT_TRACE_PACKET=\"$(pwd)/log\" GIT_TEST_SIDEBAND_ALL=1 \\\n>  \tgit -c protocol.version=2 \\\n>  \t\t-c fetch.uriprotocols=http,https \\\n> -\t\tclone \"$HTTPD_URL/smart/http_parent\" http_child &&\n> +\t\tclone \"$HTTPD_URL/smart/http_parent\" http_child \\\n> +\t\t--progress 2>progress &&\n\nSome existing tests use GIT_PROGRESS_DELAY to protect against an\noperation that is too quick to complete.  Don't we need to do the\nsame?  If not, then perhaps we need to allow delaying the progress\nmeter we add with this patch for \"too quick\" case, perhaps?\n\n>  \t# Ensure that my-blob and other-blob are in separate packfiles.\n>  \tfor idx in http_child/.git/objects/pack/*.idx\n> @@ -875,6 +876,8 @@ test_expect_success 'part of packfile response provided as URI' '\n>  \ttest -f hfound &&\n>  \ttest -f h2found &&\n>  \n> +\ttest_i18ngrep \"Downloading packs\" progress &&\n\nAlso, I am not sure with all the terminal control junk, 'grep'\nshould be expected to reliably pick this substring in the output.\nAre we expecting any other output to the standard error stream?\nSome tests in t5318 seem to just see if the output is non-empty, and\nI am wondering if that is an approach more appropriate here (not\nrhetorical---I simply do not know the answer).\n\n>  \t# Ensure that there are exactly 3 packfiles with associated .idx\n>  \tls http_child/.git/objects/pack/*.pack \\\n>  \t    http_child/.git/objects/pack/*.idx >filelist &&\n>\n> base-commit: a5828ae6b52137b913b978e16cd2334482eb4c1f\n"},{"id":"421492","messageId":"pull.907.v2.git.1618008249632.gitgitgadget@gmail.com","threadId":"55340","inReplyTo":"pull.907.git.1616007794513.gitgitgadget@gmail.com","subject":"[PATCH v2] fetch: show progress for packfile uri downloads","fromName":"Albert Cui via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-04-09T22:44:09Z","receivedAt":"2021-04-09T22:44:15Z","isPatch":true,"sender":{"key":"albertqcui@gmail.com","avatar":"https://gravatar.com/avatar/8cff5e385bd4fdd4895efb8654e2ce82c8e680820acb817daaa1cd80e2fdb707?d=mp&s=160"},"body":"From: Albert Cui <albertqcui@gmail.com>\n\nGit appears to hang when downloading packfiles as this part of the\nfetch is silent, causing user confusion. This change implements\nprogress for the number of packfiles downloaded; a progress display\nfor bytes would involve deeper changes at the http-fetch layer\ninstead of fetch-pack, the caller, so we do not do that in this\npatch.\n\nSigned-off-by: Albert Cui <albertqcui@gmail.com>\n---\n    fetch: show progress for packfile uri downloads\n    \n    Git appears to hang when downloading packfiles as this part of the fetch\n    is silent, causing user confusion. This change implements progress for\n    the number of packfiles downloaded; a progress display for bytes would\n    involve deeper changes at the http-fetch layer instead of fetch-pack,\n    the caller.\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-907%2Falbertcui%2Fprogress-v2\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-907/albertcui/progress-v2\nPull-Request: https://github.com/gitgitgadget/git/pull/907\n\nRange-diff vs v1:\n\n 1:  d6cf8c195111 ! 1:  aedb1b33d0c4 fetch: show progress for packfile uri downloads\n     @@ Commit message\n          fetch is silent, causing user confusion. This change implements\n          progress for the number of packfiles downloaded; a progress display\n          for bytes would involve deeper changes at the http-fetch layer\n     -    instead of fetch-pack, the caller.\n     +    instead of fetch-pack, the caller, so we do not do that in this\n     +    patch.\n      \n          Signed-off-by: Albert Cui <albertqcui@gmail.com>\n      \n     @@ fetch-pack.c: static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n       \t\tconst char *uri = packfile_uris.items[i].string +\n       \t\t\tthe_hash_algo->hexsz + 1;\n       \n     -+\t\tdisplay_progress(packfile_uri_progress, i+1);\n     ++\t\tdisplay_progress(packfile_uri_progress, i + 1);\n       \t\tstrvec_push(&cmd.args, \"http-fetch\");\n       \t\tstrvec_pushf(&cmd.args, \"--packfile=%.*s\",\n       \t\t\t     (int) the_hash_algo->hexsz,\n     @@ fetch-pack.c: static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n      \n       ## t/t5702-protocol-v2.sh ##\n      @@ t/t5702-protocol-v2.sh: test_expect_success 'part of packfile response provided as URI' '\n     - \tGIT_TRACE=1 GIT_TRACE_PACKET=\"$(pwd)/log\" GIT_TEST_SIDEBAND_ALL=1 \\\n     + \tconfigure_exclusion \"$P\" my-blob >h &&\n     + \tconfigure_exclusion \"$P\" other-blob >h2 &&\n     + \n     +-\tGIT_TRACE=1 GIT_TRACE_PACKET=\"$(pwd)/log\" GIT_TEST_SIDEBAND_ALL=1 \\\n     ++\tGIT_PROGRESS_DELAY=0 GIT_TRACE=1 GIT_TRACE2_EVENT=1 \\\n     ++\tGIT_TRACE_PACKET=\"$(pwd)/log\" GIT_TEST_SIDEBAND_ALL=1 \\\n       \tgit -c protocol.version=2 \\\n       \t\t-c fetch.uriprotocols=http,https \\\n      -\t\tclone \"$HTTPD_URL/smart/http_parent\" http_child &&\n\n\n fetch-pack.c           | 8 ++++++++\n t/t5702-protocol-v2.sh | 8 ++++++--\n 2 files changed, 14 insertions(+), 2 deletions(-)\n\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 6a61a464283e..7ca85f40cf23 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -23,6 +23,7 @@\n #include \"fetch-negotiator.h\"\n #include \"fsck.h\"\n #include \"shallow.h\"\n+#include \"progress.h\"\n \n static int transfer_unpack_limit = -1;\n static int fetch_unpack_limit = -1;\n@@ -1585,6 +1586,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \tstruct fetch_negotiator *negotiator;\n \tint seen_ack = 0;\n \tstruct string_list packfile_uris = STRING_LIST_INIT_DUP;\n+\tstruct progress *packfile_uri_progress;\n \tint i;\n \tstruct strvec index_pack_args = STRVEC_INIT;\n \tstruct oidset gitmodules_oids = OIDSET_INIT;\n@@ -1689,6 +1691,8 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \t\t}\n \t}\n \n+\tpackfile_uri_progress = start_progress(_(\"Downloading packs\"), packfile_uris.nr);\n+\n \tfor (i = 0; i < packfile_uris.nr; i++) {\n \t\tint j;\n \t\tstruct child_process cmd = CHILD_PROCESS_INIT;\n@@ -1696,6 +1700,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \t\tconst char *uri = packfile_uris.items[i].string +\n \t\t\tthe_hash_algo->hexsz + 1;\n \n+\t\tdisplay_progress(packfile_uri_progress, i + 1);\n \t\tstrvec_push(&cmd.args, \"http-fetch\");\n \t\tstrvec_pushf(&cmd.args, \"--packfile=%.*s\",\n \t\t\t     (int) the_hash_algo->hexsz,\n@@ -1739,6 +1744,9 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \t\t\t\t\t\t get_object_directory(),\n \t\t\t\t\t\t packname));\n \t}\n+\n+\tstop_progress(&packfile_uri_progress);\n+\n \tstring_list_clear(&packfile_uris, 0);\n \tstrvec_clear(&index_pack_args);\n \ndiff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh\nindex 2e1243ca40b0..0476b3f50455 100755\n--- a/t/t5702-protocol-v2.sh\n+++ b/t/t5702-protocol-v2.sh\n@@ -848,10 +848,12 @@ test_expect_success 'part of packfile response provided as URI' '\n \tconfigure_exclusion \"$P\" my-blob >h &&\n \tconfigure_exclusion \"$P\" other-blob >h2 &&\n \n-\tGIT_TRACE=1 GIT_TRACE_PACKET=\"$(pwd)/log\" GIT_TEST_SIDEBAND_ALL=1 \\\n+\tGIT_PROGRESS_DELAY=0 GIT_TRACE=1 GIT_TRACE2_EVENT=1 \\\n+\tGIT_TRACE_PACKET=\"$(pwd)/log\" GIT_TEST_SIDEBAND_ALL=1 \\\n \tgit -c protocol.version=2 \\\n \t\t-c fetch.uriprotocols=http,https \\\n-\t\tclone \"$HTTPD_URL/smart/http_parent\" http_child &&\n+\t\tclone \"$HTTPD_URL/smart/http_parent\" http_child \\\n+\t\t--progress 2>progress &&\n \n \t# Ensure that my-blob and other-blob are in separate packfiles.\n \tfor idx in http_child/.git/objects/pack/*.idx\n@@ -875,6 +877,8 @@ test_expect_success 'part of packfile response provided as URI' '\n \ttest -f hfound &&\n \ttest -f h2found &&\n \n+\ttest_i18ngrep \"Downloading packs\" progress &&\n+\n \t# Ensure that there are exactly 3 packfiles with associated .idx\n \tls http_child/.git/objects/pack/*.pack \\\n \t    http_child/.git/objects/pack/*.idx >filelist &&\n\nbase-commit: a5828ae6b52137b913b978e16cd2334482eb4c1f\n-- \ngitgitgadget\n"},{"id":"421523","messageId":"877dlah76e.fsf@evledraar.gmail.com","threadId":"55340","inReplyTo":"pull.907.git.1616007794513.gitgitgadget@gmail.com","subject":"Re: [PATCH] fetch: show progress for packfile uri downloads","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-04-10T08:31:37Z","receivedAt":"2021-04-10T08:31:48Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Mar 17 2021, Albert Cui via GitGitGadget wrote:\n\n> From: Albert Cui <albertqcui@gmail.com>\n> [...]\n> @@ -875,6 +876,8 @@ test_expect_success 'part of packfile response provided as URI' '\n>  \ttest -f hfound &&\n>  \ttest -f h2found &&\n>  \n> +\ttest_i18ngrep \"Downloading packs\" progress &&\n\nThis can just be \"grep\"\n"},{"id":"421561","messageId":"xmqqy2dpip4h.fsf@gitster.g","threadId":"55340","inReplyTo":"pull.907.v2.git.1618008249632.gitgitgadget@gmail.com","subject":"Re: [PATCH v2] fetch: show progress for packfile uri downloads","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-04-11T01:30:54Z","receivedAt":"2021-04-11T01:31:01Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Albert Cui via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n\n> From: Albert Cui <albertqcui@gmail.com>\n>\n> Git appears to hang when downloading packfiles as this part of the\n> fetch is silent, causing user confusion. This change implements\n> progress for the number of packfiles downloaded; a progress display\n> for bytes would involve deeper changes at the http-fetch layer\n> instead of fetch-pack, the caller, so we do not do that in this\n> patch.\n\n... meaning, hopefully later we'd hook into transport->progress and\nimplement the byte-level progress display down there?  And when that\nhappens, we'd remove this file-level progress as it would be too\nconfusing to have both at the same time?\n\nIs this start_progress() call a way to unconditionally enable the\nprogress display?  How does it interact with transport->progress\nthat is driven by transport_set_verbosity(), which in turn is called\nby builtin/fetch.c and friends?  If it doesn't, shouldn't this\ncodepath pay attention to the transport->progress and enable the\nprogress meter only when it is enabled (i.e. the stderr going to a\nterminal, or --progress explicitly being asked)?\n\n> @@ -1585,6 +1586,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n>  \tstruct fetch_negotiator *negotiator;\n>  \tint seen_ack = 0;\n>  \tstruct string_list packfile_uris = STRING_LIST_INIT_DUP;\n> +\tstruct progress *packfile_uri_progress;\n>  \tint i;\n>  \tstruct strvec index_pack_args = STRVEC_INIT;\n>  \tstruct oidset gitmodules_oids = OIDSET_INIT;\n> @@ -1689,6 +1691,8 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n>  \t\t}\n>  \t}\n>  \n> +\tpackfile_uri_progress = start_progress(_(\"Downloading packs\"), packfile_uris.nr);\n> +\n>  \tfor (i = 0; i < packfile_uris.nr; i++) {\n>  \t\tint j;\n>  \t\tstruct child_process cmd = CHILD_PROCESS_INIT;\n> @@ -1696,6 +1700,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n>  \t\tconst char *uri = packfile_uris.items[i].string +\n>  \t\t\tthe_hash_algo->hexsz + 1;\n>  \n> +\t\tdisplay_progress(packfile_uri_progress, i + 1);\n>  \t\tstrvec_push(&cmd.args, \"http-fetch\");\n>  \t\tstrvec_pushf(&cmd.args, \"--packfile=%.*s\",\n>  \t\t\t     (int) the_hash_algo->hexsz,\n> @@ -1739,6 +1744,9 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n>  \t\t\t\t\t\t get_object_directory(),\n>  \t\t\t\t\t\t packname));\n>  \t}\n> +\n> +\tstop_progress(&packfile_uri_progress);\n> +\n>  \tstring_list_clear(&packfile_uris, 0);\n>  \tstrvec_clear(&index_pack_args);\n>  \n> diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh\n> index 2e1243ca40b0..0476b3f50455 100755\n> --- a/t/t5702-protocol-v2.sh\n> +++ b/t/t5702-protocol-v2.sh\n> @@ -848,10 +848,12 @@ test_expect_success 'part of packfile response provided as URI' '\n>  \tconfigure_exclusion \"$P\" my-blob >h &&\n>  \tconfigure_exclusion \"$P\" other-blob >h2 &&\n>  \n> -\tGIT_TRACE=1 GIT_TRACE_PACKET=\"$(pwd)/log\" GIT_TEST_SIDEBAND_ALL=1 \\\n> +\tGIT_PROGRESS_DELAY=0 GIT_TRACE=1 GIT_TRACE2_EVENT=1 \\\n> +\tGIT_TRACE_PACKET=\"$(pwd)/log\" GIT_TEST_SIDEBAND_ALL=1 \\\n>  \tgit -c protocol.version=2 \\\n>  \t\t-c fetch.uriprotocols=http,https \\\n> -\t\tclone \"$HTTPD_URL/smart/http_parent\" http_child &&\n> +\t\tclone \"$HTTPD_URL/smart/http_parent\" http_child \\\n> +\t\t--progress 2>progress &&\n>  \n>  \t# Ensure that my-blob and other-blob are in separate packfiles.\n>  \tfor idx in http_child/.git/objects/pack/*.idx\n> @@ -875,6 +877,8 @@ test_expect_success 'part of packfile response provided as URI' '\n>  \ttest -f hfound &&\n>  \ttest -f h2found &&\n>  \n> +\ttest_i18ngrep \"Downloading packs\" progress &&\n> +\n>  \t# Ensure that there are exactly 3 packfiles with associated .idx\n>  \tls http_child/.git/objects/pack/*.pack \\\n>  \t    http_child/.git/objects/pack/*.idx >filelist &&\n>\n> base-commit: a5828ae6b52137b913b978e16cd2334482eb4c1f\n"},{"id":"428860","messageId":"YNz5kEs4ivfJdhP3@google.com","threadId":"55340","inReplyTo":"pull.907.v2.git.1618008249632.gitgitgadget@gmail.com","subject":"Re: [PATCH v2] fetch: show progress for packfile uri downloads","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2021-06-30T23:09:04Z","receivedAt":"2021-06-30T23:09:10Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nAlbert Cui wrote:\n\n> Git appears to hang when downloading packfiles as this part of the\n> fetch is silent, causing user confusion. This change implements\n> progress for the number of packfiles downloaded; a progress display\n> for bytes would involve deeper changes at the http-fetch layer\n> instead of fetch-pack, the caller, so we do not do that in this\n> patch.\n>\n> Signed-off-by: Albert Cui <albertqcui@gmail.com>\n> ---\n>  fetch-pack.c           | 8 ++++++++\n>  t/t5702-protocol-v2.sh | 8 ++++++--\n>  2 files changed, 14 insertions(+), 2 deletions(-)\n\nThis is something that came up at the last in-person Git Contributor\nSummit; I'm glad to see it being taken care of.\n\n> --- a/fetch-pack.c\n> +++ b/fetch-pack.c\n> @@ -23,6 +23,7 @@\n>  #include \"fetch-negotiator.h\"\n>  #include \"fsck.h\"\n>  #include \"shallow.h\"\n> +#include \"progress.h\"\n>  \n>  static int transfer_unpack_limit = -1;\n>  static int fetch_unpack_limit = -1;\n> @@ -1585,6 +1586,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n>  \tstruct fetch_negotiator *negotiator;\n>  \tint seen_ack = 0;\n>  \tstruct string_list packfile_uris = STRING_LIST_INIT_DUP;\n> +\tstruct progress *packfile_uri_progress;\n\nIt seems to be more idiomatic to initialize this to NULL.\n\n>  \tint i;\n>  \tstruct strvec index_pack_args = STRVEC_INIT;\n>  \tstruct oidset gitmodules_oids = OIDSET_INIT;\n> @@ -1689,6 +1691,8 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n>  \t\t}\n>  \t}\n>  \n> +\tpackfile_uri_progress = start_progress(_(\"Downloading packs\"), packfile_uris.nr);\n\nThat way, we can respect a --quiet option by making this remain NULL\nwhen progress is not enabled:\n\n\tif (!args->quiet && !args->no_progress)\n\t\tpackfile_uri_progress = ...;\n\n[...]\n> --- a/t/t5702-protocol-v2.sh\n> +++ b/t/t5702-protocol-v2.sh\n> @@ -848,10 +848,12 @@ test_expect_success 'part of packfile response provided as URI' '\n>  \tconfigure_exclusion \"$P\" my-blob >h &&\n>  \tconfigure_exclusion \"$P\" other-blob >h2 &&\n>  \n> -\tGIT_TRACE=1 GIT_TRACE_PACKET=\"$(pwd)/log\" GIT_TEST_SIDEBAND_ALL=1 \\\n> +\tGIT_PROGRESS_DELAY=0 GIT_TRACE=1 GIT_TRACE2_EVENT=1 \\\n\nThis puts the trace in stderr mixed with other output.  Would it make\nsense to put it in a separate file, like this?\n\n\tGIT_TRACE=1 GIT_TRACE_PACKET=\"$(pwd)/log\" \\\n\tGIT_PROGRESS_DELAY=0 GIT_TRACE2_EVENT=\"$(pwd)/trace2\" \\\n\tGIT_TEST_SIDEBAND_ALL=1 \\\n\tgit -c [etc]\n\n[...]\n> @@ -875,6 +877,8 @@ test_expect_success 'part of packfile response provided as URI' '\n>  \ttest -f hfound &&\n>  \ttest -f h2found &&\n>  \n> +\ttest_i18ngrep \"Downloading packs\" progress &&\n\nThat way, this \"grep\" could check the trace2 file which would contain\noutput intended for machines, and we wouldn't have to worry e.g. about\nANSII control codes potentially affecting the output around the space\nsome day in the progress output intended for a terminal.\n\nWith whatever subset of the changes described above make sense, this is\nReviewed-by: Jonathan Nieder <jrnieder@gmail.com>\n\nThanks.\n"}]}