{"thread":{"id":"54923","subject":"[PATCH] bundle: arguments can be read from stdin","startedAt":"2021-01-03T09:56:17Z","lastAt":"2021-06-21T09:31:01Z","messageCount":60,"participants":["Jiang Xin","Junio C Hamano","Đoàn Trần Công Danh","Ævar Arnfjörð Bjarmason","Jeff King","Felipe Contreras","Nicolas Pitre"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"413334","messageId":"20210103095457.6894-1-worldhello.net@gmail.com","threadId":"54923","inReplyTo":null,"subject":"[PATCH] bundle: arguments can be read from stdin","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-01-03T09:54:57Z","receivedAt":"2021-01-03T09:56:17Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"From: Jiang Xin \n\nIn order to create an incremental bundle, we need to pass many arguments\nto let git-bundle ignore some already packed commits. It will be more\nconvenient to pass args via stdin. But the current implementation does\nnot allow us to do this.\n\nThis is because args are parsed twice when creating bundle. The first\ntime for parsing args is in `compute_and_write_prerequisites()` by\nrunning `git-rev-list` command to write prerequisites in bundle file,\nand stdin is consumed in this step if \"--stdin\" option is provided for\n`git-bundle`. Later nothing can be read from stdin when running\n`setup_revisions()` in `create_bundle()`.\n\nRemove the entire `compute_and_write_prerequisites()` function, and\nparse the args once by `setup_revisions()`. The first step for creating\na bundle is to write prerequisites (\"-\" obj-id SP comment LF), but after\ncalling `prepare_revision_walk()`, the `revs.pending` is left empty.\nFollowing steps could not work properly without data in `revs.pending`.\nTherefore, before calling `prepare_revision_walk()` function, make a\ncopy on `revs.pending` for later use.\n\nSigned-off-by: Jiang Xin \n---\n bundle.c | 109 ++++++++++++++++++++++------------------\n t/t5607-clone-bundle.sh | 4 +-\n 2 files changed, 61 insertions(+), 52 deletions(-)\n\ndiff --git a/bundle.c b/bundle.c\nindex cb0e5931ac..693d619551 100644\n--- a/bundle.c\n+++ b/bundle.c\n@@ -338,48 +338,6 @@ static int write_pack_data(int bundle_fd, struct rev_info *revs, struct strvec *\n \treturn 0;\n }\n \n-static int compute_and_write_prerequisites(int bundle_fd,\n-\t\t\t\t\t struct rev_info *revs,\n-\t\t\t\t\t int argc, const char **argv)\n-{\n-\tstruct child_process rls = CHILD_PROCESS_INIT;\n-\tstruct strbuf buf = STRBUF_INIT;\n-\tFILE *rls_fout;\n-\tint i;\n-\n-\tstrvec_pushl(&rls.args,\n-\t\t \"rev-list\", \"--boundary\", \"--pretty=oneline\",\n-\t\t NULL);\n-\tfor (i = 1; i < argc; i++)\n-\t\tstrvec_push(&rls.args, argv[i]);\n-\trls.out = -1;\n-\trls.git_cmd = 1;\n-\tif (start_command(&rls))\n-\t\treturn -1;\n-\trls_fout = xfdopen(rls.out, \"r\");\n-\twhile (strbuf_getwholeline(&buf, rls_fout, '\\n') != EOF) {\n-\t\tstruct object_id oid;\n-\t\tif (buf.len > 0 && buf.buf[0] == '-') {\n-\t\t\twrite_or_die(bundle_fd, buf.buf, buf.len);\n-\t\t\tif (!get_oid_hex(buf.buf + 1, &oid)) {\n-\t\t\t\tstruct object *object = parse_object_or_die(&oid,\n-\t\t\t\t\t\t\t\t\t buf.buf);\n-\t\t\t\tobject->flags |= UNINTERESTING;\n-\t\t\t\tadd_pending_object(revs, object, buf.buf);\n-\t\t\t}\n-\t\t} else if (!get_oid_hex(buf.buf, &oid)) {\n-\t\t\tstruct object *object = parse_object_or_die(&oid,\n-\t\t\t\t\t\t\t\t buf.buf);\n-\t\t\tobject->flags |= SHOWN;\n-\t\t}\n-\t}\n-\tstrbuf_release(&buf);\n-\tfclose(rls_fout);\n-\tif (finish_command(&rls))\n-\t\treturn error(_(\"rev-list died\"));\n-\treturn 0;\n-}\n-\n /*\n * Write out bundle refs based on the tips already\n * parsed into revs.pending. As a side effect, may\n@@ -474,6 +432,38 @@ static int write_bundle_refs(int bundle_fd, struct rev_info *revs)\n \treturn ref_count;\n }\n \n+struct bundle_prerequisites_info {\n+\tstruct object_array *pending;\n+\tint fd;\n+};\n+\n+static void write_bundle_prerequisites(struct commit *commit, void *data)\n+{\n+\tstruct bundle_prerequisites_info *bpi = data;\n+\tstruct object *object;\n+\tstruct pretty_print_context ctx = { 0 };\n+\tstruct strbuf buf = STRBUF_INIT;\n+\n+\tif (!(commit->object.flags & BOUNDARY))\n+\t\treturn;\n+\tstrbuf_addf(&buf, \"-%s \", oid_to_hex(&commit->object.oid));\n+\twrite_or_die(bpi->fd, buf.buf, buf.len);\n+\n+\tctx.fmt = CMIT_FMT_ONELINE;\n+\tctx.output_encoding = get_log_output_encoding();\n+\tstrbuf_reset(&buf);\n+\tpretty_print_commit(&ctx, commit, &buf);\n+\tstrbuf_trim(&buf);\n+\n+\tobject = (struct object *)commit;\n+\tobject->flags |= UNINTERESTING;\n+\tadd_object_array_with_path(object, buf.buf, bpi->pending, S_IFINVALID,\n+\t\t\t\t NULL);\n+\tstrbuf_addch(&buf, '\\n');\n+\twrite_or_die(bpi->fd, buf.buf, buf.len);\n+\tstrbuf_release(&buf);\n+}\n+\n int create_bundle(struct repository *r, const char *path,\n \t\t int argc, const char **argv, struct strvec *pack_options, int version)\n {\n@@ -481,8 +471,10 @@ int create_bundle(struct repository *r, const char *path,\n \tint bundle_fd = -1;\n \tint bundle_to_stdout;\n \tint ref_count = 0;\n-\tstruct rev_info revs;\n+\tstruct rev_info revs, revs_copy;\n \tint min_version = the_hash_algo == &hash_algos[GIT_HASH_SHA1] ? 2 : 3;\n+\tstruct bundle_prerequisites_info bpi;\n+\tint i;\n \n \tbundle_to_stdout = !strcmp(path, \"-\");\n \tif (bundle_to_stdout)\n@@ -512,10 +504,6 @@ int create_bundle(struct repository *r, const char *path,\n \tsave_commit_buffer = 0;\n \trepo_init_revisions(r, &revs, NULL);\n \n-\t/* write prerequisites */\n-\tif (compute_and_write_prerequisites(bundle_fd, &revs, argc, argv))\n-\t\tgoto err;\n-\n \targc = setup_revisions(argc, argv, &revs, NULL);\n \n \tif (argc > 1) {\n@@ -523,16 +511,37 @@ int create_bundle(struct repository *r, const char *path,\n \t\tgoto err;\n \t}\n \n-\tobject_array_remove_duplicates(&revs.pending);\n+\t/* save revs.pending in revs_copy for later use */\n+\tmemcpy(&revs_copy, &revs, sizeof(revs));\n+\trevs_copy.pending.nr = 0;\n+\trevs_copy.pending.alloc = 0;\n+\trevs_copy.pending.objects = NULL;\n+\tfor (i = 0; i < revs.pending.nr; i++) {\n+\t\tstruct object_array_entry *e = revs.pending.objects + i;\n+\t\tif (e)\n+\t\t\tadd_object_array_with_path(e->item, e->name,\n+\t\t\t\t\t\t &revs_copy.pending,\n+\t\t\t\t\t\t e->mode, e->path);\n+\t}\n \n-\tref_count = write_bundle_refs(bundle_fd, &revs);\n+\t/* write prerequisites */\n+\trevs.boundary = 1;\n+\tif (prepare_revision_walk(&revs))\n+\t\tdie(\"revision walk setup failed\");\n+\tbpi.fd = bundle_fd;\n+\tbpi.pending = &revs_copy.pending;\n+\ttraverse_commit_list(&revs, write_bundle_prerequisites, NULL, &bpi);\n+\tobject_array_remove_duplicates(&revs_copy.pending);\n+\n+\t/* write bundle refs */\n+\tref_count = write_bundle_refs(bundle_fd, &revs_copy);\n \tif (!ref_count)\n \t\tdie(_(\"Refusing to create empty bundle.\"));\n \telse if (ref_count < 0)\n \t\tgoto err;\n \n \t/* write pack */\n-\tif (write_pack_data(bundle_fd, &revs, pack_options))\n+\tif (write_pack_data(bundle_fd, &revs_copy, pack_options))\n \t\tgoto err;\n \n \tif (!bundle_to_stdout) {\ndiff --git a/t/t5607-clone-bundle.sh b/t/t5607-clone-bundle.sh\nindex 26985f4b44..425258767d 100755\n--- a/t/t5607-clone-bundle.sh\n+++ b/t/t5607-clone-bundle.sh\n@@ -38,13 +38,13 @@ test_expect_success 'die if bundle file cannot be created' '\n \ttest_must_fail git bundle create adir --all\n '\n \n-test_expect_failure 'bundle --stdin' '\n+test_expect_success 'bundle --stdin' '\n \techo master | git bundle create stdin-bundle.bdl --stdin &&\n \tgit ls-remote stdin-bundle.bdl >output &&\n \tgrep master output\n '\n \n-test_expect_failure 'bundle --stdin ' '\n+test_expect_success 'bundle --stdin ' '\n \techo master | git bundle create hybrid-bundle.bdl --stdin tag &&\n \tgit ls-remote hybrid-bundle.bdl >output &&\n \tgrep master output\n-- \n2.30.0.1.gade423aef4\n\n"},{"id":"413387","messageId":"xmqqft3g6ziz.fsf@gitster.c.googlers.com","threadId":"54923","inReplyTo":"20210103095457.6894-1-worldhello.net@gmail.com","subject":"Re: [PATCH] bundle: arguments can be read from stdin","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-01-04T23:41:24Z","receivedAt":"2021-01-04T23:42:12Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jiang Xin writes:\n\n> Therefore, before calling `prepare_revision_walk()` function, make a\n> copy on `revs.pending` for later use.\n\nThe in-core objects pointed by the list elements are shared between\nthe original and the copy, and the object flag bits that are used to\ncontrol the traversal (like SEEN, SHOWN and BOUNDARY bits) would be\nsmudged during the traversal. So depending on how the \"later use\"\nuses the copied list, it may or may not be sufficient to just copy\nthe list.\n\nApparently, you've tested the updated code well enough to send to\nthe list, so it must be sufficient to make a copy of the list to\nsupport the way the updated code uses it, but it is not clear how it\nis so, only from what is in the proposed log message.\n\n"},{"id":"413466","messageId":"20210105163054.25358-1-worldhello.net@gmail.com","threadId":"54923","inReplyTo":"xmqqft3g6ziz.fsf@gitster.c.googlers.com","subject":"[PATCH v2 1/2] bundle: lost objects when removing duplicate pendings","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-01-05T16:30:53Z","receivedAt":"2021-01-05T16:31:40Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"From: Jiang Xin \n\n`git rev-list` will list one commit for the following command:\n\n $ git rev-list 'main^!'\n \n\nBut providing the same rev-list args to `git bundle`, fail to create\na bundle file.\n\n $ git bundle create - 'main^!'\n # v2 git bundle\n - \n\n fatal: Refusing to create empty bundle.\n\nThis is because when removing duplicate objects in function\n`object_array_remove_duplicates()`, one unique pending object which has\nthe same name is deleted by mistake. The revision arg 'main^!' in the\nabove example is parsed by `handle_revision_arg()`, and at lease two\ndifferent objects will be appended to `revs.pending`, one points to the\nparent commit of the \"main\" branch, and the other points to the tip\ncommit of the \"main\" branch. These two objects have the same name\n\"main\". Only one object is left with the name \"main\" after calling the\nfunction `object_array_remove_duplicates()`.\n\nAnd what's worse, when adding boundary commits into pending list, we use\none-line commit message as names, and the arbitory names may surprise\ngit-bundle.\n\nOnly comparing objects themselves (\".item\") is also not good enough,\nbecause user may want to create a bundle with two identical objects but\nwith different reference names, such as: \"HEAD\" and \"refs/heads/main\".\n\nAdd new function `contains_object()` which compare both the address and\nthe name of the object.\n\nSigned-off-by: Jiang Xin \n---\n object.c | 10 +-\n t/t6020-bundle-misc.sh | 413 +++++++++++++++++++++++++++++++++++++++++\n 2 files changed, 419 insertions(+), 4 deletions(-)\n create mode 100755 t/t6020-bundle-misc.sh\n\ndiff --git a/object.c b/object.c\nindex 68f80b0b3d..98017bed8e 100644\n--- a/object.c\n+++ b/object.c\n@@ -412,15 +412,16 @@ void object_array_clear(struct object_array *array)\n }\n \n /*\n- * Return true iff array already contains an entry with name.\n+ * Return true if array already contains an entry.\n */\n-static int contains_name(struct object_array *array, const char *name)\n+static int contains_object(struct object_array *array,\n+\t\t\t const struct object *item, const char *name)\n {\n \tunsigned nr = array->nr, i;\n \tstruct object_array_entry *object = array->objects;\n \n \tfor (i = 0; i < nr; i++, object++)\n-\t\tif (!strcmp(object->name, name))\n+\t\tif (item == object->item && !strcmp(object->name, name))\n \t\t\treturn 1;\n \treturn 0;\n }\n@@ -432,7 +433,8 @@ void object_array_remove_duplicates(struct object_array *array)\n \n \tarray->nr = 0;\n \tfor (src = 0; src < nr; src++) {\n-\t\tif (!contains_name(array, objects[src].name)) {\n+\t\tif (!contains_object(array, objects[src].item,\n+\t\t\t\t objects[src].name)) {\n \t\t\tif (src != array->nr)\n \t\t\t\tobjects[array->nr] = objects[src];\n \t\t\tarray->nr++;\ndiff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh\nnew file mode 100755\nindex 0000000000..c6613a4162\n--- /dev/null\n+++ b/t/t6020-bundle-misc.sh\n@@ -0,0 +1,413 @@\n+test_description='Test git-bundle'\n+\n+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n+\n+. ./test-lib.sh\n+\n+test_bundle_object_count () {\n+\tgit verify-pack -v \"$1\" >verify.out &&\n+\tcount=$(grep \"^$OID_REGEX \" verify.out | wc -l) &&\n+\ttest $2 = $count && return 0\n+\techo object count is $count, not $2\n+\treturn 1\n+}\n+\n+convert_bundle_to_pack () {\n+\twhile read x && test -n \"$x\"\n+\tdo\n+\t\t:;\n+\tdone\n+\tcat\n+}\n+\n+# Format the output of git commands to make a user-friendly and stable\n+# text. We can easily prepare the expect text without having to worry\n+# about future changes of the commit ID and spaces of the output.\n+make_user_friendly_and_stable_output () {\n+\tsed \\\n+\t\t-e \"s/ *\\$//\" \\\n+\t\t-e \"s/$A//g\" \\\n+\t\t-e \"s/$B//g\" \\\n+\t\t-e \"s/$C//g\" \\\n+\t\t-e \"s/$D//g\" \\\n+\t\t-e \"s/$E//g\" \\\n+\t\t-e \"s/$F//g\" \\\n+\t\t-e \"s/$G//g\" \\\n+\t\t-e \"s/$H//g\" \\\n+\t\t-e \"s/$I//g\" \\\n+\t\t-e \"s/$J//g\" \\\n+\t\t-e \"s/$K//g\" \\\n+\t\t-e \"s/$L//g\" \\\n+\t\t-e \"s/$M//g\" \\\n+\t\t-e \"s/$N//g\" \\\n+\t\t-e \"s/$O//g\" \\\n+\t\t-e \"s/$P//g\" \\\n+\t\t-e \"s/$(echo $A | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $B | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $C | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $D | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $E | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $F | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $G | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $H | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $I | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $J | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $K | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $L | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $M | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $N | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $O | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $P | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$TAG1//g\" \\\n+\t\t-e \"s/$TAG2//g\" \\\n+\t\t-e \"s/$TAG3//g\" \\\n+\t\t-e \"s/$(echo $TAG1 | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $TAG2 | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $TAG3 | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$ZERO_OID//g\"\n+}\n+\n+# (C) (D, pull/1/head, topic/1)\n+# o --- o\n+# / \\ (L, tags/v1)\n+# / \\ o (H, topic/2) (M, tags/v2)\n+# / (F) \\ / (N, tags/v3)\n+# / o --------- o (G, pull/2/head) o --- o --- o (release)\n+# / / \\ \\ / \\\n+# o --- o --- o -------- o -- o ------------------ o ------- o --- o (main)\n+# (A) (B) (E) (I) (J) (K) (O) (P)\n+#\n+test_expect_success 'setup' '\n+\t# commit A & B\n+\tcat >main.txt <<-EOF &&\n+\t\tCommit A\n+\t\tEOF\n+\tgit add main.txt &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit A\" &&\n+\n+\tcat >main.txt <<-EOF &&\n+\t\tCommit B\n+\t\tEOF\n+\tgit add main.txt &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit B\" &&\n+\tA=$(git rev-parse HEAD~) &&\n+\tB=$(git rev-parse HEAD) &&\n+\n+\t# branch topic/1\n+\tgit checkout -b topic/1 &&\n+\tcat >topic-1.txt <<-EOF &&\n+\t\tCommit C\n+\t\tEOF\n+\tgit add topic-1.txt &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit C\" &&\n+\n+\tcat >topic-1.txt <<-EOF &&\n+\t\tCommit D\n+\t\tEOF\n+\tgit add -u &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit D\" &&\n+\tgit update-ref refs/pull/1/head HEAD &&\n+\tC=$(git rev-parse topic/1~) &&\n+\tD=$(git rev-parse topic/1) &&\n+\n+\t# commit E\n+\tgit checkout main &&\n+\tcat >main.txt <<-EOF &&\n+\t\tCommit E\n+\t\tEOF\n+\tgit add main.txt &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit E\" &&\n+\tE=$(git rev-parse HEAD) &&\n+\n+\t# branch topic/2\n+\tgit checkout -b topic/2 &&\n+\tcat >topic-2.txt <<-EOF &&\n+\t\tCommit F\n+\t\tEOF\n+\tgit add topic-2.txt &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit F\" &&\n+\n+\tcat >topic-2.txt <<-EOF &&\n+\t\tCommit G\n+\t\tEOF\n+\tgit add -u &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit G\" &&\n+\tgit update-ref refs/pull/2/head HEAD &&\n+\n+\tcat >topic-2.txt <<-EOF &&\n+\t\tCommit H\n+\t\tEOF\n+\tgit add -u &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit H\" &&\n+\tF=$(git rev-parse topic/2~2) &&\n+\tG=$(git rev-parse topic/2~) &&\n+\tH=$(git rev-parse topic/2) &&\n+\n+\t# merge commit I & J\n+\tgit checkout main &&\n+\ttest_tick &&\n+\tgit merge --no-ff --no-edit topic/1 &&\n+\ttest_tick &&\n+\tgit merge --no-ff --no-edit refs/pull/2/head &&\n+\tI=$(git rev-parse HEAD~) &&\n+\tJ=$(git rev-parse HEAD) &&\n+\n+\t# commit K\n+\tgit checkout main &&\n+\tcat >main.txt <<-EOF &&\n+\t\tCommit K\n+\t\tEOF\n+\tgit add main.txt &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit K\" &&\n+\tK=$(git rev-parse HEAD) &&\n+\n+\t# branch release\n+\tgit checkout -b release &&\n+\tcat >release.txt <<-EOF &&\n+\t\tCommit L\n+\t\tEOF\n+\tgit add release.txt &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit L\" &&\n+\ttest_tick &&\n+\tgit tag -m \"v1\" v1 HEAD &&\n+\n+\tcat >release.txt <<-EOF &&\n+\t\tCommit M\n+\t\tEOF\n+\tgit add -u &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit M\" &&\n+\ttest_tick &&\n+\tgit tag -m \"v2\" v2 HEAD &&\n+\n+\tcat >release.txt <<-EOF &&\n+\t\tCommit N\n+\t\tEOF\n+\tgit add -u &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit N\" &&\n+\ttest_tick &&\n+\tgit tag -m \"v3\" v3 HEAD &&\n+\tL=$(git rev-parse HEAD~2) &&\n+\tM=$(git rev-parse HEAD~) &&\n+\tN=$(git rev-parse HEAD) &&\n+\tTAG1=$(git rev-parse refs/tags/v1) &&\n+\tTAG2=$(git rev-parse refs/tags/v2) &&\n+\tTAG3=$(git rev-parse refs/tags/v3) &&\n+\n+\t# merge commit O\n+\tgit checkout main &&\n+\ttest_tick &&\n+\tgit merge --no-ff --no-edit tags/v2 &&\n+\tO=$(git rev-parse HEAD) &&\n+\n+\t# commit P\n+\tgit checkout main &&\n+\tcat >main.txt <<-EOF &&\n+\t\tCommit P\n+\t\tEOF\n+\tgit add main.txt &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit P\" &&\n+\tP=$(git rev-parse HEAD)\n+'\n+\n+test_expect_success 'create bundle from special rev: main^!' '\n+\tgit bundle create special-rev.bdl \"main^!\" &&\n+\n+\tgit bundle list-heads special-rev.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\t refs/heads/main\n+\t\tEOF\n+\ttest_i18ncmp expect actual &&\n+\n+\tgit bundle verify special-rev.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\tThe bundle contains this ref:\n+\t\t refs/heads/main\n+\t\tThe bundle requires this ref:\n+\t\t\n+\t\tEOF\n+\ttest_i18ncmp expect actual &&\n+\n+\tconvert_bundle_to_pack special-rev.pack &&\n+\tgit index-pack special-rev.pack &&\n+\ttest_bundle_object_count special-rev.pack 3\n+'\n+\n+test_expect_success 'create bundle 1 - no prerequisites' '\n+\t# create bundle from args\n+\tgit bundle create 1.bdl topic/1 topic/2 &&\n+\n+\tcat >expect <<-EOF &&\n+\t\tThe bundle contains these 2 refs:\n+\t\t refs/heads/topic/1\n+\t\t refs/heads/topic/2\n+\t\tThe bundle records a complete history.\n+\t\tEOF\n+\n+\t# verify bundle, which has no prerequisites\n+\tgit bundle verify 1.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\tconvert_bundle_to_pack <1.bdl >1.pack &&\n+\tgit index-pack 1.pack &&\n+\ttest_bundle_object_count 1.pack 24\n+'\n+\n+test_expect_success 'create bundle 2 - has prerequisites' '\n+\t# create bundle from args\n+\tgit bundle create 2.bdl \\\n+\t\t--ignore-missing \\\n+\t\t^topic/deleted \\\n+\t\t^$D \\\n+\t\t^topic/2 \\\n+\t\trelease &&\n+\n+\tcat >expect <<-EOF &&\n+\t\tThe bundle contains this ref:\n+\t\t refs/heads/release\n+\t\tThe bundle requires these 3 refs:\n+\t\t\n+\t\t\n+\t\t\n+\t\tEOF\n+\n+\tgit bundle verify 2.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\tconvert_bundle_to_pack <2.bdl >2.pack &&\n+\tgit index-pack 2.pack &&\n+\ttest_bundle_object_count 2.pack 16\n+'\n+\n+test_expect_success 'fail to verify bundle without prerequisites' '\n+\tgit init --bare test1.git &&\n+\n+\tcat >expect <<-EOF &&\n+\t\terror: Repository lacks these prerequisite commits:\n+\t\terror: \n+\t\terror: \n+\t\terror: \n+\t\tEOF\n+\n+\ttest_must_fail git -C test1.git bundle verify ../2.bdl 2>&1 |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual\n+'\n+\n+test_expect_success 'create bundle 3 - two refs, same object' '\n+\t# create bundle from args\n+\tgit bundle create --version=3 3.bdl \\\n+\t\t^release \\\n+\t\t^topic/1 \\\n+\t\t^topic/2 \\\n+\t\tmain \\\n+\t\tHEAD &&\n+\n+\tcat >expect <<-EOF &&\n+\t\tThe bundle contains these 2 refs:\n+\t\t refs/heads/main\n+\t\t HEAD\n+\t\tThe bundle requires these 2 refs:\n+\t\t\n+\t\t\n+\t\tEOF\n+\n+\tgit bundle verify 3.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\tconvert_bundle_to_pack <3.bdl >3.pack &&\n+\tgit index-pack 3.pack &&\n+\ttest_bundle_object_count 3.pack 4\n+'\n+\n+test_expect_success 'create bundle 4 - with tags' '\n+\t# create bundle from args\n+\tgit bundle create 4.bdl \\\n+\t\t^main \\\n+\t\t^release \\\n+\t\t^topic/1 \\\n+\t\t^topic/2 \\\n+\t\t--all &&\n+\n+\tcat >expect <<-EOF &&\n+\t\tThe bundle contains these 3 refs:\n+\t\t refs/tags/v1\n+\t\t refs/tags/v2\n+\t\t refs/tags/v3\n+\t\tThe bundle records a complete history.\n+\t\tEOF\n+\n+\tgit bundle verify 4.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\tconvert_bundle_to_pack <4.bdl >4.pack &&\n+\tgit index-pack 4.pack &&\n+\ttest_bundle_object_count 4.pack 3\n+'\n+\n+test_expect_success 'clone from bundle' '\n+\tgit clone --mirror 1.bdl mirror.git &&\n+\tgit -C mirror.git show-ref |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\t refs/heads/topic/1\n+\t\t refs/heads/topic/2\n+\t\tEOF\n+\ttest_cmp expect actual &&\n+\n+\tgit -C mirror.git fetch ../2.bdl \"+refs/*:refs/*\" &&\n+\tgit -C mirror.git show-ref |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\t refs/heads/release\n+\t\t refs/heads/topic/1\n+\t\t refs/heads/topic/2\n+\t\tEOF\n+\ttest_cmp expect actual &&\n+\n+\tgit -C mirror.git fetch ../3.bdl \"+refs/*:refs/*\" &&\n+\tgit -C mirror.git show-ref |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\t refs/heads/main\n+\t\t refs/heads/release\n+\t\t refs/heads/topic/1\n+\t\t refs/heads/topic/2\n+\t\tEOF\n+\ttest_cmp expect actual &&\n+\n+\tgit -C mirror.git fetch ../4.bdl \"+refs/*:refs/*\" &&\n+\tgit -C mirror.git show-ref |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\t refs/heads/main\n+\t\t refs/heads/release\n+\t\t refs/heads/topic/1\n+\t\t refs/heads/topic/2\n+\t\t refs/tags/v1\n+\t\t refs/tags/v2\n+\t\t refs/tags/v3\n+\t\tEOF\n+\ttest_cmp expect actual\n+'\n+\n+test_done\n-- \n2.30.0.4.g09ee524cd5\n\n"},{"id":"413467","messageId":"20210105163054.25358-2-worldhello.net@gmail.com","threadId":"54923","inReplyTo":"xmqqft3g6ziz.fsf@gitster.c.googlers.com","subject":"[PATCH v2 2/2] bundle: arguments can be read from stdin","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-01-05T16:30:54Z","receivedAt":"2021-01-05T16:31:56Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"From: Jiang Xin \n\nIn order to create an incremental bundle, we need to pass many arguments\nto let git-bundle ignore some already packed commits. It will be more\nconvenient to pass args via stdin. But the current implementation does\nnot allow us to do this.\n\nThis is because args are parsed twice when creating bundle. The first\ntime for parsing args is in `compute_and_write_prerequisites()` by\nrunning `git-rev-list` command to write prerequisites in bundle file,\nand stdin is consumed in this step if \"--stdin\" option is provided for\n`git-bundle`. Later nothing can be read from stdin when running\n`setup_revisions()` in `create_bundle()`.\n\nRemove the entire `compute_and_write_prerequisites()` function, and\nparse the args once by `setup_revisions()`. The first step for creating\na bundle is to write prerequisites (\"-\" obj-id SP comment LF), but after\ncalling `prepare_revision_walk()`, the `revs.pending` is left empty.\nFollowing steps could not work properly without data in `revs.pending`.\nTherefore, before calling `prepare_revision_walk()` function, make a\ncopy `revs_copy` from `revs` for later use. Even though `revs_copy` and\n`revs` share the same objects, but changes on these objects will not\nchange the behavior of function `write_bundle_refs()` and\n`write_pack_data()`.\n\nAlso add testcases for git bundle in t6020, which read args from stdin.\n\nSigned-off-by: Jiang Xin \n---\n bundle.c | 109 ++++++++++++++++++++++------------------\n t/t5607-clone-bundle.sh | 4 +-\n t/t6020-bundle-misc.sh | 85 +++++++++++++++++++++++++++++--\n 3 files changed, 142 insertions(+), 56 deletions(-)\n\ndiff --git a/bundle.c b/bundle.c\nindex cb0e5931ac..693d619551 100644\n--- a/bundle.c\n+++ b/bundle.c\n@@ -338,48 +338,6 @@ static int write_pack_data(int bundle_fd, struct rev_info *revs, struct strvec *\n \treturn 0;\n }\n \n-static int compute_and_write_prerequisites(int bundle_fd,\n-\t\t\t\t\t struct rev_info *revs,\n-\t\t\t\t\t int argc, const char **argv)\n-{\n-\tstruct child_process rls = CHILD_PROCESS_INIT;\n-\tstruct strbuf buf = STRBUF_INIT;\n-\tFILE *rls_fout;\n-\tint i;\n-\n-\tstrvec_pushl(&rls.args,\n-\t\t \"rev-list\", \"--boundary\", \"--pretty=oneline\",\n-\t\t NULL);\n-\tfor (i = 1; i < argc; i++)\n-\t\tstrvec_push(&rls.args, argv[i]);\n-\trls.out = -1;\n-\trls.git_cmd = 1;\n-\tif (start_command(&rls))\n-\t\treturn -1;\n-\trls_fout = xfdopen(rls.out, \"r\");\n-\twhile (strbuf_getwholeline(&buf, rls_fout, '\\n') != EOF) {\n-\t\tstruct object_id oid;\n-\t\tif (buf.len > 0 && buf.buf[0] == '-') {\n-\t\t\twrite_or_die(bundle_fd, buf.buf, buf.len);\n-\t\t\tif (!get_oid_hex(buf.buf + 1, &oid)) {\n-\t\t\t\tstruct object *object = parse_object_or_die(&oid,\n-\t\t\t\t\t\t\t\t\t buf.buf);\n-\t\t\t\tobject->flags |= UNINTERESTING;\n-\t\t\t\tadd_pending_object(revs, object, buf.buf);\n-\t\t\t}\n-\t\t} else if (!get_oid_hex(buf.buf, &oid)) {\n-\t\t\tstruct object *object = parse_object_or_die(&oid,\n-\t\t\t\t\t\t\t\t buf.buf);\n-\t\t\tobject->flags |= SHOWN;\n-\t\t}\n-\t}\n-\tstrbuf_release(&buf);\n-\tfclose(rls_fout);\n-\tif (finish_command(&rls))\n-\t\treturn error(_(\"rev-list died\"));\n-\treturn 0;\n-}\n-\n /*\n * Write out bundle refs based on the tips already\n * parsed into revs.pending. As a side effect, may\n@@ -474,6 +432,38 @@ static int write_bundle_refs(int bundle_fd, struct rev_info *revs)\n \treturn ref_count;\n }\n \n+struct bundle_prerequisites_info {\n+\tstruct object_array *pending;\n+\tint fd;\n+};\n+\n+static void write_bundle_prerequisites(struct commit *commit, void *data)\n+{\n+\tstruct bundle_prerequisites_info *bpi = data;\n+\tstruct object *object;\n+\tstruct pretty_print_context ctx = { 0 };\n+\tstruct strbuf buf = STRBUF_INIT;\n+\n+\tif (!(commit->object.flags & BOUNDARY))\n+\t\treturn;\n+\tstrbuf_addf(&buf, \"-%s \", oid_to_hex(&commit->object.oid));\n+\twrite_or_die(bpi->fd, buf.buf, buf.len);\n+\n+\tctx.fmt = CMIT_FMT_ONELINE;\n+\tctx.output_encoding = get_log_output_encoding();\n+\tstrbuf_reset(&buf);\n+\tpretty_print_commit(&ctx, commit, &buf);\n+\tstrbuf_trim(&buf);\n+\n+\tobject = (struct object *)commit;\n+\tobject->flags |= UNINTERESTING;\n+\tadd_object_array_with_path(object, buf.buf, bpi->pending, S_IFINVALID,\n+\t\t\t\t NULL);\n+\tstrbuf_addch(&buf, '\\n');\n+\twrite_or_die(bpi->fd, buf.buf, buf.len);\n+\tstrbuf_release(&buf);\n+}\n+\n int create_bundle(struct repository *r, const char *path,\n \t\t int argc, const char **argv, struct strvec *pack_options, int version)\n {\n@@ -481,8 +471,10 @@ int create_bundle(struct repository *r, const char *path,\n \tint bundle_fd = -1;\n \tint bundle_to_stdout;\n \tint ref_count = 0;\n-\tstruct rev_info revs;\n+\tstruct rev_info revs, revs_copy;\n \tint min_version = the_hash_algo == &hash_algos[GIT_HASH_SHA1] ? 2 : 3;\n+\tstruct bundle_prerequisites_info bpi;\n+\tint i;\n \n \tbundle_to_stdout = !strcmp(path, \"-\");\n \tif (bundle_to_stdout)\n@@ -512,10 +504,6 @@ int create_bundle(struct repository *r, const char *path,\n \tsave_commit_buffer = 0;\n \trepo_init_revisions(r, &revs, NULL);\n \n-\t/* write prerequisites */\n-\tif (compute_and_write_prerequisites(bundle_fd, &revs, argc, argv))\n-\t\tgoto err;\n-\n \targc = setup_revisions(argc, argv, &revs, NULL);\n \n \tif (argc > 1) {\n@@ -523,16 +511,37 @@ int create_bundle(struct repository *r, const char *path,\n \t\tgoto err;\n \t}\n \n-\tobject_array_remove_duplicates(&revs.pending);\n+\t/* save revs.pending in revs_copy for later use */\n+\tmemcpy(&revs_copy, &revs, sizeof(revs));\n+\trevs_copy.pending.nr = 0;\n+\trevs_copy.pending.alloc = 0;\n+\trevs_copy.pending.objects = NULL;\n+\tfor (i = 0; i < revs.pending.nr; i++) {\n+\t\tstruct object_array_entry *e = revs.pending.objects + i;\n+\t\tif (e)\n+\t\t\tadd_object_array_with_path(e->item, e->name,\n+\t\t\t\t\t\t &revs_copy.pending,\n+\t\t\t\t\t\t e->mode, e->path);\n+\t}\n \n-\tref_count = write_bundle_refs(bundle_fd, &revs);\n+\t/* write prerequisites */\n+\trevs.boundary = 1;\n+\tif (prepare_revision_walk(&revs))\n+\t\tdie(\"revision walk setup failed\");\n+\tbpi.fd = bundle_fd;\n+\tbpi.pending = &revs_copy.pending;\n+\ttraverse_commit_list(&revs, write_bundle_prerequisites, NULL, &bpi);\n+\tobject_array_remove_duplicates(&revs_copy.pending);\n+\n+\t/* write bundle refs */\n+\tref_count = write_bundle_refs(bundle_fd, &revs_copy);\n \tif (!ref_count)\n \t\tdie(_(\"Refusing to create empty bundle.\"));\n \telse if (ref_count < 0)\n \t\tgoto err;\n \n \t/* write pack */\n-\tif (write_pack_data(bundle_fd, &revs, pack_options))\n+\tif (write_pack_data(bundle_fd, &revs_copy, pack_options))\n \t\tgoto err;\n \n \tif (!bundle_to_stdout) {\ndiff --git a/t/t5607-clone-bundle.sh b/t/t5607-clone-bundle.sh\nindex 26985f4b44..425258767d 100755\n--- a/t/t5607-clone-bundle.sh\n+++ b/t/t5607-clone-bundle.sh\n@@ -38,13 +38,13 @@ test_expect_success 'die if bundle file cannot be created' '\n \ttest_must_fail git bundle create adir --all\n '\n \n-test_expect_failure 'bundle --stdin' '\n+test_expect_success 'bundle --stdin' '\n \techo master | git bundle create stdin-bundle.bdl --stdin &&\n \tgit ls-remote stdin-bundle.bdl >output &&\n \tgrep master output\n '\n \n-test_expect_failure 'bundle --stdin ' '\n+test_expect_success 'bundle --stdin ' '\n \techo master | git bundle create hybrid-bundle.bdl --stdin tag &&\n \tgit ls-remote hybrid-bundle.bdl >output &&\n \tgrep master output\ndiff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh\nindex c6613a4162..780a66dcc5 100755\n--- a/t/t6020-bundle-misc.sh\n+++ b/t/t6020-bundle-misc.sh\n@@ -252,6 +252,13 @@ test_expect_success 'create bundle 1 - no prerequisites' '\n \t# create bundle from args\n \tgit bundle create 1.bdl topic/1 topic/2 &&\n \n+\t# create bundle from stdin\n+\tcat >input <<-EOF &&\n+\t\ttopic/1\n+\t\ttopic/2\n+\t\tEOF\n+\tgit bundle create stdin-1.bdl --stdin expect <<-EOF &&\n \t\tThe bundle contains these 2 refs:\n \t\t refs/heads/topic/1\n@@ -264,9 +271,17 @@ test_expect_success 'create bundle 1 - no prerequisites' '\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual &&\n \n+\tgit bundle verify stdin-1.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n \tconvert_bundle_to_pack <1.bdl >1.pack &&\n \tgit index-pack 1.pack &&\n-\ttest_bundle_object_count 1.pack 24\n+\ttest_bundle_object_count 1.pack 24 &&\n+\n+\tconvert_bundle_to_pack stdin-1.pack &&\n+\tgit index-pack stdin-1.pack &&\n+\ttest_bundle_object_count stdin-1.pack 24\n '\n \n test_expect_success 'create bundle 2 - has prerequisites' '\n@@ -278,6 +293,18 @@ test_expect_success 'create bundle 2 - has prerequisites' '\n \t\t^topic/2 \\\n \t\trelease &&\n \n+\t# create bundle from stdin\n+\t# input has a non-exist reference: \"topic/deleted\"\n+\tcat >input <<-EOF &&\n+\t\t^topic/deleted\n+\t\t^$D\n+\t\t^topic/2\n+\t\tEOF\n+\tgit bundle create stdin-2.bdl \\\n+\t\t--ignore-missing \\\n+\t\t--stdin \\\n+\t\trelease expect <<-EOF &&\n \t\tThe bundle contains this ref:\n \t\t refs/heads/release\n@@ -291,9 +318,17 @@ test_expect_success 'create bundle 2 - has prerequisites' '\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual &&\n \n+\tgit bundle verify stdin-2.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n \tconvert_bundle_to_pack <2.bdl >2.pack &&\n \tgit index-pack 2.pack &&\n-\ttest_bundle_object_count 2.pack 16\n+\ttest_bundle_object_count 2.pack 16 &&\n+\n+\tconvert_bundle_to_pack stdin-2.pack &&\n+\tgit index-pack stdin-2.pack &&\n+\ttest_bundle_object_count stdin-2.pack 16\n '\n \n test_expect_success 'fail to verify bundle without prerequisites' '\n@@ -308,6 +343,10 @@ test_expect_success 'fail to verify bundle without prerequisites' '\n \n \ttest_must_fail git -C test1.git bundle verify ../2.bdl 2>&1 |\n \t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_must_fail git -C test1.git bundle verify ../stdin-2.bdl 2>&1 |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual\n '\n \n@@ -320,6 +359,16 @@ test_expect_success 'create bundle 3 - two refs, same object' '\n \t\tmain \\\n \t\tHEAD &&\n \n+\t# create bundle from stdin\n+\tcat >input <<-EOF &&\n+\t\t^release\n+\t\t^topic/1\n+\t\t^topic/2\n+\t\tEOF\n+\tgit bundle create --version=3 stdin-3.bdl \\\n+\t\t--stdin \\\n+\t\tmain HEAD expect <<-EOF &&\n \t\tThe bundle contains these 2 refs:\n \t\t refs/heads/main\n@@ -333,9 +382,17 @@ test_expect_success 'create bundle 3 - two refs, same object' '\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual &&\n \n+\tgit bundle verify stdin-3.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n \tconvert_bundle_to_pack <3.bdl >3.pack &&\n \tgit index-pack 3.pack &&\n-\ttest_bundle_object_count 3.pack 4\n+\ttest_bundle_object_count 3.pack 4 &&\n+\n+\tconvert_bundle_to_pack stdin-3.pack &&\n+\tgit index-pack stdin-3.pack &&\n+\ttest_bundle_object_count stdin-3.pack 4\n '\n \n test_expect_success 'create bundle 4 - with tags' '\n@@ -347,6 +404,18 @@ test_expect_success 'create bundle 4 - with tags' '\n \t\t^topic/2 \\\n \t\t--all &&\n \n+\t# create bundle from stdin\n+\tcat >input <<-EOF &&\n+\t\t^main\n+\t\t^release\n+\t\t^topic/1\n+\t\t^topic/2\n+\t\tEOF\n+\tgit bundle create stdin-4.bdl \\\n+\t\t--ignore-missing \\\n+\t\t--stdin \\\n+\t\t--all expect <<-EOF &&\n \t\tThe bundle contains these 3 refs:\n \t\t refs/tags/v1\n@@ -359,9 +428,17 @@ test_expect_success 'create bundle 4 - with tags' '\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual &&\n \n+\tgit bundle verify stdin-4.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n \tconvert_bundle_to_pack <4.bdl >4.pack &&\n \tgit index-pack 4.pack &&\n-\ttest_bundle_object_count 4.pack 3\n+\ttest_bundle_object_count 4.pack 3 &&\n+\n+\tconvert_bundle_to_pack stdin-4.pack &&\n+\tgit index-pack stdin-4.pack &&\n+\ttest_bundle_object_count stdin-4.pack 3\n '\n \n test_expect_success 'clone from bundle' '\n-- \n2.30.0.4.g09ee524cd5\n\n"},{"id":"413688","messageId":"20210107135025.2682-3-worldhello.net@gmail.com","threadId":"54923","inReplyTo":"xmqqft3g6ziz.fsf@gitster.c.googlers.com","subject":"[PATCH v3 2/2] bundle: arguments can be read from stdin","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-01-07T13:50:25Z","receivedAt":"2021-01-07T13:51:29Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"From: Jiang Xin \n\nIn order to create an incremental bundle, we need to pass many arguments\nto let git-bundle ignore some already packed commits. It will be more\nconvenient to pass args via stdin. But the current implementation does\nnot allow us to do this.\n\nThis is because args are parsed twice when creating bundle. The first\ntime for parsing args is in `compute_and_write_prerequisites()` by\nrunning `git-rev-list` command to write prerequisites in bundle file,\nand stdin is consumed in this step if \"--stdin\" option is provided for\n`git-bundle`. Later nothing can be read from stdin when running\n`setup_revisions()` in `create_bundle()`.\n\nThe solution is to parse args once by removing the entire function\n`compute_and_write_prerequisites()` and then calling function\n`setup_revisions()`. In order to write prerequisites for bundle, will\ncall `prepare_revision_walk()` and `traverse_commit_list()`. But after\ncalling `prepare_revision_walk()`, the object array `revs.pending` is\nleft empty, and the following steps could not work properly with the\nempty object array (`revs.pending`). Therefore, make a copy of `revs`\nto `revs_copy` for later use right after calling `setup_revisions()`.\n\nThe copy of `revs_copy` is not a deep copy, it shares the same objects\nwith `revs`. The object array of `revs` has been cleared, but objects\nthemselves are still kept. Flags of objects may change after calling\n`prepare_revision_walk()`, we can use these changed flags without\ncalling the `git rev-list` command and parsing its output like the\nformer implementation.\n\nAlso add testcases for git bundle in t6020, which read args from stdin.\n\nSigned-off-by: Jiang Xin \n---\n bundle.c | 111 ++++++++++++++++++++++------------------\n t/t5607-clone-bundle.sh | 4 +-\n t/t6020-bundle-misc.sh | 77 ++++++++++++++++++++++++++--\n 3 files changed, 135 insertions(+), 57 deletions(-)\n\ndiff --git a/bundle.c b/bundle.c\nindex cb0e5931ac..72970d962a 100644\n--- a/bundle.c\n+++ b/bundle.c\n@@ -338,48 +338,6 @@ static int write_pack_data(int bundle_fd, struct rev_info *revs, struct strvec *\n \treturn 0;\n }\n \n-static int compute_and_write_prerequisites(int bundle_fd,\n-\t\t\t\t\t struct rev_info *revs,\n-\t\t\t\t\t int argc, const char **argv)\n-{\n-\tstruct child_process rls = CHILD_PROCESS_INIT;\n-\tstruct strbuf buf = STRBUF_INIT;\n-\tFILE *rls_fout;\n-\tint i;\n-\n-\tstrvec_pushl(&rls.args,\n-\t\t \"rev-list\", \"--boundary\", \"--pretty=oneline\",\n-\t\t NULL);\n-\tfor (i = 1; i < argc; i++)\n-\t\tstrvec_push(&rls.args, argv[i]);\n-\trls.out = -1;\n-\trls.git_cmd = 1;\n-\tif (start_command(&rls))\n-\t\treturn -1;\n-\trls_fout = xfdopen(rls.out, \"r\");\n-\twhile (strbuf_getwholeline(&buf, rls_fout, '\\n') != EOF) {\n-\t\tstruct object_id oid;\n-\t\tif (buf.len > 0 && buf.buf[0] == '-') {\n-\t\t\twrite_or_die(bundle_fd, buf.buf, buf.len);\n-\t\t\tif (!get_oid_hex(buf.buf + 1, &oid)) {\n-\t\t\t\tstruct object *object = parse_object_or_die(&oid,\n-\t\t\t\t\t\t\t\t\t buf.buf);\n-\t\t\t\tobject->flags |= UNINTERESTING;\n-\t\t\t\tadd_pending_object(revs, object, buf.buf);\n-\t\t\t}\n-\t\t} else if (!get_oid_hex(buf.buf, &oid)) {\n-\t\t\tstruct object *object = parse_object_or_die(&oid,\n-\t\t\t\t\t\t\t\t buf.buf);\n-\t\t\tobject->flags |= SHOWN;\n-\t\t}\n-\t}\n-\tstrbuf_release(&buf);\n-\tfclose(rls_fout);\n-\tif (finish_command(&rls))\n-\t\treturn error(_(\"rev-list died\"));\n-\treturn 0;\n-}\n-\n /*\n * Write out bundle refs based on the tips already\n * parsed into revs.pending. As a side effect, may\n@@ -425,7 +383,7 @@ static int write_bundle_refs(int bundle_fd, struct rev_info *revs)\n \t\t * constraints.\n \t\t */\n \t\tif (!(e->item->flags & SHOWN) && e->item->type == OBJ_COMMIT) {\n-\t\t\twarning(_(\"ref '%s' is excluded by the rev-list options\"),\n+\t\t\twarning(_(\"ref '%s' is excluded by the limiting options\"),\n \t\t\t\te->name);\n \t\t\tgoto skip_write_ref;\n \t\t}\n@@ -474,6 +432,38 @@ static int write_bundle_refs(int bundle_fd, struct rev_info *revs)\n \treturn ref_count;\n }\n \n+struct bundle_prerequisites_info {\n+\tstruct object_array *pending;\n+\tint fd;\n+};\n+\n+static void write_bundle_prerequisites(struct commit *commit, void *data)\n+{\n+\tstruct bundle_prerequisites_info *bpi = data;\n+\tstruct object *object;\n+\tstruct pretty_print_context ctx = { 0 };\n+\tstruct strbuf buf = STRBUF_INIT;\n+\n+\tif (!(commit->object.flags & BOUNDARY))\n+\t\treturn;\n+\tstrbuf_addf(&buf, \"-%s \", oid_to_hex(&commit->object.oid));\n+\twrite_or_die(bpi->fd, buf.buf, buf.len);\n+\n+\tctx.fmt = CMIT_FMT_ONELINE;\n+\tctx.output_encoding = get_log_output_encoding();\n+\tstrbuf_reset(&buf);\n+\tpretty_print_commit(&ctx, commit, &buf);\n+\tstrbuf_trim(&buf);\n+\n+\tobject = (struct object *)commit;\n+\tobject->flags |= UNINTERESTING;\n+\tadd_object_array_with_path(object, buf.buf, bpi->pending, S_IFINVALID,\n+\t\t\t\t NULL);\n+\tstrbuf_addch(&buf, '\\n');\n+\twrite_or_die(bpi->fd, buf.buf, buf.len);\n+\tstrbuf_release(&buf);\n+}\n+\n int create_bundle(struct repository *r, const char *path,\n \t\t int argc, const char **argv, struct strvec *pack_options, int version)\n {\n@@ -481,8 +471,10 @@ int create_bundle(struct repository *r, const char *path,\n \tint bundle_fd = -1;\n \tint bundle_to_stdout;\n \tint ref_count = 0;\n-\tstruct rev_info revs;\n+\tstruct rev_info revs, revs_copy;\n \tint min_version = the_hash_algo == &hash_algos[GIT_HASH_SHA1] ? 2 : 3;\n+\tstruct bundle_prerequisites_info bpi;\n+\tint i;\n \n \tbundle_to_stdout = !strcmp(path, \"-\");\n \tif (bundle_to_stdout)\n@@ -512,10 +504,6 @@ int create_bundle(struct repository *r, const char *path,\n \tsave_commit_buffer = 0;\n \trepo_init_revisions(r, &revs, NULL);\n \n-\t/* write prerequisites */\n-\tif (compute_and_write_prerequisites(bundle_fd, &revs, argc, argv))\n-\t\tgoto err;\n-\n \targc = setup_revisions(argc, argv, &revs, NULL);\n \n \tif (argc > 1) {\n@@ -523,16 +511,37 @@ int create_bundle(struct repository *r, const char *path,\n \t\tgoto err;\n \t}\n \n-\tobject_array_remove_duplicates(&revs.pending);\n+\t/* save revs.pending in revs_copy for later use */\n+\tmemcpy(&revs_copy, &revs, sizeof(revs));\n+\trevs_copy.pending.nr = 0;\n+\trevs_copy.pending.alloc = 0;\n+\trevs_copy.pending.objects = NULL;\n+\tfor (i = 0; i < revs.pending.nr; i++) {\n+\t\tstruct object_array_entry *e = revs.pending.objects + i;\n+\t\tif (e)\n+\t\t\tadd_object_array_with_path(e->item, e->name,\n+\t\t\t\t\t\t &revs_copy.pending,\n+\t\t\t\t\t\t e->mode, e->path);\n+\t}\n \n-\tref_count = write_bundle_refs(bundle_fd, &revs);\n+\t/* write prerequisites */\n+\trevs.boundary = 1;\n+\tif (prepare_revision_walk(&revs))\n+\t\tdie(\"revision walk setup failed\");\n+\tbpi.fd = bundle_fd;\n+\tbpi.pending = &revs_copy.pending;\n+\ttraverse_commit_list(&revs, write_bundle_prerequisites, NULL, &bpi);\n+\tobject_array_remove_duplicates(&revs_copy.pending);\n+\n+\t/* write bundle refs */\n+\tref_count = write_bundle_refs(bundle_fd, &revs_copy);\n \tif (!ref_count)\n \t\tdie(_(\"Refusing to create empty bundle.\"));\n \telse if (ref_count < 0)\n \t\tgoto err;\n \n \t/* write pack */\n-\tif (write_pack_data(bundle_fd, &revs, pack_options))\n+\tif (write_pack_data(bundle_fd, &revs_copy, pack_options))\n \t\tgoto err;\n \n \tif (!bundle_to_stdout) {\ndiff --git a/t/t5607-clone-bundle.sh b/t/t5607-clone-bundle.sh\nindex 26985f4b44..425258767d 100755\n--- a/t/t5607-clone-bundle.sh\n+++ b/t/t5607-clone-bundle.sh\n@@ -38,13 +38,13 @@ test_expect_success 'die if bundle file cannot be created' '\n \ttest_must_fail git bundle create adir --all\n '\n \n-test_expect_failure 'bundle --stdin' '\n+test_expect_success 'bundle --stdin' '\n \techo master | git bundle create stdin-bundle.bdl --stdin &&\n \tgit ls-remote stdin-bundle.bdl >output &&\n \tgrep master output\n '\n \n-test_expect_failure 'bundle --stdin ' '\n+test_expect_success 'bundle --stdin ' '\n \techo master | git bundle create hybrid-bundle.bdl --stdin tag &&\n \tgit ls-remote hybrid-bundle.bdl >output &&\n \tgrep master output\ndiff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh\nindex d15e67c8f7..1f60fe180e 100755\n--- a/t/t6020-bundle-misc.sh\n+++ b/t/t6020-bundle-misc.sh\n@@ -336,8 +336,16 @@ test_expect_success 'create bundle with --since option' '\n '\n \n test_expect_success 'create bundle 1 - no prerequisites' '\n+\t# create bundle from args\n \tgit bundle create 1.bdl topic/1 topic/2 &&\n \n+\t# create bundle from stdin\n+\tcat >input <<-EOF &&\n+\t\ttopic/1\n+\t\ttopic/2\n+\t\tEOF\n+\tgit bundle create stdin-1.bdl --stdin expect <<-EOF &&\n \t\tThe bundle contains these 2 refs:\n \t\t refs/heads/topic/1\n@@ -350,10 +358,16 @@ test_expect_success 'create bundle 1 - no prerequisites' '\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual &&\n \n-\ttest_bundle_object_count 1.bdl 24\n+\tgit bundle verify stdin-1.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 1.bdl 24 &&\n+\ttest_bundle_object_count stdin-1.bdl 24\n '\n \n test_expect_success 'create bundle 2 - has prerequisites' '\n+\t# create bundle from args\n \tgit bundle create 2.bdl \\\n \t\t--ignore-missing \\\n \t\t^topic/deleted \\\n@@ -361,6 +375,18 @@ test_expect_success 'create bundle 2 - has prerequisites' '\n \t\t^topic/2 \\\n \t\trelease &&\n \n+\t# create bundle from stdin\n+\t# input has a non-exist reference: \"topic/deleted\"\n+\tcat >input <<-EOF &&\n+\t\t^topic/deleted\n+\t\t^$D\n+\t\t^topic/2\n+\t\tEOF\n+\tgit bundle create stdin-2.bdl \\\n+\t\t--ignore-missing \\\n+\t\t--stdin \\\n+\t\trelease expect <<-EOF &&\n \t\tThe bundle contains this ref:\n \t\t refs/heads/release\n@@ -374,7 +400,12 @@ test_expect_success 'create bundle 2 - has prerequisites' '\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual &&\n \n-\ttest_bundle_object_count 2.bdl 16\n+\tgit bundle verify stdin-2.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 2.bdl 16 &&\n+\ttest_bundle_object_count stdin-2.bdl 16\n '\n \n test_expect_success 'fail to verify bundle without prerequisites' '\n@@ -389,10 +420,15 @@ test_expect_success 'fail to verify bundle without prerequisites' '\n \n \ttest_must_fail git -C test1.git bundle verify ../2.bdl 2>&1 |\n \t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_must_fail git -C test1.git bundle verify ../stdin-2.bdl 2>&1 |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual\n '\n \n test_expect_success 'create bundle 3 - two refs, same object' '\n+\t# create bundle from args\n \tgit bundle create --version=3 3.bdl \\\n \t\t^release \\\n \t\t^topic/1 \\\n@@ -400,6 +436,16 @@ test_expect_success 'create bundle 3 - two refs, same object' '\n \t\tmain \\\n \t\tHEAD &&\n \n+\t# create bundle from stdin\n+\tcat >input <<-EOF &&\n+\t\t^release\n+\t\t^topic/1\n+\t\t^topic/2\n+\t\tEOF\n+\tgit bundle create --version=3 stdin-3.bdl \\\n+\t\t--stdin \\\n+\t\tmain HEAD expect <<-EOF &&\n \t\tThe bundle contains these 2 refs:\n \t\t refs/heads/main\n@@ -413,10 +459,16 @@ test_expect_success 'create bundle 3 - two refs, same object' '\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual &&\n \n-\ttest_bundle_object_count 3.bdl 4\n+\tgit bundle verify stdin-3.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 3.bdl 4 &&\n+\ttest_bundle_object_count stdin-3.bdl 4\n '\n \n test_expect_success 'create bundle 4 - with tags' '\n+\t# create bundle from args\n \tgit bundle create 4.bdl \\\n \t\t^main \\\n \t\t^release \\\n@@ -424,6 +476,18 @@ test_expect_success 'create bundle 4 - with tags' '\n \t\t^topic/2 \\\n \t\t--all &&\n \n+\t# create bundle from stdin\n+\tcat >input <<-EOF &&\n+\t\t^main\n+\t\t^release\n+\t\t^topic/1\n+\t\t^topic/2\n+\t\tEOF\n+\tgit bundle create stdin-4.bdl \\\n+\t\t--ignore-missing \\\n+\t\t--stdin \\\n+\t\t--all expect <<-EOF &&\n \t\tThe bundle contains these 3 refs:\n \t\t refs/tags/v1\n@@ -436,7 +500,12 @@ test_expect_success 'create bundle 4 - with tags' '\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual &&\n \n-\ttest_bundle_object_count 4.bdl 3\n+\tgit bundle verify stdin-4.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 4.bdl 3 &&\n+\ttest_bundle_object_count stdin-4.bdl 3\n '\n \n test_expect_success 'clone from bundle' '\n-- \n2.30.0.2.g06d2f50715\n\n"},{"id":"413689","messageId":"20210107135025.2682-2-worldhello.net@gmail.com","threadId":"54923","inReplyTo":"xmqqft3g6ziz.fsf@gitster.c.googlers.com","subject":"[PATCH v3 1/2] bundle: lost objects when removing duplicate pendings","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-01-07T13:50:24Z","receivedAt":"2021-01-07T13:51:29Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"From: Jiang Xin \n\n`git rev-list` will list one commit for the following command:\n\n $ git rev-list 'main^!'\n \n\nBut providing the same rev-list args to `git bundle`, fail to create\na bundle file.\n\n $ git bundle create - 'main^!'\n # v2 git bundle\n - \n\n fatal: Refusing to create empty bundle.\n\nThis is because when removing duplicate objects in function\n`object_array_remove_duplicates()`, one unique pending object which has\nthe same name is deleted by mistake. The revision arg 'main^!' in the\nabove example is parsed by `handle_revision_arg()`, and at lease two\ndifferent objects will be appended to `revs.pending`, one points to the\nparent commit of the \"main\" branch, and the other points to the tip\ncommit of the \"main\" branch. These two objects have the same name\n\"main\". Only one object is left with the name \"main\" after calling the\nfunction `object_array_remove_duplicates()`.\n\nAnd what's worse, when adding boundary commits into pending list, we use\none-line commit message as names, and the arbitory names may surprise\ngit-bundle.\n\nOnly comparing objects themselves (\".item\") is also not good enough,\nbecause user may want to create a bundle with two identical objects but\nwith different reference names, such as: \"HEAD\" and \"refs/heads/main\".\n\nAdd new function `contains_object()` which compare both the address and\nthe name of the object.\n\nSigned-off-by: Jiang Xin \n---\n object.c | 10 +-\n t/t6020-bundle-misc.sh | 488 +++++++++++++++++++++++++++++++++++++++++\n 2 files changed, 494 insertions(+), 4 deletions(-)\n create mode 100755 t/t6020-bundle-misc.sh\n\ndiff --git a/object.c b/object.c\nindex 68f80b0b3d..98017bed8e 100644\n--- a/object.c\n+++ b/object.c\n@@ -412,15 +412,16 @@ void object_array_clear(struct object_array *array)\n }\n \n /*\n- * Return true iff array already contains an entry with name.\n+ * Return true if array already contains an entry.\n */\n-static int contains_name(struct object_array *array, const char *name)\n+static int contains_object(struct object_array *array,\n+\t\t\t const struct object *item, const char *name)\n {\n \tunsigned nr = array->nr, i;\n \tstruct object_array_entry *object = array->objects;\n \n \tfor (i = 0; i < nr; i++, object++)\n-\t\tif (!strcmp(object->name, name))\n+\t\tif (item == object->item && !strcmp(object->name, name))\n \t\t\treturn 1;\n \treturn 0;\n }\n@@ -432,7 +433,8 @@ void object_array_remove_duplicates(struct object_array *array)\n \n \tarray->nr = 0;\n \tfor (src = 0; src < nr; src++) {\n-\t\tif (!contains_name(array, objects[src].name)) {\n+\t\tif (!contains_object(array, objects[src].item,\n+\t\t\t\t objects[src].name)) {\n \t\t\tif (src != array->nr)\n \t\t\t\tobjects[array->nr] = objects[src];\n \t\t\tarray->nr++;\ndiff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh\nnew file mode 100755\nindex 0000000000..d15e67c8f7\n--- /dev/null\n+++ b/t/t6020-bundle-misc.sh\n@@ -0,0 +1,488 @@\n+#!/bin/sh\n+#\n+# Copyright (c) 2021 Jiang Xin\n+#\n+\n+test_description='Test git-bundle'\n+\n+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n+\n+. ./test-lib.sh\n+\n+test_bundle_object_count () {\n+\tbundle=$1 &&\n+\tpack=${bundle%.bdl}.pack &&\n+\tconvert_bundle_to_pack <\"$bundle\" >\"$pack\" &&\n+\tgit index-pack \"$pack\" &&\n+\tgit verify-pack -v \"$pack\" >verify.out &&\n+\tcount=$(grep \"^$OID_REGEX \" verify.out | wc -l) &&\n+\ttest $2 = $count && return 0\n+\techo object count for $bundle is $count, not $2\n+\treturn 1\n+}\n+\n+\n+test_thin_bundle_object_count () {\n+\tbundle=$1 &&\n+\tpack=${bundle%.bdl}.pack &&\n+\tconvert_bundle_to_pack <\"$bundle\" |\n+\t\ttest_must_fail git index-pack --stdin \"$pack\" &&\n+\trm -f \"$pack\" &&\n+\tconvert_bundle_to_pack <\"$bundle\" |\n+\t\tgit index-pack --stdin --fix-thin \"$pack\" &&\n+\tgit verify-pack -v \"$pack\" >verify.out &&\n+\tcount=$(grep \"^$OID_REGEX \" verify.out | wc -l) &&\n+\ttest $2 = $count && return 0\n+\techo object count for $bundle is $count, not $2\n+\treturn 1\n+}\n+\n+convert_bundle_to_pack () {\n+\twhile read x && test -n \"$x\"\n+\tdo\n+\t\t:;\n+\tdone\n+\tcat\n+}\n+\n+# Format the output of git commands to make a user-friendly and stable\n+# text. We can easily prepare the expect text without having to worry\n+# about future changes of the commit ID and spaces of the output.\n+make_user_friendly_and_stable_output () {\n+\tsed \\\n+\t\t-e \"s/ *\\$//\" \\\n+\t\t-e \"s/$A//g\" \\\n+\t\t-e \"s/$B//g\" \\\n+\t\t-e \"s/$C//g\" \\\n+\t\t-e \"s/$D//g\" \\\n+\t\t-e \"s/$E//g\" \\\n+\t\t-e \"s/$F//g\" \\\n+\t\t-e \"s/$G//g\" \\\n+\t\t-e \"s/$H//g\" \\\n+\t\t-e \"s/$I//g\" \\\n+\t\t-e \"s/$J//g\" \\\n+\t\t-e \"s/$K//g\" \\\n+\t\t-e \"s/$L//g\" \\\n+\t\t-e \"s/$M//g\" \\\n+\t\t-e \"s/$N//g\" \\\n+\t\t-e \"s/$O//g\" \\\n+\t\t-e \"s/$P//g\" \\\n+\t\t-e \"s/$(echo $A | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $B | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $C | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $D | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $E | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $F | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $G | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $H | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $I | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $J | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $K | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $L | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $M | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $N | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $O | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $P | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$TAG1//g\" \\\n+\t\t-e \"s/$TAG2//g\" \\\n+\t\t-e \"s/$TAG3//g\" \\\n+\t\t-e \"s/$(echo $TAG1 | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $TAG2 | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $TAG3 | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$ZERO_OID//g\"\n+}\n+\n+# (C) (D, pull/1/head, topic/1)\n+# o --- o\n+# / \\ (L)\n+# / \\ o (H, topic/2) (M, tag:v2)\n+# / (F) \\ / (N, tag:v3)\n+# / o --------- o (G, pull/2/head) o --- o --- o (release)\n+# / / \\ \\ / \\\n+# o --- o --- o -------- o -- o ------------------ o ------- o --- o (main)\n+# (A) (B) (E, tag:v1) (I) (J) (K) (O) (P)\n+#\n+test_expect_success 'setup' '\n+\t# commit A & B\n+\tcat >main.txt <<-EOF &&\n+\t\tCommit A\n+\t\tEOF\n+\tgit add main.txt &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit A\" &&\n+\n+\tcat >main.txt <<-EOF &&\n+\t\tCommit B\n+\t\tEOF\n+\tgit add main.txt &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit B\" &&\n+\tA=$(git rev-parse HEAD~) &&\n+\tB=$(git rev-parse HEAD) &&\n+\n+\t# branch topic/1\n+\tgit checkout -b topic/1 &&\n+\tcat >topic-1.txt <<-EOF &&\n+\t\tCommit C\n+\t\tEOF\n+\tgit add topic-1.txt &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit C\" &&\n+\n+\tcat >topic-1.txt <<-EOF &&\n+\t\tCommit D\n+\t\tEOF\n+\tgit add -u &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit D\" &&\n+\tgit update-ref refs/pull/1/head HEAD &&\n+\tC=$(git rev-parse topic/1~) &&\n+\tD=$(git rev-parse topic/1) &&\n+\n+\t# commit E\n+\tgit checkout main &&\n+\tcat >main.txt <<-EOF &&\n+\t\tCommit E\n+\t\tEOF\n+\tgit add main.txt &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit E\" &&\n+\tE=$(git rev-parse HEAD) &&\n+\ttest_tick &&\n+\tgit tag -m \"v1\" v1 HEAD &&\n+\tTAG1=$(git rev-parse refs/tags/v1) &&\n+\n+\t# branch topic/2\n+\tgit checkout -b topic/2 &&\n+\tcat >topic-2.txt <<-EOF &&\n+\t\tCommit F\n+\t\tEOF\n+\tgit add topic-2.txt &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit F\" &&\n+\n+\tcat >topic-2.txt <<-EOF &&\n+\t\tCommit G\n+\t\tEOF\n+\tgit add -u &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit G\" &&\n+\tgit update-ref refs/pull/2/head HEAD &&\n+\n+\tcat >topic-2.txt <<-EOF &&\n+\t\tCommit H\n+\t\tEOF\n+\tgit add -u &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit H\" &&\n+\tF=$(git rev-parse topic/2~2) &&\n+\tG=$(git rev-parse topic/2~) &&\n+\tH=$(git rev-parse topic/2) &&\n+\n+\t# merge commit I & J\n+\tgit checkout main &&\n+\ttest_tick &&\n+\tgit merge --no-ff --no-edit topic/1 &&\n+\ttest_tick &&\n+\tgit merge --no-ff --no-edit refs/pull/2/head &&\n+\tI=$(git rev-parse HEAD~) &&\n+\tJ=$(git rev-parse HEAD) &&\n+\n+\t# commit K\n+\tgit checkout main &&\n+\tcat >main.txt <<-EOF &&\n+\t\tCommit K\n+\t\tEOF\n+\tgit add main.txt &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit K\" &&\n+\tK=$(git rev-parse HEAD) &&\n+\n+\t# branch release\n+\tgit checkout -b release &&\n+\tcat >release.txt <<-EOF &&\n+\t\tCommit L\n+\t\tEOF\n+\tgit add release.txt &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit L\" &&\n+\n+\tcat >release.txt <<-EOF &&\n+\t\tCommit M\n+\t\tEOF\n+\tgit add -u &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit M\" &&\n+\ttest_tick &&\n+\tgit tag -m \"v2\" v2 HEAD &&\n+\n+\tcat >release.txt <<-EOF &&\n+\t\tCommit N\n+\t\tEOF\n+\tgit add -u &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit N\" &&\n+\ttest_tick &&\n+\tgit tag -m \"v3\" v3 HEAD &&\n+\tL=$(git rev-parse HEAD~2) &&\n+\tM=$(git rev-parse HEAD~) &&\n+\tN=$(git rev-parse HEAD) &&\n+\tTAG2=$(git rev-parse refs/tags/v2) &&\n+\tTAG3=$(git rev-parse refs/tags/v3) &&\n+\n+\t# merge commit O\n+\tgit checkout main &&\n+\ttest_tick &&\n+\tgit merge --no-ff --no-edit tags/v2 &&\n+\tO=$(git rev-parse HEAD) &&\n+\n+\t# commit P\n+\tgit checkout main &&\n+\tcat >main.txt <<-EOF &&\n+\t\tCommit P\n+\t\tEOF\n+\tgit add main.txt &&\n+\ttest_tick &&\n+\tgit commit -m \"Commit P\" &&\n+\tP=$(git rev-parse HEAD)\n+'\n+\n+test_expect_success 'create bundle from special rev: main^!' '\n+\tgit bundle create special-rev.bdl \"main^!\" &&\n+\n+\tgit bundle list-heads special-rev.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\t refs/heads/main\n+\t\tEOF\n+\ttest_i18ncmp expect actual &&\n+\n+\tgit bundle verify special-rev.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\tThe bundle contains this ref:\n+\t\t refs/heads/main\n+\t\tThe bundle requires this ref:\n+\t\t\n+\t\tEOF\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count special-rev.bdl 3\n+'\n+\n+test_expect_success 'create bundle with --max-count option' '\n+\tgit bundle create max-count.bdl --max-count 1 \\\n+\t\tmain \\\n+\t\t\"^release\" \\\n+\t\trefs/tags/v1 \\\n+\t\trefs/pull/1/head \\\n+\t\trefs/pull/2/head &&\n+\n+\tgit bundle list-heads max-count.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\t refs/heads/main\n+\t\t refs/tags/v1\n+\t\tEOF\n+\ttest_i18ncmp expect actual &&\n+\n+\tgit bundle verify max-count.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\tThe bundle contains these 2 refs:\n+\t\t refs/heads/main\n+\t\t refs/tags/v1\n+\t\tThe bundle requires this ref:\n+\t\t\n+\t\tEOF\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count max-count.bdl 4\n+'\n+\n+test_expect_success 'create bundle with --since option' '\n+\tgit bundle create since.bdl \\\n+\t\t--since \"Thu Apr 7 15:26:13 2005 -0700\" \\\n+\t\t--all &&\n+\n+\tgit bundle list-heads since.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\t refs/heads/main\n+\t\t refs/heads/release\n+\t\t refs/tags/v2\n+\t\t refs/tags/v3\n+\t\t HEAD\n+\t\tEOF\n+\ttest_i18ncmp expect actual &&\n+\n+\tgit bundle verify since.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\tThe bundle contains these 5 refs:\n+\t\t refs/heads/main\n+\t\t refs/heads/release\n+\t\t refs/tags/v2\n+\t\t refs/tags/v3\n+\t\t HEAD\n+\t\tThe bundle requires these 2 refs:\n+\t\t\n+\t\t\n+\t\tEOF\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_thin_bundle_object_count since.bdl 16\n+'\n+\n+test_expect_success 'create bundle 1 - no prerequisites' '\n+\tgit bundle create 1.bdl topic/1 topic/2 &&\n+\n+\tcat >expect <<-EOF &&\n+\t\tThe bundle contains these 2 refs:\n+\t\t refs/heads/topic/1\n+\t\t refs/heads/topic/2\n+\t\tThe bundle records a complete history.\n+\t\tEOF\n+\n+\t# verify bundle, which has no prerequisites\n+\tgit bundle verify 1.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 1.bdl 24\n+'\n+\n+test_expect_success 'create bundle 2 - has prerequisites' '\n+\tgit bundle create 2.bdl \\\n+\t\t--ignore-missing \\\n+\t\t^topic/deleted \\\n+\t\t^$D \\\n+\t\t^topic/2 \\\n+\t\trelease &&\n+\n+\tcat >expect <<-EOF &&\n+\t\tThe bundle contains this ref:\n+\t\t refs/heads/release\n+\t\tThe bundle requires these 3 refs:\n+\t\t\n+\t\t\n+\t\t\n+\t\tEOF\n+\n+\tgit bundle verify 2.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 2.bdl 16\n+'\n+\n+test_expect_success 'fail to verify bundle without prerequisites' '\n+\tgit init --bare test1.git &&\n+\n+\tcat >expect <<-EOF &&\n+\t\terror: Repository lacks these prerequisite commits:\n+\t\terror: \n+\t\terror: \n+\t\terror: \n+\t\tEOF\n+\n+\ttest_must_fail git -C test1.git bundle verify ../2.bdl 2>&1 |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual\n+'\n+\n+test_expect_success 'create bundle 3 - two refs, same object' '\n+\tgit bundle create --version=3 3.bdl \\\n+\t\t^release \\\n+\t\t^topic/1 \\\n+\t\t^topic/2 \\\n+\t\tmain \\\n+\t\tHEAD &&\n+\n+\tcat >expect <<-EOF &&\n+\t\tThe bundle contains these 2 refs:\n+\t\t refs/heads/main\n+\t\t HEAD\n+\t\tThe bundle requires these 2 refs:\n+\t\t\n+\t\t\n+\t\tEOF\n+\n+\tgit bundle verify 3.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 3.bdl 4\n+'\n+\n+test_expect_success 'create bundle 4 - with tags' '\n+\tgit bundle create 4.bdl \\\n+\t\t^main \\\n+\t\t^release \\\n+\t\t^topic/1 \\\n+\t\t^topic/2 \\\n+\t\t--all &&\n+\n+\tcat >expect <<-EOF &&\n+\t\tThe bundle contains these 3 refs:\n+\t\t refs/tags/v1\n+\t\t refs/tags/v2\n+\t\t refs/tags/v3\n+\t\tThe bundle records a complete history.\n+\t\tEOF\n+\n+\tgit bundle verify 4.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 4.bdl 3\n+'\n+\n+test_expect_success 'clone from bundle' '\n+\tgit clone --mirror 1.bdl mirror.git &&\n+\tgit -C mirror.git show-ref |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\t refs/heads/topic/1\n+\t\t refs/heads/topic/2\n+\t\tEOF\n+\ttest_cmp expect actual &&\n+\n+\tgit -C mirror.git fetch ../2.bdl \"+refs/*:refs/*\" &&\n+\tgit -C mirror.git show-ref |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\t refs/heads/release\n+\t\t refs/heads/topic/1\n+\t\t refs/heads/topic/2\n+\t\tEOF\n+\ttest_cmp expect actual &&\n+\n+\tgit -C mirror.git fetch ../3.bdl \"+refs/*:refs/*\" &&\n+\tgit -C mirror.git show-ref |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\t refs/heads/main\n+\t\t refs/heads/release\n+\t\t refs/heads/topic/1\n+\t\t refs/heads/topic/2\n+\t\tEOF\n+\ttest_cmp expect actual &&\n+\n+\tgit -C mirror.git fetch ../4.bdl \"+refs/*:refs/*\" &&\n+\tgit -C mirror.git show-ref |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\t refs/heads/main\n+\t\t refs/heads/release\n+\t\t refs/heads/topic/1\n+\t\t refs/heads/topic/2\n+\t\t refs/tags/v1\n+\t\t refs/tags/v2\n+\t\t refs/tags/v3\n+\t\tEOF\n+\ttest_cmp expect actual\n+'\n+\n+test_done\n-- \n2.30.0.2.g06d2f50715\n\n"},{"id":"413690","messageId":"20210107135025.2682-1-worldhello.net@gmail.com","threadId":"54923","inReplyTo":"xmqqft3g6ziz.fsf@gitster.c.googlers.com","subject":"[PATCH v3 0/2] improvements for git-bundle","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-01-07T13:50:23Z","receivedAt":"2021-01-07T13:51:29Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"From: Jiang Xin \n\n## Introduce two improvements for git-bundle\n\n+ Commit \"bundle: lost objects when removing duplicate pendings\",\n which fixes command like:\n\n $ git bundle create 'master^!'\n \n+ Commits \"bundle: arguments can be read from stdin\",\n which add \"--stdin\" option support for git-bundle, like:\n\n $ git bundle create verify.out &&\n ++\tbundle=$1 &&\n ++\tpack=${bundle%.bdl}.pack &&\n ++\tconvert_bundle_to_pack <\"$bundle\" >\"$pack\" &&\n ++\tgit index-pack \"$pack\" &&\n ++\tgit verify-pack -v \"$pack\" >verify.out &&\n ++\tcount=$(grep \"^$OID_REGEX \" verify.out | wc -l) &&\n ++\ttest $2 = $count && return 0\n ++\techo object count for $bundle is $count, not $2\n ++\treturn 1\n ++}\n ++\n ++\n ++test_thin_bundle_object_count () {\n ++\tbundle=$1 &&\n ++\tpack=${bundle%.bdl}.pack &&\n ++\tconvert_bundle_to_pack <\"$bundle\" |\n ++\t\ttest_must_fail git index-pack --stdin \"$pack\" &&\n ++\trm -f \"$pack\" &&\n ++\tconvert_bundle_to_pack <\"$bundle\" |\n ++\t\tgit index-pack --stdin --fix-thin \"$pack\" &&\n ++\tgit verify-pack -v \"$pack\" >verify.out &&\n +\tcount=$(grep \"^$OID_REGEX \" verify.out | wc -l) &&\n +\ttest $2 = $count && return 0\n -+\techo object count is $count, not $2\n ++\techo object count for $bundle is $count, not $2\n +\treturn 1\n +}\n +\n @@ t/t6020-bundle-misc.sh (new)\n +\n +# (C) (D, pull/1/head, topic/1)\n +# o --- o\n -+# / \\ (L, tags/v1)\n -+# / \\ o (H, topic/2) (M, tags/v2)\n -+# / (F) \\ / (N, tags/v3)\n ++# / \\ (L)\n ++# / \\ o (H, topic/2) (M, tag:v2)\n ++# / (F) \\ / (N, tag:v3)\n +# / o --------- o (G, pull/2/head) o --- o --- o (release)\n +# / / \\ \\ / \\\n +# o --- o --- o -------- o -- o ------------------ o ------- o --- o (main)\n -+# (A) (B) (E) (I) (J) (K) (O) (P)\n ++# (A) (B) (E, tag:v1) (I) (J) (K) (O) (P)\n +#\n +test_expect_success 'setup' '\n +\t# commit A & B\n @@ t/t6020-bundle-misc.sh (new)\n +\ttest_tick &&\n +\tgit commit -m \"Commit E\" &&\n +\tE=$(git rev-parse HEAD) &&\n ++\ttest_tick &&\n ++\tgit tag -m \"v1\" v1 HEAD &&\n ++\tTAG1=$(git rev-parse refs/tags/v1) &&\n +\n +\t# branch topic/2\n +\tgit checkout -b topic/2 &&\n @@ t/t6020-bundle-misc.sh (new)\n +\tgit add release.txt &&\n +\ttest_tick &&\n +\tgit commit -m \"Commit L\" &&\n -+\ttest_tick &&\n -+\tgit tag -m \"v1\" v1 HEAD &&\n +\n +\tcat >release.txt <<-EOF &&\n +\t\tCommit M\n @@ t/t6020-bundle-misc.sh (new)\n +\tL=$(git rev-parse HEAD~2) &&\n +\tM=$(git rev-parse HEAD~) &&\n +\tN=$(git rev-parse HEAD) &&\n -+\tTAG1=$(git rev-parse refs/tags/v1) &&\n +\tTAG2=$(git rev-parse refs/tags/v2) &&\n +\tTAG3=$(git rev-parse refs/tags/v3) &&\n +\n @@ t/t6020-bundle-misc.sh (new)\n +\t\tEOF\n +\ttest_i18ncmp expect actual &&\n +\n -+\tconvert_bundle_to_pack special-rev.pack &&\n -+\tgit index-pack special-rev.pack &&\n -+\ttest_bundle_object_count special-rev.pack 3\n ++\ttest_bundle_object_count special-rev.bdl 3\n ++'\n ++\n ++test_expect_success 'create bundle with --max-count option' '\n ++\tgit bundle create max-count.bdl --max-count 1 \\\n ++\t\tmain \\\n ++\t\t\"^release\" \\\n ++\t\trefs/tags/v1 \\\n ++\t\trefs/pull/1/head \\\n ++\t\trefs/pull/2/head &&\n ++\n ++\tgit bundle list-heads max-count.bdl |\n ++\t\tmake_user_friendly_and_stable_output >actual &&\n ++\tcat >expect <<-EOF &&\n ++\t\t refs/heads/main\n ++\t\t refs/tags/v1\n ++\t\tEOF\n ++\ttest_i18ncmp expect actual &&\n ++\n ++\tgit bundle verify max-count.bdl |\n ++\t\tmake_user_friendly_and_stable_output >actual &&\n ++\tcat >expect <<-EOF &&\n ++\t\tThe bundle contains these 2 refs:\n ++\t\t refs/heads/main\n ++\t\t refs/tags/v1\n ++\t\tThe bundle requires this ref:\n ++\t\t\n ++\t\tEOF\n ++\ttest_i18ncmp expect actual &&\n ++\n ++\ttest_bundle_object_count max-count.bdl 4\n ++'\n ++\n ++test_expect_success 'create bundle with --since option' '\n ++\tgit bundle create since.bdl \\\n ++\t\t--since \"Thu Apr 7 15:26:13 2005 -0700\" \\\n ++\t\t--all &&\n ++\n ++\tgit bundle list-heads since.bdl |\n ++\t\tmake_user_friendly_and_stable_output >actual &&\n ++\tcat >expect <<-EOF &&\n ++\t\t refs/heads/main\n ++\t\t refs/heads/release\n ++\t\t refs/tags/v2\n ++\t\t refs/tags/v3\n ++\t\t HEAD\n ++\t\tEOF\n ++\ttest_i18ncmp expect actual &&\n ++\n ++\tgit bundle verify since.bdl |\n ++\t\tmake_user_friendly_and_stable_output >actual &&\n ++\tcat >expect <<-EOF &&\n ++\t\tThe bundle contains these 5 refs:\n ++\t\t refs/heads/main\n ++\t\t refs/heads/release\n ++\t\t refs/tags/v2\n ++\t\t refs/tags/v3\n ++\t\t HEAD\n ++\t\tThe bundle requires these 2 refs:\n ++\t\t\n ++\t\t\n ++\t\tEOF\n ++\ttest_i18ncmp expect actual &&\n ++\n ++\ttest_thin_bundle_object_count since.bdl 16\n +'\n +\n +test_expect_success 'create bundle 1 - no prerequisites' '\n -+\t# create bundle from args\n +\tgit bundle create 1.bdl topic/1 topic/2 &&\n +\n +\tcat >expect <<-EOF &&\n @@ t/t6020-bundle-misc.sh (new)\n +\t\tmake_user_friendly_and_stable_output >actual &&\n +\ttest_i18ncmp expect actual &&\n +\n -+\tconvert_bundle_to_pack <1.bdl >1.pack &&\n -+\tgit index-pack 1.pack &&\n -+\ttest_bundle_object_count 1.pack 24\n ++\ttest_bundle_object_count 1.bdl 24\n +'\n +\n +test_expect_success 'create bundle 2 - has prerequisites' '\n -+\t# create bundle from args\n +\tgit bundle create 2.bdl \\\n +\t\t--ignore-missing \\\n +\t\t^topic/deleted \\\n @@ t/t6020-bundle-misc.sh (new)\n +\t\tmake_user_friendly_and_stable_output >actual &&\n +\ttest_i18ncmp expect actual &&\n +\n -+\tconvert_bundle_to_pack <2.bdl >2.pack &&\n -+\tgit index-pack 2.pack &&\n -+\ttest_bundle_object_count 2.pack 16\n ++\ttest_bundle_object_count 2.bdl 16\n +'\n +\n +test_expect_success 'fail to verify bundle without prerequisites' '\n @@ t/t6020-bundle-misc.sh (new)\n +'\n +\n +test_expect_success 'create bundle 3 - two refs, same object' '\n -+\t# create bundle from args\n +\tgit bundle create --version=3 3.bdl \\\n +\t\t^release \\\n +\t\t^topic/1 \\\n @@ t/t6020-bundle-misc.sh (new)\n +\t\tmake_user_friendly_and_stable_output >actual &&\n +\ttest_i18ncmp expect actual &&\n +\n -+\tconvert_bundle_to_pack <3.bdl >3.pack &&\n -+\tgit index-pack 3.pack &&\n -+\ttest_bundle_object_count 3.pack 4\n ++\ttest_bundle_object_count 3.bdl 4\n +'\n +\n +test_expect_success 'create bundle 4 - with tags' '\n -+\t# create bundle from args\n +\tgit bundle create 4.bdl \\\n +\t\t^main \\\n +\t\t^release \\\n @@ t/t6020-bundle-misc.sh (new)\n +\t\tmake_user_friendly_and_stable_output >actual &&\n +\ttest_i18ncmp expect actual &&\n +\n -+\tconvert_bundle_to_pack <4.bdl >4.pack &&\n -+\tgit index-pack 4.pack &&\n -+\ttest_bundle_object_count 4.pack 3\n ++\ttest_bundle_object_count 4.bdl 3\n +'\n +\n +test_expect_success 'clone from bundle' '\n2: a4662f44a8 ! 2: 86ad41e4d4 bundle: arguments can be read from stdin\n @@ Commit message\n `git-bundle`. Later nothing can be read from stdin when running\n `setup_revisions()` in `create_bundle()`.\n \n - Remove the entire `compute_and_write_prerequisites()` function, and\n - parse the args once by `setup_revisions()`. The first step for creating\n - a bundle is to write prerequisites (\"-\" obj-id SP comment LF), but after\n - calling `prepare_revision_walk()`, the `revs.pending` is left empty.\n - Following steps could not work properly without data in `revs.pending`.\n - Therefore, before calling `prepare_revision_walk()` function, make a\n - copy `revs_copy` from `revs` for later use. Even though `revs_copy` and\n - `revs` share the same objects, but changes on these objects will not\n - change the behavior of function `write_bundle_refs()` and\n - `write_pack_data()`.\n + The solution is to parse args once by removing the entire function\n + `compute_and_write_prerequisites()` and then calling function\n + `setup_revisions()`. In order to write prerequisites for bundle, will\n + call `prepare_revision_walk()` and `traverse_commit_list()`. But after\n + calling `prepare_revision_walk()`, the object array `revs.pending` is\n + left empty, and the following steps could not work properly with the\n + empty object array (`revs.pending`). Therefore, make a copy of `revs`\n + to `revs_copy` for later use right after calling `setup_revisions()`.\n +\n + The copy of `revs_copy` is not a deep copy, it shares the same objects\n + with `revs`. The object array of `revs` has been cleared, but objects\n + themselves are still kept. Flags of objects may change after calling\n + `prepare_revision_walk()`, we can use these changed flags without\n + calling the `git rev-list` command and parsing its output like the\n + former implementation.\n \n Also add testcases for git bundle in t6020, which read args from stdin.\n \n @@ bundle.c: static int write_pack_data(int bundle_fd, struct rev_info *revs, struc\n /*\n * Write out bundle refs based on the tips already\n * parsed into revs.pending. As a side effect, may\n +@@ bundle.c: static int write_bundle_refs(int bundle_fd, struct rev_info *revs)\n + \t\t * constraints.\n + \t\t */\n + \t\tif (!(e->item->flags & SHOWN) && e->item->type == OBJ_COMMIT) {\n +-\t\t\twarning(_(\"ref '%s' is excluded by the rev-list options\"),\n ++\t\t\twarning(_(\"ref '%s' is excluded by the limiting options\"),\n + \t\t\t\te->name);\n + \t\t\tgoto skip_write_ref;\n + \t\t}\n @@ bundle.c: static int write_bundle_refs(int bundle_fd, struct rev_info *revs)\n \treturn ref_count;\n }\n @@ t/t5607-clone-bundle.sh: test_expect_success 'die if bundle file cannot be creat\n \tgrep master output\n \n ## t/t6020-bundle-misc.sh ##\n -@@ t/t6020-bundle-misc.sh: test_expect_success 'create bundle 1 - no prerequisites' '\n - \t# create bundle from args\n +@@ t/t6020-bundle-misc.sh: test_expect_success 'create bundle with --since option' '\n + '\n + \n + test_expect_success 'create bundle 1 - no prerequisites' '\n ++\t# create bundle from args\n \tgit bundle create 1.bdl topic/1 topic/2 &&\n \n +\t# create bundle from stdin\n @@ t/t6020-bundle-misc.sh: test_expect_success 'create bundle 1 - no prerequisites'\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual &&\n \n +-\ttest_bundle_object_count 1.bdl 24\n +\tgit bundle verify stdin-1.bdl |\n +\t\tmake_user_friendly_and_stable_output >actual &&\n +\ttest_i18ncmp expect actual &&\n +\n - \tconvert_bundle_to_pack <1.bdl >1.pack &&\n - \tgit index-pack 1.pack &&\n --\ttest_bundle_object_count 1.pack 24\n -+\ttest_bundle_object_count 1.pack 24 &&\n -+\n -+\tconvert_bundle_to_pack stdin-1.pack &&\n -+\tgit index-pack stdin-1.pack &&\n -+\ttest_bundle_object_count stdin-1.pack 24\n ++\ttest_bundle_object_count 1.bdl 24 &&\n ++\ttest_bundle_object_count stdin-1.bdl 24\n '\n \n test_expect_success 'create bundle 2 - has prerequisites' '\n ++\t# create bundle from args\n + \tgit bundle create 2.bdl \\\n + \t\t--ignore-missing \\\n + \t\t^topic/deleted \\\n @@ t/t6020-bundle-misc.sh: test_expect_success 'create bundle 2 - has prerequisites' '\n \t\t^topic/2 \\\n \t\trelease &&\n @@ t/t6020-bundle-misc.sh: test_expect_success 'create bundle 2 - has prerequisites\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual &&\n \n +-\ttest_bundle_object_count 2.bdl 16\n +\tgit bundle verify stdin-2.bdl |\n +\t\tmake_user_friendly_and_stable_output >actual &&\n +\ttest_i18ncmp expect actual &&\n +\n - \tconvert_bundle_to_pack <2.bdl >2.pack &&\n - \tgit index-pack 2.pack &&\n --\ttest_bundle_object_count 2.pack 16\n -+\ttest_bundle_object_count 2.pack 16 &&\n -+\n -+\tconvert_bundle_to_pack stdin-2.pack &&\n -+\tgit index-pack stdin-2.pack &&\n -+\ttest_bundle_object_count stdin-2.pack 16\n ++\ttest_bundle_object_count 2.bdl 16 &&\n ++\ttest_bundle_object_count stdin-2.bdl 16\n '\n \n test_expect_success 'fail to verify bundle without prerequisites' '\n @@ t/t6020-bundle-misc.sh: test_expect_success 'fail to verify bundle without prere\n \ttest_i18ncmp expect actual\n '\n \n + test_expect_success 'create bundle 3 - two refs, same object' '\n ++\t# create bundle from args\n + \tgit bundle create --version=3 3.bdl \\\n + \t\t^release \\\n + \t\t^topic/1 \\\n @@ t/t6020-bundle-misc.sh: test_expect_success 'create bundle 3 - two refs, same object' '\n \t\tmain \\\n \t\tHEAD &&\n @@ t/t6020-bundle-misc.sh: test_expect_success 'create bundle 3 - two refs, same ob\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual &&\n \n +-\ttest_bundle_object_count 3.bdl 4\n +\tgit bundle verify stdin-3.bdl |\n +\t\tmake_user_friendly_and_stable_output >actual &&\n +\ttest_i18ncmp expect actual &&\n +\n - \tconvert_bundle_to_pack <3.bdl >3.pack &&\n - \tgit index-pack 3.pack &&\n --\ttest_bundle_object_count 3.pack 4\n -+\ttest_bundle_object_count 3.pack 4 &&\n -+\n -+\tconvert_bundle_to_pack stdin-3.pack &&\n -+\tgit index-pack stdin-3.pack &&\n -+\ttest_bundle_object_count stdin-3.pack 4\n ++\ttest_bundle_object_count 3.bdl 4 &&\n ++\ttest_bundle_object_count stdin-3.bdl 4\n '\n \n test_expect_success 'create bundle 4 - with tags' '\n ++\t# create bundle from args\n + \tgit bundle create 4.bdl \\\n + \t\t^main \\\n + \t\t^release \\\n @@ t/t6020-bundle-misc.sh: test_expect_success 'create bundle 4 - with tags' '\n \t\t^topic/2 \\\n \t\t--all &&\n @@ t/t6020-bundle-misc.sh: test_expect_success 'create bundle 4 - with tags' '\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual &&\n \n +-\ttest_bundle_object_count 4.bdl 3\n +\tgit bundle verify stdin-4.bdl |\n +\t\tmake_user_friendly_and_stable_output >actual &&\n +\ttest_i18ncmp expect actual &&\n +\n - \tconvert_bundle_to_pack <4.bdl >4.pack &&\n - \tgit index-pack 4.pack &&\n --\ttest_bundle_object_count 4.pack 3\n -+\ttest_bundle_object_count 4.pack 3 &&\n -+\n -+\tconvert_bundle_to_pack stdin-4.pack &&\n -+\tgit index-pack stdin-4.pack &&\n -+\ttest_bundle_object_count stdin-4.pack 3\n ++\ttest_bundle_object_count 4.bdl 3 &&\n ++\ttest_bundle_object_count stdin-4.bdl 3\n '\n \n test_expect_success 'clone from bundle' '\n\n--\n\nJiang Xin (2):\n bundle: lost objects when removing duplicate pendings\n bundle: arguments can be read from stdin\n\n bundle.c | 111 ++++----\n object.c | 10 +-\n t/t5607-clone-bundle.sh | 4 +-\n t/t6020-bundle-misc.sh | 557 ++++++++++++++++++++++++++++++++++++++++\n 4 files changed, 625 insertions(+), 57 deletions(-)\n create mode 100755 t/t6020-bundle-misc.sh\n\n-- \n2.30.0.2.g06d2f50715\n\n"},{"id":"413699","messageId":"X/cqrTgilKAW9P9G@danh.dev","threadId":"54923","inReplyTo":"20210107135025.2682-2-worldhello.net@gmail.com","subject":"Re: [PATCH v3 1/2] bundle: lost objects when removing duplicate pendings","fromName":"Đoàn Trần Công Danh","fromEmail":"congdanhqx@gmail.com","sentAt":"2021-01-07T15:37:17Z","receivedAt":"2021-01-07T15:38:02Z","isPatch":true,"sender":{"key":"congdanhqx@gmail.com","avatar":"https://avatars.githubusercontent.com/u/42673067?v=4"},"body":"On 2021-01-07 08:50:24-0500, Jiang Xin wrote:\n> From: Jiang Xin \n> \n> `git rev-list` will list one commit for the following command:\n> \n> $ git rev-list 'main^!'\n> \n> \n> But providing the same rev-list args to `git bundle`, fail to create\n> a bundle file.\n> \n> $ git bundle create - 'main^!'\n> # v2 git bundle\n> - \n> \n> fatal: Refusing to create empty bundle.\n> \n> This is because when removing duplicate objects in function\n> `object_array_remove_duplicates()`, one unique pending object which has\n> the same name is deleted by mistake. The revision arg 'main^!' in the\n> above example is parsed by `handle_revision_arg()`, and at lease two\n> different objects will be appended to `revs.pending`, one points to the\n> parent commit of the \"main\" branch, and the other points to the tip\n> commit of the \"main\" branch. These two objects have the same name\n> \"main\". Only one object is left with the name \"main\" after calling the\n> function `object_array_remove_duplicates()`.\n> \n> And what's worse, when adding boundary commits into pending list, we use\n> one-line commit message as names, and the arbitory names may surprise\n> git-bundle.\n> \n> Only comparing objects themselves (\".item\") is also not good enough,\n> because user may want to create a bundle with two identical objects but\n> with different reference names, such as: \"HEAD\" and \"refs/heads/main\".\n> \n> Add new function `contains_object()` which compare both the address and\n> the name of the object.\n> \n> Signed-off-by: Jiang Xin \n> ---\n> object.c | 10 +-\n> t/t6020-bundle-misc.sh | 488 +++++++++++++++++++++++++++++++++++++++++\n> 2 files changed, 494 insertions(+), 4 deletions(-)\n> create mode 100755 t/t6020-bundle-misc.sh\n> \n> diff --git a/object.c b/object.c\n> index 68f80b0b3d..98017bed8e 100644\n> --- a/object.c\n> +++ b/object.c\n> @@ -412,15 +412,16 @@ void object_array_clear(struct object_array *array)\n> }\n> \n> /*\n> - * Return true iff array already contains an entry with name.\n> + * Return true if array already contains an entry.\n> */\n> -static int contains_name(struct object_array *array, const char *name)\n> +static int contains_object(struct object_array *array,\n> +\t\t\t const struct object *item, const char *name)\n> {\n> \tunsigned nr = array->nr, i;\n> \tstruct object_array_entry *object = array->objects;\n> \n> \tfor (i = 0; i < nr; i++, object++)\n> -\t\tif (!strcmp(object->name, name))\n> +\t\tif (item == object->item && !strcmp(object->name, name))\n\nI think the comparison of `item == object->item` is a bit too fragile.\nYes, we all know `item` must be an entry of array.\nHowever, it's separated from its usage may lead to misuse in the\nfuture. Perhaps using `oidcmp` or adding a comment to indicate that\n`item` must be an entry of `array`.\n\n> \t\t\treturn 1;\n> \treturn 0;\n> }\n> @@ -432,7 +433,8 @@ void object_array_remove_duplicates(struct object_array *array)\n> \n> \tarray->nr = 0;\n> \tfor (src = 0; src < nr; src++) {\n> -\t\tif (!contains_name(array, objects[src].name)) {\n> +\t\tif (!contains_object(array, objects[src].item,\n> +\t\t\t\t objects[src].name)) {\n> \t\t\tif (src != array->nr)\n> \t\t\t\tobjects[array->nr] = objects[src];\n> \t\t\tarray->nr++;\n> diff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh\n> new file mode 100755\n> index 0000000000..d15e67c8f7\n> --- /dev/null\n> +++ b/t/t6020-bundle-misc.sh\n> @@ -0,0 +1,488 @@\n> +#!/bin/sh\n> +#\n> +# Copyright (c) 2021 Jiang Xin\n> +#\n> +\n> +test_description='Test git-bundle'\n> +\n> +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n> +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n> +\n> +. ./test-lib.sh\n> +\n> +test_bundle_object_count () {\n> +\tbundle=$1 &&\n> +\tpack=${bundle%.bdl}.pack &&\n> +\tconvert_bundle_to_pack <\"$bundle\" >\"$pack\" &&\n> +\tgit index-pack \"$pack\" &&\n> +\tgit verify-pack -v \"$pack\" >verify.out &&\n> +\tcount=$(grep \"^$OID_REGEX \" verify.out | wc -l) &&\n\nI think we can use 'grep -c' instead of `grep .. | wc -l`.\n\nOr\n\n\tgrep \"^$OID_REGEX \" verify.out >verify.filtered &&\n\ttest_line_count = $2 verify.filtered\n\nThe same comment applied to test_thin_bundle_object_count\n\n> +\ttest $2 = $count && return 0\n> +\techo object count for $bundle is $count, not $2\n> +\treturn 1\n> +}\n> +\n> +\n> +test_thin_bundle_object_count () {\n> +\tbundle=$1 &&\n> +\tpack=${bundle%.bdl}.pack &&\n> +\tconvert_bundle_to_pack <\"$bundle\" |\n> +\t\ttest_must_fail git index-pack --stdin \"$pack\" &&\n> +\trm -f \"$pack\" &&\n> +\tconvert_bundle_to_pack <\"$bundle\" |\n> +\t\tgit index-pack --stdin --fix-thin \"$pack\" &&\n> +\tgit verify-pack -v \"$pack\" >verify.out &&\n> +\tcount=$(grep \"^$OID_REGEX \" verify.out | wc -l) &&\n> +\ttest $2 = $count && return 0\n> +\techo object count for $bundle is $count, not $2\n> +\treturn 1\n> +}\n> +\n> +convert_bundle_to_pack () {\n> +\twhile read x && test -n \"$x\"\n> +\tdo\n> +\t\t:;\n> +\tdone\n> +\tcat\n\nI'm not sure what you would expect in this case,\nbut in my experience, I replace this whole block with\n\n\tsed '1,/^$/d'\n\nalso works.\n\nIOW, I would apply this on top of your change:\n\n----8<-----\ndiff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh\nindex 1f60fe180e..3a428454d7 100755\n--- a/t/t6020-bundle-misc.sh\n+++ b/t/t6020-bundle-misc.sh\n@@ -16,10 +16,8 @@ test_bundle_object_count () {\n \tconvert_bundle_to_pack <\"$bundle\" >\"$pack\" &&\n \tgit index-pack \"$pack\" &&\n \tgit verify-pack -v \"$pack\" >verify.out &&\n-\tcount=$(grep \"^$OID_REGEX \" verify.out | wc -l) &&\n-\ttest $2 = $count && return 0\n-\techo object count for $bundle is $count, not $2\n-\treturn 1\n+\tgrep \"^$OID_REGEX \" verify.out >verify.filtered &&\n+\ttest_line_count = $2 verify.filtered\n }\n \n \n@@ -32,18 +30,12 @@ test_thin_bundle_object_count () {\n \tconvert_bundle_to_pack <\"$bundle\" |\n \t\tgit index-pack --stdin --fix-thin \"$pack\" &&\n \tgit verify-pack -v \"$pack\" >verify.out &&\n-\tcount=$(grep \"^$OID_REGEX \" verify.out | wc -l) &&\n-\ttest $2 = $count && return 0\n-\techo object count for $bundle is $count, not $2\n-\treturn 1\n+\tgrep \"^$OID_REGEX \" verify.out >verify.filtered &&\n+\ttest_line_count = $2 verify.filtered\n }\n \n convert_bundle_to_pack () {\n-\twhile read x && test -n \"$x\"\n-\tdo\n-\t\t:;\n-\tdone\n-\tcat\n+\tsed '1,/^$/d'\n }\n \n # Format the output of git commands to make a user-friendly and stable\n----->8-----\n\nFor the below change, I haven't checked but I think test_commit should work, no?\n\n-- Danh\n\n> +}\n> +\n> +# Format the output of git commands to make a user-friendly and stable\n> +# text. We can easily prepare the expect text without having to worry\n> +# about future changes of the commit ID and spaces of the output.\n> +make_user_friendly_and_stable_output () {\n> +\tsed \\\n> +\t\t-e \"s/ *\\$//\" \\\n> +\t\t-e \"s/$A//g\" \\\n> +\t\t-e \"s/$B//g\" \\\n> +\t\t-e \"s/$C//g\" \\\n> +\t\t-e \"s/$D//g\" \\\n> +\t\t-e \"s/$E//g\" \\\n> +\t\t-e \"s/$F//g\" \\\n> +\t\t-e \"s/$G//g\" \\\n> +\t\t-e \"s/$H//g\" \\\n> +\t\t-e \"s/$I//g\" \\\n> +\t\t-e \"s/$J//g\" \\\n> +\t\t-e \"s/$K//g\" \\\n> +\t\t-e \"s/$L//g\" \\\n> +\t\t-e \"s/$M//g\" \\\n> +\t\t-e \"s/$N//g\" \\\n> +\t\t-e \"s/$O//g\" \\\n> +\t\t-e \"s/$P//g\" \\\n> +\t\t-e \"s/$(echo $A | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $B | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $C | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $D | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $E | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $F | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $G | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $H | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $I | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $J | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $K | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $L | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $M | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $N | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $O | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $P | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$TAG1//g\" \\\n> +\t\t-e \"s/$TAG2//g\" \\\n> +\t\t-e \"s/$TAG3//g\" \\\n> +\t\t-e \"s/$(echo $TAG1 | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $TAG2 | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $TAG3 | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$ZERO_OID//g\"\n> +}\n> +\n> +# (C) (D, pull/1/head, topic/1)\n> +# o --- o\n> +# / \\ (L)\n> +# / \\ o (H, topic/2) (M, tag:v2)\n> +# / (F) \\ / (N, tag:v3)\n> +# / o --------- o (G, pull/2/head) o --- o --- o (release)\n> +# / / \\ \\ / \\\n> +# o --- o --- o -------- o -- o ------------------ o ------- o --- o (main)\n> +# (A) (B) (E, tag:v1) (I) (J) (K) (O) (P)\n> +#\n> +test_expect_success 'setup' '\n> +\t# commit A & B\n> +\tcat >main.txt <<-EOF &&\n> +\t\tCommit A\n> +\t\tEOF\n> +\tgit add main.txt &&\n> +\ttest_tick &&\n> +\tgit commit -m \"Commit A\" &&\n> +\n> +\tcat >main.txt <<-EOF &&\n> +\t\tCommit B\n> +\t\tEOF\n> +\tgit add main.txt &&\n> +\ttest_tick &&\n> +\tgit commit -m \"Commit B\" &&\n> +\tA=$(git rev-parse HEAD~) &&\n> +\tB=$(git rev-parse HEAD) &&\n> +\n> +\t# branch topic/1\n> +\tgit checkout -b topic/1 &&\n> +\tcat >topic-1.txt <<-EOF &&\n> +\t\tCommit C\n> +\t\tEOF\n> +\tgit add topic-1.txt &&\n> +\ttest_tick &&\n> +\tgit commit -m \"Commit C\" &&\n> +\n> +\tcat >topic-1.txt <<-EOF &&\n> +\t\tCommit D\n> +\t\tEOF\n> +\tgit add -u &&\n> +\ttest_tick &&\n> +\tgit commit -m \"Commit D\" &&\n> +\tgit update-ref refs/pull/1/head HEAD &&\n> +\tC=$(git rev-parse topic/1~) &&\n> +\tD=$(git rev-parse topic/1) &&\n> +\n> +\t# commit E\n> +\tgit checkout main &&\n> +\tcat >main.txt <<-EOF &&\n> +\t\tCommit E\n> +\t\tEOF\n> +\tgit add main.txt &&\n> +\ttest_tick &&\n> +\tgit commit -m \"Commit E\" &&\n> +\tE=$(git rev-parse HEAD) &&\n> +\ttest_tick &&\n> +\tgit tag -m \"v1\" v1 HEAD &&\n> +\tTAG1=$(git rev-parse refs/tags/v1) &&\n> +\n> +\t# branch topic/2\n> +\tgit checkout -b topic/2 &&\n> +\tcat >topic-2.txt <<-EOF &&\n> +\t\tCommit F\n> +\t\tEOF\n> +\tgit add topic-2.txt &&\n> +\ttest_tick &&\n> +\tgit commit -m \"Commit F\" &&\n> +\n> +\tcat >topic-2.txt <<-EOF &&\n> +\t\tCommit G\n> +\t\tEOF\n> +\tgit add -u &&\n> +\ttest_tick &&\n> +\tgit commit -m \"Commit G\" &&\n> +\tgit update-ref refs/pull/2/head HEAD &&\n> +\n> +\tcat >topic-2.txt <<-EOF &&\n> +\t\tCommit H\n> +\t\tEOF\n> +\tgit add -u &&\n> +\ttest_tick &&\n> +\tgit commit -m \"Commit H\" &&\n> +\tF=$(git rev-parse topic/2~2) &&\n> +\tG=$(git rev-parse topic/2~) &&\n> +\tH=$(git rev-parse topic/2) &&\n> +\n> +\t# merge commit I & J\n> +\tgit checkout main &&\n> +\ttest_tick &&\n> +\tgit merge --no-ff --no-edit topic/1 &&\n> +\ttest_tick &&\n> +\tgit merge --no-ff --no-edit refs/pull/2/head &&\n> +\tI=$(git rev-parse HEAD~) &&\n> +\tJ=$(git rev-parse HEAD) &&\n> +\n> +\t# commit K\n> +\tgit checkout main &&\n> +\tcat >main.txt <<-EOF &&\n> +\t\tCommit K\n> +\t\tEOF\n> +\tgit add main.txt &&\n> +\ttest_tick &&\n> +\tgit commit -m \"Commit K\" &&\n> +\tK=$(git rev-parse HEAD) &&\n> +\n> +\t# branch release\n> +\tgit checkout -b release &&\n> +\tcat >release.txt <<-EOF &&\n> +\t\tCommit L\n> +\t\tEOF\n> +\tgit add release.txt &&\n> +\ttest_tick &&\n> +\tgit commit -m \"Commit L\" &&\n> +\n> +\tcat >release.txt <<-EOF &&\n> +\t\tCommit M\n> +\t\tEOF\n> +\tgit add -u &&\n> +\ttest_tick &&\n> +\tgit commit -m \"Commit M\" &&\n> +\ttest_tick &&\n> +\tgit tag -m \"v2\" v2 HEAD &&\n> +\n> +\tcat >release.txt <<-EOF &&\n> +\t\tCommit N\n> +\t\tEOF\n> +\tgit add -u &&\n> +\ttest_tick &&\n> +\tgit commit -m \"Commit N\" &&\n> +\ttest_tick &&\n> +\tgit tag -m \"v3\" v3 HEAD &&\n> +\tL=$(git rev-parse HEAD~2) &&\n> +\tM=$(git rev-parse HEAD~) &&\n> +\tN=$(git rev-parse HEAD) &&\n> +\tTAG2=$(git rev-parse refs/tags/v2) &&\n> +\tTAG3=$(git rev-parse refs/tags/v3) &&\n> +\n> +\t# merge commit O\n> +\tgit checkout main &&\n> +\ttest_tick &&\n> +\tgit merge --no-ff --no-edit tags/v2 &&\n> +\tO=$(git rev-parse HEAD) &&\n> +\n> +\t# commit P\n> +\tgit checkout main &&\n> +\tcat >main.txt <<-EOF &&\n> +\t\tCommit P\n> +\t\tEOF\n> +\tgit add main.txt &&\n> +\ttest_tick &&\n> +\tgit commit -m \"Commit P\" &&\n> +\tP=$(git rev-parse HEAD)\n> +'\n> +\n> +test_expect_success 'create bundle from special rev: main^!' '\n> +\tgit bundle create special-rev.bdl \"main^!\" &&\n> +\n> +\tgit bundle list-heads special-rev.bdl |\n> +\t\tmake_user_friendly_and_stable_output >actual &&\n> +\tcat >expect <<-EOF &&\n> +\t\t refs/heads/main\n> +\t\tEOF\n> +\ttest_i18ncmp expect actual &&\n> +\n> +\tgit bundle verify special-rev.bdl |\n> +\t\tmake_user_friendly_and_stable_output >actual &&\n> +\tcat >expect <<-EOF &&\n> +\t\tThe bundle contains this ref:\n> +\t\t refs/heads/main\n> +\t\tThe bundle requires this ref:\n> +\t\t\n> +\t\tEOF\n> +\ttest_i18ncmp expect actual &&\n> +\n> +\ttest_bundle_object_count special-rev.bdl 3\n> +'\n> +\n> +test_expect_success 'create bundle with --max-count option' '\n> +\tgit bundle create max-count.bdl --max-count 1 \\\n> +\t\tmain \\\n> +\t\t\"^release\" \\\n> +\t\trefs/tags/v1 \\\n> +\t\trefs/pull/1/head \\\n> +\t\trefs/pull/2/head &&\n> +\n> +\tgit bundle list-heads max-count.bdl |\n> +\t\tmake_user_friendly_and_stable_output >actual &&\n> +\tcat >expect <<-EOF &&\n> +\t\t refs/heads/main\n> +\t\t refs/tags/v1\n> +\t\tEOF\n> +\ttest_i18ncmp expect actual &&\n> +\n> +\tgit bundle verify max-count.bdl |\n> +\t\tmake_user_friendly_and_stable_output >actual &&\n> +\tcat >expect <<-EOF &&\n> +\t\tThe bundle contains these 2 refs:\n> +\t\t refs/heads/main\n> +\t\t refs/tags/v1\n> +\t\tThe bundle requires this ref:\n> +\t\t\n> +\t\tEOF\n> +\ttest_i18ncmp expect actual &&\n> +\n> +\ttest_bundle_object_count max-count.bdl 4\n> +'\n> +\n> +test_expect_success 'create bundle with --since option' '\n> +\tgit bundle create since.bdl \\\n> +\t\t--since \"Thu Apr 7 15:26:13 2005 -0700\" \\\n> +\t\t--all &&\n> +\n> +\tgit bundle list-heads since.bdl |\n> +\t\tmake_user_friendly_and_stable_output >actual &&\n> +\tcat >expect <<-EOF &&\n> +\t\t refs/heads/main\n> +\t\t refs/heads/release\n> +\t\t refs/tags/v2\n> +\t\t refs/tags/v3\n> +\t\t HEAD\n> +\t\tEOF\n> +\ttest_i18ncmp expect actual &&\n> +\n> +\tgit bundle verify since.bdl |\n> +\t\tmake_user_friendly_and_stable_output >actual &&\n> +\tcat >expect <<-EOF &&\n> +\t\tThe bundle contains these 5 refs:\n> +\t\t refs/heads/main\n> +\t\t refs/heads/release\n> +\t\t refs/tags/v2\n> +\t\t refs/tags/v3\n> +\t\t HEAD\n> +\t\tThe bundle requires these 2 refs:\n> +\t\t\n> +\t\t\n> +\t\tEOF\n> +\ttest_i18ncmp expect actual &&\n> +\n> +\ttest_thin_bundle_object_count since.bdl 16\n> +'\n> +\n> +test_expect_success 'create bundle 1 - no prerequisites' '\n> +\tgit bundle create 1.bdl topic/1 topic/2 &&\n> +\n> +\tcat >expect <<-EOF &&\n> +\t\tThe bundle contains these 2 refs:\n> +\t\t refs/heads/topic/1\n> +\t\t refs/heads/topic/2\n> +\t\tThe bundle records a complete history.\n> +\t\tEOF\n> +\n> +\t# verify bundle, which has no prerequisites\n> +\tgit bundle verify 1.bdl |\n> +\t\tmake_user_friendly_and_stable_output >actual &&\n> +\ttest_i18ncmp expect actual &&\n> +\n> +\ttest_bundle_object_count 1.bdl 24\n> +'\n> +\n> +test_expect_success 'create bundle 2 - has prerequisites' '\n> +\tgit bundle create 2.bdl \\\n> +\t\t--ignore-missing \\\n> +\t\t^topic/deleted \\\n> +\t\t^$D \\\n> +\t\t^topic/2 \\\n> +\t\trelease &&\n> +\n> +\tcat >expect <<-EOF &&\n> +\t\tThe bundle contains this ref:\n> +\t\t refs/heads/release\n> +\t\tThe bundle requires these 3 refs:\n> +\t\t\n> +\t\t\n> +\t\t\n> +\t\tEOF\n> +\n> +\tgit bundle verify 2.bdl |\n> +\t\tmake_user_friendly_and_stable_output >actual &&\n> +\ttest_i18ncmp expect actual &&\n> +\n> +\ttest_bundle_object_count 2.bdl 16\n> +'\n> +\n> +test_expect_success 'fail to verify bundle without prerequisites' '\n> +\tgit init --bare test1.git &&\n> +\n> +\tcat >expect <<-EOF &&\n> +\t\terror: Repository lacks these prerequisite commits:\n> +\t\terror: \n> +\t\terror: \n> +\t\terror: \n> +\t\tEOF\n> +\n> +\ttest_must_fail git -C test1.git bundle verify ../2.bdl 2>&1 |\n> +\t\tmake_user_friendly_and_stable_output >actual &&\n> +\ttest_i18ncmp expect actual\n> +'\n> +\n> +test_expect_success 'create bundle 3 - two refs, same object' '\n> +\tgit bundle create --version=3 3.bdl \\\n> +\t\t^release \\\n> +\t\t^topic/1 \\\n> +\t\t^topic/2 \\\n> +\t\tmain \\\n> +\t\tHEAD &&\n> +\n> +\tcat >expect <<-EOF &&\n> +\t\tThe bundle contains these 2 refs:\n> +\t\t refs/heads/main\n> +\t\t HEAD\n> +\t\tThe bundle requires these 2 refs:\n> +\t\t\n> +\t\t\n> +\t\tEOF\n> +\n> +\tgit bundle verify 3.bdl |\n> +\t\tmake_user_friendly_and_stable_output >actual &&\n> +\ttest_i18ncmp expect actual &&\n> +\n> +\ttest_bundle_object_count 3.bdl 4\n> +'\n> +\n> +test_expect_success 'create bundle 4 - with tags' '\n> +\tgit bundle create 4.bdl \\\n> +\t\t^main \\\n> +\t\t^release \\\n> +\t\t^topic/1 \\\n> +\t\t^topic/2 \\\n> +\t\t--all &&\n> +\n> +\tcat >expect <<-EOF &&\n> +\t\tThe bundle contains these 3 refs:\n> +\t\t refs/tags/v1\n> +\t\t refs/tags/v2\n> +\t\t refs/tags/v3\n> +\t\tThe bundle records a complete history.\n> +\t\tEOF\n> +\n> +\tgit bundle verify 4.bdl |\n> +\t\tmake_user_friendly_and_stable_output >actual &&\n> +\ttest_i18ncmp expect actual &&\n> +\n> +\ttest_bundle_object_count 4.bdl 3\n> +'\n> +\n> +test_expect_success 'clone from bundle' '\n> +\tgit clone --mirror 1.bdl mirror.git &&\n> +\tgit -C mirror.git show-ref |\n> +\t\tmake_user_friendly_and_stable_output >actual &&\n> +\tcat >expect <<-EOF &&\n> +\t\t refs/heads/topic/1\n> +\t\t refs/heads/topic/2\n> +\t\tEOF\n> +\ttest_cmp expect actual &&\n> +\n> +\tgit -C mirror.git fetch ../2.bdl \"+refs/*:refs/*\" &&\n> +\tgit -C mirror.git show-ref |\n> +\t\tmake_user_friendly_and_stable_output >actual &&\n> +\tcat >expect <<-EOF &&\n> +\t\t refs/heads/release\n> +\t\t refs/heads/topic/1\n> +\t\t refs/heads/topic/2\n> +\t\tEOF\n> +\ttest_cmp expect actual &&\n> +\n> +\tgit -C mirror.git fetch ../3.bdl \"+refs/*:refs/*\" &&\n> +\tgit -C mirror.git show-ref |\n> +\t\tmake_user_friendly_and_stable_output >actual &&\n> +\tcat >expect <<-EOF &&\n> +\t\t refs/heads/main\n> +\t\t refs/heads/release\n> +\t\t refs/heads/topic/1\n> +\t\t refs/heads/topic/2\n> +\t\tEOF\n> +\ttest_cmp expect actual &&\n> +\n> +\tgit -C mirror.git fetch ../4.bdl \"+refs/*:refs/*\" &&\n> +\tgit -C mirror.git show-ref |\n> +\t\tmake_user_friendly_and_stable_output >actual &&\n> +\tcat >expect <<-EOF &&\n> +\t\t refs/heads/main\n> +\t\t refs/heads/release\n> +\t\t refs/heads/topic/1\n> +\t\t refs/heads/topic/2\n> +\t\t refs/tags/v1\n> +\t\t refs/tags/v2\n> +\t\t refs/tags/v3\n> +\t\tEOF\n> +\ttest_cmp expect actual\n> +'\n> +\n> +test_done\n> -- \n> 2.30.0.2.g06d2f50715\n> \n\n-- \nDanh\n"},{"id":"413814","messageId":"CANYiYbG84GuwnU5QEg0NoMfYS0fWzfDbz-yruA2fr2yQfyfMhg@mail.gmail.com","threadId":"54923","inReplyTo":"X/cqrTgilKAW9P9G@danh.dev","subject":"Re: [PATCH v3 1/2] bundle: lost objects when removing duplicate pendings","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-01-08T13:14:22Z","receivedAt":"2021-01-08T13:15:31Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"Đoàn Trần Công Danh 于2021年1月7日周四 下午11:37写道：\n>\n> > -static int contains_name(struct object_array *array, const char *name)\n> > +static int contains_object(struct object_array *array,\n> > + const struct object *item, const char *name)\n> > {\n> > unsigned nr = array->nr, i;\n> > struct object_array_entry *object = array->objects;\n> >\n> > for (i = 0; i < nr; i++, object++)\n> > - if (!strcmp(object->name, name))\n> > + if (item == object->item && !strcmp(object->name, name))\n>\n> I think the comparison of `item == object->item` is a bit too fragile.\n> Yes, we all know `item` must be an entry of array.\n> However, it's separated from its usage may lead to misuse in the\n> future. Perhaps using `oidcmp` or adding a comment to indicate that\n> `item` must be an entry of `array`.\n\nYou can find the same usage on comparing address of objects in other places:\n\n+ https://github.com/git/git/blob/v2.30.0/bundle.c#L447\n+ https://github.com/git/git/blob/v2.30.0/commit.c#L954\n\nBoth `item` and `object->item` point to address of a shared object in\nparsed_object_pool of the repository, we can compare two objects by\nchecking address of them safely.\n\n> > +test_bundle_object_count () {\n> > + bundle=$1 &&\n> > + pack=${bundle%.bdl}.pack &&\n> > + convert_bundle_to_pack <\"$bundle\" >\"$pack\" &&\n> > + git index-pack \"$pack\" &&\n> > + git verify-pack -v \"$pack\" >verify.out &&\n> > + count=$(grep \"^$OID_REGEX \" verify.out | wc -l) &&\n>\n> I think we can use 'grep -c' instead of `grep .. | wc -l`.\n\nThis function is borrowed from `t5510-fetch.sh`, and will change like\nthis. Thanks.\n\n> > +\n> > +convert_bundle_to_pack () {\n> > + while read x && test -n \"$x\"\n> > + do\n> > + :;\n> > + done\n> > + cat\n>\n> I'm not sure what you would expect in this case,\n> but in my experience, I replace this whole block with\n>\n> sed '1,/^$/d'\n\nThis function is used to convert bundle file to pack file by strip the\nheader, which has a signature, prerequisites, references. This\nfunction is also borrowed from \"t5510-fetch.sh\".\n\n> For the below change, I haven't checked but I think test_commit should work, no?\n> > +test_expect_success 'setup' '\n> > + # commit A & B\n> > + cat >main.txt <<-EOF &&\n> > + Commit A\n> > + EOF\n> > + git add main.txt &&\n> > + test_tick &&\n> > + git commit -m \"Commit A\" &&\n> > +\n> > + cat >main.txt <<-EOF &&\n> > + Commit B\n> > + EOF\n> > + git add main.txt &&\n> > + test_tick &&\n> > + git commit -m \"Commit B\" &&\n> > + A=$(git rev-parse HEAD~) &&\n> > + B=$(git rev-parse HEAD) &&\n\nI should refactor these code, but I forgot. After examine the\n`test_commit` function, I have to write a new helper, because it does\nnot meet my needs.\n1. It should not create a tag every time.\n2. The tag it created is not an annotated tag.\n3. I need to store the object id to a variable.\n\nSo I write a new helper `test_commit_setvar()` in next reroll. And\nmake this testcase much simpler.\n\ntest_expect_success 'setup' '\n # branch main: commit A & B\n test_commit_setvar A \"Commit A\" main.txt &&\n test_commit_setvar B \"Commit B\" main.txt &&\n\n # branch topic/1: commit C & D, refs/pull/1/head\n git checkout -b topic/1 &&\n test_commit_setvar C \"Commit C\" topic-1.txt &&\n test_commit_setvar D \"Commit D\" topic-1.txt &&\n git update-ref refs/pull/1/head HEAD &&\n\n # branch topic/1: commit E, tag v1\n git checkout main &&\n test_commit_setvar E \"Commit E\" main.txt &&\n test_commit_setvar TAG1 --tag v1 &&\n\n # branch topic/2: commit F & G, refs/pull/2/head\n git checkout -b topic/2 &&\n test_commit_setvar F \"Commit F\" topic-2.txt &&\n test_commit_setvar G \"Commit G\" topic-2.txt &&\n git update-ref refs/pull/2/head HEAD &&\n test_commit_setvar H \"Commit H\" topic-2.txt &&\n\n # branch main: merge commit I & J\n git checkout main &&\n test_tick &&\n test_commit_setvar I --merge topic/1 \"Merge commit I\" &&\n test_commit_setvar J --merge refs/pull/2/head \"Merge commit J\" &&\n\n # branch main: commit K\n git checkout main &&\n test_commit_setvar K \"Commit K\" main.txt &&\n\n # branch release:\n git checkout -b release &&\n test_commit_setvar L \"Commit L\" release.txt &&\n test_commit_setvar M \"Commit M\" release.txt &&\n test_commit_setvar TAG2 --tag v2 &&\n test_commit_setvar N \"Commit N\" release.txt &&\n test_commit_setvar TAG3 --tag v3 &&\n\n # branch main: merge commit O, commit P\n git checkout main &&\n test_commit_setvar O --merge tags/v2 \"Merge commit O\" &&\n test_commit_setvar P \"Commit P\" main.txt\n'\n\n--\nJiang Xin\n"},{"id":"413819","messageId":"20210108144514.24805-1-worldhello.net@gmail.com","threadId":"54923","inReplyTo":"X/cqrTgilKAW9P9G@danh.dev","subject":"[PATCH v4 0/2] Improvements for git-bundle","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-01-08T14:45:12Z","receivedAt":"2021-01-08T14:46:01Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"From: Jiang Xin \n\n## Two improvements for git-bundle\n\n1. Commit \"bundle: lost objects when removing duplicate pendings\",\n which fixes command like:\n\n $ git bundle create 'master^!'\n \n2. Commits \"bundle: arguments can be read from stdin\",\n which add \"--stdin\" option support for git-bundle, like:\n\n $ git bundle create &2 \"args should be: \"\n ++\t\treturn 1\n ++\tfi\n +\tbundle=$1 &&\n +\tpack=${bundle%.bdl}.pack &&\n +\tconvert_bundle_to_pack <\"$bundle\" >\"$pack\" &&\n -+\tgit index-pack \"$pack\" &&\n -+\tgit verify-pack -v \"$pack\" >verify.out &&\n -+\tcount=$(grep \"^$OID_REGEX \" verify.out | wc -l) &&\n ++\tif test -n \"$thin\"\n ++\tthen\n ++\t\ttest_must_fail git index-pack \"$pack\" &&\n ++\t\tmv \"$pack\" \"$pack\"-thin &&\n ++\t\tcat \"$pack\"-thin |\n ++\t\t\tgit index-pack --stdin --fix-thin \"$pack\"\n ++\telse\n ++\t\tgit index-pack \"$pack\"\n ++\tfi &&\n ++\tgit verify-pack -v \"$pack\" >verify.out\n ++\tif test $? -ne 0\n ++\tthen\n ++\t\techo >&2 \"error: fail to convert $bundle to $pack\"\n ++\t\treturn 1\n ++\tfi\n ++\tcount=$(grep -c \"^$OID_REGEX \" verify.out) &&\n +\ttest $2 = $count && return 0\n -+\techo object count for $bundle is $count, not $2\n -+\treturn 1\n -+}\n -+\n -+\n -+test_thin_bundle_object_count () {\n -+\tbundle=$1 &&\n -+\tpack=${bundle%.bdl}.pack &&\n -+\tconvert_bundle_to_pack <\"$bundle\" |\n -+\t\ttest_must_fail git index-pack --stdin \"$pack\" &&\n -+\trm -f \"$pack\" &&\n -+\tconvert_bundle_to_pack <\"$bundle\" |\n -+\t\tgit index-pack --stdin --fix-thin \"$pack\" &&\n -+\tgit verify-pack -v \"$pack\" >verify.out &&\n -+\tcount=$(grep \"^$OID_REGEX \" verify.out | wc -l) &&\n -+\ttest $2 = $count && return 0\n -+\techo object count for $bundle is $count, not $2\n ++\techo >&2 \"error: object count for $bundle is $count, not $2\"\n +\treturn 1\n +}\n +\n ++# Display the pack data contained in the bundle file, bypassing the\n ++# header that contains the signature, prerequisites and references.\n +convert_bundle_to_pack () {\n +\twhile read x && test -n \"$x\"\n +\tdo\n @@ t/t6020-bundle-misc.sh (new)\n +\tcat\n +}\n +\n ++# Create a commit or tag and set the variable with the object ID.\n ++test_commit_setvar () {\n ++\tnotick= &&\n ++\tsignoff= &&\n ++\tindir= &&\n ++\tmerge= &&\n ++\ttag= &&\n ++\tvar= &&\n ++\twhile test $# != 0\n ++\tdo\n ++\t\tcase \"$1\" in\n ++\t\t--merge)\n ++\t\t\tmerge=yes\n ++\t\t\t;;\n ++\t\t--tag)\n ++\t\t\ttag=yes\n ++\t\t\t;;\n ++\t\t--notick)\n ++\t\t\tnotick=yes\n ++\t\t\t;;\n ++\t\t--signoff)\n ++\t\t\tsignoff=\"$1\"\n ++\t\t\t;;\n ++\t\t-C)\n ++\t\t\tindir=\"$2\"\n ++\t\t\tshift\n ++\t\t\t;;\n ++\t\t-*)\n ++\t\t\techo >&2 \"error: unknown option $1\"\n ++\t\t\treturn 1\n ++\t\t\t;;\n ++\t\t*)\n ++\t\t\ttest -n \"$var\" && break\n ++\t\t\tvar=$1\n ++\t\t\t;;\n ++\t\tesac\n ++\t\tshift\n ++\tdone &&\n ++\tindir=${indir:+\"$indir\"/} &&\n ++\tif test $# -eq 0\n ++\tthen\n ++\t\techo >&2 \"no args provided\"\n ++\t\treturn 1\n ++\tfi &&\n ++\tif test -z \"$notick\"\n ++\tthen\n ++\t\ttest_tick\n ++\tfi &&\n ++\tif test -n \"$merge\"\n ++\tthen\n ++\t\tgit ${indir:+ -C \"$indir\"} merge --no-edit --no-ff \\\n ++\t\t\t${2:+-m \"$2\"} \"$1\" &&\n ++\t\toid=$(git ${indir:+ -C \"$indir\"} rev-parse HEAD)\n ++\telif test -n \"$tag\"\n ++\tthen\n ++\t\tgit ${indir:+ -C \"$indir\"} tag -m \"$1\" \"$1\" &&\n ++\t\toid=$(git ${indir:+ -C \"$indir\"} rev-parse \"$1\")\n ++\telse\n ++\t\tfile=${2:-\"$1.t\"} &&\n ++\t\techo \"${3-$1}\" > \"$indir$file\" &&\n ++\t\tgit ${indir:+ -C \"$indir\"} add \"$file\" &&\n ++\t\tgit ${indir:+ -C \"$indir\"} commit $signoff -m \"$1\" &&\n ++\t\toid=$(git ${indir:+ -C \"$indir\"} rev-parse HEAD)\n ++\tfi &&\n ++\teval $var=$oid\n ++}\n ++\n ++\n +# Format the output of git commands to make a user-friendly and stable\n +# text. We can easily prepare the expect text without having to worry\n +# about future changes of the commit ID and spaces of the output.\n +make_user_friendly_and_stable_output () {\n +\tsed \\\n -+\t\t-e \"s/ *\\$//\" \\\n -+\t\t-e \"s/$A//g\" \\\n -+\t\t-e \"s/$B//g\" \\\n -+\t\t-e \"s/$C//g\" \\\n -+\t\t-e \"s/$D//g\" \\\n -+\t\t-e \"s/$E//g\" \\\n -+\t\t-e \"s/$F//g\" \\\n -+\t\t-e \"s/$G//g\" \\\n -+\t\t-e \"s/$H//g\" \\\n -+\t\t-e \"s/$I//g\" \\\n -+\t\t-e \"s/$J//g\" \\\n -+\t\t-e \"s/$K//g\" \\\n -+\t\t-e \"s/$L//g\" \\\n -+\t\t-e \"s/$M//g\" \\\n -+\t\t-e \"s/$N//g\" \\\n -+\t\t-e \"s/$O//g\" \\\n -+\t\t-e \"s/$P//g\" \\\n ++\t\t-e \"s/$A//\" \\\n ++\t\t-e \"s/$B//\" \\\n ++\t\t-e \"s/$C//\" \\\n ++\t\t-e \"s/$D//\" \\\n ++\t\t-e \"s/$E//\" \\\n ++\t\t-e \"s/$F//\" \\\n ++\t\t-e \"s/$G//\" \\\n ++\t\t-e \"s/$H//\" \\\n ++\t\t-e \"s/$I//\" \\\n ++\t\t-e \"s/$J//\" \\\n ++\t\t-e \"s/$K//\" \\\n ++\t\t-e \"s/$L//\" \\\n ++\t\t-e \"s/$M//\" \\\n ++\t\t-e \"s/$N//\" \\\n ++\t\t-e \"s/$O//\" \\\n ++\t\t-e \"s/$P//\" \\\n ++\t\t-e \"s/$TAG1//\" \\\n ++\t\t-e \"s/$TAG2//\" \\\n ++\t\t-e \"s/$TAG3//\" \\\n +\t\t-e \"s/$(echo $A | cut -c1-7)[0-9a-f]*//g\" \\\n +\t\t-e \"s/$(echo $B | cut -c1-7)[0-9a-f]*//g\" \\\n +\t\t-e \"s/$(echo $C | cut -c1-7)[0-9a-f]*//g\" \\\n @@ t/t6020-bundle-misc.sh (new)\n +\t\t-e \"s/$(echo $N | cut -c1-7)[0-9a-f]*//g\" \\\n +\t\t-e \"s/$(echo $O | cut -c1-7)[0-9a-f]*//g\" \\\n +\t\t-e \"s/$(echo $P | cut -c1-7)[0-9a-f]*//g\" \\\n -+\t\t-e \"s/$TAG1//g\" \\\n -+\t\t-e \"s/$TAG2//g\" \\\n -+\t\t-e \"s/$TAG3//g\" \\\n +\t\t-e \"s/$(echo $TAG1 | cut -c1-7)[0-9a-f]*//g\" \\\n +\t\t-e \"s/$(echo $TAG2 | cut -c1-7)[0-9a-f]*//g\" \\\n +\t\t-e \"s/$(echo $TAG3 | cut -c1-7)[0-9a-f]*//g\" \\\n -+\t\t-e \"s/$ZERO_OID//g\"\n ++\t\t-e \"s/ *\\$//\"\n +}\n +\n +# (C) (D, pull/1/head, topic/1)\n @@ t/t6020-bundle-misc.sh (new)\n +# (A) (B) (E, tag:v1) (I) (J) (K) (O) (P)\n +#\n +test_expect_success 'setup' '\n -+\t# commit A & B\n -+\tcat >main.txt <<-EOF &&\n -+\t\tCommit A\n -+\t\tEOF\n -+\tgit add main.txt &&\n -+\ttest_tick &&\n -+\tgit commit -m \"Commit A\" &&\n ++\t# Try to make a stable fixed width for abbreviated commit ID,\n ++\t# this fixed-width oid will be replaced with \"\".\n ++\tgit config core.abbrev 7 &&\n +\n -+\tcat >main.txt <<-EOF &&\n -+\t\tCommit B\n -+\t\tEOF\n -+\tgit add main.txt &&\n -+\ttest_tick &&\n -+\tgit commit -m \"Commit B\" &&\n -+\tA=$(git rev-parse HEAD~) &&\n -+\tB=$(git rev-parse HEAD) &&\n ++\t# branch main: commit A & B\n ++\ttest_commit_setvar A \"Commit A\" main.txt &&\n ++\ttest_commit_setvar B \"Commit B\" main.txt &&\n +\n -+\t# branch topic/1\n ++\t# branch topic/1: commit C & D, refs/pull/1/head\n +\tgit checkout -b topic/1 &&\n -+\tcat >topic-1.txt <<-EOF &&\n -+\t\tCommit C\n -+\t\tEOF\n -+\tgit add topic-1.txt &&\n -+\ttest_tick &&\n -+\tgit commit -m \"Commit C\" &&\n -+\n -+\tcat >topic-1.txt <<-EOF &&\n -+\t\tCommit D\n -+\t\tEOF\n -+\tgit add -u &&\n -+\ttest_tick &&\n -+\tgit commit -m \"Commit D\" &&\n ++\ttest_commit_setvar C \"Commit C\" topic-1.txt &&\n ++\ttest_commit_setvar D \"Commit D\" topic-1.txt &&\n +\tgit update-ref refs/pull/1/head HEAD &&\n -+\tC=$(git rev-parse topic/1~) &&\n -+\tD=$(git rev-parse topic/1) &&\n +\n -+\t# commit E\n ++\t# branch topic/1: commit E, tag v1\n +\tgit checkout main &&\n -+\tcat >main.txt <<-EOF &&\n -+\t\tCommit E\n -+\t\tEOF\n -+\tgit add main.txt &&\n -+\ttest_tick &&\n -+\tgit commit -m \"Commit E\" &&\n -+\tE=$(git rev-parse HEAD) &&\n -+\ttest_tick &&\n -+\tgit tag -m \"v1\" v1 HEAD &&\n -+\tTAG1=$(git rev-parse refs/tags/v1) &&\n -+\n -+\t# branch topic/2\n -+\tgit checkout -b topic/2 &&\n -+\tcat >topic-2.txt <<-EOF &&\n -+\t\tCommit F\n -+\t\tEOF\n -+\tgit add topic-2.txt &&\n -+\ttest_tick &&\n -+\tgit commit -m \"Commit F\" &&\n ++\ttest_commit_setvar E \"Commit E\" main.txt &&\n ++\ttest_commit_setvar TAG1 --tag v1 &&\n +\n -+\tcat >topic-2.txt <<-EOF &&\n -+\t\tCommit G\n -+\t\tEOF\n -+\tgit add -u &&\n -+\ttest_tick &&\n -+\tgit commit -m \"Commit G\" &&\n ++\t# branch topic/2: commit F & G, refs/pull/2/head\n ++\tgit checkout -b topic/2 &&\n ++\ttest_commit_setvar F \"Commit F\" topic-2.txt &&\n ++\ttest_commit_setvar G \"Commit G\" topic-2.txt &&\n +\tgit update-ref refs/pull/2/head HEAD &&\n ++\ttest_commit_setvar H \"Commit H\" topic-2.txt &&\n +\n -+\tcat >topic-2.txt <<-EOF &&\n -+\t\tCommit H\n -+\t\tEOF\n -+\tgit add -u &&\n -+\ttest_tick &&\n -+\tgit commit -m \"Commit H\" &&\n -+\tF=$(git rev-parse topic/2~2) &&\n -+\tG=$(git rev-parse topic/2~) &&\n -+\tH=$(git rev-parse topic/2) &&\n -+\n -+\t# merge commit I & J\n ++\t# branch main: merge commit I & J\n +\tgit checkout main &&\n -+\ttest_tick &&\n -+\tgit merge --no-ff --no-edit topic/1 &&\n -+\ttest_tick &&\n -+\tgit merge --no-ff --no-edit refs/pull/2/head &&\n -+\tI=$(git rev-parse HEAD~) &&\n -+\tJ=$(git rev-parse HEAD) &&\n -+\n -+\t# commit K\n ++\ttest_commit_setvar I --merge topic/1 \"Merge commit I\" &&\n ++\ttest_commit_setvar J --merge refs/pull/2/head \"Merge commit J\" &&\n ++\n ++\t# branch main: commit K\n +\tgit checkout main &&\n -+\tcat >main.txt <<-EOF &&\n -+\t\tCommit K\n -+\t\tEOF\n -+\tgit add main.txt &&\n -+\ttest_tick &&\n -+\tgit commit -m \"Commit K\" &&\n -+\tK=$(git rev-parse HEAD) &&\n ++\ttest_commit_setvar K \"Commit K\" main.txt &&\n +\n -+\t# branch release\n ++\t# branch release:\n +\tgit checkout -b release &&\n -+\tcat >release.txt <<-EOF &&\n -+\t\tCommit L\n -+\t\tEOF\n -+\tgit add release.txt &&\n -+\ttest_tick &&\n -+\tgit commit -m \"Commit L\" &&\n ++\ttest_commit_setvar L \"Commit L\" release.txt &&\n ++\ttest_commit_setvar M \"Commit M\" release.txt &&\n ++\ttest_commit_setvar TAG2 --tag v2 &&\n ++\ttest_commit_setvar N \"Commit N\" release.txt &&\n ++\ttest_commit_setvar TAG3 --tag v3 &&\n +\n -+\tcat >release.txt <<-EOF &&\n -+\t\tCommit M\n -+\t\tEOF\n -+\tgit add -u &&\n -+\ttest_tick &&\n -+\tgit commit -m \"Commit M\" &&\n -+\ttest_tick &&\n -+\tgit tag -m \"v2\" v2 HEAD &&\n -+\n -+\tcat >release.txt <<-EOF &&\n -+\t\tCommit N\n -+\t\tEOF\n -+\tgit add -u &&\n -+\ttest_tick &&\n -+\tgit commit -m \"Commit N\" &&\n -+\ttest_tick &&\n -+\tgit tag -m \"v3\" v3 HEAD &&\n -+\tL=$(git rev-parse HEAD~2) &&\n -+\tM=$(git rev-parse HEAD~) &&\n -+\tN=$(git rev-parse HEAD) &&\n -+\tTAG2=$(git rev-parse refs/tags/v2) &&\n -+\tTAG3=$(git rev-parse refs/tags/v3) &&\n -+\n -+\t# merge commit O\n -+\tgit checkout main &&\n -+\ttest_tick &&\n -+\tgit merge --no-ff --no-edit tags/v2 &&\n -+\tO=$(git rev-parse HEAD) &&\n -+\n -+\t# commit P\n ++\t# branch main: merge commit O, commit P\n +\tgit checkout main &&\n -+\tcat >main.txt <<-EOF &&\n -+\t\tCommit P\n -+\t\tEOF\n -+\tgit add main.txt &&\n -+\ttest_tick &&\n -+\tgit commit -m \"Commit P\" &&\n -+\tP=$(git rev-parse HEAD)\n ++\ttest_commit_setvar O --merge tags/v2 \"Merge commit O\" &&\n ++\ttest_commit_setvar P \"Commit P\" main.txt\n +'\n +\n +test_expect_success 'create bundle from special rev: main^!' '\n @@ t/t6020-bundle-misc.sh (new)\n +'\n +\n +test_expect_success 'create bundle with --since option' '\n ++\tsince=\"Thu Apr 7 15:26:13 2005 -0700\" &&\n ++\tgit log -1 --pretty=\"%ad\" $M >actual &&\n ++\techo \"$since\" >expect &&\n ++\ttest_cmp expect actual &&\n ++\n +\tgit bundle create since.bdl \\\n -+\t\t--since \"Thu Apr 7 15:26:13 2005 -0700\" \\\n -+\t\t--all &&\n ++\t\t--since \"$since\" --all &&\n +\n +\tgit bundle list-heads since.bdl |\n +\t\tmake_user_friendly_and_stable_output >actual &&\n @@ t/t6020-bundle-misc.sh (new)\n +\t\tEOF\n +\ttest_i18ncmp expect actual &&\n +\n -+\ttest_thin_bundle_object_count since.bdl 16\n ++\ttest_bundle_object_count --thin since.bdl 16\n +'\n +\n +test_expect_success 'create bundle 1 - no prerequisites' '\n2: 86ad41e4d4 = 2: 48ef4aa44e bundle: arguments can be read from stdin\n\n--\nJiang Xin (2):\n bundle: lost objects when removing duplicate pendings\n bundle: arguments can be read from stdin\n\n bundle.c | 111 ++++----\n object.c | 10 +-\n t/t5607-clone-bundle.sh | 4 +-\n t/t6020-bundle-misc.sh | 546 ++++++++++++++++++++++++++++++++++++++++\n 4 files changed, 614 insertions(+), 57 deletions(-)\n create mode 100755 t/t6020-bundle-misc.sh\n\n-- \n2.30.0.2.g06d2f50715\n\n"},{"id":"413820","messageId":"20210108144514.24805-3-worldhello.net@gmail.com","threadId":"54923","inReplyTo":"X/cqrTgilKAW9P9G@danh.dev","subject":"[PATCH v4 2/2] bundle: arguments can be read from stdin","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-01-08T14:45:14Z","receivedAt":"2021-01-08T14:46:18Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"From: Jiang Xin \n\nIn order to create an incremental bundle, we need to pass many arguments\nto let git-bundle ignore some already packed commits. It will be more\nconvenient to pass args via stdin. But the current implementation does\nnot allow us to do this.\n\nThis is because args are parsed twice when creating bundle. The first\ntime for parsing args is in `compute_and_write_prerequisites()` by\nrunning `git-rev-list` command to write prerequisites in bundle file,\nand stdin is consumed in this step if \"--stdin\" option is provided for\n`git-bundle`. Later nothing can be read from stdin when running\n`setup_revisions()` in `create_bundle()`.\n\nThe solution is to parse args once by removing the entire function\n`compute_and_write_prerequisites()` and then calling function\n`setup_revisions()`. In order to write prerequisites for bundle, will\ncall `prepare_revision_walk()` and `traverse_commit_list()`. But after\ncalling `prepare_revision_walk()`, the object array `revs.pending` is\nleft empty, and the following steps could not work properly with the\nempty object array (`revs.pending`). Therefore, make a copy of `revs`\nto `revs_copy` for later use right after calling `setup_revisions()`.\n\nThe copy of `revs_copy` is not a deep copy, it shares the same objects\nwith `revs`. The object array of `revs` has been cleared, but objects\nthemselves are still kept. Flags of objects may change after calling\n`prepare_revision_walk()`, we can use these changed flags without\ncalling the `git rev-list` command and parsing its output like the\nformer implementation.\n\nAlso add testcases for git bundle in t6020, which read args from stdin.\n\nSigned-off-by: Jiang Xin \n---\n bundle.c | 111 ++++++++++++++++++++++------------------\n t/t5607-clone-bundle.sh | 4 +-\n t/t6020-bundle-misc.sh | 77 ++++++++++++++++++++++++++--\n 3 files changed, 135 insertions(+), 57 deletions(-)\n\ndiff --git a/bundle.c b/bundle.c\nindex cb0e5931ac..72970d962a 100644\n--- a/bundle.c\n+++ b/bundle.c\n@@ -338,48 +338,6 @@ static int write_pack_data(int bundle_fd, struct rev_info *revs, struct strvec *\n \treturn 0;\n }\n \n-static int compute_and_write_prerequisites(int bundle_fd,\n-\t\t\t\t\t struct rev_info *revs,\n-\t\t\t\t\t int argc, const char **argv)\n-{\n-\tstruct child_process rls = CHILD_PROCESS_INIT;\n-\tstruct strbuf buf = STRBUF_INIT;\n-\tFILE *rls_fout;\n-\tint i;\n-\n-\tstrvec_pushl(&rls.args,\n-\t\t \"rev-list\", \"--boundary\", \"--pretty=oneline\",\n-\t\t NULL);\n-\tfor (i = 1; i < argc; i++)\n-\t\tstrvec_push(&rls.args, argv[i]);\n-\trls.out = -1;\n-\trls.git_cmd = 1;\n-\tif (start_command(&rls))\n-\t\treturn -1;\n-\trls_fout = xfdopen(rls.out, \"r\");\n-\twhile (strbuf_getwholeline(&buf, rls_fout, '\\n') != EOF) {\n-\t\tstruct object_id oid;\n-\t\tif (buf.len > 0 && buf.buf[0] == '-') {\n-\t\t\twrite_or_die(bundle_fd, buf.buf, buf.len);\n-\t\t\tif (!get_oid_hex(buf.buf + 1, &oid)) {\n-\t\t\t\tstruct object *object = parse_object_or_die(&oid,\n-\t\t\t\t\t\t\t\t\t buf.buf);\n-\t\t\t\tobject->flags |= UNINTERESTING;\n-\t\t\t\tadd_pending_object(revs, object, buf.buf);\n-\t\t\t}\n-\t\t} else if (!get_oid_hex(buf.buf, &oid)) {\n-\t\t\tstruct object *object = parse_object_or_die(&oid,\n-\t\t\t\t\t\t\t\t buf.buf);\n-\t\t\tobject->flags |= SHOWN;\n-\t\t}\n-\t}\n-\tstrbuf_release(&buf);\n-\tfclose(rls_fout);\n-\tif (finish_command(&rls))\n-\t\treturn error(_(\"rev-list died\"));\n-\treturn 0;\n-}\n-\n /*\n * Write out bundle refs based on the tips already\n * parsed into revs.pending. As a side effect, may\n@@ -425,7 +383,7 @@ static int write_bundle_refs(int bundle_fd, struct rev_info *revs)\n \t\t * constraints.\n \t\t */\n \t\tif (!(e->item->flags & SHOWN) && e->item->type == OBJ_COMMIT) {\n-\t\t\twarning(_(\"ref '%s' is excluded by the rev-list options\"),\n+\t\t\twarning(_(\"ref '%s' is excluded by the limiting options\"),\n \t\t\t\te->name);\n \t\t\tgoto skip_write_ref;\n \t\t}\n@@ -474,6 +432,38 @@ static int write_bundle_refs(int bundle_fd, struct rev_info *revs)\n \treturn ref_count;\n }\n \n+struct bundle_prerequisites_info {\n+\tstruct object_array *pending;\n+\tint fd;\n+};\n+\n+static void write_bundle_prerequisites(struct commit *commit, void *data)\n+{\n+\tstruct bundle_prerequisites_info *bpi = data;\n+\tstruct object *object;\n+\tstruct pretty_print_context ctx = { 0 };\n+\tstruct strbuf buf = STRBUF_INIT;\n+\n+\tif (!(commit->object.flags & BOUNDARY))\n+\t\treturn;\n+\tstrbuf_addf(&buf, \"-%s \", oid_to_hex(&commit->object.oid));\n+\twrite_or_die(bpi->fd, buf.buf, buf.len);\n+\n+\tctx.fmt = CMIT_FMT_ONELINE;\n+\tctx.output_encoding = get_log_output_encoding();\n+\tstrbuf_reset(&buf);\n+\tpretty_print_commit(&ctx, commit, &buf);\n+\tstrbuf_trim(&buf);\n+\n+\tobject = (struct object *)commit;\n+\tobject->flags |= UNINTERESTING;\n+\tadd_object_array_with_path(object, buf.buf, bpi->pending, S_IFINVALID,\n+\t\t\t\t NULL);\n+\tstrbuf_addch(&buf, '\\n');\n+\twrite_or_die(bpi->fd, buf.buf, buf.len);\n+\tstrbuf_release(&buf);\n+}\n+\n int create_bundle(struct repository *r, const char *path,\n \t\t int argc, const char **argv, struct strvec *pack_options, int version)\n {\n@@ -481,8 +471,10 @@ int create_bundle(struct repository *r, const char *path,\n \tint bundle_fd = -1;\n \tint bundle_to_stdout;\n \tint ref_count = 0;\n-\tstruct rev_info revs;\n+\tstruct rev_info revs, revs_copy;\n \tint min_version = the_hash_algo == &hash_algos[GIT_HASH_SHA1] ? 2 : 3;\n+\tstruct bundle_prerequisites_info bpi;\n+\tint i;\n \n \tbundle_to_stdout = !strcmp(path, \"-\");\n \tif (bundle_to_stdout)\n@@ -512,10 +504,6 @@ int create_bundle(struct repository *r, const char *path,\n \tsave_commit_buffer = 0;\n \trepo_init_revisions(r, &revs, NULL);\n \n-\t/* write prerequisites */\n-\tif (compute_and_write_prerequisites(bundle_fd, &revs, argc, argv))\n-\t\tgoto err;\n-\n \targc = setup_revisions(argc, argv, &revs, NULL);\n \n \tif (argc > 1) {\n@@ -523,16 +511,37 @@ int create_bundle(struct repository *r, const char *path,\n \t\tgoto err;\n \t}\n \n-\tobject_array_remove_duplicates(&revs.pending);\n+\t/* save revs.pending in revs_copy for later use */\n+\tmemcpy(&revs_copy, &revs, sizeof(revs));\n+\trevs_copy.pending.nr = 0;\n+\trevs_copy.pending.alloc = 0;\n+\trevs_copy.pending.objects = NULL;\n+\tfor (i = 0; i < revs.pending.nr; i++) {\n+\t\tstruct object_array_entry *e = revs.pending.objects + i;\n+\t\tif (e)\n+\t\t\tadd_object_array_with_path(e->item, e->name,\n+\t\t\t\t\t\t &revs_copy.pending,\n+\t\t\t\t\t\t e->mode, e->path);\n+\t}\n \n-\tref_count = write_bundle_refs(bundle_fd, &revs);\n+\t/* write prerequisites */\n+\trevs.boundary = 1;\n+\tif (prepare_revision_walk(&revs))\n+\t\tdie(\"revision walk setup failed\");\n+\tbpi.fd = bundle_fd;\n+\tbpi.pending = &revs_copy.pending;\n+\ttraverse_commit_list(&revs, write_bundle_prerequisites, NULL, &bpi);\n+\tobject_array_remove_duplicates(&revs_copy.pending);\n+\n+\t/* write bundle refs */\n+\tref_count = write_bundle_refs(bundle_fd, &revs_copy);\n \tif (!ref_count)\n \t\tdie(_(\"Refusing to create empty bundle.\"));\n \telse if (ref_count < 0)\n \t\tgoto err;\n \n \t/* write pack */\n-\tif (write_pack_data(bundle_fd, &revs, pack_options))\n+\tif (write_pack_data(bundle_fd, &revs_copy, pack_options))\n \t\tgoto err;\n \n \tif (!bundle_to_stdout) {\ndiff --git a/t/t5607-clone-bundle.sh b/t/t5607-clone-bundle.sh\nindex 26985f4b44..425258767d 100755\n--- a/t/t5607-clone-bundle.sh\n+++ b/t/t5607-clone-bundle.sh\n@@ -38,13 +38,13 @@ test_expect_success 'die if bundle file cannot be created' '\n \ttest_must_fail git bundle create adir --all\n '\n \n-test_expect_failure 'bundle --stdin' '\n+test_expect_success 'bundle --stdin' '\n \techo master | git bundle create stdin-bundle.bdl --stdin &&\n \tgit ls-remote stdin-bundle.bdl >output &&\n \tgrep master output\n '\n \n-test_expect_failure 'bundle --stdin ' '\n+test_expect_success 'bundle --stdin ' '\n \techo master | git bundle create hybrid-bundle.bdl --stdin tag &&\n \tgit ls-remote hybrid-bundle.bdl >output &&\n \tgrep master output\ndiff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh\nindex c4447ca88f..ace675a68b 100755\n--- a/t/t6020-bundle-misc.sh\n+++ b/t/t6020-bundle-misc.sh\n@@ -325,8 +325,16 @@ test_expect_success 'create bundle with --since option' '\n '\n \n test_expect_success 'create bundle 1 - no prerequisites' '\n+\t# create bundle from args\n \tgit bundle create 1.bdl topic/1 topic/2 &&\n \n+\t# create bundle from stdin\n+\tcat >input <<-EOF &&\n+\t\ttopic/1\n+\t\ttopic/2\n+\t\tEOF\n+\tgit bundle create stdin-1.bdl --stdin expect <<-EOF &&\n \t\tThe bundle contains these 2 refs:\n \t\t refs/heads/topic/1\n@@ -339,10 +347,16 @@ test_expect_success 'create bundle 1 - no prerequisites' '\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual &&\n \n-\ttest_bundle_object_count 1.bdl 24\n+\tgit bundle verify stdin-1.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 1.bdl 24 &&\n+\ttest_bundle_object_count stdin-1.bdl 24\n '\n \n test_expect_success 'create bundle 2 - has prerequisites' '\n+\t# create bundle from args\n \tgit bundle create 2.bdl \\\n \t\t--ignore-missing \\\n \t\t^topic/deleted \\\n@@ -350,6 +364,18 @@ test_expect_success 'create bundle 2 - has prerequisites' '\n \t\t^topic/2 \\\n \t\trelease &&\n \n+\t# create bundle from stdin\n+\t# input has a non-exist reference: \"topic/deleted\"\n+\tcat >input <<-EOF &&\n+\t\t^topic/deleted\n+\t\t^$D\n+\t\t^topic/2\n+\t\tEOF\n+\tgit bundle create stdin-2.bdl \\\n+\t\t--ignore-missing \\\n+\t\t--stdin \\\n+\t\trelease expect <<-EOF &&\n \t\tThe bundle contains this ref:\n \t\t refs/heads/release\n@@ -363,7 +389,12 @@ test_expect_success 'create bundle 2 - has prerequisites' '\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual &&\n \n-\ttest_bundle_object_count 2.bdl 16\n+\tgit bundle verify stdin-2.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 2.bdl 16 &&\n+\ttest_bundle_object_count stdin-2.bdl 16\n '\n \n test_expect_success 'fail to verify bundle without prerequisites' '\n@@ -378,10 +409,15 @@ test_expect_success 'fail to verify bundle without prerequisites' '\n \n \ttest_must_fail git -C test1.git bundle verify ../2.bdl 2>&1 |\n \t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_must_fail git -C test1.git bundle verify ../stdin-2.bdl 2>&1 |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual\n '\n \n test_expect_success 'create bundle 3 - two refs, same object' '\n+\t# create bundle from args\n \tgit bundle create --version=3 3.bdl \\\n \t\t^release \\\n \t\t^topic/1 \\\n@@ -389,6 +425,16 @@ test_expect_success 'create bundle 3 - two refs, same object' '\n \t\tmain \\\n \t\tHEAD &&\n \n+\t# create bundle from stdin\n+\tcat >input <<-EOF &&\n+\t\t^release\n+\t\t^topic/1\n+\t\t^topic/2\n+\t\tEOF\n+\tgit bundle create --version=3 stdin-3.bdl \\\n+\t\t--stdin \\\n+\t\tmain HEAD expect <<-EOF &&\n \t\tThe bundle contains these 2 refs:\n \t\t refs/heads/main\n@@ -402,10 +448,16 @@ test_expect_success 'create bundle 3 - two refs, same object' '\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual &&\n \n-\ttest_bundle_object_count 3.bdl 4\n+\tgit bundle verify stdin-3.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 3.bdl 4 &&\n+\ttest_bundle_object_count stdin-3.bdl 4\n '\n \n test_expect_success 'create bundle 4 - with tags' '\n+\t# create bundle from args\n \tgit bundle create 4.bdl \\\n \t\t^main \\\n \t\t^release \\\n@@ -413,6 +465,18 @@ test_expect_success 'create bundle 4 - with tags' '\n \t\t^topic/2 \\\n \t\t--all &&\n \n+\t# create bundle from stdin\n+\tcat >input <<-EOF &&\n+\t\t^main\n+\t\t^release\n+\t\t^topic/1\n+\t\t^topic/2\n+\t\tEOF\n+\tgit bundle create stdin-4.bdl \\\n+\t\t--ignore-missing \\\n+\t\t--stdin \\\n+\t\t--all expect <<-EOF &&\n \t\tThe bundle contains these 3 refs:\n \t\t refs/tags/v1\n@@ -425,7 +489,12 @@ test_expect_success 'create bundle 4 - with tags' '\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual &&\n \n-\ttest_bundle_object_count 4.bdl 3\n+\tgit bundle verify stdin-4.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 4.bdl 3 &&\n+\ttest_bundle_object_count stdin-4.bdl 3\n '\n \n test_expect_success 'clone from bundle' '\n-- \n2.30.0.2.g06d2f50715\n\n"},{"id":"413821","messageId":"20210108144514.24805-2-worldhello.net@gmail.com","threadId":"54923","inReplyTo":"X/cqrTgilKAW9P9G@danh.dev","subject":"[PATCH v4 1/2] bundle: lost objects when removing duplicate pendings","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-01-08T14:45:13Z","receivedAt":"2021-01-08T14:46:18Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"From: Jiang Xin \n\n`git rev-list` will list one commit for the following command:\n\n $ git rev-list 'main^!'\n \n\nBut providing the same rev-list args to `git bundle`, fail to create\na bundle file.\n\n $ git bundle create - 'main^!'\n # v2 git bundle\n - \n\n fatal: Refusing to create empty bundle.\n\nThis is because when removing duplicate objects in function\n`object_array_remove_duplicates()`, one unique pending object which has\nthe same name is deleted by mistake. The revision arg 'main^!' in the\nabove example is parsed by `handle_revision_arg()`, and at lease two\ndifferent objects will be appended to `revs.pending`, one points to the\nparent commit of the \"main\" branch, and the other points to the tip\ncommit of the \"main\" branch. These two objects have the same name\n\"main\". Only one object is left with the name \"main\" after calling the\nfunction `object_array_remove_duplicates()`.\n\nAnd what's worse, when adding boundary commits into pending list, we use\none-line commit message as names, and the arbitory names may surprise\ngit-bundle.\n\nOnly comparing objects themselves (\".item\") is also not good enough,\nbecause user may want to create a bundle with two identical objects but\nwith different reference names, such as: \"HEAD\" and \"refs/heads/main\".\n\nAdd new function `contains_object()` which compare both the address and\nthe name of the object.\n\nSigned-off-by: Jiang Xin \n---\n object.c | 10 +-\n t/t6020-bundle-misc.sh | 477 +++++++++++++++++++++++++++++++++++++++++\n 2 files changed, 483 insertions(+), 4 deletions(-)\n create mode 100755 t/t6020-bundle-misc.sh\n\ndiff --git a/object.c b/object.c\nindex 68f80b0b3d..98017bed8e 100644\n--- a/object.c\n+++ b/object.c\n@@ -412,15 +412,16 @@ void object_array_clear(struct object_array *array)\n }\n \n /*\n- * Return true iff array already contains an entry with name.\n+ * Return true if array already contains an entry.\n */\n-static int contains_name(struct object_array *array, const char *name)\n+static int contains_object(struct object_array *array,\n+\t\t\t const struct object *item, const char *name)\n {\n \tunsigned nr = array->nr, i;\n \tstruct object_array_entry *object = array->objects;\n \n \tfor (i = 0; i < nr; i++, object++)\n-\t\tif (!strcmp(object->name, name))\n+\t\tif (item == object->item && !strcmp(object->name, name))\n \t\t\treturn 1;\n \treturn 0;\n }\n@@ -432,7 +433,8 @@ void object_array_remove_duplicates(struct object_array *array)\n \n \tarray->nr = 0;\n \tfor (src = 0; src < nr; src++) {\n-\t\tif (!contains_name(array, objects[src].name)) {\n+\t\tif (!contains_object(array, objects[src].item,\n+\t\t\t\t objects[src].name)) {\n \t\t\tif (src != array->nr)\n \t\t\t\tobjects[array->nr] = objects[src];\n \t\t\tarray->nr++;\ndiff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh\nnew file mode 100755\nindex 0000000000..c4447ca88f\n--- /dev/null\n+++ b/t/t6020-bundle-misc.sh\n@@ -0,0 +1,477 @@\n+#!/bin/sh\n+#\n+# Copyright (c) 2021 Jiang Xin\n+#\n+\n+test_description='Test git-bundle'\n+\n+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n+\n+. ./test-lib.sh\n+\n+# Check count of objects in a bundle file.\n+# We can use \"--thin\" opiton to check thin pack, which must be fixed by\n+# command `git-index-pack --fix-thin --stdin`.\n+test_bundle_object_count () {\n+\tthin= &&\n+\tif test \"$1\" = \"--thin\"\n+\tthen\n+\t\tthin=yes\n+\t\tshift\n+\tfi &&\n+\tif test $# -ne 2\n+\tthen\n+\t\techo >&2 \"args should be: \"\n+\t\treturn 1\n+\tfi\n+\tbundle=$1 &&\n+\tpack=${bundle%.bdl}.pack &&\n+\tconvert_bundle_to_pack <\"$bundle\" >\"$pack\" &&\n+\tif test -n \"$thin\"\n+\tthen\n+\t\ttest_must_fail git index-pack \"$pack\" &&\n+\t\tmv \"$pack\" \"$pack\"-thin &&\n+\t\tcat \"$pack\"-thin |\n+\t\t\tgit index-pack --stdin --fix-thin \"$pack\"\n+\telse\n+\t\tgit index-pack \"$pack\"\n+\tfi &&\n+\tgit verify-pack -v \"$pack\" >verify.out\n+\tif test $? -ne 0\n+\tthen\n+\t\techo >&2 \"error: fail to convert $bundle to $pack\"\n+\t\treturn 1\n+\tfi\n+\tcount=$(grep -c \"^$OID_REGEX \" verify.out) &&\n+\ttest $2 = $count && return 0\n+\techo >&2 \"error: object count for $bundle is $count, not $2\"\n+\treturn 1\n+}\n+\n+# Display the pack data contained in the bundle file, bypassing the\n+# header that contains the signature, prerequisites and references.\n+convert_bundle_to_pack () {\n+\twhile read x && test -n \"$x\"\n+\tdo\n+\t\t:;\n+\tdone\n+\tcat\n+}\n+\n+# Create a commit or tag and set the variable with the object ID.\n+test_commit_setvar () {\n+\tnotick= &&\n+\tsignoff= &&\n+\tindir= &&\n+\tmerge= &&\n+\ttag= &&\n+\tvar= &&\n+\twhile test $# != 0\n+\tdo\n+\t\tcase \"$1\" in\n+\t\t--merge)\n+\t\t\tmerge=yes\n+\t\t\t;;\n+\t\t--tag)\n+\t\t\ttag=yes\n+\t\t\t;;\n+\t\t--notick)\n+\t\t\tnotick=yes\n+\t\t\t;;\n+\t\t--signoff)\n+\t\t\tsignoff=\"$1\"\n+\t\t\t;;\n+\t\t-C)\n+\t\t\tindir=\"$2\"\n+\t\t\tshift\n+\t\t\t;;\n+\t\t-*)\n+\t\t\techo >&2 \"error: unknown option $1\"\n+\t\t\treturn 1\n+\t\t\t;;\n+\t\t*)\n+\t\t\ttest -n \"$var\" && break\n+\t\t\tvar=$1\n+\t\t\t;;\n+\t\tesac\n+\t\tshift\n+\tdone &&\n+\tindir=${indir:+\"$indir\"/} &&\n+\tif test $# -eq 0\n+\tthen\n+\t\techo >&2 \"no args provided\"\n+\t\treturn 1\n+\tfi &&\n+\tif test -z \"$notick\"\n+\tthen\n+\t\ttest_tick\n+\tfi &&\n+\tif test -n \"$merge\"\n+\tthen\n+\t\tgit ${indir:+ -C \"$indir\"} merge --no-edit --no-ff \\\n+\t\t\t${2:+-m \"$2\"} \"$1\" &&\n+\t\toid=$(git ${indir:+ -C \"$indir\"} rev-parse HEAD)\n+\telif test -n \"$tag\"\n+\tthen\n+\t\tgit ${indir:+ -C \"$indir\"} tag -m \"$1\" \"$1\" &&\n+\t\toid=$(git ${indir:+ -C \"$indir\"} rev-parse \"$1\")\n+\telse\n+\t\tfile=${2:-\"$1.t\"} &&\n+\t\techo \"${3-$1}\" > \"$indir$file\" &&\n+\t\tgit ${indir:+ -C \"$indir\"} add \"$file\" &&\n+\t\tgit ${indir:+ -C \"$indir\"} commit $signoff -m \"$1\" &&\n+\t\toid=$(git ${indir:+ -C \"$indir\"} rev-parse HEAD)\n+\tfi &&\n+\teval $var=$oid\n+}\n+\n+\n+# Format the output of git commands to make a user-friendly and stable\n+# text. We can easily prepare the expect text without having to worry\n+# about future changes of the commit ID and spaces of the output.\n+make_user_friendly_and_stable_output () {\n+\tsed \\\n+\t\t-e \"s/$A//\" \\\n+\t\t-e \"s/$B//\" \\\n+\t\t-e \"s/$C//\" \\\n+\t\t-e \"s/$D//\" \\\n+\t\t-e \"s/$E//\" \\\n+\t\t-e \"s/$F//\" \\\n+\t\t-e \"s/$G//\" \\\n+\t\t-e \"s/$H//\" \\\n+\t\t-e \"s/$I//\" \\\n+\t\t-e \"s/$J//\" \\\n+\t\t-e \"s/$K//\" \\\n+\t\t-e \"s/$L//\" \\\n+\t\t-e \"s/$M//\" \\\n+\t\t-e \"s/$N//\" \\\n+\t\t-e \"s/$O//\" \\\n+\t\t-e \"s/$P//\" \\\n+\t\t-e \"s/$TAG1//\" \\\n+\t\t-e \"s/$TAG2//\" \\\n+\t\t-e \"s/$TAG3//\" \\\n+\t\t-e \"s/$(echo $A | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $B | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $C | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $D | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $E | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $F | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $G | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $H | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $I | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $J | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $K | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $L | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $M | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $N | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $O | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $P | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $TAG1 | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $TAG2 | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $TAG3 | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/ *\\$//\"\n+}\n+\n+# (C) (D, pull/1/head, topic/1)\n+# o --- o\n+# / \\ (L)\n+# / \\ o (H, topic/2) (M, tag:v2)\n+# / (F) \\ / (N, tag:v3)\n+# / o --------- o (G, pull/2/head) o --- o --- o (release)\n+# / / \\ \\ / \\\n+# o --- o --- o -------- o -- o ------------------ o ------- o --- o (main)\n+# (A) (B) (E, tag:v1) (I) (J) (K) (O) (P)\n+#\n+test_expect_success 'setup' '\n+\t# Try to make a stable fixed width for abbreviated commit ID,\n+\t# this fixed-width oid will be replaced with \"\".\n+\tgit config core.abbrev 7 &&\n+\n+\t# branch main: commit A & B\n+\ttest_commit_setvar A \"Commit A\" main.txt &&\n+\ttest_commit_setvar B \"Commit B\" main.txt &&\n+\n+\t# branch topic/1: commit C & D, refs/pull/1/head\n+\tgit checkout -b topic/1 &&\n+\ttest_commit_setvar C \"Commit C\" topic-1.txt &&\n+\ttest_commit_setvar D \"Commit D\" topic-1.txt &&\n+\tgit update-ref refs/pull/1/head HEAD &&\n+\n+\t# branch topic/1: commit E, tag v1\n+\tgit checkout main &&\n+\ttest_commit_setvar E \"Commit E\" main.txt &&\n+\ttest_commit_setvar TAG1 --tag v1 &&\n+\n+\t# branch topic/2: commit F & G, refs/pull/2/head\n+\tgit checkout -b topic/2 &&\n+\ttest_commit_setvar F \"Commit F\" topic-2.txt &&\n+\ttest_commit_setvar G \"Commit G\" topic-2.txt &&\n+\tgit update-ref refs/pull/2/head HEAD &&\n+\ttest_commit_setvar H \"Commit H\" topic-2.txt &&\n+\n+\t# branch main: merge commit I & J\n+\tgit checkout main &&\n+\ttest_commit_setvar I --merge topic/1 \"Merge commit I\" &&\n+\ttest_commit_setvar J --merge refs/pull/2/head \"Merge commit J\" &&\n+\n+\t# branch main: commit K\n+\tgit checkout main &&\n+\ttest_commit_setvar K \"Commit K\" main.txt &&\n+\n+\t# branch release:\n+\tgit checkout -b release &&\n+\ttest_commit_setvar L \"Commit L\" release.txt &&\n+\ttest_commit_setvar M \"Commit M\" release.txt &&\n+\ttest_commit_setvar TAG2 --tag v2 &&\n+\ttest_commit_setvar N \"Commit N\" release.txt &&\n+\ttest_commit_setvar TAG3 --tag v3 &&\n+\n+\t# branch main: merge commit O, commit P\n+\tgit checkout main &&\n+\ttest_commit_setvar O --merge tags/v2 \"Merge commit O\" &&\n+\ttest_commit_setvar P \"Commit P\" main.txt\n+'\n+\n+test_expect_success 'create bundle from special rev: main^!' '\n+\tgit bundle create special-rev.bdl \"main^!\" &&\n+\n+\tgit bundle list-heads special-rev.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\t refs/heads/main\n+\t\tEOF\n+\ttest_i18ncmp expect actual &&\n+\n+\tgit bundle verify special-rev.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\tThe bundle contains this ref:\n+\t\t refs/heads/main\n+\t\tThe bundle requires this ref:\n+\t\t\n+\t\tEOF\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count special-rev.bdl 3\n+'\n+\n+test_expect_success 'create bundle with --max-count option' '\n+\tgit bundle create max-count.bdl --max-count 1 \\\n+\t\tmain \\\n+\t\t\"^release\" \\\n+\t\trefs/tags/v1 \\\n+\t\trefs/pull/1/head \\\n+\t\trefs/pull/2/head &&\n+\n+\tgit bundle list-heads max-count.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\t refs/heads/main\n+\t\t refs/tags/v1\n+\t\tEOF\n+\ttest_i18ncmp expect actual &&\n+\n+\tgit bundle verify max-count.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\tThe bundle contains these 2 refs:\n+\t\t refs/heads/main\n+\t\t refs/tags/v1\n+\t\tThe bundle requires this ref:\n+\t\t\n+\t\tEOF\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count max-count.bdl 4\n+'\n+\n+test_expect_success 'create bundle with --since option' '\n+\tsince=\"Thu Apr 7 15:26:13 2005 -0700\" &&\n+\tgit log -1 --pretty=\"%ad\" $M >actual &&\n+\techo \"$since\" >expect &&\n+\ttest_cmp expect actual &&\n+\n+\tgit bundle create since.bdl \\\n+\t\t--since \"$since\" --all &&\n+\n+\tgit bundle list-heads since.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\t refs/heads/main\n+\t\t refs/heads/release\n+\t\t refs/tags/v2\n+\t\t refs/tags/v3\n+\t\t HEAD\n+\t\tEOF\n+\ttest_i18ncmp expect actual &&\n+\n+\tgit bundle verify since.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\tThe bundle contains these 5 refs:\n+\t\t refs/heads/main\n+\t\t refs/heads/release\n+\t\t refs/tags/v2\n+\t\t refs/tags/v3\n+\t\t HEAD\n+\t\tThe bundle requires these 2 refs:\n+\t\t\n+\t\t\n+\t\tEOF\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count --thin since.bdl 16\n+'\n+\n+test_expect_success 'create bundle 1 - no prerequisites' '\n+\tgit bundle create 1.bdl topic/1 topic/2 &&\n+\n+\tcat >expect <<-EOF &&\n+\t\tThe bundle contains these 2 refs:\n+\t\t refs/heads/topic/1\n+\t\t refs/heads/topic/2\n+\t\tThe bundle records a complete history.\n+\t\tEOF\n+\n+\t# verify bundle, which has no prerequisites\n+\tgit bundle verify 1.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 1.bdl 24\n+'\n+\n+test_expect_success 'create bundle 2 - has prerequisites' '\n+\tgit bundle create 2.bdl \\\n+\t\t--ignore-missing \\\n+\t\t^topic/deleted \\\n+\t\t^$D \\\n+\t\t^topic/2 \\\n+\t\trelease &&\n+\n+\tcat >expect <<-EOF &&\n+\t\tThe bundle contains this ref:\n+\t\t refs/heads/release\n+\t\tThe bundle requires these 3 refs:\n+\t\t\n+\t\t\n+\t\t\n+\t\tEOF\n+\n+\tgit bundle verify 2.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 2.bdl 16\n+'\n+\n+test_expect_success 'fail to verify bundle without prerequisites' '\n+\tgit init --bare test1.git &&\n+\n+\tcat >expect <<-EOF &&\n+\t\terror: Repository lacks these prerequisite commits:\n+\t\terror: \n+\t\terror: \n+\t\terror: \n+\t\tEOF\n+\n+\ttest_must_fail git -C test1.git bundle verify ../2.bdl 2>&1 |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual\n+'\n+\n+test_expect_success 'create bundle 3 - two refs, same object' '\n+\tgit bundle create --version=3 3.bdl \\\n+\t\t^release \\\n+\t\t^topic/1 \\\n+\t\t^topic/2 \\\n+\t\tmain \\\n+\t\tHEAD &&\n+\n+\tcat >expect <<-EOF &&\n+\t\tThe bundle contains these 2 refs:\n+\t\t refs/heads/main\n+\t\t HEAD\n+\t\tThe bundle requires these 2 refs:\n+\t\t\n+\t\t\n+\t\tEOF\n+\n+\tgit bundle verify 3.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 3.bdl 4\n+'\n+\n+test_expect_success 'create bundle 4 - with tags' '\n+\tgit bundle create 4.bdl \\\n+\t\t^main \\\n+\t\t^release \\\n+\t\t^topic/1 \\\n+\t\t^topic/2 \\\n+\t\t--all &&\n+\n+\tcat >expect <<-EOF &&\n+\t\tThe bundle contains these 3 refs:\n+\t\t refs/tags/v1\n+\t\t refs/tags/v2\n+\t\t refs/tags/v3\n+\t\tThe bundle records a complete history.\n+\t\tEOF\n+\n+\tgit bundle verify 4.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 4.bdl 3\n+'\n+\n+test_expect_success 'clone from bundle' '\n+\tgit clone --mirror 1.bdl mirror.git &&\n+\tgit -C mirror.git show-ref |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\t refs/heads/topic/1\n+\t\t refs/heads/topic/2\n+\t\tEOF\n+\ttest_cmp expect actual &&\n+\n+\tgit -C mirror.git fetch ../2.bdl \"+refs/*:refs/*\" &&\n+\tgit -C mirror.git show-ref |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\t refs/heads/release\n+\t\t refs/heads/topic/1\n+\t\t refs/heads/topic/2\n+\t\tEOF\n+\ttest_cmp expect actual &&\n+\n+\tgit -C mirror.git fetch ../3.bdl \"+refs/*:refs/*\" &&\n+\tgit -C mirror.git show-ref |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\t refs/heads/main\n+\t\t refs/heads/release\n+\t\t refs/heads/topic/1\n+\t\t refs/heads/topic/2\n+\t\tEOF\n+\ttest_cmp expect actual &&\n+\n+\tgit -C mirror.git fetch ../4.bdl \"+refs/*:refs/*\" &&\n+\tgit -C mirror.git show-ref |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-EOF &&\n+\t\t refs/heads/main\n+\t\t refs/heads/release\n+\t\t refs/heads/topic/1\n+\t\t refs/heads/topic/2\n+\t\t refs/tags/v1\n+\t\t refs/tags/v2\n+\t\t refs/tags/v3\n+\t\tEOF\n+\ttest_cmp expect actual\n+'\n+\n+test_done\n-- \n2.30.0.2.g06d2f50715\n\n"},{"id":"413902","messageId":"xmqqv9c6g8r4.fsf@gitster.c.googlers.com","threadId":"54923","inReplyTo":"20210108144514.24805-2-worldhello.net@gmail.com","subject":"Re: [PATCH v4 1/2] bundle: lost objects when removing duplicate pendings","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-01-09T02:10:55Z","receivedAt":"2021-01-09T02:11:59Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jiang Xin writes:\n\n> diff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh\n> new file mode 100755\n> index 0000000000..c4447ca88f\n> --- /dev/null\n> +++ b/t/t6020-bundle-misc.sh\n> @@ -0,0 +1,477 @@\n> +#!/bin/sh\n> +#\n> +# Copyright (c) 2021 Jiang Xin\n> +#\n> +\n> +test_description='Test git-bundle'\n> +\n> +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n> +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n> +\n> +. ./test-lib.sh\n> +\n> +# Check count of objects in a bundle file.\n> +# We can use \"--thin\" opiton to check thin pack, which must be fixed by\n> +# command `git-index-pack --fix-thin --stdin`.\n> +test_bundle_object_count () {\n> +\tthin= &&\n> +\tif test \"$1\" = \"--thin\"\n> +\tthen\n> +\t\tthin=yes\n> +\t\tshift\n> +\tfi &&\n> +\tif test $# -ne 2\n> +\tthen\n> +\t\techo >&2 \"args should be: \"\n> +\t\treturn 1\n> +\tfi\n> +\tbundle=$1 &&\n> +\tpack=${bundle%.bdl}.pack &&\n> +\tconvert_bundle_to_pack <\"$bundle\" >\"$pack\" &&\n> +\tif test -n \"$thin\"\n> +\tthen\n> +\t\ttest_must_fail git index-pack \"$pack\" &&\n\nThis is overly strict, isn't it? \n\nImagine a case where the objects newer revisions introduce have *no*\nresemblance to the objects in the prerequisites' trees---the\nresulting pack will have no object that is expressed as a delta\nagainst anything outside the pack, and the above \"index-pack\" would\nsucceed.\n\nBesides, \"git pack-objects --thin\" is *not* obligated to create a\npack that lacks one or more objects. The \"--thin\" option merely\n*allows* pack-objects to omit base objects if it is convenient to do\nso.\n\n> +\t\tmv \"$pack\" \"$pack\"-thin &&\n> +\t\tcat \"$pack\"-thin |\n> +\t\t\tgit index-pack --stdin --fix-thin \"$pack\"\n\nThis side is good, but do not cat a single file into a pipe.\nThe whole \"then\" clause would become\n\n\tthen\n\t\tmv \"$pack\" \"$pack-thin\" &&\n\t\tgit index-pack --stdin --fix-thin \"$pack\" <\"$pack-thin\"\n\telse\n\nI would think.\n\n> +\telse\n> +\t\tgit index-pack \"$pack\"\n> +\tfi &&\n> +\tgit verify-pack -v \"$pack\" >verify.out\n> +\tif test $? -ne 0\n> +\tthen\n> +\t\techo >&2 \"error: fail to convert $bundle to $pack\"\n> +\t\treturn 1\n> +\tfi\n\nAt this point, we are not testing the bundle subcommand, but is\ntesting \"git index-pack --fix-thin\" that we run ourselves. Is it\nessential to ensure $pack is sane here? I doubt it.\n\n> +\tcount=$(grep -c \"^$OID_REGEX \" verify.out) &&\n\nAnd if there is no need to run verify-pack, then we can do\ncount=$(git show-index \"${pack%pack}idx\" | wc -l) instead, perhaps?\n\n> +\ttest $2 = $count && return 0\n> +\techo >&2 \"error: object count for $bundle is $count, not $2\"\n> +\treturn 1\n> +}\n> +\n> +# Display the pack data contained in the bundle file, bypassing the\n> +# header that contains the signature, prerequisites and references.\n> +convert_bundle_to_pack () {\n> +\twhile read x && test -n \"$x\"\n> +\tdo\n> +\t\t:;\n> +\tdone\n> +\tcat\n> +}\n\nThis looks somewhat familiar. Perhaps extract out necessary helpers\nincluding this one into t/test-bundle-lib or something similar in a\npreparatory step before this patch?\n\n> +# Create a commit or tag and set the variable with the object ID.\n> +test_commit_setvar () {\n> +\tnotick= &&\n> +\tsignoff= &&\n> +\tindir= &&\n> +\tmerge= &&\n> +\ttag= &&\n> +\tvar= &&\n> +\twhile test $# != 0\n> +\tdo\n> +\t\tcase \"$1\" in\n> +\t\t--merge)\n> +\t\t\tmerge=yes\n> +\t\t\t;;\n> +\t\t--tag)\n> +\t\t\ttag=yes\n> +\t\t\t;;\n> +\t\t--notick)\n> +\t\t\tnotick=yes\n> +\t\t\t;;\n> +\t\t--signoff)\n> +\t\t\tsignoff=\"$1\"\n> +\t\t\t;;\n> +\t\t-C)\n> +\t\t\tindir=\"$2\"\n> +\t\t\tshift\n> +\t\t\t;;\n> +\t\t-*)\n> +\t\t\techo >&2 \"error: unknown option $1\"\n> +\t\t\treturn 1\n> +\t\t\t;;\n> +\t\t*)\n> +\t\t\ttest -n \"$var\" && break\n> +\t\t\tvar=$1\n> +\t\t\t;;\n> +\t\tesac\n> +\t\tshift\n> +\tdone &&\n\nAt this point, if $var is still empty, the caller is buggy, and ...\n\n> +\tindir=${indir:+\"$indir\"/} &&\n> +\tif test $# -eq 0\n> +\tthen\n> +\t\techo >&2 \"no args provided\"\n> +\t\treturn 1\n> +\tfi &&\n> +\tif test -z \"$notick\"\n> +\tthen\n> +\t\ttest_tick\n> +\tfi &&\n> +\tif test -n \"$merge\"\n> +\tthen\n> +\t\tgit ${indir:+ -C \"$indir\"} merge --no-edit --no-ff \\\n> +\t\t\t${2:+-m \"$2\"} \"$1\" &&\n> +\t\toid=$(git ${indir:+ -C \"$indir\"} rev-parse HEAD)\n> +\telif test -n \"$tag\"\n> +\tthen\n> +\t\tgit ${indir:+ -C \"$indir\"} tag -m \"$1\" \"$1\" &&\n> +\t\toid=$(git ${indir:+ -C \"$indir\"} rev-parse \"$1\")\n> +\telse\n> +\t\tfile=${2:-\"$1.t\"} &&\n> +\t\techo \"${3-$1}\" > \"$indir$file\" &&\n> +\t\tgit ${indir:+ -C \"$indir\"} add \"$file\" &&\n> +\t\tgit ${indir:+ -C \"$indir\"} commit $signoff -m \"$1\" &&\n> +\t\toid=$(git ${indir:+ -C \"$indir\"} rev-parse HEAD)\n> +\tfi &&\n> +\teval $var=$oid\n> +}\n\n... it will cause a failure in 'eval' we have here. Not good.\n\n> +# Format the output of git commands to make a user-friendly and stable\n> +# text. We can easily prepare the expect text without having to worry\n> +# about future changes of the commit ID and spaces of the output.\n\nHmph. This relies on 7 hexdigits being sufficient to uniquely\nidentify all objects involved in the test? It should be OK in\npractice.\n\nIs there a point in having both and ? I would\nhave expected that all these \"full object name\" conversions are\nunneeded.\n\n> +make_user_friendly_and_stable_output () {\n> +\tsed \\\n> +\t\t-e \"s/$A//\" \\\n> +\t\t-e \"s/$B//\" \\\n> +\t\t-e \"s/$C//\" \\\n> +\t\t-e \"s/$D//\" \\\n> +\t\t-e \"s/$E//\" \\\n> +\t\t-e \"s/$F//\" \\\n> +\t\t-e \"s/$G//\" \\\n> +\t\t-e \"s/$H//\" \\\n> +\t\t-e \"s/$I//\" \\\n> +\t\t-e \"s/$J//\" \\\n> +\t\t-e \"s/$K//\" \\\n> +\t\t-e \"s/$L//\" \\\n> +\t\t-e \"s/$M//\" \\\n> +\t\t-e \"s/$N//\" \\\n> +\t\t-e \"s/$O//\" \\\n> +\t\t-e \"s/$P//\" \\\n> +\t\t-e \"s/$TAG1//\" \\\n> +\t\t-e \"s/$TAG2//\" \\\n> +\t\t-e \"s/$TAG3//\" \\\n> +\t\t-e \"s/$(echo $A | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $B | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $C | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $D | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $E | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $F | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $G | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $H | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $I | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $J | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $K | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $L | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $M | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $N | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $O | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $P | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $TAG1 | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $TAG2 | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/$(echo $TAG3 | cut -c1-7)[0-9a-f]*//g\" \\\n> +\t\t-e \"s/ *\\$//\"\n> +}\n> ...\n> +test_expect_success 'create bundle from special rev: main^!' '\n> +\tgit bundle create special-rev.bdl \"main^!\" &&\n> +\n> +\tgit bundle list-heads special-rev.bdl |\n> +\t\tmake_user_friendly_and_stable_output >actual &&\n> +\tcat >expect <<-EOF &&\n> +\t\t refs/heads/main\n> +\t\tEOF\n\nWe prefer to indent these more like so:\n\n\tcat >expect <<-\\EOF &&\n\t refs/heads/main\n\tEOF\n\ni.e. the indent of the line with < writes:\n\n> diff --git a/t/t5607-clone-bundle.sh b/t/t5607-clone-bundle.sh\n> index 26985f4b44..425258767d 100755\n> --- a/t/t5607-clone-bundle.sh\n> +++ b/t/t5607-clone-bundle.sh\n> @@ -38,13 +38,13 @@ test_expect_success 'die if bundle file cannot be created' '\n> \ttest_must_fail git bundle create adir --all\n> '\n> \n> -test_expect_failure 'bundle --stdin' '\n> +test_expect_success 'bundle --stdin' '\n> \techo master | git bundle create stdin-bundle.bdl --stdin &&\n> \tgit ls-remote stdin-bundle.bdl >output &&\n> \tgrep master output\n> '\n> \n> -test_expect_failure 'bundle --stdin ' '\n> +test_expect_success 'bundle --stdin ' '\n> \techo master | git bundle create hybrid-bundle.bdl --stdin tag &&\n> \tgit ls-remote hybrid-bundle.bdl >output &&\n> \tgrep master output\n\nI think these tests that documented the known limitation came from\nf62e0a39 (t5704 (bundle): add tests for bundle --stdin, 2010-04-19).\n\nThe author(s) of the original test deserves to be notified ;-)\n\n"},{"id":"413911","messageId":"CANYiYbGcXORT-kryEngy17_J1g3FDKbty9wVXj1U5OWHu8yM8g@mail.gmail.com","threadId":"54923","inReplyTo":"xmqqv9c6g8r4.fsf@gitster.c.googlers.com","subject":"Re: [PATCH v4 1/2] bundle: lost objects when removing duplicate pendings","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-01-09T13:32:18Z","receivedAt":"2021-01-09T13:33:14Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"Junio C Hamano 于2021年1月9日周六 上午10:11写道：\n>\n> Jiang Xin writes:\n>\n> > diff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh\n> > new file mode 100755\n> > index 0000000000..c4447ca88f\n> > --- /dev/null\n> > +++ b/t/t6020-bundle-misc.sh\n> > @@ -0,0 +1,477 @@\n> > +#!/bin/sh\n> > +#\n> > +# Copyright (c) 2021 Jiang Xin\n> > +#\n> > +\n> > +test_description='Test git-bundle'\n> > +\n> > +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n> > +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n> > +\n> > +. ./test-lib.sh\n> > +\n> > +# Check count of objects in a bundle file.\n> > +# We can use \"--thin\" opiton to check thin pack, which must be fixed by\n> > +# command `git-index-pack --fix-thin --stdin`.\n> > +test_bundle_object_count () {\n> > + thin= &&\n> > + if test \"$1\" = \"--thin\"\n> > + then\n> > + thin=yes\n> > + shift\n> > + fi &&\n> > + if test $# -ne 2\n> > + then\n> > + echo >&2 \"args should be: \"\n> > + return 1\n> > + fi\n> > + bundle=$1 &&\n> > + pack=${bundle%.bdl}.pack &&\n> > + convert_bundle_to_pack <\"$bundle\" >\"$pack\" &&\n> > + if test -n \"$thin\"\n> > + then\n> > + test_must_fail git index-pack \"$pack\" &&\n>\n> This is overly strict, isn't it?\n\nI want to make sure that the created bundle file which contains a thin\npack is not changed, but now I think this check is not necessary,\ntesting the count of objects is enough.\n\n> Imagine a case where the objects newer revisions introduce have *no*\n> resemblance to the objects in the prerequisites' trees---the\n> resulting pack will have no object that is expressed as a delta\n> against anything outside the pack, and the above \"index-pack\" would\n> succeed.\n>\n> Besides, \"git pack-objects --thin\" is *not* obligated to create a\n> pack that lacks one or more objects. The \"--thin\" option merely\n> *allows* pack-objects to omit base objects if it is convenient to do\n> so.\n>\n> > + mv \"$pack\" \"$pack\"-thin &&\n> > + cat \"$pack\"-thin |\n> > + git index-pack --stdin --fix-thin \"$pack\"\n>\n> This side is good, but do not cat a single file into a pipe.\n> The whole \"then\" clause would become\n>\n> then\n> mv \"$pack\" \"$pack-thin\" &&\n> git index-pack --stdin --fix-thin \"$pack\" <\"$pack-thin\"\n> else\n>\n> I would think.\n\nThat's better.\n\n> > + else\n> > + git index-pack \"$pack\"\n> > + fi &&\n> > + git verify-pack -v \"$pack\" >verify.out\n> > + if test $? -ne 0\n> > + then\n> > + echo >&2 \"error: fail to convert $bundle to $pack\"\n> > + return 1\n> > + fi\n>\n> At this point, we are not testing the bundle subcommand, but is\n> testing \"git index-pack --fix-thin\" that we run ourselves. Is it\n> essential to ensure $pack is sane here? I doubt it.\n\nIf not check the return error code of `git index-pack`, the report\nmessage will be \"error: object count for $bundle is , not $2\". I want\nto give a specific error message for developer.\n\n> > + count=$(grep -c \"^$OID_REGEX \" verify.out) &&\n>\n> And if there is no need to run verify-pack, then we can do\n> count=$(git show-index \"${pack%pack}idx\" | wc -l) instead, perhaps?\n\nWill do.\n\n> > + test $2 = $count && return 0\n> > + echo >&2 \"error: object count for $bundle is $count, not $2\"\n> > + return 1\n> > +}\n> > +\n> > +# Display the pack data contained in the bundle file, bypassing the\n> > +# header that contains the signature, prerequisites and references.\n> > +convert_bundle_to_pack () {\n> > + while read x && test -n \"$x\"\n> > + do\n> > + :;\n> > + done\n> > + cat\n> > +}\n>\n> This looks somewhat familiar. Perhaps extract out necessary helpers\n> including this one into t/test-bundle-lib or something similar in a\n> preparatory step before this patch?\n>\n> > +# Create a commit or tag and set the variable with the object ID.\n> > +test_commit_setvar () {\n> > + notick= &&\n> > + signoff= &&\n> > + indir= &&\n> > + merge= &&\n> > + tag= &&\n> > + var= &&\n> > + while test $# != 0\n> > + do\n> > + case \"$1\" in\n> > + --merge)\n> > + merge=yes\n> > + ;;\n> > + --tag)\n> > + tag=yes\n> > + ;;\n> > + --notick)\n> > + notick=yes\n> > + ;;\n> > + --signoff)\n> > + signoff=\"$1\"\n> > + ;;\n> > + -C)\n> > + indir=\"$2\"\n> > + shift\n> > + ;;\n> > + -*)\n> > + echo >&2 \"error: unknown option $1\"\n> > + return 1\n> > + ;;\n> > + *)\n> > + test -n \"$var\" && break\n> > + var=$1\n\nThe loop ends only if $var has been assigned a value, or no other\nargs. Will report error if no other args later.\n\n> > + ;;\n> > + esac\n> > + shift\n> > + done &&\n>\n> At this point, if $var is still empty, the caller is buggy, and ...\n\nSee the above note.\n\n> > + indir=${indir:+\"$indir\"/} &&\n> > + if test $# -eq 0\n> > + then\n> > + echo >&2 \"no args provided\"\n> > + return 1\n> > + fi &&\n> > + if test -z \"$notick\"\n> > + then\n> > + test_tick\n> > + fi &&\n> > + if test -n \"$merge\"\n> > + then\n> > + git ${indir:+ -C \"$indir\"} merge --no-edit --no-ff \\\n> > + ${2:+-m \"$2\"} \"$1\" &&\n> > + oid=$(git ${indir:+ -C \"$indir\"} rev-parse HEAD)\n> > + elif test -n \"$tag\"\n> > + then\n> > + git ${indir:+ -C \"$indir\"} tag -m \"$1\" \"$1\" &&\n> > + oid=$(git ${indir:+ -C \"$indir\"} rev-parse \"$1\")\n> > + else\n> > + file=${2:-\"$1.t\"} &&\n> > + echo \"${3-$1}\" > \"$indir$file\" &&\n> > + git ${indir:+ -C \"$indir\"} add \"$file\" &&\n> > + git ${indir:+ -C \"$indir\"} commit $signoff -m \"$1\" &&\n> > + oid=$(git ${indir:+ -C \"$indir\"} rev-parse HEAD)\n> > + fi &&\n> > + eval $var=$oid\n> > +}\n>\n> ... it will cause a failure in 'eval' we have here. Not good.\n>\n> > +# Format the output of git commands to make a user-friendly and stable\n> > +# text. We can easily prepare the expect text without having to worry\n> > +# about future changes of the commit ID and spaces of the output.\n>\n> Hmph. This relies on 7 hexdigits being sufficient to uniquely\n> identify all objects involved in the test? It should be OK in\n> practice.\n>\n> Is there a point in having both and ? I would\n> have expected that all these \"full object name\" conversions are\n> unneeded.\n\nWill do.\n\n> > +make_user_friendly_and_stable_output () {\n> > + sed \\\n> > + -e \"s/$A//\" \\\n> > + -e \"s/$B//\" \\\n> > + -e \"s/$C//\" \\\n> > + -e \"s/$D//\" \\\n> > + -e \"s/$E//\" \\\n> > + -e \"s/$F//\" \\\n> > + -e \"s/$G//\" \\\n> > + -e \"s/$H//\" \\\n> > + -e \"s/$I//\" \\\n> > + -e \"s/$J//\" \\\n> > + -e \"s/$K//\" \\\n> > + -e \"s/$L//\" \\\n> > + -e \"s/$M//\" \\\n> > + -e \"s/$N//\" \\\n> > + -e \"s/$O//\" \\\n> > + -e \"s/$P//\" \\\n> > + -e \"s/$TAG1//\" \\\n> > + -e \"s/$TAG2//\" \\\n> > + -e \"s/$TAG3//\" \\\n> > + -e \"s/$(echo $A | cut -c1-7)[0-9a-f]*//g\" \\\n> > + -e \"s/$(echo $B | cut -c1-7)[0-9a-f]*//g\" \\\n> > + -e \"s/$(echo $C | cut -c1-7)[0-9a-f]*//g\" \\\n> > + -e \"s/$(echo $D | cut -c1-7)[0-9a-f]*//g\" \\\n> > + -e \"s/$(echo $E | cut -c1-7)[0-9a-f]*//g\" \\\n> > + -e \"s/$(echo $F | cut -c1-7)[0-9a-f]*//g\" \\\n> > + -e \"s/$(echo $G | cut -c1-7)[0-9a-f]*//g\" \\\n> > + -e \"s/$(echo $H | cut -c1-7)[0-9a-f]*//g\" \\\n> > + -e \"s/$(echo $I | cut -c1-7)[0-9a-f]*//g\" \\\n> > + -e \"s/$(echo $J | cut -c1-7)[0-9a-f]*//g\" \\\n> > + -e \"s/$(echo $K | cut -c1-7)[0-9a-f]*//g\" \\\n> > + -e \"s/$(echo $L | cut -c1-7)[0-9a-f]*//g\" \\\n> > + -e \"s/$(echo $M | cut -c1-7)[0-9a-f]*//g\" \\\n> > + -e \"s/$(echo $N | cut -c1-7)[0-9a-f]*//g\" \\\n> > + -e \"s/$(echo $O | cut -c1-7)[0-9a-f]*//g\" \\\n> > + -e \"s/$(echo $P | cut -c1-7)[0-9a-f]*//g\" \\\n> > + -e \"s/$(echo $TAG1 | cut -c1-7)[0-9a-f]*//g\" \\\n> > + -e \"s/$(echo $TAG2 | cut -c1-7)[0-9a-f]*//g\" \\\n> > + -e \"s/$(echo $TAG3 | cut -c1-7)[0-9a-f]*//g\" \\\n> > + -e \"s/ *\\$//\"\n> > +}\n> > ...\n> > +test_expect_success 'create bundle from special rev: main^!' '\n> > + git bundle create special-rev.bdl \"main^!\" &&\n> > +\n> > + git bundle list-heads special-rev.bdl |\n> > + make_user_friendly_and_stable_output >actual &&\n> > + cat >expect <<-EOF &&\n> > + refs/heads/main\n> > + EOF\n>\n> We prefer to indent these more like so:\n>\n> cat >expect <<-\\EOF &&\n> refs/heads/main\n> EOF\n>\n> i.e. the indent of the line with < line with the matching EOF are the same. Also, quote EOF to signal\n> that the body of the here text should be taken as-is without $var\n> substitution.\n>\n"},{"id":"413915","messageId":"CANYiYbGR0ByQ4p9oqPvZCq=GFRpbFsHZETz-UwGc_amf1QmXYw@mail.gmail.com","threadId":"54923","inReplyTo":"xmqqv9c6g8r4.fsf@gitster.c.googlers.com","subject":"Re: [PATCH v4 1/2] bundle: lost objects when removing duplicate pendings","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-01-09T15:09:38Z","receivedAt":"2021-01-09T15:10:48Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"Junio C Hamano 于2021年1月9日周六 上午10:11写道：\n> > +# Display the pack data contained in the bundle file, bypassing the\n> > +# header that contains the signature, prerequisites and references.\n> > +convert_bundle_to_pack () {\n> > + while read x && test -n \"$x\"\n> > + do\n> > + :;\n> > + done\n> > + cat\n> > +}\n>\n> This looks somewhat familiar. Perhaps extract out necessary helpers\n> including this one into t/test-bundle-lib or something similar in a\n> preparatory step before this patch?\n\nWill add a new helper \"t/test-bundle-functions.sh\". But as how to\ninclude this helper, there are several solutions, which is better?\n\n1. For those scripts which use these shared functions include this helper like:\n\n . ./test-lib.sh\n # current directory changed, so add path prefix\n . \"$TEST_DIRECTORY\"/test-bundle-functions.sh\n\n2. Include \"test-bundle-functions.sh\" in \"test-lib.sh\" like this:\n\n . \"$TEST_DIRECTORY/test-lib-functions.sh\"\n . \"$TEST_DIRECTORY/test-bundle-functions.sh\"\n\n3. Create a \"t/test-lib.d/\" directory, add add\n“test-bundle-functions.sh” inside \"t/test-lib.d/\" as an extension,\njust like chriscool introduced in sharness project.\n\n----8<-----\n # test_perf subshells can have them too\n . \"$TEST_DIRECTORY/test-lib-functions.sh\"\n\n+if test -d \"$TEST_DIRECTORY/test-lib.d\"\n+then\n+ for file in \"$TEST_DIRECTORY/test-lib.d/\"*.sh\n+ do\n+ # Ensure glob was not an empty match:\n+ test -e \"${file}\" || break\n+\n+ if test -n \"$debug\"\n+ then\n+ echo >&5 \"test-lib: loading extensions from ${file}\"\n+ fi\n+ . \"${file}\"\n+ if test $? != 0\n+ then\n+ echo >&5 \"test-lib: Error loading ${file}. Aborting.\"\n+ exit 1\n+ fi\n+ done\n+fi\n+\n----->8-----\n"},{"id":"413930","messageId":"xmqqbldxg466.fsf@gitster.c.googlers.com","threadId":"54923","inReplyTo":"CANYiYbGcXORT-kryEngy17_J1g3FDKbty9wVXj1U5OWHu8yM8g@mail.gmail.com","subject":"Re: [PATCH v4 1/2] bundle: lost objects when removing duplicate pendings","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-01-09T22:02:09Z","receivedAt":"2021-01-09T22:02:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jiang Xin writes:\n\n>> > +# Create a commit or tag and set the variable with the object ID.\n>> > +test_commit_setvar () {\n>> > + notick= &&\n>> > + signoff= &&\n>> > + indir= &&\n>> > + merge= &&\n>> > + tag= &&\n>> > + var= &&\n>> > + while test $# != 0\n>> > + do\n>> > + case \"$1\" in\n>> > ...\n>> > + -*)\n>> > + echo >&2 \"error: unknown option $1\"\n>> > + return 1\n>> > + ;;\n>> > + *)\n>> > + test -n \"$var\" && break\n>> > + var=$1\n>\n> The loop ends only if $var has been assigned a value, or no other\n> args. Will report error if no other args later.\n\n\"We see an arg that is not an dashed option. We already saw an arg\nand taken it as the name of the variable, so break\" was a misleading\nway to structure this loop.\n\nIt looked as if it was just refusing to parse the remainder of the\ncommand line, so that a check after the loop would complain if there\nis still remaining arguments (as if to warn \"var given twice\"). But\nthat is not what the post-loop check does; it expects there still\nare some argument left to be processed in mode-specific code that\nfollows, so it happens to work as intended.\n\nThat is brittle, though. The current code may always consume one or\nmore extra arguments in $merge/$tag/other specific mode in the code\nafter the loop, but a new mode that will get added in the future to\nsit next to --merge and --tag may learn all necessary info in the\ncommand line parsing loop above, without any need for extra args to\nbe processed after the loop. And $#=0 may not always be an error at\nthat point.\n\nI forgot to notice / mention it, but now you made me to look at the\nloop again, I see this part\n\n\t-C)\n\t\tindir=\"$2\"\n\t\tshift\n\t\t;;\n\ndoes not ensure we are getting something sensible in -C; that\npotential bug by the caller also happens to be covered by the\npost-loop \"we require at least one argument that we can use as an\narg\" check, but as I said already, that feels rather brittle.\n\nAnyway, let's queue the patches as-is and see what others think.\n\nThanks.\n"},{"id":"413931","messageId":"xmqq7dolg44z.fsf@gitster.c.googlers.com","threadId":"54923","inReplyTo":"CANYiYbGR0ByQ4p9oqPvZCq=GFRpbFsHZETz-UwGc_amf1QmXYw@mail.gmail.com","subject":"Re: [PATCH v4 1/2] bundle: lost objects when removing duplicate pendings","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-01-09T22:02:52Z","receivedAt":"2021-01-09T22:03:56Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jiang Xin writes:\n\n> 1. For those scripts which use these shared functions include this helper like:\n>\n> . ./test-lib.sh\n> # current directory changed, so add path prefix\n> . \"$TEST_DIRECTORY\"/test-bundle-functions.sh\n\nThis one. But without double spaces after the dot.\n"},{"id":"413969","messageId":"20210110143019.5625-1-worldhello.net@gmail.com","threadId":"54923","inReplyTo":"xmqqbldxg466.fsf@gitster.c.googlers.com","subject":"[PATCH v5 0/3] improvements for git-bundle","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-01-10T14:30:16Z","receivedAt":"2021-01-10T14:31:21Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"From: Jiang Xin \n\nIntroduce two improvements for git-bundle\n\n+ Commit \"bundle: lost objects when removing duplicate pendings\",\n which fixes command like:\n\n $ git bundle create 'master^!'\n \n+ Commits \"bundle: arguments can be read from stdin\",\n which add \"--stdin\" option support for git-bundle, like:\n\n $ git bundle create \n\nMove git-bundle related functions from t5510 to a library, and this lib\nwill be shared with a new testcase t6020 which finds a known breakage of\n\"git-bundle\".\n\nSigned-off-by: Jiang Xin \n---\n t/t5510-fetch.sh | 26 +--\n t/t6020-bundle-misc.sh | 396 +++++++++++++++++++++++++++++++++++++\n t/test-bundle-functions.sh | 47 +++++\n 3 files changed, 447 insertions(+), 22 deletions(-)\n create mode 100755 t/t6020-bundle-misc.sh\n create mode 100644 t/test-bundle-functions.sh\n\ndiff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh\nindex 2013051a64..1e398380eb 100755\n--- a/t/t5510-fetch.sh\n+++ b/t/t5510-fetch.sh\n@@ -6,22 +6,10 @@ test_description='Per branch config variables affects \"git fetch\".\n '\n \n . ./test-lib.sh\n+. \"$TEST_DIRECTORY\"/test-bundle-functions.sh\n \n D=$(pwd)\n \n-test_bundle_object_count () {\n-\tgit verify-pack -v \"$1\" >verify.out &&\n-\ttest \"$2\" = $(grep \"^$OID_REGEX \" verify.out | wc -l)\n-}\n-\n-convert_bundle_to_pack () {\n-\twhile read x && test -n \"$x\"\n-\tdo\n-\t\t:;\n-\tdone\n-\tcat\n-}\n-\n test_expect_success setup '\n \techo >file original &&\n \tgit add file &&\n@@ -312,9 +300,7 @@ test_expect_success 'unbundle 1' '\n \n test_expect_success 'bundle 1 has only 3 files ' '\n \tcd \"$D\" &&\n-\tconvert_bundle_to_pack bundle.pack &&\n-\tgit index-pack bundle.pack &&\n-\ttest_bundle_object_count bundle.pack 3\n+\ttest_bundle_object_count bundle1 3\n '\n \n test_expect_success 'unbundle 2' '\n@@ -329,9 +315,7 @@ test_expect_success 'bundle does not prerequisite objects' '\n \tgit add file2 &&\n \tgit commit -m add.file2 file2 &&\n \tgit bundle create bundle3 -1 HEAD &&\n-\tconvert_bundle_to_pack bundle.pack &&\n-\tgit index-pack bundle.pack &&\n-\ttest_bundle_object_count bundle.pack 3\n+\ttest_bundle_object_count bundle3 3\n '\n \n test_expect_success 'bundle should be able to create a full history' '\n@@ -884,9 +868,7 @@ test_expect_success 'all boundary commits are excluded' '\n \tgit merge otherside &&\n \tad=$(git log --no-walk --format=%ad HEAD) &&\n \tgit bundle create twoside-boundary.bdl main --since=\"$ad\" &&\n-\tconvert_bundle_to_pack twoside-boundary.pack &&\n-\tpack=$(git index-pack --fix-thin --stdin &2 \"error: unknown option $1\"\n+\t\t\treturn 1\n+\t\t\t;;\n+\t\t*)\n+\t\t\tbreak\n+\t\t\t;;\n+\t\tesac\n+\t\tshift\n+\tdone\n+\n+\tvar=$1\n+\tshift\n+\tif test -z \"$var\"\n+\tthen\n+\t\techo >&2 \"error: var is not defined\"\n+\t\treturn 1\n+\tfi\n+\tindir=${indir:+\"$indir\"/}\n+\tif test -z \"$notick\"\n+\tthen\n+\t\ttest_tick\n+\tfi &&\n+\tif test -n \"$merge\"\n+\tthen\n+\t\tgit ${indir:+ -C \"$indir\"} merge --no-edit --no-ff \\\n+\t\t\t${2:+-m \"$2\"} \"$1\" &&\n+\t\toid=$(git ${indir:+ -C \"$indir\"} rev-parse HEAD)\n+\telif test -n \"$tag\"\n+\tthen\n+\t\tgit ${indir:+ -C \"$indir\"} tag -m \"$1\" \"$1\" &&\n+\t\toid=$(git ${indir:+ -C \"$indir\"} rev-parse \"$1\")\n+\telse\n+\t\tfile=${2:-\"$1.t\"} &&\n+\t\techo \"${3-$1}\" > \"$indir$file\" &&\n+\t\tgit ${indir:+ -C \"$indir\"} add \"$file\" &&\n+\t\tgit ${indir:+ -C \"$indir\"} commit $signoff -m \"$1\" &&\n+\t\toid=$(git ${indir:+ -C \"$indir\"} rev-parse HEAD)\n+\tfi &&\n+\teval $var=$oid\n+}\n+\n+\n+# Format the output of git commands to make a user-friendly and stable\n+# text. We can easily prepare the expect text without having to worry\n+# about future changes of the commit ID and spaces of the output.\n+make_user_friendly_and_stable_output () {\n+\tsed \\\n+\t\t-e \"s/$(echo $A | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $B | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $C | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $D | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $E | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $F | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $G | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $H | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $I | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $J | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $K | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $L | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $M | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $N | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $O | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $P | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $TAG1 | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $TAG2 | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(echo $TAG3 | cut -c1-7)[0-9a-f]*//g\" \\\n+\t\t-e \"s/ *\\$//\"\n+}\n+\n+# (C) (D, pull/1/head, topic/1)\n+# o --- o\n+# / \\ (L)\n+# / \\ o (H, topic/2) (M, tag:v2)\n+# / (F) \\ / (N, tag:v3)\n+# / o --------- o (G, pull/2/head) o --- o --- o (release)\n+# / / \\ \\ / \\\n+# o --- o --- o -------- o -- o ------------------ o ------- o --- o (main)\n+# (A) (B) (E, tag:v1) (I) (J) (K) (O) (P)\n+#\n+test_expect_success 'setup' '\n+\t# Try to make a stable fixed width for abbreviated commit ID,\n+\t# this fixed-width oid will be replaced with \"\".\n+\tgit config core.abbrev 7 &&\n+\n+\t# branch main: commit A & B\n+\ttest_commit_setvar A \"Commit A\" main.txt &&\n+\ttest_commit_setvar B \"Commit B\" main.txt &&\n+\n+\t# branch topic/1: commit C & D, refs/pull/1/head\n+\tgit checkout -b topic/1 &&\n+\ttest_commit_setvar C \"Commit C\" topic-1.txt &&\n+\ttest_commit_setvar D \"Commit D\" topic-1.txt &&\n+\tgit update-ref refs/pull/1/head HEAD &&\n+\n+\t# branch topic/1: commit E, tag v1\n+\tgit checkout main &&\n+\ttest_commit_setvar E \"Commit E\" main.txt &&\n+\ttest_commit_setvar --tag TAG1 v1 &&\n+\n+\t# branch topic/2: commit F & G, refs/pull/2/head\n+\tgit checkout -b topic/2 &&\n+\ttest_commit_setvar F \"Commit F\" topic-2.txt &&\n+\ttest_commit_setvar G \"Commit G\" topic-2.txt &&\n+\tgit update-ref refs/pull/2/head HEAD &&\n+\ttest_commit_setvar H \"Commit H\" topic-2.txt &&\n+\n+\t# branch main: merge commit I & J\n+\tgit checkout main &&\n+\ttest_commit_setvar --merge I topic/1 \"Merge commit I\" &&\n+\ttest_commit_setvar --merge J refs/pull/2/head \"Merge commit J\" &&\n+\n+\t# branch main: commit K\n+\tgit checkout main &&\n+\ttest_commit_setvar K \"Commit K\" main.txt &&\n+\n+\t# branch release:\n+\tgit checkout -b release &&\n+\ttest_commit_setvar L \"Commit L\" release.txt &&\n+\ttest_commit_setvar M \"Commit M\" release.txt &&\n+\ttest_commit_setvar --tag TAG2 v2 &&\n+\ttest_commit_setvar N \"Commit N\" release.txt &&\n+\ttest_commit_setvar --tag TAG3 v3 &&\n+\n+\t# branch main: merge commit O, commit P\n+\tgit checkout main &&\n+\ttest_commit_setvar --merge O tags/v2 \"Merge commit O\" &&\n+\ttest_commit_setvar P \"Commit P\" main.txt\n+'\n+\n+test_expect_failure 'create bundle from special rev: main^!' '\n+\tgit bundle create special-rev.bdl \"main^!\" &&\n+\n+\tgit bundle list-heads special-rev.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-\\EOF &&\n+\t refs/heads/main\n+\tEOF\n+\ttest_i18ncmp expect actual &&\n+\n+\tgit bundle verify special-rev.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-\\EOF &&\n+\tThe bundle contains this ref:\n+\t refs/heads/main\n+\tThe bundle requires this ref:\n+\t\n+\tEOF\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count special-rev.bdl 3\n+'\n+\n+test_expect_success 'create bundle with --max-count option' '\n+\tgit bundle create max-count.bdl --max-count 1 \\\n+\t\tmain \\\n+\t\t\"^release\" \\\n+\t\trefs/tags/v1 \\\n+\t\trefs/pull/1/head \\\n+\t\trefs/pull/2/head &&\n+\n+\tgit bundle verify max-count.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-\\EOF &&\n+\tThe bundle contains these 2 refs:\n+\t refs/heads/main\n+\t refs/tags/v1\n+\tThe bundle requires this ref:\n+\t\n+\tEOF\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count max-count.bdl 4\n+'\n+\n+test_expect_success 'create bundle with --since option' '\n+\tgit log -1 --pretty=\"%ad\" $M >actual &&\n+\tcat >expect <<-\\EOF &&\n+\tThu Apr 7 15:26:13 2005 -0700\n+\tEOF\n+\ttest_cmp expect actual &&\n+\n+\tgit bundle create since.bdl \\\n+\t\t--since \"Thu Apr 7 15:27:00 2005 -0700\" \\\n+\t\t--all &&\n+\n+\tgit bundle verify since.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-\\EOF &&\n+\tThe bundle contains these 5 refs:\n+\t refs/heads/main\n+\t refs/heads/release\n+\t refs/tags/v2\n+\t refs/tags/v3\n+\t HEAD\n+\tThe bundle requires these 2 refs:\n+\t\n+\t\n+\tEOF\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count --thin since.bdl 13\n+'\n+\n+test_expect_success 'create bundle 1 - no prerequisites' '\n+\tgit bundle create 1.bdl topic/1 topic/2 &&\n+\n+\tcat >expect <<-\\EOF &&\n+\tThe bundle contains these 2 refs:\n+\t refs/heads/topic/1\n+\t refs/heads/topic/2\n+\tThe bundle records a complete history.\n+\tEOF\n+\n+\t# verify bundle, which has no prerequisites\n+\tgit bundle verify 1.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 1.bdl 24\n+'\n+\n+test_expect_success 'create bundle 2 - has prerequisites' '\n+\tgit bundle create 2.bdl \\\n+\t\t--ignore-missing \\\n+\t\t^topic/deleted \\\n+\t\t^$D \\\n+\t\t^topic/2 \\\n+\t\trelease &&\n+\n+\tcat >expect <<-\\EOF &&\n+\tThe bundle contains this ref:\n+\t refs/heads/release\n+\tThe bundle requires these 3 refs:\n+\t\n+\t\n+\t\n+\tEOF\n+\n+\tgit bundle verify 2.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 2.bdl 16\n+'\n+\n+test_expect_success 'fail to verify bundle without prerequisites' '\n+\tgit init --bare test1.git &&\n+\n+\tcat >expect <<-\\EOF &&\n+\terror: Repository lacks these prerequisite commits:\n+\terror: \n+\terror: \n+\terror: \n+\tEOF\n+\n+\ttest_must_fail git -C test1.git bundle verify ../2.bdl 2>&1 |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual\n+'\n+\n+test_expect_success 'create bundle 3 - two refs, same object' '\n+\tgit bundle create --version=3 3.bdl \\\n+\t\t^release \\\n+\t\t^topic/1 \\\n+\t\t^topic/2 \\\n+\t\tmain \\\n+\t\tHEAD &&\n+\n+\tcat >expect <<-\\EOF &&\n+\tThe bundle contains these 2 refs:\n+\t refs/heads/main\n+\t HEAD\n+\tThe bundle requires these 2 refs:\n+\t\n+\t\n+\tEOF\n+\n+\tgit bundle verify 3.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 3.bdl 4\n+'\n+\n+test_expect_success 'create bundle 4 - with tags' '\n+\tgit bundle create 4.bdl \\\n+\t\t^main \\\n+\t\t^release \\\n+\t\t^topic/1 \\\n+\t\t^topic/2 \\\n+\t\t--all &&\n+\n+\tcat >expect <<-\\EOF &&\n+\tThe bundle contains these 3 refs:\n+\t refs/tags/v1\n+\t refs/tags/v2\n+\t refs/tags/v3\n+\tThe bundle records a complete history.\n+\tEOF\n+\n+\tgit bundle verify 4.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 4.bdl 3\n+'\n+\n+test_expect_success 'clone from bundle' '\n+\tgit clone --mirror 1.bdl mirror.git &&\n+\tgit -C mirror.git show-ref |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-\\EOF &&\n+\t refs/heads/topic/1\n+\t refs/heads/topic/2\n+\tEOF\n+\ttest_cmp expect actual &&\n+\n+\tgit -C mirror.git fetch ../2.bdl \"+refs/*:refs/*\" &&\n+\tgit -C mirror.git show-ref |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-\\EOF &&\n+\t refs/heads/release\n+\t refs/heads/topic/1\n+\t refs/heads/topic/2\n+\tEOF\n+\ttest_cmp expect actual &&\n+\n+\tgit -C mirror.git fetch ../3.bdl \"+refs/*:refs/*\" &&\n+\tgit -C mirror.git show-ref |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-\\EOF &&\n+\t refs/heads/main\n+\t refs/heads/release\n+\t refs/heads/topic/1\n+\t refs/heads/topic/2\n+\tEOF\n+\ttest_cmp expect actual &&\n+\n+\tgit -C mirror.git fetch ../4.bdl \"+refs/*:refs/*\" &&\n+\tgit -C mirror.git show-ref |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-\\EOF &&\n+\t refs/heads/main\n+\t refs/heads/release\n+\t refs/heads/topic/1\n+\t refs/heads/topic/2\n+\t refs/tags/v1\n+\t refs/tags/v2\n+\t refs/tags/v3\n+\tEOF\n+\ttest_cmp expect actual\n+'\n+\n+test_done\ndiff --git a/t/test-bundle-functions.sh b/t/test-bundle-functions.sh\nnew file mode 100644\nindex 0000000000..0853eb1eca\n--- /dev/null\n+++ b/t/test-bundle-functions.sh\n@@ -0,0 +1,47 @@\n+# Library of git-bundle related functions.\n+\n+# Display the pack data contained in the bundle file, bypassing the\n+# header that contains the signature, prerequisites and references.\n+convert_bundle_to_pack () {\n+\twhile read x && test -n \"$x\"\n+\tdo\n+\t\t:;\n+\tdone\n+\tcat\n+}\n+\n+# Check count of objects in a bundle file.\n+# We can use \"--thin\" opiton to check thin pack, which must be fixed by\n+# command `git-index-pack --fix-thin --stdin`.\n+test_bundle_object_count () {\n+\tthin=\n+\tif test \"$1\" = \"--thin\"\n+\tthen\n+\t\tthin=t\n+\t\tshift\n+\tfi\n+\tif test $# -ne 2\n+\tthen\n+\t\techo >&2 \"args should be: \"\n+\t\treturn 1\n+\tfi\n+\tbundle=$1\n+\tpack=$bundle.pack\n+\tconvert_bundle_to_pack <\"$bundle\" >\"$pack\" &&\n+\tif test -n \"$thin\"\n+\tthen\n+\t\tmv \"$pack\" \"$bundle.thin.pack\" &&\n+\t\tgit index-pack --stdin --fix-thin \"$pack\" <\"$bundle.thin.pack\"\n+\telse\n+\t\tgit index-pack \"$pack\"\n+\tfi\n+\tif test $? -ne 0\n+\tthen\n+\t\techo >&2 \"error: fail to convert $bundle or index-pack\"\n+\t\treturn 1\n+\tfi\n+\tcount=$(git show-index <\"${pack%pack}idx\" | wc -l) &&\n+\ttest $2 = $count && return 0\n+\techo >&2 \"error: object count for $bundle is $count, not $2\"\n+\treturn 1\n+}\n-- \n2.30.0.2.g06d2f50715\n\n"},{"id":"413971","messageId":"20210110143019.5625-3-worldhello.net@gmail.com","threadId":"54923","inReplyTo":"xmqqbldxg466.fsf@gitster.c.googlers.com","subject":"[PATCH v5 2/3] bundle: lost objects when removing duplicate pendings","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-01-10T14:30:18Z","receivedAt":"2021-01-10T14:31:21Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"From: Jiang Xin \n\n`git rev-list` will list one commit for the following command:\n\n $ git rev-list 'main^!'\n \n\nBut providing the same rev-list args to `git bundle`, fail to create\na bundle file.\n\n $ git bundle create - 'main^!'\n # v2 git bundle\n - \n\n fatal: Refusing to create empty bundle.\n\nThis is because when removing duplicate objects in function\n`object_array_remove_duplicates()`, one unique pending object which has\nthe same name is deleted by mistake. The revision arg 'main^!' in the\nabove example is parsed by `handle_revision_arg()`, and at lease two\ndifferent objects will be appended to `revs.pending`, one points to the\nparent commit of the \"main\" branch, and the other points to the tip\ncommit of the \"main\" branch. These two objects have the same name\n\"main\". Only one object is left with the name \"main\" after calling the\nfunction `object_array_remove_duplicates()`.\n\nAnd what's worse, when adding boundary commits into pending list, we use\none-line commit message as names, and the arbitory names may surprise\ngit-bundle.\n\nOnly comparing objects themselves (\".item\") is also not good enough,\nbecause user may want to create a bundle with two identical objects but\nwith different reference names, such as: \"HEAD\" and \"refs/heads/main\".\n\nAdd new function `contains_object()` which compare both the address and\nthe name of the object.\n\nSigned-off-by: Jiang Xin \n---\n object.c | 10 ++++++----\n t/t6020-bundle-misc.sh | 2 +-\n 2 files changed, 7 insertions(+), 5 deletions(-)\n\ndiff --git a/object.c b/object.c\nindex 68f80b0b3d..98017bed8e 100644\n--- a/object.c\n+++ b/object.c\n@@ -412,15 +412,16 @@ void object_array_clear(struct object_array *array)\n }\n \n /*\n- * Return true iff array already contains an entry with name.\n+ * Return true if array already contains an entry.\n */\n-static int contains_name(struct object_array *array, const char *name)\n+static int contains_object(struct object_array *array,\n+\t\t\t const struct object *item, const char *name)\n {\n \tunsigned nr = array->nr, i;\n \tstruct object_array_entry *object = array->objects;\n \n \tfor (i = 0; i < nr; i++, object++)\n-\t\tif (!strcmp(object->name, name))\n+\t\tif (item == object->item && !strcmp(object->name, name))\n \t\t\treturn 1;\n \treturn 0;\n }\n@@ -432,7 +433,8 @@ void object_array_remove_duplicates(struct object_array *array)\n \n \tarray->nr = 0;\n \tfor (src = 0; src < nr; src++) {\n-\t\tif (!contains_name(array, objects[src].name)) {\n+\t\tif (!contains_object(array, objects[src].item,\n+\t\t\t\t objects[src].name)) {\n \t\t\tif (src != array->nr)\n \t\t\t\tobjects[array->nr] = objects[src];\n \t\t\tarray->nr++;\ndiff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh\nindex 637cdb5a8e..5c9d5ed94d 100755\n--- a/t/t6020-bundle-misc.sh\n+++ b/t/t6020-bundle-misc.sh\n@@ -169,7 +169,7 @@ test_expect_success 'setup' '\n \ttest_commit_setvar P \"Commit P\" main.txt\n '\n \n-test_expect_failure 'create bundle from special rev: main^!' '\n+test_expect_success 'create bundle from special rev: main^!' '\n \tgit bundle create special-rev.bdl \"main^!\" &&\n \n \tgit bundle list-heads special-rev.bdl |\n-- \n2.30.0.2.g06d2f50715\n\n"},{"id":"413972","messageId":"20210110143019.5625-4-worldhello.net@gmail.com","threadId":"54923","inReplyTo":"xmqqbldxg466.fsf@gitster.c.googlers.com","subject":"[PATCH v5 3/3] bundle: arguments can be read from stdin","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-01-10T14:30:19Z","receivedAt":"2021-01-10T14:31:21Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"From: Jiang Xin \n\nIn order to create an incremental bundle, we need to pass many arguments\nto let git-bundle ignore some already packed commits. It will be more\nconvenient to pass args via stdin. But the current implementation does\nnot allow us to do this.\n\nThis is because args are parsed twice when creating bundle. The first\ntime for parsing args is in `compute_and_write_prerequisites()` by\nrunning `git-rev-list` command to write prerequisites in bundle file,\nand stdin is consumed in this step if \"--stdin\" option is provided for\n`git-bundle`. Later nothing can be read from stdin when running\n`setup_revisions()` in `create_bundle()`.\n\nThe solution is to parse args once by removing the entire function\n`compute_and_write_prerequisites()` and then calling function\n`setup_revisions()`. In order to write prerequisites for bundle, will\ncall `prepare_revision_walk()` and `traverse_commit_list()`. But after\ncalling `prepare_revision_walk()`, the object array `revs.pending` is\nleft empty, and the following steps could not work properly with the\nempty object array (`revs.pending`). Therefore, make a copy of `revs`\nto `revs_copy` for later use right after calling `setup_revisions()`.\n\nThe copy of `revs_copy` is not a deep copy, it shares the same objects\nwith `revs`. The object array of `revs` has been cleared, but objects\nthemselves are still kept. Flags of objects may change after calling\n`prepare_revision_walk()`, we can use these changed flags without\ncalling the `git rev-list` command and parsing its output like the\nformer implementation.\n\nAlso add testcases for git bundle in t6020, which read args from stdin.\n\nSigned-off-by: Jiang Xin \n---\n bundle.c | 109 ++++++++++++++++++++++------------------\n t/t5607-clone-bundle.sh | 4 +-\n t/t6020-bundle-misc.sh | 77 ++++++++++++++++++++++++++--\n 3 files changed, 134 insertions(+), 56 deletions(-)\n\ndiff --git a/bundle.c b/bundle.c\nindex cb0e5931ac..693d619551 100644\n--- a/bundle.c\n+++ b/bundle.c\n@@ -338,48 +338,6 @@ static int write_pack_data(int bundle_fd, struct rev_info *revs, struct strvec *\n \treturn 0;\n }\n \n-static int compute_and_write_prerequisites(int bundle_fd,\n-\t\t\t\t\t struct rev_info *revs,\n-\t\t\t\t\t int argc, const char **argv)\n-{\n-\tstruct child_process rls = CHILD_PROCESS_INIT;\n-\tstruct strbuf buf = STRBUF_INIT;\n-\tFILE *rls_fout;\n-\tint i;\n-\n-\tstrvec_pushl(&rls.args,\n-\t\t \"rev-list\", \"--boundary\", \"--pretty=oneline\",\n-\t\t NULL);\n-\tfor (i = 1; i < argc; i++)\n-\t\tstrvec_push(&rls.args, argv[i]);\n-\trls.out = -1;\n-\trls.git_cmd = 1;\n-\tif (start_command(&rls))\n-\t\treturn -1;\n-\trls_fout = xfdopen(rls.out, \"r\");\n-\twhile (strbuf_getwholeline(&buf, rls_fout, '\\n') != EOF) {\n-\t\tstruct object_id oid;\n-\t\tif (buf.len > 0 && buf.buf[0] == '-') {\n-\t\t\twrite_or_die(bundle_fd, buf.buf, buf.len);\n-\t\t\tif (!get_oid_hex(buf.buf + 1, &oid)) {\n-\t\t\t\tstruct object *object = parse_object_or_die(&oid,\n-\t\t\t\t\t\t\t\t\t buf.buf);\n-\t\t\t\tobject->flags |= UNINTERESTING;\n-\t\t\t\tadd_pending_object(revs, object, buf.buf);\n-\t\t\t}\n-\t\t} else if (!get_oid_hex(buf.buf, &oid)) {\n-\t\t\tstruct object *object = parse_object_or_die(&oid,\n-\t\t\t\t\t\t\t\t buf.buf);\n-\t\t\tobject->flags |= SHOWN;\n-\t\t}\n-\t}\n-\tstrbuf_release(&buf);\n-\tfclose(rls_fout);\n-\tif (finish_command(&rls))\n-\t\treturn error(_(\"rev-list died\"));\n-\treturn 0;\n-}\n-\n /*\n * Write out bundle refs based on the tips already\n * parsed into revs.pending. As a side effect, may\n@@ -474,6 +432,38 @@ static int write_bundle_refs(int bundle_fd, struct rev_info *revs)\n \treturn ref_count;\n }\n \n+struct bundle_prerequisites_info {\n+\tstruct object_array *pending;\n+\tint fd;\n+};\n+\n+static void write_bundle_prerequisites(struct commit *commit, void *data)\n+{\n+\tstruct bundle_prerequisites_info *bpi = data;\n+\tstruct object *object;\n+\tstruct pretty_print_context ctx = { 0 };\n+\tstruct strbuf buf = STRBUF_INIT;\n+\n+\tif (!(commit->object.flags & BOUNDARY))\n+\t\treturn;\n+\tstrbuf_addf(&buf, \"-%s \", oid_to_hex(&commit->object.oid));\n+\twrite_or_die(bpi->fd, buf.buf, buf.len);\n+\n+\tctx.fmt = CMIT_FMT_ONELINE;\n+\tctx.output_encoding = get_log_output_encoding();\n+\tstrbuf_reset(&buf);\n+\tpretty_print_commit(&ctx, commit, &buf);\n+\tstrbuf_trim(&buf);\n+\n+\tobject = (struct object *)commit;\n+\tobject->flags |= UNINTERESTING;\n+\tadd_object_array_with_path(object, buf.buf, bpi->pending, S_IFINVALID,\n+\t\t\t\t NULL);\n+\tstrbuf_addch(&buf, '\\n');\n+\twrite_or_die(bpi->fd, buf.buf, buf.len);\n+\tstrbuf_release(&buf);\n+}\n+\n int create_bundle(struct repository *r, const char *path,\n \t\t int argc, const char **argv, struct strvec *pack_options, int version)\n {\n@@ -481,8 +471,10 @@ int create_bundle(struct repository *r, const char *path,\n \tint bundle_fd = -1;\n \tint bundle_to_stdout;\n \tint ref_count = 0;\n-\tstruct rev_info revs;\n+\tstruct rev_info revs, revs_copy;\n \tint min_version = the_hash_algo == &hash_algos[GIT_HASH_SHA1] ? 2 : 3;\n+\tstruct bundle_prerequisites_info bpi;\n+\tint i;\n \n \tbundle_to_stdout = !strcmp(path, \"-\");\n \tif (bundle_to_stdout)\n@@ -512,10 +504,6 @@ int create_bundle(struct repository *r, const char *path,\n \tsave_commit_buffer = 0;\n \trepo_init_revisions(r, &revs, NULL);\n \n-\t/* write prerequisites */\n-\tif (compute_and_write_prerequisites(bundle_fd, &revs, argc, argv))\n-\t\tgoto err;\n-\n \targc = setup_revisions(argc, argv, &revs, NULL);\n \n \tif (argc > 1) {\n@@ -523,16 +511,37 @@ int create_bundle(struct repository *r, const char *path,\n \t\tgoto err;\n \t}\n \n-\tobject_array_remove_duplicates(&revs.pending);\n+\t/* save revs.pending in revs_copy for later use */\n+\tmemcpy(&revs_copy, &revs, sizeof(revs));\n+\trevs_copy.pending.nr = 0;\n+\trevs_copy.pending.alloc = 0;\n+\trevs_copy.pending.objects = NULL;\n+\tfor (i = 0; i < revs.pending.nr; i++) {\n+\t\tstruct object_array_entry *e = revs.pending.objects + i;\n+\t\tif (e)\n+\t\t\tadd_object_array_with_path(e->item, e->name,\n+\t\t\t\t\t\t &revs_copy.pending,\n+\t\t\t\t\t\t e->mode, e->path);\n+\t}\n \n-\tref_count = write_bundle_refs(bundle_fd, &revs);\n+\t/* write prerequisites */\n+\trevs.boundary = 1;\n+\tif (prepare_revision_walk(&revs))\n+\t\tdie(\"revision walk setup failed\");\n+\tbpi.fd = bundle_fd;\n+\tbpi.pending = &revs_copy.pending;\n+\ttraverse_commit_list(&revs, write_bundle_prerequisites, NULL, &bpi);\n+\tobject_array_remove_duplicates(&revs_copy.pending);\n+\n+\t/* write bundle refs */\n+\tref_count = write_bundle_refs(bundle_fd, &revs_copy);\n \tif (!ref_count)\n \t\tdie(_(\"Refusing to create empty bundle.\"));\n \telse if (ref_count < 0)\n \t\tgoto err;\n \n \t/* write pack */\n-\tif (write_pack_data(bundle_fd, &revs, pack_options))\n+\tif (write_pack_data(bundle_fd, &revs_copy, pack_options))\n \t\tgoto err;\n \n \tif (!bundle_to_stdout) {\ndiff --git a/t/t5607-clone-bundle.sh b/t/t5607-clone-bundle.sh\nindex 26985f4b44..425258767d 100755\n--- a/t/t5607-clone-bundle.sh\n+++ b/t/t5607-clone-bundle.sh\n@@ -38,13 +38,13 @@ test_expect_success 'die if bundle file cannot be created' '\n \ttest_must_fail git bundle create adir --all\n '\n \n-test_expect_failure 'bundle --stdin' '\n+test_expect_success 'bundle --stdin' '\n \techo master | git bundle create stdin-bundle.bdl --stdin &&\n \tgit ls-remote stdin-bundle.bdl >output &&\n \tgrep master output\n '\n \n-test_expect_failure 'bundle --stdin ' '\n+test_expect_success 'bundle --stdin ' '\n \techo master | git bundle create hybrid-bundle.bdl --stdin tag &&\n \tgit ls-remote hybrid-bundle.bdl >output &&\n \tgrep master output\ndiff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh\nindex 5c9d5ed94d..e9158a22a0 100755\n--- a/t/t6020-bundle-misc.sh\n+++ b/t/t6020-bundle-misc.sh\n@@ -244,8 +244,16 @@ test_expect_success 'create bundle with --since option' '\n '\n \n test_expect_success 'create bundle 1 - no prerequisites' '\n+\t# create bundle from args\n \tgit bundle create 1.bdl topic/1 topic/2 &&\n \n+\t# create bundle from stdin\n+\tcat >input <<-\\EOF &&\n+\ttopic/1\n+\ttopic/2\n+\tEOF\n+\tgit bundle create stdin-1.bdl --stdin expect <<-\\EOF &&\n \tThe bundle contains these 2 refs:\n \t refs/heads/topic/1\n@@ -258,10 +266,16 @@ test_expect_success 'create bundle 1 - no prerequisites' '\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual &&\n \n-\ttest_bundle_object_count 1.bdl 24\n+\tgit bundle verify stdin-1.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 1.bdl 24 &&\n+\ttest_bundle_object_count stdin-1.bdl 24\n '\n \n test_expect_success 'create bundle 2 - has prerequisites' '\n+\t# create bundle from args\n \tgit bundle create 2.bdl \\\n \t\t--ignore-missing \\\n \t\t^topic/deleted \\\n@@ -269,6 +283,18 @@ test_expect_success 'create bundle 2 - has prerequisites' '\n \t\t^topic/2 \\\n \t\trelease &&\n \n+\t# create bundle from stdin\n+\t# input has a non-exist reference: \"topic/deleted\"\n+\tcat >input <<-EOF &&\n+\t^topic/deleted\n+\t^$D\n+\t^topic/2\n+\tEOF\n+\tgit bundle create stdin-2.bdl \\\n+\t\t--ignore-missing \\\n+\t\t--stdin \\\n+\t\trelease expect <<-\\EOF &&\n \tThe bundle contains this ref:\n \t refs/heads/release\n@@ -282,7 +308,12 @@ test_expect_success 'create bundle 2 - has prerequisites' '\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual &&\n \n-\ttest_bundle_object_count 2.bdl 16\n+\tgit bundle verify stdin-2.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 2.bdl 16 &&\n+\ttest_bundle_object_count stdin-2.bdl 16\n '\n \n test_expect_success 'fail to verify bundle without prerequisites' '\n@@ -297,10 +328,15 @@ test_expect_success 'fail to verify bundle without prerequisites' '\n \n \ttest_must_fail git -C test1.git bundle verify ../2.bdl 2>&1 |\n \t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_must_fail git -C test1.git bundle verify ../stdin-2.bdl 2>&1 |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual\n '\n \n test_expect_success 'create bundle 3 - two refs, same object' '\n+\t# create bundle from args\n \tgit bundle create --version=3 3.bdl \\\n \t\t^release \\\n \t\t^topic/1 \\\n@@ -308,6 +344,16 @@ test_expect_success 'create bundle 3 - two refs, same object' '\n \t\tmain \\\n \t\tHEAD &&\n \n+\t# create bundle from stdin\n+\tcat >input <<-\\EOF &&\n+\t^release\n+\t^topic/1\n+\t^topic/2\n+\tEOF\n+\tgit bundle create --version=3 stdin-3.bdl \\\n+\t\t--stdin \\\n+\t\tmain HEAD expect <<-\\EOF &&\n \tThe bundle contains these 2 refs:\n \t refs/heads/main\n@@ -321,10 +367,16 @@ test_expect_success 'create bundle 3 - two refs, same object' '\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual &&\n \n-\ttest_bundle_object_count 3.bdl 4\n+\tgit bundle verify stdin-3.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 3.bdl 4 &&\n+\ttest_bundle_object_count stdin-3.bdl 4\n '\n \n test_expect_success 'create bundle 4 - with tags' '\n+\t# create bundle from args\n \tgit bundle create 4.bdl \\\n \t\t^main \\\n \t\t^release \\\n@@ -332,6 +384,18 @@ test_expect_success 'create bundle 4 - with tags' '\n \t\t^topic/2 \\\n \t\t--all &&\n \n+\t# create bundle from stdin\n+\tcat >input <<-\\EOF &&\n+\t^main\n+\t^release\n+\t^topic/1\n+\t^topic/2\n+\tEOF\n+\tgit bundle create stdin-4.bdl \\\n+\t\t--ignore-missing \\\n+\t\t--stdin \\\n+\t\t--all expect <<-\\EOF &&\n \tThe bundle contains these 3 refs:\n \t refs/tags/v1\n@@ -344,7 +408,12 @@ test_expect_success 'create bundle 4 - with tags' '\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual &&\n \n-\ttest_bundle_object_count 4.bdl 3\n+\tgit bundle verify stdin-4.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 4.bdl 3 &&\n+\ttest_bundle_object_count stdin-4.bdl 3\n '\n \n test_expect_success 'clone from bundle' '\n-- \n2.30.0.2.g06d2f50715\n\n"},{"id":"414072","messageId":"xmqq1rer8cbz.fsf@gitster.c.googlers.com","threadId":"54923","inReplyTo":"20210110143019.5625-2-worldhello.net@gmail.com","subject":"Re: [PATCH v5 1/3] test: add helper functions for git-bundle","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-01-11T20:09:52Z","receivedAt":"2021-01-11T20:10:49Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jiang Xin writes:\n\n> +# Create a commit or tag and set the variable with the object ID.\n> +test_commit_setvar () {\n> +\tnotick=\n> +\tsignoff=\n> +\tindir=\n> +\tmerge=\n> +\ttag=\n> +\tvar=\n> +\n> +\twhile test $# != 0\n> +\tdo\n> +\t\tcase \"$1\" in\n> +\t\t--merge)\n> +\t\t\tmerge=t\n> +\t\t\t;;\n> +\t\t--tag)\n> +\t\t\ttag=t\n> +\t\t\t;;\n> +\t\t--notick)\n> +\t\t\tnotick=t\n> +\t\t\t;;\n> +\t\t--signoff)\n> +\t\t\tsignoff=\"$1\"\n> +\t\t\t;;\n> +\t\t-C)\n> +\t\t\tshift\n> +\t\t\tindir=\"$1\"\n> +\t\t\t;;\n> +\t\t-*)\n> +\t\t\techo >&2 \"error: unknown option $1\"\n> +\t\t\treturn 1\n> +\t\t\t;;\n> +\t\t*)\n> +\t\t\tbreak\n> +\t\t\t;;\n> +\t\tesac\n> +\t\tshift\n> +\tdone\n> +\n> +\tvar=$1\n> +\tshift\n> +\tif test -z \"$var\"\n> +\tthen\n> +\t\techo >&2 \"error: var is not defined\"\n> +\t\treturn 1\n> +\tfi\n\nWe need to check $# immediately after the loop to ensure that we can\ncarve out $var and at least another arg. [*Nit 1*]\n\nThe previous round required the command line to have at least one\nafter the loop (including parsing of $var) parsed it, but now we\nfall through from here when a command line were:\n\n test_commit_setvar --merge -C there VAR\n\nand because \"$1\" does not exist, such an error is propagated down to\n\"git merge\" not getting the side branch, \"git tag\" not getting the\nobject to tag, etc.\n\n> +\tindir=${indir:+\"$indir\"/}\n> +\tif test -z \"$notick\"\n> +\tthen\n> +\t\ttest_tick\n> +\tfi &&\n> +\tif test -n \"$merge\"\n> +\tthen\n> +\t\tgit ${indir:+ -C \"$indir\"} merge --no-edit --no-ff \\\n> +\t\t\t${2:+-m \"$2\"} \"$1\" &&\n> +\t\toid=$(git ${indir:+ -C \"$indir\"} rev-parse HEAD)\n> +\telif test -n \"$tag\"\n> +\tthen\n> +\t\tgit ${indir:+ -C \"$indir\"} tag -m \"$1\" \"$1\" &&\n> +\t\toid=$(git ${indir:+ -C \"$indir\"} rev-parse \"$1\")\n> +\telse\n> +\t\tfile=${2:-\"$1.t\"} &&\n> +\t\techo \"${3-$1}\" > \"$indir$file\" &&\n\nStyle? [*Nit 2*]\n\n> +\t\tgit ${indir:+ -C \"$indir\"} add \"$file\" &&\n> +\t\tgit ${indir:+ -C \"$indir\"} commit $signoff -m \"$1\" &&\n> +\t\toid=$(git ${indir:+ -C \"$indir\"} rev-parse HEAD)\n> +\tfi &&\n> +\teval $var=$oid\n> +}\n\n\n> +\n> +\n> +# Format the output of git commands to make a user-friendly and stable\n> +# text. We can easily prepare the expect text without having to worry\n> +# about future changes of the commit ID and spaces of the output.\n> +make_user_friendly_and_stable_output () {\n> +\tsed \\\n> +\t\t-e \"s/$(echo $A | cut -c1-7)[0-9a-f]*//g\" \\\n\nIs \"$(echo $A | cut -c1-7)\" the same as \"${A%${A#???????}}\"? If so,\nthe latter may be a bit shorter.\n\n> diff --git a/t/test-bundle-functions.sh b/t/test-bundle-functions.sh\n> new file mode 100644\n> index 0000000000..0853eb1eca\n> --- /dev/null\n> +++ b/t/test-bundle-functions.sh\n> @@ -0,0 +1,47 @@\n> +# Library of git-bundle related functions.\n> +\n> +# Display the pack data contained in the bundle file, bypassing the\n> +# header that contains the signature, prerequisites and references.\n> +convert_bundle_to_pack () {\n> +\twhile read x && test -n \"$x\"\n> +\tdo\n> +\t\t:;\n> +\tdone\n> +\tcat\n> +}\n> +\n> +# Check count of objects in a bundle file.\n> +# We can use \"--thin\" opiton to check thin pack, which must be fixed by\n> +# command `git-index-pack --fix-thin --stdin`.\n> +test_bundle_object_count () {\n> +\tthin=\n> +\tif test \"$1\" = \"--thin\"\n> +\tthen\n> +\t\tthin=t\n> +\t\tshift\n> +\tfi\n> +\tif test $# -ne 2\n> +\tthen\n> +\t\techo >&2 \"args should be: \"\n> +\t\treturn 1\n> +\tfi\n> +\tbundle=$1\n> +\tpack=$bundle.pack\n> +\tconvert_bundle_to_pack <\"$bundle\" >\"$pack\" &&\n> +\tif test -n \"$thin\"\n> +\tthen\n> +\t\tmv \"$pack\" \"$bundle.thin.pack\" &&\n> +\t\tgit index-pack --stdin --fix-thin \"$pack\" <\"$bundle.thin.pack\"\n> +\telse\n> +\t\tgit index-pack \"$pack\"\n> +\tfi\n\nI wonder why we shouldn't always do \"--fix-thin\", so that the caller\ndoes not even have to bother passing the \"--thin\" option.\n\nIs this to protect us from \"git bundle\" creating a bundle that\ncontains a thin pack when it should not? A caller that knows it is\nstoring a fully connected history can deliberately omit \"--thin\"\nfrom the command line and make sure \"index-pack\" that is not asked\nto do \"--fix-thin\" indeed finds the pack data fully self-contained,\nso it may be a good idea to have these two separate codepaths after\nall. OK.\n\n> +\tif test $? -ne 0\n> +\tthen\n> +\t\techo >&2 \"error: fail to convert $bundle or index-pack\"\n> +\t\treturn 1\n> +\tfi\n\nDo we even need the \"error\" message? \"git index-pack\" would have\nalready given some error message to its standard error stream, no?\nIf so\n\n\tif test -n \"$thin\"\n\tthen\n\t\t...\n\tfi || return 1\n\nwould be sufficient, I guess.\n\n> +\tcount=$(git show-index <\"${pack%pack}idx\" | wc -l) &&\n> +\ttest $2 = $count && return 0\n> +\techo >&2 \"error: object count for $bundle is $count, not $2\"\n> +\treturn 1\n> +}\n\nLooking good except for a few minor nits I mentioned above.\n\nThanks.\n"},{"id":"414073","messageId":"xmqqwnwj6xnu.fsf@gitster.c.googlers.com","threadId":"54923","inReplyTo":"20210110143019.5625-3-worldhello.net@gmail.com","subject":"Re: [PATCH v5 2/3] bundle: lost objects when removing duplicate pendings","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-01-11T20:12:05Z","receivedAt":"2021-01-11T20:12:52Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jiang Xin writes:\n\n> /*\n> - * Return true iff array already contains an entry with name.\n> + * Return true if array already contains an entry.\n> */\n\nI think the original deliberately spells \"if and only if\", but a\nnatural reading of \"return true if X\" would be \"this would never\nreturn true if not X\", so perhaps the above change is OK.\n\n> -static int contains_name(struct object_array *array, const char *name)\n> +static int contains_object(struct object_array *array,\n> +\t\t\t const struct object *item, const char *name)\n> {\n> \tunsigned nr = array->nr, i;\n> \tstruct object_array_entry *object = array->objects;\n> \n> \tfor (i = 0; i < nr; i++, object++)\n> -\t\tif (!strcmp(object->name, name))\n> +\t\tif (item == object->item && !strcmp(object->name, name))\n> \t\t\treturn 1;\n> \treturn 0;\n> }\n> @@ -432,7 +433,8 @@ void object_array_remove_duplicates(struct object_array *array)\n> \n> \tarray->nr = 0;\n> \tfor (src = 0; src < nr; src++) {\n> -\t\tif (!contains_name(array, objects[src].name)) {\n> +\t\tif (!contains_object(array, objects[src].item,\n> +\t\t\t\t objects[src].name)) {\n> \t\t\tif (src != array->nr)\n> \t\t\t\tobjects[array->nr] = objects[src];\n> \t\t\tarray->nr++;\n> diff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh\n> index 637cdb5a8e..5c9d5ed94d 100755\n> --- a/t/t6020-bundle-misc.sh\n> +++ b/t/t6020-bundle-misc.sh\n> @@ -169,7 +169,7 @@ test_expect_success 'setup' '\n> \ttest_commit_setvar P \"Commit P\" main.txt\n> '\n> \n> -test_expect_failure 'create bundle from special rev: main^!' '\n> +test_expect_success 'create bundle from special rev: main^!' '\n> \tgit bundle create special-rev.bdl \"main^!\" &&\n> \n> \tgit bundle list-heads special-rev.bdl |\nn\n"},{"id":"414089","messageId":"20210112022703.1884-1-worldhello.net@gmail.com","threadId":"54923","inReplyTo":"xmqq1rer8cbz.fsf@gitster.c.googlers.com","subject":"[PATCH v6 0/3] improvements for git-bundle","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-01-12T02:27:00Z","receivedAt":"2021-01-12T02:28:07Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"From: Jiang Xin \n\nIntroduce two improvements for git-bundle\n\n+ Commit \"bundle: lost objects when removing duplicate pendings\",\n which fixes command like:\n\n $ git bundle create 'master^!'\n\n+ Commits \"bundle: arguments can be read from stdin\",\n which add \"--stdin\" option support for git-bundle, like:\n\n $ git bundle create writes:\n>\n> Jiang Xin writes:\n>\n> > +     var=$1\n> > +     shift\n> > +     if test -z \"$var\"\n> > +     then\n> > +             echo >&2 \"error: var is not defined\"\n> > +             return 1\n> > +     fi\n>\n> We need to check $# immediately after the loop to ensure that we can\n> carve out $var and at least another arg.  [*Nit 1*]\n>\n> The previous round required the command line to have at least one\n> after the loop (including parsing of $var) parsed it, but now we\n> fall through from here when a command line were:\n>\n>     test_commit_setvar --merge -C there VAR\n>\n> and because \"$1\" does not exist, such an error is propagated down to\n> \"git merge\" not getting the side branch, \"git tag\" not getting the\n> object to tag, etc.\n\nrange-diff v5...v6 (part-1):\n\n @@ t/t6020-bundle-misc.sh (new)\n +\t\tesac\n +\t\tshift\n +\tdone\n -+\n -+\tvar=$1\n -+\tshift\n -+\tif test -z \"$var\"\n ++\tif test $# -lt 2\n +\tthen\n -+\t\techo >&2 \"error: var is not defined\"\n ++\t\techo >&2 \"error: test_commit_setvar must have at least 2 arguments\"\n +\t\treturn 1\n +\tfi\n ++\tvar=$1\n ++\tshift\n +\tindir=${indir:+\"$indir\"/}\n +\tif test -z \"$notick\"\n +\tthen\n\n> > +     else\n> > +             file=${2:-\"$1.t\"} &&\n> > +             echo \"${3-$1}\" > \"$indir$file\" &&\n>\n> Style?  [*Nit 2*]\n\nrange-diff v5...v6 (part-2):\n\n* Add new arg \"${2:-HEAD}\" for `git tag`.\n* Fix [*Nit 2*].\n\n @@ t/t6020-bundle-misc.sh (new)\n +\t\toid=$(git ${indir:+ -C \"$indir\"} rev-parse HEAD)\n +\telif test -n \"$tag\"\n +\tthen\n -+\t\tgit ${indir:+ -C \"$indir\"} tag -m \"$1\" \"$1\" &&\n ++\t\tgit ${indir:+ -C \"$indir\"} tag -m \"$1\" \"$1\" \"${2:-HEAD}\" &&\n +\t\toid=$(git ${indir:+ -C \"$indir\"} rev-parse \"$1\")\n +\telse\n +\t\tfile=${2:-\"$1.t\"} &&\n -+\t\techo \"${3-$1}\" > \"$indir$file\" &&\n ++\t\techo \"${3-$1}\" >\"$indir$file\" &&\n +\t\tgit ${indir:+ -C \"$indir\"} add \"$file\" &&\n +\t\tgit ${indir:+ -C \"$indir\"} commit $signoff -m \"$1\" &&\n +\t\toid=$(git ${indir:+ -C \"$indir\"} rev-parse HEAD)\n\n> > +# Format the output of git commands to make a user-friendly and stable\n> > +# text.  We can easily prepare the expect text without having to worry\n> > +# about future changes of the commit ID and spaces of the output.\n> > +make_user_friendly_and_stable_output () {\n> > +     sed \\\n> > +             -e \"s/$(echo $A | cut -c1-7)[0-9a-f]*//g\" \\\n>\n> Is \"$(echo $A | cut -c1-7)\" the same as \"${A%${A#???????}}\"?  If so,\n> the latter may be a bit shorter.\n\n\nrange-diff v5...v6 (part-3):\n\n @@ t/t6020-bundle-misc.sh (new)\n +\teval $var=$oid\n +}\n +\n -+\n +# Format the output of git commands to make a user-friendly and stable\n +# text. We can easily prepare the expect text without having to worry\n +# about future changes of the commit ID and spaces of the output.\n +make_user_friendly_and_stable_output () {\n +\tsed \\\n -+\t\t-e \"s/$(echo $A | cut -c1-7)[0-9a-f]*//g\" \\\n -+\t\t-e \"s/$(echo $B | cut -c1-7)[0-9a-f]*//g\" \\\n -+\t\t-e \"s/$(echo $C | cut -c1-7)[0-9a-f]*//g\" \\\n -+\t\t-e \"s/$(echo $D | cut -c1-7)[0-9a-f]*//g\" \\\n -+\t\t-e \"s/$(echo $E | cut -c1-7)[0-9a-f]*//g\" \\\n -+\t\t-e \"s/$(echo $F | cut -c1-7)[0-9a-f]*//g\" \\\n -+\t\t-e \"s/$(echo $G | cut -c1-7)[0-9a-f]*//g\" \\\n -+\t\t-e \"s/$(echo $H | cut -c1-7)[0-9a-f]*//g\" \\\n -+\t\t-e \"s/$(echo $I | cut -c1-7)[0-9a-f]*//g\" \\\n -+\t\t-e \"s/$(echo $J | cut -c1-7)[0-9a-f]*//g\" \\\n -+\t\t-e \"s/$(echo $K | cut -c1-7)[0-9a-f]*//g\" \\\n -+\t\t-e \"s/$(echo $L | cut -c1-7)[0-9a-f]*//g\" \\\n -+\t\t-e \"s/$(echo $M | cut -c1-7)[0-9a-f]*//g\" \\\n -+\t\t-e \"s/$(echo $N | cut -c1-7)[0-9a-f]*//g\" \\\n -+\t\t-e \"s/$(echo $O | cut -c1-7)[0-9a-f]*//g\" \\\n -+\t\t-e \"s/$(echo $P | cut -c1-7)[0-9a-f]*//g\" \\\n -+\t\t-e \"s/$(echo $TAG1 | cut -c1-7)[0-9a-f]*//g\" \\\n -+\t\t-e \"s/$(echo $TAG2 | cut -c1-7)[0-9a-f]*//g\" \\\n -+\t\t-e \"s/$(echo $TAG3 | cut -c1-7)[0-9a-f]*//g\" \\\n ++\t\t-e \"s/${A%${A#???????}}[0-9a-f]*//g\" \\\n ++\t\t-e \"s/${B%${B#???????}}[0-9a-f]*//g\" \\\n ++\t\t-e \"s/${C%${C#???????}}[0-9a-f]*//g\" \\\n ++\t\t-e \"s/${D%${D#???????}}[0-9a-f]*//g\" \\\n ++\t\t-e \"s/${E%${E#???????}}[0-9a-f]*//g\" \\\n ++\t\t-e \"s/${F%${F#???????}}[0-9a-f]*//g\" \\\n ++\t\t-e \"s/${G%${G#???????}}[0-9a-f]*//g\" \\\n ++\t\t-e \"s/${H%${H#???????}}[0-9a-f]*//g\" \\\n ++\t\t-e \"s/${I%${I#???????}}[0-9a-f]*//g\" \\\n ++\t\t-e \"s/${J%${J#???????}}[0-9a-f]*//g\" \\\n ++\t\t-e \"s/${K%${K#???????}}[0-9a-f]*//g\" \\\n ++\t\t-e \"s/${L%${L#???????}}[0-9a-f]*//g\" \\\n ++\t\t-e \"s/${M%${M#???????}}[0-9a-f]*//g\" \\\n ++\t\t-e \"s/${N%${N#???????}}[0-9a-f]*//g\" \\\n ++\t\t-e \"s/${O%${O#???????}}[0-9a-f]*//g\" \\\n ++\t\t-e \"s/${P%${P#???????}}[0-9a-f]*//g\" \\\n ++\t\t-e \"s/${TAG1%${TAG1#???????}}[0-9a-f]*//g\" \\\n ++\t\t-e \"s/${TAG2%${TAG2#???????}}[0-9a-f]*//g\" \\\n ++\t\t-e \"s/${TAG3%${TAG3#???????}}[0-9a-f]*//g\" \\\n +\t\t-e \"s/ *\\$//\"\n +}\n +\n\n> Do we even need the \"error\" message?  \"git index-pack\" would have\n> already given some error message to its standard error stream, no?\n> If so\n>\n>         if test -n \"$thin\"\n>         then\n>                 ...\n>         fi || return 1\n>\n> would be sufficient, I guess.\n\nrange-diff v5...v6 (part-4):\n\n1: fa7516b2ec ! 1: 900bb16178 test: add helper functions for git-bundle\n @@ t/test-bundle-functions.sh (new)\n +\t\tgit index-pack --stdin --fix-thin \"$pack\" <\"$bundle.thin.pack\"\n +\telse\n +\t\tgit index-pack \"$pack\"\n -+\tfi\n -+\tif test $? -ne 0\n -+\tthen\n -+\t\techo >&2 \"error: fail to convert $bundle or index-pack\"\n -+\t\treturn 1\n -+\tfi\n ++\tfi || return 1\n +\tcount=$(git show-index <\"${pack%pack}idx\" | wc -l) &&\n +\ttest $2 = $count && return 0\n +\techo >&2 \"error: object count for $bundle is $count, not $2\"\n2: ea543de111 = 2: 1bbf0ab213 bundle: lost objects when removing duplicate pendings\n3: 18f0d48814 = 3: 7ac0751821 bundle: arguments can be read from stdin\n\n--\n\nJiang Xin (3):\n test: add helper functions for git-bundle\n bundle: lost objects when removing duplicate pendings\n bundle: arguments can be read from stdin\n\n bundle.c | 109 +++++----\n object.c | 10 +-\n t/t5510-fetch.sh | 26 +--\n t/t5607-clone-bundle.sh | 4 +-\n t/t6020-bundle-misc.sh | 463 +++++++++++++++++++++++++++++++++++++\n t/test-bundle-functions.sh | 42 ++++\n 6 files changed, 576 insertions(+), 78 deletions(-)\n create mode 100755 t/t6020-bundle-misc.sh\n create mode 100644 t/test-bundle-functions.sh\n\n-- \n2.28.0.15.gba9e81f0bd\n"},{"id":"414090","messageId":"20210112022703.1884-2-worldhello.net@gmail.com","threadId":"54923","inReplyTo":"xmqq1rer8cbz.fsf@gitster.c.googlers.com","subject":"[PATCH v6 1/3] test: add helper functions for git-bundle","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-01-12T02:27:01Z","receivedAt":"2021-01-12T02:28:07Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"From: Jiang Xin \n\nMove git-bundle related functions from t5510 to a library, and this lib\nwill be shared with a new testcase t6020 which finds a known breakage of\n\"git-bundle\".\n\nSigned-off-by: Jiang Xin \n---\n t/t5510-fetch.sh | 26 +--\n t/t6020-bundle-misc.sh | 394 +++++++++++++++++++++++++++++++++++++\n t/test-bundle-functions.sh | 42 ++++\n 3 files changed, 440 insertions(+), 22 deletions(-)\n create mode 100755 t/t6020-bundle-misc.sh\n create mode 100644 t/test-bundle-functions.sh\n\ndiff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh\nindex 2013051a64..1e398380eb 100755\n--- a/t/t5510-fetch.sh\n+++ b/t/t5510-fetch.sh\n@@ -6,22 +6,10 @@ test_description='Per branch config variables affects \"git fetch\".\n '\n \n . ./test-lib.sh\n+. \"$TEST_DIRECTORY\"/test-bundle-functions.sh\n \n D=$(pwd)\n \n-test_bundle_object_count () {\n-\tgit verify-pack -v \"$1\" >verify.out &&\n-\ttest \"$2\" = $(grep \"^$OID_REGEX \" verify.out | wc -l)\n-}\n-\n-convert_bundle_to_pack () {\n-\twhile read x && test -n \"$x\"\n-\tdo\n-\t\t:;\n-\tdone\n-\tcat\n-}\n-\n test_expect_success setup '\n \techo >file original &&\n \tgit add file &&\n@@ -312,9 +300,7 @@ test_expect_success 'unbundle 1' '\n \n test_expect_success 'bundle 1 has only 3 files ' '\n \tcd \"$D\" &&\n-\tconvert_bundle_to_pack bundle.pack &&\n-\tgit index-pack bundle.pack &&\n-\ttest_bundle_object_count bundle.pack 3\n+\ttest_bundle_object_count bundle1 3\n '\n \n test_expect_success 'unbundle 2' '\n@@ -329,9 +315,7 @@ test_expect_success 'bundle does not prerequisite objects' '\n \tgit add file2 &&\n \tgit commit -m add.file2 file2 &&\n \tgit bundle create bundle3 -1 HEAD &&\n-\tconvert_bundle_to_pack bundle.pack &&\n-\tgit index-pack bundle.pack &&\n-\ttest_bundle_object_count bundle.pack 3\n+\ttest_bundle_object_count bundle3 3\n '\n \n test_expect_success 'bundle should be able to create a full history' '\n@@ -884,9 +868,7 @@ test_expect_success 'all boundary commits are excluded' '\n \tgit merge otherside &&\n \tad=$(git log --no-walk --format=%ad HEAD) &&\n \tgit bundle create twoside-boundary.bdl main --since=\"$ad\" &&\n-\tconvert_bundle_to_pack twoside-boundary.pack &&\n-\tpack=$(git index-pack --fix-thin --stdin &2 \"error: unknown option $1\"\n+\t\t\treturn 1\n+\t\t\t;;\n+\t\t*)\n+\t\t\tbreak\n+\t\t\t;;\n+\t\tesac\n+\t\tshift\n+\tdone\n+\tif test $# -lt 2\n+\tthen\n+\t\techo >&2 \"error: test_commit_setvar must have at least 2 arguments\"\n+\t\treturn 1\n+\tfi\n+\tvar=$1\n+\tshift\n+\tindir=${indir:+\"$indir\"/}\n+\tif test -z \"$notick\"\n+\tthen\n+\t\ttest_tick\n+\tfi &&\n+\tif test -n \"$merge\"\n+\tthen\n+\t\tgit ${indir:+ -C \"$indir\"} merge --no-edit --no-ff \\\n+\t\t\t${2:+-m \"$2\"} \"$1\" &&\n+\t\toid=$(git ${indir:+ -C \"$indir\"} rev-parse HEAD)\n+\telif test -n \"$tag\"\n+\tthen\n+\t\tgit ${indir:+ -C \"$indir\"} tag -m \"$1\" \"$1\" \"${2:-HEAD}\" &&\n+\t\toid=$(git ${indir:+ -C \"$indir\"} rev-parse \"$1\")\n+\telse\n+\t\tfile=${2:-\"$1.t\"} &&\n+\t\techo \"${3-$1}\" >\"$indir$file\" &&\n+\t\tgit ${indir:+ -C \"$indir\"} add \"$file\" &&\n+\t\tgit ${indir:+ -C \"$indir\"} commit $signoff -m \"$1\" &&\n+\t\toid=$(git ${indir:+ -C \"$indir\"} rev-parse HEAD)\n+\tfi &&\n+\teval $var=$oid\n+}\n+\n+# Format the output of git commands to make a user-friendly and stable\n+# text. We can easily prepare the expect text without having to worry\n+# about future changes of the commit ID and spaces of the output.\n+make_user_friendly_and_stable_output () {\n+\tsed \\\n+\t\t-e \"s/${A%${A#???????}}[0-9a-f]*//g\" \\\n+\t\t-e \"s/${B%${B#???????}}[0-9a-f]*//g\" \\\n+\t\t-e \"s/${C%${C#???????}}[0-9a-f]*//g\" \\\n+\t\t-e \"s/${D%${D#???????}}[0-9a-f]*//g\" \\\n+\t\t-e \"s/${E%${E#???????}}[0-9a-f]*//g\" \\\n+\t\t-e \"s/${F%${F#???????}}[0-9a-f]*//g\" \\\n+\t\t-e \"s/${G%${G#???????}}[0-9a-f]*//g\" \\\n+\t\t-e \"s/${H%${H#???????}}[0-9a-f]*//g\" \\\n+\t\t-e \"s/${I%${I#???????}}[0-9a-f]*//g\" \\\n+\t\t-e \"s/${J%${J#???????}}[0-9a-f]*//g\" \\\n+\t\t-e \"s/${K%${K#???????}}[0-9a-f]*//g\" \\\n+\t\t-e \"s/${L%${L#???????}}[0-9a-f]*//g\" \\\n+\t\t-e \"s/${M%${M#???????}}[0-9a-f]*//g\" \\\n+\t\t-e \"s/${N%${N#???????}}[0-9a-f]*//g\" \\\n+\t\t-e \"s/${O%${O#???????}}[0-9a-f]*//g\" \\\n+\t\t-e \"s/${P%${P#???????}}[0-9a-f]*//g\" \\\n+\t\t-e \"s/${TAG1%${TAG1#???????}}[0-9a-f]*//g\" \\\n+\t\t-e \"s/${TAG2%${TAG2#???????}}[0-9a-f]*//g\" \\\n+\t\t-e \"s/${TAG3%${TAG3#???????}}[0-9a-f]*//g\" \\\n+\t\t-e \"s/ *\\$//\"\n+}\n+\n+# (C) (D, pull/1/head, topic/1)\n+# o --- o\n+# / \\ (L)\n+# / \\ o (H, topic/2) (M, tag:v2)\n+# / (F) \\ / (N, tag:v3)\n+# / o --------- o (G, pull/2/head) o --- o --- o (release)\n+# / / \\ \\ / \\\n+# o --- o --- o -------- o -- o ------------------ o ------- o --- o (main)\n+# (A) (B) (E, tag:v1) (I) (J) (K) (O) (P)\n+#\n+test_expect_success 'setup' '\n+\t# Try to make a stable fixed width for abbreviated commit ID,\n+\t# this fixed-width oid will be replaced with \"\".\n+\tgit config core.abbrev 7 &&\n+\n+\t# branch main: commit A & B\n+\ttest_commit_setvar A \"Commit A\" main.txt &&\n+\ttest_commit_setvar B \"Commit B\" main.txt &&\n+\n+\t# branch topic/1: commit C & D, refs/pull/1/head\n+\tgit checkout -b topic/1 &&\n+\ttest_commit_setvar C \"Commit C\" topic-1.txt &&\n+\ttest_commit_setvar D \"Commit D\" topic-1.txt &&\n+\tgit update-ref refs/pull/1/head HEAD &&\n+\n+\t# branch topic/1: commit E, tag v1\n+\tgit checkout main &&\n+\ttest_commit_setvar E \"Commit E\" main.txt &&\n+\ttest_commit_setvar --tag TAG1 v1 &&\n+\n+\t# branch topic/2: commit F & G, refs/pull/2/head\n+\tgit checkout -b topic/2 &&\n+\ttest_commit_setvar F \"Commit F\" topic-2.txt &&\n+\ttest_commit_setvar G \"Commit G\" topic-2.txt &&\n+\tgit update-ref refs/pull/2/head HEAD &&\n+\ttest_commit_setvar H \"Commit H\" topic-2.txt &&\n+\n+\t# branch main: merge commit I & J\n+\tgit checkout main &&\n+\ttest_commit_setvar --merge I topic/1 \"Merge commit I\" &&\n+\ttest_commit_setvar --merge J refs/pull/2/head \"Merge commit J\" &&\n+\n+\t# branch main: commit K\n+\tgit checkout main &&\n+\ttest_commit_setvar K \"Commit K\" main.txt &&\n+\n+\t# branch release:\n+\tgit checkout -b release &&\n+\ttest_commit_setvar L \"Commit L\" release.txt &&\n+\ttest_commit_setvar M \"Commit M\" release.txt &&\n+\ttest_commit_setvar --tag TAG2 v2 &&\n+\ttest_commit_setvar N \"Commit N\" release.txt &&\n+\ttest_commit_setvar --tag TAG3 v3 &&\n+\n+\t# branch main: merge commit O, commit P\n+\tgit checkout main &&\n+\ttest_commit_setvar --merge O tags/v2 \"Merge commit O\" &&\n+\ttest_commit_setvar P \"Commit P\" main.txt\n+'\n+\n+test_expect_failure 'create bundle from special rev: main^!' '\n+\tgit bundle create special-rev.bdl \"main^!\" &&\n+\n+\tgit bundle list-heads special-rev.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-\\EOF &&\n+\t refs/heads/main\n+\tEOF\n+\ttest_i18ncmp expect actual &&\n+\n+\tgit bundle verify special-rev.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-\\EOF &&\n+\tThe bundle contains this ref:\n+\t refs/heads/main\n+\tThe bundle requires this ref:\n+\t\n+\tEOF\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count special-rev.bdl 3\n+'\n+\n+test_expect_success 'create bundle with --max-count option' '\n+\tgit bundle create max-count.bdl --max-count 1 \\\n+\t\tmain \\\n+\t\t\"^release\" \\\n+\t\trefs/tags/v1 \\\n+\t\trefs/pull/1/head \\\n+\t\trefs/pull/2/head &&\n+\n+\tgit bundle verify max-count.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-\\EOF &&\n+\tThe bundle contains these 2 refs:\n+\t refs/heads/main\n+\t refs/tags/v1\n+\tThe bundle requires this ref:\n+\t\n+\tEOF\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count max-count.bdl 4\n+'\n+\n+test_expect_success 'create bundle with --since option' '\n+\tgit log -1 --pretty=\"%ad\" $M >actual &&\n+\tcat >expect <<-\\EOF &&\n+\tThu Apr 7 15:26:13 2005 -0700\n+\tEOF\n+\ttest_cmp expect actual &&\n+\n+\tgit bundle create since.bdl \\\n+\t\t--since \"Thu Apr 7 15:27:00 2005 -0700\" \\\n+\t\t--all &&\n+\n+\tgit bundle verify since.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-\\EOF &&\n+\tThe bundle contains these 5 refs:\n+\t refs/heads/main\n+\t refs/heads/release\n+\t refs/tags/v2\n+\t refs/tags/v3\n+\t HEAD\n+\tThe bundle requires these 2 refs:\n+\t\n+\t\n+\tEOF\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count --thin since.bdl 13\n+'\n+\n+test_expect_success 'create bundle 1 - no prerequisites' '\n+\tgit bundle create 1.bdl topic/1 topic/2 &&\n+\n+\tcat >expect <<-\\EOF &&\n+\tThe bundle contains these 2 refs:\n+\t refs/heads/topic/1\n+\t refs/heads/topic/2\n+\tThe bundle records a complete history.\n+\tEOF\n+\n+\t# verify bundle, which has no prerequisites\n+\tgit bundle verify 1.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 1.bdl 24\n+'\n+\n+test_expect_success 'create bundle 2 - has prerequisites' '\n+\tgit bundle create 2.bdl \\\n+\t\t--ignore-missing \\\n+\t\t^topic/deleted \\\n+\t\t^$D \\\n+\t\t^topic/2 \\\n+\t\trelease &&\n+\n+\tcat >expect <<-\\EOF &&\n+\tThe bundle contains this ref:\n+\t refs/heads/release\n+\tThe bundle requires these 3 refs:\n+\t\n+\t\n+\t\n+\tEOF\n+\n+\tgit bundle verify 2.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 2.bdl 16\n+'\n+\n+test_expect_success 'fail to verify bundle without prerequisites' '\n+\tgit init --bare test1.git &&\n+\n+\tcat >expect <<-\\EOF &&\n+\terror: Repository lacks these prerequisite commits:\n+\terror: \n+\terror: \n+\terror: \n+\tEOF\n+\n+\ttest_must_fail git -C test1.git bundle verify ../2.bdl 2>&1 |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual\n+'\n+\n+test_expect_success 'create bundle 3 - two refs, same object' '\n+\tgit bundle create --version=3 3.bdl \\\n+\t\t^release \\\n+\t\t^topic/1 \\\n+\t\t^topic/2 \\\n+\t\tmain \\\n+\t\tHEAD &&\n+\n+\tcat >expect <<-\\EOF &&\n+\tThe bundle contains these 2 refs:\n+\t refs/heads/main\n+\t HEAD\n+\tThe bundle requires these 2 refs:\n+\t\n+\t\n+\tEOF\n+\n+\tgit bundle verify 3.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 3.bdl 4\n+'\n+\n+test_expect_success 'create bundle 4 - with tags' '\n+\tgit bundle create 4.bdl \\\n+\t\t^main \\\n+\t\t^release \\\n+\t\t^topic/1 \\\n+\t\t^topic/2 \\\n+\t\t--all &&\n+\n+\tcat >expect <<-\\EOF &&\n+\tThe bundle contains these 3 refs:\n+\t refs/tags/v1\n+\t refs/tags/v2\n+\t refs/tags/v3\n+\tThe bundle records a complete history.\n+\tEOF\n+\n+\tgit bundle verify 4.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 4.bdl 3\n+'\n+\n+test_expect_success 'clone from bundle' '\n+\tgit clone --mirror 1.bdl mirror.git &&\n+\tgit -C mirror.git show-ref |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-\\EOF &&\n+\t refs/heads/topic/1\n+\t refs/heads/topic/2\n+\tEOF\n+\ttest_cmp expect actual &&\n+\n+\tgit -C mirror.git fetch ../2.bdl \"+refs/*:refs/*\" &&\n+\tgit -C mirror.git show-ref |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-\\EOF &&\n+\t refs/heads/release\n+\t refs/heads/topic/1\n+\t refs/heads/topic/2\n+\tEOF\n+\ttest_cmp expect actual &&\n+\n+\tgit -C mirror.git fetch ../3.bdl \"+refs/*:refs/*\" &&\n+\tgit -C mirror.git show-ref |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-\\EOF &&\n+\t refs/heads/main\n+\t refs/heads/release\n+\t refs/heads/topic/1\n+\t refs/heads/topic/2\n+\tEOF\n+\ttest_cmp expect actual &&\n+\n+\tgit -C mirror.git fetch ../4.bdl \"+refs/*:refs/*\" &&\n+\tgit -C mirror.git show-ref |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\tcat >expect <<-\\EOF &&\n+\t refs/heads/main\n+\t refs/heads/release\n+\t refs/heads/topic/1\n+\t refs/heads/topic/2\n+\t refs/tags/v1\n+\t refs/tags/v2\n+\t refs/tags/v3\n+\tEOF\n+\ttest_cmp expect actual\n+'\n+\n+test_done\ndiff --git a/t/test-bundle-functions.sh b/t/test-bundle-functions.sh\nnew file mode 100644\nindex 0000000000..cf7ed818b2\n--- /dev/null\n+++ b/t/test-bundle-functions.sh\n@@ -0,0 +1,42 @@\n+# Library of git-bundle related functions.\n+\n+# Display the pack data contained in the bundle file, bypassing the\n+# header that contains the signature, prerequisites and references.\n+convert_bundle_to_pack () {\n+\twhile read x && test -n \"$x\"\n+\tdo\n+\t\t:;\n+\tdone\n+\tcat\n+}\n+\n+# Check count of objects in a bundle file.\n+# We can use \"--thin\" opiton to check thin pack, which must be fixed by\n+# command `git-index-pack --fix-thin --stdin`.\n+test_bundle_object_count () {\n+\tthin=\n+\tif test \"$1\" = \"--thin\"\n+\tthen\n+\t\tthin=t\n+\t\tshift\n+\tfi\n+\tif test $# -ne 2\n+\tthen\n+\t\techo >&2 \"args should be: \"\n+\t\treturn 1\n+\tfi\n+\tbundle=$1\n+\tpack=$bundle.pack\n+\tconvert_bundle_to_pack <\"$bundle\" >\"$pack\" &&\n+\tif test -n \"$thin\"\n+\tthen\n+\t\tmv \"$pack\" \"$bundle.thin.pack\" &&\n+\t\tgit index-pack --stdin --fix-thin \"$pack\" <\"$bundle.thin.pack\"\n+\telse\n+\t\tgit index-pack \"$pack\"\n+\tfi || return 1\n+\tcount=$(git show-index <\"${pack%pack}idx\" | wc -l) &&\n+\ttest $2 = $count && return 0\n+\techo >&2 \"error: object count for $bundle is $count, not $2\"\n+\treturn 1\n+}\n-- \n2.28.0.15.gba9e81f0bd\n\n"},{"id":"414091","messageId":"20210112022703.1884-4-worldhello.net@gmail.com","threadId":"54923","inReplyTo":"xmqq1rer8cbz.fsf@gitster.c.googlers.com","subject":"[PATCH v6 3/3] bundle: arguments can be read from stdin","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-01-12T02:27:03Z","receivedAt":"2021-01-12T02:28:07Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"From: Jiang Xin \n\nIn order to create an incremental bundle, we need to pass many arguments\nto let git-bundle ignore some already packed commits. It will be more\nconvenient to pass args via stdin. But the current implementation does\nnot allow us to do this.\n\nThis is because args are parsed twice when creating bundle. The first\ntime for parsing args is in `compute_and_write_prerequisites()` by\nrunning `git-rev-list` command to write prerequisites in bundle file,\nand stdin is consumed in this step if \"--stdin\" option is provided for\n`git-bundle`. Later nothing can be read from stdin when running\n`setup_revisions()` in `create_bundle()`.\n\nThe solution is to parse args once by removing the entire function\n`compute_and_write_prerequisites()` and then calling function\n`setup_revisions()`. In order to write prerequisites for bundle, will\ncall `prepare_revision_walk()` and `traverse_commit_list()`. But after\ncalling `prepare_revision_walk()`, the object array `revs.pending` is\nleft empty, and the following steps could not work properly with the\nempty object array (`revs.pending`). Therefore, make a copy of `revs`\nto `revs_copy` for later use right after calling `setup_revisions()`.\n\nThe copy of `revs_copy` is not a deep copy, it shares the same objects\nwith `revs`. The object array of `revs` has been cleared, but objects\nthemselves are still kept. Flags of objects may change after calling\n`prepare_revision_walk()`, we can use these changed flags without\ncalling the `git rev-list` command and parsing its output like the\nformer implementation.\n\nAlso add testcases for git bundle in t6020, which read args from stdin.\n\nSigned-off-by: Jiang Xin \n---\n bundle.c | 109 ++++++++++++++++++++++------------------\n t/t5607-clone-bundle.sh | 4 +-\n t/t6020-bundle-misc.sh | 77 ++++++++++++++++++++++++++--\n 3 files changed, 134 insertions(+), 56 deletions(-)\n\ndiff --git a/bundle.c b/bundle.c\nindex cb0e5931ac..693d619551 100644\n--- a/bundle.c\n+++ b/bundle.c\n@@ -338,48 +338,6 @@ static int write_pack_data(int bundle_fd, struct rev_info *revs, struct strvec *\n \treturn 0;\n }\n \n-static int compute_and_write_prerequisites(int bundle_fd,\n-\t\t\t\t\t struct rev_info *revs,\n-\t\t\t\t\t int argc, const char **argv)\n-{\n-\tstruct child_process rls = CHILD_PROCESS_INIT;\n-\tstruct strbuf buf = STRBUF_INIT;\n-\tFILE *rls_fout;\n-\tint i;\n-\n-\tstrvec_pushl(&rls.args,\n-\t\t \"rev-list\", \"--boundary\", \"--pretty=oneline\",\n-\t\t NULL);\n-\tfor (i = 1; i < argc; i++)\n-\t\tstrvec_push(&rls.args, argv[i]);\n-\trls.out = -1;\n-\trls.git_cmd = 1;\n-\tif (start_command(&rls))\n-\t\treturn -1;\n-\trls_fout = xfdopen(rls.out, \"r\");\n-\twhile (strbuf_getwholeline(&buf, rls_fout, '\\n') != EOF) {\n-\t\tstruct object_id oid;\n-\t\tif (buf.len > 0 && buf.buf[0] == '-') {\n-\t\t\twrite_or_die(bundle_fd, buf.buf, buf.len);\n-\t\t\tif (!get_oid_hex(buf.buf + 1, &oid)) {\n-\t\t\t\tstruct object *object = parse_object_or_die(&oid,\n-\t\t\t\t\t\t\t\t\t buf.buf);\n-\t\t\t\tobject->flags |= UNINTERESTING;\n-\t\t\t\tadd_pending_object(revs, object, buf.buf);\n-\t\t\t}\n-\t\t} else if (!get_oid_hex(buf.buf, &oid)) {\n-\t\t\tstruct object *object = parse_object_or_die(&oid,\n-\t\t\t\t\t\t\t\t buf.buf);\n-\t\t\tobject->flags |= SHOWN;\n-\t\t}\n-\t}\n-\tstrbuf_release(&buf);\n-\tfclose(rls_fout);\n-\tif (finish_command(&rls))\n-\t\treturn error(_(\"rev-list died\"));\n-\treturn 0;\n-}\n-\n /*\n * Write out bundle refs based on the tips already\n * parsed into revs.pending. As a side effect, may\n@@ -474,6 +432,38 @@ static int write_bundle_refs(int bundle_fd, struct rev_info *revs)\n \treturn ref_count;\n }\n \n+struct bundle_prerequisites_info {\n+\tstruct object_array *pending;\n+\tint fd;\n+};\n+\n+static void write_bundle_prerequisites(struct commit *commit, void *data)\n+{\n+\tstruct bundle_prerequisites_info *bpi = data;\n+\tstruct object *object;\n+\tstruct pretty_print_context ctx = { 0 };\n+\tstruct strbuf buf = STRBUF_INIT;\n+\n+\tif (!(commit->object.flags & BOUNDARY))\n+\t\treturn;\n+\tstrbuf_addf(&buf, \"-%s \", oid_to_hex(&commit->object.oid));\n+\twrite_or_die(bpi->fd, buf.buf, buf.len);\n+\n+\tctx.fmt = CMIT_FMT_ONELINE;\n+\tctx.output_encoding = get_log_output_encoding();\n+\tstrbuf_reset(&buf);\n+\tpretty_print_commit(&ctx, commit, &buf);\n+\tstrbuf_trim(&buf);\n+\n+\tobject = (struct object *)commit;\n+\tobject->flags |= UNINTERESTING;\n+\tadd_object_array_with_path(object, buf.buf, bpi->pending, S_IFINVALID,\n+\t\t\t\t NULL);\n+\tstrbuf_addch(&buf, '\\n');\n+\twrite_or_die(bpi->fd, buf.buf, buf.len);\n+\tstrbuf_release(&buf);\n+}\n+\n int create_bundle(struct repository *r, const char *path,\n \t\t int argc, const char **argv, struct strvec *pack_options, int version)\n {\n@@ -481,8 +471,10 @@ int create_bundle(struct repository *r, const char *path,\n \tint bundle_fd = -1;\n \tint bundle_to_stdout;\n \tint ref_count = 0;\n-\tstruct rev_info revs;\n+\tstruct rev_info revs, revs_copy;\n \tint min_version = the_hash_algo == &hash_algos[GIT_HASH_SHA1] ? 2 : 3;\n+\tstruct bundle_prerequisites_info bpi;\n+\tint i;\n \n \tbundle_to_stdout = !strcmp(path, \"-\");\n \tif (bundle_to_stdout)\n@@ -512,10 +504,6 @@ int create_bundle(struct repository *r, const char *path,\n \tsave_commit_buffer = 0;\n \trepo_init_revisions(r, &revs, NULL);\n \n-\t/* write prerequisites */\n-\tif (compute_and_write_prerequisites(bundle_fd, &revs, argc, argv))\n-\t\tgoto err;\n-\n \targc = setup_revisions(argc, argv, &revs, NULL);\n \n \tif (argc > 1) {\n@@ -523,16 +511,37 @@ int create_bundle(struct repository *r, const char *path,\n \t\tgoto err;\n \t}\n \n-\tobject_array_remove_duplicates(&revs.pending);\n+\t/* save revs.pending in revs_copy for later use */\n+\tmemcpy(&revs_copy, &revs, sizeof(revs));\n+\trevs_copy.pending.nr = 0;\n+\trevs_copy.pending.alloc = 0;\n+\trevs_copy.pending.objects = NULL;\n+\tfor (i = 0; i < revs.pending.nr; i++) {\n+\t\tstruct object_array_entry *e = revs.pending.objects + i;\n+\t\tif (e)\n+\t\t\tadd_object_array_with_path(e->item, e->name,\n+\t\t\t\t\t\t &revs_copy.pending,\n+\t\t\t\t\t\t e->mode, e->path);\n+\t}\n \n-\tref_count = write_bundle_refs(bundle_fd, &revs);\n+\t/* write prerequisites */\n+\trevs.boundary = 1;\n+\tif (prepare_revision_walk(&revs))\n+\t\tdie(\"revision walk setup failed\");\n+\tbpi.fd = bundle_fd;\n+\tbpi.pending = &revs_copy.pending;\n+\ttraverse_commit_list(&revs, write_bundle_prerequisites, NULL, &bpi);\n+\tobject_array_remove_duplicates(&revs_copy.pending);\n+\n+\t/* write bundle refs */\n+\tref_count = write_bundle_refs(bundle_fd, &revs_copy);\n \tif (!ref_count)\n \t\tdie(_(\"Refusing to create empty bundle.\"));\n \telse if (ref_count < 0)\n \t\tgoto err;\n \n \t/* write pack */\n-\tif (write_pack_data(bundle_fd, &revs, pack_options))\n+\tif (write_pack_data(bundle_fd, &revs_copy, pack_options))\n \t\tgoto err;\n \n \tif (!bundle_to_stdout) {\ndiff --git a/t/t5607-clone-bundle.sh b/t/t5607-clone-bundle.sh\nindex 26985f4b44..425258767d 100755\n--- a/t/t5607-clone-bundle.sh\n+++ b/t/t5607-clone-bundle.sh\n@@ -38,13 +38,13 @@ test_expect_success 'die if bundle file cannot be created' '\n \ttest_must_fail git bundle create adir --all\n '\n \n-test_expect_failure 'bundle --stdin' '\n+test_expect_success 'bundle --stdin' '\n \techo master | git bundle create stdin-bundle.bdl --stdin &&\n \tgit ls-remote stdin-bundle.bdl >output &&\n \tgrep master output\n '\n \n-test_expect_failure 'bundle --stdin ' '\n+test_expect_success 'bundle --stdin ' '\n \techo master | git bundle create hybrid-bundle.bdl --stdin tag &&\n \tgit ls-remote hybrid-bundle.bdl >output &&\n \tgrep master output\ndiff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh\nindex b554538e00..6249420a80 100755\n--- a/t/t6020-bundle-misc.sh\n+++ b/t/t6020-bundle-misc.sh\n@@ -242,8 +242,16 @@ test_expect_success 'create bundle with --since option' '\n '\n \n test_expect_success 'create bundle 1 - no prerequisites' '\n+\t# create bundle from args\n \tgit bundle create 1.bdl topic/1 topic/2 &&\n \n+\t# create bundle from stdin\n+\tcat >input <<-\\EOF &&\n+\ttopic/1\n+\ttopic/2\n+\tEOF\n+\tgit bundle create stdin-1.bdl --stdin expect <<-\\EOF &&\n \tThe bundle contains these 2 refs:\n \t refs/heads/topic/1\n@@ -256,10 +264,16 @@ test_expect_success 'create bundle 1 - no prerequisites' '\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual &&\n \n-\ttest_bundle_object_count 1.bdl 24\n+\tgit bundle verify stdin-1.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 1.bdl 24 &&\n+\ttest_bundle_object_count stdin-1.bdl 24\n '\n \n test_expect_success 'create bundle 2 - has prerequisites' '\n+\t# create bundle from args\n \tgit bundle create 2.bdl \\\n \t\t--ignore-missing \\\n \t\t^topic/deleted \\\n@@ -267,6 +281,18 @@ test_expect_success 'create bundle 2 - has prerequisites' '\n \t\t^topic/2 \\\n \t\trelease &&\n \n+\t# create bundle from stdin\n+\t# input has a non-exist reference: \"topic/deleted\"\n+\tcat >input <<-EOF &&\n+\t^topic/deleted\n+\t^$D\n+\t^topic/2\n+\tEOF\n+\tgit bundle create stdin-2.bdl \\\n+\t\t--ignore-missing \\\n+\t\t--stdin \\\n+\t\trelease expect <<-\\EOF &&\n \tThe bundle contains this ref:\n \t refs/heads/release\n@@ -280,7 +306,12 @@ test_expect_success 'create bundle 2 - has prerequisites' '\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual &&\n \n-\ttest_bundle_object_count 2.bdl 16\n+\tgit bundle verify stdin-2.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 2.bdl 16 &&\n+\ttest_bundle_object_count stdin-2.bdl 16\n '\n \n test_expect_success 'fail to verify bundle without prerequisites' '\n@@ -295,10 +326,15 @@ test_expect_success 'fail to verify bundle without prerequisites' '\n \n \ttest_must_fail git -C test1.git bundle verify ../2.bdl 2>&1 |\n \t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_must_fail git -C test1.git bundle verify ../stdin-2.bdl 2>&1 |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual\n '\n \n test_expect_success 'create bundle 3 - two refs, same object' '\n+\t# create bundle from args\n \tgit bundle create --version=3 3.bdl \\\n \t\t^release \\\n \t\t^topic/1 \\\n@@ -306,6 +342,16 @@ test_expect_success 'create bundle 3 - two refs, same object' '\n \t\tmain \\\n \t\tHEAD &&\n \n+\t# create bundle from stdin\n+\tcat >input <<-\\EOF &&\n+\t^release\n+\t^topic/1\n+\t^topic/2\n+\tEOF\n+\tgit bundle create --version=3 stdin-3.bdl \\\n+\t\t--stdin \\\n+\t\tmain HEAD expect <<-\\EOF &&\n \tThe bundle contains these 2 refs:\n \t refs/heads/main\n@@ -319,10 +365,16 @@ test_expect_success 'create bundle 3 - two refs, same object' '\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual &&\n \n-\ttest_bundle_object_count 3.bdl 4\n+\tgit bundle verify stdin-3.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 3.bdl 4 &&\n+\ttest_bundle_object_count stdin-3.bdl 4\n '\n \n test_expect_success 'create bundle 4 - with tags' '\n+\t# create bundle from args\n \tgit bundle create 4.bdl \\\n \t\t^main \\\n \t\t^release \\\n@@ -330,6 +382,18 @@ test_expect_success 'create bundle 4 - with tags' '\n \t\t^topic/2 \\\n \t\t--all &&\n \n+\t# create bundle from stdin\n+\tcat >input <<-\\EOF &&\n+\t^main\n+\t^release\n+\t^topic/1\n+\t^topic/2\n+\tEOF\n+\tgit bundle create stdin-4.bdl \\\n+\t\t--ignore-missing \\\n+\t\t--stdin \\\n+\t\t--all expect <<-\\EOF &&\n \tThe bundle contains these 3 refs:\n \t refs/tags/v1\n@@ -342,7 +406,12 @@ test_expect_success 'create bundle 4 - with tags' '\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_i18ncmp expect actual &&\n \n-\ttest_bundle_object_count 4.bdl 3\n+\tgit bundle verify stdin-4.bdl |\n+\t\tmake_user_friendly_and_stable_output >actual &&\n+\ttest_i18ncmp expect actual &&\n+\n+\ttest_bundle_object_count 4.bdl 3 &&\n+\ttest_bundle_object_count stdin-4.bdl 3\n '\n \n test_expect_success 'clone from bundle' '\n-- \n2.28.0.15.gba9e81f0bd\n\n"},{"id":"414092","messageId":"20210112022703.1884-3-worldhello.net@gmail.com","threadId":"54923","inReplyTo":"xmqq1rer8cbz.fsf@gitster.c.googlers.com","subject":"[PATCH v6 2/3] bundle: lost objects when removing duplicate pendings","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-01-12T02:27:02Z","receivedAt":"2021-01-12T02:28:09Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"From: Jiang Xin \n\n`git rev-list` will list one commit for the following command:\n\n $ git rev-list 'main^!'\n \n\nBut providing the same rev-list args to `git bundle`, fail to create\na bundle file.\n\n $ git bundle create - 'main^!'\n # v2 git bundle\n - \n\n fatal: Refusing to create empty bundle.\n\nThis is because when removing duplicate objects in function\n`object_array_remove_duplicates()`, one unique pending object which has\nthe same name is deleted by mistake. The revision arg 'main^!' in the\nabove example is parsed by `handle_revision_arg()`, and at lease two\ndifferent objects will be appended to `revs.pending`, one points to the\nparent commit of the \"main\" branch, and the other points to the tip\ncommit of the \"main\" branch. These two objects have the same name\n\"main\". Only one object is left with the name \"main\" after calling the\nfunction `object_array_remove_duplicates()`.\n\nAnd what's worse, when adding boundary commits into pending list, we use\none-line commit message as names, and the arbitory names may surprise\ngit-bundle.\n\nOnly comparing objects themselves (\".item\") is also not good enough,\nbecause user may want to create a bundle with two identical objects but\nwith different reference names, such as: \"HEAD\" and \"refs/heads/main\".\n\nAdd new function `contains_object()` which compare both the address and\nthe name of the object.\n\nSigned-off-by: Jiang Xin \n---\n object.c | 10 ++++++----\n t/t6020-bundle-misc.sh | 2 +-\n 2 files changed, 7 insertions(+), 5 deletions(-)\n\ndiff --git a/object.c b/object.c\nindex 68f80b0b3d..98017bed8e 100644\n--- a/object.c\n+++ b/object.c\n@@ -412,15 +412,16 @@ void object_array_clear(struct object_array *array)\n }\n \n /*\n- * Return true iff array already contains an entry with name.\n+ * Return true if array already contains an entry.\n */\n-static int contains_name(struct object_array *array, const char *name)\n+static int contains_object(struct object_array *array,\n+\t\t\t const struct object *item, const char *name)\n {\n \tunsigned nr = array->nr, i;\n \tstruct object_array_entry *object = array->objects;\n \n \tfor (i = 0; i < nr; i++, object++)\n-\t\tif (!strcmp(object->name, name))\n+\t\tif (item == object->item && !strcmp(object->name, name))\n \t\t\treturn 1;\n \treturn 0;\n }\n@@ -432,7 +433,8 @@ void object_array_remove_duplicates(struct object_array *array)\n \n \tarray->nr = 0;\n \tfor (src = 0; src < nr; src++) {\n-\t\tif (!contains_name(array, objects[src].name)) {\n+\t\tif (!contains_object(array, objects[src].item,\n+\t\t\t\t objects[src].name)) {\n \t\t\tif (src != array->nr)\n \t\t\t\tobjects[array->nr] = objects[src];\n \t\t\tarray->nr++;\ndiff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh\nindex 201f34b5c3..b554538e00 100755\n--- a/t/t6020-bundle-misc.sh\n+++ b/t/t6020-bundle-misc.sh\n@@ -167,7 +167,7 @@ test_expect_success 'setup' '\n \ttest_commit_setvar P \"Commit P\" main.txt\n '\n \n-test_expect_failure 'create bundle from special rev: main^!' '\n+test_expect_success 'create bundle from special rev: main^!' '\n \tgit bundle create special-rev.bdl \"main^!\" &&\n \n \tgit bundle list-heads special-rev.bdl |\n-- \n2.28.0.15.gba9e81f0bd\n\n"},{"id":"425584","messageId":"87bl8xmipo.fsf@evledraar.gmail.com","threadId":"54923","inReplyTo":"20210112022703.1884-2-worldhello.net@gmail.com","subject":"Runaway sed memory use in test on older sed+glibc (was \"Re: [PATCH v6 1/3] test: add helper functions for git-bundle\")","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-05-26T18:49:24Z","receivedAt":"2021-05-26T18:51:52Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Jan 11 2021, Jiang Xin wrote:\n\n> From: Jiang Xin \n>\n> Move git-bundle related functions from t5510 to a library, and this lib\n> will be shared with a new testcase t6020 which finds a known breakage of\n> \"git-bundle\".\n> [...]\n> +\n> +# Format the output of git commands to make a user-friendly and stable\n> +# text. We can easily prepare the expect text without having to worry\n> +# about future changes of the commit ID and spaces of the output.\n> +make_user_friendly_and_stable_output () {\n> +\tsed \\\n> +\t\t-e \"s/${A%${A#???????}}[0-9a-f]*//g\" \\\n> +\t\t-e \"s/${B%${B#???????}}[0-9a-f]*//g\" \\\n> +\t\t-e \"s/${C%${C#???????}}[0-9a-f]*//g\" \\\n> +\t\t-e \"s/${D%${D#???????}}[0-9a-f]*//g\" \\\n> +\t\t-e \"s/${E%${E#???????}}[0-9a-f]*//g\" \\\n> +\t\t-e \"s/${F%${F#???????}}[0-9a-f]*//g\" \\\n> +\t\t-e \"s/${G%${G#???????}}[0-9a-f]*//g\" \\\n> +\t\t-e \"s/${H%${H#???????}}[0-9a-f]*//g\" \\\n> +\t\t-e \"s/${I%${I#???????}}[0-9a-f]*//g\" \\\n> +\t\t-e \"s/${J%${J#???????}}[0-9a-f]*//g\" \\\n> +\t\t-e \"s/${K%${K#???????}}[0-9a-f]*//g\" \\\n> +\t\t-e \"s/${L%${L#???????}}[0-9a-f]*//g\" \\\n> +\t\t-e \"s/${M%${M#???????}}[0-9a-f]*//g\" \\\n> +\t\t-e \"s/${N%${N#???????}}[0-9a-f]*//g\" \\\n> +\t\t-e \"s/${O%${O#???????}}[0-9a-f]*//g\" \\\n> +\t\t-e \"s/${P%${P#???????}}[0-9a-f]*//g\" \\\n> +\t\t-e \"s/${TAG1%${TAG1#???????}}[0-9a-f]*//g\" \\\n> +\t\t-e \"s/${TAG2%${TAG2#???????}}[0-9a-f]*//g\" \\\n> +\t\t-e \"s/${TAG3%${TAG3#???????}}[0-9a-f]*//g\" \\\n> +\t\t-e \"s/ *\\$//\"\n> +}\n\nOn one of the gcc farm boxes, a i386 box (gcc45) this fails because sed\ngets killed after >500MB of memory use (I was just eyeballing it in\nhtop) on the \"reate bundle from special rev: main^!\" test. This with GNU\nsed 4.2.2.\n\nI suspect this regex pattern creates some runaway behavior in sed that's\nsince been fixed (or maybe it's the glibc regex engine?). The glibc is\n2.19-18+deb8u10:\n \n + git bundle list-heads special-rev.bdl\n + make_user_friendly_and_stable_output\n + sed -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/ *$//\n sed: couldn't re-allocate memory\n"},{"id":"425667","messageId":"20210527115226.42539-1-zhiyou.jx@alibaba-inc.com","threadId":"54923","inReplyTo":"87bl8xmipo.fsf@evledraar.gmail.com","subject":"Re: Runaway sed memory use in test on older sed+glibc (was \"Re: [PATCH v6 1/3] test: add helper functions for git-bundle\")","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-05-27T11:52:26Z","receivedAt":"2021-05-27T11:52:41Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"Ævar Arnfjörð Bjarmason 于2021年5月27日周四\n上午2:51写道：\n>\n>\n> On Mon, Jan 11 2021, Jiang Xin wrote:\n>\n> > From: Jiang Xin \n> >\n> > Move git-bundle related functions from t5510 to a library, and this\n> > lib\n> > will be shared with a new testcase t6020 which finds a known\n> > breakage of\n> > \"git-bundle\".\n> > [...]\n> > +\n> > +# Format the output of git commands to make a user-friendly and\n> > stable\n> > +# text.  We can easily prepare the expect text without having to\n> > worry\n> > +# about future changes of the commit ID and spaces of the output.\n> > +make_user_friendly_and_stable_output () {\n> > +     sed \\\n> > +             -e \"s/${A%${A#???????}}[0-9a-f]*//g\" \\\n> > +             -e \"s/${B%${B#???????}}[0-9a-f]*//g\" \\\n> > +             -e \"s/${C%${C#???????}}[0-9a-f]*//g\" \\\n> > +             -e \"s/${D%${D#???????}}[0-9a-f]*//g\" \\\n> > +             -e \"s/${E%${E#???????}}[0-9a-f]*//g\" \\\n> > +             -e \"s/${F%${F#???????}}[0-9a-f]*//g\" \\\n> > +             -e \"s/${G%${G#???????}}[0-9a-f]*//g\" \\\n> > +             -e \"s/${H%${H#???????}}[0-9a-f]*//g\" \\\n> > +             -e \"s/${I%${I#???????}}[0-9a-f]*//g\" \\\n> > +             -e \"s/${J%${J#???????}}[0-9a-f]*//g\" \\\n> > +             -e \"s/${K%${K#???????}}[0-9a-f]*//g\" \\\n> > +             -e \"s/${L%${L#???????}}[0-9a-f]*//g\" \\\n> > +             -e \"s/${M%${M#???????}}[0-9a-f]*//g\" \\\n> > +             -e \"s/${N%${N#???????}}[0-9a-f]*//g\" \\\n> > +             -e \"s/${O%${O#???????}}[0-9a-f]*//g\" \\\n> > +             -e \"s/${P%${P#???????}}[0-9a-f]*//g\" \\\n> > +             -e \"s/${TAG1%${TAG1#???????}}[0-9a-f]*//g\" \\\n> > +             -e \"s/${TAG2%${TAG2#???????}}[0-9a-f]*//g\" \\\n> > +             -e \"s/${TAG3%${TAG3#???????}}[0-9a-f]*//g\" \\\n> > +             -e \"s/ *\\$//\"\n> > +}\n>\n> On one of the gcc farm boxes, a i386 box (gcc45) this fails because\n> sed\n> gets killed after >500MB of memory use (I was just eyeballing it in\n> htop) on the \"reate bundle from special rev: main^!\" test. This with\n> GNU\n> sed 4.2.2.\n>\n> I suspect this regex pattern creates some runaway behavior in sed\n> that's\n> since been fixed (or maybe it's the glibc regex engine?). The glibc is\n> 2.19-18+deb8u10:\n>\n>     + git bundle list-heads special-rev.bdl\n>     + make_user_friendly_and_stable_output\n>     + sed -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n> s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n> s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n> s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n> s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n> s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n> s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n> s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n> s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n> s/[0-9a-f]*//g -e s/ *$//\n>     sed: couldn't re-allocate memory\n\nI wrote a program on macOS to check memory footprint for sed and perl.\nSee:\n\n    https://github.com/jiangxin/compare-sed-perl\n\nTest result:\n\n    $ go build && ./compare-sed-perl\n    Command: sed  ..., MaxRSS: 901120\n    Command: gsed ..., MaxRSS: 2056192\n    Command: perl ..., MaxRSS: 2269184\n\nIt seems that sed (both the builtin version on macOS and GNU sed v4.8)\nhas less memory consumed than perl.\n\nCan you run this program on the i386 box (gcc45) to check memory consumed\nby sed and perl?\n\nIf this issue can be resolved by replacing sed with perl, the following\npatch may help:\n\n--- >8 ---\nFrom: Jiang Xin \nDate: Thu, 27 May 2021 14:31:49 +0800\nSubject: [PATCH] test: use perl for complex text replacement\n\nÆvar reported that the function `make_user_friendly_and_stable_output()`\nfailed on a i386 box (gcc45) in the gcc farm boxes with error:\n\n sed: couldn't re-allocate memory\n\nIt seems that sed (GNU sed 4.2.2) gets killed after >500MB of memory\nuse on the \"create bundle from special rev: main^!\" test.\n\nCall perl instead of sed for complex text replacement.\n\nReported-by: Ævar Arnfjörð Bjarmason \nSigned-off-by: Jiang Xin \n---\n t/t5411/common-functions.sh | 27 ++++++++++++------------\n t/t5548-push-porcelain.sh | 20 +++++++++---------\n t/t6020-bundle-misc.sh | 42 ++++++++++++++++++-------------------\n 3 files changed, 45 insertions(+), 44 deletions(-)\n\ndiff --git a/t/t5411/common-functions.sh b/t/t5411/common-functions.sh\nindex 6694858e18..b6d33bdfdc 100644\n--- a/t/t5411/common-functions.sh\n+++ b/t/t5411/common-functions.sh\n@@ -39,19 +39,20 @@ create_commits_in () {\n # remove some locale error messages. The emitted human-readable errors are\n # redundant to the more machine-readable output the tests already assert.\n make_user_friendly_and_stable_output () {\n-\tsed \\\n-\t\t-e \"s/ *\\$//\" \\\n-\t\t-e \"s/ */ /g\" \\\n-\t\t-e \"s/'/\\\"/g\" \\\n-\t\t-e \"s/\t/ /g\" \\\n-\t\t-e \"s/$A//g\" \\\n-\t\t-e \"s/$B//g\" \\\n-\t\t-e \"s/$TAG//g\" \\\n-\t\t-e \"s/$ZERO_OID//g\" \\\n-\t\t-e \"s/$(echo $A | cut -c1-7)[0-9a-f]*//g\" \\\n-\t\t-e \"s/$(echo $B | cut -c1-7)[0-9a-f]*//g\" \\\n-\t\t-e \"s#To $URL_PREFIX/upstream.git#To #\" \\\n-\t\t-e \"/^error: / d\"\n+\tperl -ne \"\n+\t\ts/ *\\$//;\n+\t\ts/ */ /g;\n+\t\ts/'/\\\"/g;\n+\t\ts/\t/ /g;\n+\t\ts/$A//g;\n+\t\ts/$B//g;\n+\t\ts/$TAG//g;\n+\t\ts/$ZERO_OID//g;\n+\t\ts/$(echo $A | cut -c1-7)[0-9a-f]*//g;\n+\t\ts/$(echo $B | cut -c1-7)[0-9a-f]*//g;\n+\t\ts#To $URL_PREFIX/upstream.git#To #;\n+\t\tnext if /^error: .*$/;\n+\t\tprint\"\n }\n \n filter_out_user_friendly_and_stable_output () {\ndiff --git a/t/t5548-push-porcelain.sh b/t/t5548-push-porcelain.sh\nindex 5a761f3642..95e216973d 100755\n--- a/t/t5548-push-porcelain.sh\n+++ b/t/t5548-push-porcelain.sh\n@@ -44,16 +44,16 @@ create_commits_in () {\n # without having to worry about future changes of the commit ID and spaces\n # of the output.\n make_user_friendly_and_stable_output () {\n-\tsed \\\n-\t\t-e \"s/ *\\$//\" \\\n-\t\t-e \"s/ */ /g\" \\\n-\t\t-e \"s/\t/ /g\" \\\n-\t\t-e \"s/$A//g\" \\\n-\t\t-e \"s/$B//g\" \\\n-\t\t-e \"s/$ZERO_OID//g\" \\\n-\t\t-e \"s/$(echo $A | cut -c1-7)[0-9a-f]*//g\" \\\n-\t\t-e \"s/$(echo $B | cut -c1-7)[0-9a-f]*//g\" \\\n-\t\t-e \"s#To $URL_PREFIX/upstream.git#To #\"\n+\tperl -pe \"\n+\t\ts/ *\\$//;\n+\t\ts/ */ /g;\n+\t\ts/\t/ /g;\n+\t\ts/$A//g;\n+\t\ts/$B//g;\n+\t\ts/$ZERO_OID//g;\n+\t\ts/$(echo $A | cut -c1-7)[0-9a-f]*//g;\n+\t\ts/$(echo $B | cut -c1-7)[0-9a-f]*//g;\n+\t\ts#To $URL_PREFIX/upstream.git#To #\"\n }\n \n setup_upstream_and_workbench () {\ndiff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh\nindex 881f72fd44..f284be820f 100755\n--- a/t/t6020-bundle-misc.sh\n+++ b/t/t6020-bundle-misc.sh\n@@ -84,27 +84,27 @@ test_commit_setvar () {\n # text. We can easily prepare the expect text without having to worry\n # about future changes of the commit ID and spaces of the output.\n make_user_friendly_and_stable_output () {\n-\tsed \\\n-\t\t-e \"s/${A%${A#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${B%${B#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${C%${C#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${D%${D#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${E%${E#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${F%${F#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${G%${G#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${H%${H#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${I%${I#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${J%${J#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${K%${K#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${L%${L#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${M%${M#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${N%${N#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${O%${O#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${P%${P#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${TAG1%${TAG1#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${TAG2%${TAG2#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${TAG3%${TAG3#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/ *\\$//\"\n+\tperl -pe \"\n+\t\ts/${A%${A#???????}}[0-9a-f]*//g;\n+\t\ts/${B%${B#???????}}[0-9a-f]*//g;\n+\t\ts/${C%${C#???????}}[0-9a-f]*//g;\n+\t\ts/${D%${D#???????}}[0-9a-f]*//g;\n+\t\ts/${E%${E#???????}}[0-9a-f]*//g;\n+\t\ts/${F%${F#???????}}[0-9a-f]*//g;\n+\t\ts/${G%${G#???????}}[0-9a-f]*//g;\n+\t\ts/${H%${H#???????}}[0-9a-f]*//g;\n+\t\ts/${I%${I#???????}}[0-9a-f]*//g;\n+\t\ts/${J%${J#???????}}[0-9a-f]*//g;\n+\t\ts/${K%${K#???????}}[0-9a-f]*//g;\n+\t\ts/${L%${L#???????}}[0-9a-f]*//g;\n+\t\ts/${M%${M#???????}}[0-9a-f]*//g;\n+\t\ts/${N%${N#???????}}[0-9a-f]*//g;\n+\t\ts/${O%${O#???????}}[0-9a-f]*//g;\n+\t\ts/${P%${P#???????}}[0-9a-f]*//g;\n+\t\ts/${TAG1%${TAG1#???????}}[0-9a-f]*//g;\n+\t\ts/${TAG2%${TAG2#???????}}[0-9a-f]*//g;\n+\t\ts/${TAG3%${TAG3#???????}}[0-9a-f]*//g;\n+\t\ts/ *\\$//\"\n }\n \n # (C) (D, pull/1/head, topic/1)\n-- \n2.32.0.rc0\n\n"},{"id":"425670","messageId":"87tumol4tg.fsf@evledraar.gmail.com","threadId":"54923","inReplyTo":"20210527115226.42539-1-zhiyou.jx@alibaba-inc.com","subject":"Re: Runaway sed memory use in test on older sed+glibc (was \"Re: [PATCH v6 1/3] test: add helper functions for git-bundle\")","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-05-27T12:19:04Z","receivedAt":"2021-05-27T12:49:36Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, May 27 2021, Jiang Xin wrote:\n\n> Ævar Arnfjörð Bjarmason 于2021年5月27日周四\n> 上午2:51写道：\n>>\n>>\n>> On Mon, Jan 11 2021, Jiang Xin wrote:\n>>\n>> > From: Jiang Xin \n>> >\n>> > Move git-bundle related functions from t5510 to a library, and this\n>> > lib\n>> > will be shared with a new testcase t6020 which finds a known\n>> > breakage of\n>> > \"git-bundle\".\n>> > [...]\n>> > +\n>> > +# Format the output of git commands to make a user-friendly and\n>> > stable\n>> > +# text.  We can easily prepare the expect text without having to\n>> > worry\n>> > +# about future changes of the commit ID and spaces of the output.\n>> > +make_user_friendly_and_stable_output () {\n>> > +     sed \\\n>> > +             -e \"s/${A%${A#???????}}[0-9a-f]*//g\" \\\n>> > +             -e \"s/${B%${B#???????}}[0-9a-f]*//g\" \\\n>> > +             -e \"s/${C%${C#???????}}[0-9a-f]*//g\" \\\n>> > +             -e \"s/${D%${D#???????}}[0-9a-f]*//g\" \\\n>> > +             -e \"s/${E%${E#???????}}[0-9a-f]*//g\" \\\n>> > +             -e \"s/${F%${F#???????}}[0-9a-f]*//g\" \\\n>> > +             -e \"s/${G%${G#???????}}[0-9a-f]*//g\" \\\n>> > +             -e \"s/${H%${H#???????}}[0-9a-f]*//g\" \\\n>> > +             -e \"s/${I%${I#???????}}[0-9a-f]*//g\" \\\n>> > +             -e \"s/${J%${J#???????}}[0-9a-f]*//g\" \\\n>> > +             -e \"s/${K%${K#???????}}[0-9a-f]*//g\" \\\n>> > +             -e \"s/${L%${L#???????}}[0-9a-f]*//g\" \\\n>> > +             -e \"s/${M%${M#???????}}[0-9a-f]*//g\" \\\n>> > +             -e \"s/${N%${N#???????}}[0-9a-f]*//g\" \\\n>> > +             -e \"s/${O%${O#???????}}[0-9a-f]*//g\" \\\n>> > +             -e \"s/${P%${P#???????}}[0-9a-f]*//g\" \\\n>> > +             -e \"s/${TAG1%${TAG1#???????}}[0-9a-f]*//g\" \\\n>> > +             -e \"s/${TAG2%${TAG2#???????}}[0-9a-f]*//g\" \\\n>> > +             -e \"s/${TAG3%${TAG3#???????}}[0-9a-f]*//g\" \\\n>> > +             -e \"s/ *\\$//\"\n>> > +}\n>>\n>> On one of the gcc farm boxes, a i386 box (gcc45) this fails because\n>> sed\n>> gets killed after >500MB of memory use (I was just eyeballing it in\n>> htop) on the \"reate bundle from special rev: main^!\" test. This with\n>> GNU\n>> sed 4.2.2.\n>>\n>> I suspect this regex pattern creates some runaway behavior in sed\n>> that's\n>> since been fixed (or maybe it's the glibc regex engine?). The glibc is\n>> 2.19-18+deb8u10:\n>>\n>>     + git bundle list-heads special-rev.bdl\n>>     + make_user_friendly_and_stable_output\n>>     + sed -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n>> s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n>> s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n>> s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n>> s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n>> s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n>> s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n>> s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n>> s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n>> s/[0-9a-f]*//g -e s/ *$//\n>>     sed: couldn't re-allocate memory\n>\n> I wrote a program on macOS to check memory footprint for sed and perl.\n> See:\n>\n>     https://github.com/jiangxin/compare-sed-perl\n\nInteresting use of Go for as a /usr/bin/time -v replacement :)\n\nAfter changing your int64 to int32 and digging up how to cross-compile\nGo I get similar results, it's because your test has actual short SHA-1s\nin the \"-e 's///g'\"'s, but notice how in the trace I have it's\ne.g. \"s/[0-9a-f]*//g\".\n\nThat's the problem, so that Go command won't reproduce it. Anyway,\nchanging the test to emit to \"input\" first and running this shows it:\n \n avar@gcc45:/run/user/1632/git/t/trash directory.t6020-bundle-misc$ /usr/bin/time -v sed -e 's/[0-9a-f]*//g' -e 's/[0-9a-f]*//g' -e 's/[0-9a-f]*//g' -e 's/[0-9a-f]*//g' -e 's/[0-9a-f]*//g' -e 's/[0-9a-f]*//g' -e 's/[0-9a-f]*//g' -e 's/[0-9a-f]*//g' -e 's/[0-9a-f]*//g' -e 's/[0-9a-f]*//g' -e 's/[0-9a-f]*//g' -e 's/[0-9a-f]*//g' -e 's/[0-9a-f]*//g' -e 's/[0-9a-f]*//g' -e 's/[0-9a-f]*//g' -e 's/[0-9a-f]*//g' -e 's/[0-9a-f]*//g' -e 's/[0-9a-f]*//g' -e 's/[0-9a-f]*//g' -e 's/ *$//' /g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e s/ *$//\"\n User time (seconds): 130.00\n System time (seconds): 2.42\n Percent of CPU this job got: 100%\n Elapsed (wall clock) time (h:mm:ss or m:ss): 2:12.41\n Average shared text size (kbytes): 0\n Average unshared data size (kbytes): 0\n Average stack size (kbytes): 0\n Average total size (kbytes): 0\n Maximum resident set size (kbytes): 1030968\n Average resident set size (kbytes): 0\n Major (requiring I/O) page faults: 0\n Minor (reclaiming a frame) page faults: 257333\n Voluntary context switches: 1\n Involuntary context switches: 12578\n Swaps: 0\n File system inputs: 0\n File system outputs: 0\n Socket messages sent: 0\n Socket messages received: 0\n Signals delivered: 0\n Page size (bytes): 4096\n Exit status: 4\n\nBut no, the issue as it turns out is not Perl v.s. Sed, it's that\nthere's some bug in the shellscript / tooling version (happens with both\ndash 0.5.7-4 and bash 4.3-11+deb8u2 on that box) where those expansions\nlike ${A%${A#??????0?}} resolve to nothing.\n\nSo if we make that:\n\n cat >input &&\n cat input >&2 &&\n sed -e \"s/${A%${A#??????0?}}[0-9a-f]*//g\" input.tmp && mv input.tmp input &&\n cat input >&2 &&\n sed -e \"s/${B%${B#???????}}[0-9a-f]*//g\" input.tmp && mv input.tmp input &&\n cat input >&2 &&\n\nWe get things like:\n \n + sed -e s/[0-9a-f]*//g\n + mv input.tmp input\n + cat input\n rs/hs/min\n + sed -e s/[0-9a-f]*//g\n + mv input.tmp input\n + cat input\n <COMMIT-A> <COMMIT-A>r<COMMIT-A>s<COMMIT-A>/<COMMIT-A>h<COMMIT-A>s<COMMIT-A>/<COMMIT-A>m<COMMIT-A>i<COMMIT-A>n<COMMIT-A>\n [...]\n\netc. I.e. it's the sed expression itself that's the issue. I.e. you\nshould be able to reproduce this locally with something like:\n\n echo 0 | sed -e 's/[0-9]*/0/g' -e 's/[0-9]*/0/g' -e 's/[0-9]*/0/g' -e 's/[0-9]*/0/g' -e 's/[0-9]*/0/g' -e 's/[0-9]*/0/g' -e 's/[0-9]*/0/g' -e 's/[0-9]*/0/g'\n\nIf not just copy the -e a few more times.\n\nAnyway, looking at this whole test file with fresh eyes this pattern\nseems very strange. You duplicated most of test_commit with this\ntest_commit_setvar. It's a bit more verbosity but why not just use:\n\n test_commit ...\n A=$(git rev-parse HEAD)\n\nOr teach test_commit a --rev-parse option or something and:\n\n A=$(test_commit ...)\n\nThis make_user_friendly_and_stable_output then actually loses\ninformation, e.g. sometimes the bundle output you're testing emits\ntrailing spaces, but the normalization function overzelously trims that.\n\nI think this whole thing would be much simpler with the above and then\nsomething like:\n \n @@ -146,7 +126,8 @@ test_expect_success 'setup' '\n \n # branch main: merge commit I & J\n git checkout main &&\n - test_commit_setvar --merge I topic/1 \"Merge commit I\" &&\n + git merge --no-edit --no-ff -m\"Merge commit I\" topic/1 &&\n + I=$(git rev-parse HEAD) &&\n test_commit_setvar --merge J refs/pull/2/head \"Merge commit J\" &&\n \n # branch main: commit K\n @@ -172,18 +153,18 @@ test_expect_success 'create bundle from special rev: main^!' '\n \n git bundle list-heads special-rev.bdl |\n make_user_friendly_and_stable_output >actual &&\n - cat >expect <<-\\EOF &&\n - refs/heads/main\n + cat >expect <<-EOF &&\n + $P refs/heads/main\n EOF\n test_cmp expect actual &&\n\nOr just add a --merge option to test_commit itself.\n"},{"id":"425673","messageId":"YK+jGxlHHZ63o94N@coredump.intra.peff.net","threadId":"54923","inReplyTo":"87tumol4tg.fsf@evledraar.gmail.com","subject":"Re: Runaway sed memory use in test on older sed+glibc (was \"Re: [PATCH v6 1/3] test: add helper functions for git-bundle\")","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-05-27T13:48:11Z","receivedAt":"2021-05-27T13:48:15Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, May 27, 2021 at 02:19:04PM +0200, Ævar Arnfjörð Bjarmason wrote:\n\n> Anyway, looking at this whole test file with fresh eyes this pattern\n> seems very strange. You duplicated most of test_commit with this\n> test_commit_setvar. It's a bit more verbosity but why not just use:\n> \n> test_commit ...\n> A=$(git rev-parse HEAD)\n> \n> Or teach test_commit a --rev-parse option or something and:\n> \n> A=$(test_commit ...)\n\nThe latter would be nice (and maybe even could be the default, as the\nstdout is usually just going to --verbose output anyway). But I don't\nthink it works, because the $() is a subshell, so we lose the side\neffect of test_tick incrementing the timestamp counter.\n\nI wasn't following this thread carefully, but your suggestion to just\ndo \"A=$(git rev-parse HEAD)\" seems quite readable to me.\n\n-Peff\n"},{"id":"425705","messageId":"60aff0a416f40_265302082c@natae.notmuch","threadId":"54923","inReplyTo":"87tumol4tg.fsf@evledraar.gmail.com","subject":"Re: Runaway sed memory use in test on older sed+glibc (was \"Re: [PATCH v6 1/3] test: add helper functions for git-bundle\")","fromName":"Felipe Contreras","fromEmail":"felipe.contreras@gmail.com","sentAt":"2021-05-27T19:19:00Z","receivedAt":"2021-05-27T19:19:07Z","isPatch":true,"sender":{"key":"felipe.contreras@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8358?v=4"},"body":"Ævar Arnfjörð Bjarmason wrote:\n> \n> On Thu, May 27 2021, Jiang Xin wrote:\n> \n> > Ævar Arnfjörð Bjarmason 于2021年5月27日周四\n> > 上午2:51写道：\n> >>\n> >>\n> >> On Mon, Jan 11 2021, Jiang Xin wrote:\n> >>\n> >> > From: Jiang Xin \n> >> >\n> >> > Move git-bundle related functions from t5510 to a library, and this\n> >> > lib\n> >> > will be shared with a new testcase t6020 which finds a known\n> >> > breakage of\n> >> > \"git-bundle\".\n> >> > [...]\n> >> > +\n> >> > +# Format the output of git commands to make a user-friendly and\n> >> > stable\n> >> > +# text.  We can easily prepare the expect text without having to\n> >> > worry\n> >> > +# about future changes of the commit ID and spaces of the output.\n> >> > +make_user_friendly_and_stable_output () {\n> >> > +     sed \\\n> >> > +             -e \"s/${A%${A#???????}}[0-9a-f]*//g\" \\\n> >> > +             -e \"s/${B%${B#???????}}[0-9a-f]*//g\" \\\n> >> > +             -e \"s/${C%${C#???????}}[0-9a-f]*//g\" \\\n> >> > +             -e \"s/${D%${D#???????}}[0-9a-f]*//g\" \\\n> >> > +             -e \"s/${E%${E#???????}}[0-9a-f]*//g\" \\\n> >> > +             -e \"s/${F%${F#???????}}[0-9a-f]*//g\" \\\n> >> > +             -e \"s/${G%${G#???????}}[0-9a-f]*//g\" \\\n> >> > +             -e \"s/${H%${H#???????}}[0-9a-f]*//g\" \\\n> >> > +             -e \"s/${I%${I#???????}}[0-9a-f]*//g\" \\\n> >> > +             -e \"s/${J%${J#???????}}[0-9a-f]*//g\" \\\n> >> > +             -e \"s/${K%${K#???????}}[0-9a-f]*//g\" \\\n> >> > +             -e \"s/${L%${L#???????}}[0-9a-f]*//g\" \\\n> >> > +             -e \"s/${M%${M#???????}}[0-9a-f]*//g\" \\\n> >> > +             -e \"s/${N%${N#???????}}[0-9a-f]*//g\" \\\n> >> > +             -e \"s/${O%${O#???????}}[0-9a-f]*//g\" \\\n> >> > +             -e \"s/${P%${P#???????}}[0-9a-f]*//g\" \\\n> >> > +             -e \"s/${TAG1%${TAG1#???????}}[0-9a-f]*//g\" \\\n> >> > +             -e \"s/${TAG2%${TAG2#???????}}[0-9a-f]*//g\" \\\n> >> > +             -e \"s/${TAG3%${TAG3#???????}}[0-9a-f]*//g\" \\\n> >> > +             -e \"s/ *\\$//\"\n> >> > +}\n> >>\n> >> On one of the gcc farm boxes, a i386 box (gcc45) this fails because\n> >> sed\n> >> gets killed after >500MB of memory use (I was just eyeballing it in\n> >> htop) on the \"reate bundle from special rev: main^!\" test. This with\n> >> GNU\n> >> sed 4.2.2.\n> >>\n> >> I suspect this regex pattern creates some runaway behavior in sed\n> >> that's\n> >> since been fixed (or maybe it's the glibc regex engine?). The glibc is\n> >> 2.19-18+deb8u10:\n> >>\n> >>     + git bundle list-heads special-rev.bdl\n> >>     + make_user_friendly_and_stable_output\n> >>     + sed -e s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n> >> s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n> >> s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n> >> s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n> >> s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n> >> s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n> >> s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n> >> s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n> >> s/[0-9a-f]*//g -e s/[0-9a-f]*//g -e\n> >> s/[0-9a-f]*//g -e s/ *$//\n> >>     sed: couldn't re-allocate memory\n> >\n> > I wrote a program on macOS to check memory footprint for sed and perl.\n> > See:\n> >\n> >     https://github.com/jiangxin/compare-sed-perl\n> \n> Interesting use of Go for as a /usr/bin/time -v replacement :)\n\nHere's a Ruby version:\nhttps://dpaste.com/FYT2QKHJE\n\nI'm not sure if will be useful in this particular case, but Ruby code\nalways ends up simpler ;)\n\n-- \nFelipe Contreras"},{"id":"426103","messageId":"CANYiYbG1vVqN-De0n2Ukexh0Jn8e_b2G1CPv24T1fbYgyhKjfg@mail.gmail.com","threadId":"54923","inReplyTo":"87tumol4tg.fsf@evledraar.gmail.com","subject":"Re: Runaway sed memory use in test on older sed+glibc (was \"Re: [PATCH v6 1/3] test: add helper functions for git-bundle\")","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-06-01T09:42:19Z","receivedAt":"2021-06-01T09:42:41Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"Ævar Arnfjörð Bjarmason 于2021年5月27日周四 下午8:49写道：\n>\n> But no, the issue as it turns out is not Perl v.s. Sed, it's that\n> there's some bug in the shellscript / tooling version (happens with both\n> dash 0.5.7-4 and bash 4.3-11+deb8u2 on that box) where those expansions\n> like ${A%${A#??????0?}} resolve to nothing.\n\nThat's the root cause. It can be reproduced by running the following\ntest script:\n\n```\n#!/bin/sh\n# test script: test.sh\n\ntest_commit_setvar () {\n var=$1 &&\n oid=1234567890123456789012345678901234567890 &&\n eval $var=$oid\n}\n\ntest_commit_setvar A\necho \"A: $A\"\necho \"Abbrev of A: ${A%${A#???????}}\"\n```\n\nBy running different version of dash, we can see that dash 0.5.7 fail the test:\n\n```\n$ /opt/dash/0.5.11/bin/dash test.sh\nA: 1234567890123456789012345678901234567890\nAbbrev of A: 1234567\n\n$ /opt/dash/0.5.7/bin/dash test.sh\nA: 1234567890123456789012345678901234567890\nAbbrev of A:\n```\n\nThis issue can be fixed using the following example:\n\n```\n#!/bin/sh\n\ntest_commit_setvar () {\n var=$1 &&\n oid=1234567890123456789012345678901234567890 &&\n suffix=${oid#???????} &&\n oid=${oid%$suffix} &&\n eval $var=$oid\n}\n\ntest_commit_setvar A\necho \"Abbrev of A: $A\"\n```\n\n> Anyway, looking at this whole test file with fresh eyes this pattern\n> seems very strange. You duplicated most of test_commit with this\n> test_commit_setvar. It's a bit more verbosity but why not just use:\n>\n> test_commit ...\n> A=$(git rev-parse HEAD)\n\nThe function \"test_commit()\" in \"test-lib-function.sh\" always creates\ntags and it cannot make merge commit. So I rewrite a new function\nwhich reuse the scaffold of \"test_commit\".\n\nBTW, sorry for the late reply, will send patch later.\n\n--\nJiang Xin\n"},{"id":"426105","messageId":"CANYiYbGLuUjUgKZzuDTB7cCaJzA3o7jdXLUTQ=_h8hBvbVmnHA@mail.gmail.com","threadId":"54923","inReplyTo":"60aff0a416f40_265302082c@natae.notmuch","subject":"Re: Runaway sed memory use in test on older sed+glibc (was \"Re: [PATCH v6 1/3] test: add helper functions for git-bundle\")","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-06-01T09:45:21Z","receivedAt":"2021-06-01T09:45:36Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"Felipe Contreras 于2021年5月28日周五 上午3:19写道：\n>\n> Ævar Arnfjörð Bjarmason wrote:\n> >\n> > On Thu, May 27 2021, Jiang Xin wrote:\n> > > I wrote a program on macOS to check memory footprint for sed and perl.\n> > > See:\n> > >\n> > > https://github.com/jiangxin/compare-sed-perl\n> >\n> > Interesting use of Go for as a /usr/bin/time -v replacement :)\n>\n> Here's a Ruby version:\n> https://dpaste.com/FYT2QKHJE\n\nNice, it's much simpler.\n\n> I'm not sure if will be useful in this particular case, but Ruby code\n> always ends up simpler ;)\n>\n> --\n> Felipe Contreras\n"},{"id":"426116","messageId":"874keh94ga.fsf@evledraar.gmail.com","threadId":"54923","inReplyTo":"CANYiYbG1vVqN-De0n2Ukexh0Jn8e_b2G1CPv24T1fbYgyhKjfg@mail.gmail.com","subject":"Re: Runaway sed memory use in test on older sed+glibc (was \"Re: [PATCH v6 1/3] test: add helper functions for git-bundle\")","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-06-01T11:50:52Z","receivedAt":"2021-06-01T12:04:12Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Jun 01 2021, Jiang Xin wrote:\n\n> Ævar Arnfjörð Bjarmason 于2021年5月27日周四 下午8:49写道：\n>>\n>> But no, the issue as it turns out is not Perl v.s. Sed, it's that\n>> there's some bug in the shellscript / tooling version (happens with both\n>> dash 0.5.7-4 and bash 4.3-11+deb8u2 on that box) where those expansions\n>> like ${A%${A#??????0?}} resolve to nothing.\n>\n> That's the root cause. It can be reproduced by running the following\n> test script:\n>\n> ```\n> #!/bin/sh\n> # test script: test.sh\n>\n> test_commit_setvar () {\n> var=$1 &&\n> oid=1234567890123456789012345678901234567890 &&\n> eval $var=$oid\n> }\n>\n> test_commit_setvar A\n> echo \"A: $A\"\n> echo \"Abbrev of A: ${A%${A#???????}}\"\n> ```\n>\n> By running different version of dash, we can see that dash 0.5.7 fail the test:\n>\n> ```\n> $ /opt/dash/0.5.11/bin/dash test.sh\n> A: 1234567890123456789012345678901234567890\n> Abbrev of A: 1234567\n>\n> $ /opt/dash/0.5.7/bin/dash test.sh\n> A: 1234567890123456789012345678901234567890\n> Abbrev of A:\n> ```\n>\n> This issue can be fixed using the following example:\n>\n> ```\n> #!/bin/sh\n>\n> test_commit_setvar () {\n> var=$1 &&\n> oid=1234567890123456789012345678901234567890 &&\n> suffix=${oid#???????} &&\n> oid=${oid%$suffix} &&\n> eval $var=$oid\n> }\n>\n> test_commit_setvar A\n> echo \"Abbrev of A: $A\"\n> ```\n\n*nod*\n\n>> Anyway, looking at this whole test file with fresh eyes this pattern\n>> seems very strange. You duplicated most of test_commit with this\n>> test_commit_setvar. It's a bit more verbosity but why not just use:\n>>\n>> test_commit ...\n>> A=$(git rev-parse HEAD)\n>\n> The function \"test_commit()\" in \"test-lib-function.sh\" always creates\n> tags and it cannot make merge commit. So I rewrite a new function\n> which reuse the scaffold of \"test_commit\".\n\nIt's had a --no-tag since 3803a3a099 (t: add --no-tag option to\ntest_commit, 2021-02-09). I also have patches in \"next\" to add more\noptions, you can just add more, having a --merge and maybe a way to tell\nit to eval the rev-parse into a given variable seem like sensible\nadditions.\n\n> BTW, sorry for the late reply, will send patch later.\n\nMy main point was that looking at this I think it's very much over the\ncomplexity v.s. benefit line on the \"complexity\" side.\n\nEven if there wasn't a --no-tag just using \"test_commit\" with a \"git tag\n-d\" and \"commit_X=$(git rev-parse HEAD)\" is less magical and more\nreadable.\n\nI.e. the mostly copy/pasted from test-lib-functions.sh function is ~70\nlines, the whole setup function is 50 lines.\n\nAnd as I noted with the whitespace getting lost in the munging the end\nresult is actually less reliable than just doing a test_cmp with $(git\nrev-parse ...) instead of .\n\nIf you were trying to avoid the whitespace warnings then see the\n\"'s/Z$//'\" pattern in t0000-basic.sh for how we've usually tested that,\ni.e. had a \"Z\" at the end mark intentional whitespace for\ntest_cmp-alike.\n\nThere's a big value in the test suite being mostly consistent (which it\nsomewhat isn't, but we're hopefully getting there). I.e. the goal isn't\nto optimize each test file to be as small as possible, but to e.g. have\nthe next person maintaining it not wondering where comes\nfrom, understanding some test_commit-alike that eval's variables into\nexistence, how it's subtly different (if at all) from test_commit etc.\n"},{"id":"426123","messageId":"CANYiYbEXiaJ-iN+wLijoKGB_ebEF3eDK8BBwtYz0unE0mKtR8w@mail.gmail.com","threadId":"54923","inReplyTo":"874keh94ga.fsf@evledraar.gmail.com","subject":"Re: Runaway sed memory use in test on older sed+glibc (was \"Re: [PATCH v6 1/3] test: add helper functions for git-bundle\")","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-06-01T13:20:45Z","receivedAt":"2021-06-01T13:20:59Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"Ævar Arnfjörð Bjarmason 于2021年6月1日周二 下午8:04写道：\n>\n>\n> On Tue, Jun 01 2021, Jiang Xin wrote:\n>\n> > Ævar Arnfjörð Bjarmason 于2021年5月27日周四 下午8:49写道：\n> >>\n> >> But no, the issue as it turns out is not Perl v.s. Sed, it's that\n> >> there's some bug in the shellscript / tooling version (happens with both\n> >> dash 0.5.7-4 and bash 4.3-11+deb8u2 on that box) where those expansions\n> >> like ${A%${A#??????0?}} resolve to nothing.\n> >\n> > That's the root cause. It can be reproduced by running the following\n> > test script:\n> >\n> > ```\n> > #!/bin/sh\n> > # test script: test.sh\n> >\n> > test_commit_setvar () {\n> > var=$1 &&\n> > oid=1234567890123456789012345678901234567890 &&\n> > eval $var=$oid\n> > }\n> >\n> > test_commit_setvar A\n> > echo \"A: $A\"\n> > echo \"Abbrev of A: ${A%${A#???????}}\"\n> > ```\n> >\n> > By running different version of dash, we can see that dash 0.5.7 fail the test:\n> >\n> > ```\n> > $ /opt/dash/0.5.11/bin/dash test.sh\n> > A: 1234567890123456789012345678901234567890\n> > Abbrev of A: 1234567\n> >\n> > $ /opt/dash/0.5.7/bin/dash test.sh\n> > A: 1234567890123456789012345678901234567890\n> > Abbrev of A:\n> > ```\n> >\n> > This issue can be fixed using the following example:\n> >\n> > ```\n> > #!/bin/sh\n> >\n> > test_commit_setvar () {\n> > var=$1 &&\n> > oid=1234567890123456789012345678901234567890 &&\n> > suffix=${oid#???????} &&\n> > oid=${oid%$suffix} &&\n> > eval $var=$oid\n> > }\n> >\n> > test_commit_setvar A\n> > echo \"Abbrev of A: $A\"\n> > ```\n>\n> *nod*\n>\n> >> Anyway, looking at this whole test file with fresh eyes this pattern\n> >> seems very strange. You duplicated most of test_commit with this\n> >> test_commit_setvar. It's a bit more verbosity but why not just use:\n> >>\n> >> test_commit ...\n> >> A=$(git rev-parse HEAD)\n> >\n> > The function \"test_commit()\" in \"test-lib-function.sh\" always creates\n> > tags and it cannot make merge commit. So I rewrite a new function\n> > which reuse the scaffold of \"test_commit\".\n>\n> It's had a --no-tag since 3803a3a099 (t: add --no-tag option to\n> test_commit, 2021-02-09). I also have patches in \"next\" to add more\n> options, you can just add more, having a --merge and maybe a way to tell\n> it to eval the rev-parse into a given variable seem like sensible\n> additions.\n>\n> > BTW, sorry for the late reply, will send patch later.\n>\n> My main point was that looking at this I think it's very much over the\n> complexity v.s. benefit line on the \"complexity\" side.\n>\n> Even if there wasn't a --no-tag just using \"test_commit\" with a \"git tag\n> -d\" and \"commit_X=$(git rev-parse HEAD)\" is less magical and more\n> readable.\n>\n> I.e. the mostly copy/pasted from test-lib-functions.sh function is ~70\n> lines, the whole setup function is 50 lines.\n>\n> And as I noted with the whitespace getting lost in the munging the end\n> result is actually less reliable than just doing a test_cmp with $(git\n> rev-parse ...) instead of .\n>\n> If you were trying to avoid the whitespace warnings then see the\n> \"'s/Z$//'\" pattern in t0000-basic.sh for how we've usually tested that,\n> i.e. had a \"Z\" at the end mark intentional whitespace for\n> test_cmp-alike.\n>\n> There's a big value in the test suite being mostly consistent (which it\n> somewhat isn't, but we're hopefully getting there). I.e. the goal isn't\n> to optimize each test file to be as small as possible, but to e.g. have\n> the next person maintaining it not wondering where comes\n> from, understanding some test_commit-alike that eval's variables into\n> existence, how it's subtly different (if at all) from test_commit etc.\n\nWill send a patch for quick fix for t6020 which is broken on older\nversion of bash.\n\nAfter changes on \"test_commit()\" of \"test-lib-function.sh\" has been\nmerge to master branch, I will try to refactor t6020 again to remove\n`test_commit_setvar()` and reuse `test_commit()`.\n"},{"id":"426133","messageId":"20210601144901.31375-1-worldhello.net@gmail.com","threadId":"54923","inReplyTo":"CANYiYbEXiaJ-iN+wLijoKGB_ebEF3eDK8BBwtYz0unE0mKtR8w@mail.gmail.com","subject":"[PATCH 1/2] t6020: fix bash incompatible issue","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-06-01T14:49:00Z","receivedAt":"2021-06-01T14:49:37Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"From: Jiang Xin \n\nÆvar reported that the function `make_user_friendly_and_stable_output()`\nfailed on a i386 box (gcc45) in the gcc farm boxes with error:\n\n sed: couldn't re-allocate memory\n\nIt turns out that older versions of bash (4.3) or dash (0.5.7) cannot\nevaluate expression like `${A%${A#???????}}` used to get the leading 7\ncharacters of variable A.\n\nReplace the complex expressions so that t6020 works on older version of\nbash or dash.\n\nReported-by: Ævar Arnfjörð Bjarmason \nSigned-off-by: Jiang Xin \n---\n t/t6020-bundle-misc.sh | 41 +++++++++++++++++++++--------------------\n 1 file changed, 21 insertions(+), 20 deletions(-)\n\ndiff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh\nindex 881f72fd44..c6a8ea7f76 100755\n--- a/t/t6020-bundle-misc.sh\n+++ b/t/t6020-bundle-misc.sh\n@@ -77,7 +77,8 @@ test_commit_setvar () {\n \t\tgit ${indir:+ -C \"$indir\"} commit $signoff -m \"$1\" &&\n \t\toid=$(git ${indir:+ -C \"$indir\"} rev-parse HEAD)\n \tfi &&\n-\teval $var=$oid\n+\tsuffix=${oid#???????} &&\n+\teval $var=${oid%$suffix}\n }\n \n # Format the output of git commands to make a user-friendly and stable\n@@ -85,25 +86,25 @@ test_commit_setvar () {\n # about future changes of the commit ID and spaces of the output.\n make_user_friendly_and_stable_output () {\n \tsed \\\n-\t\t-e \"s/${A%${A#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${B%${B#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${C%${C#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${D%${D#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${E%${E#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${F%${F#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${G%${G#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${H%${H#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${I%${I#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${J%${J#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${K%${K#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${L%${L#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${M%${M#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${N%${N#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${O%${O#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${P%${P#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${TAG1%${TAG1#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${TAG2%${TAG2#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${TAG3%${TAG3#???????}}[0-9a-f]*//g\" \\\n+\t\t-e \"s/$A[0-9a-f]*//g\" \\\n+\t\t-e \"s/$B[0-9a-f]*//g\" \\\n+\t\t-e \"s/$C[0-9a-f]*//g\" \\\n+\t\t-e \"s/$D[0-9a-f]*//g\" \\\n+\t\t-e \"s/$E[0-9a-f]*//g\" \\\n+\t\t-e \"s/$F[0-9a-f]*//g\" \\\n+\t\t-e \"s/$G[0-9a-f]*//g\" \\\n+\t\t-e \"s/$H[0-9a-f]*//g\" \\\n+\t\t-e \"s/$I[0-9a-f]*//g\" \\\n+\t\t-e \"s/$J[0-9a-f]*//g\" \\\n+\t\t-e \"s/$K[0-9a-f]*//g\" \\\n+\t\t-e \"s/$L[0-9a-f]*//g\" \\\n+\t\t-e \"s/$M[0-9a-f]*//g\" \\\n+\t\t-e \"s/$N[0-9a-f]*//g\" \\\n+\t\t-e \"s/$O[0-9a-f]*//g\" \\\n+\t\t-e \"s/$P[0-9a-f]*//g\" \\\n+\t\t-e \"s/$TAG1[0-9a-f]*//g\" \\\n+\t\t-e \"s/$TAG2[0-9a-f]*//g\" \\\n+\t\t-e \"s/$TAG3[0-9a-f]*//g\" \\\n \t\t-e \"s/ *\\$//\"\n }\n \n-- \n2.32.0.rc0\n\n"},{"id":"426134","messageId":"20210601144901.31375-2-worldhello.net@gmail.com","threadId":"54923","inReplyTo":"CANYiYbEXiaJ-iN+wLijoKGB_ebEF3eDK8BBwtYz0unE0mKtR8w@mail.gmail.com","subject":"[PATCH 2/2] t6020: do not mangle trailing spaces in output","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-06-01T14:49:01Z","receivedAt":"2021-06-01T14:49:39Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"From: Jiang Xin \n\nWe used to call `make_user_friendly_and_stable_output` to mangle\ntrailing spaces in output before comparing with the expect file.\nÆvar recommends generating expect file using pattern \"'s/Z$//'\" to\ncompare expect file with raw output.\n\nSuggested-by: Ævar Arnfjörð Bjarmason \nSigned-off-by: Jiang Xin \n---\n t/t6020-bundle-misc.sh | 89 ++++++++++++++++++++++--------------------\n 1 file changed, 46 insertions(+), 43 deletions(-)\n\ndiff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh\nindex c6a8ea7f76..7ee43fb1df 100755\n--- a/t/t6020-bundle-misc.sh\n+++ b/t/t6020-bundle-misc.sh\n@@ -83,7 +83,7 @@ test_commit_setvar () {\n \n # Format the output of git commands to make a user-friendly and stable\n # text. We can easily prepare the expect text without having to worry\n-# about future changes of the commit ID and spaces of the output.\n+# about future changes of the commit ID.\n make_user_friendly_and_stable_output () {\n \tsed \\\n \t\t-e \"s/$A[0-9a-f]*//g\" \\\n@@ -104,8 +104,11 @@ make_user_friendly_and_stable_output () {\n \t\t-e \"s/$P[0-9a-f]*//g\" \\\n \t\t-e \"s/$TAG1[0-9a-f]*//g\" \\\n \t\t-e \"s/$TAG2[0-9a-f]*//g\" \\\n-\t\t-e \"s/$TAG3[0-9a-f]*//g\" \\\n-\t\t-e \"s/ *\\$//\"\n+\t\t-e \"s/$TAG3[0-9a-f]*//g\"\n+}\n+\n+format_and_save_expect () {\n+\tsed -e 's/^> //' -e 's/Z$//' >expect\n }\n \n # (C) (D, pull/1/head, topic/1)\n@@ -180,11 +183,11 @@ test_expect_success 'create bundle from special rev: main^!' '\n \n \tgit bundle verify special-rev.bdl |\n \t\tmake_user_friendly_and_stable_output >actual &&\n-\tcat >expect <<-\\EOF &&\n-\tThe bundle contains this ref:\n-\t refs/heads/main\n-\tThe bundle requires this ref:\n-\t\n+\tformat_and_save_expect <<-\\EOF &&\n+\t> The bundle contains this ref:\n+\t> refs/heads/main\n+\t> The bundle requires this ref:\n+\t> Z\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -201,12 +204,12 @@ test_expect_success 'create bundle with --max-count option' '\n \n \tgit bundle verify max-count.bdl |\n \t\tmake_user_friendly_and_stable_output >actual &&\n-\tcat >expect <<-\\EOF &&\n-\tThe bundle contains these 2 refs:\n-\t refs/heads/main\n-\t refs/tags/v1\n-\tThe bundle requires this ref:\n-\t\n+\tformat_and_save_expect <<-\\EOF &&\n+\t> The bundle contains these 2 refs:\n+\t> refs/heads/main\n+\t> refs/tags/v1\n+\t> The bundle requires this ref:\n+\t> Z\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -226,16 +229,16 @@ test_expect_success 'create bundle with --since option' '\n \n \tgit bundle verify since.bdl |\n \t\tmake_user_friendly_and_stable_output >actual &&\n-\tcat >expect <<-\\EOF &&\n-\tThe bundle contains these 5 refs:\n-\t refs/heads/main\n-\t refs/heads/release\n-\t refs/tags/v2\n-\t refs/tags/v3\n-\t HEAD\n-\tThe bundle requires these 2 refs:\n-\t\n-\t\n+\tformat_and_save_expect <<-\\EOF &&\n+\t> The bundle contains these 5 refs:\n+\t> refs/heads/main\n+\t> refs/heads/release\n+\t> refs/tags/v2\n+\t> refs/tags/v3\n+\t> HEAD\n+\t> The bundle requires these 2 refs:\n+\t> Z\n+\t> Z\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -294,13 +297,13 @@ test_expect_success 'create bundle 2 - has prerequisites' '\n \t\t--stdin \\\n \t\trelease expect <<-\\EOF &&\n-\tThe bundle contains this ref:\n-\t refs/heads/release\n-\tThe bundle requires these 3 refs:\n-\t\n-\t\n-\t\n+\tformat_and_save_expect <<-\\EOF &&\n+\t> The bundle contains this ref:\n+\t> refs/heads/release\n+\t> The bundle requires these 3 refs:\n+\t> Z\n+\t> Z\n+\t> Z\n \tEOF\n \n \tgit bundle verify 2.bdl |\n@@ -318,11 +321,11 @@ test_expect_success 'create bundle 2 - has prerequisites' '\n test_expect_success 'fail to verify bundle without prerequisites' '\n \tgit init --bare test1.git &&\n \n-\tcat >expect <<-\\EOF &&\n-\terror: Repository lacks these prerequisite commits:\n-\terror: \n-\terror: \n-\terror: \n+\tformat_and_save_expect <<-\\EOF &&\n+\t> error: Repository lacks these prerequisite commits:\n+\t> error: Z\n+\t> error: Z\n+\t> error: Z\n \tEOF\n \n \ttest_must_fail git -C test1.git bundle verify ../2.bdl 2>&1 |\n@@ -353,13 +356,13 @@ test_expect_success 'create bundle 3 - two refs, same object' '\n \t\t--stdin \\\n \t\tmain HEAD expect <<-\\EOF &&\n-\tThe bundle contains these 2 refs:\n-\t refs/heads/main\n-\t HEAD\n-\tThe bundle requires these 2 refs:\n-\t\n-\t\n+\tformat_and_save_expect <<-\\EOF &&\n+\t> The bundle contains these 2 refs:\n+\t> refs/heads/main\n+\t> HEAD\n+\t> The bundle requires these 2 refs:\n+\t> Z\n+\t> Z\n \tEOF\n \n \tgit bundle verify 3.bdl |\n-- \n2.32.0.rc0\n\n"},{"id":"426497","messageId":"87im2s5jjm.fsf@evledraar.gmail.com","threadId":"54923","inReplyTo":"20210601144901.31375-2-worldhello.net@gmail.com","subject":"Re: [PATCH 2/2] t6020: do not mangle trailing spaces in output","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-06-05T17:02:41Z","receivedAt":"2021-06-05T17:06:33Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Jun 01 2021, Jiang Xin wrote:\n\n> From: Jiang Xin \n>\n> We used to call `make_user_friendly_and_stable_output` to mangle\n> trailing spaces in output before comparing with the expect file.\n> Ævar recommends generating expect file using pattern \"'s/Z$//'\" to\n> compare expect file with raw output.\n\nI've tested both of these patches and they fix the reported issue on\nthat gcc45 machine.\n\n> Suggested-by: Ævar Arnfjörð Bjarmason \n> Signed-off-by: Jiang Xin \n> ---\n> t/t6020-bundle-misc.sh | 89 ++++++++++++++++++++++--------------------\n> 1 file changed, 46 insertions(+), 43 deletions(-)\n>\n> diff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh\n> index c6a8ea7f76..7ee43fb1df 100755\n> --- a/t/t6020-bundle-misc.sh\n> +++ b/t/t6020-bundle-misc.sh\n> @@ -83,7 +83,7 @@ test_commit_setvar () {\n> \n> # Format the output of git commands to make a user-friendly and stable\n> # text. We can easily prepare the expect text without having to worry\n> -# about future changes of the commit ID and spaces of the output.\n> +# about future changes of the commit ID.\n> make_user_friendly_and_stable_output () {\n> \tsed \\\n> \t\t-e \"s/$A[0-9a-f]*//g\" \\\n> @@ -104,8 +104,11 @@ make_user_friendly_and_stable_output () {\n> \t\t-e \"s/$P[0-9a-f]*//g\" \\\n> \t\t-e \"s/$TAG1[0-9a-f]*//g\" \\\n> \t\t-e \"s/$TAG2[0-9a-f]*//g\" \\\n> -\t\t-e \"s/$TAG3[0-9a-f]*//g\" \\\n> -\t\t-e \"s/ *\\$//\"\n> +\t\t-e \"s/$TAG3[0-9a-f]*//g\"\n> +}\n> +\n> +format_and_save_expect () {\n> +\tsed -e 's/^> //' -e 's/Z$//' >expect\n> }\n> \n> # (C) (D, pull/1/head, topic/1)\n> @@ -180,11 +183,11 @@ test_expect_success 'create bundle from special rev: main^!' '\n> \n> \tgit bundle verify special-rev.bdl |\n> \t\tmake_user_friendly_and_stable_output >actual &&\n> -\tcat >expect <<-\\EOF &&\n> -\tThe bundle contains this ref:\n> -\t refs/heads/main\n> -\tThe bundle requires this ref:\n> -\t\n> +\tformat_and_save_expect <<-\\EOF &&\n> +\t> The bundle contains this ref:\n> +\t> refs/heads/main\n> +\t> The bundle requires this ref:\n> +\t> Z\n> \tEOF\n> \ttest_cmp expect actual &&\n\nI think for this test it would be better to just have the \"Z\" suffix and\nleave off prefixing all the lines with \">\". That's done in t0000.sh\nbecause we have various empty lines, but here that's not the case.\n\nI don't think that nit is worth a re-roll, and as you noted you're doing\nsome larger changes to these tests in follow-up patches.\n\nIt's just convenient to have the test_cmp expect input as close to\ncopy/paste-able as possible.\n"},{"id":"427170","messageId":"20210612050711.4057-4-worldhello.net@gmail.com","threadId":"54923","inReplyTo":"87im2s5jjm.fsf@evledraar.gmail.com","subject":"[PATCH v2 3/4] sideband: append suffix for message whose CR in next pktline","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-06-12T05:07:10Z","receivedAt":"2021-06-12T05:07:38Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"From: Jiang Xin \n\nWhen calling \"demultiplex_sideband\" on a sideband-2 message, will try to\nsplit the message by line breaks, and append a suffix to each nonempty\nline to clear the end of the screen line. But in the following example,\nthere will be no suffix (8 spaces) for \"\":\n\n PKT-LINE(\\2 CR CR )\n PKT-LINE(\\2 CR CR CR)\n\nThis is because the line break of \"\" is placed in the next\npktline message.\n\nWithout this fix, t5411 must remove trailing spaces of the actual output\nof \"git-push\" command before comparing.\n\nSigned-off-by: Jiang Xin \n---\n sideband.c | 4 ++++\n 1 file changed, 4 insertions(+)\n\ndiff --git a/sideband.c b/sideband.c\nindex 6f9e026732..abf2be98e1 100644\n--- a/sideband.c\n+++ b/sideband.c\n@@ -185,6 +185,10 @@ int demultiplex_sideband(const char *me, int status,\n \n \t\t\tif (!scratch->len)\n \t\t\t\tstrbuf_addstr(scratch, DISPLAY_PREFIX);\n+\t\t\telse if (!linelen)\n+\t\t\t\t/* buf has a leading CR which ends the remaining\n+\t\t\t\t * scratch of last round of \"demultiplex_sideband\" */\n+\t\t\t\tstrbuf_addstr(scratch, suffix);\n \t\t\tif (linelen > 0) {\n \t\t\t\tmaybe_colorize_sideband(scratch, b, linelen);\n \t\t\t\tstrbuf_addstr(scratch, suffix);\n-- \n2.32.0.rc0.27.g7b1e85181b\n\n"},{"id":"427171","messageId":"20210612050711.4057-1-worldhello.net@gmail.com","threadId":"54923","inReplyTo":"87im2s5jjm.fsf@evledraar.gmail.com","subject":"[PATCH v2 0/4] Fixed t6020 bash compatible issue and fixed wrong sideband suffix issue","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-06-12T05:07:07Z","receivedAt":"2021-06-12T05:07:38Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"From: Jiang Xin \n\nIn addition to fix the bash incompatible issue of t6020, find another\nissue when try to rewrite t5411 to compare raw command output.\n\nFunction \"demultiplex_sideband()\" will try to split the sideband-2\nmessage by line breaks, and append a suffix to each nonempty line\nto clear the end of the screen line. But in the following example,\nthere will be no suffix (8 spaces) for \"\":\n \n PKT-LINE(\\2 CR CR )\n PKT-LINE(\\2 CR CR CR)\n \nThis is because the line break of \"\" is placed in the\nnext pktline message.\n\nWith this fix, we could rewrite t5411 to test raw output of \"git push\"\nwhich has a stable suffix for each remote sideband-2 message.\n\nJiang Xin (4):\n t6020: fix bash incompatible issue\n test: refactor create_commits_in() for t5411 and t5548\n sideband: append suffix for message whose CR in next pktline\n test: compare raw output, not mangle tabs and spaces\n\n sideband.c | 4 +\n t/t5411/common-functions.sh | 54 +++--\n t/t5411/test-0000-standard-git-push.sh | 82 +++----\n .../test-0001-standard-git-push--porcelain.sh | 90 ++++----\n ...st-0003-pre-receive-declined--porcelain.sh | 8 +-\n t/t5411/test-0011-no-hook-error.sh | 40 ++--\n t/t5411/test-0012-no-hook-error--porcelain.sh | 42 ++--\n t/t5411/test-0013-bad-protocol.sh | 62 +++---\n t/t5411/test-0014-bad-protocol--porcelain.sh | 80 +++----\n t/t5411/test-0020-report-ng.sh | 32 +--\n t/t5411/test-0021-report-ng--porcelain.sh | 36 ++--\n t/t5411/test-0022-report-unexpect-ref.sh | 26 +--\n ...est-0023-report-unexpect-ref--porcelain.sh | 28 +--\n t/t5411/test-0024-report-unknown-ref.sh | 18 +-\n ...test-0025-report-unknown-ref--porcelain.sh | 20 +-\n t/t5411/test-0026-push-options.sh | 58 ++---\n t/t5411/test-0027-push-options--porcelain.sh | 62 +++---\n t/t5411/test-0030-report-ok.sh | 20 +-\n t/t5411/test-0031-report-ok--porcelain.sh | 22 +-\n t/t5411/test-0032-report-with-options.sh | 186 ++++++++--------\n ...est-0033-report-with-options--porcelain.sh | 200 +++++++++---------\n t/t5411/test-0034-report-ft.sh | 22 +-\n t/t5411/test-0035-report-ft--porcelain.sh | 24 +--\n ...t-0036-report-multi-rewrite-for-one-ref.sh | 132 ++++++------\n ...rt-multi-rewrite-for-one-ref--porcelain.sh | 138 ++++++------\n t/t5411/test-0038-report-mixed-refs.sh | 74 +++----\n .../test-0039-report-mixed-refs--porcelain.sh | 76 +++----\n t/t5411/test-0040-process-all-refs.sh | 80 +++----\n .../test-0041-process-all-refs--porcelain.sh | 82 +++----\n ...t-0050-proc-receive-refs-with-modifiers.sh | 90 ++++----\n t/t5548-push-porcelain.sh | 97 +++++----\n t/t6020-bundle-misc.sh | 93 ++++----\n 32 files changed, 1047 insertions(+), 1031 deletions(-)\n\n-- \n2.32.0.rc0.27.g7b1e85181b\n\n"},{"id":"427172","messageId":"20210612050711.4057-2-worldhello.net@gmail.com","threadId":"54923","inReplyTo":"87im2s5jjm.fsf@evledraar.gmail.com","subject":"[PATCH v2 1/4] t6020: fix bash incompatible issue","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-06-12T05:07:08Z","receivedAt":"2021-06-12T05:08:35Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"From: Jiang Xin \n\nÆvar reported that the function `make_user_friendly_and_stable_output()`\nfailed on a i386 box (gcc45) in the gcc farm boxes with error:\n\n sed: couldn't re-allocate memory\n\nIt turns out that older versions of bash (4.3) or dash (0.5.7) cannot\nevaluate expression like `${A%${A#???????}}` used to get the leading 7\ncharacters of variable A.\n\nReplace the complex expressions so that t6020 works on older version of\nbash or dash.\n\nReported-by: Ævar Arnfjörð Bjarmason \nSigned-off-by: Jiang Xin \n---\n t/t6020-bundle-misc.sh | 50 ++++++++++++++++++++++++++----------------\n 1 file changed, 31 insertions(+), 19 deletions(-)\n\ndiff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh\nindex 881f72fd44..3140ca4fdc 100755\n--- a/t/t6020-bundle-misc.sh\n+++ b/t/t6020-bundle-misc.sh\n@@ -80,30 +80,42 @@ test_commit_setvar () {\n \teval $var=$oid\n }\n \n+get_abbrev_oid () {\n+\toid=$1 &&\n+\tsuffix=${oid#???????} &&\n+\toid=${oid%$suffix} &&\n+\tif test -n \"$oid\"\n+\tthen\n+\t\techo \"$oid\"\n+\telse\n+\t\techo \"undefined-oid\"\n+\tfi\n+}\n+\n # Format the output of git commands to make a user-friendly and stable\n # text. We can easily prepare the expect text without having to worry\n # about future changes of the commit ID and spaces of the output.\n make_user_friendly_and_stable_output () {\n \tsed \\\n-\t\t-e \"s/${A%${A#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${B%${B#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${C%${C#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${D%${D#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${E%${E#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${F%${F#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${G%${G#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${H%${H#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${I%${I#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${J%${J#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${K%${K#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${L%${L#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${M%${M#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${N%${N#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${O%${O#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${P%${P#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${TAG1%${TAG1#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${TAG2%${TAG2#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${TAG3%${TAG3#???????}}[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $A)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $B)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $C)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $D)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $E)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $F)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $G)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $H)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $I)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $J)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $K)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $L)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $M)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $N)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $O)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $P)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $TAG1)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $TAG2)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $TAG3)[0-9a-f]*//g\" \\\n \t\t-e \"s/ *\\$//\"\n }\n \n-- \n2.32.0.rc0.27.g7b1e85181b\n\n"},{"id":"427173","messageId":"20210612050711.4057-3-worldhello.net@gmail.com","threadId":"54923","inReplyTo":"87im2s5jjm.fsf@evledraar.gmail.com","subject":"[PATCH v2 2/4] test: refactor create_commits_in() for t5411 and t5548","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-06-12T05:07:09Z","receivedAt":"2021-06-12T05:08:35Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"From: Jiang Xin \n\nUse \"test_commit\" in \"create_commits_in\" to create commit and get abbrev\nobject ID by using function \"get_abbrev_oid\".\n\nSigned-off-by: Jiang Xin \n---\n t/t5411/common-functions.sh | 45 ++++++++---------\n t/t5411/test-0000-standard-git-push.sh | 6 +--\n .../test-0001-standard-git-push--porcelain.sh | 6 +--\n t/t5411/test-0022-report-unexpect-ref.sh | 2 +-\n ...est-0023-report-unexpect-ref--porcelain.sh | 2 +-\n t/t5411/test-0032-report-with-options.sh | 8 +--\n ...est-0033-report-with-options--porcelain.sh | 8 +--\n ...t-0036-report-multi-rewrite-for-one-ref.sh | 10 ++--\n ...rt-multi-rewrite-for-one-ref--porcelain.sh | 10 ++--\n t/t5411/test-0038-report-mixed-refs.sh | 4 +-\n .../test-0039-report-mixed-refs--porcelain.sh | 4 +-\n t/t5411/test-0040-process-all-refs.sh | 8 +--\n .../test-0041-process-all-refs--porcelain.sh | 8 +--\n ...t-0050-proc-receive-refs-with-modifiers.sh | 4 +-\n t/t5548-push-porcelain.sh | 49 +++++++++----------\n 15 files changed, 84 insertions(+), 90 deletions(-)\n\ndiff --git a/t/t5411/common-functions.sh b/t/t5411/common-functions.sh\nindex 6694858e18..6398f5f2a3 100644\n--- a/t/t5411/common-functions.sh\n+++ b/t/t5411/common-functions.sh\n@@ -6,29 +6,28 @@\n # NOTE: Never calling this function from a subshell since variable\n # assignments will disappear when subshell exits.\n create_commits_in () {\n-\trepo=\"$1\" &&\n-\tif ! parent=$(git -C \"$repo\" rev-parse HEAD^{} --)\n-\tthen\n-\t\tparent=\n-\tfi &&\n-\tT=$(git -C \"$repo\" write-tree) &&\n+\trepo=\"$1\" && test -d \"$repo\" ||\n+\terror \"Repository $repo does not exist.\"\n \tshift &&\n \twhile test $# -gt 0\n \tdo\n \t\tname=$1 &&\n-\t\ttest_tick &&\n-\t\tif test -z \"$parent\"\n-\t\tthen\n-\t\t\toid=$(echo $name | git -C \"$repo\" commit-tree $T)\n-\t\telse\n-\t\t\toid=$(echo $name | git -C \"$repo\" commit-tree -p $parent $T)\n-\t\tfi &&\n-\t\teval $name=$oid &&\n-\t\tparent=$oid &&\n-\t\tshift ||\n-\t\treturn 1\n-\tdone &&\n-\tgit -C \"$repo\" update-ref refs/heads/main $oid\n+\t\tshift &&\n+\t\ttest_commit -C \"$repo\" --no-tag \"$name\" &&\n+\t\teval $name=$(git -C \"$repo\" rev-parse HEAD)\n+\tdone\n+}\n+\n+get_abbrev_oid () {\n+\toid=$1 &&\n+\tsuffix=${oid#???????} &&\n+\toid=${oid%$suffix} &&\n+\tif test -n \"$oid\"\n+\tthen\n+\t\techo \"$oid\"\n+\telse\n+\t\techo \"undefined-oid\"\n+\tfi\n }\n \n # Format the output of git-push, git-show-ref and other commands to make a\n@@ -44,12 +43,10 @@ make_user_friendly_and_stable_output () {\n \t\t-e \"s/ */ /g\" \\\n \t\t-e \"s/'/\\\"/g\" \\\n \t\t-e \"s/\t/ /g\" \\\n-\t\t-e \"s/$A//g\" \\\n-\t\t-e \"s/$B//g\" \\\n-\t\t-e \"s/$TAG//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $A)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $B)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $TAG)[0-9a-f]*//g\" \\\n \t\t-e \"s/$ZERO_OID//g\" \\\n-\t\t-e \"s/$(echo $A | cut -c1-7)[0-9a-f]*//g\" \\\n-\t\t-e \"s/$(echo $B | cut -c1-7)[0-9a-f]*//g\" \\\n \t\t-e \"s#To $URL_PREFIX/upstream.git#To #\" \\\n \t\t-e \"/^error: / d\"\n }\ndiff --git a/t/t5411/test-0000-standard-git-push.sh b/t/t5411/test-0000-standard-git-push.sh\nindex e1e0175c12..2c69cf60d4 100644\n--- a/t/t5411/test-0000-standard-git-push.sh\n+++ b/t/t5411/test-0000-standard-git-push.sh\n@@ -15,7 +15,7 @@ test_expect_success \"git-push ($PROTOCOL)\" '\n \tremote: post-receive< refs/heads/main\n \tremote: post-receive< refs/heads/next\n \tTo \n-\t .. -> main\n+\t .. -> main\n \t * [new branch] HEAD -> next\n \tEOF\n \ttest_cmp expect actual &&\n@@ -69,7 +69,7 @@ test_expect_success \"non-fast-forward git-push ($PROTOCOL)\" '\n \tremote: # post-receive hook\n \tremote: post-receive< refs/heads/next\n \tTo \n-\t .. -> next\n+\t .. -> next\n \t ! [rejected] main -> main (non-fast-forward)\n \tEOF\n \ttest_cmp expect actual &&\n@@ -106,7 +106,7 @@ test_expect_success \"git-push -f ($PROTOCOL)\" '\n \tremote: post-receive< refs/review/main/topic\n \tremote: post-receive< refs/heads/a/b/c\n \tTo \n-\t + ... main -> main (forced update)\n+\t + ... main -> main (forced update)\n \t - [deleted] next\n \t * [new tag] v123 -> v123\n \t * [new reference] main -> refs/review/main/topic\ndiff --git a/t/t5411/test-0001-standard-git-push--porcelain.sh b/t/t5411/test-0001-standard-git-push--porcelain.sh\nindex bcbda72341..7b982c8395 100644\n--- a/t/t5411/test-0001-standard-git-push--porcelain.sh\n+++ b/t/t5411/test-0001-standard-git-push--porcelain.sh\n@@ -15,7 +15,7 @@ test_expect_success \"git-push ($PROTOCOL/porcelain)\" '\n \tremote: post-receive< refs/heads/main\n \tremote: post-receive< refs/heads/next\n \tTo \n-\t :refs/heads/main ..\n+\t :refs/heads/main ..\n \t* HEAD:refs/heads/next [new branch]\n \tDone\n \tEOF\n@@ -71,7 +71,7 @@ test_expect_success \"non-fast-forward git-push ($PROTOCOL/porcelain)\" '\n \tremote: # post-receive hook\n \tremote: post-receive< refs/heads/next\n \tTo \n-\t :refs/heads/next ..\n+\t :refs/heads/next ..\n \t! refs/heads/main:refs/heads/main [rejected] (non-fast-forward)\n \tDone\n \tEOF\n@@ -109,7 +109,7 @@ test_expect_success \"git-push -f ($PROTOCOL/porcelain)\" '\n \tremote: post-receive< refs/review/main/topic\n \tremote: post-receive< refs/heads/a/b/c\n \tTo \n-\t+ refs/heads/main:refs/heads/main ... (forced update)\n+\t+ refs/heads/main:refs/heads/main ... (forced update)\n \t- :refs/heads/next [deleted]\n \t* refs/tags/v123:refs/tags/v123 [new tag]\n \t* refs/heads/main:refs/review/main/topic [new reference]\ndiff --git a/t/t5411/test-0022-report-unexpect-ref.sh b/t/t5411/test-0022-report-unexpect-ref.sh\nindex dbed467186..a482ff931a 100644\n--- a/t/t5411/test-0022-report-unexpect-ref.sh\n+++ b/t/t5411/test-0022-report-unexpect-ref.sh\n@@ -26,7 +26,7 @@ test_expect_success \"proc-receive: report unexpected ref ($PROTOCOL)\" '\n \tremote: # post-receive hook\n \tremote: post-receive< refs/heads/main\n \tTo \n-\t .. -> main\n+\t .. -> main\n \t ! [remote rejected] HEAD -> refs/for/main/topic (proc-receive failed to report status)\n \tEOF\n \ttest_cmp expect actual &&\ndiff --git a/t/t5411/test-0023-report-unexpect-ref--porcelain.sh b/t/t5411/test-0023-report-unexpect-ref--porcelain.sh\nindex e89096fa13..c586cd70a0 100644\n--- a/t/t5411/test-0023-report-unexpect-ref--porcelain.sh\n+++ b/t/t5411/test-0023-report-unexpect-ref--porcelain.sh\n@@ -26,7 +26,7 @@ test_expect_success \"proc-receive: report unexpected ref ($PROTOCOL/porcelain)\"\n \tremote: # post-receive hook\n \tremote: post-receive< refs/heads/main\n \tTo \n-\t :refs/heads/main ..\n+\t :refs/heads/main ..\n \t! HEAD:refs/for/main/topic [remote rejected] (proc-receive failed to report status)\n \tDone\n \tEOF\ndiff --git a/t/t5411/test-0032-report-with-options.sh b/t/t5411/test-0032-report-with-options.sh\nindex 437ade012d..bbd09ecdfb 100644\n--- a/t/t5411/test-0032-report-with-options.sh\n+++ b/t/t5411/test-0032-report-with-options.sh\n@@ -123,7 +123,7 @@ test_expect_success \"proc-receive: report option refname and old-oid ($PROTOCOL)\n \tremote: # post-receive hook\n \tremote: post-receive< refs/pull/123/head\n \tTo \n-\t .. HEAD -> refs/pull/123/head\n+\t .. HEAD -> refs/pull/123/head\n \tEOF\n \ttest_cmp expect actual\n '\n@@ -155,7 +155,7 @@ test_expect_success \"proc-receive: report option old-oid ($PROTOCOL)\" '\n \tremote: # post-receive hook\n \tremote: post-receive< refs/for/main/topic\n \tTo \n-\t .. HEAD -> refs/for/main/topic\n+\t .. HEAD -> refs/for/main/topic\n \tEOF\n \ttest_cmp expect actual\n '\n@@ -189,7 +189,7 @@ test_expect_success \"proc-receive: report option old-oid and new-oid ($PROTOCOL)\n \tremote: # post-receive hook\n \tremote: post-receive< refs/for/main/topic\n \tTo \n-\t .. HEAD -> refs/for/main/topic\n+\t .. HEAD -> refs/for/main/topic\n \tEOF\n \ttest_cmp expect actual\n '\n@@ -243,7 +243,7 @@ test_expect_success \"proc-receive: report with multiple rewrites ($PROTOCOL)\" '\n \tTo \n \t * [new reference] HEAD -> refs/pull/123/head\n \t * [new reference] HEAD -> refs/for/a/b/c/topic\n-\t + ... HEAD -> refs/pull/124/head (forced update)\n+\t + ... HEAD -> refs/pull/124/head (forced update)\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0033-report-with-options--porcelain.sh b/t/t5411/test-0033-report-with-options--porcelain.sh\nindex 11486720ee..d6a24d60ff 100644\n--- a/t/t5411/test-0033-report-with-options--porcelain.sh\n+++ b/t/t5411/test-0033-report-with-options--porcelain.sh\n@@ -127,7 +127,7 @@ test_expect_success \"proc-receive: report option refname and old-oid ($PROTOCOL/\n \tremote: # post-receive hook\n \tremote: post-receive< refs/pull/123/head\n \tTo \n-\t HEAD:refs/pull/123/head ..\n+\t HEAD:refs/pull/123/head ..\n \tDone\n \tEOF\n \ttest_cmp expect actual\n@@ -160,7 +160,7 @@ test_expect_success \"proc-receive: report option old-oid ($PROTOCOL/porcelain)\"\n \tremote: # post-receive hook\n \tremote: post-receive< refs/for/main/topic\n \tTo \n-\t HEAD:refs/for/main/topic ..\n+\t HEAD:refs/for/main/topic ..\n \tDone\n \tEOF\n \ttest_cmp expect actual\n@@ -195,7 +195,7 @@ test_expect_success \"proc-receive: report option old-oid and new-oid ($PROTOCOL/\n \tremote: # post-receive hook\n \tremote: post-receive< refs/for/main/topic\n \tTo \n-\t HEAD:refs/for/main/topic ..\n+\t HEAD:refs/for/main/topic ..\n \tDone\n \tEOF\n \ttest_cmp expect actual\n@@ -251,7 +251,7 @@ test_expect_success \"proc-receive: report with multiple rewrites ($PROTOCOL/porc\n \tTo \n \t* HEAD:refs/pull/123/head [new reference]\n \t* HEAD:refs/for/a/b/c/topic [new reference]\n-\t+ HEAD:refs/pull/124/head ... (forced update)\n+\t+ HEAD:refs/pull/124/head ... (forced update)\n \tDone\n \tEOF\n \ttest_cmp expect actual &&\ndiff --git a/t/t5411/test-0036-report-multi-rewrite-for-one-ref.sh b/t/t5411/test-0036-report-multi-rewrite-for-one-ref.sh\nindex be9b18b2b6..604656824b 100644\n--- a/t/t5411/test-0036-report-multi-rewrite-for-one-ref.sh\n+++ b/t/t5411/test-0036-report-multi-rewrite-for-one-ref.sh\n@@ -60,9 +60,9 @@ test_expect_success \"proc-receive: multiple rewrite for one ref, no refname for\n \tremote: post-receive< refs/changes/24/124/1\n \tremote: post-receive< refs/changes/25/125/1\n \tTo \n-\t .. HEAD -> refs/for/main/topic\n+\t .. HEAD -> refs/for/main/topic\n \t * [new reference] HEAD -> refs/changes/24/124/1\n-\t .. HEAD -> refs/changes/25/125/1\n+\t .. HEAD -> refs/changes/25/125/1\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -136,8 +136,8 @@ test_expect_success \"proc-receive: multiple rewrites for one ref, no refname for\n \tremote: post-receive< refs/changes/25/125/1\n \tTo \n \t * [new reference] HEAD -> refs/changes/24/124/1\n-\t .. HEAD -> refs/for/main/topic\n-\t + ... HEAD -> refs/changes/25/125/1 (forced update)\n+\t .. HEAD -> refs/for/main/topic\n+\t + ... HEAD -> refs/changes/25/125/1 (forced update)\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -198,7 +198,7 @@ test_expect_success \"proc-receive: multiple rewrites for one ref ($PROTOCOL)\" '\n \tremote: post-receive< refs/changes/24/124/2\n \tTo \n \t * [new reference] HEAD -> refs/changes/23/123/1\n-\t .. HEAD -> refs/changes/24/124/2\n+\t .. HEAD -> refs/changes/24/124/2\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0037-report-multi-rewrite-for-one-ref--porcelain.sh b/t/t5411/test-0037-report-multi-rewrite-for-one-ref--porcelain.sh\nindex 95fb89c031..6cc0c78a2a 100644\n--- a/t/t5411/test-0037-report-multi-rewrite-for-one-ref--porcelain.sh\n+++ b/t/t5411/test-0037-report-multi-rewrite-for-one-ref--porcelain.sh\n@@ -45,9 +45,9 @@ test_expect_success \"proc-receive: multiple rewrite for one ref, no refname for\n \tremote: post-receive< refs/changes/24/124/1\n \tremote: post-receive< refs/changes/25/125/1\n \tTo \n-\t HEAD:refs/for/main/topic ..\n+\t HEAD:refs/for/main/topic ..\n \t* HEAD:refs/changes/24/124/1 [new reference]\n-\t HEAD:refs/changes/25/125/1 ..\n+\t HEAD:refs/changes/25/125/1 ..\n \tDone\n \tEOF\n \ttest_cmp expect actual &&\n@@ -107,8 +107,8 @@ test_expect_success \"proc-receive: multiple rewrites for one ref, no refname for\n \tremote: post-receive< refs/changes/25/125/1\n \tTo \n \t* HEAD:refs/changes/24/124/1 [new reference]\n-\t HEAD:refs/for/main/topic ..\n-\t+ HEAD:refs/changes/25/125/1 ... (forced update)\n+\t HEAD:refs/for/main/topic ..\n+\t+ HEAD:refs/changes/25/125/1 ... (forced update)\n \tDone\n \tEOF\n \ttest_cmp expect actual &&\n@@ -155,7 +155,7 @@ test_expect_success \"proc-receive: multiple rewrites for one ref ($PROTOCOL/porc\n \tremote: post-receive< refs/changes/24/124/2\n \tTo \n \t* HEAD:refs/changes/23/123/1 [new reference]\n-\t HEAD:refs/changes/24/124/2 ..\n+\t HEAD:refs/changes/24/124/2 ..\n \tDone\n \tEOF\n \ttest_cmp expect actual &&\ndiff --git a/t/t5411/test-0038-report-mixed-refs.sh b/t/t5411/test-0038-report-mixed-refs.sh\nindex 5e005299cc..9260644814 100644\n--- a/t/t5411/test-0038-report-mixed-refs.sh\n+++ b/t/t5411/test-0038-report-mixed-refs.sh\n@@ -55,12 +55,12 @@ test_expect_success \"proc-receive: report update of mixed refs ($PROTOCOL)\" '\n \tremote: post-receive< refs/heads/foo\n \tremote: post-receive< refs/for/main/topic\n \tTo \n-\t .. -> main\n+\t .. -> main\n \t * [new branch] HEAD -> bar\n \t * [new branch] HEAD -> baz\n \t * [new reference] HEAD -> refs/for/next/topic2\n \t * [new branch] HEAD -> foo\n-\t .. HEAD -> refs/for/main/topic\n+\t .. HEAD -> refs/for/main/topic\n \t ! [remote rejected] HEAD -> refs/for/next/topic1 (fail to call Web API)\n \t ! [remote rejected] HEAD -> refs/for/next/topic3 (proc-receive failed to report status)\n \tEOF\ndiff --git a/t/t5411/test-0039-report-mixed-refs--porcelain.sh b/t/t5411/test-0039-report-mixed-refs--porcelain.sh\nindex 8f891c5385..4fe37683f8 100644\n--- a/t/t5411/test-0039-report-mixed-refs--porcelain.sh\n+++ b/t/t5411/test-0039-report-mixed-refs--porcelain.sh\n@@ -55,12 +55,12 @@ test_expect_success \"proc-receive: report update of mixed refs ($PROTOCOL/porcel\n \tremote: post-receive< refs/heads/foo\n \tremote: post-receive< refs/for/main/topic\n \tTo \n-\t :refs/heads/main ..\n+\t :refs/heads/main ..\n \t* HEAD:refs/heads/bar [new branch]\n \t* HEAD:refs/heads/baz [new branch]\n \t* HEAD:refs/for/next/topic2 [new reference]\n \t* HEAD:refs/heads/foo [new branch]\n-\t HEAD:refs/for/main/topic ..\n+\t HEAD:refs/for/main/topic ..\n \t! HEAD:refs/for/next/topic1 [remote rejected] (fail to call Web API)\n \t! HEAD:refs/for/next/topic3 [remote rejected] (proc-receive failed to report status)\n \tDone\ndiff --git a/t/t5411/test-0040-process-all-refs.sh b/t/t5411/test-0040-process-all-refs.sh\nindex fdcdcc7c2e..33a7f49a50 100644\n--- a/t/t5411/test-0040-process-all-refs.sh\n+++ b/t/t5411/test-0040-process-all-refs.sh\n@@ -85,11 +85,11 @@ test_expect_success \"proc-receive: process all refs ($PROTOCOL)\" '\n \tremote: post-receive< refs/pull/123/head\n \tremote: post-receive< refs/pull/124/head\n \tTo \n-\t .. -> bar\n+\t .. -> bar\n \t - [deleted] foo\n-\t + ... HEAD -> main (forced update)\n-\t .. HEAD -> refs/pull/123/head\n-\t + ... HEAD -> refs/pull/124/head (forced update)\n+\t + ... HEAD -> main (forced update)\n+\t .. HEAD -> refs/pull/123/head\n+\t + ... HEAD -> refs/pull/124/head (forced update)\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0041-process-all-refs--porcelain.sh b/t/t5411/test-0041-process-all-refs--porcelain.sh\nindex 73b35fe0aa..07dce47a7d 100644\n--- a/t/t5411/test-0041-process-all-refs--porcelain.sh\n+++ b/t/t5411/test-0041-process-all-refs--porcelain.sh\n@@ -85,11 +85,11 @@ test_expect_success \"proc-receive: process all refs ($PROTOCOL/porcelain)\" '\n \tremote: post-receive< refs/pull/123/head\n \tremote: post-receive< refs/pull/124/head\n \tTo \n-\t :refs/heads/bar ..\n+\t :refs/heads/bar ..\n \t- :refs/heads/foo [deleted]\n-\t+ HEAD:refs/heads/main ... (forced update)\n-\t HEAD:refs/pull/123/head ..\n-\t+ HEAD:refs/pull/124/head ... (forced update)\n+\t+ HEAD:refs/heads/main ... (forced update)\n+\t HEAD:refs/pull/123/head ..\n+\t+ HEAD:refs/pull/124/head ... (forced update)\n \tDone\n \tEOF\n \ttest_cmp expect actual &&\ndiff --git a/t/t5411/test-0050-proc-receive-refs-with-modifiers.sh b/t/t5411/test-0050-proc-receive-refs-with-modifiers.sh\nindex 7214647ada..906d75e62d 100644\n--- a/t/t5411/test-0050-proc-receive-refs-with-modifiers.sh\n+++ b/t/t5411/test-0050-proc-receive-refs-with-modifiers.sh\n@@ -46,7 +46,7 @@ test_expect_success \"proc-receive: update branch and new tag ($PROTOCOL)\" '\n \tremote: post-receive< refs/pull/123/head\n \tremote: post-receive< refs/pull/124/head\n \tTo \n-\t .. -> refs/pull/123/head\n+\t .. -> refs/pull/123/head\n \t * [new reference] v123 -> refs/pull/124/head\n \tEOF\n \ttest_cmp expect actual &&\n@@ -116,7 +116,7 @@ test_expect_success \"proc-receive: create/delete branch, and delete tag ($PROTOC\n \tremote: post-receive< refs/pull/124/head\n \tTo \n \t - [deleted] refs/pull/123/head\n-\t .. -> topic\n+\t .. -> topic\n \t - [deleted] v123\n \t * [new reference] -> refs/pull/124/head\n \tEOF\ndiff --git a/t/t5548-push-porcelain.sh b/t/t5548-push-porcelain.sh\nindex 5a761f3642..335abe85a7 100755\n--- a/t/t5548-push-porcelain.sh\n+++ b/t/t5548-push-porcelain.sh\n@@ -14,29 +14,28 @@ test_description='Test git push porcelain output'\n # NOTE: Never calling this function from a subshell since variable\n # assignments will disappear when subshell exits.\n create_commits_in () {\n-\trepo=\"$1\" &&\n-\tif ! parent=$(git -C \"$repo\" rev-parse HEAD^{} --)\n-\tthen\n-\t\tparent=\n-\tfi &&\n-\tT=$(git -C \"$repo\" write-tree) &&\n+\trepo=\"$1\" && test -d \"$repo\" ||\n+\terror \"Repository $repo does not exist.\"\n \tshift &&\n \twhile test $# -gt 0\n \tdo\n \t\tname=$1 &&\n-\t\ttest_tick &&\n-\t\tif test -z \"$parent\"\n-\t\tthen\n-\t\t\toid=$(echo $name | git -C \"$repo\" commit-tree $T)\n-\t\telse\n-\t\t\toid=$(echo $name | git -C \"$repo\" commit-tree -p $parent $T)\n-\t\tfi &&\n-\t\teval $name=$oid &&\n-\t\tparent=$oid &&\n-\t\tshift ||\n-\t\treturn 1\n-\tdone &&\n-\tgit -C \"$repo\" update-ref refs/heads/main $oid\n+\t\tshift &&\n+\t\ttest_commit -C \"$repo\" --no-tag \"$name\" &&\n+\t\teval $name=$(git -C \"$repo\" rev-parse HEAD)\n+\tdone\n+}\n+\n+get_abbrev_oid () {\n+\toid=$1 &&\n+\tsuffix=${oid#???????} &&\n+\toid=${oid%$suffix} &&\n+\tif test -n \"$oid\"\n+\tthen\n+\t\techo \"$oid\"\n+\telse\n+\t\techo \"undefined-oid\"\n+\tfi\n }\n \n # Format the output of git-push, git-show-ref and other commands to make a\n@@ -48,11 +47,9 @@ make_user_friendly_and_stable_output () {\n \t\t-e \"s/ *\\$//\" \\\n \t\t-e \"s/ */ /g\" \\\n \t\t-e \"s/\t/ /g\" \\\n-\t\t-e \"s/$A//g\" \\\n-\t\t-e \"s/$B//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $A)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $B)[0-9a-f]*//g\" \\\n \t\t-e \"s/$ZERO_OID//g\" \\\n-\t\t-e \"s/$(echo $A | cut -c1-7)[0-9a-f]*//g\" \\\n-\t\t-e \"s/$(echo $B | cut -c1-7)[0-9a-f]*//g\" \\\n \t\t-e \"s#To $URL_PREFIX/upstream.git#To #\"\n }\n \n@@ -114,9 +111,9 @@ run_git_push_porcelain_output_test() {\n \t\tcat >expect <<-EOF &&\n \t\tTo \n \t\t= refs/heads/baz:refs/heads/baz [up to date]\n-\t\t :refs/heads/bar ..\n+\t\t :refs/heads/bar ..\n \t\t- :refs/heads/foo [deleted]\n-\t\t+ refs/heads/main:refs/heads/main ... (forced update)\n+\t\t+ refs/heads/main:refs/heads/main ... (forced update)\n \t\t* refs/heads/next:refs/heads/next [new branch]\n \t\tDone\n \t\tEOF\n@@ -231,7 +228,7 @@ run_git_push_porcelain_output_test() {\n \t\tTo \n \t\t= refs/heads/next:refs/heads/next [up to date]\n \t\t- :refs/heads/baz [deleted]\n-\t\t refs/heads/main:refs/heads/main ..\n+\t\t refs/heads/main:refs/heads/main ..\n \t\t! refs/heads/bar:refs/heads/bar [rejected] (non-fast-forward)\n \t\tDone\n \t\tEOF\n-- \n2.32.0.rc0.27.g7b1e85181b\n\n"},{"id":"427174","messageId":"20210612050711.4057-5-worldhello.net@gmail.com","threadId":"54923","inReplyTo":"87im2s5jjm.fsf@evledraar.gmail.com","subject":"[PATCH v2 4/4] test: compare raw output, not mangle tabs and spaces","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-06-12T05:07:11Z","receivedAt":"2021-06-12T05:08:35Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"From: Jiang Xin \n\nWe used to call `make_user_friendly_and_stable_output` to mangle\ntrailing spaces in output before comparing with the expect file.\nÆvar recommends generating expect file using pattern \"'s/Z$//'\" to\ncompare expect file with raw output.\n\nSuggested-by: Ævar Arnfjörð Bjarmason \nSigned-off-by: Jiang Xin \n---\n t/t5411/common-functions.sh | 9 +-\n t/t5411/test-0000-standard-git-push.sh | 82 +++----\n .../test-0001-standard-git-push--porcelain.sh | 90 ++++----\n ...st-0003-pre-receive-declined--porcelain.sh | 8 +-\n t/t5411/test-0011-no-hook-error.sh | 40 ++--\n t/t5411/test-0012-no-hook-error--porcelain.sh | 42 ++--\n t/t5411/test-0013-bad-protocol.sh | 62 +++---\n t/t5411/test-0014-bad-protocol--porcelain.sh | 80 +++----\n t/t5411/test-0020-report-ng.sh | 32 +--\n t/t5411/test-0021-report-ng--porcelain.sh | 36 ++--\n t/t5411/test-0022-report-unexpect-ref.sh | 26 +--\n ...est-0023-report-unexpect-ref--porcelain.sh | 28 +--\n t/t5411/test-0024-report-unknown-ref.sh | 18 +-\n ...test-0025-report-unknown-ref--porcelain.sh | 20 +-\n t/t5411/test-0026-push-options.sh | 58 ++---\n t/t5411/test-0027-push-options--porcelain.sh | 62 +++---\n t/t5411/test-0030-report-ok.sh | 20 +-\n t/t5411/test-0031-report-ok--porcelain.sh | 22 +-\n t/t5411/test-0032-report-with-options.sh | 186 ++++++++--------\n ...est-0033-report-with-options--porcelain.sh | 200 +++++++++---------\n t/t5411/test-0034-report-ft.sh | 22 +-\n t/t5411/test-0035-report-ft--porcelain.sh | 24 +--\n ...t-0036-report-multi-rewrite-for-one-ref.sh | 132 ++++++------\n ...rt-multi-rewrite-for-one-ref--porcelain.sh | 138 ++++++------\n t/t5411/test-0038-report-mixed-refs.sh | 74 +++----\n .../test-0039-report-mixed-refs--porcelain.sh | 76 +++----\n t/t5411/test-0040-process-all-refs.sh | 80 +++----\n .../test-0041-process-all-refs--porcelain.sh | 82 +++----\n ...t-0050-proc-receive-refs-with-modifiers.sh | 90 ++++----\n t/t5548-push-porcelain.sh | 54 ++---\n t/t6020-bundle-misc.sh | 45 ++--\n 31 files changed, 972 insertions(+), 966 deletions(-)\n\ndiff --git a/t/t5411/common-functions.sh b/t/t5411/common-functions.sh\nindex 6398f5f2a3..3c747782c1 100644\n--- a/t/t5411/common-functions.sh\n+++ b/t/t5411/common-functions.sh\n@@ -32,17 +32,14 @@ get_abbrev_oid () {\n \n # Format the output of git-push, git-show-ref and other commands to make a\n # user-friendly and stable text. We can easily prepare the expect text\n-# without having to worry about future changes of the commit ID and spaces\n+# without having to worry about changes of the commit ID (full or abbrev.)\n # of the output. Single quotes are replaced with double quotes, because\n # it is boring to prepare unquoted single quotes in expect text. We also\n # remove some locale error messages. The emitted human-readable errors are\n # redundant to the more machine-readable output the tests already assert.\n make_user_friendly_and_stable_output () {\n \tsed \\\n-\t\t-e \"s/ *\\$//\" \\\n-\t\t-e \"s/ */ /g\" \\\n \t\t-e \"s/'/\\\"/g\" \\\n-\t\t-e \"s/\t/ /g\" \\\n \t\t-e \"s/$(get_abbrev_oid $A)[0-9a-f]*//g\" \\\n \t\t-e \"s/$(get_abbrev_oid $B)[0-9a-f]*//g\" \\\n \t\t-e \"s/$(get_abbrev_oid $TAG)[0-9a-f]*//g\" \\\n@@ -56,6 +53,10 @@ filter_out_user_friendly_and_stable_output () {\n \t\tsed -n ${1+\"$@\"}\n }\n \n+format_and_save_expect () {\n+\tsed -e 's/^> //' -e 's/Z$//' >expect\n+}\n+\n test_cmp_refs () {\n \tindir=\n \tif test \"$1\" = \"-C\"\ndiff --git a/t/t5411/test-0000-standard-git-push.sh b/t/t5411/test-0000-standard-git-push.sh\nindex 2c69cf60d4..ce64bb660b 100644\n--- a/t/t5411/test-0000-standard-git-push.sh\n+++ b/t/t5411/test-0000-standard-git-push.sh\n@@ -7,16 +7,16 @@ test_expect_success \"git-push ($PROTOCOL)\" '\n \t\tHEAD:refs/heads/next \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/heads/main\n-\tremote: pre-receive< refs/heads/next\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/heads/main\n-\tremote: post-receive< refs/heads/next\n-\tTo \n-\t .. -> main\n-\t * [new branch] HEAD -> next\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/heads/main Z\n+\t> remote: pre-receive< refs/heads/next Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/heads/main Z\n+\t> remote: post-receive< refs/heads/next Z\n+\t> To \n+\t> .. -> main\n+\t> * [new branch] HEAD -> next\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -38,10 +38,10 @@ test_expect_success \"git-push --atomic ($PROTOCOL)\" '\n \t\t-e \"/^To / { p; }\" \\\n \t\t-e \"/^ ! / { p; }\" \\\n \t\tactual &&\n-\tcat >expect <<-EOF &&\n-\tTo \n-\t ! [rejected] main -> main (non-fast-forward)\n-\t ! [rejected] -> next (atomic push failed)\n+\tformat_and_save_expect <<-EOF &&\n+\t> To \n+\t> ! [rejected] main -> main (non-fast-forward)\n+\t> ! [rejected] -> next (atomic push failed)\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -63,14 +63,14 @@ test_expect_success \"non-fast-forward git-push ($PROTOCOL)\" '\n \t\t$B:refs/heads/next \\\n \t\t>out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/heads/next\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/heads/next\n-\tTo \n-\t .. -> next\n-\t ! [rejected] main -> main (non-fast-forward)\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/heads/next Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/heads/next Z\n+\t> To \n+\t> .. -> next\n+\t> ! [rejected] main -> main (non-fast-forward)\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -92,25 +92,25 @@ test_expect_success \"git-push -f ($PROTOCOL)\" '\n \t\tHEAD:refs/heads/a/b/c \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/heads/main\n-\tremote: pre-receive< refs/heads/next\n-\tremote: pre-receive< refs/tags/v123\n-\tremote: pre-receive< refs/review/main/topic\n-\tremote: pre-receive< refs/heads/a/b/c\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/heads/main\n-\tremote: post-receive< refs/heads/next\n-\tremote: post-receive< refs/tags/v123\n-\tremote: post-receive< refs/review/main/topic\n-\tremote: post-receive< refs/heads/a/b/c\n-\tTo \n-\t + ... main -> main (forced update)\n-\t - [deleted] next\n-\t * [new tag] v123 -> v123\n-\t * [new reference] main -> refs/review/main/topic\n-\t * [new branch] HEAD -> a/b/c\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/heads/main Z\n+\t> remote: pre-receive< refs/heads/next Z\n+\t> remote: pre-receive< refs/tags/v123 Z\n+\t> remote: pre-receive< refs/review/main/topic Z\n+\t> remote: pre-receive< refs/heads/a/b/c Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/heads/main Z\n+\t> remote: post-receive< refs/heads/next Z\n+\t> remote: post-receive< refs/tags/v123 Z\n+\t> remote: post-receive< refs/review/main/topic Z\n+\t> remote: post-receive< refs/heads/a/b/c Z\n+\t> To \n+\t> + ... main -> main (forced update)\n+\t> - [deleted] next\n+\t> * [new tag] v123 -> v123\n+\t> * [new reference] main -> refs/review/main/topic\n+\t> * [new branch] HEAD -> a/b/c\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0001-standard-git-push--porcelain.sh b/t/t5411/test-0001-standard-git-push--porcelain.sh\nindex 7b982c8395..373ec3d865 100644\n--- a/t/t5411/test-0001-standard-git-push--porcelain.sh\n+++ b/t/t5411/test-0001-standard-git-push--porcelain.sh\n@@ -7,17 +7,17 @@ test_expect_success \"git-push ($PROTOCOL/porcelain)\" '\n \t\tHEAD:refs/heads/next \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/heads/main\n-\tremote: pre-receive< refs/heads/next\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/heads/main\n-\tremote: post-receive< refs/heads/next\n-\tTo \n-\t :refs/heads/main ..\n-\t* HEAD:refs/heads/next [new branch]\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/heads/main Z\n+\t> remote: pre-receive< refs/heads/next Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/heads/main Z\n+\t> remote: post-receive< refs/heads/next Z\n+\t> To \n+\t> \t:refs/heads/main\t..\n+\t> *\tHEAD:refs/heads/next\t[new branch]\n+\t> Done\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -38,12 +38,12 @@ test_expect_success \"git-push --atomic ($PROTOCOL/porcelain)\" '\n \tfilter_out_user_friendly_and_stable_output \\\n \t\t-e \"s/^# GETTEXT POISON #//\" \\\n \t\t-e \"/^To / { p; }\" \\\n-\t\t-e \"/^! / { p; }\" \\\n+\t\t-e \"/^!/ { p; }\" \\\n \t\tactual &&\n-\tcat >expect <<-EOF &&\n-\tTo \n-\t! refs/heads/main:refs/heads/main [rejected] (non-fast-forward)\n-\t! :refs/heads/next [rejected] (atomic push failed)\n+\tformat_and_save_expect <<-EOF &&\n+\t> To \n+\t> !\trefs/heads/main:refs/heads/main\t[rejected] (non-fast-forward)\n+\t> !\t:refs/heads/next\t[rejected] (atomic push failed)\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -65,15 +65,15 @@ test_expect_success \"non-fast-forward git-push ($PROTOCOL/porcelain)\" '\n \t\t$B:refs/heads/next \\\n \t\t>out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/heads/next\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/heads/next\n-\tTo \n-\t :refs/heads/next ..\n-\t! refs/heads/main:refs/heads/main [rejected] (non-fast-forward)\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/heads/next Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/heads/next Z\n+\t> To \n+\t> \t:refs/heads/next\t..\n+\t> !\trefs/heads/main:refs/heads/main\t[rejected] (non-fast-forward)\n+\t> Done\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -95,26 +95,26 @@ test_expect_success \"git-push -f ($PROTOCOL/porcelain)\" '\n \t\tHEAD:refs/heads/a/b/c \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/heads/main\n-\tremote: pre-receive< refs/heads/next\n-\tremote: pre-receive< refs/tags/v123\n-\tremote: pre-receive< refs/review/main/topic\n-\tremote: pre-receive< refs/heads/a/b/c\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/heads/main\n-\tremote: post-receive< refs/heads/next\n-\tremote: post-receive< refs/tags/v123\n-\tremote: post-receive< refs/review/main/topic\n-\tremote: post-receive< refs/heads/a/b/c\n-\tTo \n-\t+ refs/heads/main:refs/heads/main ... (forced update)\n-\t- :refs/heads/next [deleted]\n-\t* refs/tags/v123:refs/tags/v123 [new tag]\n-\t* refs/heads/main:refs/review/main/topic [new reference]\n-\t* HEAD:refs/heads/a/b/c [new branch]\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/heads/main Z\n+\t> remote: pre-receive< refs/heads/next Z\n+\t> remote: pre-receive< refs/tags/v123 Z\n+\t> remote: pre-receive< refs/review/main/topic Z\n+\t> remote: pre-receive< refs/heads/a/b/c Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/heads/main Z\n+\t> remote: post-receive< refs/heads/next Z\n+\t> remote: post-receive< refs/tags/v123 Z\n+\t> remote: post-receive< refs/review/main/topic Z\n+\t> remote: post-receive< refs/heads/a/b/c Z\n+\t> To \n+\t> +\trefs/heads/main:refs/heads/main\t... (forced update)\n+\t> -\t:refs/heads/next\t[deleted]\n+\t> *\trefs/tags/v123:refs/tags/v123\t[new tag]\n+\t> *\trefs/heads/main:refs/review/main/topic\t[new reference]\n+\t> *\tHEAD:refs/heads/a/b/c\t[new branch]\n+\t> Done\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0003-pre-receive-declined--porcelain.sh b/t/t5411/test-0003-pre-receive-declined--porcelain.sh\nindex e9c9db5d1f..2393b04ad9 100644\n--- a/t/t5411/test-0003-pre-receive-declined--porcelain.sh\n+++ b/t/t5411/test-0003-pre-receive-declined--porcelain.sh\n@@ -14,10 +14,10 @@ test_expect_success \"git-push is declined ($PROTOCOL/porcelain)\" '\n \t\tHEAD:refs/heads/next \\\n \t\t>out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tTo \n-\t! :refs/heads/main [remote rejected] (pre-receive hook declined)\n-\t! HEAD:refs/heads/next [remote rejected] (pre-receive hook declined)\n+\tformat_and_save_expect <<-EOF &&\n+\t> To \n+\t> !\t:refs/heads/main\t[remote rejected] (pre-receive hook declined)\n+\t> !\tHEAD:refs/heads/next\t[remote rejected] (pre-receive hook declined)\n \tDone\n \tEOF\n \ttest_cmp expect actual &&\ndiff --git a/t/t5411/test-0011-no-hook-error.sh b/t/t5411/test-0011-no-hook-error.sh\nindex 3ef136e6ef..d35002b1f0 100644\n--- a/t/t5411/test-0011-no-hook-error.sh\n+++ b/t/t5411/test-0011-no-hook-error.sh\n@@ -7,16 +7,16 @@ test_expect_success \"proc-receive: no hook, fail to push special ref ($PROTOCOL)\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/heads/next\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: error: cannot find hook \"proc-receive\"\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/heads/next\n-\tTo \n-\t * [new branch] HEAD -> next\n-\t ! [remote rejected] HEAD -> refs/for/main/topic (fail to run proc-receive hook)\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/heads/next Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: error: cannot find hook \"proc-receive\" Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/heads/next Z\n+\t> To \n+\t> * [new branch] HEAD -> next\n+\t> ! [remote rejected] HEAD -> refs/for/main/topic (fail to run proc-receive hook)\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -41,16 +41,16 @@ test_expect_success \"proc-receive: no hook, all failed for atomic push ($PROTOCO\n \t\tHEAD:next \\\n \t\tHEAD:refs/for/main/topic >out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/heads/main\n-\tremote: pre-receive< refs/heads/next\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: error: cannot find hook \"proc-receive\"\n-\tTo \n-\t ! [remote rejected] -> main (fail to run proc-receive hook)\n-\t ! [remote rejected] HEAD -> next (fail to run proc-receive hook)\n-\t ! [remote rejected] HEAD -> refs/for/main/topic (fail to run proc-receive hook)\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/heads/main Z\n+\t> remote: pre-receive< refs/heads/next Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: error: cannot find hook \"proc-receive\" Z\n+\t> To \n+\t> ! [remote rejected] -> main (fail to run proc-receive hook)\n+\t> ! [remote rejected] HEAD -> next (fail to run proc-receive hook)\n+\t> ! [remote rejected] HEAD -> refs/for/main/topic (fail to run proc-receive hook)\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0012-no-hook-error--porcelain.sh b/t/t5411/test-0012-no-hook-error--porcelain.sh\nindex 19f66fbd7d..04468b5018 100644\n--- a/t/t5411/test-0012-no-hook-error--porcelain.sh\n+++ b/t/t5411/test-0012-no-hook-error--porcelain.sh\n@@ -7,16 +7,16 @@ test_expect_success \"proc-receive: no hook, fail to push special ref ($PROTOCOL/\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/heads/next\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: error: cannot find hook \"proc-receive\"\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/heads/next\n-\tTo \n-\t* HEAD:refs/heads/next [new branch]\n-\t! HEAD:refs/for/main/topic [remote rejected] (fail to run proc-receive hook)\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/heads/next Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: error: cannot find hook \"proc-receive\" Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/heads/next Z\n+\t> To \n+\t> *\tHEAD:refs/heads/next\t[new branch]\n+\t> !\tHEAD:refs/for/main/topic\t[remote rejected] (fail to run proc-receive hook)\n \tDone\n \tEOF\n \ttest_cmp expect actual &&\n@@ -42,17 +42,17 @@ test_expect_success \"proc-receive: no hook, all failed for atomic push ($PROTOCO\n \t\tHEAD:next \\\n \t\tHEAD:refs/for/main/topic >out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/heads/main\n-\tremote: pre-receive< refs/heads/next\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: error: cannot find hook \"proc-receive\"\n-\tTo \n-\t! :refs/heads/main [remote rejected] (fail to run proc-receive hook)\n-\t! HEAD:refs/heads/next [remote rejected] (fail to run proc-receive hook)\n-\t! HEAD:refs/for/main/topic [remote rejected] (fail to run proc-receive hook)\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/heads/main Z\n+\t> remote: pre-receive< refs/heads/next Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: error: cannot find hook \"proc-receive\" Z\n+\t> To \n+\t> !\t:refs/heads/main\t[remote rejected] (fail to run proc-receive hook)\n+\t> !\tHEAD:refs/heads/next\t[remote rejected] (fail to run proc-receive hook)\n+\t> !\tHEAD:refs/for/main/topic\t[remote rejected] (fail to run proc-receive hook)\n+\t> Done\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0013-bad-protocol.sh b/t/t5411/test-0013-bad-protocol.sh\nindex 095e613f6f..c08a00ded2 100644\n--- a/t/t5411/test-0013-bad-protocol.sh\n+++ b/t/t5411/test-0013-bad-protocol.sh\n@@ -29,8 +29,8 @@ test_expect_success \"proc-receive: bad protocol (unknown version, $PROTOCOL)\" '\n \t# message (\"remote: fatal: the remote end hung up unexpectedly\") which\n \t# is different from the remote HTTP server with different locale settings.\n \tgrep \"^remote: error:\" actual-error &&\n-\tcat >expect <<-EOF &&\n-\tremote: error: proc-receive version \"2\" is not supported\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: error: proc-receive version \"2\" is not supported Z\n \tEOF\n \ttest_cmp expect actual-error &&\n \n@@ -208,17 +208,17 @@ test_expect_success \"proc-receive: bad protocol (no report, $PROTOCOL)\" '\n \t\tHEAD:refs/heads/next \\\n \t\tHEAD:refs/for/main/topic >out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/heads/next\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/heads/next\n-\tTo \n-\t * [new branch] HEAD -> next\n-\t ! [remote rejected] HEAD -> refs/for/main/topic (proc-receive failed to report status)\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/heads/next Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/heads/next Z\n+\t> To \n+\t> * [new branch] HEAD -> next\n+\t> ! [remote rejected] HEAD -> refs/for/main/topic (proc-receive failed to report status)\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -251,15 +251,15 @@ test_expect_success \"proc-receive: bad protocol (no ref, $PROTOCOL)\" '\n \t\tHEAD:refs/for/main/topic\\\n \t\t>out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok\n-\tremote: error: proc-receive reported incomplete status line: \"ok\"\n-\tTo \n-\t ! [remote rejected] HEAD -> refs/for/main/topic (proc-receive failed to report status)\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok Z\n+\t> remote: error: proc-receive reported incomplete status line: \"ok\" Z\n+\t> To \n+\t> ! [remote rejected] HEAD -> refs/for/main/topic (proc-receive failed to report status)\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -284,15 +284,15 @@ test_expect_success \"proc-receive: bad protocol (unknown status, $PROTOCOL)\" '\n \t\t\tHEAD:refs/for/main/topic \\\n \t\t\t>out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> xx refs/for/main/topic\n-\tremote: error: proc-receive reported bad status \"xx\" on ref \"refs/for/main/topic\"\n-\tTo \n-\t ! [remote rejected] HEAD -> refs/for/main/topic (proc-receive failed to report status)\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> xx refs/for/main/topic Z\n+\t> remote: error: proc-receive reported bad status \"xx\" on ref \"refs/for/main/topic\" Z\n+\t> To \n+\t> ! [remote rejected] HEAD -> refs/for/main/topic (proc-receive failed to report status)\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0014-bad-protocol--porcelain.sh b/t/t5411/test-0014-bad-protocol--porcelain.sh\nindex a44649789c..3eaa597e0f 100644\n--- a/t/t5411/test-0014-bad-protocol--porcelain.sh\n+++ b/t/t5411/test-0014-bad-protocol--porcelain.sh\n@@ -20,7 +20,7 @@ test_expect_success \"proc-receive: bad protocol (unknown version, $PROTOCOL/porc\n \t\tactual-report &&\n \tcat >expect <<-EOF &&\n \tTo \n-\t! HEAD:refs/for/main/topic [remote rejected] (fail to run proc-receive hook)\n+\t!\tHEAD:refs/for/main/topic\t[remote rejected] (fail to run proc-receive hook)\n \tDone\n \tEOF\n \ttest_cmp expect actual-report &&\n@@ -29,8 +29,8 @@ test_expect_success \"proc-receive: bad protocol (unknown version, $PROTOCOL/porc\n \t# message (\"remote: fatal: the remote end hung up unexpectedly\") which\n \t# is different from the remote HTTP server with different locale settings.\n \tgrep \"^remote: error:\" actual-error &&\n-\tcat >expect <<-EOF &&\n-\tremote: error: proc-receive version \"2\" is not supported\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: error: proc-receive version \"2\" is not supported Z\n \tEOF\n \ttest_cmp expect actual-error &&\n \n@@ -58,7 +58,7 @@ test_expect_success \"proc-receive: bad protocol (hook --die-read-version, $PROTO\n \t\tactual &&\n \tcat >expect <<-EOF &&\n \tTo \n-\t! HEAD:refs/for/main/topic [remote rejected] (fail to run proc-receive hook)\n+\t!\tHEAD:refs/for/main/topic\t[remote rejected] (fail to run proc-receive hook)\n \tDone\n \tEOF\n \ttest_cmp expect actual &&\n@@ -89,7 +89,7 @@ test_expect_success \"proc-receive: bad protocol (hook --die-write-version, $PROT\n \t\tactual &&\n \tcat >expect <<-EOF &&\n \tTo \n-\t! HEAD:refs/for/main/topic [remote rejected] (fail to run proc-receive hook)\n+\t!\tHEAD:refs/for/main/topic\t[remote rejected] (fail to run proc-receive hook)\n \tDone\n \tEOF\n \ttest_cmp expect actual &&\n@@ -120,7 +120,7 @@ test_expect_success \"proc-receive: bad protocol (hook --die-read-commands, $PROT\n \t\tactual &&\n \tcat >expect <<-EOF &&\n \tTo \n-\t! HEAD:refs/for/main/topic [remote rejected] (fail to run proc-receive hook)\n+\t!\tHEAD:refs/for/main/topic\t[remote rejected] (fail to run proc-receive hook)\n \tDone\n \tEOF\n \ttest_cmp expect actual &&\n@@ -152,7 +152,7 @@ test_expect_success \"proc-receive: bad protocol (hook --die-read-push-options, $\n \t\tactual &&\n \tcat >expect <<-EOF &&\n \tTo \n-\t! HEAD:refs/for/main/topic [remote rejected] (fail to run proc-receive hook)\n+\t!\tHEAD:refs/for/main/topic\t[remote rejected] (fail to run proc-receive hook)\n \tDone\n \tEOF\n \ttest_cmp expect actual &&\n@@ -182,7 +182,7 @@ test_expect_success \"proc-receive: bad protocol (hook --die-write-report, $PROTO\n \t\tactual &&\n \tcat >expect <<-EOF &&\n \tTo \n-\t! HEAD:refs/for/main/topic [remote rejected] (fail to run proc-receive hook)\n+\t!\tHEAD:refs/for/main/topic\t[remote rejected] (fail to run proc-receive hook)\n \tDone\n \tEOF\n \ttest_cmp expect actual &&\n@@ -208,18 +208,18 @@ test_expect_success \"proc-receive: bad protocol (no report, $PROTOCOL/porcelain)\n \t\tHEAD:refs/heads/next \\\n \t\tHEAD:refs/for/main/topic >out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/heads/next\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/heads/next\n-\tTo \n-\t* HEAD:refs/heads/next [new branch]\n-\t! HEAD:refs/for/main/topic [remote rejected] (proc-receive failed to report status)\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/heads/next Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/heads/next Z\n+\t> To \n+\t> *\tHEAD:refs/heads/next\t[new branch]\n+\t> !\tHEAD:refs/for/main/topic\t[remote rejected] (proc-receive failed to report status)\n+\t> Done\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -251,16 +251,16 @@ test_expect_success \"proc-receive: bad protocol (no ref, $PROTOCOL/porcelain)\" '\n \t\tHEAD:refs/for/main/topic\\\n \t\t>out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok\n-\tremote: error: proc-receive reported incomplete status line: \"ok\"\n-\tTo \n-\t! HEAD:refs/for/main/topic [remote rejected] (proc-receive failed to report status)\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok Z\n+\t> remote: error: proc-receive reported incomplete status line: \"ok\" Z\n+\t> To \n+\t> !\tHEAD:refs/for/main/topic\t[remote rejected] (proc-receive failed to report status)\n+\t> Done\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -285,16 +285,16 @@ test_expect_success \"proc-receive: bad protocol (unknown status, $PROTOCOL/porce\n \t\t\tHEAD:refs/for/main/topic \\\n \t\t\t>out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> xx refs/for/main/topic\n-\tremote: error: proc-receive reported bad status \"xx\" on ref \"refs/for/main/topic\"\n-\tTo \n-\t! HEAD:refs/for/main/topic [remote rejected] (proc-receive failed to report status)\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> xx refs/for/main/topic Z\n+\t> remote: error: proc-receive reported bad status \"xx\" on ref \"refs/for/main/topic\" Z\n+\t> To \n+\t> !\tHEAD:refs/for/main/topic\t[remote rejected] (proc-receive failed to report status)\n+\t> Done\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0020-report-ng.sh b/t/t5411/test-0020-report-ng.sh\nindex ad2c8f6535..e915dbc28d 100644\n--- a/t/t5411/test-0020-report-ng.sh\n+++ b/t/t5411/test-0020-report-ng.sh\n@@ -14,14 +14,14 @@ test_expect_success \"proc-receive: fail to update (ng, no message, $PROTOCOL)\" '\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ng refs/for/main/topic\n-\tTo \n-\t ! [remote rejected] HEAD -> refs/for/main/topic (failed)\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ng refs/for/main/topic Z\n+\t> To \n+\t> ! [remote rejected] HEAD -> refs/for/main/topic (failed)\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -46,14 +46,14 @@ test_expect_success \"proc-receive: fail to update (ng, with message, $PROTOCOL)\"\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ng refs/for/main/topic error msg\n-\tTo \n-\t ! [remote rejected] HEAD -> refs/for/main/topic (error msg)\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ng refs/for/main/topic error msg Z\n+\t> To \n+\t> ! [remote rejected] HEAD -> refs/for/main/topic (error msg)\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0021-report-ng--porcelain.sh b/t/t5411/test-0021-report-ng--porcelain.sh\nindex d8ae9d3414..2a392e099b 100644\n--- a/t/t5411/test-0021-report-ng--porcelain.sh\n+++ b/t/t5411/test-0021-report-ng--porcelain.sh\n@@ -14,15 +14,15 @@ test_expect_success \"proc-receive: fail to update (ng, no message, $PROTOCOL/por\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ng refs/for/main/topic\n-\tTo \n-\t! HEAD:refs/for/main/topic [remote rejected] (failed)\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ng refs/for/main/topic Z\n+\t> To \n+\t> !\tHEAD:refs/for/main/topic\t[remote rejected] (failed)\n+\t> Done\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -47,15 +47,15 @@ test_expect_success \"proc-receive: fail to update (ng, with message, $PROTOCOL/p\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ng refs/for/main/topic error msg\n-\tTo \n-\t! HEAD:refs/for/main/topic [remote rejected] (error msg)\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ng refs/for/main/topic error msg Z\n+\t> To \n+\t> !\tHEAD:refs/for/main/topic\t[remote rejected] (error msg)\n+\t> Done\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0022-report-unexpect-ref.sh b/t/t5411/test-0022-report-unexpect-ref.sh\nindex a482ff931a..f7a494bdb9 100644\n--- a/t/t5411/test-0022-report-unexpect-ref.sh\n+++ b/t/t5411/test-0022-report-unexpect-ref.sh\n@@ -15,19 +15,19 @@ test_expect_success \"proc-receive: report unexpected ref ($PROTOCOL)\" '\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/heads/main\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/heads/main\n-\tremote: error: proc-receive reported status on unexpected ref: refs/heads/main\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/heads/main\n-\tTo \n-\t .. -> main\n-\t ! [remote rejected] HEAD -> refs/for/main/topic (proc-receive failed to report status)\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/heads/main Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/heads/main Z\n+\t> remote: error: proc-receive reported status on unexpected ref: refs/heads/main Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/heads/main Z\n+\t> To \n+\t> .. -> main\n+\t> ! [remote rejected] HEAD -> refs/for/main/topic (proc-receive failed to report status)\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0023-report-unexpect-ref--porcelain.sh b/t/t5411/test-0023-report-unexpect-ref--porcelain.sh\nindex c586cd70a0..63c479e975 100644\n--- a/t/t5411/test-0023-report-unexpect-ref--porcelain.sh\n+++ b/t/t5411/test-0023-report-unexpect-ref--porcelain.sh\n@@ -15,20 +15,20 @@ test_expect_success \"proc-receive: report unexpected ref ($PROTOCOL/porcelain)\"\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/heads/main\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/heads/main\n-\tremote: error: proc-receive reported status on unexpected ref: refs/heads/main\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/heads/main\n-\tTo \n-\t :refs/heads/main ..\n-\t! HEAD:refs/for/main/topic [remote rejected] (proc-receive failed to report status)\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/heads/main Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/heads/main Z\n+\t> remote: error: proc-receive reported status on unexpected ref: refs/heads/main Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/heads/main Z\n+\t> To \n+\t> \t:refs/heads/main\t..\n+\t> !\tHEAD:refs/for/main/topic\t[remote rejected] (proc-receive failed to report status)\n+\t> Done\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0024-report-unknown-ref.sh b/t/t5411/test-0024-report-unknown-ref.sh\nindex 77204244b8..af055aa086 100644\n--- a/t/t5411/test-0024-report-unknown-ref.sh\n+++ b/t/t5411/test-0024-report-unknown-ref.sh\n@@ -14,15 +14,15 @@ test_expect_success \"proc-receive: report unknown reference ($PROTOCOL)\" '\n \t\tHEAD:refs/for/a/b/c/my/topic \\\n \t\t>out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/a/b/c/my/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/a/b/c/my/topic\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: error: proc-receive reported status on unknown ref: refs/for/main/topic\n-\tTo \n-\t ! [remote rejected] HEAD -> refs/for/a/b/c/my/topic (proc-receive failed to report status)\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/a/b/c/my/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/a/b/c/my/topic Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: error: proc-receive reported status on unknown ref: refs/for/main/topic Z\n+\t> To \n+\t> ! [remote rejected] HEAD -> refs/for/a/b/c/my/topic (proc-receive failed to report status)\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0025-report-unknown-ref--porcelain.sh b/t/t5411/test-0025-report-unknown-ref--porcelain.sh\nindex eeb1ce6b2c..99601ca321 100644\n--- a/t/t5411/test-0025-report-unknown-ref--porcelain.sh\n+++ b/t/t5411/test-0025-report-unknown-ref--porcelain.sh\n@@ -14,16 +14,16 @@ test_expect_success \"proc-receive: report unknown reference ($PROTOCOL/porcelain\n \t\tHEAD:refs/for/a/b/c/my/topic \\\n \t\t>out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/a/b/c/my/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/a/b/c/my/topic\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: error: proc-receive reported status on unknown ref: refs/for/main/topic\n-\tTo \n-\t! HEAD:refs/for/a/b/c/my/topic [remote rejected] (proc-receive failed to report status)\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/a/b/c/my/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/a/b/c/my/topic Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: error: proc-receive reported status on unknown ref: refs/for/main/topic Z\n+\t> To \n+\t> !\tHEAD:refs/for/a/b/c/my/topic\t[remote rejected] (proc-receive failed to report status)\n+\t> Done\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0026-push-options.sh b/t/t5411/test-0026-push-options.sh\nindex 1ec2cb95bc..fec5f95793 100644\n--- a/t/t5411/test-0026-push-options.sh\n+++ b/t/t5411/test-0026-push-options.sh\n@@ -52,19 +52,19 @@ test_expect_success \"proc-receive: ignore push-options for version 0 ($PROTOCOL)\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/heads/next\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/heads/next\n-\tremote: post-receive< refs/for/main/topic\n-\tTo \n-\t * [new branch] HEAD -> next\n-\t * [new reference] HEAD -> refs/for/main/topic\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/heads/next Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/heads/next Z\n+\t> remote: post-receive< refs/for/main/topic Z\n+\t> To \n+\t> * [new branch] HEAD -> next\n+\t> * [new reference] HEAD -> refs/for/main/topic\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -101,22 +101,22 @@ test_expect_success \"proc-receive: push with options ($PROTOCOL)\" '\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/heads/next\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive: atomic push_options\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive< issue=123\n-\tremote: proc-receive< reviewer=user1\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/heads/next\n-\tremote: post-receive< refs/for/main/topic\n-\tTo \n-\t * [new branch] HEAD -> next\n-\t * [new reference] HEAD -> refs/for/main/topic\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/heads/next Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive: atomic push_options Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive< issue=123 Z\n+\t> remote: proc-receive< reviewer=user1 Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/heads/next Z\n+\t> remote: post-receive< refs/for/main/topic Z\n+\t> To \n+\t> * [new branch] HEAD -> next\n+\t> * [new reference] HEAD -> refs/for/main/topic\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0027-push-options--porcelain.sh b/t/t5411/test-0027-push-options--porcelain.sh\nindex 447fbfec0c..8fb75a8789 100644\n--- a/t/t5411/test-0027-push-options--porcelain.sh\n+++ b/t/t5411/test-0027-push-options--porcelain.sh\n@@ -54,20 +54,20 @@ test_expect_success \"proc-receive: ignore push-options for version 0 ($PROTOCOL/\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/heads/next\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/heads/next\n-\tremote: post-receive< refs/for/main/topic\n-\tTo \n-\t* HEAD:refs/heads/next [new branch]\n-\t* HEAD:refs/for/main/topic [new reference]\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/heads/next Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/heads/next Z\n+\t> remote: post-receive< refs/for/main/topic Z\n+\t> To \n+\t> *\tHEAD:refs/heads/next\t[new branch]\n+\t> *\tHEAD:refs/for/main/topic\t[new reference]\n+\t> Done\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -105,23 +105,23 @@ test_expect_success \"proc-receive: push with options ($PROTOCOL/porcelain)\" '\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/heads/next\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive: atomic push_options\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive< issue=123\n-\tremote: proc-receive< reviewer=user1\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/heads/next\n-\tremote: post-receive< refs/for/main/topic\n-\tTo \n-\t* HEAD:refs/heads/next [new branch]\n-\t* HEAD:refs/for/main/topic [new reference]\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/heads/next Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive: atomic push_options Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive< issue=123 Z\n+\t> remote: proc-receive< reviewer=user1 Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/heads/next Z\n+\t> remote: post-receive< refs/for/main/topic Z\n+\t> To \n+\t> *\tHEAD:refs/heads/next\t[new branch]\n+\t> *\tHEAD:refs/for/main/topic\t[new reference]\n+\t> Done\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0030-report-ok.sh b/t/t5411/test-0030-report-ok.sh\nindex 8acb4f204f..a3a6278213 100644\n--- a/t/t5411/test-0030-report-ok.sh\n+++ b/t/t5411/test-0030-report-ok.sh\n@@ -14,16 +14,16 @@ test_expect_success \"proc-receive: ok ($PROTOCOL)\" '\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/for/main/topic\n-\tTo \n-\t * [new reference] HEAD -> refs/for/main/topic\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/for/main/topic Z\n+\t> To \n+\t> * [new reference] HEAD -> refs/for/main/topic\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0031-report-ok--porcelain.sh b/t/t5411/test-0031-report-ok--porcelain.sh\nindex a967718046..0e175388b6 100644\n--- a/t/t5411/test-0031-report-ok--porcelain.sh\n+++ b/t/t5411/test-0031-report-ok--porcelain.sh\n@@ -14,17 +14,17 @@ test_expect_success \"proc-receive: ok ($PROTOCOL/porcelain)\" '\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/for/main/topic\n-\tTo \n-\t* HEAD:refs/for/main/topic [new reference]\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/for/main/topic Z\n+\t> To \n+\t> *\tHEAD:refs/for/main/topic\t[new reference]\n+\t> Done\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0032-report-with-options.sh b/t/t5411/test-0032-report-with-options.sh\nindex bbd09ecdfb..988a4302a6 100644\n--- a/t/t5411/test-0032-report-with-options.sh\n+++ b/t/t5411/test-0032-report-with-options.sh\n@@ -15,16 +15,16 @@ test_expect_success \"proc-receive: report option without matching ok ($PROTOCOL)\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> option refname refs/pull/123/head\n-\tremote: proc-receive> option old-oid \n-\tremote: error: proc-receive reported \"option\" without a matching \"ok/ng\" directive\n-\tTo \n-\t ! [remote rejected] HEAD -> refs/for/main/topic (proc-receive failed to report status)\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/pull/123/head Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: error: proc-receive reported \"option\" without a matching \"ok/ng\" directive Z\n+\t> To \n+\t> ! [remote rejected] HEAD -> refs/for/main/topic (proc-receive failed to report status)\n \tEOF\n \ttest_cmp expect actual\n '\n@@ -46,17 +46,17 @@ test_expect_success \"proc-receive: report option refname ($PROTOCOL)\" '\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option refname refs/pull/123/head\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/pull/123/head\n-\tTo \n-\t * [new reference] HEAD -> refs/pull/123/head\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/pull/123/head Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/pull/123/head Z\n+\t> To \n+\t> * [new reference] HEAD -> refs/pull/123/head\n \tEOF\n \ttest_cmp expect actual\n '\n@@ -78,18 +78,18 @@ test_expect_success \"proc-receive: report option refname and forced-update ($PRO\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option refname refs/pull/123/head\n-\tremote: proc-receive> option forced-update\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/pull/123/head\n-\tTo \n-\t * [new reference] HEAD -> refs/pull/123/head\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/pull/123/head Z\n+\t> remote: proc-receive> option forced-update Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/pull/123/head Z\n+\t> To \n+\t> * [new reference] HEAD -> refs/pull/123/head\n \tEOF\n \ttest_cmp expect actual\n '\n@@ -112,18 +112,18 @@ test_expect_success \"proc-receive: report option refname and old-oid ($PROTOCOL)\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option refname refs/pull/123/head\n-\tremote: proc-receive> option old-oid \n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/pull/123/head\n-\tTo \n-\t .. HEAD -> refs/pull/123/head\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/pull/123/head Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/pull/123/head Z\n+\t> To \n+\t> .. HEAD -> refs/pull/123/head\n \tEOF\n \ttest_cmp expect actual\n '\n@@ -145,17 +145,17 @@ test_expect_success \"proc-receive: report option old-oid ($PROTOCOL)\" '\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option old-oid \n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/for/main/topic\n-\tTo \n-\t .. HEAD -> refs/for/main/topic\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/for/main/topic Z\n+\t> To \n+\t> .. HEAD -> refs/for/main/topic\n \tEOF\n \ttest_cmp expect actual\n '\n@@ -178,18 +178,18 @@ test_expect_success \"proc-receive: report option old-oid and new-oid ($PROTOCOL)\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option new-oid \n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/for/main/topic\n-\tTo \n-\t .. HEAD -> refs/for/main/topic\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/for/main/topic Z\n+\t> To \n+\t> .. HEAD -> refs/for/main/topic\n \tEOF\n \ttest_cmp expect actual\n '\n@@ -219,31 +219,31 @@ test_expect_success \"proc-receive: report with multiple rewrites ($PROTOCOL)\" '\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/next/topic\n-\tremote: pre-receive< refs/for/a/b/c/topic\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/next/topic\n-\tremote: proc-receive< refs/for/a/b/c/topic\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/for/a/b/c/topic\n-\tremote: proc-receive> ok refs/for/next/topic\n-\tremote: proc-receive> option refname refs/pull/123/head\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option refname refs/pull/124/head\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option forced-update\n-\tremote: proc-receive> option new-oid \n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/pull/123/head\n-\tremote: post-receive< refs/for/a/b/c/topic\n-\tremote: post-receive< refs/pull/124/head\n-\tTo \n-\t * [new reference] HEAD -> refs/pull/123/head\n-\t * [new reference] HEAD -> refs/for/a/b/c/topic\n-\t + ... HEAD -> refs/pull/124/head (forced update)\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/next/topic Z\n+\t> remote: pre-receive< refs/for/a/b/c/topic Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/next/topic Z\n+\t> remote: proc-receive< refs/for/a/b/c/topic Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/for/a/b/c/topic Z\n+\t> remote: proc-receive> ok refs/for/next/topic Z\n+\t> remote: proc-receive> option refname refs/pull/123/head Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/pull/124/head Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option forced-update Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/pull/123/head Z\n+\t> remote: post-receive< refs/for/a/b/c/topic Z\n+\t> remote: post-receive< refs/pull/124/head Z\n+\t> To \n+\t> * [new reference] HEAD -> refs/pull/123/head\n+\t> * [new reference] HEAD -> refs/for/a/b/c/topic\n+\t> + ... HEAD -> refs/pull/124/head (forced update)\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0033-report-with-options--porcelain.sh b/t/t5411/test-0033-report-with-options--porcelain.sh\nindex d6a24d60ff..daacb3d69d 100644\n--- a/t/t5411/test-0033-report-with-options--porcelain.sh\n+++ b/t/t5411/test-0033-report-with-options--porcelain.sh\n@@ -15,17 +15,17 @@ test_expect_success \"proc-receive: report option without matching ok ($PROTOCOL/\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> option refname refs/pull/123/head\n-\tremote: proc-receive> option old-oid \n-\tremote: error: proc-receive reported \"option\" without a matching \"ok/ng\" directive\n-\tTo \n-\t! HEAD:refs/for/main/topic [remote rejected] (proc-receive failed to report status)\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/pull/123/head Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: error: proc-receive reported \"option\" without a matching \"ok/ng\" directive Z\n+\t> To \n+\t> !\tHEAD:refs/for/main/topic\t[remote rejected] (proc-receive failed to report status)\n+\t> Done\n \tEOF\n \ttest_cmp expect actual\n '\n@@ -47,18 +47,18 @@ test_expect_success \"proc-receive: report option refname ($PROTOCOL/porcelain)\"\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option refname refs/pull/123/head\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/pull/123/head\n-\tTo \n-\t* HEAD:refs/pull/123/head [new reference]\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/pull/123/head Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/pull/123/head Z\n+\t> To \n+\t> *\tHEAD:refs/pull/123/head\t[new reference]\n+\t> Done\n \tEOF\n \ttest_cmp expect actual\n '\n@@ -81,19 +81,19 @@ test_expect_success \"proc-receive: report option refname and forced-update ($PRO\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option refname refs/pull/123/head\n-\tremote: proc-receive> option forced-update\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/pull/123/head\n-\tTo \n-\t* HEAD:refs/pull/123/head [new reference]\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/pull/123/head Z\n+\t> remote: proc-receive> option forced-update Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/pull/123/head Z\n+\t> To \n+\t> *\tHEAD:refs/pull/123/head\t[new reference]\n+\t> Done\n \tEOF\n \ttest_cmp expect actual\n '\n@@ -116,19 +116,19 @@ test_expect_success \"proc-receive: report option refname and old-oid ($PROTOCOL/\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option refname refs/pull/123/head\n-\tremote: proc-receive> option old-oid \n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/pull/123/head\n-\tTo \n-\t HEAD:refs/pull/123/head ..\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/pull/123/head Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/pull/123/head Z\n+\t> To \n+\t> \tHEAD:refs/pull/123/head\t..\n+\t> Done\n \tEOF\n \ttest_cmp expect actual\n '\n@@ -150,18 +150,18 @@ test_expect_success \"proc-receive: report option old-oid ($PROTOCOL/porcelain)\"\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option old-oid \n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/for/main/topic\n-\tTo \n-\t HEAD:refs/for/main/topic ..\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/for/main/topic Z\n+\t> To \n+\t> \tHEAD:refs/for/main/topic\t..\n+\t> Done\n \tEOF\n \ttest_cmp expect actual\n '\n@@ -184,19 +184,19 @@ test_expect_success \"proc-receive: report option old-oid and new-oid ($PROTOCOL/\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option new-oid \n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/for/main/topic\n-\tTo \n-\t HEAD:refs/for/main/topic ..\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/for/main/topic Z\n+\t> To \n+\t> \tHEAD:refs/for/main/topic\t..\n+\t> Done\n \tEOF\n \ttest_cmp expect actual\n '\n@@ -227,32 +227,32 @@ test_expect_success \"proc-receive: report with multiple rewrites ($PROTOCOL/porc\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/next/topic\n-\tremote: pre-receive< refs/for/a/b/c/topic\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/next/topic\n-\tremote: proc-receive< refs/for/a/b/c/topic\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/for/a/b/c/topic\n-\tremote: proc-receive> ok refs/for/next/topic\n-\tremote: proc-receive> option refname refs/pull/123/head\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option refname refs/pull/124/head\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option forced-update\n-\tremote: proc-receive> option new-oid \n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/pull/123/head\n-\tremote: post-receive< refs/for/a/b/c/topic\n-\tremote: post-receive< refs/pull/124/head\n-\tTo \n-\t* HEAD:refs/pull/123/head [new reference]\n-\t* HEAD:refs/for/a/b/c/topic [new reference]\n-\t+ HEAD:refs/pull/124/head ... (forced update)\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/next/topic Z\n+\t> remote: pre-receive< refs/for/a/b/c/topic Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/next/topic Z\n+\t> remote: proc-receive< refs/for/a/b/c/topic Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/for/a/b/c/topic Z\n+\t> remote: proc-receive> ok refs/for/next/topic Z\n+\t> remote: proc-receive> option refname refs/pull/123/head Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/pull/124/head Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option forced-update Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/pull/123/head Z\n+\t> remote: post-receive< refs/for/a/b/c/topic Z\n+\t> remote: post-receive< refs/pull/124/head Z\n+\t> To \n+\t> *\tHEAD:refs/pull/123/head\t[new reference]\n+\t> *\tHEAD:refs/for/a/b/c/topic\t[new reference]\n+\t> +\tHEAD:refs/pull/124/head\t... (forced update)\n+\t> Done\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0034-report-ft.sh b/t/t5411/test-0034-report-ft.sh\nindex 6e0d08b327..73a47d1ffd 100644\n--- a/t/t5411/test-0034-report-ft.sh\n+++ b/t/t5411/test-0034-report-ft.sh\n@@ -15,17 +15,17 @@ test_expect_success \"proc-receive: fall throught, let receive-pack to execute ($\n \t\t$B:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option fall-through\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/for/main/topic\n-\tTo \n-\t * [new reference] -> refs/for/main/topic\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option fall-through Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/for/main/topic Z\n+\t> To \n+\t> * [new reference] -> refs/for/main/topic\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0035-report-ft--porcelain.sh b/t/t5411/test-0035-report-ft--porcelain.sh\nindex 81bae9f2ec..c350201107 100644\n--- a/t/t5411/test-0035-report-ft--porcelain.sh\n+++ b/t/t5411/test-0035-report-ft--porcelain.sh\n@@ -15,18 +15,18 @@ test_expect_success \"proc-receive: fall throught, let receive-pack to execute ($\n \t\t$B:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option fall-through\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/for/main/topic\n-\tTo \n-\t* :refs/for/main/topic [new reference]\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option fall-through Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/for/main/topic Z\n+\t> To \n+\t> *\t:refs/for/main/topic\t[new reference]\n+\t> Done\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0036-report-multi-rewrite-for-one-ref.sh b/t/t5411/test-0036-report-multi-rewrite-for-one-ref.sh\nindex 604656824b..8c8a6c16e1 100644\n--- a/t/t5411/test-0036-report-multi-rewrite-for-one-ref.sh\n+++ b/t/t5411/test-0036-report-multi-rewrite-for-one-ref.sh\n@@ -39,30 +39,30 @@ test_expect_success \"proc-receive: multiple rewrite for one ref, no refname for\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option new-oid \n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option refname refs/changes/24/124/1\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option new-oid \n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option refname refs/changes/25/125/1\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option new-oid \n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/for/main/topic\n-\tremote: post-receive< refs/changes/24/124/1\n-\tremote: post-receive< refs/changes/25/125/1\n-\tTo \n-\t .. HEAD -> refs/for/main/topic\n-\t * [new reference] HEAD -> refs/changes/24/124/1\n-\t .. HEAD -> refs/changes/25/125/1\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/changes/24/124/1 Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/changes/25/125/1 Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/for/main/topic Z\n+\t> remote: post-receive< refs/changes/24/124/1 Z\n+\t> remote: post-receive< refs/changes/25/125/1 Z\n+\t> To \n+\t> .. HEAD -> refs/for/main/topic\n+\t> * [new reference] HEAD -> refs/changes/24/124/1\n+\t> .. HEAD -> refs/changes/25/125/1\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -113,31 +113,31 @@ test_expect_success \"proc-receive: multiple rewrites for one ref, no refname for\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option refname refs/changes/24/124/1\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option new-oid \n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option new-oid \n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option refname refs/changes/25/125/1\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option new-oid \n-\tremote: proc-receive> option forced-update\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/changes/24/124/1\n-\tremote: post-receive< refs/for/main/topic\n-\tremote: post-receive< refs/changes/25/125/1\n-\tTo \n-\t * [new reference] HEAD -> refs/changes/24/124/1\n-\t .. HEAD -> refs/for/main/topic\n-\t + ... HEAD -> refs/changes/25/125/1 (forced update)\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/changes/24/124/1 Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/changes/25/125/1 Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: proc-receive> option forced-update Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/changes/24/124/1 Z\n+\t> remote: post-receive< refs/for/main/topic Z\n+\t> remote: post-receive< refs/changes/25/125/1 Z\n+\t> To \n+\t> * [new reference] HEAD -> refs/changes/24/124/1\n+\t> .. HEAD -> refs/for/main/topic\n+\t> + ... HEAD -> refs/changes/25/125/1 (forced update)\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -182,23 +182,23 @@ test_expect_success \"proc-receive: multiple rewrites for one ref ($PROTOCOL)\" '\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option refname refs/changes/23/123/1\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option refname refs/changes/24/124/2\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option new-oid \n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/changes/23/123/1\n-\tremote: post-receive< refs/changes/24/124/2\n-\tTo \n-\t * [new reference] HEAD -> refs/changes/23/123/1\n-\t .. HEAD -> refs/changes/24/124/2\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/changes/23/123/1 Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/changes/24/124/2 Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/changes/23/123/1 Z\n+\t> remote: post-receive< refs/changes/24/124/2 Z\n+\t> To \n+\t> * [new reference] HEAD -> refs/changes/23/123/1\n+\t> .. HEAD -> refs/changes/24/124/2\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0037-report-multi-rewrite-for-one-ref--porcelain.sh b/t/t5411/test-0037-report-multi-rewrite-for-one-ref--porcelain.sh\nindex 6cc0c78a2a..bc44810f33 100644\n--- a/t/t5411/test-0037-report-multi-rewrite-for-one-ref--porcelain.sh\n+++ b/t/t5411/test-0037-report-multi-rewrite-for-one-ref--porcelain.sh\n@@ -24,31 +24,31 @@ test_expect_success \"proc-receive: multiple rewrite for one ref, no refname for\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option new-oid \n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option refname refs/changes/24/124/1\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option new-oid \n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option refname refs/changes/25/125/1\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option new-oid \n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/for/main/topic\n-\tremote: post-receive< refs/changes/24/124/1\n-\tremote: post-receive< refs/changes/25/125/1\n-\tTo \n-\t HEAD:refs/for/main/topic ..\n-\t* HEAD:refs/changes/24/124/1 [new reference]\n-\t HEAD:refs/changes/25/125/1 ..\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/changes/24/124/1 Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/changes/25/125/1 Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/for/main/topic Z\n+\t> remote: post-receive< refs/changes/24/124/1 Z\n+\t> remote: post-receive< refs/changes/25/125/1 Z\n+\t> To \n+\t> \tHEAD:refs/for/main/topic\t..\n+\t> *\tHEAD:refs/changes/24/124/1\t[new reference]\n+\t> \tHEAD:refs/changes/25/125/1\t..\n+\t> Done\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -84,32 +84,32 @@ test_expect_success \"proc-receive: multiple rewrites for one ref, no refname for\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option refname refs/changes/24/124/1\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option new-oid \n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option new-oid \n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option refname refs/changes/25/125/1\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option new-oid \n-\tremote: proc-receive> option forced-update\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/changes/24/124/1\n-\tremote: post-receive< refs/for/main/topic\n-\tremote: post-receive< refs/changes/25/125/1\n-\tTo \n-\t* HEAD:refs/changes/24/124/1 [new reference]\n-\t HEAD:refs/for/main/topic ..\n-\t+ HEAD:refs/changes/25/125/1 ... (forced update)\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/changes/24/124/1 Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/changes/25/125/1 Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: proc-receive> option forced-update Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/changes/24/124/1 Z\n+\t> remote: post-receive< refs/for/main/topic Z\n+\t> remote: post-receive< refs/changes/25/125/1 Z\n+\t> To \n+\t> *\tHEAD:refs/changes/24/124/1\t[new reference]\n+\t> \tHEAD:refs/for/main/topic\t..\n+\t> +\tHEAD:refs/changes/25/125/1\t... (forced update)\n+\t> Done\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -139,24 +139,24 @@ test_expect_success \"proc-receive: multiple rewrites for one ref ($PROTOCOL/porc\n \t\tHEAD:refs/for/main/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option refname refs/changes/23/123/1\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option refname refs/changes/24/124/2\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option new-oid \n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/changes/23/123/1\n-\tremote: post-receive< refs/changes/24/124/2\n-\tTo \n-\t* HEAD:refs/changes/23/123/1 [new reference]\n-\t HEAD:refs/changes/24/124/2 ..\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/changes/23/123/1 Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/changes/24/124/2 Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/changes/23/123/1 Z\n+\t> remote: post-receive< refs/changes/24/124/2 Z\n+\t> To \n+\t> *\tHEAD:refs/changes/23/123/1\t[new reference]\n+\t> \tHEAD:refs/changes/24/124/2\t..\n+\t> Done\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0038-report-mixed-refs.sh b/t/t5411/test-0038-report-mixed-refs.sh\nindex 9260644814..e63fe7ba11 100644\n--- a/t/t5411/test-0038-report-mixed-refs.sh\n+++ b/t/t5411/test-0038-report-mixed-refs.sh\n@@ -26,43 +26,43 @@ test_expect_success \"proc-receive: report update of mixed refs ($PROTOCOL)\" '\n \t\tHEAD:refs/for/next/topic3 \\\n \t\t>out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/heads/main\n-\tremote: pre-receive< refs/heads/bar\n-\tremote: pre-receive< refs/heads/baz\n-\tremote: pre-receive< refs/for/next/topic2\n-\tremote: pre-receive< refs/for/next/topic1\n-\tremote: pre-receive< refs/heads/foo\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: pre-receive< refs/for/next/topic3\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/next/topic2\n-\tremote: proc-receive< refs/for/next/topic1\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive< refs/for/next/topic3\n-\tremote: proc-receive> ok refs/for/next/topic2\n-\tremote: proc-receive> ng refs/for/next/topic1 fail to call Web API\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option refname refs/for/main/topic\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option new-oid \n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/heads/main\n-\tremote: post-receive< refs/heads/bar\n-\tremote: post-receive< refs/heads/baz\n-\tremote: post-receive< refs/for/next/topic2\n-\tremote: post-receive< refs/heads/foo\n-\tremote: post-receive< refs/for/main/topic\n-\tTo \n-\t .. -> main\n-\t * [new branch] HEAD -> bar\n-\t * [new branch] HEAD -> baz\n-\t * [new reference] HEAD -> refs/for/next/topic2\n-\t * [new branch] HEAD -> foo\n-\t .. HEAD -> refs/for/main/topic\n-\t ! [remote rejected] HEAD -> refs/for/next/topic1 (fail to call Web API)\n-\t ! [remote rejected] HEAD -> refs/for/next/topic3 (proc-receive failed to report status)\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/heads/main Z\n+\t> remote: pre-receive< refs/heads/bar Z\n+\t> remote: pre-receive< refs/heads/baz Z\n+\t> remote: pre-receive< refs/for/next/topic2 Z\n+\t> remote: pre-receive< refs/for/next/topic1 Z\n+\t> remote: pre-receive< refs/heads/foo Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: pre-receive< refs/for/next/topic3 Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/next/topic2 Z\n+\t> remote: proc-receive< refs/for/next/topic1 Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive< refs/for/next/topic3 Z\n+\t> remote: proc-receive> ok refs/for/next/topic2 Z\n+\t> remote: proc-receive> ng refs/for/next/topic1 fail to call Web API Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/for/main/topic Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/heads/main Z\n+\t> remote: post-receive< refs/heads/bar Z\n+\t> remote: post-receive< refs/heads/baz Z\n+\t> remote: post-receive< refs/for/next/topic2 Z\n+\t> remote: post-receive< refs/heads/foo Z\n+\t> remote: post-receive< refs/for/main/topic Z\n+\t> To \n+\t> .. -> main\n+\t> * [new branch] HEAD -> bar\n+\t> * [new branch] HEAD -> baz\n+\t> * [new reference] HEAD -> refs/for/next/topic2\n+\t> * [new branch] HEAD -> foo\n+\t> .. HEAD -> refs/for/main/topic\n+\t> ! [remote rejected] HEAD -> refs/for/next/topic1 (fail to call Web API)\n+\t> ! [remote rejected] HEAD -> refs/for/next/topic3 (proc-receive failed to report status)\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0039-report-mixed-refs--porcelain.sh b/t/t5411/test-0039-report-mixed-refs--porcelain.sh\nindex 4fe37683f8..99d17b73af 100644\n--- a/t/t5411/test-0039-report-mixed-refs--porcelain.sh\n+++ b/t/t5411/test-0039-report-mixed-refs--porcelain.sh\n@@ -26,44 +26,44 @@ test_expect_success \"proc-receive: report update of mixed refs ($PROTOCOL/porcel\n \t\tHEAD:refs/for/next/topic3 \\\n \t\t>out-$test_count 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/heads/main\n-\tremote: pre-receive< refs/heads/bar\n-\tremote: pre-receive< refs/heads/baz\n-\tremote: pre-receive< refs/for/next/topic2\n-\tremote: pre-receive< refs/for/next/topic1\n-\tremote: pre-receive< refs/heads/foo\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: pre-receive< refs/for/next/topic3\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/for/next/topic2\n-\tremote: proc-receive< refs/for/next/topic1\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive< refs/for/next/topic3\n-\tremote: proc-receive> ok refs/for/next/topic2\n-\tremote: proc-receive> ng refs/for/next/topic1 fail to call Web API\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option refname refs/for/main/topic\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option new-oid \n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/heads/main\n-\tremote: post-receive< refs/heads/bar\n-\tremote: post-receive< refs/heads/baz\n-\tremote: post-receive< refs/for/next/topic2\n-\tremote: post-receive< refs/heads/foo\n-\tremote: post-receive< refs/for/main/topic\n-\tTo \n-\t :refs/heads/main ..\n-\t* HEAD:refs/heads/bar [new branch]\n-\t* HEAD:refs/heads/baz [new branch]\n-\t* HEAD:refs/for/next/topic2 [new reference]\n-\t* HEAD:refs/heads/foo [new branch]\n-\t HEAD:refs/for/main/topic ..\n-\t! HEAD:refs/for/next/topic1 [remote rejected] (fail to call Web API)\n-\t! HEAD:refs/for/next/topic3 [remote rejected] (proc-receive failed to report status)\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/heads/main Z\n+\t> remote: pre-receive< refs/heads/bar Z\n+\t> remote: pre-receive< refs/heads/baz Z\n+\t> remote: pre-receive< refs/for/next/topic2 Z\n+\t> remote: pre-receive< refs/for/next/topic1 Z\n+\t> remote: pre-receive< refs/heads/foo Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: pre-receive< refs/for/next/topic3 Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/for/next/topic2 Z\n+\t> remote: proc-receive< refs/for/next/topic1 Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive< refs/for/next/topic3 Z\n+\t> remote: proc-receive> ok refs/for/next/topic2 Z\n+\t> remote: proc-receive> ng refs/for/next/topic1 fail to call Web API Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/for/main/topic Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/heads/main Z\n+\t> remote: post-receive< refs/heads/bar Z\n+\t> remote: post-receive< refs/heads/baz Z\n+\t> remote: post-receive< refs/for/next/topic2 Z\n+\t> remote: post-receive< refs/heads/foo Z\n+\t> remote: post-receive< refs/for/main/topic Z\n+\t> To \n+\t> \t:refs/heads/main\t..\n+\t> *\tHEAD:refs/heads/bar\t[new branch]\n+\t> *\tHEAD:refs/heads/baz\t[new branch]\n+\t> *\tHEAD:refs/for/next/topic2\t[new reference]\n+\t> *\tHEAD:refs/heads/foo\t[new branch]\n+\t> \tHEAD:refs/for/main/topic\t..\n+\t> !\tHEAD:refs/for/next/topic1\t[remote rejected] (fail to call Web API)\n+\t> !\tHEAD:refs/for/next/topic3\t[remote rejected] (proc-receive failed to report status)\n+\t> Done\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0040-process-all-refs.sh b/t/t5411/test-0040-process-all-refs.sh\nindex 33a7f49a50..2f405adefa 100644\n--- a/t/t5411/test-0040-process-all-refs.sh\n+++ b/t/t5411/test-0040-process-all-refs.sh\n@@ -50,46 +50,46 @@ test_expect_success \"proc-receive: process all refs ($PROTOCOL)\" '\n \t\tHEAD:refs/for/next/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/heads/bar\n-\tremote: pre-receive< refs/heads/foo\n-\tremote: pre-receive< refs/heads/main\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: pre-receive< refs/for/next/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/heads/bar\n-\tremote: proc-receive< refs/heads/foo\n-\tremote: proc-receive< refs/heads/main\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive< refs/for/next/topic\n-\tremote: proc-receive> ok refs/heads/main\n-\tremote: proc-receive> option fall-through\n-\tremote: proc-receive> ok refs/heads/foo\n-\tremote: proc-receive> option fall-through\n-\tremote: proc-receive> ok refs/heads/bar\n-\tremote: proc-receive> option fall-through\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option refname refs/pull/123/head\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option new-oid \n-\tremote: proc-receive> ok refs/for/next/topic\n-\tremote: proc-receive> option refname refs/pull/124/head\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option new-oid \n-\tremote: proc-receive> option forced-update\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/heads/bar\n-\tremote: post-receive< refs/heads/foo\n-\tremote: post-receive< refs/heads/main\n-\tremote: post-receive< refs/pull/123/head\n-\tremote: post-receive< refs/pull/124/head\n-\tTo \n-\t .. -> bar\n-\t - [deleted] foo\n-\t + ... HEAD -> main (forced update)\n-\t .. HEAD -> refs/pull/123/head\n-\t + ... HEAD -> refs/pull/124/head (forced update)\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/heads/bar Z\n+\t> remote: pre-receive< refs/heads/foo Z\n+\t> remote: pre-receive< refs/heads/main Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: pre-receive< refs/for/next/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/heads/bar Z\n+\t> remote: proc-receive< refs/heads/foo Z\n+\t> remote: proc-receive< refs/heads/main Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive< refs/for/next/topic Z\n+\t> remote: proc-receive> ok refs/heads/main Z\n+\t> remote: proc-receive> option fall-through Z\n+\t> remote: proc-receive> ok refs/heads/foo Z\n+\t> remote: proc-receive> option fall-through Z\n+\t> remote: proc-receive> ok refs/heads/bar Z\n+\t> remote: proc-receive> option fall-through Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/pull/123/head Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: proc-receive> ok refs/for/next/topic Z\n+\t> remote: proc-receive> option refname refs/pull/124/head Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: proc-receive> option forced-update Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/heads/bar Z\n+\t> remote: post-receive< refs/heads/foo Z\n+\t> remote: post-receive< refs/heads/main Z\n+\t> remote: post-receive< refs/pull/123/head Z\n+\t> remote: post-receive< refs/pull/124/head Z\n+\t> To \n+\t> .. -> bar\n+\t> - [deleted] foo\n+\t> + ... HEAD -> main (forced update)\n+\t> .. HEAD -> refs/pull/123/head\n+\t> + ... HEAD -> refs/pull/124/head (forced update)\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0041-process-all-refs--porcelain.sh b/t/t5411/test-0041-process-all-refs--porcelain.sh\nindex 07dce47a7d..c88405792e 100644\n--- a/t/t5411/test-0041-process-all-refs--porcelain.sh\n+++ b/t/t5411/test-0041-process-all-refs--porcelain.sh\n@@ -50,47 +50,47 @@ test_expect_success \"proc-receive: process all refs ($PROTOCOL/porcelain)\" '\n \t\tHEAD:refs/for/next/topic \\\n \t\t>out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/heads/bar\n-\tremote: pre-receive< refs/heads/foo\n-\tremote: pre-receive< refs/heads/main\n-\tremote: pre-receive< refs/for/main/topic\n-\tremote: pre-receive< refs/for/next/topic\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/heads/bar\n-\tremote: proc-receive< refs/heads/foo\n-\tremote: proc-receive< refs/heads/main\n-\tremote: proc-receive< refs/for/main/topic\n-\tremote: proc-receive< refs/for/next/topic\n-\tremote: proc-receive> ok refs/heads/main\n-\tremote: proc-receive> option fall-through\n-\tremote: proc-receive> ok refs/heads/foo\n-\tremote: proc-receive> option fall-through\n-\tremote: proc-receive> ok refs/heads/bar\n-\tremote: proc-receive> option fall-through\n-\tremote: proc-receive> ok refs/for/main/topic\n-\tremote: proc-receive> option refname refs/pull/123/head\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option new-oid \n-\tremote: proc-receive> ok refs/for/next/topic\n-\tremote: proc-receive> option refname refs/pull/124/head\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option new-oid \n-\tremote: proc-receive> option forced-update\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/heads/bar\n-\tremote: post-receive< refs/heads/foo\n-\tremote: post-receive< refs/heads/main\n-\tremote: post-receive< refs/pull/123/head\n-\tremote: post-receive< refs/pull/124/head\n-\tTo \n-\t :refs/heads/bar ..\n-\t- :refs/heads/foo [deleted]\n-\t+ HEAD:refs/heads/main ... (forced update)\n-\t HEAD:refs/pull/123/head ..\n-\t+ HEAD:refs/pull/124/head ... (forced update)\n-\tDone\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/heads/bar Z\n+\t> remote: pre-receive< refs/heads/foo Z\n+\t> remote: pre-receive< refs/heads/main Z\n+\t> remote: pre-receive< refs/for/main/topic Z\n+\t> remote: pre-receive< refs/for/next/topic Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/heads/bar Z\n+\t> remote: proc-receive< refs/heads/foo Z\n+\t> remote: proc-receive< refs/heads/main Z\n+\t> remote: proc-receive< refs/for/main/topic Z\n+\t> remote: proc-receive< refs/for/next/topic Z\n+\t> remote: proc-receive> ok refs/heads/main Z\n+\t> remote: proc-receive> option fall-through Z\n+\t> remote: proc-receive> ok refs/heads/foo Z\n+\t> remote: proc-receive> option fall-through Z\n+\t> remote: proc-receive> ok refs/heads/bar Z\n+\t> remote: proc-receive> option fall-through Z\n+\t> remote: proc-receive> ok refs/for/main/topic Z\n+\t> remote: proc-receive> option refname refs/pull/123/head Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: proc-receive> ok refs/for/next/topic Z\n+\t> remote: proc-receive> option refname refs/pull/124/head Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: proc-receive> option forced-update Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/heads/bar Z\n+\t> remote: post-receive< refs/heads/foo Z\n+\t> remote: post-receive< refs/heads/main Z\n+\t> remote: post-receive< refs/pull/123/head Z\n+\t> remote: post-receive< refs/pull/124/head Z\n+\t> To \n+\t> \t:refs/heads/bar\t..\n+\t> -\t:refs/heads/foo\t[deleted]\n+\t> +\tHEAD:refs/heads/main\t... (forced update)\n+\t> \tHEAD:refs/pull/123/head\t..\n+\t> +\tHEAD:refs/pull/124/head\t... (forced update)\n+\t> Done\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5411/test-0050-proc-receive-refs-with-modifiers.sh b/t/t5411/test-0050-proc-receive-refs-with-modifiers.sh\nindex 906d75e62d..31989f0185 100644\n--- a/t/t5411/test-0050-proc-receive-refs-with-modifiers.sh\n+++ b/t/t5411/test-0050-proc-receive-refs-with-modifiers.sh\n@@ -29,25 +29,25 @@ test_expect_success \"proc-receive: update branch and new tag ($PROTOCOL)\" '\n \t\t$B:refs/heads/main \\\n \t\tv123 >out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/heads/main\n-\tremote: pre-receive< refs/tags/v123\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/heads/main\n-\tremote: proc-receive< refs/tags/v123\n-\tremote: proc-receive> ok refs/heads/main\n-\tremote: proc-receive> option refname refs/pull/123/head\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option new-oid \n-\tremote: proc-receive> ok refs/tags/v123\n-\tremote: proc-receive> option refname refs/pull/124/head\n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/pull/123/head\n-\tremote: post-receive< refs/pull/124/head\n-\tTo \n-\t .. -> refs/pull/123/head\n-\t * [new reference] v123 -> refs/pull/124/head\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/heads/main Z\n+\t> remote: pre-receive< refs/tags/v123 Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/heads/main Z\n+\t> remote: proc-receive< refs/tags/v123 Z\n+\t> remote: proc-receive> ok refs/heads/main Z\n+\t> remote: proc-receive> option refname refs/pull/123/head Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: proc-receive> ok refs/tags/v123 Z\n+\t> remote: proc-receive> option refname refs/pull/124/head Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/pull/123/head Z\n+\t> remote: post-receive< refs/pull/124/head Z\n+\t> To \n+\t> .. -> refs/pull/123/head\n+\t> * [new reference] v123 -> refs/pull/124/head\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -93,32 +93,32 @@ test_expect_success \"proc-receive: create/delete branch, and delete tag ($PROTOC\n \t\t$A:refs/heads/next \\\n \t\t:refs/tags/v123 >out 2>&1 &&\n \tmake_user_friendly_and_stable_output actual &&\n-\tcat >expect <<-EOF &&\n-\tremote: # pre-receive hook\n-\tremote: pre-receive< refs/heads/main\n-\tremote: pre-receive< refs/heads/topic\n-\tremote: pre-receive< refs/tags/v123\n-\tremote: pre-receive< refs/heads/next\n-\tremote: # proc-receive hook\n-\tremote: proc-receive< refs/heads/main\n-\tremote: proc-receive< refs/heads/next\n-\tremote: proc-receive> ok refs/heads/main\n-\tremote: proc-receive> option refname refs/pull/123/head\n-\tremote: proc-receive> option old-oid \n-\tremote: proc-receive> option new-oid \n-\tremote: proc-receive> ok refs/heads/next\n-\tremote: proc-receive> option refname refs/pull/124/head\n-\tremote: proc-receive> option new-oid \n-\tremote: # post-receive hook\n-\tremote: post-receive< refs/pull/123/head\n-\tremote: post-receive< refs/heads/topic\n-\tremote: post-receive< refs/tags/v123\n-\tremote: post-receive< refs/pull/124/head\n-\tTo \n-\t - [deleted] refs/pull/123/head\n-\t .. -> topic\n-\t - [deleted] v123\n-\t * [new reference] -> refs/pull/124/head\n+\tformat_and_save_expect <<-EOF &&\n+\t> remote: # pre-receive hook Z\n+\t> remote: pre-receive< refs/heads/main Z\n+\t> remote: pre-receive< refs/heads/topic Z\n+\t> remote: pre-receive< refs/tags/v123 Z\n+\t> remote: pre-receive< refs/heads/next Z\n+\t> remote: # proc-receive hook Z\n+\t> remote: proc-receive< refs/heads/main Z\n+\t> remote: proc-receive< refs/heads/next Z\n+\t> remote: proc-receive> ok refs/heads/main Z\n+\t> remote: proc-receive> option refname refs/pull/123/head Z\n+\t> remote: proc-receive> option old-oid Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: proc-receive> ok refs/heads/next Z\n+\t> remote: proc-receive> option refname refs/pull/124/head Z\n+\t> remote: proc-receive> option new-oid Z\n+\t> remote: # post-receive hook Z\n+\t> remote: post-receive< refs/pull/123/head Z\n+\t> remote: post-receive< refs/heads/topic Z\n+\t> remote: post-receive< refs/tags/v123 Z\n+\t> remote: post-receive< refs/pull/124/head Z\n+\t> To \n+\t> - [deleted] refs/pull/123/head\n+\t> .. -> topic\n+\t> - [deleted] v123\n+\t> * [new reference] -> refs/pull/124/head\n \tEOF\n \ttest_cmp expect actual &&\n \ndiff --git a/t/t5548-push-porcelain.sh b/t/t5548-push-porcelain.sh\nindex 335abe85a7..f11ff57e54 100755\n--- a/t/t5548-push-porcelain.sh\n+++ b/t/t5548-push-porcelain.sh\n@@ -44,15 +44,16 @@ get_abbrev_oid () {\n # of the output.\n make_user_friendly_and_stable_output () {\n \tsed \\\n-\t\t-e \"s/ *\\$//\" \\\n-\t\t-e \"s/ */ /g\" \\\n-\t\t-e \"s/\t/ /g\" \\\n \t\t-e \"s/$(get_abbrev_oid $A)[0-9a-f]*//g\" \\\n \t\t-e \"s/$(get_abbrev_oid $B)[0-9a-f]*//g\" \\\n \t\t-e \"s/$ZERO_OID//g\" \\\n \t\t-e \"s#To $URL_PREFIX/upstream.git#To #\"\n }\n \n+format_and_save_expect () {\n+\tsed -e 's/^> //' -e 's/Z$//' >expect\n+}\n+\n setup_upstream_and_workbench () {\n \t# Upstream after setup : main(B) foo(A) bar(A) baz(A)\n \t# Workbench after setup : main(A)\n@@ -108,14 +109,14 @@ run_git_push_porcelain_output_test() {\n \t\t\t\tnext\n \t\t) >out &&\n \t\tmake_user_friendly_and_stable_output actual &&\n-\t\tcat >expect <<-EOF &&\n-\t\tTo \n-\t\t= refs/heads/baz:refs/heads/baz [up to date]\n-\t\t :refs/heads/bar ..\n-\t\t- :refs/heads/foo [deleted]\n-\t\t+ refs/heads/main:refs/heads/main ... (forced update)\n-\t\t* refs/heads/next:refs/heads/next [new branch]\n-\t\tDone\n+\t\tformat_and_save_expect <<-EOF &&\n+\t\t> To \n+\t\t> =\trefs/heads/baz:refs/heads/baz\t[up to date]\n+\t\t> \t:refs/heads/bar\t..\n+\t\t> -\t:refs/heads/foo\t[deleted]\n+\t\t> +\trefs/heads/main:refs/heads/main\t... (forced update)\n+\t\t> *\trefs/heads/next:refs/heads/next\t[new branch]\n+\t\t> Done\n \t\tEOF\n \t\ttest_cmp expect actual &&\n \n@@ -145,12 +146,12 @@ run_git_push_porcelain_output_test() {\n \t\t\t\tnext\n \t\t) >out &&\n \t\tmake_user_friendly_and_stable_output actual &&\n-\t\tcat >expect <<-EOF &&\n+\t\tformat_and_save_expect <<-EOF &&\n \t\tTo \n-\t\t= refs/heads/next:refs/heads/next [up to date]\n-\t\t! refs/heads/bar:refs/heads/bar [rejected] (non-fast-forward)\n-\t\t! (delete):refs/heads/baz [rejected] (atomic push failed)\n-\t\t! refs/heads/main:refs/heads/main [rejected] (atomic push failed)\n+\t\t> =\trefs/heads/next:refs/heads/next\t[up to date]\n+\t\t> !\trefs/heads/bar:refs/heads/bar\t[rejected] (non-fast-forward)\n+\t\t> !\t(delete):refs/heads/baz\t[rejected] (atomic push failed)\n+\t\t> !\trefs/heads/main:refs/heads/main\t[rejected] (atomic push failed)\n \t\tDone\n \t\tEOF\n \t\ttest_cmp expect actual &&\n@@ -165,6 +166,7 @@ run_git_push_porcelain_output_test() {\n \t\tEOF\n \t\ttest_cmp expect actual\n \t'\n+\n \ttest_expect_success \"prepare pre-receive hook ($PROTOCOL)\" '\n \t\twrite_script \"$upstream/hooks/pre-receive\" <<-EOF\n \t\texit 1\n@@ -186,12 +188,12 @@ run_git_push_porcelain_output_test() {\n \t\t\t\tnext\n \t\t) >out &&\n \t\tmake_user_friendly_and_stable_output actual &&\n-\t\tcat >expect <<-EOF &&\n+\t\tformat_and_save_expect <<-EOF &&\n \t\tTo \n-\t\t= refs/heads/next:refs/heads/next [up to date]\n-\t\t! refs/heads/bar:refs/heads/bar [remote rejected] (pre-receive hook declined)\n-\t\t! :refs/heads/baz [remote rejected] (pre-receive hook declined)\n-\t\t! refs/heads/main:refs/heads/main [remote rejected] (pre-receive hook declined)\n+\t\t> =\trefs/heads/next:refs/heads/next\t[up to date]\n+\t\t> !\trefs/heads/bar:refs/heads/bar\t[remote rejected] (pre-receive hook declined)\n+\t\t> !\t:refs/heads/baz\t[remote rejected] (pre-receive hook declined)\n+\t\t> !\trefs/heads/main:refs/heads/main\t[remote rejected] (pre-receive hook declined)\n \t\tDone\n \t\tEOF\n \t\ttest_cmp expect actual &&\n@@ -224,12 +226,12 @@ run_git_push_porcelain_output_test() {\n \t\t\t\tnext\n \t\t) >out &&\n \t\tmake_user_friendly_and_stable_output actual &&\n-\t\tcat >expect <<-EOF &&\n+\t\tformat_and_save_expect <<-EOF &&\n \t\tTo \n-\t\t= refs/heads/next:refs/heads/next [up to date]\n-\t\t- :refs/heads/baz [deleted]\n-\t\t refs/heads/main:refs/heads/main ..\n-\t\t! refs/heads/bar:refs/heads/bar [rejected] (non-fast-forward)\n+\t\t> =\trefs/heads/next:refs/heads/next\t[up to date]\n+\t\t> -\t:refs/heads/baz\t[deleted]\n+\t\t> \trefs/heads/main:refs/heads/main\t..\n+\t\t> !\trefs/heads/bar:refs/heads/bar\t[rejected] (non-fast-forward)\n \t\tDone\n \t\tEOF\n \t\ttest_cmp expect actual &&\ndiff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh\nindex 3140ca4fdc..b13e8a52a9 100755\n--- a/t/t6020-bundle-misc.sh\n+++ b/t/t6020-bundle-misc.sh\n@@ -94,7 +94,7 @@ get_abbrev_oid () {\n \n # Format the output of git commands to make a user-friendly and stable\n # text. We can easily prepare the expect text without having to worry\n-# about future changes of the commit ID and spaces of the output.\n+# about future changes of the commit ID.\n make_user_friendly_and_stable_output () {\n \tsed \\\n \t\t-e \"s/$(get_abbrev_oid $A)[0-9a-f]*//g\" \\\n@@ -115,8 +115,11 @@ make_user_friendly_and_stable_output () {\n \t\t-e \"s/$(get_abbrev_oid $P)[0-9a-f]*//g\" \\\n \t\t-e \"s/$(get_abbrev_oid $TAG1)[0-9a-f]*//g\" \\\n \t\t-e \"s/$(get_abbrev_oid $TAG2)[0-9a-f]*//g\" \\\n-\t\t-e \"s/$(get_abbrev_oid $TAG3)[0-9a-f]*//g\" \\\n-\t\t-e \"s/ *\\$//\"\n+\t\t-e \"s/$(get_abbrev_oid $TAG3)[0-9a-f]*//g\"\n+}\n+\n+format_and_save_expect () {\n+\tsed -e 's/Z$//' >expect\n }\n \n # (C) (D, pull/1/head, topic/1)\n@@ -191,11 +194,11 @@ test_expect_success 'create bundle from special rev: main^!' '\n \n \tgit bundle verify special-rev.bdl |\n \t\tmake_user_friendly_and_stable_output >actual &&\n-\tcat >expect <<-\\EOF &&\n+\tformat_and_save_expect <<-\\EOF &&\n \tThe bundle contains this ref:\n \t refs/heads/main\n \tThe bundle requires this ref:\n-\t\n+\t Z\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -212,12 +215,12 @@ test_expect_success 'create bundle with --max-count option' '\n \n \tgit bundle verify max-count.bdl |\n \t\tmake_user_friendly_and_stable_output >actual &&\n-\tcat >expect <<-\\EOF &&\n+\tformat_and_save_expect <<-\\EOF &&\n \tThe bundle contains these 2 refs:\n \t refs/heads/main\n \t refs/tags/v1\n \tThe bundle requires this ref:\n-\t\n+\t Z\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -237,7 +240,7 @@ test_expect_success 'create bundle with --since option' '\n \n \tgit bundle verify since.bdl |\n \t\tmake_user_friendly_and_stable_output >actual &&\n-\tcat >expect <<-\\EOF &&\n+\tformat_and_save_expect <<-\\EOF &&\n \tThe bundle contains these 5 refs:\n \t refs/heads/main\n \t refs/heads/release\n@@ -245,8 +248,8 @@ test_expect_success 'create bundle with --since option' '\n \t refs/tags/v3\n \t HEAD\n \tThe bundle requires these 2 refs:\n-\t\n-\t\n+\t Z\n+\t Z\n \tEOF\n \ttest_cmp expect actual &&\n \n@@ -305,13 +308,13 @@ test_expect_success 'create bundle 2 - has prerequisites' '\n \t\t--stdin \\\n \t\trelease expect <<-\\EOF &&\n+\tformat_and_save_expect <<-\\EOF &&\n \tThe bundle contains this ref:\n \t refs/heads/release\n \tThe bundle requires these 3 refs:\n-\t\n-\t\n-\t\n+\t Z\n+\t Z\n+\t Z\n \tEOF\n \n \tgit bundle verify 2.bdl |\n@@ -329,11 +332,11 @@ test_expect_success 'create bundle 2 - has prerequisites' '\n test_expect_success 'fail to verify bundle without prerequisites' '\n \tgit init --bare test1.git &&\n \n-\tcat >expect <<-\\EOF &&\n+\tformat_and_save_expect <<-\\EOF &&\n \terror: Repository lacks these prerequisite commits:\n-\terror: \n-\terror: \n-\terror: \n+\terror: Z\n+\terror: Z\n+\terror: Z\n \tEOF\n \n \ttest_must_fail git -C test1.git bundle verify ../2.bdl 2>&1 |\n@@ -364,13 +367,13 @@ test_expect_success 'create bundle 3 - two refs, same object' '\n \t\t--stdin \\\n \t\tmain HEAD expect <<-\\EOF &&\n+\tformat_and_save_expect <<-\\EOF &&\n \tThe bundle contains these 2 refs:\n \t refs/heads/main\n \t HEAD\n \tThe bundle requires these 2 refs:\n-\t\n-\t\n+\t Z\n+\t Z\n \tEOF\n \n \tgit bundle verify 3.bdl |\n-- \n2.32.0.rc0.27.g7b1e85181b\n\n"},{"id":"427226","messageId":"87eed619w3.fsf@evledraar.gmail.com","threadId":"54923","inReplyTo":"20210612050711.4057-4-worldhello.net@gmail.com","subject":"Re: [PATCH v2 3/4] sideband: append suffix for message whose CR in next pktline","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-06-13T07:47:22Z","receivedAt":"2021-06-13T07:50:09Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Sat, Jun 12 2021, Jiang Xin wrote:\n\n> From: Jiang Xin \n>\n> When calling \"demultiplex_sideband\" on a sideband-2 message, will try to\n> split the message by line breaks, and append a suffix to each nonempty\n> line to clear the end of the screen line. But in the following example,\n> there will be no suffix (8 spaces) for \"\":\n>\n> PKT-LINE(\\2 CR CR )\n> PKT-LINE(\\2 CR CR CR)\n>\n> This is because the line break of \"\" is placed in the next\n> pktline message.\n>\n> Without this fix, t5411 must remove trailing spaces of the actual output\n> of \"git-push\" command before comparing.\n\nNice, i.e. let's generally fix the output instead.\n\n> Signed-off-by: Jiang Xin \n> ---\n> sideband.c | 4 ++++\n> 1 file changed, 4 insertions(+)\n>\n> diff --git a/sideband.c b/sideband.c\n> index 6f9e026732..abf2be98e1 100644\n> --- a/sideband.c\n> +++ b/sideband.c\n> @@ -185,6 +185,10 @@ int demultiplex_sideband(const char *me, int status,\n> \n> \t\t\tif (!scratch->len)\n> \t\t\t\tstrbuf_addstr(scratch, DISPLAY_PREFIX);\n> +\t\t\telse if (!linelen)\n> +\t\t\t\t/* buf has a leading CR which ends the remaining\n> +\t\t\t\t * scratch of last round of \"demultiplex_sideband\" */\n> +\t\t\t\tstrbuf_addstr(scratch, suffix);\n> \t\t\tif (linelen > 0) {\n\nI haven't thought about this carefully but isn't there some way to\ncombine these if/else if/if statementsn that's clearer?\n\nI.e. here we're doing an \"if\" check for a !lineline and then an \"if\"\nthat can't be true if !linelen.\n\nIsn't this the same as:\n\nif (!scratch->len) {\n ...\n} else {\n if (!linelen)\n ...\n else if (linelen > 0)\n ...\n}\n\nOr are there cases where we take that \"linelen > 0\" arm if\n!scratch->len?\n\n> \t\t\t\tmaybe_colorize_sideband(scratch, b, linelen);\n> \t\t\t\tstrbuf_addstr(scratch, suffix);\n\n"},{"id":"427263","messageId":"xmqqim2hyuj1.fsf@gitster.g","threadId":"54923","inReplyTo":"20210612050711.4057-4-worldhello.net@gmail.com","subject":"Re: [PATCH v2 3/4] sideband: append suffix for message whose CR in next pktline","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-06-14T03:50:10Z","receivedAt":"2021-06-14T03:50:18Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jiang Xin writes:\n\n> From: Jiang Xin \n>\n> When calling \"demultiplex_sideband\" on a sideband-2 message, will try to\n> split the message by line breaks, and append a suffix to each nonempty\n> line to clear the end of the screen line.\n\nSubject of \"will try\" and \"append\" is missing. Do you mean that\nthe helper function in question does these two things? I.e.\n\n\tdemultiplex_sideband() used on a sideband #2 will try\n\tto... and appends ...\n\n> But in the following example,\n> there will be no suffix (8 spaces) for \"\":\n>\n> PKT-LINE(\\2 CR CR )\n> PKT-LINE(\\2 CR CR CR)\n\nThat description may mechanically correct, but\n\n after , we fail to clear to the end of line\n\nmay make it easier to understand what the problem we are trying to\nsolve for those who do not remember what these suffix games are\nabout.\n\n> This is because the line break of \"\" is placed in the next\n> pktline message.\n>\n> Without this fix, t5411 must remove trailing spaces of the actual output\n> of \"git-push\" command before comparing.\n>\n> Signed-off-by: Jiang Xin \n> ---\n> sideband.c | 4 ++++\n> 1 file changed, 4 insertions(+)\n>\n> diff --git a/sideband.c b/sideband.c\n> index 6f9e026732..abf2be98e1 100644\n> --- a/sideband.c\n> +++ b/sideband.c\n> @@ -185,6 +185,10 @@ int demultiplex_sideband(const char *me, int status,\n> \n> \t\t\tif (!scratch->len)\n> \t\t\t\tstrbuf_addstr(scratch, DISPLAY_PREFIX);\n> +\t\t\telse if (!linelen)\n> +\t\t\t\t/* buf has a leading CR which ends the remaining\n> +\t\t\t\t * scratch of last round of \"demultiplex_sideband\" */\n> +\t\t\t\tstrbuf_addstr(scratch, suffix);\n\nThe style of multi-line comment needs fixing, but the contents of\nthe comment is a bit hard to grok.\n\n> \t\t\tif (linelen > 0) {\n> \t\t\t\tmaybe_colorize_sideband(scratch, b, linelen);\n> \t\t\t\tstrbuf_addstr(scratch, suffix);\n\nI wonder if the following is simpler to read, though.\n\n-- >8 --\nSubject: [PATCH] sideband: don't lose clear-to-eol at packet boundary\n\nWhen demultiplex_sideband() sees a CR or LF on the sideband #2, it\nadds \"suffix\" string to clear to the end of the current line, which\nhelps when relaying a progress display whose records are terminated\nwith CRs.\n\nThe code however forgot that depending on the length of the payload\nline, such a CR may fall exactly at the packet boundary and the\nnumber of bytes before the CR from the beginning of the packet could\nbe zero. In such a case, the message that was terminated by the CR\nwere leftover in the \"scratch\" buffer in the previous call to the\nfunction and we still need to clear to the end of the current line.\n\nJust remove the unnecessary check on linelen; maybe_colorize_sideband()\non 0-byte payload turns into a no-op, and we should be adding clear-to-eol\nfor each and every CR/LF anyway.\n\n sideband.c | 7 +++----\n 1 file changed, 3 insertions(+), 4 deletions(-)\n\ndiff --git c/sideband.c w/sideband.c\nindex 6f9e026732..1575bf16dd 100644\n--- c/sideband.c\n+++ w/sideband.c\n@@ -185,10 +185,9 @@ int demultiplex_sideband(const char *me, int status,\n \n \t\t\tif (!scratch->len)\n \t\t\t\tstrbuf_addstr(scratch, DISPLAY_PREFIX);\n-\t\t\tif (linelen > 0) {\n-\t\t\t\tmaybe_colorize_sideband(scratch, b, linelen);\n-\t\t\t\tstrbuf_addstr(scratch, suffix);\n-\t\t\t}\n+\n+\t\t\tmaybe_colorize_sideband(scratch, b, linelen);\n+\t\t\tstrbuf_addstr(scratch, suffix);\n \n \t\t\tstrbuf_addch(scratch, *brk);\n \t\t\txwrite(2, scratch->buf, scratch->len);\n"},{"id":"427264","messageId":"xmqqa6ntytks.fsf@gitster.g","threadId":"54923","inReplyTo":"20210612050711.4057-1-worldhello.net@gmail.com","subject":"Re: [PATCH v2 0/4] Fixed t6020 bash compatible issue and fixed wrong sideband suffix issue","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-06-14T04:10:43Z","receivedAt":"2021-06-14T04:11:34Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jiang Xin writes:\n\n> From: Jiang Xin \n>\n> In addition to fix the bash incompatible issue of t6020, find another\n> issue when try to rewrite t5411 to compare raw command output.\n\nDo the three later patches depend on the t6020 fix, or is this made\na 4-patch series only for the convenience of sending them out?\n\nIt's not like get_abbrev_oid() used in t6020 is defined in a common\npart of the test library and later used by other tests (instead, the\npatches duplicate this helper function into yet two more files).\n\n"},{"id":"427327","messageId":"CANYiYbGtfgZepnfTWGjbmOh2bxa8tZ7bvgtVTo6qTQpCP9MPag@mail.gmail.com","threadId":"54923","inReplyTo":"xmqqim2hyuj1.fsf@gitster.g","subject":"Re: [PATCH v2 3/4] sideband: append suffix for message whose CR in next pktline","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-06-14T11:51:23Z","receivedAt":"2021-06-14T11:53:16Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"Junio C Hamano 于2021年6月14日周一 上午11:50写道：\n>\n> Jiang Xin writes:\n>\n> > From: Jiang Xin \n> >\n> > When calling \"demultiplex_sideband\" on a sideband-2 message, will try to\n> > split the message by line breaks, and append a suffix to each nonempty\n> > line to clear the end of the screen line.\n>\n> Subject of \"will try\" and \"append\" is missing. Do you mean that\n> the helper function in question does these two things? I.e.\n>\n> demultiplex_sideband() used on a sideband #2 will try\n> to... and appends ...\n>\n> > But in the following example,\n> > there will be no suffix (8 spaces) for \"\":\n> >\n> > PKT-LINE(\\2 CR CR )\n> > PKT-LINE(\\2 CR CR CR)\n>\n> That description may mechanically correct, but\n>\n> after , we fail to clear to the end of line\n>\n> may make it easier to understand what the problem we are trying to\n> solve for those who do not remember what these suffix games are\n> about.\n>\n> > This is because the line break of \"\" is placed in the next\n> > pktline message.\n> >\n> > Without this fix, t5411 must remove trailing spaces of the actual output\n> > of \"git-push\" command before comparing.\n> >\n> > Signed-off-by: Jiang Xin \n> > ---\n> > sideband.c | 4 ++++\n> > 1 file changed, 4 insertions(+)\n> >\n> > diff --git a/sideband.c b/sideband.c\n> > index 6f9e026732..abf2be98e1 100644\n> > --- a/sideband.c\n> > +++ b/sideband.c\n> > @@ -185,6 +185,10 @@ int demultiplex_sideband(const char *me, int status,\n> >\n> > if (!scratch->len)\n> > strbuf_addstr(scratch, DISPLAY_PREFIX);\n> > + else if (!linelen)\n> > + /* buf has a leading CR which ends the remaining\n> > + * scratch of last round of \"demultiplex_sideband\" */\n> > + strbuf_addstr(scratch, suffix);\n>\n> The style of multi-line comment needs fixing, but the contents of\n> the comment is a bit hard to grok.\n>\n> > if (linelen > 0) {\n> > maybe_colorize_sideband(scratch, b, linelen);\n> > strbuf_addstr(scratch, suffix);\n>\n> I wonder if the following is simpler to read, though.\n>\n> -- >8 --\n> Subject: [PATCH] sideband: don't lose clear-to-eol at packet boundary\n>\n> When demultiplex_sideband() sees a CR or LF on the sideband #2, it\n> adds \"suffix\" string to clear to the end of the current line, which\n> helps when relaying a progress display whose records are terminated\n> with CRs.\n>\n> The code however forgot that depending on the length of the payload\n> line, such a CR may fall exactly at the packet boundary and the\n> number of bytes before the CR from the beginning of the packet could\n> be zero. In such a case, the message that was terminated by the CR\n> were leftover in the \"scratch\" buffer in the previous call to the\n> function and we still need to clear to the end of the current line.\n>\n> Just remove the unnecessary check on linelen; maybe_colorize_sideband()\n> on 0-byte payload turns into a no-op, and we should be adding clear-to-eol\n> for each and every CR/LF anyway.\n>\n> sideband.c | 7 +++----\n> 1 file changed, 3 insertions(+), 4 deletions(-)\n>\n> diff --git c/sideband.c w/sideband.c\n> index 6f9e026732..1575bf16dd 100644\n> --- c/sideband.c\n> +++ w/sideband.c\n> @@ -185,10 +185,9 @@ int demultiplex_sideband(const char *me, int status,\n>\n> if (!scratch->len)\n> strbuf_addstr(scratch, DISPLAY_PREFIX);\n> - if (linelen > 0) {\n> - maybe_colorize_sideband(scratch, b, linelen);\n> - strbuf_addstr(scratch, suffix);\n> - }\n> +\n> + maybe_colorize_sideband(scratch, b, linelen);\n> + strbuf_addstr(scratch, suffix);\n>\n> strbuf_addch(scratch, *brk);\n> xwrite(2, scratch->buf, scratch->len);\n\nThe above changes will add suffix to the end of each line, and even an\nempty lines. However, according to the comment in commit ebe8fa738d\n(fix display overlap between remote and local progress, 2007-11-04)\nwhich introduced the suffix implementation for the first time, no\nsuffix should be appended for empty lines.\n\n /*\n * Let's insert a suffix to clear the end\n * of the screen line, but only if current\n * line data actually contains something.\n */\n\nSo my implementation is to try not to break the original\nimplementation, and keep the linelen unchanged.\n\nThe strbuf \"scratch\" will be reset at line 18th in the while block, so\nthe nonempty scratch at line 7 indicates the parameter scratch of\ndemultiplex_sideband() is not empty. With the following patch,\nadditional suffix is only added before a leading CR in a packet which\nis seperated with its message by packet boundary.\n\n```\n01 while ((brk = strpbrk(b, \"\\n\\r\"))) {\n02 int linelen = brk - b;\n03\n04 + /* Has no empty scratch from last call of \"demultiplex_sideband\"\n05 + * and has a leading CR in buf.\n06 + */\n07 + if (scratch->len && !linelen)\n08 + strbuf_addstr(scratch, suffix);\n09 if (!scratch->len)\n10 strbuf_addstr(scratch, DISPLAY_PREFIX);\n11 if (linelen > 0) {\n12 maybe_colorize_sideband(scratch, b, linelen);\n13 strbuf_addstr(scratch, suffix);\n14 }\n15\n16 strbuf_addch(scratch, *brk);\n17 xwrite(2, scratch->buf, scratch->len);\n18 strbuf_reset(scratch);\n19\n20 b = brk + 1;\n21 }\n```\n"},{"id":"427393","messageId":"0pqq6s9-n238-95ss-nsqr-42o75sr933r@syhkavp.arg","threadId":"54923","inReplyTo":"CANYiYbG6bKwheThWz2ecHUTh7JFmTsi-F9FtYkcoS0az=f=m6Q@mail.gmail.com","subject":"Re: [PATCH v2 3/4] sideband: append suffix for message whose CR in next pktline","fromName":"Nicolas Pitre","fromEmail":"nico@fluxnic.net","sentAt":"2021-06-15T02:11:37Z","receivedAt":"2021-06-15T02:29:14Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Tue, 15 Jun 2021, Jiang Xin wrote:\n\n> Junio C Hamano 于2021年6月15日周二 上午9:17写道：\n> >\n> > Jiang Xin writes:\n> >\n> > > /*\n> > > * Let's insert a suffix to clear the end\n> > > * of the screen line, but only if current\n> > > * line data actually contains something.\n> > > */\n> > >\n> > > So my implementation is to try not to break the original\n> > > implementation, and keep the linelen unchanged.\n> >\n> > I knew what you wanted to do from your code---I am questioning if\n> > that \"only when something is there\" was really sensible, or if it\n> > was just attracting bugs.\n> >\n> \n> @Nicolas, what's your opinion? Is it ok to add clear-to-eol suffix to\n> each line even empty ones?\n\nThat would be the simplest thing to do.\n\nBut there must have been a reason for doing it otherwise. I just don't \nremember anymore.\n\nMaybe it had to do with progress reporting that does a bunch of \npercentage updates followed by '\\r' to remain on the same line, and at \nthe end a single '\\n' to move to the next line without erasing the final \nstatus report line. That would be a case for not clearing empty lines.\n\n\nNicolas"},{"id":"427396","messageId":"xmqqk0mwylhc.fsf@gitster.g","threadId":"54923","inReplyTo":"CANYiYbGtfgZepnfTWGjbmOh2bxa8tZ7bvgtVTo6qTQpCP9MPag@mail.gmail.com","subject":"Re: [PATCH v2 3/4] sideband: append suffix for message whose CR in next pktline","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-06-15T01:17:51Z","receivedAt":"2021-06-15T02:34:14Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jiang Xin writes:\n\n> /*\n> * Let's insert a suffix to clear the end\n> * of the screen line, but only if current\n> * line data actually contains something.\n> */\n>\n> So my implementation is to try not to break the original\n> implementation, and keep the linelen unchanged.\n\nI knew what you wanted to do from your code---I am questioning if\nthat \"only when something is there\" was really sensible, or if it\nwas just attracting bugs.\n\nThanks.\n\n"},{"id":"427399","messageId":"CANYiYbG6bKwheThWz2ecHUTh7JFmTsi-F9FtYkcoS0az=f=m6Q@mail.gmail.com","threadId":"54923","inReplyTo":"xmqqk0mwylhc.fsf@gitster.g","subject":"Re: [PATCH v2 3/4] sideband: append suffix for message whose CR in next pktline","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-06-15T01:47:39Z","receivedAt":"2021-06-15T02:57:21Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"Junio C Hamano 于2021年6月15日周二 上午9:17写道：\n>\n> Jiang Xin writes:\n>\n> > /*\n> > * Let's insert a suffix to clear the end\n> > * of the screen line, but only if current\n> > * line data actually contains something.\n> > */\n> >\n> > So my implementation is to try not to break the original\n> > implementation, and keep the linelen unchanged.\n>\n> I knew what you wanted to do from your code---I am questioning if\n> that \"only when something is there\" was really sensible, or if it\n> was just attracting bugs.\n>\n\n@Nicolas, what's your opinion? Is it ok to add clear-to-eol suffix to\neach line even empty ones?\n\n--\nJiang Xin\n"},{"id":"427401","messageId":"CANYiYbFsqj9j_UT9-mF+BSOAHU4=On+BUxCU4w6rXhhMaeo=zg@mail.gmail.com","threadId":"54923","inReplyTo":"0pqq6s9-n238-95ss-nsqr-42o75sr933r@syhkavp.arg","subject":"Re: [PATCH v2 3/4] sideband: append suffix for message whose CR in next pktline","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-06-15T03:04:55Z","receivedAt":"2021-06-15T03:05:10Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"Nicolas Pitre 于2021年6月15日周二 上午10:11写道：\n>\n> On Tue, 15 Jun 2021, Jiang Xin wrote:\n>\n> > Junio C Hamano 于2021年6月15日周二 上午9:17写道：\n> > >\n> > > Jiang Xin writes:\n> > >\n> > > > /*\n> > > > * Let's insert a suffix to clear the end\n> > > > * of the screen line, but only if current\n> > > > * line data actually contains something.\n> > > > */\n> > > >\n> > > > So my implementation is to try not to break the original\n> > > > implementation, and keep the linelen unchanged.\n> > >\n> > > I knew what you wanted to do from your code---I am questioning if\n> > > that \"only when something is there\" was really sensible, or if it\n> > > was just attracting bugs.\n> > >\n> >\n> > @Nicolas, what's your opinion? Is it ok to add clear-to-eol suffix to\n> > each line even empty ones?\n>\n> That would be the simplest thing to do.\n>\n> But there must have been a reason for doing it otherwise. I just don't\n> remember anymore.\n>\n> Maybe it had to do with progress reporting that does a bunch of\n> percentage updates followed by '\\r' to remain on the same line, and at\n> the end a single '\\n' to move to the next line without erasing the final\n> status report line. That would be a case for not clearing empty lines.\n>\n\nThank @Nicolas for helping me understand the story behinds the code.\n\nIf there are two sideband #2 packets like this:\n\n PKTLINE(\\2 \"\" CR \"\" CR)\n PKTLINE(\\2 \"\" LF \"\" LF)\n\nWe should append clear-to-eol suffix to \"\", \"\"\nand \"\" to erase the last message displayed on the same\nline. Even though there is no need to add the clear-to-eol suffix to\n\"\", always adding suffix before line breaks (CR or LF) of\nnonempty message make it simple to program.\n\nIf there are empty messages in sideband #2 packets like this:\n\n PKTLINE(\\2 \"\" CR LF \"\" LF)\n PKTLINE(\\2 \"\" LF)\n\nFor the empty message between \"\" and \"\",\nnothing to display and no need to add clear-to-eol suffix.\n\nThe issue this patch try to fix is like the following example:\n\n PKTLINE(\\2 \"\" CR \"\")\n PKTLINE(\\2 CR \"\" LF)\n\nThe message \"\" is displayed without a proper clear-to-eol\nsuffix, because it's eol (CR) is in another pktline.\n\nSince we can distinguished this case by checking the size of\n\"scratch\", IMHO, it better not add suffix before all line breaks.\n\n--\nJiang Xin\n"},{"id":"427404","messageId":"CANYiYbHA+obZPPNw3Oc0h5BbtdVyqpxOK-u3dxDB-sQbdcY0yQ@mail.gmail.com","threadId":"54923","inReplyTo":"xmqqa6ntytks.fsf@gitster.g","subject":"Re: [PATCH v2 0/4] Fixed t6020 bash compatible issue and fixed wrong sideband suffix issue","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-06-15T03:11:39Z","receivedAt":"2021-06-15T03:11:52Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"Junio C Hamano 于2021年6月14日周一 下午12:10写道：\n>\n> Jiang Xin writes:\n>\n> > From: Jiang Xin \n> >\n> > In addition to fix the bash incompatible issue of t6020, find another\n> > issue when try to rewrite t5411 to compare raw command output.\n>\n> Do the three later patches depend on the t6020 fix, or is this made\n> a 4-patch series only for the convenience of sending them out?\n>\n> It's not like get_abbrev_oid() used in t6020 is defined in a common\n> part of the test library and later used by other tests (instead, the\n> patches duplicate this helper function into yet two more files).\n>\n\nWill split it into two patch series. One will fix bash incompatible\nparameter expansion in t6020, another will fix clear-to-eol at packet\nboundary issue of sideband and try to test raw output in t6020, t5548\nand t5411.\n\nI also queue another patch series, which add \"--bare\" support to\n\"test_create_repo\", and replace \"git init\" command in test cases to\nadapt to variable GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME.\n\n--\nJiang Xin\n"},{"id":"427406","messageId":"r8ss67nn-31s4-o429-828p-s787sr2481@syhkavp.arg","threadId":"54923","inReplyTo":"CANYiYbFsqj9j_UT9-mF+BSOAHU4=On+BUxCU4w6rXhhMaeo=zg@mail.gmail.com","subject":"Re: [PATCH v2 3/4] sideband: append suffix for message whose CR in next pktline","fromName":"Nicolas Pitre","fromEmail":"nico@fluxnic.net","sentAt":"2021-06-15T03:26:35Z","receivedAt":"2021-06-15T03:26:44Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Tue, 15 Jun 2021, Jiang Xin wrote:\n\n> The issue this patch try to fix is like the following example:\n> \n> PKTLINE(\\2 \"\" CR \"\")\n> PKTLINE(\\2 CR \"\" LF)\n> \n> The message \"\" is displayed without a proper clear-to-eol\n> suffix, because it's eol (CR) is in another pktline.\n\nI'd fix this issue with the following logic:\n\nbool pending_clear_to_eol;\n\nmy_putchar(c) {\n\tswitch (c) {\n\tcase '\\r':\n\tcase '\\n':\n\t\tpending_clear_to_eol = true;\n\t\tbreak;\n\tdefault:\n\t\tif (pending_clear_to_eol) {\n\t\t\tclear_to_eol();\n\t\t\tpending_clear_to_eol = false;\n\t\t}\n\t\tbreak;\n\t}\n\tputchar(c);\n}\n\nIn other words, you clear the line after printing \"remote:\" but only if \nthere is a non \\n or \\r coming next.\n\n\nNicolas\n"},{"id":"427412","messageId":"xmqqh7hzwx9f.fsf@gitster.g","threadId":"54923","inReplyTo":"r8ss67nn-31s4-o429-828p-s787sr2481@syhkavp.arg","subject":"Re: [PATCH v2 3/4] sideband: append suffix for message whose CR in next pktline","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-06-15T04:46:20Z","receivedAt":"2021-06-15T04:46:27Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Nicolas Pitre writes:\n\n> On Tue, 15 Jun 2021, Jiang Xin wrote:\n>\n>> The issue this patch try to fix is like the following example:\n>> \n>> PKTLINE(\\2 \"\" CR \"\")\n>> PKTLINE(\\2 CR \"\" LF)\n>> \n>> The message \"\" is displayed without a proper clear-to-eol\n>> suffix, because it's eol (CR) is in another pktline.\n>\n> I'd fix this issue with the following logic:\n>\n> bool pending_clear_to_eol;\n>\n> my_putchar(c) {\n> \tswitch (c) {\n> \tcase '\\r':\n> \tcase '\\n':\n> \t\tpending_clear_to_eol = true;\n> \t\tbreak;\n> \tdefault:\n> \t\tif (pending_clear_to_eol) {\n> \t\t\tclear_to_eol();\n> \t\t\tpending_clear_to_eol = false;\n> \t\t}\n> \t\tbreak;\n> \t}\n> \tputchar(c);\n> }\n>\n> In other words, you clear the line after printing \"remote:\" but only if \n> there is a non \\n or \\r coming next.\n\nWhat puzzles me the most in this discussion is why we do this for\nLF. I do understand why we need it for CR---the line we are going\nto show message on after emitting CR would be full of leftover\nletters we previously have written before emitting CR, so we'd show\nthe message (to overwrite the initial part enough to show our own\nmessage) and then clear to the end with either ANSI sequence of\nsufficient number of whitespaces. But line feed would take us to a\nfresh and blank line---there is nothing to clear, no?\n\nThanks.\n"},{"id":"427420","messageId":"CANYiYbFw3sOj8xbrc6evHu0onw3VT1TMnfEeo=BKtS-qu4Hy+A@mail.gmail.com","threadId":"54923","inReplyTo":"xmqqh7hzwx9f.fsf@gitster.g","subject":"Re: [PATCH v2 3/4] sideband: append suffix for message whose CR in next pktline","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-06-15T07:17:13Z","receivedAt":"2021-06-15T07:17:29Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"Junio C Hamano 于2021年6月15日周二 下午12:46写道：\n>\n> Nicolas Pitre writes:\n>\n> > On Tue, 15 Jun 2021, Jiang Xin wrote:\n> >\n> >> The issue this patch try to fix is like the following example:\n> >>\n> >> PKTLINE(\\2 \"\" CR \"\")\n> >> PKTLINE(\\2 CR \"\" LF)\n> >>\n> >> The message \"\" is displayed without a proper clear-to-eol\n> >> suffix, because it's eol (CR) is in another pktline.\n> >\n> > I'd fix this issue with the following logic:\n> >\n> > bool pending_clear_to_eol;\n> >\n> > my_putchar(c) {\n> > switch (c) {\n> > case '\\r':\n> > case '\\n':\n> > pending_clear_to_eol = true;\n> > break;\n> > default:\n> > if (pending_clear_to_eol) {\n> > clear_to_eol();\n> > pending_clear_to_eol = false;\n> > }\n> > break;\n> > }\n> > putchar(c);\n> > }\n> >\n> > In other words, you clear the line after printing \"remote:\" but only if\n> > there is a non \\n or \\r coming next.\n>\n> What puzzles me the most in this discussion is why we do this for\n> LF. I do understand why we need it for CR---the line we are going\n> to show message on after emitting CR would be full of leftover\n> letters we previously have written before emitting CR, so we'd show\n> the message (to overwrite the initial part enough to show our own\n> message) and then clear to the end with either ANSI sequence of\n> sufficient number of whitespaces. But line feed would take us to a\n> fresh and blank line---there is nothing to clear, no?\n\nI guess this may because sideband #2 messages are printed on the\nscreen in a background process, it never know a line where it starts\nto print has characters on the right. So it is safe to write an\nadditional clear-to-eol suffix no matter the message ends with CR or\nLF.\n\n--\nJiang Xin\n"},{"id":"427481","messageId":"5098143-7947-3pr7-31o7-9396s4no82s@syhkavp.arg","threadId":"54923","inReplyTo":"xmqqh7hzwx9f.fsf@gitster.g","subject":"Re: [PATCH v2 3/4] sideband: append suffix for message whose CR in next pktline","fromName":"Nicolas Pitre","fromEmail":"nico@fluxnic.net","sentAt":"2021-06-15T14:46:10Z","receivedAt":"2021-06-15T14:46:14Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Tue, 15 Jun 2021, Junio C Hamano wrote:\n\n> Nicolas Pitre writes:\n> \n> > On Tue, 15 Jun 2021, Jiang Xin wrote:\n> >\n> >> The issue this patch try to fix is like the following example:\n> >> \n> >> PKTLINE(\\2 \"\" CR \"\")\n> >> PKTLINE(\\2 CR \"\" LF)\n> >> \n> >> The message \"\" is displayed without a proper clear-to-eol\n> >> suffix, because it's eol (CR) is in another pktline.\n> >\n> > I'd fix this issue with the following logic:\n> >\n> > bool pending_clear_to_eol;\n> >\n> > my_putchar(c) {\n> > \tswitch (c) {\n> > \tcase '\\r':\n> > \tcase '\\n':\n> > \t\tpending_clear_to_eol = true;\n> > \t\tbreak;\n> > \tdefault:\n> > \t\tif (pending_clear_to_eol) {\n> > \t\t\tclear_to_eol();\n> > \t\t\tpending_clear_to_eol = false;\n> > \t\t}\n> > \t\tbreak;\n> > \t}\n> > \tputchar(c);\n> > }\n> >\n> > In other words, you clear the line after printing \"remote:\" but only if \n> > there is a non \\n or \\r coming next.\n> \n> What puzzles me the most in this discussion is why we do this for\n> LF. I do understand why we need it for CR---the line we are going\n> to show message on after emitting CR would be full of leftover\n> letters we previously have written before emitting CR, so we'd show\n> the message (to overwrite the initial part enough to show our own\n> message) and then clear to the end with either ANSI sequence of\n> sufficient number of whitespaces. But line feed would take us to a\n> fresh and blank line---there is nothing to clear, no?\n\nDepends. Suppose the local process is doing the progress report with CR.\n\nThen the remote sends a single line with LF.\n\nYou expects the remote line to be displayed over the local progress \nreport and the local progress report to be resumed on the following \nline. Without the line clearing you might have leftover garbage on the \nremote message line.\n\n\nNicolas\n"},{"id":"427654","messageId":"20210617031411.80684-1-zhiyou.jx@alibaba-inc.com","threadId":"54923","inReplyTo":"CANYiYbHA+obZPPNw3Oc0h5BbtdVyqpxOK-u3dxDB-sQbdcY0yQ@mail.gmail.com","subject":"[PATCH v3] t6020: fix incompatible parameter expansion","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2021-06-17T03:14:11Z","receivedAt":"2021-06-17T03:14:31Z","isPatch":true,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"Ævar reported that the function `make_user_friendly_and_stable_output()`\nfailed on a i386 box (gcc45) in the gcc farm boxes with error:\n\n sed: couldn't re-allocate memory\n\nIt turns out that older versions of bash (4.3) or dash (0.5.7) cannot\nevaluate expression like `${A%${A#???????}}` used to get the leading 7\ncharacters of variable A.\n\nReplace the incompatible parameter expansion so that t6020 works on\nolder version of bash or dash.\n\nReported-by: Ævar Arnfjörð Bjarmason \nSigned-off-by: Jiang Xin \n---\n t/t6020-bundle-misc.sh | 50 ++++++++++++++++++++++++++----------------\n 1 file changed, 31 insertions(+), 19 deletions(-)\n\ndiff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh\nindex 881f72fd44..3140ca4fdc 100755\n--- a/t/t6020-bundle-misc.sh\n+++ b/t/t6020-bundle-misc.sh\n@@ -80,30 +80,42 @@ test_commit_setvar () {\n \teval $var=$oid\n }\n \n+get_abbrev_oid () {\n+\toid=$1 &&\n+\tsuffix=${oid#???????} &&\n+\toid=${oid%$suffix} &&\n+\tif test -n \"$oid\"\n+\tthen\n+\t\techo \"$oid\"\n+\telse\n+\t\techo \"undefined-oid\"\n+\tfi\n+}\n+\n # Format the output of git commands to make a user-friendly and stable\n # text. We can easily prepare the expect text without having to worry\n # about future changes of the commit ID and spaces of the output.\n make_user_friendly_and_stable_output () {\n \tsed \\\n-\t\t-e \"s/${A%${A#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${B%${B#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${C%${C#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${D%${D#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${E%${E#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${F%${F#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${G%${G#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${H%${H#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${I%${I#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${J%${J#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${K%${K#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${L%${L#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${M%${M#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${N%${N#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${O%${O#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${P%${P#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${TAG1%${TAG1#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${TAG2%${TAG2#???????}}[0-9a-f]*//g\" \\\n-\t\t-e \"s/${TAG3%${TAG3#???????}}[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $A)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $B)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $C)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $D)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $E)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $F)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $G)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $H)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $I)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $J)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $K)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $L)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $M)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $N)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $O)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $P)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $TAG1)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $TAG2)[0-9a-f]*//g\" \\\n+\t\t-e \"s/$(get_abbrev_oid $TAG3)[0-9a-f]*//g\" \\\n \t\t-e \"s/ *\\$//\"\n }\n \n-- \n2.32.0.rc0.27.g7b1e85181b\n\n"},{"id":"428025","messageId":"87v96736ph.fsf@evledraar.gmail.com","threadId":"54923","inReplyTo":"20210617031411.80684-1-zhiyou.jx@alibaba-inc.com","subject":"Re: [PATCH v3] t6020: fix incompatible parameter expansion","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-06-21T08:41:36Z","receivedAt":"2021-06-21T09:31:01Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Jun 17 2021, Jiang Xin wrote:\n\n> Ævar reported that the function `make_user_friendly_and_stable_output()`\n> failed on a i386 box (gcc45) in the gcc farm boxes with error:\n>\n> sed: couldn't re-allocate memory\n>\n> It turns out that older versions of bash (4.3) or dash (0.5.7) cannot\n> evaluate expression like `${A%${A#???????}}` used to get the leading 7\n> characters of variable A.\n>\n> Replace the incompatible parameter expansion so that t6020 works on\n> older version of bash or dash.\n>\n> Reported-by: Ævar Arnfjörð Bjarmason \n> Signed-off-by: Jiang Xin \n> ---\n\nFor what it's worth I've also tested this v3 on gcc45, it works too.\n"}]}