{"thread":{"id":"64870","subject":"[PATCH V1][RFC] t/perf/p3400: speed up setup using fast-import","startedAt":"2026-01-26T16:56:28Z","lastAt":"2026-01-30T17:10:09Z","messageCount":14,"participants":["Tian Yuchen","Johannes Sixt","Phillip Wood","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"534674","messageId":"20260126165618.596944-1-a3205153416@gmail.com","threadId":"64870","inReplyTo":null,"subject":"[PATCH V1][RFC] t/perf/p3400: speed up setup using fast-import","fromName":"Tian Yuchen","fromEmail":"a3205153416@gmail.com","sentAt":"2026-01-26T16:56:18Z","receivedAt":"2026-01-26T16:56:28Z","isPatch":true,"sender":{"key":"cat@malon.dev","avatar":"https://avatars.githubusercontent.com/u/232002048?v=4"},"body":"The setup phase in 't/perf/p3400-rebase.sh' generates 100 commits to\nsimulate a noisy history. It currently uses a shell loop that invokes\n'git add', 'git commit', 'test_seq', and 'sort' in each iteration.\nThis makes the whole process slow.\n\nOptimize the setup by using 'git fast-import' to generate the commit\nhistory in a single stream. Additionally, pre-compute the forward and\nreversed file contents to avoid repetitive execution of 'seq' and 'sort'.\n\nPerformance enhancement:\n Before: 29.045s\n After: 18.081s\n\nMeasured on Lenovo Yoga 2020, Ubuntu 24.04.\n\nSigned-off-by: Tian Yuchen \n---\n t/perf/p3400-rebase.sh | 53 +++++++++++++++++++++++++++++-------------\n 1 file changed, 37 insertions(+), 16 deletions(-)\n\ndiff --git a/t/perf/p3400-rebase.sh b/t/perf/p3400-rebase.sh\nindex e6b0277729..9251a09b4f 100755\n--- a/t/perf/p3400-rebase.sh\n+++ b/t/perf/p3400-rebase.sh\n@@ -9,25 +9,46 @@ test_expect_success 'setup rebasing on top of a lot of changes' '\n \tgit checkout -f -B base &&\n \tgit checkout -B to-rebase &&\n \tgit checkout -B upstream &&\n-\tfor i in $(test_seq 100)\n-\tdo\n-\t\t# simulate huge diffs\n-\t\techo change$i >unrelated-file$i &&\n-\t\ttest_seq 1000 >>unrelated-file$i &&\n-\t\tgit add unrelated-file$i &&\n-\t\ttest_tick &&\n-\t\tgit commit -m commit$i unrelated-file$i &&\n-\t\techo change$i >unrelated-file$i &&\n-\t\ttest_seq 1000 | sort -nr >>unrelated-file$i &&\n-\t\tgit add unrelated-file$i &&\n-\t\ttest_tick &&\n-\t\tgit commit -m commit$i-reverse unrelated-file$i ||\n-\t\treturn 1\n-\tdone &&\n+\n+\ttest_seq 1000 >content_fwd &&\n+\ttest_seq 1000 | sort -nr >content_rev &&\n+\n+\t(\n+\t\tfor i in $(test_seq 100)\n+\t\tdo\n+\t\t\techo \"commit refs/heads/upstream\" &&\n+\t\t\techo \"committer WGYDY $i +0000\" &&\n+\t\t\techo \"data < $i +0000\" &&\n+\t\t\techo \"data < writes：\n>\n> I noticed that while the setup phase is much faster (as shown in the commit\n> message), the actual rebase performance test regressed slightly by about 3\n> seconds (from ~2s to ~5s). I don't quite understand what led to this outcome.\n>\n> Additionally, I haven't delved deeply into the technical details of\n> `git fast-import`,\n> and I'm not entirely sure if my approach is reasonable and compliant with the\n> specifications.\n>\n> I would greatly appreciate any guidance from those knowledgeable in these\n> area!\n>\n> Regards,\n> Yuchen\n\nI highly suspect that git fast-import generate the packfile that is NOT delta\ncompressed. According to git fast-import documentation:\n\n>...For this reason it is strongly recommended that users repack the repository with\n>'git repack -a -d' after fast-import completes, allowing Git to reorganize the packfiles\n>for faster data access...\n\nCould this be the reason why it's even slower than handling “loose\nobjects”? I'm working\non it currently.\n\nRegards,\nYuchen\n"},{"id":"534767","messageId":"CA+rU_o6+Q89_xJyhuuO20kbpmEttKW8oTf4XFGM=R7V4ZygawQ@mail.gmail.com","threadId":"64870","inReplyTo":"CA+rU_o7dZB=OCJHk9nGSbs4RLXmE9A3VUopa+6vF6oX0E9vxuQ@mail.gmail.com","subject":"Re: [PATCH V1][RFC] t/perf/p3400: speed up setup using fast-import","fromName":"Tian Yuchen","fromEmail":"a3205153416@gmail.com","sentAt":"2026-01-28T15:25:41Z","receivedAt":"2026-01-28T15:25:55Z","isPatch":true,"sender":{"key":"cat@malon.dev","avatar":"https://avatars.githubusercontent.com/u/232002048?v=4"},"body":">...For this reason it is strongly recommended that users repack the repository with\n>'git repack -a -d' after fast-import completes, allowing Git to reorganize the packfiles\n>for faster data access...\n\nAfter adding 'git repack -a -d', the time does changed:\n original(no change) patch v1 NEW\n real 29.845 18.081 22.231\nrebase 13.34 14.94 12.78\n\nIt seems that the 'git repack -a -d' line does reduce runtime.\nHowever, the total duration\nhas seen a noticeable performance regression. I'm still trying to figure it out.\n\nWith regards,\nYuchen\n"},{"id":"534769","messageId":"20260128160717.611391-1-a3205153416@gmail.com","threadId":"64870","inReplyTo":"20260126165618.596944-1-a3205153416@gmail.com","subject":"[PATCH v2] t/perf/p3400: speed up setup using fast-import","fromName":"Tian Yuchen","fromEmail":"a3205153416@gmail.com","sentAt":"2026-01-28T16:07:17Z","receivedAt":"2026-01-28T16:07:37Z","isPatch":true,"sender":{"key":"cat@malon.dev","avatar":"https://avatars.githubusercontent.com/u/232002048?v=4"},"body":"The setup phase in 't/perf/p3400-rebase.sh' generates 100 commits to\nsimulate a noisy history. It currently uses a shell loop that invokes\n'git add', 'git commit', 'test_seq', and 'sort' in each iteration.\nThis incurs significant overhead due to repeated process spawning.\n\nOptimize the setup by using 'git fast-import' to generate the commit\nhistory in a single stream. Additionally, pre-compute the forward and\nreversed file contents to avoid repetitive execution of 'seq' and 'sort'.\n\nTo ensure the test measures rebase performance against a consistent\nobject layout (rather than the suboptimal pack/loose objects created\nby the raw import), perform a full repack (`git repack -a -d`) at the\nend of the setup.\n\nThis reduces the setup time significantly while maintaining the validity\nof the subsequent performance tests.\n\nPerformance enhancement:\n Real Rebase\n Before: 29.045s 13.34s\n After: 22.231s 12.78s\n\nMeasured on Lenovo Yoga 2020, Ubuntu 24.04.\n\nSigned-off-by: Tian Yuchen \n---\n\nChanges since v1:\n- Added 'git repack -a -d' at the end of the setup phase.\n- This fixes a performance regression observed in the subsequent rebase\n test, ensuring the object layout is normalized before testing.\n- Updated setup timing in the commit message to reflect the repack overhead.\n\n t/perf/p3400-rebase.sh | 54 +++++++++++++++++++++++++++++-------------\n 1 file changed, 38 insertions(+), 16 deletions(-)\n\ndiff --git a/t/perf/p3400-rebase.sh b/t/perf/p3400-rebase.sh\nindex e6b0277729..9f4251aed6 100755\n--- a/t/perf/p3400-rebase.sh\n+++ b/t/perf/p3400-rebase.sh\n@@ -9,25 +9,47 @@ test_expect_success 'setup rebasing on top of a lot of changes' '\n \tgit checkout -f -B base &&\n \tgit checkout -B to-rebase &&\n \tgit checkout -B upstream &&\n-\tfor i in $(test_seq 100)\n-\tdo\n-\t\t# simulate huge diffs\n-\t\techo change$i >unrelated-file$i &&\n-\t\ttest_seq 1000 >>unrelated-file$i &&\n-\t\tgit add unrelated-file$i &&\n-\t\ttest_tick &&\n-\t\tgit commit -m commit$i unrelated-file$i &&\n-\t\techo change$i >unrelated-file$i &&\n-\t\ttest_seq 1000 | sort -nr >>unrelated-file$i &&\n-\t\tgit add unrelated-file$i &&\n-\t\ttest_tick &&\n-\t\tgit commit -m commit$i-reverse unrelated-file$i ||\n-\t\treturn 1\n-\tdone &&\n+\n+\ttest_seq 1000 >content_fwd &&\n+\ttest_seq 1000 | sort -nr >content_rev &&\n+\n+\t(\n+\t\tfor i in $(test_seq 100)\n+\t\tdo\n+\t\t\techo \"commit refs/heads/upstream\" &&\n+\t\t\techo \"committer WGYDY $i +0000\" &&\n+\t\t\techo \"data < $i +0000\" &&\n+\t\t\techo \"data < +\ttest_seq 1000 >content_fwd &&\n> +\ttest_seq 1000 | sort -nr >content_rev &&\n\nRemove another pipe with\n\n\tsort -nr content_fwd >content_rev &&\n\n> +\n> +\t(\n> +\t\tfor i in $(test_seq 100)\n> +\t\tdo\n> +\t\t\t...\n> +\t\t\tcat content_rev &&\n> +\t\t\techo \"EOF\" || return 1\n\nThis is an incorrect use of `return` outside a function. This must be\n`|| exit 1` (or just `|| exit`) to actually break out of the loop.\n\nOf course, the error code is ignored, because it is in the upstream of a\npipe, so in order to fail the complete command, it would be necessary\nthat the original error left an incorrect fast-import stream such that\nthe down-stream `git fast-import` fails. Otherwise, the entire command\ncan complete successfully in an unexpected way.\n\nMaybe instead of a pipe, dump the stream into a temporary file, so that\nthe loop's exit code can be captured?\n\n> +\t\tdone\n> +\t) | git fast-import &&\n-- Hannes\n\n"},{"id":"534858","messageId":"edc47d68-7e34-49af-b719-cef3076f6eeb@kdbg.org","threadId":"64870","inReplyTo":"04005f67-3ec5-4fa7-9fd7-37ef9034286a@kdbg.org","subject":"Re: [PATCH v2] t/perf/p3400: speed up setup using fast-import","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2026-01-30T09:55:39Z","receivedAt":"2026-01-30T09:55:43Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 30.01.26 um 07:41 schrieb Johannes Sixt:\n> Am 28.01.26 um 17:07 schrieb Tian Yuchen:\n>> +\n>> +\t(\n>> +\t\tfor i in $(test_seq 100)\n>> +\t\tdo\n>> +\t\t\t...\n>> +\t\t\tcat content_rev &&\n>> +\t\t\techo \"EOF\" || return 1\n> \n> This is an incorrect use of `return` outside a function. This must be\n> `|| exit 1` (or just `|| exit`) to actually break out of the loop.\n\nI am wrong here, because my earlier tests were too narrow. This use of\n`return` actually does work as intended, because it happens inside a\nfunction. Neither the subshell nor being in the upstream of a pipe\nnegates this fact.\n\nThe rest of the comment still stands, though.\n\n> Of course, the error code is ignored, because it is in the upstream of a\n> pipe, so in order to fail the complete command, it would be necessary\n> that the original error left an incorrect fast-import stream such that\n> the down-stream `git fast-import` fails. Otherwise, the entire command\n> can complete successfully in an unexpected way.\n> \n> Maybe instead of a pipe, dump the stream into a temporary file, so that\n> the loop's exit code can be captured?\n> \n>> +\t\tdone\n>> +\t) | git fast-import &&\n-- Hannes\n\n"},{"id":"534881","messageId":"39b421df-a2cc-4ab4-9aa8-b79c5c172d86@gmail.com","threadId":"64870","inReplyTo":"20260128160717.611391-1-a3205153416@gmail.com","subject":"Re: [PATCH v2] t/perf/p3400: speed up setup using fast-import","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2026-01-30T14:31:58Z","receivedAt":"2026-01-30T14:32:02Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 28/01/2026 16:07, Tian Yuchen wrote:\n> The setup phase in 't/perf/p3400-rebase.sh' generates 100 commits to\n> simulate a noisy history. It currently uses a shell loop that invokes\n> 'git add', 'git commit', 'test_seq', and 'sort' in each iteration.\n> This incurs significant overhead due to repeated process spawning.\n> \n> Optimize the setup by using 'git fast-import' to generate the commit\n> history in a single stream. Additionally, pre-compute the forward and\n> reversed file contents to avoid repetitive execution of 'seq' and 'sort'.\n> \n> To ensure the test measures rebase performance against a consistent\n> object layout (rather than the suboptimal pack/loose objects created\n> by the raw import), perform a full repack (`git repack -a -d`) at the\n> end of the setup.\n> \n> This reduces the setup time significantly while maintaining the validity\n> of the subsequent performance tests.\n> \n> Performance enhancement:\n> Real Rebase\n> Before: 29.045s 13.34s\n> After: 22.231s 12.78s\n\nThat's a nice speedup in the test setup\n\n> diff --git a/t/perf/p3400-rebase.sh b/t/perf/p3400-rebase.sh\n> index e6b0277729..9f4251aed6 100755\n> --- a/t/perf/p3400-rebase.sh\n> +++ b/t/perf/p3400-rebase.sh\n> @@ -9,25 +9,47 @@ test_expect_success 'setup rebasing on top of a lot of changes' '\n> \tgit checkout -f -B base &&\n> \tgit checkout -B to-rebase &&\n> \tgit checkout -B upstream &&\n> -\tfor i in $(test_seq 100)\n> -\tdo\n> -\t\t# simulate huge diffs\n> -\t\techo change$i >unrelated-file$i &&\n> -\t\ttest_seq 1000 >>unrelated-file$i &&\n> -\t\tgit add unrelated-file$i &&\n> -\t\ttest_tick &&\n> -\t\tgit commit -m commit$i unrelated-file$i &&\n> -\t\techo change$i >unrelated-file$i &&\n> -\t\ttest_seq 1000 | sort -nr >>unrelated-file$i &&\n> -\t\tgit add unrelated-file$i &&\n> -\t\ttest_tick &&\n> -\t\tgit commit -m commit$i-reverse unrelated-file$i ||\n> -\t\treturn 1\n> -\tdone &&\n> +\n> +\ttest_seq 1000 >content_fwd &&\n> +\ttest_seq 1000 | sort -nr >content_rev &&\n> +\n> +\t(\n> +\t\tfor i in $(test_seq 100)\n> +\t\tdo\n> +\t\t\techo \"commit refs/heads/upstream\" &&\n> +\t\t\techo \"committer WGYDY $i +0000\" &&\n\nYou can keep the same author and committer as the original with\n\n\ttest_tick &&\n\techo \"author $GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL> $GIT_AUTHOR_DATE\" &&\n\techo \"committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> \n$GIT_COMMITTER_DATE\" &&\n\nhere and below\n\n> +\t\t\techo \"data < +\t\t\techo \"commit$i\" &&\n> +\t\t\techo \"EOF\" &&\n> +\t\t\t\n> +\t\t\tif test \"$i\" = 1; then\n> +\t\t\t\techo \"from refs/heads/upstream^0\"\n> +\t\t\tfi &&\n> +\n> +\t\t\techo \"M 100644 inline unrelated-file$i\" &&\n> +\t\t\techo \"data < +\t\t\techo \"change$i\" &&\n> +\t\t\tcat content_fwd &&\n> +\t\t\techo \"EOF\" &&\n> +\n> +\t\t\techo \"commit refs/heads/upstream\" &&\n> +\t\t\techo \"committer WGYDY $i +0000\" &&\n> +\t\t\techo \"data < +\t\t\techo \"commit$i-reversed\" &&\n\nThe commit message in the original is \"commit$i-reverse\", not \"reversed\"\n\n> +\t\t\techo \"EOF\" &&\n> +\t\t\techo \"M 100644 inline unrelated-file$i\" &&\n> +\t\t\techo \"data < +\t\t\techo \"change$i\" &&\n> +\t\t\tcat content_rev &&\n> +\t\t\techo \"EOF\" || return 1\n> +\t\tdone\n\nAs Johannes pointed out we'll ignore the any failure above. We can \naddress that by adding \"echo done\" here and adding \"--done\" to \"git \nfast-import\" below. That will cause \"git fast-import\" to fail because if \nthere is an error in the loop as the last line of input to fast-import \nwill not be \"done\"\n\nThanks\n\nPhillip\n\n> +\t) | git fast-import &&\n> +\n> +\tgit repack -a -d &&\n> +\tgit checkout -f upstream &&\n> \tgit checkout to-rebase &&\n> \ttest_commit our-patch interesting-file\n> '\n> -\n> test_perf 'rebase on top of a lot of unrelated changes' '\n> \tgit rebase --onto upstream HEAD^ &&\n> \tgit rebase --onto base HEAD^\n\n"},{"id":"534885","messageId":"CA+rU_o79w=LPqRZyM4hWt2U6mWMr36Z92ebennrs+P_Ge9JpVA@mail.gmail.com","threadId":"64870","inReplyTo":"39b421df-a2cc-4ab4-9aa8-b79c5c172d86@gmail.com","subject":"Re: [PATCH v2] t/perf/p3400: speed up setup using fast-import","fromName":"Tian Yuchen","fromEmail":"a3205153416@gmail.com","sentAt":"2026-01-30T15:40:09Z","receivedAt":"2026-01-30T15:40:22Z","isPatch":true,"sender":{"key":"cat@malon.dev","avatar":"https://avatars.githubusercontent.com/u/232002048?v=4"},"body":"Hi Johannes, Hi Phillip,\n\nThanks a lot for the detailed review and the great suggestions!\n\n@Johannes: I will adopt the 'sort -nr content_fwd' optimization to\nsave a process.\n\n@Phillip: Your point about the pipe swallowing the exit code is very\ninsightful. I will\n implement the '--done' protocol to ensure any failure in the loop\ncauses the test to fail.\n\nI will also update the author/committer generation to use test_tick\nand the standard\nenvironment variables, and fix the \"reversed\" typo.\n\nI will prepare and send a v3 patch shortly incorporating these changes.\n\nThanks,\nTian Yuchen\n"},{"id":"534888","messageId":"xmqqcy2r83rx.fsf@gitster.g","threadId":"64870","inReplyTo":"04005f67-3ec5-4fa7-9fd7-37ef9034286a@kdbg.org","subject":"Re: [PATCH v2] t/perf/p3400: speed up setup using fast-import","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-01-30T16:27:14Z","receivedAt":"2026-01-30T16:27:16Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Sixt writes:\n\n> Am 28.01.26 um 17:07 schrieb Tian Yuchen:\n>> +\ttest_seq 1000 >content_fwd &&\n>> +\ttest_seq 1000 | sort -nr >content_rev &&\n>\n> Remove another pipe with\n>\n> \tsort -nr content_fwd >content_rev &&\n\nI agree with the outcome (i.e., we do not run test_seq twice) but\nnot necessarily with the motivation (i.e., we do not assume test_seq\nmay fail and care about its failure code, so no need to avoid pipes\nfor the sake of avoiding pipes).\n\n>> +\t(\n>> +\t\tfor i in $(test_seq 100)\n>> +\t\tdo\n>> +\t\t\t...\n>> +\t\t\tcat content_rev &&\n>> +\t\t\techo \"EOF\" || return 1\n>\n> This is an incorrect use of `return` outside a function. This must be\n> `|| exit 1` (or just `|| exit`) to actually break out of the loop.\n\nGood.\n\n> Of course, the error code is ignored, because it is in the upstream of a\n> pipe, so in order to fail the complete command, it would be necessary\n> that the original error left an incorrect fast-import stream such that\n> the down-stream `git fast-import` fails. Otherwise, the entire command\n> can complete successfully in an unexpected way.\n>\n> Maybe instead of a pipe, dump the stream into a temporary file, so that\n> the loop's exit code can be captured?\n\nI like this suggestion. A temporary file, unless it is excessively\nlarge, has an additional advantage that it allows you to inspect\nafter \"git fast-import\" dies on it.\n\n>> +\t\tdone\n>> +\t) | git fast-import &&\n> -- Hannes\n"},{"id":"534889","messageId":"20260130162927.638672-1-a3205153416@gmail.com","threadId":"64870","inReplyTo":"20260128160717.611391-1-a3205153416@gmail.com","subject":"[PATCH v3] t/perf/p3400: speed up setup using fast-import","fromName":"Tian Yuchen","fromEmail":"a3205153416@gmail.com","sentAt":"2026-01-30T16:29:27Z","receivedAt":"2026-01-30T16:29:34Z","isPatch":true,"sender":{"key":"cat@malon.dev","avatar":"https://avatars.githubusercontent.com/u/232002048?v=4"},"body":"The setup phase in 't/perf/p3400-rebase.sh' generates 100 commits to\nsimulate a noisy history. It currently uses a shell loop that invokes\n'git add', 'git commit', 'test_seq', and 'sort' in each iteration.\nThis incurs significant overhead due to repeated process spawning.\n\nOptimize the setup by using 'git fast-import' to generate the commit\nhistory in a single stream. Additionally, pre-compute the forward and\nreversed file contents to avoid repetitive execution of 'seq' and 'sort'.\n\nTo ensure the test measures rebase performance against a consistent\nobject layout (rather than the suboptimal pack/loose objects created\nby the raw import), perform a full repack (`git repack -a -d`) at the\nend of the setup.\n\nThis reduces the setup time significantly while maintaining the validity\nof the subsequent performance tests.\n\nPerformance enhancement (Average value of 5 tests):\n Real Rebase\n Before: 29.045s 13.34s\n After: 22.431s 12.98s\n\nMeasured on Lenovo Yoga 2020, Ubuntu 24.04.\n\nSigned-off-by: Tian Yuchen \n---\n\nChanges since v2:\n- Optimized `content_rev` generation by sorting `content_fwd` directly (Johannes Sixt).\n- Used `test_tick` and standard `$GIT_COMMITTER_*` variables for consistency (Phillip Wood).\n- Fixed typo \"reversed\" -> \"reverse\" to match original test (Phillip Wood).\n- Added \"done\" command and used `git fast-import --done` to prevent errors in the loop being ignored by the pipe (Phillip Wood).\n\n t/perf/p3400-rebase.sh | 56 ++++++++++++++++++++++++++++++------------\n 1 file changed, 40 insertions(+), 16 deletions(-)\n\ndiff --git a/t/perf/p3400-rebase.sh b/t/perf/p3400-rebase.sh\nindex e6b0277729..6bb58282d6 100755\n--- a/t/perf/p3400-rebase.sh\n+++ b/t/perf/p3400-rebase.sh\n@@ -9,25 +9,49 @@ test_expect_success 'setup rebasing on top of a lot of changes' '\n \tgit checkout -f -B base &&\n \tgit checkout -B to-rebase &&\n \tgit checkout -B upstream &&\n-\tfor i in $(test_seq 100)\n-\tdo\n-\t\t# simulate huge diffs\n-\t\techo change$i >unrelated-file$i &&\n-\t\ttest_seq 1000 >>unrelated-file$i &&\n-\t\tgit add unrelated-file$i &&\n-\t\ttest_tick &&\n-\t\tgit commit -m commit$i unrelated-file$i &&\n-\t\techo change$i >unrelated-file$i &&\n-\t\ttest_seq 1000 | sort -nr >>unrelated-file$i &&\n-\t\tgit add unrelated-file$i &&\n-\t\ttest_tick &&\n-\t\tgit commit -m commit$i-reverse unrelated-file$i ||\n-\t\treturn 1\n-\tdone &&\n+\n+\ttest_seq 1000 >content_fwd &&\n+\tsort -nr content_fwd >content_rev &&\n+\n+\t(\n+\t\tfor i in $(test_seq 100)\n+\t\tdo\n+\t\t\ttest_tick &&\n+\t\t\techo \"commit refs/heads/upstream\" &&\n+\t\t\techo \"committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE\" &&\n+\t\t\techo \"data < $GIT_COMMITTER_DATE\" &&\n+\t\t\techo \"data < I will send V4 shortly.\nIn this case, please also undo the unnecessary removal of a blank line\nat the end of the patch.\n\n-- Hannes\n\n"},{"id":"534897","messageId":"20260130170123.642344-1-a3205153416@gmail.com","threadId":"64870","inReplyTo":"20260130162927.638672-1-a3205153416@gmail.com","subject":"[PATCH v4] t/perf/p3400: speed up setup using fast-import","fromName":"Tian Yuchen","fromEmail":"a3205153416@gmail.com","sentAt":"2026-01-30T17:01:23Z","receivedAt":"2026-01-30T17:01:32Z","isPatch":true,"sender":{"key":"cat@malon.dev","avatar":"https://avatars.githubusercontent.com/u/232002048?v=4"},"body":"The setup phase in 't/perf/p3400-rebase.sh' generates 100 commits to\nsimulate a noisy history. It currently uses a shell loop that invokes\n'git add', 'git commit', 'test_seq', and 'sort' in each iteration.\nThis incurs significant overhead due to repeated process spawning.\n\nOptimize the setup by using 'git fast-import' to generate the commit\nhistory. Additionally, pre-compute the forward and reversed file contents\nto avoid repetitive execution of 'seq' and 'sort'.\n\nTo ensure the test measures rebase performance against a consistent\nobject layout (rather than the suboptimal pack/loose objects created\nby the raw import), perform a full repack (`git repack -a -d`) at the\nend of the setup.\n\nThis reduces the setup time significantly while maintaining the validity\nof the subsequent performance tests.\n\nPerformance enhancement (Average value of 5 tests):\n Real Rebase\n Before: 29.045s 13.34s\n After: 21.989s 12.84s\n\nMeasured on Lenovo Yoga 2020, Ubuntu 24.04.\n\nSigned-off-by: Tian Yuchen \n---\n\nChanges since v3:\n- Refactored the fast-import generation to write to a temporary file \n (`fast_import_stream`) instead of using a pipe. This allows for better \n debugging if fast-import fails, as suggested by Johannes Sixt and \n endorsed by Junio C Hamano.\n- Switched from `return 1` to `exit 1` inside the subshell to correctly \n propagate exit codes.\n- Undid the unnecessary removal of a blank line.\n\n t/perf/p3400-rebase.sh | 53 ++++++++++++++++++++++++++++++------------\n 1 file changed, 38 insertions(+), 15 deletions(-)\n\ndiff --git a/t/perf/p3400-rebase.sh b/t/perf/p3400-rebase.sh\nindex e6b0277729..425f484681 100755\n--- a/t/perf/p3400-rebase.sh\n+++ b/t/perf/p3400-rebase.sh\n@@ -9,21 +9,44 @@ test_expect_success 'setup rebasing on top of a lot of changes' '\n \tgit checkout -f -B base &&\n \tgit checkout -B to-rebase &&\n \tgit checkout -B upstream &&\n-\tfor i in $(test_seq 100)\n-\tdo\n-\t\t# simulate huge diffs\n-\t\techo change$i >unrelated-file$i &&\n-\t\ttest_seq 1000 >>unrelated-file$i &&\n-\t\tgit add unrelated-file$i &&\n-\t\ttest_tick &&\n-\t\tgit commit -m commit$i unrelated-file$i &&\n-\t\techo change$i >unrelated-file$i &&\n-\t\ttest_seq 1000 | sort -nr >>unrelated-file$i &&\n-\t\tgit add unrelated-file$i &&\n-\t\ttest_tick &&\n-\t\tgit commit -m commit$i-reverse unrelated-file$i ||\n-\t\treturn 1\n-\tdone &&\n+\ttest_seq 1000 >content_fwd &&\n+\tsort -nr content_fwd >content_rev &&\n+\t(\n+\t\tfor i in $(test_seq 100)\n+\t\tdo\n+\t\t\ttest_tick &&\n+\t\t\techo \"commit refs/heads/upstream\" &&\n+\t\t\techo \"committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE\" &&\n+\t\t\techo \"data < $GIT_COMMITTER_DATE\" &&\n+\t\t\techo \"data <fast_import_stream &&\n+\n+\tgit fast-import writes:\n\n> Changes since v2:\n> - Optimized `content_rev` generation by sorting `content_fwd` directly (Johannes Sixt).\n> - Used `test_tick` and standard `$GIT_COMMITTER_*` variables for consistency (Phillip Wood).\n> - Fixed typo \"reversed\" -> \"reverse\" to match original test (Phillip Wood).\n> - Added \"done\" command and used `git fast-import --done` to prevent errors in the loop being ignored by the pipe (Phillip Wood).\n\nThanks. Will replace.\n"}]}