{"thread":{"id":"55808","subject":"Slow fetch on repo with tags per each commit (à la Vim)","startedAt":"2021-05-31T06:36:14Z","lastAt":"2021-06-01T03:39:31Z","messageCount":3,"participants":["Bagas Sanjaya","Felipe Contreras"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"425971","messageId":"f9c1115a-2e17-9d3e-02f5-138eab8048d9@gmail.com","threadId":"55808","inReplyTo":null,"subject":"Slow fetch on repo with tags per each commit (à la Vim)","fromName":"Bagas Sanjaya","fromEmail":"bagasdotme@gmail.com","sentAt":"2021-05-31T06:36:07Z","receivedAt":"2021-05-31T06:36:14Z","isPatch":false,"sender":{"key":"bagasdotme@gmail.com","avatar":"https://avatars.githubusercontent.com/u/40219486?v=4"},"body":"Hi,\n\nI have full clone of Vim repository [1] on my computer. This repo has \ntags for each commit, meaning that when there is a new commit landed on \ndefault (master) branch, it will be tagged (something close to release \ntags but not same).\n\nWhile on the other repo, doing git fetch will fetch new commits from the \nremote quickly (the only limit is network bandwidth), on vim repo it \ntakes longer.\n\nExamining the trace (run by `GIT_TRACE=1 git fetch`), I got:\n\n> 13:21:50.429633 git.c:447               trace: built-in: git fetch\n> 13:21:50.470410 run-command.c:667       trace: run_command: GIT_DIR=.git git remote-https origin https://github.com/vim/vim.git\n> 13:21:50.492785 git.c:733               trace: exec: git-remote-https origin https://github.com/vim/vim.git\n> 13:21:50.492849 run-command.c:667       trace: run_command: git-remote-https origin https://github.com/vim/vim.git\n> remote: Enumerating objects: 137, done.\n> remote: Counting objects: 100% (137/137), done.\n> remote: Compressing objects: 100% (20/20), done.\n> 13:23:31.589963 run-command.c:667       trace: run_command: git index-pack --stdin -v --fix-thin '--keep=fetch-pack 10745 on ubuntu' --pack_header=2,137\n> 13:23:31.593505 git.c:447               trace: built-in: git index-pack --stdin -v --fix-thin '--keep=fetch-pack 10745 on ubuntu' --pack_header=2,137\n> remote: Total 137 (delta 116), reused 134 (delta 113), pack-reused 0\n> Receiving objects: 100% (137/137), 117.44 KiB | 871.00 KiB/s, done.\n> Resolving deltas: 100% (116/116), completed with 101 local objects.\n> 13:23:32.682368 run-command.c:667       trace: run_command: git rev-list --objects --stdin --not --all --quiet --alternate-refs\n> 13:23:32.684986 git.c:447               trace: built-in: git rev-list --objects --stdin --not --all --quiet --alternate-refs\n> From https://github.com/vim/vim\n>    0b4964848..2fa9384ca  master     -> origin/master\n>  * [new tag]             v8.2.2914  -> v8.2.2914\n> 13:23:33.239629 run-command.c:667       trace: run_command: GIT_DIR=.git git remote-https origin https://github.com/vim/vim.git\n> 13:23:33.241760 git.c:733               trace: exec: git-remote-https origin https://github.com/vim/vim.git\n> 13:23:33.241815 run-command.c:667       trace: run_command: git-remote-https origin https://github.com/vim/vim.git\n> 13:23:33.251931 run-command.c:667       trace: run_command: git rev-list --objects --stdin --not --all --quiet --alternate-refs\n> 13:23:33.540444 run-command.c:667       trace: run_command: git rev-list --objects --stdin --not --all --quiet --alternate-refs\n> 13:23:33.542831 git.c:447               trace: built-in: git rev-list --objects --stdin --not --all --quiet --alternate-refs\n>  * [new tag]             v8.2.2911  -> v8.2.2911\n>  * [new tag]             v8.2.2912  -> v8.2.2912\n>  * [new tag]             v8.2.2913  -> v8.2.2913\n> 13:23:33.828590 run-command.c:1628      run_processes_parallel: preparing to run up to 1 tasks\n> 13:23:33.828702 run-command.c:1660      run_processes_parallel: done\n> 13:23:33.829024 run-command.c:667       trace: run_command: git maintenance run --auto --no-quiet\n> 13:23:33.831040 git.c:447               trace: built-in: git maintenance run --auto --no-quiet\n> 1.80user 1.89system 1:43.54elapsed 3%CPU (0avgtext+0avgdata 25700maxresident)k\n> 149520inputs+4552outputs (268major+20701minor)pagefaults 0swaps\n\ngit fetch took about 100 seconds to complete, including fetching 3 new \ntags. On the other repo, the same command took around 10-20 seconds, \ndepending on how large data that must be downloaded.\n\nNote that Vim repo has 13952 commits and 12284 tags; and for testing \nabove I use Git 2.31.1.\n\nThanks.\n\n[1]: https://github.com/vim/vim\n\n-- \nAn old man doll... just what I always wanted! - Clara\n"},{"id":"426024","messageId":"60b50829553df_24d2820813@natae.notmuch","threadId":"55808","inReplyTo":"f9c1115a-2e17-9d3e-02f5-138eab8048d9@gmail.com","subject":"RE: Slow fetch on repo with tags per each commit (à la Vim)","fromName":"Felipe Contreras","fromEmail":"felipe.contreras@gmail.com","sentAt":"2021-05-31T16:00:41Z","receivedAt":"2021-05-31T17:24:52Z","isPatch":false,"sender":{"key":"felipe.contreras@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8358?v=4"},"body":"Bagas Sanjaya wrote:\n> I have full clone of Vim repository [1] on my computer. This repo has \n> tags for each commit, meaning that when there is a new commit landed on \n> default (master) branch, it will be tagged (something close to release \n> tags but not same).\n\nI sometimes work on that repo and I just have to say how much of a\npleasure it is to work on a properly maintained repo such as git.git.\n\nMy last bisection on vim.git landed on this beauty of a commit:\n\n    Update runtime files\n\n runtime/autoload/netrw.vim         | 1982 ++++++++++++++++++++++++++++++++++----------------------\n runtime/autoload/netrwSettings.vim |   10 +-\n runtime/doc/eval.txt               |   17 +-\n runtime/doc/filetype.txt           |   12 +-\n runtime/doc/options.txt            |    2 +-\n runtime/doc/pi_netrw.txt           |   94 +--\n runtime/doc/popup.txt              |    8 +-\n runtime/doc/quickfix.txt           |    8 +-\n runtime/doc/quickref.txt           |    3 +-\n runtime/doc/syntax.txt             |    3 +-\n runtime/doc/tags                   |    5 +\n runtime/doc/todo.txt               |   45 +-\n runtime/doc/usr_27.txt             |   14 +-\n runtime/doc/version8.txt           |   24 +-\n runtime/doc/windows.txt            |    4 +-\n runtime/ftplugin/qf.vim            |   10 +-\n runtime/lang/menu_sr_rs.utf-8.vim  |   72 +-\n runtime/optwin.vim                 |    4 +-\n runtime/plugin/netrwPlugin.vim     |   14 +-\n runtime/syntax/lisp.vim            |    8 +-\n runtime/syntax/vim.vim             |   66 +-\n runtime/tutor/README.txt           |    2 +-\n src/po/sr.po                       |  677 ++++++++++---------\n 23 files changed, 1797 insertions(+), 1287 deletions(-)\n\nThree years of development of a plugin in a single commit.\n\nThat's a yikes.\n\nhttps://github.com/vim/vim/commit/85850f3a5\n\n-- \nFelipe Contreras\n"},{"id":"426093","messageId":"11b3e4e2-4b2c-e7d8-01d7-d7249b7a0424@gmail.com","threadId":"55808","inReplyTo":"60b50829553df_24d2820813@natae.notmuch","subject":"Re: Slow fetch on repo with tags per each commit (à la Vim)","fromName":"Bagas Sanjaya","fromEmail":"bagasdotme@gmail.com","sentAt":"2021-06-01T03:39:23Z","receivedAt":"2021-06-01T03:39:31Z","isPatch":false,"sender":{"key":"bagasdotme@gmail.com","avatar":"https://avatars.githubusercontent.com/u/40219486?v=4"},"body":"Hi Felipe,\n\nOn 31/05/21 23.00, Felipe Contreras wrote:\n> Bagas Sanjaya wrote:\n>> I have full clone of Vim repository [1] on my computer. This repo has\n>> tags for each commit, meaning that when there is a new commit landed on\n>> default (master) branch, it will be tagged (something close to release\n>> tags but not same).\n> \n> I sometimes work on that repo and I just have to say how much of a\n> pleasure it is to work on a properly maintained repo such as git.git.\n> \n> My last bisection on vim.git landed on this beauty of a commit:\n> \n>      Update runtime files\n> \n\nWhat were you doing there? I'm discussing about performance issue instead.\n\n-- \nAn old man doll... just what I always wanted! - Clara\n"}]}