{"thread":{"id":"53283","subject":"Git 2.26 fetches many times more objects than it should, wasting gigabytes","startedAt":"2020-04-22T08:43:33Z","lastAt":"2020-04-24T05:32:07Z","messageCount":18,"participants":["Lubomir Rintel","Jeff King","Junio C Hamano","Jonathan Nieder","Jonathan Tan"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"395888","messageId":"20200422084254.GA27502@furthur.local","threadId":"53283","inReplyTo":null,"subject":"Git 2.26 fetches many times more objects than it should, wasting gigabytes","fromName":"Lubomir Rintel","fromEmail":"lkundrak@v3.sk","sentAt":"2020-04-22T08:42:54Z","receivedAt":"2020-04-22T08:43:33Z","isPatch":false,"sender":{"key":"lkundrak@v3.sk","avatar":"https://gravatar.com/avatar/0c78b09297e4f43bda3282627ada927e031aabd945d9a02df75ae525a18952a8?d=mp&s=160"},"body":"Hi,\n\nmy git repository with Linux grows several gigabytes each time I fetch:\n\n  [lkundrak@furthur linux]$ git fetch --all\n  Fetching origin\n  Receiving objects: 100% (431/431), 72.19 KiB | 345.00 KiB/s, done.\n  Fetching stable\n  Receiving objects: 100% (127228/127228), 22.01 MiB | 1.93 MiB/s, done.\n  Fetching next\n  Receiving objects: 100% (31113/31113), 6.51 MiB | 1.11 MiB/s, done.\n  Fetching net\n  Receiving objects: 100% (7331963/7331963), 1.20 GiB | 2.48 MiB/s, done.\n  Fetching tip\n  Receiving objects: 100% (7334643/7334643), 1.20 GiB | 2.44 MiB/s, done.\n  Fetching irqchip\n  Receiving objects: 100% (7333669/7333669), 1.20 GiB | 2.44 MiB/s, done.\n  Fetching drm\n  Receiving objects:  26% (1931483/7336388), 687.05 MiB | 1.55 MiB/s\n  ...\n\nNote the 1.2 GiB fetches from irqchip, tip, drm, net, etc. It looks like\nthe whole history gets fetched instead of the few changes that were\nadded since the fetch.\n\nWhen I've first noticed this happening I've thrown away the repository,\ninitialized a new one with Git 2.26.0 and fetched everything anew, but\nthat didn't help.\n\nI have very little clue about how to debug this. I'd be thankful for\nsuggestions about how to provide more details if necessary. I'm using\ngit from a Fedora package with this version number:\n\n  [lkundrak@furthur linux]$ rpm -q git\n  git-2.26.0-1.fc32.x86_64\n\nHere's a full log of my today's unfortunate fetch (still running...)\n\n  [lkundrak@furthur linux]$ git fetch --all\n  Fetching origin\n  remote: Enumerating objects: 766, done.\n  remote: Counting objects: 100% (636/636), done.\n  remote: Compressing objects: 100% (154/154), done.\n  remote: Total 431 (delta 355), reused 335 (delta 275)\n  Receiving objects: 100% (431/431), 72.19 KiB | 345.00 KiB/s, done.\n  Resolving deltas: 100% (355/355), completed with 120 local objects.\n  From git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux\n     ae83d0b416db00..18bf34080c4c3b  master     -> origin/master\n  Fetching stable\n  remote: Enumerating objects: 164963, done.\n  remote: Counting objects: 100% (156255/156255), done.\n  remote: Compressing objects: 100% (35062/35062), done.\n  remote: Total 127228 (delta 109912), reused 101371 (delta 91954)\n  Receiving objects: 100% (127228/127228), 22.01 MiB | 1.93 MiB/s, done.\n  Resolving deltas: 100% (109912/109912), completed with 12616 local objects.\n  From git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux\n     8488c3f3bc867e..8e2406c8518775  linux-4.19.y -> stable/linux-4.19.y\n     dc4059d21d87e8..6ccc74c083c0d4  linux-5.4.y  -> stable/linux-5.4.y\n     0634aa9416af81..937381741d02cc  linux-5.5.y  -> stable/linux-5.5.y\n     f07f08b09f05e3..7c572703216073  linux-5.6.y  -> stable/linux-5.6.y\n   * [new tag]                       v4.19.117    -> v4.19.117\n   * [new tag]                       v5.4.34      -> v5.4.34\n   * [new tag]                       v5.5.19      -> v5.5.19\n   * [new tag]                       v5.6.6       -> v5.6.6\n  Fetching next\n  remote: Enumerating objects: 75381, done.\n  remote: Counting objects: 100% (38266/38266), done.\n  remote: Compressing objects: 100% (9052/9052), done.\n  remote: Total 31113 (delta 26421), reused 26574 (delta 22000)\n  Receiving objects: 100% (31113/31113), 6.51 MiB | 1.11 MiB/s, done.\n  Resolving deltas: 100% (26421/26421), completed with 4591 local objects.\n  From git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next\n   + e98ff732aff9f7...d7b7d5f7953a08 akpm          -> next/akpm  (forced update)\n   + 29d027a641745b...8e32e79bb64e7e akpm-base     -> next/akpm-base  (forced update)\n   + 6735c84f78e417...a5840f9618a90e master        -> next/master  (forced update)\n   + f507be28f9e551...aa411fab3c05eb pending-fixes -> next/pending-fixes  (forced update)\n     ae83d0b416db00..18bf34080c4c3b  stable        -> next/stable\n   * [new tag]                       next-20200422 -> next-20200422\n  Fetching xo\n  From github.com:hackerspace/olpc-xo175-linux\n   * [new branch]                    lr/8250-json-schema-v2 -> xo/lr/8250-json-schema-v2\n   + 3942092b6c20ea...483c7451896cff lr/ariel               -> xo/lr/ariel  (forced update)\n   * [new branch]                    lr/ch7033-v4           -> xo/lr/ch7033-v4\n     0472b4080244b7..d2339c1aeb192d  lr/mmp-adma            -> xo/lr/mmp-adma\n   * [new branch]                    lr/mmp-dts             -> xo/lr/mmp-dts\n   + 3dc167b0785d17...2a1d6d9af30e19 lr/mmp2-clk-audio-gpu  -> xo/lr/mmp2-clk-audio-gpu  (forced update)\n  Fetching olpc\n  Fetching spi\n  From git://git.kernel.org/pub/scm/linux/kernel/git/broonie/spi\n     0dadde344d9655..0392727c261bab  for-5.7    -> spi/for-5.7\n     59fc9ad5cb108b..2f5f5302c569f7  for-5.8    -> spi/for-5.8\n   + 5e60c07c8615e8...bedad93ec5f83a for-linus  -> spi/for-linus  (forced update)\n   + 36792a4aa66c21...c5a7b42434ff12 for-next   -> spi/for-next  (forced update)\n  Fetching arm-soc\n  From git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc\n   + 512e8d40f91d7e...e9801213465aa8 arm/fixes  -> arm-soc/arm/fixes  (forced update)\n   + 512e8d40f91d7e...e9801213465aa8 for-next   -> arm-soc/for-next  (forced update)\n  Fetching net\n  remote: Enumerating objects: 7331963, done.\n  remote: Counting objects: 100% (7331963/7331963), done.\n  remote: Compressing objects: 100% (1114459/1114459), done.\n  remote: Total 7331963 (delta 6171286), reused 7329526 (delta 6169706)\n  Receiving objects: 100% (7331963/7331963), 1.20 GiB | 2.48 MiB/s, done.\n  Resolving deltas: 100% (6171286/6171286), done.\n  From git://git.kernel.org/pub/scm/linux/kernel/git/davem/net\n     9bacd256f13548..b9663b7ca6ff78  master     -> net/master\n  Fetching net-next\n  From git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next\n     0fde6e3b55a15a..44dd5efc97dae0  master     -> net-next/master\n  Fetching arm\n  From git://git.armlinux.org.uk/~rmk/linux-arm\n     89604523a76eb3..8f3d9f35428674  fixes      -> arm/fixes\n   + 52d3b2f98483c3...365a6327cd643e for-next   -> arm/for-next  (forced update)\n     8632e9b5645bbc..ae83d0b416db00  master     -> arm/master\n  Fetching tip\n  remote: Enumerating objects: 7334643, done.\n  remote: Counting objects: 100% (7334643/7334643), done.\n  remote: Compressing objects: 100% (1115060/1115060), done.\n  Receiving objects: 100% (7334643/7334643), 1.20 GiB | 2.44 MiB/s, done.\n  remote: Total 7334643 (delta 6173502), reused 7332019 (delta 6171782)\n  Resolving deltas: 100% (6173502/6173502), done.\n  From git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip\n   + 36d1b5ecb41546...f091a1f17abdba auto-latest   -> tip/auto-latest  (forced update)\n     11e31f608b499f..6e0d6ac5f3d9d9  core/core     -> tip/core/core\n   + 37d2e30492116b...22105aa089a7f5 master        -> tip/master  (forced update)\n     94d440d6184678..ac84bac4062e7f  timers/urgent -> tip/timers/urgent\n     4caffe6a28d315..675a59b7dec6e0  x86/build     -> tip/x86/build\n     aa61ee7b9ee3cb..a85573f7e74191  x86/mm        -> tip/x86/mm\n     79a3aaa7b82e31..cd2f45b7514cdd  x86/vdso      -> tip/x86/vdso\n  Fetching irqchip\n  remote: Enumerating objects: 7333669, done.\n  remote: Counting objects: 100% (7333669/7333669), done.\n  remote: Compressing objects: 100% (1114570/1114570), done.\n  Receiving objects: 100% (7333669/7333669), 1.20 GiB | 2.44 MiB/s, done.\n  remote: Total 7333669 (delta 6172952), reused 7331159 (delta 6171299)\n  Resolving deltas: 100% (6172952/6172952), done.\n  From git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms\n   + a2ffd41b964081...41932c0b588f27 hack/vim3l-crap          -> irqchip/hack/vim3l-crap  (forced update)\n   + a97bd23c9e3762...751551c45552e3 kvm-arm64/nv-5.7-rc1-WIP -> irqchip/kvm-arm64/nv-5.7-rc1-WIP  (forced update)\n  Fetching gpio\n  Fetching dvhart\n  remote: Enumerating objects: 23, done.\n  remote: Counting objects: 100% (23/23), done.\n  remote: Total 28 (delta 23), reused 23 (delta 23), pack-reused 5\n  Unpacking objects: 100% (28/28), 5.19 KiB | 13.00 KiB/s, done.\n  From https://github.com/dvhart/linux-pdx86\n     8f3d9f35428674..f7ea285b626682  for-next            -> dvhart/for-next\n   * [new branch]                    ib-pdx86-properties -> dvhart/ib-pdx86-properties\n     00086336a8d96a..ae83d0b416db00  master              -> dvhart/master\n   + 79777a3891c69e...5a93adbdbb42e9 review-andy         -> dvhart/review-andy  (forced update)\n  Fetching power-supply\n  Fetching mmp\n  Fetching usb\n  From git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb\n     be34a5854b4606..8f97250c21f0cf  usb-linus  -> usb/usb-linus\n  Fetching pinchartl\n  Fetching drm\n  remote: Enumerating objects: 7336388, done.\n  remote: Counting objects: 100% (7336388/7336388), done.\n  remote: Compressing objects: 100% (1091614/1091614), done.\n  Receiving objects:  26% (1931483/7336388), 687.05 MiB | 1.55 MiB/s\n\nHere's my .git/config:\n\n  [lkundrak@furthur linux]$ cat .git/config \n  [core]\n  \trepositoryformatversion = 0\n  \tfilemode = true\n  \tbare = false\n  \tlogallrefupdates = true\n  [gui]\n  \twmstate = normal\n  \tgeometry = 1400x954+-1+26 453 376\n  [merge]\n  \trenamelimit = 65535\n  [remote \"origin\"]\n  \turl = git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git\n  \tfetch = +refs/heads/*:refs/remotes/origin/*\n  [branch \"master\"]\n  \tremote = origin\n  \tmerge = refs/heads/master\n  [remote \"stable\"]\n  \turl = git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git\n  \tfetch = +refs/heads/*:refs/remotes/stable/*\n  [remote \"next\"]\n  \turl = git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git\n  \tfetch = +refs/heads/*:refs/remotes/next/*\n  [remote \"xo\"]\n  \turl = git@github.com:hackerspace/olpc-xo175-linux.git\n  \tfetch = +refs/heads/*:refs/remotes/xo/*\n  [branch \"lr/olpc-xo175\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175\n  [remote \"olpc\"]\n  \turl = http://dev.laptop.org/git/olpc-kernel/\n  \tfetch = +refs/heads/*:refs/remotes/olpc/*\n  [remote \"spi\"]\n  \turl = git://git.kernel.org/pub/scm/linux/kernel/git/broonie/spi.git\n  \tfetch = +refs/heads/*:refs/remotes/spi/*\n  [branch \"lr/olpc-xo175-fixes5-mmp\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-fixes5-mmp\n  [branch \"lr/olpc-xo175-fixes4-ap-sp\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-fixes4-ap-sp\n  [branch \"lr/olpc-xo175-fixes4-ec\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-fixes4-ec\n  [branch \"lr/olpc-xo175-fixes2-trivial\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-fixes2-trivial\n  [branch \"lr/olpc-xo175-fixes3-mmp-camera\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-fixes3-mmp-camera\n  [branch \"lr/olpc-xo175-drm\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-drm\n  [remote \"arm-soc\"]\n  \turl = git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc.git\n  \tfetch = +refs/heads/*:refs/remotes/arm-soc/*\n  [branch \"lr/olpc-xo175-fixes5-ec\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-fixes5-ec\n  [branch \"lr/olpc-xo175-fixes7-ec\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-fixes7-ec\n  [branch \"lr/olpc-xo175-fixes7-battery\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-fixes7-battery\n  [branch \"lr/olpc-xo175-fixes4-mmp-camera\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-fixes4-mmp-camera\n  [branch \"lr/olpc-xo175-fixes6\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-fixes6\n  [branch \"lr/olpc-xo175-fixes1-drm\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-fixes1-drm\n  [branch \"lr/olpc-xo175-fixes1-drm-dt\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-fixes1-drm-dt\n  [branch \"lr/olpc-xo175-fixes2-drm\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-fixes2-drm\n  [remote \"net\"]\n  \turl = git://git.kernel.org/pub/scm/linux/kernel/git/davem/net.git\n  \tfetch = +refs/heads/*:refs/remotes/net/*\n  [remote \"net-next\"]\n  \turl = git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next.git\n  \tfetch = +refs/heads/*:refs/remotes/net-next/*\n  #[remote \"olpc-wifi\"]\n  #\turl = git://dev.laptop.org/users/javier/wireless-testing\n  #\tfetch = +refs/heads/*:refs/remotes/olpc-wifi/*\n  [remote \"arm\"]\n  \turl = git://git.armlinux.org.uk/~rmk/linux-arm.git\n  \tfetch = +refs/heads/*:refs/remotes/arm/*\n  [branch \"lr/olpc-xo175-fixes3-drm\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-fixes3-drm\n  [remote \"tip\"]\n  \turl = git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git\n  \tfetch = +refs/heads/*:refs/remotes/tip/*\n  [remote \"irqchip\"]\n  \turl = git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git\n  \tfetch = +refs/heads/*:refs/remotes/irqchip/*\n  [branch \"lr/olpc-xo175-fixes6-next\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-fixes6-next\n  [remote \"gpio\"]\n  \turl = git://git.kernel.org/pub/scm/linux/kernel/git/linusw/linux-gpio.git\n  \tfetch = +refs/heads/*:refs/remotes/gpio/*\n  [branch \"lr/olpc-xo175-fixes8-battery\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-fixes8-battery\n  [remote \"dvhart\"]\n  \turl = https://github.com/dvhart/linux-pdx86.git\n  \tfetch = +refs/heads/*:refs/remotes/dvhart/*\n  [remote \"power-supply\"]\n  \turl = git://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-power-supply.git\n  \tfetch = +refs/heads/*:refs/remotes/power-supply/*\n  [branch \"lr/olpc-xo175-battery-v6\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-battery-v6\n  [branch \"lr/olpc-xo175-fixes2-drm-dt\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-fixes2-drm-dt\n  [branch \"lr/olpc-xo175-fixes4-drm\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-fixes4-drm\n  [branch \"olpc-5.0\"]\n  \tremote = xo\n  \tmerge = refs/heads/olpc-5.0\n  [branch \"lr/olpc-xo175-fixes8-ec\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-fixes8-ec\n  [branch \"lr/olpc-xo175-drm-dt-v3\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-drm-dt-v3\n  [branch \"lr/olpc-xo175-mmp-camera-v5\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-mmp-camera-v5\n  [branch \"lr/olpc-xo175-drm-dt-v4\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-drm-dt-v4\n  [branch \"lr/olpc-xo175-ec-v6\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-ec-v6\n  [branch \"lr/olpc-xo175-ec-v7\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-ec-v7\n  [branch \"lr/olpc-xo175-galcore-v1\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-galcore-v1\n  [branch \"lr/mmp3-1\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/mmp3-1\n  [branch \"lr/mmp3-2\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/mmp3-2\n  [branch \"lr/mmp3-4\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/mmp3-4\n  [branch \"lr/ch7033\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/ch7033\n  [branch \"lr/ch7033-1\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/ch7033-1\n  [branch \"lr/w000000t\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/w000000t\n  [branch \"lr/ariel\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/ariel\n  [remote \"mmp\"]\n  \turl = git://git.kernel.org/pub/scm/linux/kernel/git/lkundrak/linux-mmp\n  \tpushurl = git@gitolite.kernel.org:pub/scm/linux/kernel/git/lkundrak/linux-mmp\n  \tfetch = +refs/heads/*:refs/remotes/mmp/*\n  [branch \"lr/mmp3-hsic\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/mmp3-hsic\n  [branch \"lr/mmp3-hsic-v2\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/mmp3-hsic-v2\n  [branch \"lr/mmp2-galcore-v1\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/olpc-xo175-galcore-v1\n  [remote \"usb\"]\n  \turl = git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb.git\n  \tfetch = +refs/heads/*:refs/remotes/usb/*\n  [remote \"pinchartl\"]\n  \turl = git://linuxtv.org/pinchartl/media.git\n  \tfetch = +refs/heads/*:refs/remotes/pinchartl/*\n  [remote \"drm\"]\n  \turl = git://anongit.freedesktop.org/drm/drm\n  \tfetch = +refs/heads/*:refs/remotes/drm/*\n  [branch \"lr/marvell-dt-validation-v2\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/marvell-dt-validation-v2\n  [branch \"lr/mmp3-thermal-v1\"]\n  \tremote = xo\n  \tmerge = refs/heads/lr/mmp3-thermal-v1\n  [remote \"clk\"]\n  \turl = git://git.kernel.org/pub/scm/linux/kernel/git/clk/linux.git\n  \tfetch = +refs/heads/*:refs/remotes/clk/*\n  [remote \"drm-misc\"]\n  \turl = git://anongit.freedesktop.org/drm/drm-misc\n  \tfetch = +refs/heads/*:refs/remotes/drm-misc/*\n\nThank you\nLubo\n"},{"id":"395891","messageId":"20200422095702.GA475060@coredump.intra.peff.net","threadId":"53283","inReplyTo":"20200422084254.GA27502@furthur.local","subject":"Re: Git 2.26 fetches many times more objects than it should, wasting gigabytes","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-04-22T09:57:02Z","receivedAt":"2020-04-22T09:57:09Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Apr 22, 2020 at 10:42:54AM +0200, Lubomir Rintel wrote:\n\n> my git repository with Linux grows several gigabytes each time I fetch:\n\nThanks for this report. We've been tracking the issue but have had\ntrouble reproducing it.\n\nTo get you unstuck, the immediate workaround is to drop back to the\nolder protocol, like:\n\n  git -c protocol.version=0 fetch --all\n\n>   [lkundrak@furthur linux]$ git fetch --all\n\nHere's a recipe based on your fetches that shows the problem.\n\n  # start with an up-to-date regular clone of linus's tree; I had one\n  # lying around from https://github.com/torvalds/linux, but the source\n  # shouldn't matter\n  rm -rf repo.git\n  git clone --bare /path/to/linux repo.git\n  cd repo.git\n\n  git remote add next git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next\n  git remote add xo git@github.com:hackerspace/olpc-xo175-linux\n  git fetch --all\n\nThe \"next\" fetch grabs about 30MB of objects. But the xo one downloads\n1.5GB from 7.4M objects. That's using v2.26.2, so protocol 2.\n\nIf I switch to the v0 protocol like:\n\n  git -c protocol.version=0 fetch --all\n\nthen the xo fetch is only 48k objects, 23MB. So this is definitely\nexhibiting the problem.\n\nThere are a few data points we've been wanting to collect:\n\n - does setting fetch.negotiationAlgorithm=skipping help? Yes, but not\n   as much as the v0 protocol does. It sends 84k objects, 33MB.\n\n - does the same fetch over v0 stateless-http have similar problems? No,\n   swapping out the second \"remote add\" for:\n\n     git remote add xo https://github.com/hackerspace/olpc-xo175-linux\n\n   results in the same 48k, 32MB fetch. The v0 conversation involved 10\n   POST requests. The v2 conversation only took 6 (and generates the\n   same big response as the ssh session, unsurprisingly).\n\nSo it really does seem like something in v2 is not trying as hard to\nnegotiate as v0 did, even when using stateless-http.\n\nI'm attaching for-each-ref output before and after the xo fetch. That\nshould be sufficient to recreate the situation synthetically even once\nthese repos have moved on.\n\nI have GIT_TRACE_PACKET output showing the whole negotiation, but it's\npretty hard to look at. I _think_ a lot more is said in the v0\nconversation, but it's difficult to sort out because there's a lot of\nextra packet framing as we shuttle bits back and forth between\nremote-curl and fetch-pack.\n\n-Peff\n"},{"id":"395893","messageId":"20200422103011.GA545254@coredump.intra.peff.net","threadId":"53283","inReplyTo":"20200422095702.GA475060@coredump.intra.peff.net","subject":"Re: Git 2.26 fetches many times more objects than it should, wasting gigabytes","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-04-22T10:30:11Z","receivedAt":"2020-04-22T10:30:46Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Apr 22, 2020 at 05:57:02AM -0400, Jeff King wrote:\n\n> I'm attaching for-each-ref output before and after the xo fetch. That\n> should be sufficient to recreate the situation synthetically even once\n> these repos have moved on.\n> \n> I have GIT_TRACE_PACKET output showing the whole negotiation, but it's\n> pretty hard to look at. I _think_ a lot more is said in the v0\n> conversation, but it's difficult to sort out because there's a lot of\n> extra packet framing as we shuttle bits back and forth between\n> remote-curl and fetch-pack.\n\nHere's what I did manage to pull out if it. In both versions we send a\nbunch of \"have\" lines, and get a bunch of NAKs back. Which makes sense.\nWe have a bunch of refs that the other side doesn't know about.\n\nIn the v0 http session, we finally get an ACK on\n8f3d9f354286745c751374f5f1fcafee6b3f3136, which is Linus's 5.7-rc1. We\nsend that commit as a \"have\" in the 9th batch.\n\nIn the v2 session, we never get an ACK at all (which unsurprisingly\nleads to the other side sending the full history). But that commit id is\nnowhere to be found in the trace! We appear to give up after 5 rounds.\n\nSo it really just seems like v2 does not try hard enough. I think the\nculprit is the MAX_IN_VAIN setting. If I do this:\n\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 1734a573b0..016a413d49 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -46,7 +46,7 @@ static struct strbuf fsck_msg_types = STRBUF_INIT;\n  * After sending this many \"have\"s if we do not get any new ACK , we\n  * give up traversing our history.\n  */\n-#define MAX_IN_VAIN 256\n+#define MAX_IN_VAIN 20000\n \n static int multi_ack, use_sideband;\n /* Allow specifying sha1 if it is a ref tip. */\n\nthen I get that same 48k objects, 23MB fetch that v0 does.\n\n-Peff\n"},{"id":"395894","messageId":"20200422104000.GA551233@coredump.intra.peff.net","threadId":"53283","inReplyTo":"20200422103011.GA545254@coredump.intra.peff.net","subject":"Re: Git 2.26 fetches many times more objects than it should, wasting gigabytes","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-04-22T10:40:00Z","receivedAt":"2020-04-22T10:40:13Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Apr 22, 2020 at 06:30:11AM -0400, Jeff King wrote:\n\n> So it really just seems like v2 does not try hard enough. I think the\n> culprit is the MAX_IN_VAIN setting. If I do this:\n> \n> diff --git a/fetch-pack.c b/fetch-pack.c\n> index 1734a573b0..016a413d49 100644\n> --- a/fetch-pack.c\n> +++ b/fetch-pack.c\n> @@ -46,7 +46,7 @@ static struct strbuf fsck_msg_types = STRBUF_INIT;\n>   * After sending this many \"have\"s if we do not get any new ACK , we\n>   * give up traversing our history.\n>   */\n> -#define MAX_IN_VAIN 256\n> +#define MAX_IN_VAIN 20000\n>  \n>  static int multi_ack, use_sideband;\n>  /* Allow specifying sha1 if it is a ref tip. */\n> \n> then I get that same 48k objects, 23MB fetch that v0 does.\n\nI don't quite think that's the solution, though. Both old and new are\nsupposed to be respecting MAX_IN_VAIN. So it's not at all clear to me\nwhy it restricts the number of haves we'll send in v2, but not in v0.\n\nMaybe somebody more familiar with the negotiation code can comment\nfurther.\n\n-Peff\n"},{"id":"395904","messageId":"xmqqwo67k00z.fsf@gitster.c.googlers.com","threadId":"53283","inReplyTo":"20200422104000.GA551233@coredump.intra.peff.net","subject":"Re: Git 2.26 fetches many times more objects than it should, wasting gigabytes","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-04-22T15:33:48Z","receivedAt":"2020-04-22T15:33:58Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> On Wed, Apr 22, 2020 at 06:30:11AM -0400, Jeff King wrote:\n>\n>> So it really just seems like v2 does not try hard enough. I think the\n>> culprit is the MAX_IN_VAIN setting. If I do this:\n>> \n>> diff --git a/fetch-pack.c b/fetch-pack.c\n>> index 1734a573b0..016a413d49 100644\n>> --- a/fetch-pack.c\n>> +++ b/fetch-pack.c\n>> @@ -46,7 +46,7 @@ static struct strbuf fsck_msg_types = STRBUF_INIT;\n>>   * After sending this many \"have\"s if we do not get any new ACK , we\n>>   * give up traversing our history.\n>>   */\n>> -#define MAX_IN_VAIN 256\n>> +#define MAX_IN_VAIN 20000\n>>  \n>>  static int multi_ack, use_sideband;\n>>  /* Allow specifying sha1 if it is a ref tip. */\n>> \n>> then I get that same 48k objects, 23MB fetch that v0 does.\n>\n> I don't quite think that's the solution, though. Both old and new are\n> supposed to be respecting MAX_IN_VAIN. So it's not at all clear to me\n> why it restricts the number of haves we'll send in v2, but not in v0.\n\nThanks for digging.  I tend to agree with your assessment that the\nsetting should not make a difference, if v0 find the common out of\nthe exchange within the same number of \"have\"s.\n\nI am guilty of introducing the hardcoded \"give up after this many\nnaks\", which I admit I was never fond of, back in the days there was\nonly one original protocol.  In retrospect, I probably should have\ndone \"after this many naks, stop sending each and every commit but\nstart skipping exponentially (or fibonacci)\" instead.  After all,\nthis was meant to prevent walking all the way down to a different\nroot commit when you have more of them than the repository you are\nfetching from---but (1) skipping exponentially down to root is way\nless expensive, even if it is a bit more expensive than not walking\nat all, and (2) if we find a common tree, even if it is distant, it\nis way better than not having any common tree at all.\n\nIf we had such a code, however, it would probably have swept the\nreal cause of the issue people are reporting under the rug, though.\n\n"},{"id":"395906","messageId":"20200422154025.GA91734@google.com","threadId":"53283","inReplyTo":"20200422095702.GA475060@coredump.intra.peff.net","subject":"Re: Git 2.26 fetches many times more objects than it should, wasting gigabytes","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2020-04-22T15:40:25Z","receivedAt":"2020-04-22T15:40:29Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"(+cc: Jonathan Tan)\nHi,\n\nJeff King wrote:\n\n> Here's a recipe based on your fetches that shows the problem.\n>\n>   # start with an up-to-date regular clone of linus's tree; I had one\n>   # lying around from https://github.com/torvalds/linux, but the source\n>   # shouldn't matter\n>   rm -rf repo.git\n>   git clone --bare /path/to/linux repo.git\n>   cd repo.git\n>\n>   git remote add next git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next\n>   git remote add xo git@github.com:hackerspace/olpc-xo175-linux\n>   git fetch --all\n>\n> The \"next\" fetch grabs about 30MB of objects. But the xo one downloads\n> 1.5GB from 7.4M objects. That's using v2.26.2, so protocol 2.\n\nThanks!  I'll give it a try.\n\n[...]\n> There are a few data points we've been wanting to collect:\n>\n>  - does setting fetch.negotiationAlgorithm=skipping help? Yes, but not\n>    as much as the v0 protocol does. It sends 84k objects, 33MB.\n\nThat's pretty good.  Tightening it further would require changing the\nprotocol to allow the client to say \"please don't send me a pack; I want\nto continue with negotiation\".\n\n>  - does the same fetch over v0 stateless-http have similar problems? No,\n>    swapping out the second \"remote add\" for:\n>\n>      git remote add xo https://github.com/hackerspace/olpc-xo175-linux\n>\n>    results in the same 48k, 32MB fetch. The v0 conversation involved 10\n>    POST requests. The v2 conversation only took 6 (and generates the\n>    same big response as the ssh session, unsurprisingly).\n>\n> So it really does seem like something in v2 is not trying as hard to\n> negotiate as v0 did, even when using stateless-http.\n\nInteresting!  So it sounds like some refs that are not being fetched\nare important here to the negotiation.  And the default (non-skipping)\nnegotiation algorithm is doing a bad job of exploring that part of\nhistory.\n\nWill take a closer look.\n\nI think this still suggests that we should go ahead and switch\nnegotiation algorithms, both because it avoids this MAX_IN_VAIN and\nbecause it reduces the number of rounds needed to make progress.\n\nI'd also be tempted to get rid of MAX_IN_VAIN.  If we're at the point\nof giving up, shouldn't we error out instead of having the server send\na copy of the entirety of history?\n\nJonathan\n"},{"id":"395908","messageId":"20200422155047.GB91734@google.com","threadId":"53283","inReplyTo":"20200422095702.GA475060@coredump.intra.peff.net","subject":"[PATCH] Revert \"fetch: default to protocol version 2\"","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2020-04-22T15:50:47Z","receivedAt":"2020-04-22T15:50:53Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"This reverts commit 684ceae32dae726c6a5c693b257b156926aba8b7.\n\nUsers fetching from linux-next and other kernel remotes are reporting\nthat the limited ref advertisement causes negotiation to reach\nMAX_IN_VAIN, resulting in too-large fetches.\n\nReported-by: Lubomir Rintel <lkundrak@v3.sk>\nReported-by: \"Dixit, Ashutosh\" <ashutosh.dixit@intel.com>\nReported-by: Jiri Slaby <jslaby@suse.cz>\nReported-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\nHi again,\n\nJeff King wrote:\n\n> Thanks for this report. We've been tracking the issue but have had\n> trouble reproducing it.\n>\n> To get you unstuck, the immediate workaround is to drop back to the\n> older protocol, like:\n>\n>   git -c protocol.version=0 fetch --all\n\nBy the way, I'd recommend the immediate workaround of\n\n\tgit fetch --negotiation-tip=refs/remotes/xo/* xo\n\ninstead.  But that's a separate subject.\n\n[...]\n> There are a few data points we've been wanting to collect:\n[...]\n> I'm attaching for-each-ref output before and after the xo fetch. That\n> should be sufficient to recreate the situation synthetically even once\n> these repos have moved on.\n\nExcellent --- I think this is enough for us to have something to use\nto investigate, switching users to protocol v0 in the meantime.\n\nThanks,\nJonathan\n\n Documentation/config/protocol.txt | 2 +-\n protocol.c                        | 2 +-\n 2 files changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/config/protocol.txt b/Documentation/config/protocol.txt\nindex 756591d77b0..0b40141613e 100644\n--- a/Documentation/config/protocol.txt\n+++ b/Documentation/config/protocol.txt\n@@ -48,7 +48,7 @@ protocol.version::\n \tIf set, clients will attempt to communicate with a server\n \tusing the specified protocol version.  If the server does\n \tnot support it, communication falls back to version 0.\n-\tIf unset, the default is `2`.\n+\tIf unset, the default is `0`.\n \tSupported versions:\n +\n --\ndiff --git a/protocol.c b/protocol.c\nindex 803bef5c87e..d390391ebac 100644\n--- a/protocol.c\n+++ b/protocol.c\n@@ -39,7 +39,7 @@ enum protocol_version get_protocol_version_config(void)\n \t\treturn env;\n \t}\n \n-\treturn protocol_v2;\n+\treturn protocol_v0;\n }\n \n enum protocol_version determine_protocol_version_server(void)\n-- \n2.26.2.303.gf8c07b1a785\n\n"},{"id":"395913","messageId":"20200422165358.GB140314@google.com","threadId":"53283","inReplyTo":"20200422095702.GA475060@coredump.intra.peff.net","subject":"Re: Git 2.26 fetches many times more objects than it should, wasting gigabytes","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2020-04-22T16:53:58Z","receivedAt":"2020-04-22T16:54:05Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Jeff King wrote:\n\n> So it really does seem like something in v2 is not trying as hard to\n> negotiate as v0 did, even when using stateless-http.\n>\n> I'm attaching for-each-ref output before and after the xo fetch. That\n> should be sufficient to recreate the situation synthetically even once\n> these repos have moved on.\n\nThanks again.  Looked a little closer and the advertised refs shouldn't\nmatter.  We just have two completely different code paths for\nnegotiation, one for v0 and one for v2.  In v0, we do\n\n\tfetch_negotiator_init(r, negotiator);\n\tmark_complete_and_common_ref(negotiator, args, &ref);\n\tfilter_refs(argrs, &ref, sought, nr_sought);\n\tfind_common(negotiator, args, &ref, sought, nr_sought);\n\nfind_common does\n\n\tcount = 0;\n\twhile ((oid = negotiator->next(negotiator)) {\n\t\twrite \"have %s\\n\", oid\n\t\tif (++count >= flush_at) {\n\t\t\t... flush ...\n\t\t\tflush_at = next_flush(args->stateless_rpc, count);\n\t\t\t...\n\nIn v2, the corresponding code is in add_haves:\n\n\twhile ((oid = negotiator->next(negotiator))) {\n\t\twrite \"have %s\\n\", oid\n\t\tif (++haves_added >= *haves_to_send)\n\t\t\tbreak;\n\t}\n\t*in_vain += haves_added;\n\tif (!haves_added || *in_vain >= MAX_IN_VAIN)\n\t\twrite \"done\"\n\t*haves_to_send = next_flush(1, *haves_to_send);\n\nIn other words, in v2 we reset the counter on each round whereas in v0\nwe keep a running total.  That would be expected to produce larger\nnegotiate requests in v2 than v0 (which looks like an unintentional\ndifference, but not the one producing this bug).\n\nSo much for flush_at handling.  in_vain seems like another area to\ncompare: in v2, the driving logic is in do_fetch_pack_v2:\n\n\thaves_to_send = INITIAL_FLUSH;\n negotiate_loop:\n\twhile (!send_fetch_request(negotiator, fd, args, ref,\n\t\t\t\t   &common, &haves_to_send,\n\t\t\t\t   &in_vain, reader.use_sideband))\n\t\tswitch (process_acks(negotiator, &reader, &common)) {\n\t\tcase 1:\n\t\t\tin_vain = 0; /* renewed hope!\n\t\t\tcontinue;\n\t\tcase 2:\n\t\t\tbreak negotiate_loop; /* time to move on */\n\t\tdefault:\n\t\t\tcontinue;\n\t\t}\n\nWhen process_acks sees an ACK, it passes it on to the negotiator.\nIt wants to record that it received an ack to reset in_vain, but\nit forgets to!  The function is initialized and read but never\nwritten to.\n\nSo I'd expect the following to help:\n\ndiff --git i/fetch-pack.c w/fetch-pack.c\nindex 1734a573b01..a1d743e1f61 100644\n--- i/fetch-pack.c\n+++ w/fetch-pack.c\n@@ -1287,6 +1287,8 @@ static int process_acks(struct fetch_negotiator *negotiator,\n \t\t\tstruct object_id oid;\n \t\t\tif (!get_oid_hex(arg, &oid)) {\n \t\t\t\tstruct commit *commit;\n+\n+\t\t\t\treceived_ack = 1;\n \t\t\t\toidset_insert(common, &oid);\n \t\t\t\tcommit = lookup_commit(the_repository, &oid);\n \t\t\t\tif (negotiator)\n"},{"id":"395917","messageId":"xmqq7dy7juix.fsf@gitster.c.googlers.com","threadId":"53283","inReplyTo":"20200422165358.GB140314@google.com","subject":"Re: Git 2.26 fetches many times more objects than it should, wasting gigabytes","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-04-22T17:32:38Z","receivedAt":"2020-04-22T17:32:43Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Nieder <jrnieder@gmail.com> writes:\n\n> When process_acks sees an ACK, it passes it on to the negotiator.\n> It wants to record that it received an ack to reset in_vain, but\n> it forgets to!  The function is initialized and read but never\n> written to.\n\nYeah, this smells like the right solution ;-)\n\ns/function/variable/ though.\n\n>\n> So I'd expect the following to help:\n>\n> diff --git i/fetch-pack.c w/fetch-pack.c\n> index 1734a573b01..a1d743e1f61 100644\n> --- i/fetch-pack.c\n> +++ w/fetch-pack.c\n> @@ -1287,6 +1287,8 @@ static int process_acks(struct fetch_negotiator *negotiator,\n>  \t\t\tstruct object_id oid;\n>  \t\t\tif (!get_oid_hex(arg, &oid)) {\n>  \t\t\t\tstruct commit *commit;\n> +\n> +\t\t\t\treceived_ack = 1;\n>  \t\t\t\toidset_insert(common, &oid);\n>  \t\t\t\tcommit = lookup_commit(the_repository, &oid);\n>  \t\t\t\tif (negotiator)\n"},{"id":"395924","messageId":"xmqqtv1bidlu.fsf@gitster.c.googlers.com","threadId":"53283","inReplyTo":"20200422155047.GB91734@google.com","subject":"Re: [PATCH] Revert \"fetch: default to protocol version 2\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-04-22T18:23:25Z","receivedAt":"2020-04-22T18:23:33Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Nieder <jrnieder@gmail.com> writes:\n\n> This reverts commit 684ceae32dae726c6a5c693b257b156926aba8b7.\n>\n> Users fetching from linux-next and other kernel remotes are reporting\n> that the limited ref advertisement causes negotiation to reach\n> MAX_IN_VAIN, resulting in too-large fetches.\n>\n> Reported-by: Lubomir Rintel <lkundrak@v3.sk>\n> Reported-by: \"Dixit, Ashutosh\" <ashutosh.dixit@intel.com>\n> Reported-by: Jiri Slaby <jslaby@suse.cz>\n> Reported-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>\n> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>\n> ---\n\nWith the one-liner fix for resetting the \"we haven't heard an Ack\"\ntimer, I am no longer so worried by hurting users by keeping them on\nv2 by default, but it will take some time for the fix to trickle\ndown from the 'master' track to a maintenance release, so I am OK to\napply this patch in the meantime.  But let's flip the default back\nto v2 after the one-liner fix lands to see if (1) the fix truly\nhelps and (2) people hit other issues in the difference between v0\nand v2.\n\nThanks.\n"},{"id":"395931","messageId":"20200422191841.GA558336@coredump.intra.peff.net","threadId":"53283","inReplyTo":"20200422165358.GB140314@google.com","subject":"Re: Git 2.26 fetches many times more objects than it should, wasting gigabytes","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-04-22T19:18:41Z","receivedAt":"2020-04-22T19:18:44Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Apr 22, 2020 at 09:53:58AM -0700, Jonathan Nieder wrote:\n\n> When process_acks sees an ACK, it passes it on to the negotiator.\n> It wants to record that it received an ack to reset in_vain, but\n> it forgets to!  The function is initialized and read but never\n> written to.\n\nI wondered if it might be something like this, too (and this might well\nbe an independent bug), but...\n\n> So I'd expect the following to help:\n> \n> diff --git i/fetch-pack.c w/fetch-pack.c\n> index 1734a573b01..a1d743e1f61 100644\n> --- i/fetch-pack.c\n> +++ w/fetch-pack.c\n> @@ -1287,6 +1287,8 @@ static int process_acks(struct fetch_negotiator *negotiator,\n>  \t\t\tstruct object_id oid;\n>  \t\t\tif (!get_oid_hex(arg, &oid)) {\n>  \t\t\t\tstruct commit *commit;\n> +\n> +\t\t\t\treceived_ack = 1;\n>  \t\t\t\toidset_insert(common, &oid);\n>  \t\t\t\tcommit = lookup_commit(the_repository, &oid);\n>  \t\t\t\tif (negotiator)\n\nIt doesn't. We never get any ACK from the server at all, because we give\nup on sending haves before hitting any common commit.\n\n-Peff\n"},{"id":"395933","messageId":"20200422193324.GB558336@coredump.intra.peff.net","threadId":"53283","inReplyTo":"xmqqwo67k00z.fsf@gitster.c.googlers.com","subject":"Re: Git 2.26 fetches many times more objects than it should, wasting gigabytes","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-04-22T19:33:24Z","receivedAt":"2020-04-22T19:33:26Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Apr 22, 2020 at 08:33:48AM -0700, Junio C Hamano wrote:\n\n> > I don't quite think that's the solution, though. Both old and new are\n> > supposed to be respecting MAX_IN_VAIN. So it's not at all clear to me\n> > why it restricts the number of haves we'll send in v2, but not in v0.\n> \n> Thanks for digging.  I tend to agree with your assessment that the\n> setting should not make a difference, if v0 find the common out of\n> the exchange within the same number of \"have\"s.\n\nI think v0 sends many more haves. Again, it's hard to compare the\nprotocol traces because of the framing, but if I simplify each one like:\n\n  perl -lne '/fetch-pack([<>] .*)/ and print \"fetch$1\"' <packet-v0.trace  >small-v0.trace\n  perl -lne '/fetch([<>] .*)/ and print \"fetch$1\"' <packet-v2.trace  >small-v2.trace\n\nI think we can get an apples-to-apples-ish comparison. And the results\nare quite different:\n\n  $ grep -c have small-v0.trace\n  11342\n  $ grep -c have small-v2.trace\n  496\n\nSo I think the two protocols are treating MAX_IN_VAIN quite differently.\n\nIt looks like v0 only respects it after seeing a \"continue\" (or maybe\nany non-common ACK; they all seem to trigger got_continue), but v2 will\nuse it to limit the haves we send when the other side is just NAKing.\n\n> I am guilty of introducing the hardcoded \"give up after this many\n> naks\", which I admit I was never fond of, back in the days there was\n> only one original protocol.  In retrospect, I probably should have\n> done \"after this many naks, stop sending each and every commit but\n> start skipping exponentially (or fibonacci)\" instead.  After all,\n> this was meant to prevent walking all the way down to a different\n> root commit when you have more of them than the repository you are\n> fetching from---but (1) skipping exponentially down to root is way\n> less expensive, even if it is a bit more expensive than not walking\n> at all, and (2) if we find a common tree, even if it is distant, it\n> is way better than not having any common tree at all.\n\nI think fetch.negotiationAlgorithm=skipping is that thing. And it _does_\npaper over the problem (the most horrific case goes away, but you end up\nwith twice as many objects as v2 finds).\n\nLimiting the amount of work we're willing to spend digging in history\ndoes make sense, but it seems like we'd always want to at least dig a\nlittle on each ref. For example, imagine a pathological case like this:\n\n  - the client has 10,001 refs; the first 10,000 (sorted alphabetically)\n    point to commit graph X. The last one points to some disjoint commit\n    graph Y.\n\n  - the server only cares about Y, and it has some Y' that adds one\n    commit on top\n\nWe _should_ be able to serve that fetch with a single commit (Y->Y').\nAnd we could find it trivially by feeding all of the ref tips as \"have\"\nlines. But I suspect we wouldn't with v2, as we'd feed the first couple\nhundred haves and then give up.\n\n-Peff\n"},{"id":"395934","messageId":"20200422193658.GC558336@coredump.intra.peff.net","threadId":"53283","inReplyTo":"20200422154025.GA91734@google.com","subject":"Re: Git 2.26 fetches many times more objects than it should, wasting gigabytes","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-04-22T19:36:58Z","receivedAt":"2020-04-22T19:37:01Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Apr 22, 2020 at 08:40:25AM -0700, Jonathan Nieder wrote:\n\n> I think this still suggests that we should go ahead and switch\n> negotiation algorithms, both because it avoids this MAX_IN_VAIN and\n> because it reduces the number of rounds needed to make progress.\n\nI'd be happier about that if the other algorithm turned up the same pack\nthat v0 does. But it has twice as many objects.\n\nIt looks to me like v0 is just more aggressive about digging in the\nhistory. That _does_ cost more rounds, but this example shows that\nthere's a benefit to doing so for real-world cases.\n\n> I'd also be tempted to get rid of MAX_IN_VAIN.  If we're at the point\n> of giving up, shouldn't we error out instead of having the server send\n> a copy of the entirety of history?\n\nHow would you fetch in cases where the client and server legitimately\ndon't have any common commits?\n\nYou could add a flag to force it, but I don't know that you're really\nmaking users any happier. Fetching the whole history is annoying, but\nrefusing to fetch at all is perhaps more so. From the user's perspective\neither the full fetch is what they want, or Git is broken.\n\n-Peff\n"},{"id":"395935","messageId":"20200422194047.GD558336@coredump.intra.peff.net","threadId":"53283","inReplyTo":"20200422155047.GB91734@google.com","subject":"Re: [PATCH] Revert \"fetch: default to protocol version 2\"","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-04-22T19:40:47Z","receivedAt":"2020-04-22T19:40:50Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Apr 22, 2020 at 08:50:47AM -0700, Jonathan Nieder wrote:\n\n> This reverts commit 684ceae32dae726c6a5c693b257b156926aba8b7.\n> \n> Users fetching from linux-next and other kernel remotes are reporting\n> that the limited ref advertisement causes negotiation to reach\n> MAX_IN_VAIN, resulting in too-large fetches.\n\nOK, now that we have data I think this strategy is reasonable.\n\nThat said, it will take a while to make it to a release, so we very well\nmay have brought v2 and v0 to parity in the meantime.\n\n> > To get you unstuck, the immediate workaround is to drop back to the\n> > older protocol, like:\n> >\n> >   git -c protocol.version=0 fetch --all\n> \n> By the way, I'd recommend the immediate workaround of\n> \n> \tgit fetch --negotiation-tip=refs/remotes/xo/* xo\n> \n> instead.  But that's a separate subject.\n\nIt seems like if we are fetching with refspec X/*:Y/* that we should\nperhaps automatically select our local Y/* negotiation tips.\n\nThat said, neither it (nor the manual version above) would help the case\nI've been testing with. It's a first fetch from \"xo\", which can reuse\nhistory we already have from other remotes.\n\nI agree it's a good workaround for folks doing their daily fetches,\nthough.\n\n-Peff\n"},{"id":"395936","messageId":"20200422194733.GA561178@coredump.intra.peff.net","threadId":"53283","inReplyTo":"20200422194047.GD558336@coredump.intra.peff.net","subject":"Re: [PATCH] Revert \"fetch: default to protocol version 2\"","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-04-22T19:47:33Z","receivedAt":"2020-04-22T19:47:36Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Apr 22, 2020 at 03:40:47PM -0400, Jeff King wrote:\n\n> > By the way, I'd recommend the immediate workaround of\n> > \n> > \tgit fetch --negotiation-tip=refs/remotes/xo/* xo\n> > \n> > instead.  But that's a separate subject.\n> \n> It seems like if we are fetching with refspec X/*:Y/* that we should\n> perhaps automatically select our local Y/* negotiation tips.\n\nActually, I guess that option is not \"please prioritize these tips\", but\nrather \"use only these tips\". So using it can sometimes hurt.\n\n> That said, neither it (nor the manual version above) would help the case\n> I've been testing with. It's a first fetch from \"xo\", which can reuse\n> history we already have from other remotes.\n> \n> I agree it's a good workaround for folks doing their daily fetches,\n> though.\n\nI did confirm that re-running my test with --negotiation-tip=master (to\npoint to Linus's tree, avoiding the clogging of \"next\" commits) results\nin the usual good pack in a reasonable time.\n\n-Peff\n"},{"id":"396044","messageId":"20200423213735.242662-1-jonathantanmy@google.com","threadId":"53283","inReplyTo":"20200422104000.GA551233@coredump.intra.peff.net","subject":"Re: Git 2.26 fetches many times more objects than it should, wasting gigabytes","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2020-04-23T21:37:35Z","receivedAt":"2020-04-23T21:37:42Z","isPatch":false,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> On Wed, Apr 22, 2020 at 06:30:11AM -0400, Jeff King wrote:\n> \n> > So it really just seems like v2 does not try hard enough. I think the\n> > culprit is the MAX_IN_VAIN setting. If I do this:\n> > \n> > diff --git a/fetch-pack.c b/fetch-pack.c\n> > index 1734a573b0..016a413d49 100644\n> > --- a/fetch-pack.c\n> > +++ b/fetch-pack.c\n> > @@ -46,7 +46,7 @@ static struct strbuf fsck_msg_types = STRBUF_INIT;\n> >   * After sending this many \"have\"s if we do not get any new ACK , we\n> >   * give up traversing our history.\n> >   */\n> > -#define MAX_IN_VAIN 256\n> > +#define MAX_IN_VAIN 20000\n> >  \n> >  static int multi_ack, use_sideband;\n> >  /* Allow specifying sha1 if it is a ref tip. */\n> > \n> > then I get that same 48k objects, 23MB fetch that v0 does.\n> \n> I don't quite think that's the solution, though. Both old and new are\n> supposed to be respecting MAX_IN_VAIN. So it's not at all clear to me\n> why it restricts the number of haves we'll send in v2, but not in v0.\n> \n> Maybe somebody more familiar with the negotiation code can comment\n> further.\n\nThanks for the reproduction recipe (in [1]) and your analysis. I took a\nlook, and it's because the check for in_vain is done differently. In v0:\n\n  if (got_continue && MAX_IN_VAIN < in_vain) {\n\nreflecting the documentation in pack-protocol.txt:\n\n  However, the 256 limit *only* turns on in the canonical client\n  implementation if we have received at least one \"ACK %s continue\"\n  during a prior round.  This helps to ensure that at least one common\n  ancestor is found before we give up entirely.\n\n(Note that both the code and the documentation call it \"continue\", but\nthe code also correctly handles multi_ack_detailed, which instructs the\nserver to send \"ACK common\" and \"ACK ready\" in lieu of \"ACK continue\".)\n\nWhen debugging, I noticed that in_vain was increasing far in excess of\nMAX_IN_VAIN, but because got_continue was false, the client did not give\nup.\n\nBut in v2:\n\n  if (!haves_added || *in_vain >= MAX_IN_VAIN) {\n\n(\"haves_added\" is irrelevant to this discussion. It is another\ntermination condition - when we have run out of \"have\"s to send.)\n\nSo there is no check that \"continue\" was sent. We probably should change\nv2 to match v0. I can start writing a patch unless someone else would\nlike to take a further look at it.\n\n[1] https://lore.kernel.org/git/20200422095702.GA475060@coredump.intra.peff.net/\n"},{"id":"396053","messageId":"xmqq7dy5g95n.fsf@gitster.c.googlers.com","threadId":"53283","inReplyTo":"20200423213735.242662-1-jonathantanmy@google.com","subject":"Re: Git 2.26 fetches many times more objects than it should, wasting gigabytes","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-04-23T21:54:44Z","receivedAt":"2020-04-23T21:54:49Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan <jonathantanmy@google.com> writes:\n\n> But in v2:\n>\n>   if (!haves_added || *in_vain >= MAX_IN_VAIN) {\n>\n> (\"haves_added\" is irrelevant to this discussion. It is another\n> termination condition - when we have run out of \"have\"s to send.)\n>\n> So there is no check that \"continue\" was sent. We probably should change\n> v2 to match v0.\n\nThat sounds like a good change.\n\nThanks.\n\n"},{"id":"396108","messageId":"20200424053204.GD1648190@coredump.intra.peff.net","threadId":"53283","inReplyTo":"20200423213735.242662-1-jonathantanmy@google.com","subject":"Re: Git 2.26 fetches many times more objects than it should, wasting gigabytes","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-04-24T05:32:04Z","receivedAt":"2020-04-24T05:32:07Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Apr 23, 2020 at 02:37:35PM -0700, Jonathan Tan wrote:\n\n> Thanks for the reproduction recipe (in [1]) and your analysis. I took a\n> look, and it's because the check for in_vain is done differently. In v0:\n> \n>   if (got_continue && MAX_IN_VAIN < in_vain) {\n> \n> reflecting the documentation in pack-protocol.txt:\n> \n>   However, the 256 limit *only* turns on in the canonical client\n>   implementation if we have received at least one \"ACK %s continue\"\n>   during a prior round.  This helps to ensure that at least one common\n>   ancestor is found before we give up entirely.\n\nAh, thanks for that; I hadn't though to look in that file for more\nclues.\n\n> When debugging, I noticed that in_vain was increasing far in excess of\n> MAX_IN_VAIN, but because got_continue was false, the client did not give\n> up.\n> \n> But in v2:\n> \n>   if (!haves_added || *in_vain >= MAX_IN_VAIN) {\n> \n> (\"haves_added\" is irrelevant to this discussion. It is another\n> termination condition - when we have run out of \"have\"s to send.)\n> \n> So there is no check that \"continue\" was sent. We probably should change\n> v2 to match v0. I can start writing a patch unless someone else would\n> like to take a further look at it.\n\nYeah, this fills in the final pieces of the puzzle I was chasing in:\n\n https://lore.kernel.org/git/20200422193324.GB558336@coredump.intra.peff.net/\n\nAnd the patch you suggest sounds like the best solution.\n\nI think there's some room for discussion about what the optimal\nstrategies are (e.g., v0 does send a lot more haves than v2 in this\ninstance, and it wouldn't always be helpful). But it makes sense to me\nto put v2 and v0 on the same footing for now, especially given the\nregressions people have mentioned, and then we can explore new options\nat our convenience (like switching on the skipping negotiation\nalgorithm).\n\n-Peff\n"}]}