{"thread":{"id":"55608","subject":"A couple issues with check-connectivity on fetch","startedAt":"2021-05-02T22:21:11Z","lastAt":"2021-05-02T22:21:11Z","messageCount":1,"participants":["Mike Hommey"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"423450","messageId":"20210502215316.2y4vaedxn2j5gouc@glandium.org","threadId":"55608","inReplyTo":null,"subject":"A couple issues with check-connectivity on fetch","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2021-05-02T21:53:16Z","receivedAt":"2021-05-02T22:21:11Z","isPatch":false,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"Hi,\n\nIn the context of a remote helper using the `import` capability, or one\nusing the `fetch` capability but the without `check-connectivity`\ncapability, a large amount of time may be spent in check_connected() on\nfetch, on large repos with a large number of fetched refs that also\nhappen not to actually add commits.\n\n[A little bit of background: in git-cinnabar, which allows to talk to\nMercurial repositories, tags are fetched from a different remote, so\nwhen one wants to fetch tags, the transport system never calls `import`\nbecause the `list` ended up giving a large list of refs with resolved\nsha1s, all of which are already locally available]\n\nThe first thing I noticed is that check_connected() is called twice\nduring the fetch: once from check_exist_and_connected(), and once from\nstore_updated_refs(). When each call is slow, it makes things doubly so.\nI'm not sure why check_connected() is called twice like this.  It seems\nit could be an oversight, but I could be missing something.\n\nThe second thing is that git-cinnabar uses the `import` capability,\nand in that case, the transport layer never tried to use the\n`check-connectivity` capability. I'm not sure whether the capability\nshould be exposed there, or if the checks should be skipped altogether\nassuming `import` helpers use `fast-import` (although, technically,\ngit-cinnabar doesn't use the one provided by git).\n\nBut in any case, the `check-connectivity` capability doesn't actually\nmatter because no `fetch` or `import` even happens: fetch only does a\n`list` and uses that result. In this case, it would seem\n`check_connected` should be skipped entirely.\n\nHere is a testcase that takes >20s to fetch on my machine (see\ninstructions at the beginning of the file)\nhttp://glandium.org/files/git-check-connectivity-testcase\n(the equivalent takes > 1 minute with git-cinnabar because of the\nextra refs for git-cinnabar's metadata ; note the testcase is using\nthe `fetch` capability)\n\nA workaround is to create temporary refs for those tags. That makes\ncheck_connected()'s git-rev-list work quickly, and the fetch takes < 1s.\n\nI'm ready to send patches to improve the situation, but I wanted some\nfeedback first about the right things that would need to change.\n\nThoughts?\n\nMike\n"}]}