{"thread":{"id":"53235","subject":"Fetching 24 Linux commits = 1.2 GiB","startedAt":"2020-04-15T08:02:10Z","lastAt":"2020-05-25T18:37:41Z","messageCount":20,"participants":["Jiri Slaby","Andreas Schwab","Kevin Daudt","Konstantin Ryabitsev","Junio C Hamano","Jeff King","Jonathan Nieder","Martin Wilck","Michal Suchánek","Lukas Wunner"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"395474","messageId":"b7f5bfb9-61fb-2552-4399-b744428728e4@suse.cz","threadId":"53235","inReplyTo":null,"subject":"Fetching 24 Linux commits = 1.2 GiB","fromName":"Jiri Slaby","fromEmail":"jslaby@suse.cz","sentAt":"2020-04-15T08:01:46Z","receivedAt":"2020-04-15T08:02:10Z","isPatch":false,"sender":{"key":"jslaby@suse.cz","avatar":null},"body":"Hi,\n\nI was at 8f3d9f354286 of:\ngit://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git\n\nI did git remote update today and it fetched:\nReceiving objects: 100% (7330823/7330823), 1.20 GiB\nIt updated master: 8f3d9f354286..8632e9b5645b, that is 24 small commits.\n\nOne colleague of mine fetched 1324 commits:\nReceiving objects: 100% (6820/6820), 4.21 MiB | 6.70 MiB/s, done.\nResolving deltas: 100% (5114/5114), completed with 1035 local objects.\nFrom git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux\n   7e63420847ae..8632e9b5645b  master     -> origin/master\n\nAnother colleague fetched the same what I and:\n  Receiving objects: 100% (7330823/7330823), 1.20 GiB\ntoo.\n\nI did git gc --prune && git prune now and I am at 1.7G back from 3.5 G.\n\nIs that a bug? What info should I provide?\n\nthanks,\n-- \njs\nsuse labs\n"},{"id":"395475","messageId":"878sixdv7a.fsf@igel.home","threadId":"53235","inReplyTo":"b7f5bfb9-61fb-2552-4399-b744428728e4@suse.cz","subject":"Re: Fetching 24 Linux commits = 1.2 GiB","fromName":"Andreas Schwab","fromEmail":"schwab@linux-m68k.org","sentAt":"2020-04-15T08:11:53Z","receivedAt":"2020-04-15T08:12:03Z","isPatch":false,"sender":{"key":"schwab@linux-m68k.org","avatar":"https://avatars.githubusercontent.com/u/2175493?v=4"},"body":"On Apr 15 2020, Jiri Slaby wrote:\n\n> I was at 8f3d9f354286 of:\n> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git\n>\n> I did git remote update today and it fetched:\n> Receiving objects: 100% (7330823/7330823), 1.20 GiB\n> It updated master: 8f3d9f354286..8632e9b5645b, that is 24 small commits.\n\nWhat's your git version?  I just did exactly the same update with git\n2.26.1, and it only fetched 144 objects:\n\nReceiving objects: 100% (144/144), 50.50 KiB | 1.87 MiB/s, done.\n\nAndreas.\n\n-- \nAndreas Schwab, schwab@linux-m68k.org\nGPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1\n\"And now for something completely different.\"\n"},{"id":"395476","messageId":"c35ac17a-fe28-684f-94de-2d2b63b7f4ee@suse.cz","threadId":"53235","inReplyTo":"878sixdv7a.fsf@igel.home","subject":"Re: Fetching 24 Linux commits = 1.2 GiB","fromName":"Jiri Slaby","fromEmail":"jslaby@suse.cz","sentAt":"2020-04-15T08:16:59Z","receivedAt":"2020-04-15T08:17:04Z","isPatch":false,"sender":{"key":"jslaby@suse.cz","avatar":null},"body":"On 15. 04. 20, 10:11, Andreas Schwab wrote:\n> On Apr 15 2020, Jiri Slaby wrote:\n> \n>> I was at 8f3d9f354286 of:\n>> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git\n>>\n>> I did git remote update today and it fetched:\n>> Receiving objects: 100% (7330823/7330823), 1.20 GiB\n>> It updated master: 8f3d9f354286..8632e9b5645b, that is 24 small commits.\n> \n> What's your git version?  I just did exactly the same update with git\n> 2.26.1, and it only fetched 144 objects:\n> \n> Receiving objects: 100% (144/144), 50.50 KiB | 1.87 MiB/s, done.\n\n$ git --version\ngit version 2.26.1\n\nThe same as you have.\n\n-- \njs\nsuse labs\n"},{"id":"395477","messageId":"874ktldunk.fsf@igel.home","threadId":"53235","inReplyTo":"c35ac17a-fe28-684f-94de-2d2b63b7f4ee@suse.cz","subject":"Re: Fetching 24 Linux commits = 1.2 GiB","fromName":"Andreas Schwab","fromEmail":"schwab@linux-m68k.org","sentAt":"2020-04-15T08:23:43Z","receivedAt":"2020-04-15T08:23:51Z","isPatch":false,"sender":{"key":"schwab@linux-m68k.org","avatar":"https://avatars.githubusercontent.com/u/2175493?v=4"},"body":"On Apr 15 2020, Jiri Slaby wrote:\n\n> On 15. 04. 20, 10:11, Andreas Schwab wrote:\n>> On Apr 15 2020, Jiri Slaby wrote:\n>> \n>>> I was at 8f3d9f354286 of:\n>>> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git\n>>>\n>>> I did git remote update today and it fetched:\n>>> Receiving objects: 100% (7330823/7330823), 1.20 GiB\n>>> It updated master: 8f3d9f354286..8632e9b5645b, that is 24 small commits.\n>> \n>> What's your git version?  I just did exactly the same update with git\n>> 2.26.1, and it only fetched 144 objects:\n>> \n>> Receiving objects: 100% (144/144), 50.50 KiB | 1.87 MiB/s, done.\n>\n> $ git --version\n> git version 2.26.1\n\nWas this the first time you used git >= 2.26.0?\n\nAndreas.\n\n-- \nAndreas Schwab, schwab@linux-m68k.org\nGPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1\n\"And now for something completely different.\"\n"},{"id":"395478","messageId":"89363e9d-8739-3061-3d81-806099005e3f@suse.cz","threadId":"53235","inReplyTo":"874ktldunk.fsf@igel.home","subject":"Re: Fetching 24 Linux commits = 1.2 GiB","fromName":"Jiri Slaby","fromEmail":"jslaby@suse.cz","sentAt":"2020-04-15T08:27:07Z","receivedAt":"2020-04-15T08:27:39Z","isPatch":false,"sender":{"key":"jslaby@suse.cz","avatar":null},"body":"On 15. 04. 20, 10:23, Andreas Schwab wrote:\n> On Apr 15 2020, Jiri Slaby wrote:\n> \n>> On 15. 04. 20, 10:11, Andreas Schwab wrote:\n>>> On Apr 15 2020, Jiri Slaby wrote:\n>>>\n>>>> I was at 8f3d9f354286 of:\n>>>> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git\n>>>>\n>>>> I did git remote update today and it fetched:\n>>>> Receiving objects: 100% (7330823/7330823), 1.20 GiB\n>>>> It updated master: 8f3d9f354286..8632e9b5645b, that is 24 small commits.\n>>>\n>>> What's your git version?  I just did exactly the same update with git\n>>> 2.26.1, and it only fetched 144 objects:\n>>>\n>>> Receiving objects: 100% (144/144), 50.50 KiB | 1.87 MiB/s, done.\n>>\n>> $ git --version\n>> git version 2.26.1\n> \n> Was this the first time you used git >= 2.26.0?\n\nI doubt that:\n$ grep git-core /var/log/zypp/history|sed 's@x86_64.*@@'\n2020-03-23 16:14:24|install|git-core|2.25.2-467.2|\n2020-03-24 17:19:37|install|git-core|2.26.0-468.1|\n2020-03-29 09:38:34|install|git-core|2.26.0-471.1|\n2020-04-06 06:34:36|install|git-core|2.26.0-473.1|\n2020-04-15 08:21:37|install|git-core|2.26.1-474.1|\n\nThis is the first time I used 2.26.1, though.\n\nthanks,\n-- \njs\nsuse labs\n"},{"id":"395480","messageId":"87zhbdcflb.fsf@igel.home","threadId":"53235","inReplyTo":"89363e9d-8739-3061-3d81-806099005e3f@suse.cz","subject":"Re: Fetching 24 Linux commits = 1.2 GiB","fromName":"Andreas Schwab","fromEmail":"schwab@linux-m68k.org","sentAt":"2020-04-15T08:34:24Z","receivedAt":"2020-04-15T08:34:37Z","isPatch":false,"sender":{"key":"schwab@linux-m68k.org","avatar":"https://avatars.githubusercontent.com/u/2175493?v=4"},"body":"On Apr 15 2020, Jiri Slaby wrote:\n\n> This is the first time I used 2.26.1, though.\n\nSame for me.\n\nAndreas.\n\n-- \nAndreas Schwab, schwab@linux-m68k.org\nGPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1\n\"And now for something completely different.\"\n"},{"id":"395488","messageId":"20200415105247.GA1710231@alpha","threadId":"53235","inReplyTo":"b7f5bfb9-61fb-2552-4399-b744428728e4@suse.cz","subject":"Re: Fetching 24 Linux commits = 1.2 GiB","fromName":"Kevin Daudt","fromEmail":"me@ikke.info","sentAt":"2020-04-15T10:52:47Z","receivedAt":"2020-04-15T11:10:30Z","isPatch":false,"sender":{"key":"me@ikke.info","avatar":"https://avatars.githubusercontent.com/u/135698?v=4"},"body":"On Wed, Apr 15, 2020 at 10:01:46AM +0200, Jiri Slaby wrote:\n> Hi,\n> \n> I was at 8f3d9f354286 of:\n> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git\n> \n> I did git remote update today and it fetched:\n> Receiving objects: 100% (7330823/7330823), 1.20 GiB\n> It updated master: 8f3d9f354286..8632e9b5645b, that is 24 small commits.\n> \n> One colleague of mine fetched 1324 commits:\n> Receiving objects: 100% (6820/6820), 4.21 MiB | 6.70 MiB/s, done.\n> Resolving deltas: 100% (5114/5114), completed with 1035 local objects.\n> From git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux\n>    7e63420847ae..8632e9b5645b  master     -> origin/master\n> \n> Another colleague fetched the same what I and:\n>   Receiving objects: 100% (7330823/7330823), 1.20 GiB\n> too.\n> \n> I did git gc --prune && git prune now and I am at 1.7G back from 3.5 G.\n> \n> Is that a bug? What info should I provide?\n> \n> thanks,\n> -- \n> js\n> suse labs\n\nNot as big as you report, but I recall a user on IRC (guardian) was\nwondering as well why a packfile was 240MB while they claimed that the\nobjects were committed months ago.\n\n\n"},{"id":"395492","messageId":"20200415135627.vx75hsphbpmgrquv@chatter.i7.local","threadId":"53235","inReplyTo":"b7f5bfb9-61fb-2552-4399-b744428728e4@suse.cz","subject":"Re: Fetching 24 Linux commits = 1.2 GiB","fromName":"Konstantin Ryabitsev","fromEmail":"konstantin@linuxfoundation.org","sentAt":"2020-04-15T13:56:27Z","receivedAt":"2020-04-15T13:57:02Z","isPatch":false,"sender":{"key":"konstantin@linuxfoundation.org","avatar":"https://gravatar.com/avatar/7cb8827c6de56e1bd2dea16508c6708aa43feed3bf3813bcdacecdf96ceadd79?d=mp&s=160"},"body":"On Wed, Apr 15, 2020 at 10:01:46AM +0200, Jiri Slaby wrote:\n> Hi,\n> \n> I was at 8f3d9f354286 of:\n> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git\n> \n> I did git remote update today and it fetched:\n> Receiving objects: 100% (7330823/7330823), 1.20 GiB\n> It updated master: 8f3d9f354286..8632e9b5645b, that is 24 small commits.\n> \n> One colleague of mine fetched 1324 commits:\n> Receiving objects: 100% (6820/6820), 4.21 MiB | 6.70 MiB/s, done.\n> Resolving deltas: 100% (5114/5114), completed with 1035 local objects.\n> From git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux\n>    7e63420847ae..8632e9b5645b  master     -> origin/master\n> \n> Another colleague fetched the same what I and:\n>   Receiving objects: 100% (7330823/7330823), 1.20 GiB\n> too.\n> \n> I did git gc --prune && git prune now and I am at 1.7G back from 3.5 G.\n> \n> Is that a bug? What info should I provide?\n\nI've helped sfr troubleshoot the same issue last week -- it's most \nlikely due to 2.26 turning on protocol version=2 by default.  \nUnfortunately, reproducing this has been tricky, so if you can reliably \nmake this happen again, then providing a full copy of your local tree as \nwell as the remote you're trying to fetch may greatly help narrow it \ndown.\n\nWith sfr (for whom fetching 1.2G from .au is a bit of a big deal), we \nsolved it by forcing protocol.version=1.\n\n-K\n"},{"id":"395499","messageId":"xmqq1roo947y.fsf@gitster.c.googlers.com","threadId":"53235","inReplyTo":"20200415135627.vx75hsphbpmgrquv@chatter.i7.local","subject":"Re: Fetching 24 Linux commits = 1.2 GiB","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-04-15T15:08:17Z","receivedAt":"2020-04-15T15:08:27Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Do these (and I think we saw other reports) make us rethink the\nstatus of protocol v2 as the default?  Are all of these fallouts \nwe saw so far easy-to-fix bugs, or are there more fundamental issues\nin the v2 protocol design?\n\nThanks.\n\nKonstantin Ryabitsev <konstantin@linuxfoundation.org> writes:\n\n> On Wed, Apr 15, 2020 at 10:01:46AM +0200, Jiri Slaby wrote:\n>> Hi,\n>> \n>> I was at 8f3d9f354286 of:\n>> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git\n>> \n>> I did git remote update today and it fetched:\n>> Receiving objects: 100% (7330823/7330823), 1.20 GiB\n>> It updated master: 8f3d9f354286..8632e9b5645b, that is 24 small commits.\n>> \n>> One colleague of mine fetched 1324 commits:\n>> Receiving objects: 100% (6820/6820), 4.21 MiB | 6.70 MiB/s, done.\n>> Resolving deltas: 100% (5114/5114), completed with 1035 local objects.\n>> From git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux\n>>    7e63420847ae..8632e9b5645b  master     -> origin/master\n>> \n>> Another colleague fetched the same what I and:\n>>   Receiving objects: 100% (7330823/7330823), 1.20 GiB\n>> too.\n>> \n>> I did git gc --prune && git prune now and I am at 1.7G back from 3.5 G.\n>> \n>> Is that a bug? What info should I provide?\n>\n> I've helped sfr troubleshoot the same issue last week -- it's most \n> likely due to 2.26 turning on protocol version=2 by default.  \n> Unfortunately, reproducing this has been tricky, so if you can reliably \n> make this happen again, then providing a full copy of your local tree as \n> well as the remote you're trying to fetch may greatly help narrow it \n> down.\n>\n> With sfr (for whom fetching 1.2G from .au is a bit of a big deal), we \n> solved it by forcing protocol.version=1.\n>\n> -K\n"},{"id":"395501","messageId":"20200415151652.GB2464307@coredump.intra.peff.net","threadId":"53235","inReplyTo":"xmqq1roo947y.fsf@gitster.c.googlers.com","subject":"Re: Fetching 24 Linux commits = 1.2 GiB","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-04-15T15:16:52Z","receivedAt":"2020-04-15T15:16:56Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Apr 15, 2020 at 08:08:17AM -0700, Junio C Hamano wrote:\n\n> Do these (and I think we saw other reports) make us rethink the\n> status of protocol v2 as the default?  Are all of these fallouts \n> we saw so far easy-to-fix bugs, or are there more fundamental issues\n> in the v2 protocol design?\n\nI don't think we know yet.\n\nI agree with Konstantin that the v2 switch is the likely culprit for\nthese issues, but without having been able to reproduce, I don't think\nwe know exactly what the problem is yet. It could be a protocol design\nissue, or it could be a minor implementation bug.\n\nNote that there is one other issue that's turned up, that I discussed\nhere:\n\n  https://lore.kernel.org/git/20200328154936.GA1217052@coredump.intra.peff.net/\n\nThat's more fundamental to the v2 design, but:\n\n  - it only happens when one side drops the connection, so it's not\n    impacting normal operation (it does turn an error case into a hang,\n    though, which can be rather annoying)\n\n  - it's not in the network protocol itself, but rather the protocol\n    between Git and the remote helper. So we could solve it purely as a\n    client-side fix.\n\n-Peff\n"},{"id":"395523","messageId":"20200415185230.GA216285@google.com","threadId":"53235","inReplyTo":"xmqq1roo947y.fsf@gitster.c.googlers.com","subject":"Re: Fetching 24 Linux commits = 1.2 GiB","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2020-04-15T18:52:30Z","receivedAt":"2020-04-15T19:17:02Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\n>> On Wed, Apr 15, 2020 at 10:01:46AM +0200, Jiri Slaby wrote:\n\n>>> I was at 8f3d9f354286 of:\n>>> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git\n>>>\n>>> I did git remote update today and it fetched:\n>>> Receiving objects: 100% (7330823/7330823), 1.20 GiB\n>>> It updated master: 8f3d9f354286..8632e9b5645b, that is 24 small commits.\n\nI suspect this is due to negotiation differing in protocol v2: in\nprotocols that do not maintain server side state, we have to record\npreviously matched \"have\"s at each round and the number of additional\n\"have\"s sent on top leads the server to have insufficient information\nabout what the client has.  In other words, I suspect that with\nhttps:// in protocol v0 you would experience the same thing.\n\nDoes\n\n\tgit config --global fetch.negotiationStrategy skipping\n\nhelp?\n\nThanks,\nJonathan\n"},{"id":"395575","messageId":"a5bce92f-ad19-75d4-1e34-dacc73321536@suse.cz","threadId":"53235","inReplyTo":"20200415135627.vx75hsphbpmgrquv@chatter.i7.local","subject":"Re: Fetching 24 Linux commits = 1.2 GiB","fromName":"Jiri Slaby","fromEmail":"jslaby@suse.cz","sentAt":"2020-04-16T06:31:55Z","receivedAt":"2020-04-16T06:32:07Z","isPatch":false,"sender":{"key":"jslaby@suse.cz","avatar":null},"body":"On 15. 04. 20, 15:56, Konstantin Ryabitsev wrote:\n> On Wed, Apr 15, 2020 at 10:01:46AM +0200, Jiri Slaby wrote:\n>> Hi,\n>>\n>> I was at 8f3d9f354286 of:\n>> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git\n>>\n>> I did git remote update today and it fetched:\n>> Receiving objects: 100% (7330823/7330823), 1.20 GiB\n>> It updated master: 8f3d9f354286..8632e9b5645b, that is 24 small commits.\n>>\n>> One colleague of mine fetched 1324 commits:\n>> Receiving objects: 100% (6820/6820), 4.21 MiB | 6.70 MiB/s, done.\n>> Resolving deltas: 100% (5114/5114), completed with 1035 local objects.\n>> From git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux\n>>    7e63420847ae..8632e9b5645b  master     -> origin/master\n>>\n>> Another colleague fetched the same what I and:\n>>   Receiving objects: 100% (7330823/7330823), 1.20 GiB\n>> too.\n>>\n>> I did git gc --prune && git prune now and I am at 1.7G back from 3.5 G.\n>>\n>> Is that a bug? What info should I provide?\n> \n> I've helped sfr troubleshoot the same issue last week -- it's most \n> likely due to 2.26 turning on protocol version=2 by default.  \n> Unfortunately, reproducing this has been tricky, so if you can reliably \n> make this happen again, then providing a full copy of your local tree as \n> well as the remote you're trying to fetch may greatly help narrow it \n> down.\n\nI tried hard, but cannot reproduce. I noticed a difference between\n2.25.1 and 2.25.1+protocol.version=2, though:\n\n$ git config protocol.version 1 # the default in 2.25\n$ git fetch\ngit://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git\n8f3d9f354286745c751374f5f1fcafee6b3f3136\nerror: Server does not allow request for unadvertised object\n8f3d9f354286745c751374f5f1fcafee6b3f3136\n$ git config protocol.version 2\n$ git fetch\ngit://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git\n8f3d9f354286745c751374f5f1fcafee6b3f3136\nremote: Enumerating objects: 1433262, done.\n...\n\nDoing fetch v5.7-rc1 (which is 8f3d9f3 above) with proto 1 works. So the\nserver obviously advertises different set of objects with proto 1 and 2.\n\nthanks,\n-- \njs\nsuse labs\n"},{"id":"395817","messageId":"6cb9d009026cfb9918b2ceab3e43ec1cbc303de4.camel@suse.com","threadId":"53235","inReplyTo":"a5bce92f-ad19-75d4-1e34-dacc73321536@suse.cz","subject":"Re: Fetching 24 Linux commits = 1.2 GiB","fromName":"Martin Wilck","fromEmail":"mwilck@suse.com","sentAt":"2020-04-21T10:02:45Z","receivedAt":"2020-04-21T10:02:50Z","isPatch":false,"sender":{"key":"mwilck@suse.com","avatar":null},"body":"On Thu, 2020-04-16 at 08:31 +0200, Jiri Slaby wrote:\n> On 15. 04. 20, 15:56, Konstantin Ryabitsev wrote:\n> > \n> I tried hard, but cannot reproduce. I noticed a difference between\n> 2.25.1 and 2.25.1+protocol.version=2, though:\n> \n> $ git config protocol.version 1 # the default in 2.25\n> $ git fetch\n> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git\n> 8f3d9f354286745c751374f5f1fcafee6b3f3136\n> error: Server does not allow request for unadvertised object\n> 8f3d9f354286745c751374f5f1fcafee6b3f3136\n> $ git config protocol.version 2\n> $ git fetch\n> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git\n> 8f3d9f354286745c751374f5f1fcafee6b3f3136\n> remote: Enumerating objects: 1433262, done.\n> ...\n> \n> Doing fetch v5.7-rc1 (which is 8f3d9f3 above) with proto 1 works. So\n> the\n> server obviously advertises different set of objects with proto 1 and\n> 2.\n\nI just had the same issue with several kernel repos. The affected repo \nis a bare clone of Linus' kernel repo, plus \"stable\" and various\nmaintainer repos as additional remotes.\n\nInterestingly, a single \"git fetch --dry-run\" with protol version 1\n\"fixed\" the issue. I hadn't expected that to happen, so unfortunately I\nhadn't made a  backup of the repo before. As this is a 50GB bare repo,\nI'd appreciate instructions on how exactly to archive/upload it for\ndebugging if this should happen again.\n\nmwilck@apollon:linux.git[BARE:master]> git fetch --dry-run -v net\nLooking up git.kernel.org ... done.\nConnecting to git.kernel.org (port 9418) ... 136.144.49.103 done.\nremote: Enumerating objects: 7331255, done.\nremote: Counting objects: 100% (7331255/7331255), done.\nremote: Compressing objects: 100% (1113751/1113751), done.\n^C\nmwilck@apollon:linux.git[BARE:master]> git config --get\nprotocol.version\nmwilck@apollon:linux.git[BARE:master]> git config protocol.version 1\nmwilck@apollon:linux.git[BARE:master]> git fetch --dry-run -v net\nLooking up git.kernel.org ... done.\nConnecting to git.kernel.org (port 9418) ... 136.144.49.103 done.\nremote: Enumerating objects: 71, done.\nremote: Counting objects: 100% (71/71), done.\nremote: Compressing objects: 100% (48/48), done.\nremote: Total 53 (delta 42), reused 0 (delta 0)\nUnpacking objects: 100% (53/53), 8.06 KiB | 5.00 KiB/s, done.\n^C\n\n(this fixed the issue despite \"--dry-run\" and myself interrupting the\nprocess).\n\nmwilck@apollon:linux.git[BARE:master]> git config protocol.version 2\nmwilck@apollon:linux.git[BARE:master]> git fetch --dry-run -v net\nLooking up git.kernel.org ... done.\nConnecting to git.kernel.org (port 9418) ... 136.144.49.103 done.\nFrom git://git.kernel.org/pub/scm/linux/kernel/git/davem/net\n   9bacd25..a460fc5      master       -> net/master\n = [up to date]          v2.6.11      -> v2.6.11\n...\n\nRegards\nMartin\n\n\n"},{"id":"398240","messageId":"87sgfvq967.fsf@igel.home","threadId":"53235","inReplyTo":"xmqq1roo947y.fsf@gitster.c.googlers.com","subject":"Re: Fetching 24 Linux commits = 1.2 GiB","fromName":"Andreas Schwab","fromEmail":"schwab@linux-m68k.org","sentAt":"2020-05-20T08:53:36Z","receivedAt":"2020-05-20T08:53:46Z","isPatch":false,"sender":{"key":"schwab@linux-m68k.org","avatar":"https://avatars.githubusercontent.com/u/2175493?v=4"},"body":"On Apr 15 2020, Junio C Hamano wrote:\n\n> Do these (and I think we saw other reports) make us rethink the\n> status of protocol v2 as the default?  Are all of these fallouts \n> we saw so far easy-to-fix bugs, or are there more fundamental issues\n> in the v2 protocol design?\n\nI'm now seeing the issue myself, and can provide a backup of the\noffending repository.\n\n$ git count-objects -v\ncount: 17\nsize: 76\nin-pack: 387240\npacks: 37\nsize-pack: 203738\nprune-packable: 0\ngarbage: 0\nsize-garbage: 0\nalternate: /home/andreas/src/linux/git/torvalds/linux.git/objects\nalternate: /home/andreas/src/linux/git/stable/linux-stable.git/objects\n$ GIT_TRACE=1 git fetch\n10:40:32.829450 git.c:439               trace: built-in: git fetch\n10:40:33.133448 run-command.c:663       trace: run_command: unset GIT_DIR GIT_IMPLICIT_WORK_TREE GIT_PREFIX; git --git-dir=/daten/src/linux/git/torvalds/linux.git for-each-ref '--format=%(objectname)'\n10:40:33.135756 git.c:439               trace: built-in: git for-each-ref '--format=%(objectname)'\n10:40:33.143936 run-command.c:663       trace: run_command: unset GIT_DIR GIT_IMPLICIT_WORK_TREE GIT_PREFIX; git --git-dir=/daten/src/linux/git/stable/linux-stable.git for-each-ref '--format=%(objectname)'\n10:40:33.146087 git.c:439               trace: built-in: git for-each-ref '--format=%(objectname)'\nremote: Enumerating objects: 30796, done.\nremote: Counting objects: 100% (30796/30796), done.\nremote: Compressing objects: 100% (6965/6965), done.\n10:40:40.102198 run-command.c:663       trace: run_command: git index-pack --stdin -v --fix-thin '--keep=fetch-pack 12342 on igel.home' --pack_header=2,7350969\n10:40:40.104872 git.c:439               trace: built-in: git index-pack --stdin -v --fix-thin '--keep=fetch-pack 12342 on igel.home' --pack_header=2,7350969\n^Cceiving objects:   0% (3092/7350969), 1.32 MiB | 891.00 KiB/s  \n\nAndreas.\n\n-- \nAndreas Schwab, schwab@linux-m68k.org\nGPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1\n\"And now for something completely different.\"\n"},{"id":"398241","messageId":"87o8qjq90d.fsf@igel.home","threadId":"53235","inReplyTo":"87sgfvq967.fsf@igel.home","subject":"Re: Fetching 24 Linux commits = 1.2 GiB","fromName":"Andreas Schwab","fromEmail":"schwab@linux-m68k.org","sentAt":"2020-05-20T08:57:06Z","receivedAt":"2020-05-20T08:57:12Z","isPatch":false,"sender":{"key":"schwab@linux-m68k.org","avatar":"https://avatars.githubusercontent.com/u/2175493?v=4"},"body":"On Mai 20 2020, Andreas Schwab wrote:\n\n> On Apr 15 2020, Junio C Hamano wrote:\n>\n>> Do these (and I think we saw other reports) make us rethink the\n>> status of protocol v2 as the default?  Are all of these fallouts \n>> we saw so far easy-to-fix bugs, or are there more fundamental issues\n>> in the v2 protocol design?\n>\n> I'm now seeing the issue myself, and can provide a backup of the\n> offending repository.\n>\n> $ git count-objects -v\n> count: 17\n> size: 76\n> in-pack: 387240\n> packs: 37\n> size-pack: 203738\n> prune-packable: 0\n> garbage: 0\n> size-garbage: 0\n> alternate: /home/andreas/src/linux/git/torvalds/linux.git/objects\n> alternate: /home/andreas/src/linux/git/stable/linux-stable.git/objects\n> $ GIT_TRACE=1 git fetch\n> 10:40:32.829450 git.c:439               trace: built-in: git fetch\n> 10:40:33.133448 run-command.c:663       trace: run_command: unset GIT_DIR GIT_IMPLICIT_WORK_TREE GIT_PREFIX; git --git-dir=/daten/src/linux/git/torvalds/linux.git for-each-ref '--format=%(objectname)'\n> 10:40:33.135756 git.c:439               trace: built-in: git for-each-ref '--format=%(objectname)'\n> 10:40:33.143936 run-command.c:663       trace: run_command: unset GIT_DIR GIT_IMPLICIT_WORK_TREE GIT_PREFIX; git --git-dir=/daten/src/linux/git/stable/linux-stable.git for-each-ref '--format=%(objectname)'\n> 10:40:33.146087 git.c:439               trace: built-in: git for-each-ref '--format=%(objectname)'\n> remote: Enumerating objects: 30796, done.\n> remote: Counting objects: 100% (30796/30796), done.\n> remote: Compressing objects: 100% (6965/6965), done.\n> 10:40:40.102198 run-command.c:663       trace: run_command: git index-pack --stdin -v --fix-thin '--keep=fetch-pack 12342 on igel.home' --pack_header=2,7350969\n> 10:40:40.104872 git.c:439               trace: built-in: git index-pack --stdin -v --fix-thin '--keep=fetch-pack 12342 on igel.home' --pack_header=2,7350969\n> ^Cceiving objects:   0% (3092/7350969), 1.32 MiB | 891.00 KiB/s  \n\n$ git -c protocol.version=1 fetch\nremote: Enumerating objects: 13226, done.\nremote: Counting objects: 100% (10881/10881), done.\nremote: Compressing objects: 100% (2322/2322), done.\nremote: Total 10276 (delta 7860), reused 10215 (delta 7818), pack-reused 0\nReceiving objects: 100% (10276/10276), 4.13 MiB | 615.00 KiB/s, done.\nResolving deltas: 100% (7860/7860), completed with 192 local objects.\nFrom git://git.kernel.org/pub/scm/linux/kernel/git/geert/linux-m68k\n + c7630f6386a3...4684d38e4d47 m68k-queue -> m68k-queue  (forced update)\n   39d91d3c3b47..e886fc082483  master     -> master\n\nAndreas.\n\n-- \nAndreas Schwab, schwab@linux-m68k.org\nGPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1\n\"And now for something completely different.\"\n"},{"id":"398242","messageId":"20200520100528.GV25173@kitsune.suse.cz","threadId":"53235","inReplyTo":"87sgfvq967.fsf@igel.home","subject":"Re: Fetching 24 Linux commits = 1.2 GiB","fromName":"Michal Suchánek","fromEmail":"msuchanek@suse.de","sentAt":"2020-05-20T10:05:28Z","receivedAt":"2020-05-20T10:05:33Z","isPatch":false,"sender":{"key":"msuchanek@suse.de","avatar":"https://avatars.githubusercontent.com/u/787652?v=4"},"body":"On Wed, May 20, 2020 at 10:53:36AM +0200, Andreas Schwab wrote:\n> On Apr 15 2020, Junio C Hamano wrote:\n> \n> > Do these (and I think we saw other reports) make us rethink the\n> > status of protocol v2 as the default?  Are all of these fallouts \n> > we saw so far easy-to-fix bugs, or are there more fundamental issues\n> > in the v2 protocol design?\n> \n> I'm now seeing the issue myself, and can provide a backup of the\n> offending repository.\nWhat git version? Should be fixed in git 2.26.2 in SUSE and git 2.26.3\nupstream.\n\nThanks\n\nMichal\n"},{"id":"398243","messageId":"87k116rj1x.fsf@igel.home","threadId":"53235","inReplyTo":"20200520100528.GV25173@kitsune.suse.cz","subject":"Re: Fetching 24 Linux commits = 1.2 GiB","fromName":"Andreas Schwab","fromEmail":"schwab@linux-m68k.org","sentAt":"2020-05-20T10:34:50Z","receivedAt":"2020-05-20T10:34:56Z","isPatch":false,"sender":{"key":"schwab@linux-m68k.org","avatar":"https://avatars.githubusercontent.com/u/2175493?v=4"},"body":"On Mai 20 2020, Michal Suchánek wrote:\n\n> What git version?\n\n2.26.2\n\n> git 2.26.3\n\nI don't see that anywhere.\n\nAndreas.\n\n-- \nAndreas Schwab, schwab@linux-m68k.org\nGPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1\n\"And now for something completely different.\"\n"},{"id":"398290","messageId":"20200520194019.GA340985@coredump.intra.peff.net","threadId":"53235","inReplyTo":"87sgfvq967.fsf@igel.home","subject":"Re: Fetching 24 Linux commits = 1.2 GiB","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-05-20T19:40:19Z","receivedAt":"2020-05-20T19:40:22Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, May 20, 2020 at 10:53:36AM +0200, Andreas Schwab wrote:\n\n> On Apr 15 2020, Junio C Hamano wrote:\n> \n> > Do these (and I think we saw other reports) make us rethink the\n> > status of protocol v2 as the default?  Are all of these fallouts \n> > we saw so far easy-to-fix bugs, or are there more fundamental issues\n> > in the v2 protocol design?\n> \n> I'm now seeing the issue myself, and can provide a backup of the\n> offending repository.\n\nThe \"too big fetch\" issue has since been fixed in \"master\", as well as\nreverting the switch to the v2 protocol (which I think is just\nbelt-and-suspenders; AFAIK there are no known issues after the fix).\nBoth will be in v2.27. I don't see anything on \"maint\", but they _could_\nbe part of an eventual v2.26.3.\n\nThe fix was merged in 0b07eecf6e (Merge branch 'jt/v2-fetch-nego-fix',\n2020-05-01) for reference.\n\n-Peff\n"},{"id":"398502","messageId":"20200525182205.7hcffndaro2pgink@wunner.de","threadId":"53235","inReplyTo":"20200520194019.GA340985@coredump.intra.peff.net","subject":"Re: Fetching 24 Linux commits = 1.2 GiB","fromName":"Lukas Wunner","fromEmail":"lukas@wunner.de","sentAt":"2020-05-25T18:22:05Z","receivedAt":"2020-05-25T18:22:11Z","isPatch":false,"sender":{"key":"lukas@wunner.de","avatar":null},"body":"On Wed, May 20, 2020 at 03:40:19PM -0400, Jeff King wrote:\n> The \"too big fetch\" issue has since been fixed in \"master\", as well as\n> reverting the switch to the v2 protocol (which I think is just\n> belt-and-suspenders; AFAIK there are no known issues after the fix).\n> Both will be in v2.27. I don't see anything on \"maint\", but they _could_\n> be part of an eventual v2.26.3.\n> \n> The fix was merged in 0b07eecf6e (Merge branch 'jt/v2-fetch-nego-fix',\n> 2020-05-01) for reference.\n\nPlease consider cutting a v2.26.3 release with this fix at your\nearliest convenience.  The waste of bandwidth is mind-boggling.\n(> 1 GByte whenever fetching from a kernel remote.)\n\nThanks,\n\nLukas\n"},{"id":"398503","messageId":"xmqq4ks3q2s4.fsf@gitster.c.googlers.com","threadId":"53235","inReplyTo":"20200525182205.7hcffndaro2pgink@wunner.de","subject":"Re: Fetching 24 Linux commits = 1.2 GiB","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-05-25T18:37:31Z","receivedAt":"2020-05-25T18:37:41Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Lukas Wunner <lukas@wunner.de> writes:\n\n> On Wed, May 20, 2020 at 03:40:19PM -0400, Jeff King wrote:\n>> The \"too big fetch\" issue has since been fixed in \"master\", as well as\n>> reverting the switch to the v2 protocol (which I think is just\n>> belt-and-suspenders; AFAIK there are no known issues after the fix).\n>> Both will be in v2.27. I don't see anything on \"maint\", but they _could_\n>> be part of an eventual v2.26.3.\n>> \n>> The fix was merged in 0b07eecf6e (Merge branch 'jt/v2-fetch-nego-fix',\n>> 2020-05-01) for reference.\n>\n> Please consider cutting a v2.26.3 release with this fix at your\n> earliest convenience.  The waste of bandwidth is mind-boggling.\n> (> 1 GByte whenever fetching from a kernel remote.)\n\nIn the meantime, v2.27.0 rc2 will be out tomorrow.  Please consider\nhelping us polish it by testing that version.\n\nThanks.\n"}]}