{"thread":{"id":"65454","subject":"Git 2.54.0-rc1, subtests of t5310, t5326, t5327","startedAt":"2026-04-07T23:30:02Z","lastAt":"2026-04-10T08:10:19Z","messageCount":29,"participants":["rsbecker@nexbridge.com","Jeff King","Junio C Hamano","brian m. carlson","Patrick Steinhardt","Phillip Wood","Johannes Sixt"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"541110","messageId":"00f401dcc6e6$7183c0f0$548b42d0$@nexbridge.com","threadId":"65454","inReplyTo":null,"subject":"Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2026-04-07T23:29:50Z","receivedAt":"2026-04-07T23:30:02Z","isPatch":false,"body":"I can getting numerous issues in t5310, t5326, t2527 relating to the\nfollowing use of --git-dir:\n\nIn t5310:\nfatal: not a git repository: 'clone.git'\nnot ok 55 - fetch (full bitmap)\n#\n#                       git --git-dir=clone.git fetch origin second:second\n&&\n#                       git rev-parse HEAD >expect &&\n#                       git --git-dir=clone.git rev-parse HEAD >actual &&\n#                       test_cmp expect actual\n#\n\nIn t5326 and t5327:\nfatal: writev error: Invalid function argument\nfetch-pack: unexpected disconnect while reading sideband packet\nfatal: early EOF\nfatal: fetch-pack: invalid index-pack output\nnot ok 24 - clone from bitmapped repository\n#\n#                       rm -fr clone.git &&\n#                       git clone --no-local --bare . clone.git &&\n#                       git rev-parse HEAD >expect &&\n#                       git --git-dir=clone.git rev-parse HEAD >actual &&\n#                       test_cmp expect actual\n#\n\n\nIt appears that the --git-dir argument is not working properly in this\nrelease. Any opinions?\n\nI would have expected HOME or .gitconfig to have been set or redirected\nelsewhere in the test.\n\n--Randall\n\n\n--\nBrief whoami: NonStop&UNIX developer since approximately\nUNIX(421664400)\nNonStop(211288444200000000)\n-- In real life, I talk too much.\n\n\n"},{"id":"541114","messageId":"20260408041716.GA1324339@coredump.intra.peff.net","threadId":"65454","inReplyTo":"00f401dcc6e6$7183c0f0$548b42d0$@nexbridge.com","subject":"Re: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2026-04-08T04:17:16Z","receivedAt":"2026-04-08T04:17:17Z","isPatch":false,"body":"On Tue, Apr 07, 2026 at 07:29:50PM -0400, rsbecker@nexbridge.com wrote:\n\n> I can getting numerous issues in t5310, t5326, t2527 relating to the\n> following use of --git-dir:\n> \n> In t5310:\n> fatal: not a git repository: 'clone.git'\n> not ok 55 - fetch (full bitmap)\n> #\n> #                       git --git-dir=clone.git fetch origin second:second\n> &&\n> #                       git rev-parse HEAD >expect &&\n> #                       git --git-dir=clone.git rev-parse HEAD >actual &&\n> #                       test_cmp expect actual\n> #\n\nThis test hasn't changed recently. The clone.git directory should have\nbeen created by an earlier test. Can you try running with \"-i\" and make\nsure that this is the first failing test, and we didn't fail earlier?\n\nEspecially because...\n\n> In t5326 and t5327:\n> fatal: writev error: Invalid function argument\n> fetch-pack: unexpected disconnect while reading sideband packet\n> fatal: early EOF\n> fatal: fetch-pack: invalid index-pack output\n> not ok 24 - clone from bitmapped repository\n> #\n> #                       rm -fr clone.git &&\n> #                       git clone --no-local --bare . clone.git &&\n> #                       git rev-parse HEAD >expect &&\n> #                       git --git-dir=clone.git rev-parse HEAD >actual &&\n> #                       test_cmp expect actual\n> #\n\n...it looks less like --git-dir is a problem here, and more like the\nintroduction of writev() is. It is now used in sideband_send(), so it\nseems plausible that a similar failure might have broken the git-clone\noperation that the other test was using to create clone.git.\n\nAs for why writev() is failing, I don't know. If it were totally broken\non your system I'd expect almost everything to be failing. But maybe try\nbuilding with \"make NO_WRITEV=Nope\" and see if that makes the problems\ngo away? The compat implementation just does a series of write() calls,\nwhich is what send_sideband() was doing before.\n\n-Peff\n"},{"id":"541138","messageId":"011701dcc767$8c2ab400$a4801c00$@nexbridge.com","threadId":"65454","inReplyTo":"20260408041716.GA1324339@coredump.intra.peff.net","subject":"RE: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2026-04-08T14:54:00Z","receivedAt":"2026-04-08T14:54:09Z","isPatch":false,"body":"On April 8, 2026 12:17 AM, Jeff King wrote:\n>On Tue, Apr 07, 2026 at 07:29:50PM -0400, rsbecker@nexbridge.com wrote:\n>\n>> I can getting numerous issues in t5310, t5326, t2527 relating to the\n>> following use of --git-dir:\n>>\n>> In t5310:\n>> fatal: not a git repository: 'clone.git'\n>> not ok 55 - fetch (full bitmap)\n>> #\n>> #                       git --git-dir=clone.git fetch origin second:second\n>> &&\n>> #                       git rev-parse HEAD >expect &&\n>> #                       git --git-dir=clone.git rev-parse HEAD >actual &&\n>> #                       test_cmp expect actual\n>> #\n>\n>This test hasn't changed recently. The clone.git directory should have been created\n>by an earlier test. Can you try running with \"-i\" and make sure that this is the first\n>failing test, and we didn't fail earlier?\n>\n>Especially because...\n>\n>> In t5326 and t5327:\n>> fatal: writev error: Invalid function argument\n>> fetch-pack: unexpected disconnect while reading sideband packet\n>> fatal: early EOF\n>> fatal: fetch-pack: invalid index-pack output not ok 24 - clone from\n>> bitmapped repository #\n>> #                       rm -fr clone.git &&\n>> #                       git clone --no-local --bare . clone.git &&\n>> #                       git rev-parse HEAD >expect &&\n>> #                       git --git-dir=clone.git rev-parse HEAD >actual &&\n>> #                       test_cmp expect actual\n>> #\n>\n>...it looks less like --git-dir is a problem here, and more like the introduction of\n>writev() is. It is now used in sideband_send(), so it seems plausible that a similar\n>failure might have broken the git-clone operation that the other test was using to\n>create clone.git.\n>\n>As for why writev() is failing, I don't know. If it were totally broken on your system\n>I'd expect almost everything to be failing. But maybe try building with \"make\n>NO_WRITEV=Nope\" and see if that makes the problems go away? The compat\n>implementation just does a series of write() calls, which is what send_sideband()\n>was doing before.\n\nFirst fail is as follows in subtest 25:\n\nexpecting success of 5310.25 'clone from bitmapped repository':\n                rm -fr clone.git &&\n                git clone --no-local --bare . clone.git &&\n                git rev-parse HEAD >expect &&\n                git --git-dir=clone.git rev-parse HEAD >actual &&\n                test_cmp expect actual\n\nCloning into bare repository 'clone.git'...\nremote: Enumerating objects: 629, done.\nfatal: writev error: Invalid function argument\nfetch-pack: unexpected disconnect while reading sideband packet\nfatal: early EOF\nfatal: fetch-pack: invalid index-pack output\nnot ok 25 - clone from bitmapped repository\n\nI think the invalid function argument maybe an ioctl or socketioctl not supported for the file type.\n\n\n"},{"id":"541145","messageId":"013301dcc774$5e9fffb0$1bdfff10$@nexbridge.com","threadId":"65454","inReplyTo":"011701dcc767$8c2ab400$a4801c00$@nexbridge.com","subject":"RE: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2026-04-08T16:25:47Z","receivedAt":"2026-04-08T16:25:55Z","isPatch":false,"body":"On April 8, 2026 10:54 AM, I wrote:\n>To: 'Jeff King' <peff@peff.net>\n>Cc: git@vger.kernel.org\n>Subject: RE: Git 2.54.0-rc1, subtests of t5310, t5326, t5327\n>\n>On April 8, 2026 12:17 AM, Jeff King wrote:\n>>On Tue, Apr 07, 2026 at 07:29:50PM -0400, rsbecker@nexbridge.com wrote:\n>>\n>>> I can getting numerous issues in t5310, t5326, t2527 relating to the\n>>> following use of --git-dir:\n>>>\n>>> In t5310:\n>>> fatal: not a git repository: 'clone.git'\n>>> not ok 55 - fetch (full bitmap)\n>>> #\n>>> #                       git --git-dir=clone.git fetch origin second:second\n>>> &&\n>>> #                       git rev-parse HEAD >expect &&\n>>> #                       git --git-dir=clone.git rev-parse HEAD >actual &&\n>>> #                       test_cmp expect actual\n>>> #\n>>\n>>This test hasn't changed recently. The clone.git directory should have been created\n>>by an earlier test. Can you try running with \"-i\" and make sure that this is the first\n>>failing test, and we didn't fail earlier?\n>>\n>>Especially because...\n>>\n>>> In t5326 and t5327:\n>>> fatal: writev error: Invalid function argument\n>>> fetch-pack: unexpected disconnect while reading sideband packet\n>>> fatal: early EOF\n>>> fatal: fetch-pack: invalid index-pack output not ok 24 - clone from\n>>> bitmapped repository #\n>>> #                       rm -fr clone.git &&\n>>> #                       git clone --no-local --bare . clone.git &&\n>>> #                       git rev-parse HEAD >expect &&\n>>> #                       git --git-dir=clone.git rev-parse HEAD >actual &&\n>>> #                       test_cmp expect actual\n>>> #\n>>\n>>...it looks less like --git-dir is a problem here, and more like the introduction of\n>>writev() is. It is now used in sideband_send(), so it seems plausible that a similar\n>>failure might have broken the git-clone operation that the other test was using to\n>>create clone.git.\n>>\n>>As for why writev() is failing, I don't know. If it were totally broken on your system\n>>I'd expect almost everything to be failing. But maybe try building with \"make\n>>NO_WRITEV=Nope\" and see if that makes the problems go away? The compat\n>>implementation just does a series of write() calls, which is what send_sideband()\n>>was doing before.\n>\n>First fail is as follows in subtest 25:\n>\n>expecting success of 5310.25 'clone from bitmapped repository':\n>                rm -fr clone.git &&\n>                git clone --no-local --bare . clone.git &&\n>                git rev-parse HEAD >expect &&\n>                git --git-dir=clone.git rev-parse HEAD >actual &&\n>                test_cmp expect actual\n>\n>Cloning into bare repository 'clone.git'...\n>remote: Enumerating objects: 629, done.\n>fatal: writev error: Invalid function argument\n>fetch-pack: unexpected disconnect while reading sideband packet\n>fatal: early EOF\n>fatal: fetch-pack: invalid index-pack output\n>not ok 25 - clone from bitmapped repository\n>\n>I think the invalid function argument maybe an ioctl or socketioctl not supported for\n>the file type.\n\nThis is also impacting t5608 and t7700. Anywhere where writev() is used, seemingly. We went through\nMAX_IO_SIZE issues years ago, instead of using ssize_t as a basis of how big communication is. I think\nwritev() is not valid. It worked on Lunix, but had issues elsewhere. This broke the compat layer.\n\n"},{"id":"541158","messageId":"20260408173714.GA2850002@coredump.intra.peff.net","threadId":"65454","inReplyTo":"011701dcc767$8c2ab400$a4801c00$@nexbridge.com","subject":"Re: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2026-04-08T17:37:14Z","receivedAt":"2026-04-08T17:37:15Z","isPatch":false,"body":"On Wed, Apr 08, 2026 at 10:54:00AM -0400, rsbecker@nexbridge.com wrote:\n\n> First fail is as follows in subtest 25:\n> \n> expecting success of 5310.25 'clone from bitmapped repository':\n>                 rm -fr clone.git &&\n>                 git clone --no-local --bare . clone.git &&\n>                 git rev-parse HEAD >expect &&\n>                 git --git-dir=clone.git rev-parse HEAD >actual &&\n>                 test_cmp expect actual\n> \n> Cloning into bare repository 'clone.git'...\n> remote: Enumerating objects: 629, done.\n> fatal: writev error: Invalid function argument\n> fetch-pack: unexpected disconnect while reading sideband packet\n> fatal: early EOF\n> fatal: fetch-pack: invalid index-pack output\n> not ok 25 - clone from bitmapped repository\n\nOK, good. Well, not good, but at least absolves --git-dir, and we know\nthe problem is just writev() everywhere.\n\n> I think the invalid function argument maybe an ioctl or socketioctl\n> not supported for the file type.\n\nYeah, this is weird. We are just calling writev() here, and not trying\nto do anything exotic. I could believe that writev() isn't supported on\ncertain descriptor types or something, but this should just be a pipe.\nBut why does it consistently fail here, but not in every clone? Weird.\n\nIt might be interesting to use strace or a debugger (or just printf from\nwritev_or_die()) to see if there's something interesting about the\narguments here. But finding the root cause may not actually be that\nhelpful (or at least not worth the effort).\n\nDoes building with NO_WRITEV make the problem go away?\n\n-Peff\n"},{"id":"541159","messageId":"20260408173949.GB2850002@coredump.intra.peff.net","threadId":"65454","inReplyTo":"013301dcc774$5e9fffb0$1bdfff10$@nexbridge.com","subject":"Re: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2026-04-08T17:39:49Z","receivedAt":"2026-04-08T17:39:51Z","isPatch":false,"body":"On Wed, Apr 08, 2026 at 12:25:47PM -0400, rsbecker@nexbridge.com wrote:\n\n> This is also impacting t5608 and t7700. Anywhere where writev() is\n> used, seemingly. We went through MAX_IO_SIZE issues years ago, instead\n> of using ssize_t as a basis of how big communication is. I think\n> writev() is not valid. It worked on Lunix, but had issues elsewhere.\n> This broke the compat layer.\n\nI wondered briefly if the problem could be that we're violating\nMAX_IO_SIZE here, as our use of writev() does not respect it at all. But\nthe only spot that uses it is feeding pkt-line packets, which max out at\n64k. So unless your MAX_IO_SIZE is smaller than that, I doubt that is\nthe problem.\n\n-Peff\n"},{"id":"541165","messageId":"xmqq4illz5g9.fsf@gitster.g","threadId":"65454","inReplyTo":"20260408173949.GB2850002@coredump.intra.peff.net","subject":"Re: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-04-08T18:12:54Z","receivedAt":"2026-04-08T18:12:57Z","isPatch":false,"body":"Jeff King <peff@peff.net> writes:\n\n> On Wed, Apr 08, 2026 at 12:25:47PM -0400, rsbecker@nexbridge.com wrote:\n>\n>> This is also impacting t5608 and t7700. Anywhere where writev() is\n>> used, seemingly. We went through MAX_IO_SIZE issues years ago, instead\n>> of using ssize_t as a basis of how big communication is. I think\n>> writev() is not valid. It worked on Lunix, but had issues elsewhere.\n>> This broke the compat layer.\n>\n> I wondered briefly if the problem could be that we're violating\n> MAX_IO_SIZE here, as our use of writev() does not respect it at all. But\n> the only spot that uses it is feeding pkt-line packets, which max out at\n> 64k. So unless your MAX_IO_SIZE is smaller than that, I doubt that is\n> the problem.\n\nGood point.  The original did not use write(2) directly but used\nwrite_or_die(), that is write_in_full(), that loops over xwrite(),\nso it would have worked even with a lot lower MAX_IO_SIZE limit.\n\nAccording to man7.org, writev() is allowed to transfer fewer bytes\nthan requested, so our use of writev() may have to be a bit more\ncareful, though.\n"},{"id":"541168","messageId":"014801dcc786$9ff5bf60$dfe13e20$@nexbridge.com","threadId":"65454","inReplyTo":"20260408173949.GB2850002@coredump.intra.peff.net","subject":"RE: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2026-04-08T18:36:27Z","receivedAt":"2026-04-08T18:36:36Z","isPatch":false,"body":"On April 8, 2026 1:40 PM, Jeff King wrote:\n>On Wed, Apr 08, 2026 at 12:25:47PM -0400, rsbecker@nexbridge.com wrote:\n>\n>> This is also impacting t5608 and t7700. Anywhere where writev() is\n>> used, seemingly. We went through MAX_IO_SIZE issues years ago, instead\n>> of using ssize_t as a basis of how big communication is. I think\n>> writev() is not valid. It worked on Lunix, but had issues elsewhere.\n>> This broke the compat layer.\n>\n>I wondered briefly if the problem could be that we're violating MAX_IO_SIZE here,\n>as our use of writev() does not respect it at all. But the only spot that uses it is\n>feeding pkt-line packets, which max out at 64k. So unless your MAX_IO_SIZE is\n>smaller than that, I doubt that is the problem.\n\nSSIZE_MAX on platform is 53248, so yes. We expected git-compat-util.h at line\n696 to be honoured.\n\n"},{"id":"541170","messageId":"014e01dcc793$8a9bab90$9fd302b0$@nexbridge.com","threadId":"65454","inReplyTo":"xmqq4illz5g9.fsf@gitster.g","subject":"RE: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2026-04-08T20:08:55Z","receivedAt":"2026-04-08T20:09:08Z","isPatch":false,"body":"On April 8, 2026 2:13 PM, Junio C Hamano wrote:\n>Jeff King <peff@peff.net> writes:\n>\n>> On Wed, Apr 08, 2026 at 12:25:47PM -0400, rsbecker@nexbridge.com wrote:\n>>\n>>> This is also impacting t5608 and t7700. Anywhere where writev() is\n>>> used, seemingly. We went through MAX_IO_SIZE issues years ago,\n>>> instead of using ssize_t as a basis of how big communication is. I\n>>> think\n>>> writev() is not valid. It worked on Lunix, but had issues elsewhere.\n>>> This broke the compat layer.\n>>\n>> I wondered briefly if the problem could be that we're violating\n>> MAX_IO_SIZE here, as our use of writev() does not respect it at all.\n>> But the only spot that uses it is feeding pkt-line packets, which max\n>> out at 64k. So unless your MAX_IO_SIZE is smaller than that, I doubt\n>> that is the problem.\n>\n>Good point.  The original did not use write(2) directly but used\nwrite_or_die(), that\n>is write_in_full(), that loops over xwrite(), so it would have worked even\nwith a lot\n>lower MAX_IO_SIZE limit.\n>\n>According to man7.org, writev() is allowed to transfer fewer bytes than\nrequested,\n>so our use of writev() may have to be a bit more careful, though.\n\nOn my box, I have the following note:\n\nSpecifying  the sum of the iov_len values in the iov array greater than\nthe OSS I/O size limit for that open causes the  writev()  function  to\nreturn  -1  and  set errno to [EINVAL].\n\nThis can be either 52K (SSIZE_MAX) or 868K depending on the build settings.\nWe are currently stuck with 32-bit builds, with large file support, so\nreally no\nlimit on the file sizes, but due to dependencies that are stuck at 32-bit\nuntil I\nconvince the powers that be to fix those. Please, can we be compliant with\nMAX_IO_SIZE?\n\n"},{"id":"541171","messageId":"xmqqqzopxkxa.fsf@gitster.g","threadId":"65454","inReplyTo":"014e01dcc793$8a9bab90$9fd302b0$@nexbridge.com","subject":"Re: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-04-08T20:21:37Z","receivedAt":"2026-04-08T20:21:40Z","isPatch":false,"body":"<rsbecker@nexbridge.com> writes:\n\n> On my box, I have the following note:\n>\n> Specifying  the sum of the iov_len values in the iov array greater than\n> the OSS I/O size limit for that open causes the  writev()  function  to\n> return  -1  and  set errno to [EINVAL].\n\nThat is unexpected.\n\nwritev() may fail if the sum of iov_len would not fit within ssize_t\nwith EINVAL, but unless your \"the OSS I/O size limit\" is the same as\nSSIZE_MAX, what you have above is not quite the same.\n\nDoes your build work with NO_WRITEV=Nope?  I think I saw it asked a\nfew times but I do not recall seeing it answered.  At least we know\nxwrite() seems to work well enough on your system, which is what the\nwritev() emulation is written in terms of, so I suspect it would.\n\n\n"},{"id":"541184","messageId":"016b01dcc79e$87472860$95d57920$@nexbridge.com","threadId":"65454","inReplyTo":"xmqqqzopxkxa.fsf@gitster.g","subject":"RE: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2026-04-08T21:27:34Z","receivedAt":"2026-04-08T21:27:44Z","isPatch":false,"body":"On April 8, 2026 4:22 PM, Junio C Hamano wrote:\n><rsbecker@nexbridge.com> writes:\n>\n>> On my box, I have the following note:\n>>\n>> Specifying  the sum of the iov_len values in the iov array greater\n>> than the OSS I/O size limit for that open causes the  writev()\n>> function  to return  -1  and  set errno to [EINVAL].\n>\n>That is unexpected.\n>\n>writev() may fail if the sum of iov_len would not fit within ssize_t with\nEINVAL, but\n>unless your \"the OSS I/O size limit\" is the same as SSIZE_MAX, what you\nhave above\n>is not quite the same.\n>\n>Does your build work with NO_WRITEV=Nope?  I think I saw it asked a few\ntimes\n>but I do not recall seeing it answered.  At least we know\n>xwrite() seems to work well enough on your system, which is what the\n>writev() emulation is written in terms of, so I suspect it would.\n\nYes, NO_WRITEV=Nope does compile and execute. I am including it\nin our CI/CD job for now. Can we plan on a fix for this?\n\n"},{"id":"541186","messageId":"xmqqcy09xh53.fsf@gitster.g","threadId":"65454","inReplyTo":"016b01dcc79e$87472860$95d57920$@nexbridge.com","subject":"Re: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-04-08T21:43:20Z","receivedAt":"2026-04-08T21:43:22Z","isPatch":false,"body":"<rsbecker@nexbridge.com> writes:\n\n> On April 8, 2026 4:22 PM, Junio C Hamano wrote:\n>><rsbecker@nexbridge.com> writes:\n>>\n>>> On my box, I have the following note:\n>>>\n>>> Specifying  the sum of the iov_len values in the iov array greater\n>>> than the OSS I/O size limit for that open causes the  writev()\n>>> function  to return  -1  and  set errno to [EINVAL].\n>>\n>>That is unexpected.\n>>\n>>writev() may fail if the sum of iov_len would not fit within ssize_t with\n> EINVAL, but\n>>unless your \"the OSS I/O size limit\" is the same as SSIZE_MAX, what you\n> have above\n>>is not quite the same.\n>>\n>>Does your build work with NO_WRITEV=Nope?  I think I saw it asked a few\n> times\n>>but I do not recall seeing it answered.  At least we know\n>>xwrite() seems to work well enough on your system, which is what the\n>>writev() emulation is written in terms of, so I suspect it would.\n>\n> Yes, NO_WRITEV=Nope does compile and execute. I am including it\n> in our CI/CD job for now. Can we plan on a fix for this?\n\nWhat I have heard so far indicate that the code that uses writev()\nwould need to loop over to prepare for short writes, but your\nwritev() that fails for \"the OSS I/O size limit\" (whatever it is)\ndoes not sound like something we want to change the callers to chomp\nthe writev() calls into smaller chunks for.  Such a platform is far\nbetter off using the compat/writev for the code path we recently\nstarted using writev() in.\n\nTo be quite honest, I am not sure if it is even worth using writev()\nif we need a loop that protects against shrot writes, so unless I am\ngrossly mistaken (e.g., perhaps there is some guarantee that there\nwon't be any short writes for writev() that sends data smaller than\n64k that I missed in the docs), the best course of action might be\nto revert the change to use writev() and use the two write(2)s as\nbefore, *if* we actually observe that the current code is broken by\nshort writes.\n\n"},{"id":"541188","messageId":"016e01dcc7a3$9f0d06e0$dd2714a0$@nexbridge.com","threadId":"65454","inReplyTo":"xmqqcy09xh53.fsf@gitster.g","subject":"RE: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2026-04-08T22:04:01Z","receivedAt":"2026-04-08T22:04:11Z","isPatch":false,"body":"On April 8, 2026 5:43 PM, Junio C Hamano wrote:\n><rsbecker@nexbridge.com> writes:\n>\n>> On April 8, 2026 4:22 PM, Junio C Hamano wrote:\n>>><rsbecker@nexbridge.com> writes:\n>>>\n>>>> On my box, I have the following note:\n>>>>\n>>>> Specifying  the sum of the iov_len values in the iov array greater\n>>>> than the OSS I/O size limit for that open causes the  writev()\n>>>> function  to return  -1  and  set errno to [EINVAL].\n>>>\n>>>That is unexpected.\n>>>\n>>>writev() may fail if the sum of iov_len would not fit within ssize_t\n>>>with\n>> EINVAL, but\n>>>unless your \"the OSS I/O size limit\" is the same as SSIZE_MAX, what\n>>>you\n>> have above\n>>>is not quite the same.\n>>>\n>>>Does your build work with NO_WRITEV=Nope?  I think I saw it asked a\n>>>few\n>> times\n>>>but I do not recall seeing it answered.  At least we know\n>>>xwrite() seems to work well enough on your system, which is what the\n>>>writev() emulation is written in terms of, so I suspect it would.\n>>\n>> Yes, NO_WRITEV=Nope does compile and execute. I am including it in our\n>> CI/CD job for now. Can we plan on a fix for this?\n>\n>What I have heard so far indicate that the code that uses writev() would\nneed to\n>loop over to prepare for short writes, but your\n>writev() that fails for \"the OSS I/O size limit\" (whatever it is) does not\nsound like\n>something we want to change the callers to chomp the writev() calls into\nsmaller\n>chunks for.  Such a platform is far better off using the compat/writev for\nthe code\n>path we recently started using writev() in.\n>\n>To be quite honest, I am not sure if it is even worth using writev() if we\nneed a loop\n>that protects against shrot writes, so unless I am grossly mistaken (e.g.,\nperhaps\n>there is some guarantee that there won't be any short writes for writev()\nthat sends\n>data smaller than 64k that I missed in the docs), the best course of action\nmight be\n>to revert the change to use writev() and use the two write(2)s as before,\n*if* we\n>actually observe that the current code is broken by short writes.\n\nI am 100% sure that EINVAL is returned by writev() on NonStop if the size\nexceeds 52K\nIn 32-bit models. Whether it supports 868K for 64-bit is conjecture.\nNO_WRITEV=Nope\nworks, which I am trying for everything at RC1, then we can live with this.\n\n"},{"id":"541189","messageId":"20260408221447.GA2873736@coredump.intra.peff.net","threadId":"65454","inReplyTo":"014801dcc786$9ff5bf60$dfe13e20$@nexbridge.com","subject":"Re: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2026-04-08T22:14:47Z","receivedAt":"2026-04-08T22:14:54Z","isPatch":false,"body":"On Wed, Apr 08, 2026 at 02:36:27PM -0400, rsbecker@nexbridge.com wrote:\n\n> On April 8, 2026 1:40 PM, Jeff King wrote:\n> >On Wed, Apr 08, 2026 at 12:25:47PM -0400, rsbecker@nexbridge.com wrote:\n> >\n> >> This is also impacting t5608 and t7700. Anywhere where writev() is\n> >> used, seemingly. We went through MAX_IO_SIZE issues years ago, instead\n> >> of using ssize_t as a basis of how big communication is. I think\n> >> writev() is not valid. It worked on Lunix, but had issues elsewhere.\n> >> This broke the compat layer.\n> >\n> >I wondered briefly if the problem could be that we're violating MAX_IO_SIZE here,\n> >as our use of writev() does not respect it at all. But the only spot that uses it is\n> >feeding pkt-line packets, which max out at 64k. So unless your MAX_IO_SIZE is\n> >smaller than that, I doubt that is the problem.\n> \n> SSIZE_MAX on platform is 53248, so yes. We expected git-compat-util.h at line\n> 696 to be honoured.\n\nOof, that is small. So yeah, that is almost certainly the problem (and\nexplains why it only happens for some writes in the test suite).\n\n-Peff\n"},{"id":"541190","messageId":"xmqqzf3dw0o8.fsf@gitster.g","threadId":"65454","inReplyTo":"xmqqcy09xh53.fsf@gitster.g","subject":"Re: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-04-08T22:24:23Z","receivedAt":"2026-04-08T22:25:40Z","isPatch":false,"body":"Junio C Hamano <gitster@pobox.com> writes:\n\n> To be quite honest, I am not sure if it is even worth using writev()\n> if we need a loop that protects against shrot writes, so unless I am\n> grossly mistaken (e.g., perhaps there is some guarantee that there\n> won't be any short writes for writev() that sends data smaller than\n> 64k that I missed in the docs), the best course of action might be\n> to revert the change to use writev() and use the two write(2)s as\n> before, *if* we actually observe that the current code is broken by\n> short writes.\n\nAh, sorry, I should have double checked the actual code.  We already\nuse a looping writev_in_full() that wraps writev(), so there is\nnothing extra that we still need to do to prepare for short writes.\n\nUnfortunately, comparing write_in_full() vs writev_in_full(), there\nis nothing that corresponds to xwrite() that can be used to hide the\nshort writes and chomps an originally larger I/O into smaller\npieces.  Unlike write() that we may receive a single linear large\nsequence of bytes, which we can choose to chomp into artificially\nsmaller pieces and write them out (up to 8MB by default), writev()\nAPI lets the caller to prepare chunks of memory and I do not think\nthere is a good way for the writev_in_full() at the lower layer to\nchomp these into smaller pieces, and even if we could, that would\ndefeat the whole reason why we rewrote the original code that used\nwrite_in_full() into using writev(), i.e., to avoid extra allocation\n(and extra system calls---but if your I/O layer is limited to very\nsmall writes, no matter how we chop it, you will have to issue extra\nsystem calls to flush all of the data out).\n\nSo, I dunno.\n\n\n"},{"id":"541191","messageId":"20260408223233.GB2873736@coredump.intra.peff.net","threadId":"65454","inReplyTo":"xmqqcy09xh53.fsf@gitster.g","subject":"Re: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2026-04-08T22:32:33Z","receivedAt":"2026-04-08T22:32:34Z","isPatch":false,"body":"On Wed, Apr 08, 2026 at 02:43:20PM -0700, Junio C Hamano wrote:\n\n> > Yes, NO_WRITEV=Nope does compile and execute. I am including it\n> > in our CI/CD job for now. Can we plan on a fix for this?\n> \n> What I have heard so far indicate that the code that uses writev()\n> would need to loop over to prepare for short writes, but your\n> writev() that fails for \"the OSS I/O size limit\" (whatever it is)\n> does not sound like something we want to change the callers to chomp\n> the writev() calls into smaller chunks for.  Such a platform is far\n> better off using the compat/writev for the code path we recently\n> started using writev() in.\n\nYeah, we definitely do not want callers to worry about this. I think it\nwould be possible to have xwritev() handle this, but it gets ugly. If we\nare worried about an individual iovec being larger than MAX_IO_SIZE,\nthen we may need to rewrite the iovec array to break it apart. And if we\nare worried about the sum of the pieces being larger than MAX_IO_SIZE,\nthen we end up with multiple writev() calls anyway.\n\nAt which point the least-painful thing is probably just calling write()\non each segment anyway, which is exactly what the compat wrapper does.\n\n> To be quite honest, I am not sure if it is even worth using writev()\n> if we need a loop that protects against shrot writes, so unless I am\n> grossly mistaken (e.g., perhaps there is some guarantee that there\n> won't be any short writes for writev() that sends data smaller than\n> 64k that I missed in the docs), the best course of action might be\n> to revert the change to use writev() and use the two write(2)s as\n> before, *if* we actually observe that the current code is broken by\n> short writes.\n\nI think writev() is buying us something when it works (it is halving the\nnumber of writes for sideband packets). And it works when either:\n\n  1. the platform is OK with writing up to 64k in a single writev()\n\n  2. the platform has a limit that is small (like NonStop here), but\n     writes less than MAX_IO_SIZE work and will save a write() call\n\nIf we just care about (1), then the right solution is to declare that\nwritev() isn't fully functional for us on some platforms, and they\nshould build with NO_WRITEV. And we should probably embed that in\nconfig.mak.uname.\n\nIf we want to care about (2), then we could make the decision at\nruntime, something like:\n\ndiff --git a/compat/writev.c b/compat/writev.c\nindex 3a94870a2f..cf2fbb39c9 100644\n--- a/compat/writev.c\n+++ b/compat/writev.c\n@@ -1,7 +1,7 @@\n #include \"../git-compat-util.h\"\n #include \"../wrapper.h\"\n \n-ssize_t git_writev(int fd, const struct iovec *iov, int iovcnt)\n+ssize_t git_writev_with_write(int fd, const struct iovec *iov, int iovcnt)\n {\n \tsize_t total_written = 0;\n \tsize_t sum = 0;\n@@ -42,3 +42,17 @@ ssize_t git_writev(int fd, const struct iovec *iov, int iovcnt)\n out:\n \treturn (ssize_t) total_written;\n }\n+\n+ssize_t git_writev(int fd, const struct iovec *iov, int iovcnt)\n+{\n+\tsize_t total = 0;\n+\tfor (int i = 0; i < iovcnt; i++)\n+\t\ttotal += iov[i].iov_len;\n+\tif (total > MAX_IO_SIZE) {\n+\t\t/* too big; bail to wrapper which will limit individual writes */\n+\t\treturn git_writev_with_write(fd, iov, iovcnt);\n+\t}\n+\n+\t/* otherwise, use the real system writev */\n+\treturn writev(fd, iov, iovcnt);\n+}\n\nI'm not sure that complexity is worth it, though. If your write-limit is\nless than 64k you are already doing lots of extra write() calls, and\ntrying to squeeze out a tiny bit of performance by omitting a few of\nthem is not worth the trouble.\n\nThough it does make things Just Work on such platforms without having to\nset another build-time knob.\n\nI do wonder what it all means for systems with MAX_IO_SIZE that is more\nreasonable. Right now we are using writev() only for sideband packets.\nBut one of the stated reasons for using it there (instead of tweaking\nthe packet buffer so that there's room for the header in it) is that\npeople wanted to be able to start using writev() elsewhere. If we start\nfeeding unbounded data to it (say, a blob buffer), then we're going to\nbe violating MAX_IO_SIZE for those calls. So there may be more of this\nheadache down the road.\n\n-Peff\n"},{"id":"541192","messageId":"xmqqv7e1w05u.fsf@gitster.g","threadId":"65454","inReplyTo":"xmqqzf3dw0o8.fsf@gitster.g","subject":"Re: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-04-08T22:35:25Z","receivedAt":"2026-04-08T22:35:27Z","isPatch":false,"body":"Junio C Hamano <gitster@pobox.com> writes:\n\n> Junio C Hamano <gitster@pobox.com> writes:\n>\n>> To be quite honest, I am not sure if it is even worth using writev()\n>> if we need a loop that protects against shrot writes, so unless I am\n>> grossly mistaken (e.g., perhaps there is some guarantee that there\n>> won't be any short writes for writev() that sends data smaller than\n>> 64k that I missed in the docs), the best course of action might be\n>> to revert the change to use writev() and use the two write(2)s as\n>> before, *if* we actually observe that the current code is broken by\n>> short writes.\n>\n> Ah, sorry, I should have double checked the actual code.  We already\n> use a looping writev_in_full() that wraps writev(), so there is\n> nothing extra that we still need to do to prepare for short writes.\n>\n> Unfortunately, comparing write_in_full() vs writev_in_full(), there\n> is nothing that corresponds to xwrite() that can be used to hide the\n> short writes and chomps an originally larger I/O into smaller\n> pieces.\n\nOops, the beauty of having xwrite() is *not* that it hides short\nwrites (it doesn't), but it can be used to pretend that short writes\nhappened on platforms with unreasonably small I/O limit by setting\nMAX_IO_SIZE to unusually low.  But the point that ...\n\n> Unlike write() that we may receive a single linear large\n> sequence of bytes, which we can choose to chomp into artificially\n> smaller pieces and write them out (up to 8MB by default), writev()\n> API lets the caller to prepare chunks of memory and I do not think\n> there is a good way for the writev_in_full() at the lower layer to\n> chomp these into smaller pieces, and even if we could, that would\n> defeat the whole reason why we rewrote the original code that used\n> write_in_full() into using writev(), i.e., to avoid extra allocation\n> (and extra system calls---but if your I/O layer is limited to very\n> small writes, no matter how we chop it, you will have to issue extra\n> system calls to flush all of the data out).\n>\n> So, I dunno.\n\n... I doubt that there exists a good way to have xwritev() that\nwraps around writev() and pretend that a short write happened,\ninstead of issuing a large I/O, still stands.\n\n\n"},{"id":"541193","messageId":"018701dcc7ad$8c3addd0$a4b09970$@nexbridge.com","threadId":"65454","inReplyTo":"xmqqv7e1w05u.fsf@gitster.g","subject":"RE: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2026-04-08T23:15:05Z","receivedAt":"2026-04-08T23:15:15Z","isPatch":false,"body":"On April 8, 2026 6:35 PM, Junio C Hamano wrote\n>Junio C Hamano <gitster@pobox.com> writes:\n>\n>> Junio C Hamano <gitster@pobox.com> writes:\n>>\n>>> To be quite honest, I am not sure if it is even worth using writev()\n>>> if we need a loop that protects against shrot writes, so unless I am\n>>> grossly mistaken (e.g., perhaps there is some guarantee that there\n>>> won't be any short writes for writev() that sends data smaller than\n>>> 64k that I missed in the docs), the best course of action might be to\n>>> revert the change to use writev() and use the two write(2)s as\n>>> before, *if* we actually observe that the current code is broken by\n>>> short writes.\n>>\n>> Ah, sorry, I should have double checked the actual code.  We already\n>> use a looping writev_in_full() that wraps writev(), so there is\n>> nothing extra that we still need to do to prepare for short writes.\n>>\n>> Unfortunately, comparing write_in_full() vs writev_in_full(), there is\n>> nothing that corresponds to xwrite() that can be used to hide the\n>> short writes and chomps an originally larger I/O into smaller pieces.\n>\n>Oops, the beauty of having xwrite() is *not* that it hides short writes (it\ndoesn't),\n>but it can be used to pretend that short writes happened on platforms with\n>unreasonably small I/O limit by setting MAX_IO_SIZE to unusually low.  But\nthe\n>point that ...\n>\n>> Unlike write() that we may receive a single linear large sequence of\n>> bytes, which we can choose to chomp into artificially smaller pieces\n>> and write them out (up to 8MB by default), writev() API lets the\n>> caller to prepare chunks of memory and I do not think there is a good\n>> way for the writev_in_full() at the lower layer to chomp these into\n>> smaller pieces, and even if we could, that would defeat the whole\n>> reason why we rewrote the original code that used\n>> write_in_full() into using writev(), i.e., to avoid extra allocation\n>> (and extra system calls---but if your I/O layer is limited to very\n>> small writes, no matter how we chop it, you will have to issue extra\n>> system calls to flush all of the data out).\n>>\n>> So, I dunno.\n>\n>... I doubt that there exists a good way to have xwritev() that wraps\naround writev()\n>and pretend that a short write happened, instead of issuing a large I/O,\nstill stands.\n\nI am partial to Peff's runtime approach, personally.\n\nIf it helps (probably does not), the limit is due to the DMA/VMM limits on\nthe\nhardware. The NonStop Message system is blindingly fast when sending\nmessages around between processes on other CPUs - far faster than I have\nseen\non most other platforms. But there are physical limits in the chipset. It is\ninteresting\nto use it in an abstracted way, like hacking the TCP/IP stack to pass\nhandles around.\n\n"},{"id":"541195","messageId":"adbwyvQ-R2Ag1vox@fruit.crustytoothpaste.net","threadId":"65454","inReplyTo":"20260408223233.GB2873736@coredump.intra.peff.net","subject":"Re: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2026-04-09T00:20:26Z","receivedAt":"2026-04-09T00:20:34Z","isPatch":false,"body":"On 2026-04-08 at 22:32:33, Jeff King wrote:\n> I think writev() is buying us something when it works (it is hlving the\n> number of writes for sideband packets). And it works when either:\n> \n>   1. the platform is OK with writing up to 64k in a single writev()\n> \n>   2. the platform has a limit that is small (like NonStop here), but\n>      writes less than MAX_IO_SIZE work and will save a write() call\n> \n> If we just care about (1), then the right solution is to declare that\n> writev() isn't fully functional for us on some platforms, and they\n> should build with NO_WRITEV. And we should probably embed that in\n> config.mak.uname.\n\nLooking at POSIX, there doesn't seem to be any constraints on the size\nof individual vectors other than that they must total to less than\nSSIZE_MAX.  iovcnt can be limited to 16, but I don't think we're hitting\nthat here.  POSIX does say that SSIZE_MAX does not need to exceed 32767,\nwhich may be what's going on here, although that does seem like an\nunreasonable value for a real system.  Linux, FreeBSD, and NetBSD all\nset SSIZE_MAX to either INT_MAX or LONG_MAX.\n\nI also think that 64 KiB is more than reasonable in terms of the size\nthat people should be able to send.  I'd personally expect to be able to\nsend values much larger, at least 512 KiB, and I have code that expects\neven larger (16 MiB).\n\nSo I'd simply say that for systems that have a constraint on the size\nthat is \"too small\", they should just use NO_WRITEV.\n\nHowever, I don't have a strong opinion on this and if people want to do\nthe proposal for option 2, that's fine with me.\n\nI will say that we may find ourselves in a pickle with Rust code in the\nfuture if we use `write_vectored` since that will probably just use the\nOS implementation, but we can worry about that when we get there.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"541235","messageId":"addgkjiB80pgKw69@pks.im","threadId":"65454","inReplyTo":"adbwyvQ-R2Ag1vox@fruit.crustytoothpaste.net","subject":"Re: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-04-09T08:17:22Z","receivedAt":"2026-04-09T08:17:32Z","isPatch":false,"body":"On Thu, Apr 09, 2026 at 12:20:26AM +0000, brian m. carlson wrote:\n> On 2026-04-08 at 22:32:33, Jeff King wrote:\n> > I think writev() is buying us something when it works (it is hlving the\n> > number of writes for sideband packets). And it works when either:\n> > \n> >   1. the platform is OK with writing up to 64k in a single writev()\n> > \n> >   2. the platform has a limit that is small (like NonStop here), but\n> >      writes less than MAX_IO_SIZE work and will save a write() call\n> > \n> > If we just care about (1), then the right solution is to declare that\n> > writev() isn't fully functional for us on some platforms, and they\n> > should build with NO_WRITEV. And we should probably embed that in\n> > config.mak.uname.\n\nYeah, agreed. I think we shouldn't make ourselves a hostage to platforms\nthat don't have reasonable support for writev(3p), as it does buy us\nsomething on the majority of platforms that actually support it well.\n\nThat of course doesn't mean that we shouldn't support such platforms.\n\n> Looking at POSIX, there doesn't seem to be any constraints on the size\n> of individual vectors other than that they must total to less than\n> SSIZE_MAX.  iovcnt can be limited to 16, but I don't think we're hitting\n> that here.  POSIX does say that SSIZE_MAX does not need to exceed 32767,\n> which may be what's going on here, although that does seem like an\n> unreasonable value for a real system.  Linux, FreeBSD, and NetBSD all\n> set SSIZE_MAX to either INT_MAX or LONG_MAX.\n> \n> I also think that 64 KiB is more than reasonable in terms of the size\n> that people should be able to send.  I'd personally expect to be able to\n> send values much larger, at least 512 KiB, and I have code that expects\n> even larger (16 MiB).\n> \n> So I'd simply say that for systems that have a constraint on the size\n> that is \"too small\", they should just use NO_WRITEV.\n\nI would be happy with this as an intermediate step, as Randall has\nconfirmed it would fix the issue. It is the least intrusive step and has\nthe lowest risk.\n\n> However, I don't have a strong opinion on this and if people want to do\n> the proposal for option 2, that's fine with me.\n\nI think in the long term this is the most sensible approach though so\nthat we don't have to special-case platforms. I've crafted the below\nalternative to Peff's patch, and I think it's ultimately not too bad.\n\nOne question to Randall though: does MAX_IO_SIZE apply to the overall\nsize of the iovec or to the individual iovec entries? I think it should\nbe the latter, but I cannot easily verify and couldn't find any docs\naround this. So could you please try the patch at the end of this mail\nto verify that it works on your system?\n\nIn any case, I've tested that my patch also works when defining\nMAX_IO_SIZE to 128 bytes on my system, which hopefully demonstrates that\nit works as expected:\n\ndiff --git a/git-compat-util.h b/git-compat-util.h\nindex 4b4ea2498f..8e02b5f673 100644\n--- a/git-compat-util.h\n+++ b/git-compat-util.h\n@@ -690,14 +690,8 @@ static inline uint64_t u64_add(uint64_t a, uint64_t b)\n  * to override this, if the definition of SSIZE_MAX given by the platform\n  * is broken.\n  */\n-#ifndef MAX_IO_SIZE\n-# define MAX_IO_SIZE_DEFAULT (8*1024*1024)\n-# if defined(SSIZE_MAX) && (SSIZE_MAX < MAX_IO_SIZE_DEFAULT)\n-#  define MAX_IO_SIZE SSIZE_MAX\n-# else\n-#  define MAX_IO_SIZE MAX_IO_SIZE_DEFAULT\n-# endif\n-#endif\n+#undef MAX_IO_SIZE\n+#define MAX_IO_SIZE 128\n \n #ifdef HAVE_ALLOCA_H\n # include <alloca.h>\n\nI'm happy to go either way, but think that we should definitely aim for\nthe below patch eventually. Just let me know which way you prefer and\nI'm happy to polish up the patch.\n\nPatrick\n\ndiff --git a/wrapper.c b/wrapper.c\nindex be8fa575e6..645dbc5f20 100644\n--- a/wrapper.c\n+++ b/wrapper.c\n@@ -323,21 +323,50 @@ ssize_t write_in_full(int fd, const void *buf, size_t count)\n \treturn total;\n }\n \n+ssize_t xwritev(int fd, struct iovec *iov, int iovcnt)\n+{\n+\tssize_t bytes_written;\n+\tint i;\n+\n+\t/*\n+\t * We need to make sure that no individual iovec entry exceeds\n+\t * `MAX_IO_SIZE`. If there's any entry that does exceed this limit\n+\t * we'll pass all entries up to it to `writev()`, and then process the\n+\t * exceeding entry via a call to `xwrite()`.\n+\t */\n+\tfor (i = 0; i < iovcnt; i++)\n+\t\tif (iov[i].iov_len > MAX_IO_SIZE)\n+\t\t\tbreak;\n+\tif (i < iovcnt) {\n+\t\t/*\n+\t\t * The first entry exceeds MAX_IO_SIZE, so we pass it to\n+\t\t * xwrite, which knows to handle his case.\n+\t\t */\n+\t\tif (!i)\n+\t\t\treturn xwrite(fd, iov->iov_base, iov->iov_len);\n+\t\tiovcnt = i;\n+\t}\n+\n+\tbytes_written = writev(fd, iov, iovcnt);\n+\tif (!bytes_written) {\n+\t\terrno = ENOSPC;\n+\t\treturn -1;\n+\t}\n+\n+\treturn bytes_written;\n+}\n+\n ssize_t writev_in_full(int fd, struct iovec *iov, int iovcnt)\n {\n \tssize_t total_written = 0;\n \n \twhile (iovcnt) {\n-\t\tssize_t bytes_written = writev(fd, iov, iovcnt);\n-\t\tif (bytes_written < 0) {\n+\t\tssize_t bytes_written = xwritev(fd, iov, iovcnt);\n+\t\tif (bytes_written <= 0) {\n \t\t\tif (errno == EINTR || errno == EAGAIN)\n \t\t\t\tcontinue;\n \t\t\treturn -1;\n \t\t}\n-\t\tif (!bytes_written) {\n-\t\t\terrno = ENOSPC;\n-\t\t\treturn -1;\n-\t\t}\n \n \t\ttotal_written += bytes_written;\n \ndiff --git a/wrapper.h b/wrapper.h\nindex 27519b32d1..a6287d7f4d 100644\n--- a/wrapper.h\n+++ b/wrapper.h\n@@ -16,6 +16,7 @@ void *xmmap_gently(void *start, size_t length, int prot, int flags, int fd, off_\n int xopen(const char *path, int flags, ...);\n ssize_t xread(int fd, void *buf, size_t len);\n ssize_t xwrite(int fd, const void *buf, size_t len);\n+ssize_t xwritev(int fd, struct iovec *iov, int iovcnt);\n ssize_t xpread(int fd, void *buf, size_t len, off_t offset);\n int xdup(int fd);\n FILE *xfopen(const char *path, const char *mode);\n"},{"id":"541238","messageId":"90c6112d-6447-45e0-8d15-a0a3f1f25013@gmail.com","threadId":"65454","inReplyTo":"addgkjiB80pgKw69@pks.im","subject":"Re: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2026-04-09T09:48:02Z","receivedAt":"2026-04-09T09:48:07Z","isPatch":false,"body":"On 09/04/2026 09:17, Patrick Steinhardt wrote:\n> \n> One question to Randall though: does MAX_IO_SIZE apply to the overall\n> size of the iovec or to the individual iovec entries? \n\nIn <014e01dcc793$8a9bab90$9fd302b0$@nexbridge.com> Randall says\n\n     Specifying  the sum of the iov_len values in the iov array greater\n     than the OSS I/O size limit for that open causes the  writev()\n     function  to return  -1  and  set errno to [EINVAL].\n\nSo it is the overall size which fits with POSIX limiting to overall size \nto SSIZE_MAX.\n\nThanks\n\nPhillip\n\n> I think it should\n> be the latter, but I cannot easily verify and couldn't find any docs\n> around this. So could you please try the patch at the end of this mail\n> to verify that it works on your system?\n> \n> In any case, I've tested that my patch also works when defining\n> MAX_IO_SIZE to 128 bytes on my system, which hopefully demonstrates that\n> it works as expected:\n> \n> diff --git a/git-compat-util.h b/git-compat-util.h\n> index 4b4ea2498f..8e02b5f673 100644\n> --- a/git-compat-util.h\n> +++ b/git-compat-util.h\n> @@ -690,14 +690,8 @@ static inline uint64_t u64_add(uint64_t a, uint64_t b)\n>    * to override this, if the definition of SSIZE_MAX given by the platform\n>    * is broken.\n>    */\n> -#ifndef MAX_IO_SIZE\n> -# define MAX_IO_SIZE_DEFAULT (8*1024*1024)\n> -# if defined(SSIZE_MAX) && (SSIZE_MAX < MAX_IO_SIZE_DEFAULT)\n> -#  define MAX_IO_SIZE SSIZE_MAX\n> -# else\n> -#  define MAX_IO_SIZE MAX_IO_SIZE_DEFAULT\n> -# endif\n> -#endif\n> +#undef MAX_IO_SIZE\n> +#define MAX_IO_SIZE 128\n>   \n>   #ifdef HAVE_ALLOCA_H\n>   # include <alloca.h>\n> \n> I'm happy to go either way, but think that we should definitely aim for\n> the below patch eventually. Just let me know which way you prefer and\n> I'm happy to polish up the patch.\n> \n> Patrick\n> \n> diff --git a/wrapper.c b/wrapper.c\n> index be8fa575e6..645dbc5f20 100644\n> --- a/wrapper.c\n> +++ b/wrapper.c\n> @@ -323,21 +323,50 @@ ssize_t write_in_full(int fd, const void *buf, size_t count)\n>   \treturn total;\n>   }\n>   \n> +ssize_t xwritev(int fd, struct iovec *iov, int iovcnt)\n> +{\n> +\tssize_t bytes_written;\n> +\tint i;\n> +\n> +\t/*\n> +\t * We need to make sure that no individual iovec entry exceeds\n> +\t * `MAX_IO_SIZE`. If there's any entry that does exceed this limit\n> +\t * we'll pass all entries up to it to `writev()`, and then process the\n> +\t * exceeding entry via a call to `xwrite()`.\n> +\t */\n> +\tfor (i = 0; i < iovcnt; i++)\n> +\t\tif (iov[i].iov_len > MAX_IO_SIZE)\n> +\t\t\tbreak;\n> +\tif (i < iovcnt) {\n> +\t\t/*\n> +\t\t * The first entry exceeds MAX_IO_SIZE, so we pass it to\n> +\t\t * xwrite, which knows to handle his case.\n> +\t\t */\n> +\t\tif (!i)\n> +\t\t\treturn xwrite(fd, iov->iov_base, iov->iov_len);\n> +\t\tiovcnt = i;\n> +\t}\n> +\n> +\tbytes_written = writev(fd, iov, iovcnt);\n> +\tif (!bytes_written) {\n> +\t\terrno = ENOSPC;\n> +\t\treturn -1;\n> +\t}\n> +\n> +\treturn bytes_written;\n> +}\n> +\n>   ssize_t writev_in_full(int fd, struct iovec *iov, int iovcnt)\n>   {\n>   \tssize_t total_written = 0;\n>   \n>   \twhile (iovcnt) {\n> -\t\tssize_t bytes_written = writev(fd, iov, iovcnt);\n> -\t\tif (bytes_written < 0) {\n> +\t\tssize_t bytes_written = xwritev(fd, iov, iovcnt);\n> +\t\tif (bytes_written <= 0) {\n>   \t\t\tif (errno == EINTR || errno == EAGAIN)\n>   \t\t\t\tcontinue;\n>   \t\t\treturn -1;\n>   \t\t}\n> -\t\tif (!bytes_written) {\n> -\t\t\terrno = ENOSPC;\n> -\t\t\treturn -1;\n> -\t\t}\n>   \n>   \t\ttotal_written += bytes_written;\n>   \n> diff --git a/wrapper.h b/wrapper.h\n> index 27519b32d1..a6287d7f4d 100644\n> --- a/wrapper.h\n> +++ b/wrapper.h\n> @@ -16,6 +16,7 @@ void *xmmap_gently(void *start, size_t length, int prot, int flags, int fd, off_\n>   int xopen(const char *path, int flags, ...);\n>   ssize_t xread(int fd, void *buf, size_t len);\n>   ssize_t xwrite(int fd, const void *buf, size_t len);\n> +ssize_t xwritev(int fd, struct iovec *iov, int iovcnt);\n>   ssize_t xpread(int fd, void *buf, size_t len, off_t offset);\n>   int xdup(int fd);\n>   FILE *xfopen(const char *path, const char *mode);\n> \n\n"},{"id":"541249","messageId":"adeNjEBSlXd7ykEx@pks.im","threadId":"65454","inReplyTo":"90c6112d-6447-45e0-8d15-a0a3f1f25013@gmail.com","subject":"Re: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-04-09T11:29:16Z","receivedAt":"2026-04-09T11:29:27Z","isPatch":false,"body":"On Thu, Apr 09, 2026 at 10:48:02AM +0100, Phillip Wood wrote:\n> On 09/04/2026 09:17, Patrick Steinhardt wrote:\n> > \n> > One question to Randall though: does MAX_IO_SIZE apply to the overall\n> > size of the iovec or to the individual iovec entries?\n> \n> In <014e01dcc793$8a9bab90$9fd302b0$@nexbridge.com> Randall says\n> \n>     Specifying  the sum of the iov_len values in the iov array greater\n>     than the OSS I/O size limit for that open causes the  writev()\n>     function  to return  -1  and  set errno to [EINVAL].\n> \n> So it is the overall size which fits with POSIX limiting to overall size to\n> SSIZE_MAX.\n\nAh, thanks for the pointer. I've adapted the patch a bit to the below\none. Again, I've tested it with `#define MAX_IO_SIZE 100` to verify that\nit works as expected.\n\nI guess I'll send a polished version to the mailing list in a bit.\n\nPatrick\n\ndiff --git a/wrapper.c b/wrapper.c\nindex be8fa575e6..d989c78b4b 100644\n--- a/wrapper.c\n+++ b/wrapper.c\n@@ -323,21 +323,60 @@ ssize_t write_in_full(int fd, const void *buf, size_t count)\n \treturn total;\n }\n \n+ssize_t xwritev(int fd, struct iovec *iov, int iovcnt)\n+{\n+\tssize_t bytes_written;\n+\tsize_t total_length;\n+\tint i;\n+\n+\t/*\n+\t * We need to make sure that writev(3p) call does not write more than\n+\t * `MAX_IO_SIZE` many bytes. If we do exceed that limit, we only pass\n+\t * those iovecs to writev(3p) that sum up to less than the limit.\n+\t *\n+\t * If on the other hand the first iovec entry already exceeds this\n+\t * limit we'll instead use xwrite() to write it, which knows to handle\n+\t * `MAX_IO_SIZE` for us.\n+\t */\n+\tfor (i = 0, total_length = 0; i < iovcnt; i++) {\n+\t\tif (unsigned_add_overflows(total_length, iov[i].iov_len))\n+\t\t\tbreak;\n+\n+\t\ttotal_length += iov[i].iov_len;\n+\t\tif (total_length > MAX_IO_SIZE)\n+\t\t\tbreak;\n+\t}\n+\n+\tif (i < iovcnt) {\n+\t\t/*\n+\t\t * The first entry exceeds MAX_IO_SIZE, so we pass it to\n+\t\t * xwrite, which knows to handle this case.\n+\t\t */\n+\t\tif (!i)\n+\t\t\treturn xwrite(fd, iov->iov_base, iov->iov_len);\n+\t\tiovcnt = i;\n+\t}\n+\n+\tbytes_written = writev(fd, iov, iovcnt);\n+\tif (!bytes_written) {\n+\t\terrno = ENOSPC;\n+\t\treturn -1;\n+\t}\n+\n+\treturn bytes_written;\n+}\n+\n ssize_t writev_in_full(int fd, struct iovec *iov, int iovcnt)\n {\n \tssize_t total_written = 0;\n \n \twhile (iovcnt) {\n-\t\tssize_t bytes_written = writev(fd, iov, iovcnt);\n-\t\tif (bytes_written < 0) {\n+\t\tssize_t bytes_written = xwritev(fd, iov, iovcnt);\n+\t\tif (bytes_written <= 0) {\n \t\t\tif (errno == EINTR || errno == EAGAIN)\n \t\t\t\tcontinue;\n \t\t\treturn -1;\n \t\t}\n-\t\tif (!bytes_written) {\n-\t\t\terrno = ENOSPC;\n-\t\t\treturn -1;\n-\t\t}\n \n \t\ttotal_written += bytes_written;\n \ndiff --git a/wrapper.h b/wrapper.h\nindex 27519b32d1..a6287d7f4d 100644\n--- a/wrapper.h\n+++ b/wrapper.h\n@@ -16,6 +16,7 @@ void *xmmap_gently(void *start, size_t length, int prot, int flags, int fd, off_\n int xopen(const char *path, int flags, ...);\n ssize_t xread(int fd, void *buf, size_t len);\n ssize_t xwrite(int fd, const void *buf, size_t len);\n+ssize_t xwritev(int fd, struct iovec *iov, int iovcnt);\n ssize_t xpread(int fd, void *buf, size_t len, off_t offset);\n int xdup(int fd);\n FILE *xfopen(const char *path, const char *mode);\n"},{"id":"541271","messageId":"021a01dcc827$4e6342c0$eb29c840$@nexbridge.com","threadId":"65454","inReplyTo":"addgkjiB80pgKw69@pks.im","subject":"RE: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2026-04-09T13:46:39Z","receivedAt":"2026-04-09T13:47:03Z","isPatch":false,"body":"On April 9, 2026 4:17 AM, Patrick Steinhardt wrote\n>On Thu, Apr 09, 2026 at 12:20:26AM +0000, brian m. carlson wrote:\n>> On 2026-04-08 at 22:32:33, Jeff King wrote:\n>> > I think writev() is buying us something when it works (it is hlving\n>> > the number of writes for sideband packets). And it works when either:\n>> >\n>> >   1. the platform is OK with writing up to 64k in a single writev()\n>> >\n>> >   2. the platform has a limit that is small (like NonStop here), but\n>> >      writes less than MAX_IO_SIZE work and will save a write() call\n>> >\n>> > If we just care about (1), then the right solution is to declare\n>> > that\n>> > writev() isn't fully functional for us on some platforms, and they\n>> > should build with NO_WRITEV. And we should probably embed that in\n>> > config.mak.uname.\n>\n>Yeah, agreed. I think we shouldn't make ourselves a hostage to platforms\nthat don't\n>have reasonable support for writev(3p), as it does buy us something on the\n>majority of platforms that actually support it well.\n>\n>That of course doesn't mean that we shouldn't support such platforms.\n>\n>> Looking at POSIX, there doesn't seem to be any constraints on the size\n>> of individual vectors other than that they must total to less than\n>> SSIZE_MAX.  iovcnt can be limited to 16, but I don't think we're\n>> hitting that here.  POSIX does say that SSIZE_MAX does not need to\n>> exceed 32767, which may be what's going on here, although that does\n>> seem like an unreasonable value for a real system.  Linux, FreeBSD,\n>> and NetBSD all set SSIZE_MAX to either INT_MAX or LONG_MAX.\n>>\n>> I also think that 64 KiB is more than reasonable in terms of the size\n>> that people should be able to send.  I'd personally expect to be able\n>> to send values much larger, at least 512 KiB, and I have code that\n>> expects even larger (16 MiB).\n>>\n>> So I'd simply say that for systems that have a constraint on the size\n>> that is \"too small\", they should just use NO_WRITEV.\n>\n>I would be happy with this as an intermediate step, as Randall has\nconfirmed it\n>would fix the issue. It is the least intrusive step and has the lowest\nrisk.\n>\n>> However, I don't have a strong opinion on this and if people want to\n>> do the proposal for option 2, that's fine with me.\n>\n>I think in the long term this is the most sensible approach though so that\nwe don't\n>have to special-case platforms. I've crafted the below alternative to\nPeff's patch, and\n>I think it's ultimately not too bad.\n>\n>One question to Randall though: does MAX_IO_SIZE apply to the overall size\nof the\n>iovec or to the individual iovec entries? I think it should be the latter,\nbut I cannot\n>easily verify and couldn't find any docs around this. So could you please\ntry the\n>patch at the end of this mail to verify that it works on your system?\n>\n>In any case, I've tested that my patch also works when defining MAX_IO_SIZE\nto\n>128 bytes on my system, which hopefully demonstrates that it works as\nexpected:\n>\n>diff --git a/git-compat-util.h b/git-compat-util.h index\n4b4ea2498f..8e02b5f673\n>100644\n>--- a/git-compat-util.h\n>+++ b/git-compat-util.h\n>@@ -690,14 +690,8 @@ static inline uint64_t u64_add(uint64_t a, uint64_t b)\n>  * to override this, if the definition of SSIZE_MAX given by the platform\n>  * is broken.\n>  */\n>-#ifndef MAX_IO_SIZE\n>-# define MAX_IO_SIZE_DEFAULT (8*1024*1024) -# if defined(SSIZE_MAX) &&\n>(SSIZE_MAX < MAX_IO_SIZE_DEFAULT) -#  define MAX_IO_SIZE SSIZE_MAX -# else\n-\n>#  define MAX_IO_SIZE MAX_IO_SIZE_DEFAULT -# endif -#endif\n>+#undef MAX_IO_SIZE\n>+#define MAX_IO_SIZE 128\n>\n> #ifdef HAVE_ALLOCA_H\n> # include <alloca.h>\n>\n>I'm happy to go either way, but think that we should definitely aim for the\nbelow\n>patch eventually. Just let me know which way you prefer and I'm happy to\npolish up\n>the patch.\n>\n>Patrick\n>\n>diff --git a/wrapper.c b/wrapper.c\n>index be8fa575e6..645dbc5f20 100644\n>--- a/wrapper.c\n>+++ b/wrapper.c\n>@@ -323,21 +323,50 @@ ssize_t write_in_full(int fd, const void *buf, size_t\ncount)\n> \treturn total;\n> }\n>\n>+ssize_t xwritev(int fd, struct iovec *iov, int iovcnt) {\n>+\tssize_t bytes_written;\n>+\tint i;\n>+\n>+\t/*\n>+\t * We need to make sure that no individual iovec entry exceeds\n>+\t * `MAX_IO_SIZE`. If there's any entry that does exceed this limit\n>+\t * we'll pass all entries up to it to `writev()`, and then process\nthe\n>+\t * exceeding entry via a call to `xwrite()`.\n>+\t */\n>+\tfor (i = 0; i < iovcnt; i++)\n>+\t\tif (iov[i].iov_len > MAX_IO_SIZE)\n>+\t\t\tbreak;\n>+\tif (i < iovcnt) {\n>+\t\t/*\n>+\t\t * The first entry exceeds MAX_IO_SIZE, so we pass it to\n>+\t\t * xwrite, which knows to handle his case.\n>+\t\t */\n>+\t\tif (!i)\n>+\t\t\treturn xwrite(fd, iov->iov_base, iov->iov_len);\n>+\t\tiovcnt = i;\n>+\t}\n>+\n>+\tbytes_written = writev(fd, iov, iovcnt);\n>+\tif (!bytes_written) {\n>+\t\terrno = ENOSPC;\n>+\t\treturn -1;\n>+\t}\n>+\n>+\treturn bytes_written;\n>+}\n>+\n> ssize_t writev_in_full(int fd, struct iovec *iov, int iovcnt)  {\n> \tssize_t total_written = 0;\n>\n> \twhile (iovcnt) {\n>-\t\tssize_t bytes_written = writev(fd, iov, iovcnt);\n>-\t\tif (bytes_written < 0) {\n>+\t\tssize_t bytes_written = xwritev(fd, iov, iovcnt);\n>+\t\tif (bytes_written <= 0) {\n> \t\t\tif (errno == EINTR || errno == EAGAIN)\n> \t\t\t\tcontinue;\n> \t\t\treturn -1;\n> \t\t}\n>-\t\tif (!bytes_written) {\n>-\t\t\terrno = ENOSPC;\n>-\t\t\treturn -1;\n>-\t\t}\n>\n> \t\ttotal_written += bytes_written;\n>\n>diff --git a/wrapper.h b/wrapper.h\n>index 27519b32d1..a6287d7f4d 100644\n>--- a/wrapper.h\n>+++ b/wrapper.h\n>@@ -16,6 +16,7 @@ void *xmmap_gently(void *start, size_t length, int prot,\nint\n>flags, int fd, off_  int xopen(const char *path, int flags, ...);  ssize_t\nxread(int fd, void\n>*buf, size_t len);  ssize_t xwrite(int fd, const void *buf, size_t len);\n>+ssize_t xwritev(int fd, struct iovec *iov, int iovcnt);\n> ssize_t xpread(int fd, void *buf, size_t len, off_t offset);  int xdup(int\nfd);  FILE\n>*xfopen(const char *path, const char *mode);\n\nPlease do not make the change in git-compat-util. This will break xwrite().\nWe already have MAX_IO_SIZE working and verified from years ago. Changing\nthat will remove our platform from being supportable.\n\n"},{"id":"541302","messageId":"20260409203338.GB3076846@coredump.intra.peff.net","threadId":"65454","inReplyTo":"021a01dcc827$4e6342c0$eb29c840$@nexbridge.com","subject":"Re: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2026-04-09T20:33:38Z","receivedAt":"2026-04-09T20:33:40Z","isPatch":false,"body":"On Thu, Apr 09, 2026 at 09:46:39AM -0400, rsbecker@nexbridge.com wrote:\n\n> >--- a/git-compat-util.h\n> >+++ b/git-compat-util.h\n> >@@ -690,14 +690,8 @@ static inline uint64_t u64_add(uint64_t a, uint64_t b)\n> >  * to override this, if the definition of SSIZE_MAX given by the platform\n> >  * is broken.\n> >  */\n> >-#ifndef MAX_IO_SIZE\n> >-# define MAX_IO_SIZE_DEFAULT (8*1024*1024) -# if defined(SSIZE_MAX) &&\n> >(SSIZE_MAX < MAX_IO_SIZE_DEFAULT) -#  define MAX_IO_SIZE SSIZE_MAX -# else\n> -\n> >#  define MAX_IO_SIZE MAX_IO_SIZE_DEFAULT -# endif -#endif\n> >+#undef MAX_IO_SIZE\n> >+#define MAX_IO_SIZE 128\n> [...]\n> Please do not make the change in git-compat-util. This will break xwrite().\n> We already have MAX_IO_SIZE working and verified from years ago. Changing\n> that will remove our platform from being supportable.\n\nI think that was just there to demonstrate that the patch works\nregardless of the size, and would not be included in the final.\nBuilding with:\n\n  make CFLAGS=-DMAX_IO_SIZE=128\n\nis probably a nicer way of doing that, though. ;)\n\n-Peff\n"},{"id":"541304","messageId":"20260409205145.GC3076846@coredump.intra.peff.net","threadId":"65454","inReplyTo":"addgkjiB80pgKw69@pks.im","subject":"Re: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2026-04-09T20:51:45Z","receivedAt":"2026-04-09T20:51:47Z","isPatch":false,"body":"On Thu, Apr 09, 2026 at 10:17:22AM +0200, Patrick Steinhardt wrote:\n\n> > that people should be able to send.  I'd personally expect to be able to\n> > send values much larger, at least 512 KiB, and I have code that expects\n> > even larger (16 MiB).\n> > \n> > So I'd simply say that for systems that have a constraint on the size\n> > that is \"too small\", they should just use NO_WRITEV.\n> \n> I would be happy with this as an intermediate step, as Randall has\n> confirmed it would fix the issue. It is the least intrusive step and has\n> the lowest risk.\n\nThe only downside is that if there are other systems with a small\nSSIZE_MAX, they'll need similar treatment.\n\nI took a look at auto-detecting this at the preprocessor stage. The\nconditional itself is not too bad, but we have to compile writev.o\nunconditionally, as well as re-order some definitions:\n\ndiff --git a/Makefile b/Makefile\nindex 5d22394c2e..f9b4dd8ae3 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1125,6 +1125,7 @@ LIB_OBJS += compat/nonblock.o\n LIB_OBJS += compat/obstack.o\n LIB_OBJS += compat/open.o\n LIB_OBJS += compat/terminal.o\n+LIB_OBJS += compat/writev.o\n LIB_OBJS += compiler-tricks/not-constant.o\n LIB_OBJS += config.o\n LIB_OBJS += connect.o\n@@ -2031,7 +2032,6 @@ ifdef NO_PREAD\n endif\n ifdef NO_WRITEV\n \tCOMPAT_CFLAGS += -DNO_WRITEV\n-\tCOMPAT_OBJS += compat/writev.o\n endif\n ifdef NO_FAST_WORKING_DIRECTORY\n \tBASIC_CFLAGS += -DNO_FAST_WORKING_DIRECTORY\ndiff --git a/compat/posix.h b/compat/posix.h\nindex 94699a03fa..57c11938c7 100644\n--- a/compat/posix.h\n+++ b/compat/posix.h\n@@ -326,6 +326,39 @@ int git_lstat(const char *, struct stat *);\n ssize_t git_pread(int fd, void *buf, size_t count, off_t offset);\n #endif\n \n+/*\n+ * Limit size of IO chunks, because huge chunks only cause pain.  OS X\n+ * 64-bit is buggy, returning EINVAL if len >= INT_MAX; and even in\n+ * the absence of bugs, large chunks can result in bad latencies when\n+ * you decide to kill the process.\n+ *\n+ * We pick 8 MiB as our default, but if the platform defines SSIZE_MAX\n+ * that is smaller than that, clip it to SSIZE_MAX, as a call to\n+ * read(2) or write(2) larger than that is allowed to fail.  As the last\n+ * resort, we allow a port to pass via CFLAGS e.g. \"-DMAX_IO_SIZE=value\"\n+ * to override this, if the definition of SSIZE_MAX given by the platform\n+ * is broken.\n+ */\n+#ifndef MAX_IO_SIZE\n+# define MAX_IO_SIZE_DEFAULT (8*1024*1024)\n+# if defined(SSIZE_MAX) && (SSIZE_MAX < MAX_IO_SIZE_DEFAULT)\n+#  define MAX_IO_SIZE SSIZE_MAX\n+# else\n+#  define MAX_IO_SIZE MAX_IO_SIZE_DEFAULT\n+# endif\n+#endif\n+\n+/*\n+ * Temporary hack: our use of writev() does not respect MAX_IO_SIZE, but we\n+ * never feed more than a single pkt-line packet to it. Disable writev()\n+ * and use our fallback wrapper if the system can't support that size.\n+ *\n+ * This should be removed once writev() supports MAX_IO_SIZE.\n+ */\n+#if !defined(NO_WRITEV) && (MAX_IO_SIZE < 65536)\n+#define NO_WRITEV\n+#endif\n+\n #ifdef NO_WRITEV\n #define writev git_writev\n #define iovec git_iovec\ndiff --git a/git-compat-util.h b/git-compat-util.h\nindex 4b4ea2498f..60108459f3 100644\n--- a/git-compat-util.h\n+++ b/git-compat-util.h\n@@ -677,28 +677,6 @@ static inline uint64_t u64_add(uint64_t a, uint64_t b)\n \treturn a + b;\n }\n \n-/*\n- * Limit size of IO chunks, because huge chunks only cause pain.  OS X\n- * 64-bit is buggy, returning EINVAL if len >= INT_MAX; and even in\n- * the absence of bugs, large chunks can result in bad latencies when\n- * you decide to kill the process.\n- *\n- * We pick 8 MiB as our default, but if the platform defines SSIZE_MAX\n- * that is smaller than that, clip it to SSIZE_MAX, as a call to\n- * read(2) or write(2) larger than that is allowed to fail.  As the last\n- * resort, we allow a port to pass via CFLAGS e.g. \"-DMAX_IO_SIZE=value\"\n- * to override this, if the definition of SSIZE_MAX given by the platform\n- * is broken.\n- */\n-#ifndef MAX_IO_SIZE\n-# define MAX_IO_SIZE_DEFAULT (8*1024*1024)\n-# if defined(SSIZE_MAX) && (SSIZE_MAX < MAX_IO_SIZE_DEFAULT)\n-#  define MAX_IO_SIZE SSIZE_MAX\n-# else\n-#  define MAX_IO_SIZE MAX_IO_SIZE_DEFAULT\n-# endif\n-#endif\n-\n #ifdef HAVE_ALLOCA_H\n # include <alloca.h>\n # define xalloca(size)      (alloca(size))\n\n\nIt's not _too_ complicated, but it's a little more than I'd like for an\n-rc2. Yet another option is to swap out NO_WRITEV for USE_WRITEV,\neffectively reverting the feature temporarily until it supports\nMAX_IO_SIZE. But that's also slightly non-trivial.\n\nSo I dunno. Do we want just:\n\ndiff --git a/config.mak.uname b/config.mak.uname\nindex ccb3f71881..b672b1d077 100644\n--- a/config.mak.uname\n+++ b/config.mak.uname\n@@ -654,6 +654,7 @@ ifeq ($(uname_S),NONSTOP_KERNEL)\n \tCSPRNG_METHOD = openssl\n \tSANE_TOOL_PATH = /usr/coreutils/bin:/usr/local/bin\n \tSHELL_PATH = /usr/coreutils/bin/bash\n+\tNO_WRITEV = NotUntilItSupportsMaxIOSize\n endif\n ifeq ($(uname_S),OS/390)\n \tNO_SYS_POLL_H = YesPlease\n\nand hope it's the only one?\n\n-Peff\n"},{"id":"541314","messageId":"029701dcc871$d055dd20$71019760$@nexbridge.com","threadId":"65454","inReplyTo":"20260409203338.GB3076846@coredump.intra.peff.net","subject":"RE: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2026-04-09T22:40:00Z","receivedAt":"2026-04-09T22:40:16Z","isPatch":false,"body":"On April 9, 2026 4:34 PM, Jeff King wrote:\n>On Thu, Apr 09, 2026 at 09:46:39AM -0400, rsbecker@nexbridge.com wrote:\n>\n>> >--- a/git-compat-util.h\n>> >+++ b/git-compat-util.h\n>> >@@ -690,14 +690,8 @@ static inline uint64_t u64_add(uint64_t a,\n>> >uint64_t b)\n>> >  * to override this, if the definition of SSIZE_MAX given by the\n>> >platform\n>> >  * is broken.\n>> >  */\n>> >-#ifndef MAX_IO_SIZE\n>> >-# define MAX_IO_SIZE_DEFAULT (8*1024*1024) -# if defined(SSIZE_MAX)\n>> >&& (SSIZE_MAX < MAX_IO_SIZE_DEFAULT) -#  define MAX_IO_SIZE SSIZE_MAX\n>> >-# else\n>> -\n>> >#  define MAX_IO_SIZE MAX_IO_SIZE_DEFAULT -# endif -#endif\n>> >+#undef MAX_IO_SIZE\n>> >+#define MAX_IO_SIZE 128\n>> [...]\n>> Please do not make the change in git-compat-util. This will break xwrite().\n>> We already have MAX_IO_SIZE working and verified from years ago.\n>> Changing that will remove our platform from being supportable.\n>\n>I think that was just there to demonstrate that the patch works regardless of the\n>size, and would not be included in the final.\n>Building with:\n>\n>  make CFLAGS=-DMAX_IO_SIZE=128\n>\n>is probably a nicer way of doing that, though. ;)\n\nWe had that set properly in git-compat-util.h for years. MAX_IO_SIZE should be set to SSIZE_MAX if SSIZE_MAX is defined.\n#ifndef MAX_IO_SIZE\n# define MAX_IO_SIZE_DEFAULT (8*1024*1024)\n# if defined(SSIZE_MAX) && (SSIZE_MAX < MAX_IO_SIZE_DEFAULT)\n#  define MAX_IO_SIZE SSIZE_MAX\n# else\n#  define MAX_IO_SIZE MAX_IO_SIZE_DEFAULT\n\n"},{"id":"541320","messageId":"20260409225806.GA3133902@coredump.intra.peff.net","threadId":"65454","inReplyTo":"029701dcc871$d055dd20$71019760$@nexbridge.com","subject":"Re: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2026-04-09T22:58:06Z","receivedAt":"2026-04-09T22:58:08Z","isPatch":false,"body":"On Thu, Apr 09, 2026 at 06:40:00PM -0400, rsbecker@nexbridge.com wrote:\n\n> >> Please do not make the change in git-compat-util. This will break xwrite().\n> >> We already have MAX_IO_SIZE working and verified from years ago.\n> >> Changing that will remove our platform from being supportable.\n> >\n> >I think that was just there to demonstrate that the patch works regardless of the\n> >size, and would not be included in the final.\n> >Building with:\n> >\n> >  make CFLAGS=-DMAX_IO_SIZE=128\n> >\n> >is probably a nicer way of doing that, though. ;)\n> \n> We had that set properly in git-compat-util.h for years. MAX_IO_SIZE should be set to SSIZE_MAX if SSIZE_MAX is defined.\n> #ifndef MAX_IO_SIZE\n> # define MAX_IO_SIZE_DEFAULT (8*1024*1024)\n> # if defined(SSIZE_MAX) && (SSIZE_MAX < MAX_IO_SIZE_DEFAULT)\n> #  define MAX_IO_SIZE SSIZE_MAX\n> # else\n> #  define MAX_IO_SIZE MAX_IO_SIZE_DEFAULT\n\nRight. We would retain that. I think the point was that Patrick was\ndropping MAX_IO_SIZE artificially on his Linux system to exercise the\nnew code.\n\n-Peff\n"},{"id":"541323","messageId":"adh92BCD47PlK9BM@pks.im","threadId":"65454","inReplyTo":"20260409225806.GA3133902@coredump.intra.peff.net","subject":"Re: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-04-10T04:34:32Z","receivedAt":"2026-04-10T04:34:39Z","isPatch":false,"body":"On Thu, Apr 09, 2026 at 06:58:06PM -0400, Jeff King wrote:\n> On Thu, Apr 09, 2026 at 06:40:00PM -0400, rsbecker@nexbridge.com wrote:\n> \n> > >> Please do not make the change in git-compat-util. This will break xwrite().\n> > >> We already have MAX_IO_SIZE working and verified from years ago.\n> > >> Changing that will remove our platform from being supportable.\n> > >\n> > >I think that was just there to demonstrate that the patch works regardless of the\n> > >size, and would not be included in the final.\n> > >Building with:\n> > >\n> > >  make CFLAGS=-DMAX_IO_SIZE=128\n> > >\n> > >is probably a nicer way of doing that, though. ;)\n> > \n> > We had that set properly in git-compat-util.h for years. MAX_IO_SIZE should be set to SSIZE_MAX if SSIZE_MAX is defined.\n> > #ifndef MAX_IO_SIZE\n> > # define MAX_IO_SIZE_DEFAULT (8*1024*1024)\n> > # if defined(SSIZE_MAX) && (SSIZE_MAX < MAX_IO_SIZE_DEFAULT)\n> > #  define MAX_IO_SIZE SSIZE_MAX\n> > # else\n> > #  define MAX_IO_SIZE MAX_IO_SIZE_DEFAULT\n> \n> Right. We would retain that. I think the point was that Patrick was\n> dropping MAX_IO_SIZE artificially on his Linux system to exercise the\n> new code.\n\nYup, exactly. I don't have any intent to change the above snippet in our\ncode base.\n\nPatrick\n"},{"id":"541330","messageId":"1b4c3562-8501-433e-afaf-2cb3a295b4ac@kdbg.org","threadId":"65454","inReplyTo":"addgkjiB80pgKw69@pks.im","subject":"Re: Git 2.54.0-rc1, subtests of t5310, t5326, t5327","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2026-04-10T07:35:34Z","receivedAt":"2026-04-10T08:10:19Z","isPatch":false,"body":"Am 09.04.26 um 10:17 schrieb Patrick Steinhardt:\n> Yeah, agreed. I think we shouldn't make ourselves a hostage to platforms\n> that don't have reasonable support for writev(3p), as it does buy us\n> something on the majority of platforms that actually support it well.\n> \n> That of course doesn't mean that we shouldn't support such platforms.\n\nMy take on this matter is that the use writev in Git's code gives POSIX\ncentered contributors a false sense of security. POSIX's writev offers a\nnumber of guarantees that a compatibility implementation cannot provide.\nIf uses of writev proliferate, I forsee the time when someone reports a\nproblem on a compatibility platform, and one of us will point at POSIX\nand say \"but that cannot happen because we have this and that guarantee\".\n\nWhat did we gain with writev? There are half that many system calls. So\nwhat? Have there been any hard numbers about better performance, for\nexample? Not even the call sites became simpler, see the commit that\nintroduced the only caller: 26986f4cbaf3 (\"sideband: use writev(3p) to\nsend pktlines\", 2026-03-13).\n\n-- Hannes\n\n"}]}