git/list[1] front-page[2] threads[3] people[4] search[5] about
 

RE: Git 2.54.0-rc1, subtests of t5310, t5326, t5327

From
rsbecker@nexbridge.com <rsbecker@nexbridge.com>
Date
Apr 8, 2026, 23:15 UTC
Message-ID
<018701dcc7ad$8c3addd0$a4b09970$@nexbridge.com>
In-Reply-To
<xmqqv7e1w05u.fsf@gitster.g>
On April 8, 2026 6:35 PM, Junio C Hamano wrote
Show 22 quoted lines
>Junio C Hamano <gitster@pobox.com> writes:
>
>> Junio C Hamano <gitster@pobox.com> writes:
>>
>>> To be quite honest, I am not sure if it is even worth using writev()
>>> if we need a loop that protects against shrot writes, so unless I am
>>> grossly mistaken (e.g., perhaps there is some guarantee that there
>>> won't be any short writes for writev() that sends data smaller than
>>> 64k that I missed in the docs), the best course of action might be to
>>> revert the change to use writev() and use the two write(2)s as
>>> before, *if* we actually observe that the current code is broken by
>>> short writes.
>>
>> Ah, sorry, I should have double checked the actual code.  We already
>> use a looping writev_in_full() that wraps writev(), so there is
>> nothing extra that we still need to do to prepare for short writes.
>>
>> Unfortunately, comparing write_in_full() vs writev_in_full(), there is
>> nothing that corresponds to xwrite() that can be used to hide the
>> short writes and chomps an originally larger I/O into smaller pieces.
>
>Oops, the beauty of having xwrite() is *not* that it hides short writes (it
doesn't),
>but it can be used to pretend that short writes happened on platforms with
>unreasonably small I/O limit by setting MAX_IO_SIZE to unusually low.  But
the
Show 17 quoted lines
>point that ...
>
>> Unlike write() that we may receive a single linear large sequence of
>> bytes, which we can choose to chomp into artificially smaller pieces
>> and write them out (up to 8MB by default), writev() API lets the
>> caller to prepare chunks of memory and I do not think there is a good
>> way for the writev_in_full() at the lower layer to chomp these into
>> smaller pieces, and even if we could, that would defeat the whole
>> reason why we rewrote the original code that used
>> write_in_full() into using writev(), i.e., to avoid extra allocation
>> (and extra system calls---but if your I/O layer is limited to very
>> small writes, no matter how we chop it, you will have to issue extra
>> system calls to flush all of the data out).
>>
>> So, I dunno.
>
>... I doubt that there exists a good way to have xwritev() that wraps
around writev()
>and pretend that a short write happened, instead of issuing a large I/O,
still stands.
I am partial to Peff's runtime approach, personally.

If it helps (probably does not), the limit is due to the DMA/VMM limits on the hardware. The NonStop Message system is blindingly fast when sending messages around between processes on other CPUs - far faster than I have seen on most other platforms. But there are physical limits in the chipset. It is interesting to use it in an abstracted way, like hacking the TCP/IP stack to pass handles around.

Previous: Junio C HamanoNext: Jeff King
Message 14 of 29 in “Git 2.54.0-rc1, subtests of t5310, t5326, t5327”
  1. rsbecker@nexbridge.comApr 7, 2026
  2. Jeff KingApr 8, 2026
  3. rsbecker@nexbridge.comApr 8, 2026
  4. rsbecker@nexbridge.comApr 8, 2026
  5. Jeff KingApr 8, 2026
  6. Junio C HamanoApr 8, 2026
  7. rsbecker@nexbridge.comApr 8, 2026
  8. Junio C HamanoApr 8, 2026
  9. rsbecker@nexbridge.comApr 8, 2026
  10. Junio C HamanoApr 8, 2026
  11. rsbecker@nexbridge.comApr 8, 2026
  12. Junio C HamanoApr 8, 2026
  13. Junio C HamanoApr 8, 2026
  14. rsbecker@nexbridge.comApr 8, 2026
  15. Jeff KingApr 8, 2026
  16. brian m. carlsonApr 9, 2026
  17. Patrick SteinhardtApr 9, 2026
  18. Phillip WoodApr 9, 2026
  19. Patrick SteinhardtApr 9, 2026
  20. rsbecker@nexbridge.comApr 9, 2026
  21. Jeff KingApr 9, 2026
  22. rsbecker@nexbridge.comApr 9, 2026
  23. Jeff KingApr 9, 2026
  24. Patrick SteinhardtApr 10, 2026
  25. Jeff KingApr 9, 2026
  26. Johannes SixtApr 10, 2026
  27. rsbecker@nexbridge.comApr 8, 2026
  28. Jeff KingApr 8, 2026
  29. Jeff KingApr 8, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.