git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: data loss when doing ls-remote and piped to command

From
MGMike Galbraith <efault@gmx.de>
Date
Sep 18, 2021, 06:33 UTC
Message-ID
<9bd95c0ad91bd490adf2b6e57495411a0f72fe50.camel@gmx.de>
In-Reply-To
<2722184.bRktqFsmb4@devpool47>
On Fri, 2021-09-17 at 08:59 +0200, Rolf Eike Beer wrote:
Show 5 quoted lines
>
> What you need is a _fast_ git server. kernel.org or github.com seem to be too
> slow for this if you don't sit somewhere in their datacenter. Use something in
> your local network, a Xeon E5 with lot's of RAM and connected with 1GBit/s
> Ethernet in my case.

Even faster: what's coming across that wire should be a constant (is?), variable is only delivery/consumption jitter. If there's really really a pipe problem lurking, you should also be able to trigger by saving the data once, and just catting it, letting interrupts etc provide jitter. Which stdout is left of '|' in a script shouldn't matter one whit to the interpreter/kernel conversation, they're all the same.

That said, if I had a reproducer I was confident pointed to the kernel, I'd try to bisect.. boring as hell, but highly effective.

	-Mike
Previous: Linus Torvalds
Message 13 of 13 in “data loss when doing ls-remote and piped to command”
  1. Rolf Eike BeerSep 15, 2021
  2. Junio C HamanoSep 15, 2021
  3. Rolf Eike BeerSep 16, 2021
  4. Tobias UlmerSep 16, 2021
  5. Rolf Eike BeerSep 16, 2021
  6. Mike GalbraithSep 16, 2021
  7. Mike GalbraithSep 17, 2021
  8. Linus TorvaldsSep 16, 2021
  9. Junio C HamanoSep 16, 2021
  10. Rolf Eike BeerSep 17, 2021
  11. Jeff KingSep 17, 2021
  12. Linus TorvaldsSep 17, 2021
  13. Mike GalbraithSep 18, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.