git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 0/2] fix http deadlock on giant ref negotiations

From
Jeff King <peff@peff.net>
Date
May 15, 2015, 08:53 UTC
Message-ID
<20150515085355.GA18890@peff.net>
In-Reply-To
<1431679490.14042.42.camel@kaarsemaker.net>
On Fri, May 15, 2015 at 10:44:50AM +0200, Dennis Kaarsemaker wrote:
Show 12 quoted lines
> > I applied the test from your patch, but couldn't get it to fail even
> > with stock git.  The test above it shrunk a bit, but I was able to tweak
> > yours to generate tags from 2001..100000, which I thought would have
> > worked.  I suspect it's something silly like the size not being quite
> > big enough for the pipe buffer on my system, or something like that.
> > Though I couldn't get it to fail even with 200,000 tags, so perhaps it's
> > something else.
> 
> The shrinkage in the test above it will actually work around the issue,
> as there are now fewer already-fetched tags to negotiate. Either
> reverting that shrinkage or executing the new test twice should do the
> trick.

Ah, right, that makes sense. I was creating the right number of tags, but not with half of them already in the repo when I did the critical fetch. I got it to fail by adding and fetching another 48,000, and then adding and fetching another 50,000 on top of that.

Interestingly, with my patch the _first_ fetch fails, that otherwise succeeds with stock git. My patch sets a maximum size on the spool buffer, and we exceed it. I guess 1MB isn't enough for pathological cases. I'm hesitant to let it expand indefinitely for security reasons, but we could probably bump it to 10MB or something.

I dunno. I'm not excited about introducing new size restrictions that were not there before.

Maybe it's time to implement git-over-websockets. ;)
-Peff
Previous: Dennis KaarsemakerNext: Dennis Kaarsemaker
Message 23 of 24 in “Clone hangs when done over http with --reference”
  1. Konstantin RyabitsevMay 13, 2015
  2. Jeff KingMay 14, 2015
  3. http-backend: fix die recursion with custom handlerJeff King, May 14, 2015
  4. Jeff KingMay 15, 2015
  5. Konstantin RyabitsevMay 14, 2015
  6. Jeff KingMay 14, 2015
  7. 0/2 fix http deadlock on giant ref negotiationsJeff King, May 15, 2015
  8. 1/2 http-backend: fix die recursion with custom handlerJeff King, May 15, 2015
  9. 2/2 http-backend: spool ref negotiation requests to bufferJeff King, May 15, 2015
  10. Junio C HamanoMay 15, 2015
  11. Konstantin RyabitsevMay 15, 2015
  12. 0/3 fix http deadlock on giant ref negotiationsJeff King, May 20, 2015
  13. 1/3 http-backend: fix die recursion with custom handlerJeff King, May 20, 2015
  14. 2/3 t5551: factor out tag creationJeff King, May 20, 2015
  15. 3/3 http-backend: spool ref negotiation requests to bufferJeff King, May 20, 2015
  16. Konstantin RyabitsevMay 26, 2015
  17. Jeff KingMay 26, 2015
  18. Junio C HamanoMay 26, 2015
  19. Jeff KingMay 15, 2015
  20. Dennis KaarsemakerMay 15, 2015
  21. Jeff KingMay 15, 2015
  22. Dennis KaarsemakerMay 15, 2015
  23. Jeff KingMay 15, 2015
  24. Dennis KaarsemakerMay 15, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.