git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 0/2] fix http deadlock on giant ref negotiations

From
Dennis Kaarsemaker <dennis@kaarsemaker.net>
Date
May 15, 2015, 07:41 UTC
Message-ID
<1431675680.14042.39.camel@kaarsemaker.net>
In-Reply-To
<20150515062901.GA30768@peff.net>
On vr, 2015-05-15 at 02:29 -0400, Jeff King wrote:
Show 26 quoted lines
> On Wed, May 13, 2015 at 08:47:24PM -0400, Jeff King wrote:
> 
> > The fundamental problem is the deadlock on the server side, which is
> > producing bogus protocol output. And that's a mismatch between what
> > Apache expects (that the CGI will read all of the input request and then
> > generate an output request) and what the CGI wants to do (stream output
> > as it reads the input).
> 
> At first I was irritated with Apache for this. But thinking on it more,
> it's really due to our shoe-horning of a full-duplex protocol into the
> half-duplex HTTP protocol. Even if we could convince Apache to work in a
> full-duplex way here, and even if our client is full-duplex (since
> otherwise we are just trading pipe buffers for TCP buffers), we still
> may face arbitrary HTTP proxies or other infrastructure in the middle.
> 
> So here's a series to try to address the issue. The first patch is a
> fixed version of the die-recursion fixup I posted earlier. The second is
> the interesting one.
> 
>   [1/2]: http-backend: fix die recursion with custom handler
>   [2/2]: http-backend: spool ref negotiation requests to buffer
> 
> I have no clue how to write a test that would trigger this reliably
> without requiring a gigantic test fixture. However, I did confirm that
> it fixes the problem on the chromium case you provided (which otherwise
> deadlocks reliably for me).

This looks similar to the failure I posted about alst year in http://thread.gmane.org/gmane.comp.version-control.git/258514

Though the issue is different, it has the same 'hanging git fetch' symptom due to the deadlock between upload-pack and http-backend.

The patch I sent back then is suboptimal, as it can cause larger packs than necessary (we still use it though, as the alternative is a non-working git), but it does include a test you may be able to use to verify your fix, if this is indeed the same issue.

-- 
Dennis Kaarsemaker
http://www.kaarsemaker.net
Previous: Jeff KingNext: Jeff King
Message 20 of 24 in “Clone hangs when done over http with --reference”
  1. Konstantin RyabitsevMay 13, 2015
  2. Jeff KingMay 14, 2015
  3. http-backend: fix die recursion with custom handlerJeff King, May 14, 2015
  4. Jeff KingMay 15, 2015
  5. Konstantin RyabitsevMay 14, 2015
  6. Jeff KingMay 14, 2015
  7. 0/2 fix http deadlock on giant ref negotiationsJeff King, May 15, 2015
  8. 1/2 http-backend: fix die recursion with custom handlerJeff King, May 15, 2015
  9. 2/2 http-backend: spool ref negotiation requests to bufferJeff King, May 15, 2015
  10. Junio C HamanoMay 15, 2015
  11. Konstantin RyabitsevMay 15, 2015
  12. 0/3 fix http deadlock on giant ref negotiationsJeff King, May 20, 2015
  13. 1/3 http-backend: fix die recursion with custom handlerJeff King, May 20, 2015
  14. 2/3 t5551: factor out tag creationJeff King, May 20, 2015
  15. 3/3 http-backend: spool ref negotiation requests to bufferJeff King, May 20, 2015
  16. Konstantin RyabitsevMay 26, 2015
  17. Jeff KingMay 26, 2015
  18. Junio C HamanoMay 26, 2015
  19. Jeff KingMay 15, 2015
  20. Dennis KaarsemakerMay 15, 2015
  21. Jeff KingMay 15, 2015
  22. Dennis KaarsemakerMay 15, 2015
  23. Jeff KingMay 15, 2015
  24. Dennis KaarsemakerMay 15, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.