git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: upload-pack is slow with lots of refs

From
Johannes Sixt <j6t@kdbg.org>
Date
Oct 9, 2012, 20:30 UTC
Message-ID
<5074894D.90307@kdbg.org>
In-Reply-To
<CAJo=hJsJgqZqPxucRcSgYSa0N3pcw5seT9vcu2BE8WwfJVrvKQ@mail.gmail.com>
Am 09.10.2012 08:46, schrieb Shawn Pearce:
Show 8 quoted lines
> On Mon, Oct 8, 2012 at 8:05 AM, Johannes Sixt <j6t@kdbg.org> wrote:
>> Am 05.10.2012 18:57, schrieb Shawn Pearce:
>>> Smart HTTP is not bidirectional. The client can't cut off the server.
>>
>> Smart HTTP does not need it: you already posted a better solution (I'm
>> refering to "&v=2").
> 
> Yes but then it diverges even further from the native bidirectional protocol.

I won't argue here because I know next to nothing about Smart HTTP. But it sounds like you either have compatibility, but a diverging protocol or at least implementation, or no compatibility.

Show 17 quoted lines
>> +static int client_spoke(void)
>> +{
>> +       struct pollfd pfd;
>> +       pfd.fd = 0;
>> +       pfd.events = POLLIN;
>> +       return poll(&pfd, 1, 0) > 0 &&
>> +               (pfd.revents & (POLLIN|POLLHUP));
> 
> Except doing this in Java is harder on an arbitrary InputStream type.
> I guess we really only care about basic TCP, in which case we can use
> NIO to implement an emulation of poll, and SSH, where MINA SSHD
> probably doesn't provide a way to see if the client has given us data
> without blocking. That makes supporting v2 really hard in e.g. Gerrit
> Code Review. You could argue that its improper to attempt to implement
> a network protocol in a language whose standard libraries have gone
> out of their way to prevent you from polling to see if data is
> immediately available, but I prefer to ignore such arguments.

Can't you read the inbound stream in a second thread while the first thread writes the advertisements to the outbound stream? Then you don't even need to poll; you can just read the 4-byte length header, stash it away and set a flag. The implementation of client_spoke() would only amount to check that flag.

> As it turns out we don't really have this problem with git://. Clients
> can bury a v2 request in the extended headers where the host line
> appears today.

I tried, but it seems that todays git-daemons are too strict and accept only \0host=foo\0, nothing else :-(

-- Hannes
Previous: Shawn PearceNext: Johannes Sixt
Message 14 of 24 in “upload-pack is slow with lots of refs”
  1. Ævar Arnfjörð BjarmasonOct 3, 2012
  2. Nguyen Thai Ngoc DuyOct 3, 2012
  3. Jeff KingOct 3, 2012
  4. Junio C HamanoOct 3, 2012
  5. Jeff KingOct 3, 2012
  6. Shawn PearceOct 3, 2012
  7. Jeff KingOct 3, 2012
  8. Sascha CunzOct 4, 2012
  9. Jeff KingOct 5, 2012
  10. Johannes SixtOct 5, 2012
  11. Shawn PearceOct 5, 2012
  12. Johannes SixtOct 8, 2012
  13. Shawn PearceOct 9, 2012
  14. Johannes SixtOct 9, 2012
  15. Johannes SixtOct 9, 2012
  16. Ævar Arnfjörð BjarmasonOct 3, 2012
  17. Jeff KingOct 3, 2012
  18. Ævar Arnfjörð BjarmasonOct 3, 2012
  19. Jeff KingOct 3, 2012
  20. Ævar Arnfjörð BjarmasonOct 3, 2012
  21. Ævar Arnfjörð BjarmasonOct 3, 2012
  22. Jeff KingOct 3, 2012
  23. Ævar Arnfjörð BjarmasonOct 3, 2012
  24. Junio C HamanoOct 3, 2012

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.