git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Git-aware HTTP transport

From
TTarmigan <tarmigan+git@gmail.com>
Date
Sep 1, 2008, 16:05 UTC
Message-ID
<905315640809010905w20f4ceeo43e7b0a14abd48a3@mail.gmail.com>
In-Reply-To
<20080829173954.GG7403@spearce.org>
On Fri, Aug 29, 2008 at 10:39 AM, Shawn O. Pearce <spearce@spearce.org> wrote:
> Yet another draft follows.  I believe that I have covered all
> comments with this draft.  But I welcome any additional ones,
> as thus far it has been a very constructive process.
Sorry I'm jumping into this a bit late, but something just occurred to me.
Show 7 quoted lines
>
> The updated protocol looks more like the current native protocol
> does.  This should make it easier to reuse code between the two
> protocol implementations.
>
> --8<--
> Smart HTTP transfer protocols
[...]
Show 10 quoted lines
> HTTP Redirects
> --------------
>
> If a POST request results in an HTTP 302 or 303 redirect response
> clients should retry the request by updating the URL and POSTing
> the same request to the new location.  Subsequent requests should
> still be sent to the original URL.
>
> This redirect behavior is unrelated to the in-payload redirect
> that is described below in "Service show-ref".

I just want to see smart http could support a new feature (please yell if git:// already supports this and I am not aware of it). The idea is from http://lkml.org/lkml/2008/8/21/347, the relevant portion being:

Greg KH wrote:
Show 12 quoted lines
>David Vrabel wrote:
>> Or you can pull the changes from the uwb branch of
>>
>> git://pear.davidvrabel.org.uk/git/uwb.git
>>
>> (Please don't clone the entire tree from here as I have very limited
>> bandwidth.)
>
> If this is an issue, I think you can use the --reference option to
> git-clone when creating the tree to reference an external tree (like
> Linus's).  That way you don't have the whole tree on your server for
> stuff like this.

I do not believe that the server (either git:// or http://) can currently be setup with --reference to redirect to another server for certain refs, but perhaps with smart http and the POST 302/303 redirect responses, this would now be possible as a way to reduce bandwidth for people's home servers? I have also seen similar requests before ("don't pull the whole kernel from me, just add my repo as a remote after you've cloned linus-2.6"), so for larger projects, it might be a nice feature. Would that be something desirable to support?

Would the current proposal be able to support this kind of partial redirect? I don't quite see how it would, but it seems very close. Perhaps if the show-ref redirect could appear partway through the show-ref response and then the client could go off, fetch the some refs from that server and then return to the original server for the remainder? Or maybe in the upload-pack negotiations, there could be a special redirect command as part of the "status continue" response that told the client to run off and look for a specific sha at another url? Something like

status continue
 S: 0014status continue
       S: 0034common <S_COMMON #1>...........................
       S: 0034common <S_COMMON #2>...........................
       ...

Otherwise, it looks very cool, but I have a few more minor questions to help my general understanding...

Show 6 quoted lines
>     If the client has sent 256 HAVE commits and has not yet
>     received one of those back from S_COMMON, or the client
>     has emptied C_PENDING it should include a "give-up"
>     command to let the server know it won't proceed:
>
>        C: 000cgive-up

What does the server do after a 000cgive-up ? Does the server send back a complete pack (like a new clone) or if not, how does clone work over smart http? Does that mean that if I fall more than 256 commits behind, I have to redownload the whole repo? Or am I missing something about the the C_PENDING commits being sparse and doing some kind of smart back-off (I'm not at all familiar with the existing receive-pack/upload-pack)?

Show 15 quoted lines
>  (s) Parse the upload-pack request:
>
>      Verify all objects in WANT are reachable from refs.  As
>      this may require walking backwards through history to
>      the very beginning on invalid requests the server may
>      use a reasonable limit of commits (e.g. 1000) walked
>      beyond any ref tip before giving up.
>
>      If no WANT objects are received, send an error:
>
>        S: 0019status error no want
>
>      If any WANT object is not reachable, send an error:
>
>        S: 001estatus error invalid want

So again, if the client falls more than 1000 commits behind (not hard to do for example during the linux merge window), and then the client WANTs HEAD^1001, what happens? Does the get nothing from the server, or does the client essentially reclone, or I am missing something?

Show 12 quoted lines
>  (s) Send the upload-pack response:
>
>     If the server has found a closed set of objects to pack or the
>     request contains "give-up", it replies with the pack and the
>     enabled capabilities.  The set of enabled capabilities is limited
>     to the intersection of what the client requested and what the
>     server supports.
>
>        S: 0010status pack
>        C: 001bcapability include-tag
>        C: 0019capability thin-pack
>        S: 000c.PACK...
Should these be all S: ... ?

Thanks, Tarmigan

Previous: Nicolas PitreNext: Tarmigan
Message 41 of 57 in “Git-aware HTTP transport”
  1. Shawn O. PearceAug 26, 2008
  2. H. Peter AnvinAug 26, 2008
  3. Shawn O. PearceAug 26, 2008
  4. david@lang.hmAug 26, 2008
  5. H. Peter AnvinAug 26, 2008
  6. david@lang.hmAug 26, 2008
  7. H. Peter AnvinAug 26, 2008
  8. Imran M YousufAug 26, 2008
  9. Nicolas PitreAug 26, 2008
  10. Shawn O. PearceAug 26, 2008
  11. H. Peter AnvinAug 26, 2008
  12. Shawn O. PearceAug 26, 2008
  13. Shawn O. PearceAug 26, 2008
  14. H. Peter AnvinAug 26, 2008
  15. Shawn O. PearceAug 26, 2008
  16. H. Peter AnvinAug 26, 2008
  17. Imran M YousufAug 27, 2008
  18. Shawn O. PearceAug 28, 2008
  19. H. Peter AnvinAug 28, 2008
  20. Shawn O. PearceAug 28, 2008
  21. H. Peter AnvinAug 28, 2008
  22. Imran M YousufAug 28, 2008
  23. Junio C HamanoAug 28, 2008
  24. Shawn O. PearceAug 28, 2008
  25. david@lang.hmAug 28, 2008
  26. Shawn O. PearceAug 28, 2008
  27. david@lang.hmAug 28, 2008
  28. Daniel StenbergAug 28, 2008
  29. Shawn O. PearceAug 28, 2008
  30. H. Peter AnvinAug 28, 2008
  31. Mike HommeyAug 28, 2008
  32. H. Peter AnvinAug 28, 2008
  33. david@lang.hmAug 28, 2008
  34. H. Peter AnvinAug 28, 2008
  35. david@lang.hmAug 28, 2008
  36. Junio C HamanoAug 29, 2008
  37. H. Peter AnvinAug 29, 2008
  38. Junio C HamanoAug 29, 2008
  39. Shawn O. PearceAug 29, 2008
  40. Nicolas PitreAug 29, 2008
  41. TarmiganSep 1, 2008
  42. TarmiganSep 1, 2008
  43. Shawn O. PearceSep 2, 2008
  44. H. Peter AnvinSep 2, 2008
  45. Shawn O. PearceSep 2, 2008
  46. TarmiganSep 2, 2008
  47. H. Peter AnvinAug 28, 2008
  48. Shawn O. PearceAug 28, 2008
  49. H. Peter AnvinAug 28, 2008
  50. Shawn O. PearceAug 28, 2008
  51. H. Peter AnvinAug 28, 2008
  52. Shawn O. PearceAug 28, 2008
  53. Nicolas PitreAug 28, 2008
  54. H. Peter AnvinAug 28, 2008
  55. H. Peter AnvinFeb 13, 2013
  56. Scott ChaconFeb 13, 2013
  57. Junio C HamanoFeb 13, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.