git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [Patch] Prevent cloning over http from spewing

From
Jeff King <peff@peff.net>
Date
Jun 3, 2009, 19:24 UTC
Message-ID
<20090603192420.GA29610@coredump.intra.peff.net>
In-Reply-To
<20090603191555.GL3355@spearce.org>
On Wed, Jun 03, 2009 at 12:15:55PM -0700, Shawn O. Pearce wrote:
Show 6 quoted lines
> What we could do is try to organize the fetch queue by object type,
> get all commits, then all trees, then blobs.  The blobs are the
> bulk of the data, and by the time we hit them, we should be able
> to give some estimate on progress because we have all of the ones
> we need to fetch in our fetch queue.  But its only a "object count"
> sort of thing, not a byte count.

That's clever, and I think an "object count" would be fine (after all, that is all that git:// fetching provides). However, I'm not sure how it would work in practice. When we follow a walk to a commit in a pack, do we really want to try to pull _just_ that commit?

For one thing, we would need the server to support partial fetches (and it is my assumption that we don't bother with that at all now). I don't know how widespread that is these days (and of course we would still need to fall back to fetching the full pack). But even if we _could_, would we get killed by http protocol overhead for each object? Certainly it would be no worse than fetching a totally unpacked repo, but I kind of assume such a fetch would be painful.

Or given that the packs should be organized by type, are you proposing to fetch just the "commit part" as a single entity, then "tree part", then the "blob part"? I'm a little hesitant to rely too much on what is basically a performance heuristic for the pack organization (and god forbid packv4 ever gets finished ;) ).

-Peff
Previous: Shawn O. PearceNext: Shawn O. Pearce
Message 7 of 23 in “Prevent cloning over http from spewing”
  1. Prevent cloning over http from spewingsparse@infidigm.net, Jun 2, 2009
  2. Erik Faye-LundJun 3, 2009
  3. Jakub NarebskiJun 3, 2009
  4. Junio C HamanoJun 3, 2009
  5. Jeff KingJun 3, 2009
  6. Shawn O. PearceJun 3, 2009
  7. Jeff KingJun 3, 2009
  8. Shawn O. PearceJun 3, 2009
  9. Jeff KingJun 3, 2009
  10. Shawn O. PearceJun 3, 2009
  11. Tay Ray ChuanJun 4, 2009
  12. Jeff KingJun 4, 2009
  13. Tay Ray ChuanJun 7, 2009
  14. Tay Ray ChuanJun 7, 2009
  15. Jeff KingJun 8, 2009
  16. Tay Ray ChuanJun 10, 2009
  17. Tay Ray ChuanJun 10, 2009
  18. Jeff KingJun 11, 2009
  19. Tay Ray ChuanJun 22, 2009
  20. Tay Ray ChuanJul 20, 2009
  21. Jeff KingJun 8, 2009
  22. Tay Ray ChuanJun 7, 2009
  23. Jakub NarebskiJun 5, 2009

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.