git/list[1] front-page[2] threads[3] people[4] search[5] about
 

git-fetch per-repository speed issues

From
Keith Packard <keithp@keithp.com>
Date
Jul 3, 2006, 18:02 UTC
Message-ID
<1151949764.4723.51.camel@neko.keithp.com>

Ok, so maybe X.org is using git in an unexpected (or even wrong) fashion. Our environment has split development across dozens of separate repositories which match ABI interfaces. With CVS, we were able to keep this all in one giant CVS repository with separate modules, but git doesn't have that notion (which is mostly good). As such, we could use cvsup or rsync to update the entire collection of modules.

With git, we'd prefer to use the git protocol instead of rsync for the usual pack-related reasons, but that is limited to a single repository at a time. And, it's painfully slow, even when the repository is up to date:

$ cd lib/libXrandr $ time git-fetch origin ...

real 0m17.035s user 0m2.584s sys 0m0.576s

This is a repository with 24 files and perhaps 50 revisions. Given X.org's 307 git repositories, I'll clearly need to find a faster way than running git-fetch on every one.

One thing I noticed was that the git+ssh syntax found in remotes files doesn't do what I thought it did -- I assumed this would use 'git' for fetch and 'ssh' for push, when in fact it just uses ssh for everything. This slows down the connection process by several seconds.

-- 
keith.packard@intel.com
Next: Linus Torvalds
Message 1 of 30 in “git-fetch per-repository speed issues”
  1. Keith PackardJul 3, 2006
  2. Linus TorvaldsJul 3, 2006
  3. Jeff KingJul 4, 2006
  4. Ryan AndersonJul 4, 2006
  5. Jeff KingJul 4, 2006
  6. Ryan AndersonJul 4, 2006
  7. Linus TorvaldsJul 4, 2006
  8. Jeff KingJul 5, 2006
  9. Linus TorvaldsJul 5, 2006
  10. Jakub NarebskiJul 4, 2006
  11. Jakub NarebskiJul 4, 2006
  12. Thomas GlanzmannJul 4, 2006
  13. Junio C HamanoJul 4, 2006
  14. Linus TorvaldsJul 4, 2006
  15. Junio C HamanoJul 4, 2006
  16. David WoodhouseJul 6, 2006
  17. Linus TorvaldsJul 4, 2006
  18. Junio C HamanoJul 4, 2006
  19. Linus TorvaldsJul 4, 2006
  20. Keith PackardJul 4, 2006
  21. Andreas EricssonJul 4, 2006
  22. Matthias KestenholzJul 4, 2006
  23. Andreas EricssonJul 4, 2006
  24. Keith PackardJul 4, 2006
  25. Linus TorvaldsJul 4, 2006
  26. Keith PackardJul 4, 2006
  27. Linus TorvaldsJul 4, 2006
  28. Junio C HamanoJul 4, 2006
  29. Keith PackardJul 4, 2006
  30. Linus TorvaldsJul 4, 2006

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.