git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Git Scaling: What factors most affect Git performance for a large repo?

From
Duy Nguyen <pclouds@gmail.com>
Date
Feb 19, 2015, 23:38 UTC
Message-ID
<CACsJy8Dortn4fHwF8xSgJ=KoJ9o1qzmc_UyaVq003D2sxFZEuQ@mail.gmail.com>
In-Reply-To
<CAH8BJxEr_v+aitpZduXPC4oiRhMuySpc7Wb1aGcYe_p3mWjn+w@mail.gmail.com>

On Fri, Feb 20, 2015 at 4:26 AM, Stephen Morton <stephen.c.morton@gmail.com> wrote:

Show 5 quoted lines
> By 'performance', I guess I mean speed of day to day operations for devs.
>
>    * (Obviously, trivially, a (non-local) clone will be slow with a large repo.)
>    * Will a few simultaneous clones from the central server also slow down
>      other concurrent operations for other users?

There are no locks in server when cloning, so in theory cloning does not affect other operations. Cloning can use lots of memory though (and a lot of cpu unless you turn on reachability bitmap feature, which you should).

>    * Will 'git pull' be slow?

If we exclude the server side, the size of your tree is the main factor, but your 25k files should be fine (linux has 48k files).

>    * 'git push'?

This one is not affected by how deep your repo's history is, or how wide your tree is, so should be quick..

Ah the number of refs may affect both git-push and git-pull. I think Stefan knows better than I in this area.

>    * 'git commit'? (It is listed as slow in reference [3].)
>    * 'git stautus'? (Slow again in reference 3 though I don't see it.)
(also git-add)

Again, the size of your tree. I'm trying to address problems in [3], but at your repo's size, I don't think you need to worry about it.

Show 5 quoted lines
>    * Some operations might not seem to be day-to-day but if they are called
>      frequently by the web front-end to GitLab/Stash/GitHub etc then
>      they can become bottlenecks. (e.g. 'git branch --contains' seems terribly
>      adversely affected by large numbers of branches.)
>    * Others?
git-blame could be slow when a file is modified a lot.
-- 
Duy
Previous: Duy NguyenNext: David Turner
Message 17 of 28 in “Git Scaling: What factors most affect Git performance for a large repo?”
  1. Stephen MortonFeb 19, 2015
  2. Stefan BellerFeb 19, 2015
  3. Stephen MortonFeb 19, 2015
  4. Stefan BellerFeb 19, 2015
  5. Ævar Arnfjörð BjarmasonFeb 19, 2015
  6. Duy NguyenFeb 20, 2015
  7. Ævar Arnfjörð BjarmasonFeb 20, 2015
  8. Ævar Arnfjörð BjarmasonFeb 20, 2015
  9. Ævar Arnfjörð BjarmasonFeb 20, 2015
  10. Junio C HamanoFeb 20, 2015
  11. Ævar Arnfjörð BjarmasonMar 2, 2015
  12. Junio C HamanoMar 2, 2015
  13. Sebastian SchuberthFeb 20, 2015
  14. Michael HaggertyFeb 24, 2015
  15. Ævar Arnfjörð BjarmasonMar 2, 2015
  16. Duy NguyenFeb 21, 2015
  17. Duy NguyenFeb 19, 2015
  18. David TurnerFeb 20, 2015
  19. Junio C HamanoFeb 20, 2015
  20. David TurnerFeb 23, 2015
  21. Duy NguyenFeb 21, 2015
  22. Duy NguyenFeb 25, 2015
  23. brian m. carlsonFeb 20, 2015
  24. Stephen MortonFeb 20, 2015
  25. Matthieu MoyFeb 20, 2015
  26. brian m. carlsonFeb 20, 2015
  27. Sebastian SchuberthFeb 20, 2015
  28. brian m. carlsonFeb 20, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.