git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Git Scaling: What factors most affect Git performance for a large repo?

From
Stefan Beller <sbeller@google.com>
Date
Feb 19, 2015, 22:21 UTC
Message-ID
<CAGZ79kaPhuCXgYvZ49NwVRvrKLVJLTWd0oWHv3jX31sOVCWKVQ@mail.gmail.com>
In-Reply-To
<CAH8BJxEr_v+aitpZduXPC4oiRhMuySpc7Wb1aGcYe_p3mWjn+w@mail.gmail.com>

On Thu, Feb 19, 2015 at 1:26 PM, Stephen Morton <stephen.c.morton@gmail.com> wrote:

Show 9 quoted lines
> I posted this to comp.version-control.git.user and didn't get any response. I
> think the question is plumbing-related enough that I can ask it here.
>
> I'm evaluating the feasibility of moving my team from SVN to git. We have a very
> large repo. [1]
>
> [1] (Yes, I'm investigating ways to make our repo not so large etc. That's
>     beyond the scope of the discussion I'd like to have with this
>     question. Thanks.)
What do you mean by large?
* lots of files
* large files
* or even large binary files (bad to diff/merge)
* long history (i.e. lots of small changes)
* impactful history (changes which rewrite nearly everything from scratch)
For reference, the linux
* has 48414 files, in 3128 directories
* the largest file is 1.1M, the whole repo is 600M
* no really large binary files
* more than 500051 changes/commits including merges
* started in 2004 (when git was invented essentially)
* the .git folder is 1.4G compared to the 600M files,
   indicating it may have been rewritting 3 times (well this
   metric is bogus, there is lots of compression
   going on in .git)
and linux seems to be doing ok with git.

So as long as you cannot pinpoint your question on what you are exactly concerned about, there will be no helpful answer I guess.

linux is by no means a really large project, there are other projects way larger than that (I am thinking about the KDE project for example) and they do fine as well.

Thanks, Stefan

Previous: Stephen MortonNext: Stephen Morton
Message 2 of 28 in “Git Scaling: What factors most affect Git performance for a large repo?”
  1. Stephen MortonFeb 19, 2015
  2. Stefan BellerFeb 19, 2015
  3. Stephen MortonFeb 19, 2015
  4. Stefan BellerFeb 19, 2015
  5. Ævar Arnfjörð BjarmasonFeb 19, 2015
  6. Duy NguyenFeb 20, 2015
  7. Ævar Arnfjörð BjarmasonFeb 20, 2015
  8. Ævar Arnfjörð BjarmasonFeb 20, 2015
  9. Ævar Arnfjörð BjarmasonFeb 20, 2015
  10. Junio C HamanoFeb 20, 2015
  11. Ævar Arnfjörð BjarmasonMar 2, 2015
  12. Junio C HamanoMar 2, 2015
  13. Sebastian SchuberthFeb 20, 2015
  14. Michael HaggertyFeb 24, 2015
  15. Ævar Arnfjörð BjarmasonMar 2, 2015
  16. Duy NguyenFeb 21, 2015
  17. Duy NguyenFeb 19, 2015
  18. David TurnerFeb 20, 2015
  19. Junio C HamanoFeb 20, 2015
  20. David TurnerFeb 23, 2015
  21. Duy NguyenFeb 21, 2015
  22. Duy NguyenFeb 25, 2015
  23. brian m. carlsonFeb 20, 2015
  24. Stephen MortonFeb 20, 2015
  25. Matthieu MoyFeb 20, 2015
  26. brian m. carlsonFeb 20, 2015
  27. Sebastian SchuberthFeb 20, 2015
  28. brian m. carlsonFeb 20, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.