git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Git performance results on a large repository

From
Matt Graham <mdg149@gmail.com>
Date
Feb 3, 2012, 23:05 UTC
Message-ID
<CALts4TT49VAWPZ6XO9qahDTu=2E425QcRvXx5-75Jv8n4yp8RA@mail.gmail.com>
In-Reply-To
<CB5179E9.3B751%joshua.redstone@fb.com>
Hi Josh,
On Fri, Feb 3, 2012 at 17:00, Joshua Redstone <joshua.redstone@fb.com> wrote:
Show 16 quoted lines
> Thanks for the comments.  I've included a bunch more info on the test repo
> below.  It is based on a growth model of two of our current repositories
> (I.e., it's not a perforce import). We already have some of the easily
> separable projects in separate repositories, like HPHP.   If we could
> split our largest repos into multiple ones, that would help the scaling
> issue.  However, the code in those repos is rather interdependent and we
> believe it'd hurt more than help to split it up, at least for the
> medium-term future.  We derive a fair amount of benefit from the code
> sharing and keeping things together in a single repo, so it's not clear
> when it'd make sense to get more aggressive splitting things up.
>
> Some more information on the test repository:   The working directory is
> 9.5 GB, the median file size is 2 KB.  The average depth of a directory
> (counting the number of '/'s) is 3.6 levels and the average depth of a
> file is 4.6.  More detailed histograms of the repository composition is
> below:

Do you have a histogram of the types of files in the repo? And as suggested earlier, is svn working for you now because it allows sparse checkout? I imagine the stats for svn on the full repo would be comparable or worse to what you measured with git?

Previous: Nguyen Thai Ngoc DuyNext: Evgeny Sazhin
Message 7 of 34 in “Git performance results on a large repository”
  1. Joshua RedstoneFeb 3, 2012
  2. Ævar Arnfjörð BjarmasonFeb 3, 2012
  3. Joshua RedstoneFeb 3, 2012
  4. Sam VilainFeb 3, 2012
  5. Sam VilainFeb 3, 2012
  6. Nguyen Thai Ngoc DuyFeb 7, 2012
  7. Matt GrahamFeb 3, 2012
  8. Evgeny SazhinFeb 4, 2012
  9. Chris LeeFeb 3, 2012
  10. Zeki MokhtarzadaFeb 4, 2012
  11. Joey HessFeb 4, 2012
  12. Nguyen Thai Ngoc DuyFeb 4, 2012
  13. Joshua RedstoneFeb 4, 2012
  14. Nguyen Thai Ngoc DuyFeb 5, 2012
  15. Joey HessFeb 6, 2012
  16. Nguyen Thai Ngoc DuyFeb 7, 2012
  17. Joshua RedstoneFeb 9, 2012
  18. Nguyen Thai Ngoc DuyFeb 10, 2012
  19. Christian CouderFeb 10, 2012
  20. Nguyen Thai Ngoc DuyFeb 10, 2012
  21. David MohsFeb 6, 2012
  22. Matt GrahamFeb 6, 2012
  23. Joshua RedstoneFeb 6, 2012
  24. Greg TroxelFeb 6, 2012
  25. david@lang.hmFeb 7, 2012
  26. Sam VilainFeb 6, 2012
  27. Joshua RedstoneFeb 4, 2012
  28. Tomas CarneckyFeb 5, 2012
  29. Nguyen Thai Ngoc DuyFeb 5, 2012
  30. slinkyFeb 4, 2012
  31. Greg TroxelFeb 4, 2012
  32. david@lang.hmFeb 5, 2012
  33. David BarrFeb 5, 2012
  34. Emanuele ZattinFeb 7, 2012

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.