git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 1/4] gitfaq: add advice on monorepos

From
brian m. carlson <sandals@crustytoothpaste.net>
Date
Oct 20, 2021, 21:19 UTC
Message-ID
<YXCHxdYot+fCHwl1@camp.crustytoothpaste.net>
In-Reply-To
<211020.86h7dcndlm.gmgdl@evledraar.gmail.com>
On 2021-10-20 at 10:54:47, Ævar Arnfjörð Bjarmason wrote:
Show 36 quoted lines
> 
> On Wed, Oct 20 2021, brian m. carlson wrote:
> 
> > +[[monorepos]]
> > +Should we use a monorepo or many individual repos?::
> > +	This is a decision that is typically made based on an organization's needs and
> > +	desires for their projects.  Git has several features, such as shallow clone,
> > +	partial clone, and sparse checkout to make working with large repositories
> > +	easier, and there is active development on making the monorepo experience
> > +	better.
> > ++
> > +However, at a certain size, the performance of a monorepo will likely become
> > +unacceptable _unless_ you use these features.  If you choose to start with a
> > +monorepo and continue to grow, you may end up unhappy with the performance
> > +characteristics at a point where making a change is difficult.  The performance
> > +of using many smaller repositories will almost always be much better and will
> > +generally not necessitate the use of these more advanced features.  If you are
> > +concerned about future performance of your repository and related tools, you may
> > +wish to avoid a monorepo.
> > ++
> > +Ultimately, you should make a decision fully informed about the potential
> > +benefits and downsides, including the capabilities, performance, and future
> > +requirements for your repository and related tools, including your hosting
> > +platform, build tools, and other programs you typically use as part of your
> > +workflow.
> 
> In the context of git development we're typically talking about really
> big repos when we're talking about monorepos, saying "monorepo"
> communicates among other things that the user of that pattern is
> unwilling to use splitting up as a way to address any scalability issues
> they may have.
> 
> But a monorepo doesn't really say anything about size per-se, and it
> would be confusing to conflate the two in a FAQ. I may be wrong, perhaps
> the term has really come to exclusively refer to colossal size, but I
> haven't seen or heard it exclusively (or even mainly) used like that

I routinely hear "monorepo" used to imply repositories of specifically large size. However, I'm happy to rephrase to make it clearer.

Show 7 quoted lines
> I bet that the vast majority of monorepo users are never going to
> experience scaling problems, e.g. having your laptop dotfiles and
> automation of /etc in one repo is a "monorepo", and most companies/teams
> that use monorepos I'd bet are in the long tail of size
> distribution. They're not going to grow to the size of a MS's, FB's
> etc. monorepo, but they might benefit (or not) from the monorepo
> /workflow/.

I almost never hear individuals refer to such a configuration as a monorepo. Technically, it is one, yes, but I almost always hear it in the context of an organization's repository covering all of their services or the entirety of one major project.

I will point out that I personally would run into scaling issues if I put all of my projects in the same repository. I have many projects, and that would quickly become unsustainable, since the resources I have at my disposal are more limited than most organizations.

Show 5 quoted lines
> Anyway, all of the above can be read as a suggestion that we should
> split any discussion of "large repo [that runs into scaling issues]"
> from "monorepo", the latter should of course make a passing reference to
> scaling (as the pattern will lead to that sooner than not), but IMO not
> conflate the two.

I'm happy to clarify, but I think we need to mention the word "monorepo" specifically because (a) that's the term that's commonly used for this approach and (b) that approach is one that tends to lead to significantly greater growth in a single repository leading to scale problems.

-- 
brian m. carlson (he/him or they/them)
Toronto, Ontario, CA
Previous: Ævar Arnfjörð BjarmasonNext: Johannes Schindelin
Message 21 of 30 in “Additional FAQ entries”
  1. 0/4 Additional FAQ entriesbrian m. carlson, Oct 20, 2021
  2. 3/4 gitfaq: give advice on using eol attribute in gitattributesbrian m. carlson, Oct 20, 2021
  3. Eric SunshineOct 20, 2021
  4. brian m. carlsonOct 20, 2021
  5. Johannes SchindelinOct 20, 2021
  6. brian m. carlsonOct 20, 2021
  7. Johannes SchindelinOct 21, 2021
  8. 4/4 doc: add a FAQ entry about syncing working treesbrian m. carlson, Oct 20, 2021
  9. Bagas SanjayaOct 20, 2021
  10. Philip OakleyOct 20, 2021
  11. Ævar Arnfjörð BjarmasonOct 20, 2021
  12. brian m. carlsonOct 21, 2021
  13. Ævar Arnfjörð BjarmasonOct 21, 2021
  14. 4/4 gitfaq: add entry about syncing working treesbrian m. carlson, Oct 20, 2021
  15. Eric SunshineOct 20, 2021
  16. brian m. carlsonOct 20, 2021
  17. Johannes SchindelinOct 20, 2021
  18. 1/4 gitfaq: add advice on monoreposbrian m. carlson, Oct 20, 2021
  19. Bagas SanjayaOct 20, 2021
  20. Ævar Arnfjörð BjarmasonOct 20, 2021
  21. brian m. carlsonOct 20, 2021
  22. Johannes SchindelinOct 20, 2021
  23. Philip OakleyOct 20, 2021
  24. brian m. carlsonOct 20, 2021
  25. Philip OakleyOct 25, 2021
  26. 2/4 gitfaq: add documentation on proxiesbrian m. carlson, Oct 20, 2021
  27. Johannes SchindelinOct 20, 2021
  28. brian m. carlsonOct 20, 2021
  29. Junio C HamanoOct 20, 2021
  30. brian m. carlsonOct 20, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.