Re: [PATCH 1/4] gitfaq: add advice on monorepos

From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Oct 20, 2021, 10:54 UTC
Message-ID: <211020.86h7dcndlm.gmgdl@evledraar.gmail.com>
In-Reply-To: <20211020010624.675562-2-sandals@crustytoothpaste.net>

On Wed, Oct 20 2021, brian m. carlson wrote:

Show 22 quoted lines

> +[[monorepos]]
> +Should we use a monorepo or many individual repos?::
> +	This is a decision that is typically made based on an organization's needs and
> +	desires for their projects.  Git has several features, such as shallow clone,
> +	partial clone, and sparse checkout to make working with large repositories
> +	easier, and there is active development on making the monorepo experience
> +	better.
> ++
> +However, at a certain size, the performance of a monorepo will likely become
> +unacceptable _unless_ you use these features.  If you choose to start with a
> +monorepo and continue to grow, you may end up unhappy with the performance
> +characteristics at a point where making a change is difficult.  The performance
> +of using many smaller repositories will almost always be much better and will
> +generally not necessitate the use of these more advanced features.  If you are
> +concerned about future performance of your repository and related tools, you may
> +wish to avoid a monorepo.
> ++
> +Ultimately, you should make a decision fully informed about the potential
> +benefits and downsides, including the capabilities, performance, and future
> +requirements for your repository and related tools, including your hosting
> +platform, build tools, and other programs you typically use as part of your
> +workflow.

In the context of git development we're typically talking about really big repos when we're talking about monorepos, saying "monorepo" communicates among other things that the user of that pattern is unwilling to use splitting up as a way to address any scalability issues they may have.

But a monorepo doesn't really say anything about size per-se, and it would be confusing to conflate the two in a FAQ. I may be wrong, perhaps the term has really come to exclusively refer to colossal size, but I haven't seen or heard it exclusively (or even mainly) used like that

My understanding of what a monorepo in the context of software development is a collection of all your "main" code and all its dependencies in one repository, such that a person working on it rarely or never has to worry about N=N dependencies between different pieces of that collection, they'll all move in unison. You are able to atomically change a function and all its users.

I think people have a different understanding of "all its dependencies". Some monorepo users really mean it and try to e.g. fold their system configuration system that might manage files in /etc in with their monorepo, others might have "another monorepo" for that software, etc.

I bet that the vast majority of monorepo users are never going to experience scaling problems, e.g. having your laptop dotfiles and automation of /etc in one repo is a "monorepo", and most companies/teams that use monorepos I'd bet are in the long tail of size distribution. They're not going to grow to the size of a MS's, FB's etc. monorepo, but they might benefit (or not) from the monorepo /workflow/.

Anyway, all of the above can be read as a suggestion that we should split any discussion of "large repo [that runs into scaling issues]" from "monorepo", the latter should of course make a passing reference to scaling (as the pattern will lead to that sooner than not), but IMO not conflate the two.