Re: [PATCH 1/4] gitfaq: add advice on monorepos

From: Johannes Schindelin <johannes.schindelin@gmx.de>
Date: Oct 20, 2021, 11:55 UTC
Message-ID: <nycvar.QRO.7.76.6.2110201349570.56@tvgsbejvaqbjf.bet>
In-Reply-To: <20211020010624.675562-2-sandals@crustytoothpaste.net>

Hi brian,

thank you for this excellent idea to talk about monorepos.

On Wed, 20 Oct 2021, brian m. carlson wrote:

Show 16 quoted lines

> diff --git a/Documentation/gitfaq.txt b/Documentation/gitfaq.txt
> index 8c1f2d5675..946691c153 100644
> --- a/Documentation/gitfaq.txt
> +++ b/Documentation/gitfaq.txt
> @@ -241,6 +241,32 @@ How do I know if I want to do a fetch or a pull?::
>  	ignore the upstream changes.  A pull consists of a fetch followed
>  	immediately by either a merge or rebase.  See linkgit:git-pull[1].
>
> +Design
> +------
> +
> +[[monorepos]]
> +Should we use a monorepo or many individual repos?::
> +	This is a decision that is typically made based on an organization's needs and
> +	desires for their projects.  Git has several features, such as shallow clone,
> +	partial clone, and sparse checkout to make working with large repositories

May I request taking out shallow clones? A user new to Git might think that shallow clones are a sane way to clone a large repository. In practice, this only makes sense for "throw-away" clones, though. As soon as you fetch in such clones, performance will be so horrible that it is frequently a better idea to start with a partial clone instead.

At the same time, I would like to swap in "sparse index" for "shallow clone" because it _does_ have the best potential of all currently discussed new features to improve performance with monorepos.

Show 5 quoted lines

> +	easier, and there is active development on making the monorepo experience
> +	better.
> ++
> +However, at a certain size, the performance of a monorepo will likely become
> +unacceptable _unless_ you use these features.  If you choose to start with a

I would like to add a plug for Scalar here. Maybe we can link to this "opinionated tool based on Git" here? I wouldn't ask if I didn't _know_ that it helps monorepo users out there.

Show 12 quoted lines

> +monorepo and continue to grow, you may end up unhappy with the performance
> +characteristics at a point where making a change is difficult.  The performance
> +of using many smaller repositories will almost always be much better and will
> +generally not necessitate the use of these more advanced features.  If you are
> +concerned about future performance of your repository and related tools, you may
> +wish to avoid a monorepo.
> ++
> +Ultimately, you should make a decision fully informed about the potential
> +benefits and downsides, including the capabilities, performance, and future
> +requirements for your repository and related tools, including your hosting
> +platform, build tools, and other programs you typically use as part of your
> +workflow.

I wish we had a good article to link to, here. Yes, it is a decision that should be fully informed, and yes, this FAQ entry is not the place for a thorough discussion of monorepos and how Git can be asked to handle them more efficiently.

Do you know of any good resource that we could use here?

Thanks, Dscho

Show 5 quoted lines

> +
>  Merging and Rebasing
>  --------------------
>
>