Re: [PATCH/RFC] gitperformance: add new documentation about git performance tuning

From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Apr 3, 2017, 21:57 UTC
Message-ID: <CACBZZX7R5svNJ+Ak3LFh8+kY48i6V7Yo6JDS+PSDJCkZ5vHb6w@mail.gmail.com>
In-Reply-To: <20170403213440.GA1409@whir>

On Mon, Apr 3, 2017 at 11:34 PM, Eric Wong <e@80x24.org> wrote:

Show 30 quoted lines

> Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>> Add a new manpage that gives an overview of how to tweak git's
>> performance.
>>
>> There's currently no good single resource for things a git site
>> administrator might want to look into to improve performance for his
>> site & his users. This unfinished documentation aims to be the first
>> thing someone might want to look at when investigating ways to improve
>> git performance.
>>
>> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>> ---
>>
>> I've been wanting to get something like this started for a while. It's
>> obviously woefully incomplete. Pointers about what to include would be
>> great & whether including something like this makes sense.
>
> Thanks for doing this.  I hope something like this can give
> server operators more confidence to host their own git servers.
>
>> Things I have on my TODO list:
>
> <snip>
>
>>  - Should we be covering good practices for your repo going forward to
>>    maintain good performance? E.g. don't have some huge tree all in
>>    one directory (use subdirs), don't add binary (rather
>>    un-delta-able) content if you can help it etc.
>
> Yes, I think so.

I'll try to write something up.

> I think avoiding ever growing ChangeLog-type files should also
> be added to things to avoid.

How were those bad specifically? They should delta quite well, it's expensive to commit large files but no more because they're ever-growing.

One issue with e.g. storing logs (I keep my IRC logs in git) is that if you're constantly committing large (text) files without repack your .git grows by a *lot* in a very short amount of time until a very expensive repack, so now I split my IRC logs by month.

But I'm probably forgetting some obvious case where the ChangeLog use-case is bad.

Show 6 quoted lines

>> --- /dev/null
>> +++ b/Documentation/gitperformance.txt
>> @@ -0,0 +1,107 @@
>> +giteveryday(7)
>
> gitperformance(7)

Oops, thanks.

Show 8 quoted lines

>> +Server options to help clients
>> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> +
>> +These features can be enabled on git servers, they won't help the
>> +performance of the servers themselves,
>
> Is that true for bitmaps?  I thought they reduced CPU usage on
> the server side...

I'm not sure, JK? From my reading of the repack.writeBitmaps docs it seems to only help clone/fetch for the client, but maybe they do more than that.

I also see we should mention pack.writeBitmapHashCache, which according to my reading of v2.0.0-rc0~13^2~8 only helps clone/fetch.

Show 9 quoted lines

> A sidenote: I wonder if bitmaps should be the default for bare
> repos, since bare repos are likely used on servers.
>
>> but will help clients that need
>> +to talk to those servers.
>> +
>> +- config: "repack.writeBitmaps=true" (see
>> +  linkgit:git-config[1]). Spend more time during repack to produce
>> +  bitmap index, helps clients with "fetch" & "clone" performance.