threads / discuss / 44484

Git status takes too long- How to improve the performance of git

Subject: Git status takes too long- How to improve the performance of git

## tl;dr

6 messages between Nov 15, 2016 and Nov 24, 2016.

replies: 5people: 5as markdown or json

ravalika· Nov 15, 2016, 09:33 UTC · lore
Hi All,

We are using git-1.8.2 version for version control. It is an centralized server and git status takes too long

How to improve the performance of git status
Git repo details:

Size of the .git folder is 8.9MB Number of commits approx 53838 (git rev-list HEAD --count) Number of branches - 330 Number of files - 63883 Working tree clone size is 4.3GB

time git status shows real 0m23.673s user 0m9.432s sys 0m3.793s

then after 5 mins real 0m4.864s user 0m1.417s sys 0m4.710s

And I have experimented the following ways 
-          -  Setting core.ignorestat to true
-          -  Git gc &git clean
-          -  Shallow clone – Reducing number of commits
-          -  Clone only one branch 
      - Git repacking - git repack -ad && git prune
-     - Cold/warm cache 

Could you please let me know, what are the ways to improve the git performance ? I have gone through the mailing lists.

Thank you, Renuka

-- View this message in context: http://git.661346.n2.nabble.com/Git-status-takes-too-long-How-to-improve-the-performance-of-git-tp7657456.html Sent from the git mailing list archive at Nabble.com.

Fredrik Gustafsson· Nov 15, 2016, 10:24 UTC · re: ravalika · lore

Re: Git status takes too long- How to improve the performance of git

Hi,
On Tue, Nov 15, 2016 at 02:33:12AM -0700, ravalika wrote:
> We are using git-1.8.2 version for version control.

That's a three (almost four) year old version of git. Your first test should be to see if an upgrade to a recent version will improve things.

> It is an centralized server and git status takes too long

A centralized server? How? git is designed to be runned locally. If you're running git on a network file system, the performance will suffer. Could you elaborate on how your environment is setup?

Show 10 quoted lines
> 
> How to improve the performance of git status
> 
> Git repo details:
> 
> Size of the .git folder is 8.9MB
> Number of commits approx 53838  (git rev-list HEAD --count)
> Number of branches -  330  
> Number of files - 63883
> Working tree clone size is 4.3GB

.git folder of 8.9 MEGABYTE and working tree of 4.3 GIGABYTE? Is this a typo?

Show 10 quoted lines
> 
> time git status shows
> real	0m23.673s
> user	0m9.432s
> sys	0m3.793s
> 
> then after 5 mins
> real    0m4.864s
> user    0m1.417s
> sys     0m4.710s

A slow disc and empty caches are slow. Two ways of improving this is to have faster discs or make sure your cache is up to date. When I'd a really slow disc, I'd my shell to run a git status in the background to load the cache everytime I started working on a project. This is however an ugly hack that wasn't approved to be a part of git.

Show 12 quoted lines
> 
> And I have experimented the following ways 
> -          -  Setting core.ignorestat to true
> -          -  Git gc &git clean
> -          -  Shallow clone – Reducing number of commits
> -          -  Clone only one branch 
>       - Git repacking - git repack -ad && git prune
> -     - Cold/warm cache 
> 
> Could you please let me know, what are the ways to improve the git
> performance ?
> I have gone through the mailing lists.

You could always check the --assume-unchanged bit, see the manual page for git update-index. However this is quite extreme and demanding for the user.

-- 
Fredrik Gustafsson

phone: +46 733-608274
e-mail: iveqy@iveqy.com
website: http://www.iveqy.com
Christian Couder· Nov 15, 2016, 11:44 UTC · re: Fredrik Gustafsson · lore

Re: Git status takes too long- How to improve the performance of git

On Tue, Nov 15, 2016 at 11:24 AM, Fredrik Gustafsson <iveqy@iveqy.com> wrote:
> On Tue, Nov 15, 2016 at 02:33:12AM -0700, ravalika wrote:
[...]
Show 15 quoted lines
>> And I have experimented the following ways
>> -          -  Setting core.ignorestat to true
>> -          -  Git gc &git clean
>> -          -  Shallow clone – Reducing number of commits
>> -          -  Clone only one branch
>>       - Git repacking - git repack -ad && git prune
>> -     - Cold/warm cache
>>
>> Could you please let me know, what are the ways to improve the git
>> performance ?
>> I have gone through the mailing lists.
>
> You could always check the --assume-unchanged bit, see the manual page
> for git update-index. However this is quite extreme and demanding for
> the user.

If you install a recent version version, you may be able to use the untracked cache feature. (See "core.untrackedCache" in the git config documentation and --untracked-cache in the git update-index documentation.)

Heiko Voigt· Nov 15, 2016, 15:10 UTC · re: ravalika · lore

Re: Git status takes too long- How to improve the performance of git

On Tue, Nov 15, 2016 at 02:33:12AM -0700, ravalika wrote:
> Number of files - 63883

Since you also posted this to the "Git for Windows" mailinglist I assume that you are using Windows. Reduce the number of files. For example split the repository into two one for documentation and one for source. Thats what I did with a converted repository that had to many files.

Windows is unfortunately very slow when it comes to handling many files and if I recall correctly ~30000 files was in a nicely handleable range for a Git repository on Windows, but that might have changed...

Cheers Heiko
MaryTurner· Nov 24, 2016, 08:38 UTC · re: ravalika · lore

Re: Git status takes too long- How to improve the performance of git

Nice

-- View this message in context: http://git.661346.n2.nabble.com/Git-status-takes-too-long-How-to-improve-the-performance-of-git-tp7657456p7657457.html Sent from the git mailing list archive at Nabble.com.

Fredrik Gustafsson· Nov 16, 2016, 13:21 UTC · lore

Re: Git status takes too long- How to improve the performance of git

On Wed, Nov 16, 2016 at 05:13:57PM +0530, Renuka Pampana wrote:
Show 14 quoted lines
> > On Tue, Nov 15, 2016 at 02:33:12AM -0700, ravalika wrote:
> > > It is an centralized server and git status takes too long
> >
> > A centralized server? How? git is designed to be runned locally. If
> > you're running git on a network file system, the performance will
> > suffer. Could you elaborate on how your environment is setup?
> >
> >
> We have setup main git repository in remote location on Linux server
> And created a git repository in local Linux server, as a reference for the
> remote git repository,
> And update the local git repository for every 15 min in local server
> 
> Users will be able to access the  local git repository through NFS

And each user will have their own copy of the repository locally on their machine? That is having done a git clone?

Show 8 quoted lines
> 
> All users will clone the git repository from remote project url  by using
> local git repo as reference
> 
>  For example : git clone --reference ~/gitcaches/reference user@drupal
> :/home/project/drupal.git
> 
> All the users have ssh credentials for the remote server
Why are you using --reference for a 8.9MB big clone?
Show 6 quoted lines
> 
> 
> What is the best way to implement remote git repo and able to access the
> git repo from other location, without any performance glitches?
> Users should be able to access git repo from different servers and from
> different locations.

The best way is to have it locally cloned. Yes the initial clone will be expensive but operations after that will be fairly smooth. You do not(!) want to execute git on one machine and having the repository beeing on an other machine (for example via a network file system, except git clone, git fetch, git push, etc.).

Show 18 quoted lines
> >
> > > How to improve the performance of git status
> > >
> > > Git repo details:
> > >
> > > Size of the .git folder is 8.9MB
> > > Number of commits approx 53838  (git rev-list HEAD --count)
> > > Number of branches -  330
> > > Number of files - 63883
> > > Working tree clone size is 4.3GB
> >
> > .git folder of 8.9 MEGABYTE and working tree of 4.3 GIGABYTE? Is this a
> > typo?
> >
> > All git related information is stored in .git directory of the working
> directory
>   It is 8.9M
> And size of the local workspace is 4.3G

Can you please elaborate on this? How can you store 8.9 MB of data that will result in a 4.3 G workspace?

-- 
Fredrik Gustafsson

phone: +46 733-608274
e-mail: iveqy@iveqy.com
website: http://www.iveqy.com

← back to recent threads