Re: Git is not scalable with too many refs/*

From: Thomas Rast <trast@student.ethz.ch>
Date: Sep 9, 2011, 01:05 UTC
Message-ID: <201109090305.15896.trast@student.ethz.ch>
In-Reply-To: <1315529522448-6774328.post@n2.nabble.com>

Martin Fick wrote:

Show 12 quoted lines

> An update, I bisected it down to this commit:
> 
>   88a21979c5717e3f37b9691e90b6dbf2b94c751a
> 
>    fetch/pull: recurse into submodules when necessary
> 
> Since this can be disabled with the --no-recurse-submodules switch, I tried
> that and indeed, even with the latest 1.7.7rc it becomes fast (~8mins)
> again. The strange part about this is that the repository does not have any
> submodules. Anyway, I hope that this can be useful to others since it is a
> workaround which speed things up enormously. Let me know if you have any
> other tests that you want me to perform,

Jens should know about this, so let's Cc him.

I took a quick look and I'm guessing that there's at least one quadratic behaviour: in check_for_new_submodule_commits(), I see

+       const char *argv[] = {NULL, NULL, "--not", "--all", NULL};
+       int argc = ARRAY_SIZE(argv) - 1;
+
+       init_revisions(&rev, NULL);

which means that the --all needs to walk all commits reachable from all refs and flag them as uninteresting. But that function is called for every ref update, so IIUC the time spent is on the order of #ref updates*#commits.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch