Multiple threads of compression

Hi,

I’m asking here informally first, because my information relates to a quite old version (the one from lenny-backports). A tl;dr is at the end.

On a multi-core machine, the garbage collection of git, as well as pack compression on the server side when someone clones a repository remotely, the compression is normally done automatically using multiple threads of execution.

That may be fine for your typical setups, but in my cases, I have two scenarios where it isn’t:

ⓐ The machine where I want it to use only, say, 2 of my 4 or 8 cores
  as I’m also running some VMs on the box which eat up a lot of CPU
  and which I don’t want to slow down.

ⓑ The server VM which has been given 2 or 3 VCPUs to cope with all
  the load done by clients, but which is RAM-constrained to only
  512 or, when lucky, 768 MiB. It previously served only http/https
  and *yuk* Subversion, but now, git comes into the play, and I’ve
  seen the one server box I think about go down *HARD* because git
  ate up all RAM *and* swap when someone wanted to update their clone
  of a repository after someone else committed… well, an ~100 MiB large
  binary file they shouldn’t. (It required manual intervention on the
  server to kill that revision and then the objects coupled with it,
  but even *that* didn’t work, read on for more.)

In both cases, I had to apply a quick hack. One I can reproduce by now is, that, on the first box, I added a --threads=2 to the line calling git pack-objects in /usr/lib/git-core/git-repack, like this:

   83 args="$args $local ${GIT_QUIET:+-q} $no_reuse$extra"
   84 names=$(git pack-objects --threads=2 --keep-true-parents --honor-pack-
keep --non-empty --all --reflog $arg
   85         exit 1

(By the way, wrapping source code at 80c is still way to go IMHO.)

On the second box, IIRC I added --threads=1, but that box got subsequently upgraded from lenny to wheezy so any local modification is lost (luckily, the problem didn’t occur again recently, or at least I didn’t notice it, save for the VM load going up to 6-8 several times a day).

tl;dr: I would like to have a *global* option for git to restrict the number of threads of execution it uses. Several subcommands, like pack-objects, are already equipped with an optioin for this, but unfortunately, these are seldom invoked by hand¹, so this can’t work in my situations.

① automatic garbage collection, “git gc --aggressive --prune=now”,
  and cloning are the use cases I have at hand right now.

À propos, while here: is gc --aggressive safe to run on a live, online-shared repository, or does it break other users accessing the repository concurrently? (If it’s safe I’d very much like to do that in a, say weekly, cronjob on FusionForge, our hosting system.)

Thanks in advance! //mirabilos

On Sun, Nov 25, 2012 at 8:27 AM, Thorsten Glaser <tg@debian.org> wrote:

See the git-config man page.

The number of threads that pack uses can be configured in the global or system gitconfig file by setting pack.threads.

e.g.

   $ git config --system pack.threads 1

Also, modern git accepts a '-c' option which allows you to set configuration options on the command line, e.g. 'git -c pack.threads=1 gc'.

The other setting you should probably look at is pack.windowMemory which should help you control the amount of memory git uses while packing. Also look at core.packedGitWindowSize and core.packedGitLimit if your repository is really large.

Running 'git gc' with --aggressive should be as safe as running it without --aggressive.

But, you should think about whether you really need to run it more than once, or at all. When you use --aggressive, git will perform the entire delta search again for _every_ object in the repository. The general usecase for --aggressive is when you suspect that the original delta search produced sub-optimal deltas or if you modify the size of the delta search window or depth and want to regenerate your packed objects to improve compression or access speed. Even then, you will not likely get much benefit from running with --aggressive a second time.

hth, -Brandon

Brandon Casey dixit:

[…]

OK, thanks a lot!

I can’t really say much about the repositories beforehand because it’s a generic code hosting platform, several instances of which we run at my employer’s place (I also run one privately now), and which is also run by e.g. Debian. But I’ll try to figure out some somewhat sensible defaults.

OK, thanks.

[…]

Great explanation!

I think that I’d want to run it once, after the repository has been begun to be used (probably not correct English but you know what I want to say), but have to figure out a way to do so… but I’ll just leave out the --aggressive from the cronjob then.

Much appreciated, //mirabilos

Hi,

[Thorsten Glaser <tg@debian.org>, 2012-11-25 17:27]:

unfortunately I can't really speak to the git side of things, but both of these cases just sound like standard resource starvation. So why don't you address them using the usual OS mechanisms?

If you want to prevent git from sucking up CPU, nice(1) it, and if it eats too much RAM, use the parent shell's ulimit mechanism.

Granted, this might also require some changes to git, but wouldn't that be a simpler and more general approach to solving starvation problems?