cloning the linxu kernel repo at a VPS with small RAM

Situation:

The command
     git clone
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
fails at a virtual server with about 2 GiB RAM under a recent Debian
bookworm with git 2.39.2. What works for me:
       git clone --depth=1
https://git.kernel.org/pub/scm/linux/kernel/git/stable
       cd ./linux
       git config gc.auto 0
       git config pack.threads 1
       git fetch --tags
as seen in [1].

Q: I do wonder if Git could automatically try to deal with only 1.5 GiB available RAM?

TIA

[1] https://github.com/toralf/tor-relays/blob/main/playbooks/roles/setup/tasks/kernel-git.yaml#L22 -- Toralf

Hello Toralf,

On 2024-02-08 18:32, Toralf Förster wrote:

Here's an excerpt from my ~/.bashrc, which sets the things up on my Pinebook Pro laptop with an RK3399 SoC and 4 GB of RAM, which is also thermally constrained:

# Missing nproc(1) is handled properly REASONABLE_THREADS=$(nproc 2> /dev/null || echo 1) REASONABLE_THREADS=$((${REASONABLE_THREADS} / 2)) ((${REASONABLE_THREADS} == 0)) && REASONABLE_THREADS=1

export GIT_CONFIG_COUNT=3 export GIT_CONFIG_KEY_0='grep.threads' export GIT_CONFIG_VALUE_0=${REASONABLE_THREADS} export GIT_CONFIG_KEY_1='index.threads' export GIT_CONFIG_VALUE_1=${REASONABLE_THREADS} export GIT_CONFIG_KEY_2='pack.threads' export GIT_CONFIG_VALUE_2=${REASONABLE_THREADS}

export ZSTD_NBTHREADS=${REASONABLE_THREADS} unset REASONABLE_THREADS

Obviously, this does a bit more than configuring git only.

Perhaps modifying this to additionally take the amount of RAM into the calculation of REASONABLE_THREADS could be a solution for your use case?

On 2024-02-08 21:16, Dragan Simic wrote:

Actually, perhaps we could provide the following files as part of the git project, which would take the above-described approach, and let the distributions package them:

  - /etc/profile.d/git.sh
  - /etc/profile.d/git.csh

If everyone agrees, I'd we willing to take a crack on these. Of course, all suggestions and thoughts are more than welcome.

Toralf Förster <toralf.foerster@gmx.de> wrote:

I have to use pack.threads=1 myself on SMP machines, too.

_SC_AVPHYS_PAGES should probably be accounted for in addition to _SC_NPROCESSORS_CONF when calculating the default pack.threads value. Calculating good defaults is hard, though...

I've also been meaning to try the newer tombstone-free khashl to replace khash in our tree for memory savings:

https://attractivechaos.wordpress.com/2019/12/28/deletion-from-hash-tables-without-tombstones/

Perhaps you (or someone else) can take the lead, there.

On 2/8/24 18:32, Toralf Förster wrote:

Given that git fails for the linux kernel repo - where Git was used first in the wild (?) - it should definitely works for that repo even with a small RAM, right?

-- Toralf