git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: cloning the linxu kernel repo at a VPS with small RAM

From
DSDragan Simic <dsimic@manjaro.org>
Date
Feb 8, 2024, 20:35 UTC
Message-ID
<b532a9fd7b363b1dd8da2e8d15d995cc@manjaro.org>
In-Reply-To
<6c4e309aa77e534cffdba9ae56032b99@manjaro.org>
On 2024-02-08 21:16, Dragan Simic wrote:
Show 45 quoted lines
> On 2024-02-08 18:32, Toralf Förster wrote:
>> Situation:
>> 
>> The command
>>     git clone
>> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
>> fails at a virtual server with about 2 GiB RAM under a recent Debian
>> bookworm with git 2.39.2. What works for me:
>>       git clone --depth=1
>> https://git.kernel.org/pub/scm/linux/kernel/git/stable
>>       cd ./linux
>>       git config gc.auto 0
>>       git config pack.threads 1
>>       git fetch --tags
>> as seen in [1].
>> 
>> Q:
>> I do wonder if Git could automatically try to deal with only 1.5 GiB
>> available RAM?
> 
> Here's an excerpt from my ~/.bashrc, which sets the things up
> on my Pinebook Pro laptop with an RK3399 SoC and 4 GB of RAM,
> which is also thermally constrained:
> 
> # Missing nproc(1) is handled properly
> REASONABLE_THREADS=$(nproc 2> /dev/null || echo 1)
> REASONABLE_THREADS=$((${REASONABLE_THREADS} / 2))
> ((${REASONABLE_THREADS} == 0)) && REASONABLE_THREADS=1
> 
> export GIT_CONFIG_COUNT=3
> export GIT_CONFIG_KEY_0='grep.threads'
> export GIT_CONFIG_VALUE_0=${REASONABLE_THREADS}
> export GIT_CONFIG_KEY_1='index.threads'
> export GIT_CONFIG_VALUE_1=${REASONABLE_THREADS}
> export GIT_CONFIG_KEY_2='pack.threads'
> export GIT_CONFIG_VALUE_2=${REASONABLE_THREADS}
> 
> export ZSTD_NBTHREADS=${REASONABLE_THREADS}
> unset REASONABLE_THREADS
> 
> Obviously, this does a bit more than configuring git only.
> 
> Perhaps modifying this to additionally take the amount of RAM
> into the calculation of REASONABLE_THREADS could be a solution
> for your use case?

Actually, perhaps we could provide the following files as part of the git project, which would take the above-described approach, and let the distributions package them:

  - /etc/profile.d/git.sh
  - /etc/profile.d/git.csh

If everyone agrees, I'd we willing to take a crack on these. Of course, all suggestions and thoughts are more than welcome.

Previous: Dragan SimicNext: Eric Wong
Message 3 of 5 in “cloning the linxu kernel repo at a VPS with small RAM”
  1. Toralf FörsterFeb 8, 2024
  2. Dragan SimicFeb 8, 2024
  3. Dragan SimicFeb 8, 2024
  4. Eric WongFeb 9, 2024
  5. Toralf FörsterFeb 10, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.