git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Question about git gc on large text files

From
KZKai Zhang <kai@netskope.com>
Date
Aug 17, 2017, 17:40 UTC
Message-ID
<9D8F1E07-747B-49CF-B644-B8BDB63E94CE@netskope.com>
In-Reply-To
<20170817173314.qjzii4e4rybqqn34@sigill.intra.peff.net>
Show 21 quoted lines
> On Aug 17, 2017, at 10:33 AM, Jeff King <peff@peff.net> wrote:
> 
> On Thu, Aug 17, 2017 at 10:28:00AM -0700, Kai Zhang wrote:
> 
>> I have a git repository maintaining one large json file (along with
>> several other small files). With commits for large json file, the
>> repository become bigger and bigger, so I tried to run command "git gc
>> --prune=now --aggressive" to reduce disk usage, then I found .git
>> folder size did not change. I had wonderful experience with git gc
>> against files around 10M to 20M, and I am wondering if there is any
>> configuration need to tweak for large text files?
>> 
>> Here I have more details:
>> 1. Json file size: 1G
> 
> Git won't try to delta-compress anything over 500MB by default. Try:
> 
>  git config core.bigfilethreshold 2G
>  git gc --aggressive
> 
> -Peff
It works! Thank you so much!
Kai
Previous: Jeff King
Message 3 of 3 in “Question about git gc on large text files”
  1. Kai ZhangAug 17, 2017
  2. Jeff KingAug 17, 2017
  3. Kai ZhangAug 17, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.