git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: out of memory error with git push and pull

From
QHQingning Huo <qingninghuo@gmail.com>
Date
Jun 2, 2011, 22:13 UTC
Message-ID
<BANLkTimSoqbg6LgunovaKRefX78OH=xLMg@mail.gmail.com>
In-Reply-To
<20110602044603.GB5081@gnu.kitenet.net>
Hi Joey,
On Thu, Jun 2, 2011 at 5:46 AM, Joey Hess <joey@kitenet.net> wrote:
Show 20 quoted lines
> Qingning Huo wrote:
>> I tried to use git to manage my digital photos but encountered some
>> problems. The typical file sizes are hundreds of KB or a few MB. In
>> total, about 15GB data in about 10,000 files. My intention is to get
>> them into a git repository and cloned into a few computers. Probably I
>> will make some occasionally changes like editing and deleting. But I
>> think most of the files would stay at version one.
>
> I try not to mention git-annex too much here, but this is a perfect
> use-case for it. http://git-annex.branchable.com/
>
> Well, it would be more perfect if you had enough data in your repo that
> you didn't necessarily want to clone it all to every computer. Like so:
>
> # git annex status
> local annex size: 58 megabytes
> total annex keys: 38158
> total annex size: 6 terabytes
>
> :)

Thanks a lot for the pointer. I'd love to use git-annex if I can get my hand to it. I had a look of the web site and searched a bit on the web, but there does not seem to be an easy way to install it on windows/cygwin.

I might try the bigFileThreshold setting first. And maybe git-bigfiles.
Show 15 quoted lines
>
>> I wonder whether anyone has tried using git in a similar scenario. Is
>> git capable of handling this kind of data? And, are there any settings
>> and/or command line options that I should use? I had a quick look of
>> git help push (and pull/fetch) but cannot see anything obvious.
>
> There is a tunable you can use to improve things, see core.bigFileThreshold
>
> That originally came from this project.
> http://caca.zoy.org/wiki/git-bigfiles -- it may have some other
> improvements that have not landed in git, I'm not sure.
>
> --
> see shy jo
>

Thanks Qingning

Previous: Joey Hess
Message 3 of 3 in “out of memory error with git push and pull”
  1. Qingning HuoJun 1, 2011
  2. Joey HessJun 2, 2011
  3. Qingning HuoJun 2, 2011

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.