threads / discuss / 24558

git and binary files

Subject: git and binary files

## tl;dr

3 messages between Jul 28, 2010 and Jul 28, 2010.

replies: 2people: 3as markdown or json

Jan Dittmer· Jul 28, 2010, 14:17 UTC · lore

Hi,

I'm using git to keep track of directories full of jpegs and movies (19 GB .git directory, 5825 files, 18+ GB).

I now that this is also possible with rsync and friends, but I like to have full version control and history, so please no discussion about that.

The problem I'm facing is, that 'git status' is very very slow. They seem to read through every file in the directory tree. Is it possible to tell git to just use mtime/size information for an initial guess? I already tried to turn of rename detection with diff.renames=false, but still the problem remains.

Any hint? git version is 1.7.1.

Thanks,

Jan

-- 
Jan Dittmer <jdi@l4x.org>

Matthieu Moy· Jul 28, 2010, 14:29 UTC · re: Jan Dittmer · lore

Re: git and binary files

Jan Dittmer <jdi@l4x.org> writes:

> The problem I'm facing is, that 'git status' is very
> very slow. They seem to read through every file in the directory
> tree. Is it possible to tell git to just use mtime/size information for
> an initial guess?

This is what it does by default. You probably need an initial "git status" to be slow to get this information right in the index, though.

Have a look at git-bigfiles too.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Avery Pennarun· Jul 28, 2010, 20:17 UTC · re: Matthieu Moy · lore

Re: git and binary files

On Wed, Jul 28, 2010 at 10:29 AM, Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote:

Show 8 quoted lines

> Jan Dittmer <jdi@l4x.org> writes:
>> The problem I'm facing is, that 'git status' is very
>> very slow. They seem to read through every file in the directory
>> tree. Is it possible to tell git to just use mtime/size information for
>> an initial guess?
>
> This is what it does by default. You probably need an initial "git
> status" to be slow to get this information right in the index, though.

Judging from the description, it sounds like 'git status' is the problem, not the solution. Although at 18 GB, it was probably pretty slow to run, so maybe Jan only ran it once and the second time it would have been fast :)

Depending on what the requirements are (ie. just keeping multiple versions for posterity/safety as opposed to actually checking out old versions on a regular basis), bup might be more useful than git in this case: http://github.com/apenwarr/bup .

Have fun,

Avery