[BUG] Incorrect/misleading error when `git rev-list --objects --all` hits the max open files limit

Peter Rabbitson· Mar 3, 2013, 23:29 UTC · lore

Hi!

I was tinkering with a massive git repository (actually a bup repository, but it is a standard valid git repo underneath). While validating that a repack ran succesfully I executed the command:

 git rev-list --objects --all > rev.list

And got back this:

error: packfile ./objects/pack/pack-d9808b7515419737806d0c621a0a1910f71c5cba.pack cannot be accessed fatal: missing blob object '27a8cf44da85b958aef2b5074931e7913e08ae95'

Several hours later after successful fsck, and after chasing down trees blobs etc, I realized that the problem is too many open files. The hint came from ls-tree which lists the correct error (among a lot of spurious junk):

 git ls-tree -r c636a5f51d4e > /dev/null

 error: packfile ./objects/pack/pack-d9808b7515419737806d0c621a0a1910f71c5cba.pack cannot be accessed
 error: packfile ./objects/pack/pack-841e375f5e6c793a52fd1a3a2aea0b76219c4cdd.pack cannot be accessed
 error: packfile ./objects/pack/pack-e67d9bf75e0840fc6113170b314d2d5a32cbb45a.pack cannot be accessed
 error: packfile ./objects/pack/pack-b8fd8f083461c391fe6ec396840c328620d912e2.pack cannot be accessed
 error: packfile ./objects/pack/pack-d9808b7515419737806d0c621a0a1910f71c5cba.pack cannot be accessed
 error: packfile ./objects/pack/pack-804e0fadf56e2a165c157ef257620369adeea595.pack cannot be accessed
 error: unable to open object pack directory: ./objects/pack: Too many open files
 error: packfile ./objects/pack/pack-804e0fadf56e2a165c157ef257620369adeea595.pack cannot be accessed
 error: Could not read 32a050cb7e54a1e817d135d25ab251480e8d9e3c

Failure to report the correct message verified with git 1.7.2.5 and 1.8.2 (debian testing and experimental).

I hope this is sufficient description to address the underlying issue. I will keep the un-repacked "many files" repo around just in case you need more info/testing (though lowering the ulimit works equally well on normal-size repos).

Cheers

Pete Wyckoff· Mar 5, 2013, 15:26 UTC · re: Peter Rabbitson · lore

Re: [BUG] Incorrect/misleading error when `git rev-list --objects --all` hits the max open files limit

rabbit@rabbit.us wrote on Mon, 04 Mar 2013 10:29 +1100:

Show 35 quoted lines

> I was tinkering with a massive git repository (actually a bup 
> repository, but it is a standard valid git repo underneath). While 
> validating that a repack ran succesfully I executed the command:
> 
>  git rev-list --objects --all > rev.list
> 
> And got back this:
> 
> error: packfile ./objects/pack/pack-d9808b7515419737806d0c621a0a1910f71c5cba.pack cannot be accessed
> fatal: missing blob object '27a8cf44da85b958aef2b5074931e7913e08ae95'
> 
> Several hours later after successful fsck, and after chasing down trees 
> blobs etc, I realized that the problem is too many open files. The hint 
> came from ls-tree which lists the correct error (among a lot of spurious 
> junk):
> 
>  git ls-tree -r c636a5f51d4e > /dev/null
> 
>  error: packfile ./objects/pack/pack-d9808b7515419737806d0c621a0a1910f71c5cba.pack cannot be accessed
>  error: packfile ./objects/pack/pack-841e375f5e6c793a52fd1a3a2aea0b76219c4cdd.pack cannot be accessed
>  error: packfile ./objects/pack/pack-e67d9bf75e0840fc6113170b314d2d5a32cbb45a.pack cannot be accessed
>  error: packfile ./objects/pack/pack-b8fd8f083461c391fe6ec396840c328620d912e2.pack cannot be accessed
>  error: packfile ./objects/pack/pack-d9808b7515419737806d0c621a0a1910f71c5cba.pack cannot be accessed
>  error: packfile ./objects/pack/pack-804e0fadf56e2a165c157ef257620369adeea595.pack cannot be accessed
>  error: unable to open object pack directory: ./objects/pack: Too many open files
>  error: packfile ./objects/pack/pack-804e0fadf56e2a165c157ef257620369adeea595.pack cannot be accessed
>  error: Could not read 32a050cb7e54a1e817d135d25ab251480e8d9e3c
> 
> Failure to report the correct message verified with git 1.7.2.5 and 
> 1.8.2 (debian testing and experimental).
> 
> I hope this is sufficient description to address the underlying issue. I 
> will keep the un-repacked "many files" repo around just in case you need 
> more info/testing (though lowering the ulimit works equally well on 
> normal-size repos).

I've run into this twice:

    1.  user with restrictive ulimit on #files.
    2.  forgot to do "git gc" after regular fast-imports

Here's one thread:

    http://thread.gmane.org/gmane.comp.version-control.git/179231/focus=179292

I've got a patch in cold storage. It's not beautiful because there are too many paths that can end up hitting EMFILE. I'll dust it off and see if it is worth considering.

		-- Pete