"git commit" of single file takes 5 minutes, mounted fileystem/diskimage with 50G GIT repo + 900G of builds articles

Based on a previous thread “First Git status takes 40+ minutes, when mounting fileystem/diskimage with 50G GIT repo + 900G of builds articles”

The context is that 
  1. git wokspace was clone(50G)
  2. some 30 platorms build(900G)
  3. This tree was then copied into a new diskimage/filesystem + git update-index --refresh" was done to update the index to the new filesystem, then frozen.
  4. New workspaces created by cloning this frozen diskimage(< 30 seconds)
  5. This diskimage was mounted on a new machine
  6. A file was modified and "git add/commit" was done

I have done “git update-index –refresh”, in the mounted filesystem, as above This has improved “git status/diff” timing from 40+ minutes to 1.5 minutes for the first time and <5 seconds for subsequent calls.

But "git commit -m "dummy commit" of a 1 line change in 1 file takes about 5-6 minutes, everytime in this workspace. Tracing shows a whole bunch. The entire 5-6 minutes worth of the following sort of trace logs. 3:13:50.320930 trace git-lfs: filepathfilter: rejecting "x/y/z.o.command" via [] 13:13:50.320940 trace git-lfs: filepathfilter: accepting " x/y/z.o.command " 13:13:50.320862 trace git-lfs: filepathfilter: rejecting "a/b/c/d.o.command" via [] 13:13:50.320972 trace git-lfs: filepathfilter: accepting " a/b/c/d..o.command"

Does anyone have any insights on what could be causing this?

On the other hand, if I had 
 
Thanks,
Sarvi
Occam’s Razor Rules

Any suggestion or advice how to optimize this in GIT ?

 
Thanks,
Sarvi
Occam’s Razor Rules

On 8/26/19, 1:43 PM, "Saravanan Shanmugham (sarvi)" <sarvi@cisco.com> wrote:

    
    Based on a previous thread “First Git status takes 40+ minutes, when mounting fileystem/diskimage with 50G GIT repo + 900G of builds articles”
    
    The context is that 
      1. git wokspace was clone(50G)
      2. some 30 platorms build(900G)
      3. This tree was then copied into a new diskimage/filesystem + git update-index --refresh" was done to update the index to the new filesystem, then frozen.
      4. New workspaces created by cloning this frozen diskimage(< 30 seconds)
      5. This diskimage was mounted on a new machine
      6. A file was modified and "git add/commit" was done
    
    I have done “git update-index –refresh”, in the mounted filesystem, as above
    This has improved “git status/diff” timing from 40+ minutes to 1.5 minutes for the first time and <5 seconds for subsequent calls.
    
    But "git commit -m "dummy commit" of a 1 line change in 1 file takes about 5-6 minutes, everytime in this workspace.
    Tracing shows a whole bunch. The entire 5-6 minutes worth of the following sort of trace logs.
    3:13:50.320930 trace git-lfs: filepathfilter: rejecting "x/y/z.o.command" via []
    13:13:50.320940 trace git-lfs: filepathfilter: accepting " x/y/z.o.command "
    13:13:50.320862 trace git-lfs: filepathfilter: rejecting "a/b/c/d.o.command" via []
    13:13:50.320972 trace git-lfs: filepathfilter: accepting " a/b/c/d..o.command"
    
    Does anyone have any insights on what could be causing this?
         
    Thanks,
    Sarvi
    Occam’s Razor Rules
    
    

Hi Saravanan,

On Mon, Aug 26, 2019 at 08:43:34PM +0000, Saravanan Shanmugham (sarvi) wrote:

These are messages from the Git LFS tool, and this list does not provide support for Git LFS, since it is a separate project not affiliated with Git.

I would recommend that you create an issue on Git LFS's tracker at:

  https://github.com/git-lfs/git-lfs/issues/new

and mention that you are seeing lots of 'filepathfilter' logs. My speculation is that this could be a slowdown in the 'filepathfilter' code paths, or an entire-tree scan where 'filepathfilter' is invoked every so often. I'd mention both of those as possibilities (or send a link to this email on public-inbox), and see if anything comes up.

Thanks, Taylor