{"thread":{"id":"51747","subject":"\"git commit\" of single file takes 5 minutes, mounted fileystem/diskimage with 50G GIT repo + 900G of builds articles","startedAt":"2019-08-26T20:43:41Z","lastAt":"2019-08-27T18:44:37Z","messageCount":3,"participants":["Saravanan Shanmugham (sarvi)","Taylor Blau"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"381253","messageId":"7A1C0468-E07D-4D10-8891-01AC1FA1CC1B@cisco.com","threadId":"51747","inReplyTo":null,"subject":"\"git commit\" of single file takes 5 minutes, mounted fileystem/diskimage with 50G GIT repo + 900G of builds articles","fromName":"Saravanan Shanmugham (sarvi)","fromEmail":"sarvi@cisco.com","sentAt":"2019-08-26T20:43:34Z","receivedAt":"2019-08-26T20:43:41Z","isPatch":false,"sender":{"key":"sarvi@cisco.com","avatar":null},"body":"\nBased on a previous thread “First Git status takes 40+ minutes, when mounting fileystem/diskimage with 50G GIT repo + 900G of builds articles”\n\nThe context is that \n  1. git wokspace was clone(50G)\n  2. some 30 platorms build(900G)\n  3. This tree was then copied into a new diskimage/filesystem + git update-index --refresh\" was done to update the index to the new filesystem, then frozen.\n  4. New workspaces created by cloning this frozen diskimage(< 30 seconds)\n  5. This diskimage was mounted on a new machine\n  6. A file was modified and \"git add/commit\" was done\n\nI have done “git update-index –refresh”, in the mounted filesystem, as above\nThis has improved “git status/diff” timing from 40+ minutes to 1.5 minutes for the first time and <5 seconds for subsequent calls.\n\nBut \"git commit -m \"dummy commit\" of a 1 line change in 1 file takes about 5-6 minutes, everytime in this workspace.\nTracing shows a whole bunch. The entire 5-6 minutes worth of the following sort of trace logs.\n3:13:50.320930 trace git-lfs: filepathfilter: rejecting \"x/y/z.o.command\" via []\n13:13:50.320940 trace git-lfs: filepathfilter: accepting \" x/y/z.o.command \"\n13:13:50.320862 trace git-lfs: filepathfilter: rejecting \"a/b/c/d.o.command\" via []\n13:13:50.320972 trace git-lfs: filepathfilter: accepting \" a/b/c/d..o.command\"\n\nDoes anyone have any insights on what could be causing this?\n\nOn the other hand, if I had \n \nThanks,\nSarvi\nOccam’s Razor Rules\n\n"},{"id":"381374","messageId":"C09D54C7-6759-4205-B4D7-66C847B62450@cisco.com","threadId":"51747","inReplyTo":"7A1C0468-E07D-4D10-8891-01AC1FA1CC1B@cisco.com","subject":"Re: \"git commit\" of single file takes 5 minutes, mounted fileystem/diskimage with 50G GIT repo + 900G of builds articles","fromName":"Saravanan Shanmugham (sarvi)","fromEmail":"sarvi@cisco.com","sentAt":"2019-08-27T17:13:47Z","receivedAt":"2019-08-27T17:13:53Z","isPatch":false,"sender":{"key":"sarvi@cisco.com","avatar":null},"body":"Any suggestion or advice how to optimize this in GIT ?\n\n \nThanks,\nSarvi\nOccam’s Razor Rules\n\nOn 8/26/19, 1:43 PM, \"Saravanan Shanmugham (sarvi)\" <sarvi@cisco.com> wrote:\n\n    \n    Based on a previous thread “First Git status takes 40+ minutes, when mounting fileystem/diskimage with 50G GIT repo + 900G of builds articles”\n    \n    The context is that \n      1. git wokspace was clone(50G)\n      2. some 30 platorms build(900G)\n      3. This tree was then copied into a new diskimage/filesystem + git update-index --refresh\" was done to update the index to the new filesystem, then frozen.\n      4. New workspaces created by cloning this frozen diskimage(< 30 seconds)\n      5. This diskimage was mounted on a new machine\n      6. A file was modified and \"git add/commit\" was done\n    \n    I have done “git update-index –refresh”, in the mounted filesystem, as above\n    This has improved “git status/diff” timing from 40+ minutes to 1.5 minutes for the first time and <5 seconds for subsequent calls.\n    \n    But \"git commit -m \"dummy commit\" of a 1 line change in 1 file takes about 5-6 minutes, everytime in this workspace.\n    Tracing shows a whole bunch. The entire 5-6 minutes worth of the following sort of trace logs.\n    3:13:50.320930 trace git-lfs: filepathfilter: rejecting \"x/y/z.o.command\" via []\n    13:13:50.320940 trace git-lfs: filepathfilter: accepting \" x/y/z.o.command \"\n    13:13:50.320862 trace git-lfs: filepathfilter: rejecting \"a/b/c/d.o.command\" via []\n    13:13:50.320972 trace git-lfs: filepathfilter: accepting \" a/b/c/d..o.command\"\n    \n    Does anyone have any insights on what could be causing this?\n         \n    Thanks,\n    Sarvi\n    Occam’s Razor Rules\n    \n    \n\n"},{"id":"381380","messageId":"20190827184434.GA63526@syl.lan","threadId":"51747","inReplyTo":"7A1C0468-E07D-4D10-8891-01AC1FA1CC1B@cisco.com","subject":"Re: \"git commit\" of single file takes 5 minutes, mounted fileystem/diskimage with 50G GIT repo + 900G of builds articles","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2019-08-27T18:44:34Z","receivedAt":"2019-08-27T18:44:37Z","isPatch":false,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"Hi Saravanan,\n\nOn Mon, Aug 26, 2019 at 08:43:34PM +0000, Saravanan Shanmugham (sarvi) wrote:\n>\n> Based on a previous thread “First Git status takes 40+ minutes, when mounting fileystem/diskimage with 50G GIT repo + 900G of builds articles”\n>\n> The context is that\n>   1. git wokspace was clone(50G)\n>   2. some 30 platorms build(900G)\n>   3. This tree was then copied into a new diskimage/filesystem + git update-index --refresh\" was done to update the index to the new filesystem, then frozen.\n>   4. New workspaces created by cloning this frozen diskimage(< 30 seconds)\n>   5. This diskimage was mounted on a new machine\n>   6. A file was modified and \"git add/commit\" was done\n>\n> I have done “git update-index –refresh”, in the mounted filesystem, as above\n> This has improved “git status/diff” timing from 40+ minutes to 1.5 minutes for the first time and <5 seconds for subsequent calls.\n>\n> But \"git commit -m \"dummy commit\" of a 1 line change in 1 file takes about 5-6 minutes, everytime in this workspace.\n> Tracing shows a whole bunch. The entire 5-6 minutes worth of the following sort of trace logs.\n> 3:13:50.320930 trace git-lfs: filepathfilter: rejecting \"x/y/z.o.command\" via []\n> 13:13:50.320940 trace git-lfs: filepathfilter: accepting \" x/y/z.o.command \"\n> 13:13:50.320862 trace git-lfs: filepathfilter: rejecting \"a/b/c/d.o.command\" via []\n> 13:13:50.320972 trace git-lfs: filepathfilter: accepting \" a/b/c/d..o.command\"\n>\n> Does anyone have any insights on what could be causing this?\n\nThese are messages from the Git LFS tool, and this list does not provide\nsupport for Git LFS, since it is a separate project not affiliated with\nGit.\n\nI would recommend that you create an issue on Git LFS's tracker at:\n\n  https://github.com/git-lfs/git-lfs/issues/new\n\nand mention that you are seeing lots of 'filepathfilter' logs. My\nspeculation is that this could be a slowdown in the 'filepathfilter'\ncode paths, or an entire-tree scan where 'filepathfilter' is invoked\nevery so often. I'd mention both of those as possibilities (or send a\nlink to this email on public-inbox), and see if anything comes up.\n\n> On the other hand, if I had\n>\n> Thanks,\n> Sarvi\n> Occam’s Razor Rules\n\nThanks,\nTaylor\n"}]}