{"thread":{"id":"53641","subject":"git stash --include-untracked walks ignored directories","startedAt":"2020-06-09T19:33:42Z","lastAt":"2020-06-10T05:57:29Z","messageCount":3,"participants":["Brian Malehorn","Elijah Newren"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"399362","messageId":"CAJB88a39G-GkGu1LuRE8na45Pav_cWb3cJLQFJM+FuW3BnromA@mail.gmail.com","threadId":"53641","inReplyTo":null,"subject":"git stash --include-untracked walks ignored directories","fromName":"Brian Malehorn","fromEmail":"bmalehorn@gmail.com","sentAt":"2020-06-09T19:33:00Z","receivedAt":"2020-06-09T19:33:42Z","isPatch":false,"sender":{"key":"bmalehorn@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3344958?v=4"},"body":"Hi,\n\nNot sure if this is the right place to send this, but I'm here to\nreport a performance regression with git stash --include-untracked.\n\nHere's a quick way to reproduce:\n\n1. make a directory with a lot of ignored files\n\n$ find ignored -type f | wc -l\n   50000\n\n$ cat .gitignore\nignored\n\n2. touch foo\n\n3. time git stash --include-untracked\n\ngit version 2.26.0:\nreal    0m0.094s\n\ngit version 2.27.0.83.g0313f36c6e:\nreal    0m1.913s\n\nThis is a much bigger pain point on my work repo, which has 1.4\nmillion ignored files(!). As you can imagine it takes a long time to\nrun git stash. While it might be valid to question why anyone would\nneed that many files for any purpose, the bottom line is that I told\ngit to ignore this directory, and it didn't ignore it.\n\nIn the meantime I've reverted to 2.26.0 which doesn't have this\nperformance regression. Let me know if you want any other information\nrelated to this issue.\n\nThanks,\nBrian\n"},{"id":"399383","messageId":"CABPp-BHv+5XSXMpZ=-kM=a3C0Y+v=JY5m11s-QWj_krjCvvO4g@mail.gmail.com","threadId":"53641","inReplyTo":"CAJB88a39G-GkGu1LuRE8na45Pav_cWb3cJLQFJM+FuW3BnromA@mail.gmail.com","subject":"Re: git stash --include-untracked walks ignored directories","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2020-06-10T03:21:17Z","receivedAt":"2020-06-10T03:21:40Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Tue, Jun 9, 2020 at 1:39 PM Brian Malehorn <bmalehorn@gmail.com> wrote:\n>\n> Hi,\n>\n> Not sure if this is the right place to send this, but I'm here to\n> report a performance regression with git stash --include-untracked.\n>\n> Here's a quick way to reproduce:\n>\n> 1. make a directory with a lot of ignored files\n>\n> $ find ignored -type f | wc -l\n>    50000\n>\n> $ cat .gitignore\n> ignored\n>\n> 2. touch foo\n>\n> 3. time git stash --include-untracked\n>\n> git version 2.26.0:\n> real    0m0.094s\n>\n> git version 2.27.0.83.g0313f36c6e:\n> real    0m1.913s\n>\n> This is a much bigger pain point on my work repo, which has 1.4\n> million ignored files(!). As you can imagine it takes a long time to\n> run git stash. While it might be valid to question why anyone would\n> need that many files for any purpose, the bottom line is that I told\n> git to ignore this directory, and it didn't ignore it.\n>\n> In the meantime I've reverted to 2.26.0 which doesn't have this\n> performance regression. Let me know if you want any other information\n> related to this issue.\n>\n> Thanks,\n> Brian\n\nI seem to be missing some important step to reproduce; what else is\nneeded?  Here's what I see:\n\n<Set path to use git-2.26.0>\n$ ./repro.sh\nNumber of files in ignored before: 50000\nSaved working directory and index state WIP on master: e2b0471 initial\n\nreal 0m0.029s\nuser 0m0.014s\nsys 0m0.014s\ngit version 2.26.0\nNumber of files in ignored after: 50000\n\n<Set path to use git-2.27.0>\n$ ./repro.sh\nNumber of files in ignored before: 50000\nSaved working directory and index state WIP on master: 5c596b8 initial\n\nreal 0m0.052s\nuser 0m0.014s\nsys 0m0.034s\ngit version 2.27.0\nNumber of files in ignored after: 50000\n\n\nWhere repro.sh is:\n\n#!/bin/bash\n\nrm -rf stupid\ngit init -q stupid\ncd stupid\n\necho ignored >.gitignore\nseq 1 10 >numbers-tracked\ngit add numbers-tracked .gitignore\ngit commit -q -m initial\n\nseq 11 20 >>numbers-tracked\nseq 21 30 >numbers-untracked\n\nmkdir ignored\ncd ignored\nfor i in $(seq 1 50000); do >$i; done\ncd ..\n\necho \"Number of files in ignored before: $(find ignored -type f | wc -l)\"\ntime git stash --include-untracked\ngit --version\necho \"Number of files in ignored after: $(find ignored -type f | wc -l)\"\n"},{"id":"399388","messageId":"CAJB88a23uU2WfB0mnB9NfNbtgmABhNOWNOEMBt7rRVu7uL_C9A@mail.gmail.com","threadId":"53641","inReplyTo":"CABPp-BHv+5XSXMpZ=-kM=a3C0Y+v=JY5m11s-QWj_krjCvvO4g@mail.gmail.com","subject":"Re: git stash --include-untracked walks ignored directories","fromName":"Brian Malehorn","fromEmail":"bmalehorn@gmail.com","sentAt":"2020-06-10T05:56:48Z","receivedAt":"2020-06-10T05:57:29Z","isPatch":false,"sender":{"key":"bmalehorn@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3344958?v=4"},"body":"Ah, my original message was a bit misleading. I believe git slows down\nbased on the number of directories, not the number of files. Here's an\nupdated version of your script that creates a lot of directories\ninstead of a lot of files:\n\n#!/bin/bash\n\nrm -rf stupid\ngit init -q stupid\ncd stupid\n\necho ignored >.gitignore\nseq 1 10 >numbers-tracked\ngit add numbers-tracked .gitignore\ngit commit -q -m initial\n\nseq 11 20 >>numbers-tracked\nseq 21 30 >numbers-untracked\n\nmkdir ignored\ncd ignored\nfor i in $(seq 1 50); do\n  for j in $(seq 1 1000); do\n    echo \"$i/$j\"\n  done\ndone | xargs mkdir -p\ncd ..\n\necho \"Number of directories in ignored before: $(find ignored -type d | wc -l)\"\ntime git stash --include-untracked\ngit --version\necho \"Number of directories in ignored after: $(find ignored -type d | wc -l)\"\n\n----\n\nI got it to reproduce with the versions I have handy:\n\n$ ./repro.sh\nNumber of directories in ignored before: 50051\nSaved working directory and index state WIP on master: 0dabddf initial\n\nreal    0m0.023s\nuser    0m0.012s\nsys     0m0.010s\ngit version 2.25.1\nNumber of directories in ignored after: 50051\n\n\n$./repro.sh\nNumber of directories in ignored before: 50051\nSaved working directory and index state WIP on master: 175d4ce initial\n\nreal    0m0.619s\nuser    0m0.157s\nsys     0m0.452s\ngit version 2.27.0.83.g0313f36c6e\nNumber of directories in ignored after: 50051\n\nOn Tue, Jun 9, 2020 at 8:21 PM Elijah Newren <newren@gmail.com> wrote:\n>\n> On Tue, Jun 9, 2020 at 1:39 PM Brian Malehorn <bmalehorn@gmail.com> wrote:\n> >\n> > Hi,\n> >\n> > Not sure if this is the right place to send this, but I'm here to\n> > report a performance regression with git stash --include-untracked.\n> >\n> > Here's a quick way to reproduce:\n> >\n> > 1. make a directory with a lot of ignored files\n> >\n> > $ find ignored -type f | wc -l\n> >    50000\n> >\n> > $ cat .gitignore\n> > ignored\n> >\n> > 2. touch foo\n> >\n> > 3. time git stash --include-untracked\n> >\n> > git version 2.26.0:\n> > real    0m0.094s\n> >\n> > git version 2.27.0.83.g0313f36c6e:\n> > real    0m1.913s\n> >\n> > This is a much bigger pain point on my work repo, which has 1.4\n> > million ignored files(!). As you can imagine it takes a long time to\n> > run git stash. While it might be valid to question why anyone would\n> > need that many files for any purpose, the bottom line is that I told\n> > git to ignore this directory, and it didn't ignore it.\n> >\n> > In the meantime I've reverted to 2.26.0 which doesn't have this\n> > performance regression. Let me know if you want any other information\n> > related to this issue.\n> >\n> > Thanks,\n> > Brian\n>\n> I seem to be missing some important step to reproduce; what else is\n> needed?  Here's what I see:\n>\n> <Set path to use git-2.26.0>\n> $ ./repro.sh\n> Number of files in ignored before: 50000\n> Saved working directory and index state WIP on master: e2b0471 initial\n>\n> real 0m0.029s\n> user 0m0.014s\n> sys 0m0.014s\n> git version 2.26.0\n> Number of files in ignored after: 50000\n>\n> <Set path to use git-2.27.0>\n> $ ./repro.sh\n> Number of files in ignored before: 50000\n> Saved working directory and index state WIP on master: 5c596b8 initial\n>\n> real 0m0.052s\n> user 0m0.014s\n> sys 0m0.034s\n> git version 2.27.0\n> Number of files in ignored after: 50000\n>\n>\n> Where repro.sh is:\n>\n> #!/bin/bash\n>\n> rm -rf stupid\n> git init -q stupid\n> cd stupid\n>\n> echo ignored >.gitignore\n> seq 1 10 >numbers-tracked\n> git add numbers-tracked .gitignore\n> git commit -q -m initial\n>\n> seq 11 20 >>numbers-tracked\n> seq 21 30 >numbers-untracked\n>\n> mkdir ignored\n> cd ignored\n> for i in $(seq 1 50000); do >$i; done\n> cd ..\n>\n> echo \"Number of files in ignored before: $(find ignored -type f | wc -l)\"\n> time git stash --include-untracked\n> git --version\n> echo \"Number of files in ignored after: $(find ignored -type f | wc -l)\"\n"}]}