Re: Filters should be parallelized if needed
- From
Lars Schneider <larsxschneider@gmail.com>
- Date
- Jun 21, 2016, 09:36 UTC
- Message-ID
- <A61012AD-BC22-4640-AE99-000D99D7375F@gmail.com>
- In-Reply-To
- <ABCC694798EF214E91DB2ACCA61703690BBADA33@MMXDC2642.hermes.si.socgen>
Show 18 quoted lines
> On 21 Jun 2016, at 09:50, LUCAS Thomas (EXT) <thomas.lucas-ext@sgcib.com> wrote: > > Setup: > - 2.9.0.windows.1 x64 on a windows 7 16g ram, ssd > - No particular installation settings, default. > > Details: > - Running git from either a cmd or GitExtensions, I encountered an issue while staging or checkout of lfs files. I am processing 3000 files for a total of 500MB. > Files are proccessed sequentially which is too slow for this amount of files, while lfs offer in some cases a way to run it from the command line in parallel, I think the git filters should be run in parallel. I think there is no need (or I am forgetting something) to run it sequentially. > > If I were to use a filter processing files in a parallel way, I would have my time to checkout these files be reduced from 1h30min to 1min30s. > > What actually happened instead? > - This is a no go for our enterprise to use git-lfs because this is too slow. > > I can't give you my repository because it is confidential. > > Anyway, I hope you can do something about it.
Hi Lucas,
this is a known issue and there are a few ideas how to improve the situation: https://github.com/github/git-lfs/issues/1059 https://github.com/github/git-lfs/issues/931#issuecomment-172939381
I am not sure if anyone is working on the batch/parallel smudge filter processing already (I might have some time for that later this year). AFAIK this is not a trivial change.
That being said, I think the Git mailing list is the wrong place to discuss GitLFS issues as these two projects are separate. I recommend to raise an issue here: https://github.com/github/git-lfs/issues
Cheers, Lars