# Filters should be parallelized if needed 2 messages from 2016-06-21 to 2016-06-21. Participants: LUCAS Thomas (EXT), Lars Schneider. Thread: https://gitlist.dev/t/42677 ## LUCAS Thomas (EXT), 2016-06-21 07:50 Subject: Filters should be parallelized if needed Message-ID: URL: https://gitlist.dev/e/ABCC694798EF214E91DB2ACCA61703690BBADA33%40MMXDC2642.hermes.si.socgen ``` Setup: - 2.9.0.windows.1 x64 on a windows 7 16g ram, ssd - No particular installation settings, default. Details: - Running git from either a cmd or GitExtensions, I encountered an issue while staging or checkout of lfs files. I am processing 3000 files for a total of 500MB. Files are proccessed sequentially which is too slow for this amount of files, while lfs offer in some cases a way to run it from the command line in parallel, I think the git filters should be run in parallel. I think there is no need (or I am forgetting something) to run it sequentially. If I were to use a filter processing files in a parallel way, I would have my time to checkout these files be reduced from 1h30min to 1min30s. What actually happened instead? - This is a no go for our enterprise to use git-lfs because this is too slow. I can't give you my repository because it is confidential. Anyway, I hope you can do something about it. ************************************************************************* This message and any attachments (the "message") are confidential, intended solely for the addresse(s), and may contain legally privileged information. Any unauthorized use or dissemination is prohibited. E-mails are susceptible to alteration. Neither SOCIETE GENERALE nor any of its subsidiaries or affiliates shall be liable for the message if altered, changed or falsified. Please visit http://swapdisclosure.sgcib.com for important information with respect to derivative products. ************ Ce message et toutes les pieces jointes (ci-apres le "message") sont confidentiels et susceptibles de contenir des informations couvertes par le secret professionnel. Ce message est etabli a l'intention exclusive de ses destinataires. Toute utilisation ou diffusion non autorisee est interdite. Tout message electronique est susceptible d'alteration. La SOCIETE GENERALE et ses filiales declinent toute responsabilite au titre de ce message s'il a ete altere, deforme ou falsifie. Veuillez consulter le site http://swapdisclosure.sgcib.com afin de recueillir d'importantes informations sur les produits derives. ************************************************************************* ``` ## Lars Schneider, 2016-06-21 09:36 Subject: Re: Filters should be parallelized if needed Message-ID: URL: https://gitlist.dev/e/A61012AD-BC22-4640-AE99-000D99D7375F%40gmail.com In-Reply-To: ``` > On 21 Jun 2016, at 09:50, LUCAS Thomas (EXT) wrote: > > Setup: > - 2.9.0.windows.1 x64 on a windows 7 16g ram, ssd > - No particular installation settings, default. > > Details: > - Running git from either a cmd or GitExtensions, I encountered an issue while staging or checkout of lfs files. I am processing 3000 files for a total of 500MB. > Files are proccessed sequentially which is too slow for this amount of files, while lfs offer in some cases a way to run it from the command line in parallel, I think the git filters should be run in parallel. I think there is no need (or I am forgetting something) to run it sequentially. > > If I were to use a filter processing files in a parallel way, I would have my time to checkout these files be reduced from 1h30min to 1min30s. > > What actually happened instead? > - This is a no go for our enterprise to use git-lfs because this is too slow. > > I can't give you my repository because it is confidential. > > Anyway, I hope you can do something about it. Hi Lucas, this is a known issue and there are a few ideas how to improve the situation: https://github.com/github/git-lfs/issues/1059 https://github.com/github/git-lfs/issues/931#issuecomment-172939381 I am not sure if anyone is working on the batch/parallel smudge filter processing already (I might have some time for that later this year). AFAIK this is not a trivial change. That being said, I think the Git mailing list is the wrong place to discuss GitLFS issues as these two projects are separate. I recommend to raise an issue here: https://github.com/github/git-lfs/issues Cheers, Lars ```