[GSoC] My project blog

Hi, everyone

I've been posting about the progress of my GSoC project at https://matheustavares.gitlab.io/gsoc/ I should have sent the link earlier but, please, let me know if you have any comments on my previous posts.

You can read my latest update here: https://matheustavares.gitlab.io/posts/week-7-race-conditions-on-delta-base-cache I've been working to allow parallel decompression when reading objects for git-grep.

Thanks, Matheus

Hi, everyone

I just posted a new update about my GSoC project here: https://matheustavares.gitlab.io/posts/week-8-a-working-parallel-inflation Please, feel free to leave any comments. I've finally solved the race condition problem and now we have a working parallel inflation :) But there's still some tasks to be done on it before I have a proper patchset.

Thanks, Matheus

Hi Matheus,

On Sun, Jul 14, 2019 at 9:38 AM Matheus Tavares Bernardino <matheus.bernardino@usp.br> wrote:

Great that you solved the race condition!

I agree with the tasks you list in the next steps. I would even suggest that, right after splitting the changes into logical patches, you ask your mentors (Olga and me) to review it, and then send it as an RFC patch series to the mailing list. When you do that though, please list the other tasks and limitations of your patch series in the cover letter.

Thanks, Christian.

On Sun, Jul 14, 2019 at 8:00 AM Christian Couder <christian.couder@gmail.com> wrote:

OK, I will do that.

Thanks, Matheus

Hi, everyone

Unfortunately, not quite much to report this week :( I'm attending DebConf until July 28th but I'm trying to conciliate that with the work on my project. You can see my brief report of last week here: https://matheustavares.gitlab.io/posts/week-9-an-rfc-on-parallel-inflation

As always, every comment is highly appreciated.

Thanks, Matheus

Hi, everyone

I just posted an update about my project here: https://matheustavares.gitlab.io/posts/week-10-a-bug-in-git-grep-submodules This one is focused on a patch to correct a bug at git-grep --recurse-submodules and some tests for the parallel inflation patchset. Please, feel free to leave any comments.

Thanks, Matheus

Hi,

Here's my report from last week: https://matheustavares.gitlab.io/posts/week-11-wip-grep-protecting-textconv-and-submodules I'm working to protect the operations I left behind on the first version of the patchset[1]. And for that, I used a lot of the code Duy provided[2] me as an example in the early days of this project. The race conditions are now majorly gone, but the patches still need some refactoring and there are still some problems to overcome.

Thanks, Matheus

[1]: https://public-inbox.org/git/cover.1563570204.git.matheus.bernardino@usp.br/ [2]: https://gitlab.com/snippets/1834613#L3

Hi Matheus,

On Tue, Aug 6, 2019 at 4:54 AM Matheus Tavares Bernardino <matheus.bernardino@usp.br> wrote:

Thank you for another great report!

This looks good to me! I think you are now way past the point where you know much more about this than me, but anyway the following might give you some ideas or perhaps in some other ways help you:

  - About "Try to remove the obj_read_lock", which is the last point
on your TODO list, it doesn't look necessary before you submit a patch
series, unless there is no performance improvement without it. You
could, as far as I can tell, do it in a later patch series.

  - I guess " --recurse-submodules" is not used very often with git
grep. For example I never use it on the Git repo even when I perhaps
should. So if it would simplify things to just disable the
improvements you made when " --recurse-submodules" is used, then you
might consider doing that. You could later send another patch series
that would enable your improvements when " --recurse-submodules" is
used.

Thanks, Christian.

On Tue, Aug 6, 2019 at 4:16 AM Christian Couder <christian.couder@gmail.com> wrote:

Thanks for the suggestions, Christian. So, I think I could finish and send the patchset without the optimization for --recurse-submodules and --textconv, while I don't finish support for them, right? And I think 'obj_read_lock' may be removed in a future patch, as you said, because we already have good speedup without removing it. Finally, should I also add a "NOTES" section to Documentation/git-grep.txt saying which options --threads is not compatible with?

Thanks, Matheus

Hi, everyone

I just posted a new update on my project: https://matheustavares.gitlab.io/posts/simplified-version-of-parallel-inflation

This week I sent a simplified version of the series I was working on last week (which would still take some time to finish). And I'm working to improve it with support to --textconv and --recurse-submodules in the added optimizations. I've been also working on a patch to make git-grep stop adding submodules to the alternates list.

Thanks, Matheus

Hi Matheus,

On Tue, Aug 13, 2019 at 6:06 AM Matheus Tavares Bernardino <matheus.bernardino@usp.br> wrote:

Great blog post as usual!

I don't know what refactoring you think is needed on your patch:

https://github.com/matheustavares/git/commit/05536a503b3315d36ad13276af7728adf674ed51

I think it's worth sending to the list with or without addressing the small nits that I commented on.

Thanks, Christian.

Hi, everyone

I just posted the penultimate report on my project: https://matheustavares.gitlab.io/posts/going-for-a-too-big-step This week I’ve been working on a v2 of threaded git-grep w/ parallel inflation, to allow threads when grepping submodules. I also tried some more optimizations along the way.

As always, any comments/suggestions will be highly appreciated.

Thanks, Matheus

вт, 20 авг. 2019 г. в 07:59, Matheus Tavares Bernardino <matheus.bernardino@usp.br>:

Thank you for great blog post! You have done so many things, it is impressive.

I absolutely agree with your plans to structure the job and split it to smaller pieces, it should definitely help. Moreover, if you want, you can make something like a documentation and delegate some parts.

I hope you enjoyed this summer :) Thank you for your readiness to continue contributing after GSoC. It is not mandatory, you do not have to finish the project, but if you want - all the community will be so glad to see you as the contributor.

Thank you so much, Olga

Hi Matheus,

On Tue, Aug 20, 2019 at 1:28 PM Olga Telezhnaya <olyatelezhnaya@gmail.com> wrote:

Yeah, great blog post again!

I also agree with the above. I think it would be really nice for example if we could have something in Documentation/technical/ describing how multi-threading in `git grep` works and how it could be improved. (You can of course copy paste from your blog.)

About static function-scope variables I wonder if we could have one day some .h and .c files with only thread safe functions in them.

Thank you for your work and for considering continuing after the GSoC, Christian.

On Tue, Aug 20, 2019 at 9:16 AM Christian Couder <christian.couder@gmail.com> wrote:

Thank you both for the always encouraging comments :)

Hm, good idea! I would have to revisit the important points and better formulate them, but that could hopefully help in future optimizations to git-grep.

Yes, that would probably require quite some work but would definitely be very helpful!

Thank you both, again. It's been a great summer of code! (although it's winter here haha) I do hope to finish the project and keep contributing with whatever I can :)

Best, Matheus

Hi, everyone

Just wanted to share with you the final report on my GSoC project: https://matheustavares.gitlab.io/posts/gsoc-final-report I plan to continue the project (as it isn't finished yet) and keep contributing with whatever I can :) Thank you all for the support during this SoC!

Best, Matheus