git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 11/19] parallel-checkout: make it truly parallel

From
Matheus Tavares Bernardino <matheus.bernardino@usp.br>
Date
Sep 30, 2020, 14:02 UTC
Message-ID
<CAHd-oW7ErMKqeeoF40Mq418SusbVhkL2R7W73=4mbXUiamY1mA@mail.gmail.com>
In-Reply-To
<CAN0heSoJvkZ+gvQwZMPW5Q0HqB0mOUnVqXiOqnPk48cDxMzAqw@mail.gmail.com>
Hi, Martin
On Tue, Sep 29, 2020 at 4:52 PM Martin Ågren <martin.agren@gmail.com> wrote:
Show 20 quoted lines
>
> Hi Matheus,
>
> On Wed, 23 Sep 2020 at 00:53, Matheus Tavares <matheus.bernardino@usp.br> wrote:
> > --- a/Documentation/config/checkout.txt
> > +++ b/Documentation/config/checkout.txt
> > @@ -16,3 +16,24 @@ will checkout the '<something>' branch on another remote,
> >  and by linkgit:git-worktree[1] when 'git worktree add' refers to a
> >  remote branch. This setting might be used for other checkout-like
> >  commands or functionality in the future.
> > +
> > +checkout.workers::
> > +       The number of parallel workers to use when updating the working tree.
> > +       The default is one, i.e. sequential execution. If set to a value less
> > +       than one, Git will use as many workers as the number of logical cores
> > +       available. This setting and checkout.thresholdForParallelism affect all
>
> If you end up rerolling this patch series for other reasons, you might
> want to consider using `backticks` around
> `checkout.thresholdForParallelism` so that it gets typeset as monospace.
Sure, thanks!
Show 10 quoted lines
> > +       commands that perform checkout. E.g. checkout, switch, clone, reset,
> > +       sparse-checkout, read-tree, etc.
>
> Similarly here. Or perhaps go for "linkgit:git-checkout[1],
> linkgit:git-switch[1]" etc.
>
> BTW, as far as ".e.g.," goes, this list looks quite long. :) I almost
> get the feeling you've made it fairly exhaustive and added the "e.g.,"
> more as future proofing than anything else. I don't think anyone would
> complain if you left out, say, the plumbing `git read-tree` from the list.
Yeah, I might have gotten a little too carried away there hehe :)
Show 11 quoted lines
> > +Note: parallel checkout usually delivers better performance for repositories
> > +located on SSDs or over NFS. For repositories on spinning disks and/or machines
> > +with a small number of cores, the default sequential checkout often performs
> > +better. The size and compression level of a repository might also influence how
> > +well the parallel version performs.
> > +
> > +checkout.thresholdForParallelism::
>
> Sorry if this has already been discussed, but this "For" looks somewhat
> odd. Basically every config knob is "somethingForSomething". ;-) Could
> this be `checkout.parallelismThreshold`?

Thanks for the suggestion. TBH, I spent more time than I'd like to admit trying to come up with a name for this setting... But `checkout.parallelismThreshold` does sound better.

Previous: Martin ÅgrenNext: Matheus Tavares
Message 45 of 61 in “[RFC] Parallel checkout”
  1. 00/21 [RFC] Parallel checkoutMatheus Tavares, Aug 10, 2020
  2. 01/21 convert: make convert_attrs() and convert structs publicMatheus Tavares, Aug 10, 2020
  3. 02/21 convert: add [async_]convert_to_working_tree_ca() variantsMatheus Tavares, Aug 10, 2020
  4. 03/21 convert: add get_stream_filter_ca() variantMatheus Tavares, Aug 10, 2020
  5. 04/21 convert: add conv_attrs classificationMatheus Tavares, Aug 10, 2020
  6. 05/21 entry: extract a header file for entry.c functionsMatheus Tavares, Aug 10, 2020
  7. 06/21 entry: make fstat_output() and read_blob_entry() publicMatheus Tavares, Aug 10, 2020
  8. 07/21 entry: extract cache_entry update from write_entry()Matheus Tavares, Aug 10, 2020
  9. 08/21 entry: move conv_attrs lookup up to checkout_entry()Matheus Tavares, Aug 10, 2020
  10. 09/21 entry: add checkout_entry_ca() which takes preloaded conv_attrsMatheus Tavares, Aug 10, 2020
  11. 10/21 unpack-trees: add basic support for parallel checkoutMatheus Tavares, Aug 10, 2020
  12. 11/21 parallel-checkout: make it truly parallelMatheus Tavares, Aug 10, 2020
  13. Jeff HostetlerAug 19, 2020
  14. Matheus Tavares BernardinoAug 20, 2020
  15. Jeff HostetlerAug 20, 2020
  16. 12/21 parallel-checkout: add configuration optionsMatheus Tavares, Aug 10, 2020
  17. 13/21 parallel-checkout: support progress displayingMatheus Tavares, Aug 10, 2020
  18. 14/21 make_transient_cache_entry(): optionally alloc from mem_poolMatheus Tavares, Aug 10, 2020
  19. 15/21 builtin/checkout.c: complete parallel checkout supportMatheus Tavares, Aug 10, 2020
  20. 16/21 checkout-index: add parallel checkout supportMatheus Tavares, Aug 10, 2020
  21. 17/21 parallel-checkout: avoid stat() calls in workersMatheus Tavares, Aug 10, 2020
  22. 18/21 entry: use is_dir_sep() when checking leading dirsMatheus Tavares, Aug 10, 2020
  23. 19/21 symlinks: make has_dirs_only_path() track FL_NOENTMatheus Tavares, Aug 10, 2020
  24. 20/21 parallel-checkout: create leading dirs in workersMatheus Tavares, Aug 10, 2020
  25. 21/21 parallel-checkout: skip checking the working tree on cloneMatheus Tavares, Aug 10, 2020
  26. Jeff HostetlerAug 12, 2020
  27. 00/19 Parallel Checkout (part I)Matheus Tavares, Sep 22, 2020
  28. 01/19 convert: make convert_attrs() and convert structs publicMatheus Tavares, Sep 22, 2020
  29. 03/19 convert: add get_stream_filter_ca() variantMatheus Tavares, Sep 22, 2020
  30. 02/19 convert: add [async_]convert_to_working_tree_ca() variantsMatheus Tavares, Sep 22, 2020
  31. 04/19 convert: add conv_attrs classificationMatheus Tavares, Sep 22, 2020
  32. 05/19 entry: extract a header file for entry.c functionsMatheus Tavares, Sep 22, 2020
  33. 06/19 entry: make fstat_output() and read_blob_entry() publicMatheus Tavares, Sep 22, 2020
  34. 07/19 entry: extract cache_entry update from write_entry()Matheus Tavares, Sep 22, 2020
  35. 08/19 entry: move conv_attrs lookup up to checkout_entry()Matheus Tavares, Sep 22, 2020
  36. Jeff HostetlerOct 1, 2020
  37. Jeff HostetlerOct 1, 2020
  38. 09/19 entry: add checkout_entry_ca() which takes preloaded conv_attrsMatheus Tavares, Sep 22, 2020
  39. 10/19 unpack-trees: add basic support for parallel checkoutMatheus Tavares, Sep 22, 2020
  40. parallel-checkout: drop unused checkout state parameterJeff King, Oct 5, 2020
  41. Matheus Tavares BernardinoOct 5, 2020
  42. Jeff KingOct 5, 2020
  43. 11/19 parallel-checkout: make it truly parallelMatheus Tavares, Sep 22, 2020
  44. Martin ÅgrenSep 29, 2020
  45. Matheus Tavares BernardinoSep 30, 2020
  46. 12/19 parallel-checkout: support progress displayingMatheus Tavares, Sep 22, 2020
  47. 13/19 make_transient_cache_entry(): optionally alloc from mem_poolMatheus Tavares, Sep 22, 2020
  48. 14/19 builtin/checkout.c: complete parallel checkout supportMatheus Tavares, Sep 22, 2020
  49. 15/19 checkout-index: add parallel checkout supportMatheus Tavares, Sep 22, 2020
  50. 16/19 parallel-checkout: add tests for basic operationsMatheus Tavares, Sep 22, 2020
  51. Jonathan NiederOct 20, 2020
  52. Taylor BlauOct 20, 2020
  53. Matheus Tavares BernardinoOct 20, 2020
  54. Junio C HamanoOct 20, 2020
  55. Matheus Tavares BernardinoOct 20, 2020
  56. Jonathan NiederOct 20, 2020
  57. Junio C HamanoOct 20, 2020
  58. 17/19 parallel-checkout: add tests related to clone collisionsMatheus Tavares, Sep 22, 2020
  59. 18/19 parallel-checkout: add tests related to .gitattributesMatheus Tavares, Sep 22, 2020
  60. 19/19 ci: run test round with parallel-checkout enabledMatheus Tavares, Sep 22, 2020
  61. Jeff HostetlerOct 1, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.