git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 00/10] send-email: various optimizations to speed up by >2x

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
May 21, 2021, 06:23 UTC
Message-ID
<875yzcpo52.fsf@evledraar.gmail.com>
In-Reply-To
<xmqqv97drmge.fsf@gitster.g>
On Fri, May 21 2021, Junio C Hamano wrote:
Show 14 quoted lines
> Jeff King <peff@peff.net> writes:
>
>> I like all of this, except for the change in the interface of
>> Git::config_regexp(). You mention that it's new-ish, and probably not in
>> wide use. And I agree that's probably true. But it feels like violating
>> a principle of not breaking APIs, and we should stick to that principle
>> and not bend it for "well, it's not that old an API".
>>
>> I'd find it more compelling if it the existing interface was broken or
>> hard to avoid changing. But couldn't we just add a new function with the
>> extra info (config_regexp_with_values() or something)?
>
> We seem to use it in-tree only once, but we have no control over
> what out-of-tree users have been doing.

I thought people might have bought my argument in v1 that we just document these as "public" as part of some copy/pasting in Git.pm, but sure, I'll re-roll and make this a new function or something...

Show 19 quoted lines
> I do like your proposed name for the fact that it has _with_values
> in it; the original gave us only a list of configuration variable
> names, and now we are getting name-value tuples.
>
> BUT
>
> I am not sure if I like the new interface, and I do not know if the
> the name "config_with_values" is a good match for what it does.
>
> Namely, when a function is described as:
>
>     =item config_regexp ( RE )
>
>     Retrieve the list of configuration key names matching the regular
>     expression C<RE>. The return value is an ARRAY of key-value pairs.
>
> I would expect it to return ([key1, value1], [key2, value2], ...),
> but the implementation returns (key1, value1, key2, value2, ...),
> i.e. a flattened list, if I am not mistaken.
...
Show 33 quoted lines
>     my @cmd = ('config', '--null', '--get-regexp', $regex);
>     unshift @cmd, $self if $self;
>     my $data = command(@cmd);
>     my (@kv) = map { split /\n/, $_, 2 } split /\0/, $data;
>     return @kv;
>
> We get NUL terminated records, so we first split the entire output
> at NULs (to get a list of key-value records).  Each key-value record
> has the key followed by LF followed by value, so we split it at the
> first LF (i.e. a value with an embedded LF would behave correctly)
> to extract key and value out of it.  But the resulting list is a
> flat list with alternating key and value.
>
> The side that works on the returned value of courese knows that it
> is getting a flattened list and acts accordingly:
>
>     my @kv = Git::config_regexp("^sende?mail[.]");
>     while (my ($k, $v) = splice @kv, 0, 2) {
>         push @{$known_config_keys{$k}} => $v;
>     }
>
> Perhaps it may be more performant than a more obvious and
> straight-forward "list of tuples", i.e.
>
>     return map { [split /\n/, $_, 2] } split /\0/, $data;
>
>     my @kv = Git::config_regexp("^sende?mail[.]");
>     for my $tuple (@kv) {
>         push @{$known_config_keys{$tuple->[0]}}, $tuple->[1];
>     }
>
> but somehow the flattened list felt unnatural at least to a naïve
> reader like me.

The "performant" really doesn't matter here, we're comparing shelling out to getting a small number of config keys back. So I wasn't trying to optimize this.

Returning a flattened list is idiomatic in Perl, it means that a caller can do any of:

    # I only care about the last value for a key, or only about
    # existence checks
    my %hash = func();
Or:
    # I want all key-values to iterate over
    my @kv = func();

Returning touples like this makes that less convenient for both, who'll need to do more work to unpack them.

For what it's worth In Perl "return a list" means "flattened list", the term "flattened list" I think comes from other languages. You'd call what you're suggesting a list of arrays, or (if a top-level reference) an array of arrays, AoA for short, AoH for array (ref) of hashes etc.

Previous: Junio C HamanoNext: Junio C Hamano
Message 39 of 108 in “send-email: various optimizations to speed up by >2x”
  1. 0/9 send-email: various optimizations to speed up by >2xÆvar Arnfjörð Bjarmason, May 12, 2021
  2. 1/9 send-email: remove non-working support for "sendemail.smtpssl"Ævar Arnfjörð Bjarmason, May 12, 2021
  3. Đoàn Trần Công DanhMay 12, 2021
  4. 2/9 send-email: refactor sendemail.smtpencryption config parsingÆvar Arnfjörð Bjarmason, May 12, 2021
  5. 3/9 send-email: lazily load config for a big speedupÆvar Arnfjörð Bjarmason, May 12, 2021
  6. 4/9 send-email: lazily shell out to "git var"Ævar Arnfjörð Bjarmason, May 12, 2021
  7. 5/9 send-email: use function syntax instead of barewordsÆvar Arnfjörð Bjarmason, May 12, 2021
  8. Jeff KingMay 12, 2021
  9. 6/9 send-email: get rid of indirect object syntaxÆvar Arnfjörð Bjarmason, May 12, 2021
  10. 7/9 send-email: lazily load modules for a big speedupÆvar Arnfjörð Bjarmason, May 12, 2021
  11. 8/9 perl: lazily load some common Git.pm setup codeÆvar Arnfjörð Bjarmason, May 12, 2021
  12. Eric WongMay 12, 2021
  13. 9/9 send-email: move trivial config handling to PerlÆvar Arnfjörð Bjarmason, May 12, 2021
  14. Eric SunshineMay 12, 2021
  15. Jeff KingMay 12, 2021
  16. Felipe ContrerasMay 13, 2021
  17. Jeff KingMay 13, 2021
  18. Felipe ContrerasMay 13, 2021
  19. Ævar Arnfjörð BjarmasonMay 13, 2021
  20. Ævar Arnfjörð BjarmasonMay 13, 2021
  21. Eric WongMay 12, 2021
  22. Jeff KingMay 12, 2021
  23. Jeff KingMay 12, 2021
  24. Ævar Arnfjörð BjarmasonMay 13, 2021
  25. Jeff KingMay 13, 2021
  26. 00/10 send-email: various optimizations to speed up by >2xÆvar Arnfjörð Bjarmason, May 20, 2021
  27. 01/10 send-email: remove non-working support for "sendemail.smtpssl"Ævar Arnfjörð Bjarmason, May 20, 2021
  28. 02/10 send-email: refactor sendemail.smtpencryption config parsingÆvar Arnfjörð Bjarmason, May 20, 2021
  29. 03/10 send-email: lazily load config for a big speedupÆvar Arnfjörð Bjarmason, May 20, 2021
  30. 04/10 send-email: lazily shell out to "git var"Ævar Arnfjörð Bjarmason, May 20, 2021
  31. 05/10 send-email: use function syntax instead of barewordsÆvar Arnfjörð Bjarmason, May 20, 2021
  32. 06/10 send-email: get rid of indirect object syntaxÆvar Arnfjörð Bjarmason, May 20, 2021
  33. 08/10 perl: lazily load some common Git.pm setup codeÆvar Arnfjörð Bjarmason, May 20, 2021
  34. 07/10 send-email: lazily load modules for a big speedupÆvar Arnfjörð Bjarmason, May 20, 2021
  35. 10/10 perl: nano-optimize by replacing Cwd::cwd() with Cwd::getcwd()Ævar Arnfjörð Bjarmason, May 20, 2021
  36. 09/10 send-email: move trivial config handling to PerlÆvar Arnfjörð Bjarmason, May 20, 2021
  37. Jeff KingMay 20, 2021
  38. Junio C HamanoMay 20, 2021
  39. Ævar Arnfjörð BjarmasonMay 21, 2021
  40. Junio C HamanoMay 21, 2021
  41. Jeff KingMay 21, 2021
  42. Ævar Arnfjörð BjarmasonMay 21, 2021
  43. Jeff KingMay 21, 2021
  44. Felipe ContrerasMay 28, 2021
  45. Ævar Arnfjörð BjarmasonMay 29, 2021
  46. Felipe ContrerasMay 29, 2021
  47. Ævar Arnfjörð BjarmasonMay 30, 2021
  48. Felipe ContrerasMay 30, 2021
  49. Felipe ContrerasMay 28, 2021
  50. 00/13 send-email: various optimizations to speed up by >2xÆvar Arnfjörð Bjarmason, May 23, 2021
  51. 01/13 send-email tests: support GIT_TEST_PERL_FATAL_WARNINGS=trueÆvar Arnfjörð Bjarmason, May 23, 2021
  52. 02/13 send-email tests: test for boolean variables without a valueÆvar Arnfjörð Bjarmason, May 23, 2021
  53. 03/13 send-email: remove non-working support for "sendemail.smtpssl"Ævar Arnfjörð Bjarmason, May 23, 2021
  54. 04/13 send-email: refactor sendemail.smtpencryption config parsingÆvar Arnfjörð Bjarmason, May 23, 2021
  55. 05/13 send-email: copy "config_regxp" into git-send-email.perlÆvar Arnfjörð Bjarmason, May 23, 2021
  56. 06/13 send-email: lazily load config for a big speedupÆvar Arnfjörð Bjarmason, May 23, 2021
  57. 07/13 send-email: lazily shell out to "git var"Ævar Arnfjörð Bjarmason, May 23, 2021
  58. 09/13 send-email: get rid of indirect object syntaxÆvar Arnfjörð Bjarmason, May 23, 2021
  59. 08/13 send-email: use function syntax instead of barewordsÆvar Arnfjörð Bjarmason, May 23, 2021
  60. 10/13 send-email: lazily load modules for a big speedupÆvar Arnfjörð Bjarmason, May 23, 2021
  61. 11/13 perl: lazily load some common Git.pm setup codeÆvar Arnfjörð Bjarmason, May 23, 2021
  62. 12/13 send-email: move trivial config handling to PerlÆvar Arnfjörð Bjarmason, May 23, 2021
  63. 13/13 perl: nano-optimize by replacing Cwd::cwd() with Cwd::getcwd()Ævar Arnfjörð Bjarmason, May 23, 2021
  64. Junio C HamanoMay 24, 2021
  65. 00/13 send-email: various optimizations to speed up by >2xÆvar Arnfjörð Bjarmason, May 24, 2021
  66. 01/13 send-email tests: support GIT_TEST_PERL_FATAL_WARNINGS=trueÆvar Arnfjörð Bjarmason, May 24, 2021
  67. 02/13 send-email tests: test for boolean variables without a valueÆvar Arnfjörð Bjarmason, May 24, 2021
  68. 03/13 send-email: remove non-working support for "sendemail.smtpssl"Ævar Arnfjörð Bjarmason, May 24, 2021
  69. 04/13 send-email: refactor sendemail.smtpencryption config parsingÆvar Arnfjörð Bjarmason, May 24, 2021
  70. 07/13 send-email: lazily shell out to "git var"Ævar Arnfjörð Bjarmason, May 24, 2021
  71. 06/13 send-email: lazily load config for a big speedupÆvar Arnfjörð Bjarmason, May 24, 2021
  72. 05/13 send-email: copy "config_regxp" into git-send-email.perlÆvar Arnfjörð Bjarmason, May 24, 2021
  73. 09/13 send-email: get rid of indirect object syntaxÆvar Arnfjörð Bjarmason, May 24, 2021
  74. 10/13 send-email: lazily load modules for a big speedupÆvar Arnfjörð Bjarmason, May 24, 2021
  75. Junio C HamanoMay 27, 2021
  76. Ævar Arnfjörð BjarmasonMay 27, 2021
  77. 11/13 perl: lazily load some common Git.pm setup codeÆvar Arnfjörð Bjarmason, May 24, 2021
  78. 12/13 send-email: move trivial config handling to PerlÆvar Arnfjörð Bjarmason, May 24, 2021
  79. Jeff KingMay 27, 2021
  80. 08/13 send-email: use function syntax instead of barewordsÆvar Arnfjörð Bjarmason, May 24, 2021
  81. 13/13 perl: nano-optimize by replacing Cwd::cwd() with Cwd::getcwd()Ævar Arnfjörð Bjarmason, May 24, 2021
  82. Jeff KingMay 27, 2021
  83. 00/13 send-email: various optimizations to speed up by >2xÆvar Arnfjörð Bjarmason, May 28, 2021
  84. 01/13 send-email tests: support GIT_TEST_PERL_FATAL_WARNINGS=trueÆvar Arnfjörð Bjarmason, May 28, 2021
  85. 03/13 send-email: remove non-working support for "sendemail.smtpssl"Ævar Arnfjörð Bjarmason, May 28, 2021
  86. 02/13 send-email tests: test for boolean variables without a valueÆvar Arnfjörð Bjarmason, May 28, 2021
  87. Jeff KingJul 10, 2021
  88. 04/13 send-email: refactor sendemail.smtpencryption config parsingÆvar Arnfjörð Bjarmason, May 28, 2021
  89. 06/13 send-email: lazily load config for a big speedupÆvar Arnfjörð Bjarmason, May 28, 2021
  90. 05/13 send-email: copy "config_regxp" into git-send-email.perlÆvar Arnfjörð Bjarmason, May 28, 2021
  91. 07/13 send-email: lazily shell out to "git var"Ævar Arnfjörð Bjarmason, May 28, 2021
  92. 09/13 send-email: get rid of indirect object syntaxÆvar Arnfjörð Bjarmason, May 28, 2021
  93. 08/13 send-email: use function syntax instead of barewordsÆvar Arnfjörð Bjarmason, May 28, 2021
  94. Felipe ContrerasMay 28, 2021
  95. Ævar Arnfjörð BjarmasonMay 29, 2021
  96. Felipe ContrerasMay 29, 2021
  97. 10/13 send-email: lazily load modules for a big speedupÆvar Arnfjörð Bjarmason, May 28, 2021
  98. Felipe ContrerasMay 28, 2021
  99. Ævar Arnfjörð BjarmasonMay 29, 2021
  100. Felipe ContrerasMay 29, 2021
  101. 12/13 send-email: move trivial config handling to PerlÆvar Arnfjörð Bjarmason, May 28, 2021
  102. 11/13 perl: lazily load some common Git.pm setup codeÆvar Arnfjörð Bjarmason, May 28, 2021
  103. 13/13 perl: nano-optimize by replacing Cwd::cwd() with Cwd::getcwd()Ævar Arnfjörð Bjarmason, May 28, 2021
  104. Felipe ContrerasMay 28, 2021
  105. Jeff KingMay 31, 2021
  106. Ævar Arnfjörð BjarmasonMay 31, 2021
  107. Jeff KingMay 31, 2021
  108. Elijah NewrenMay 27, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.