git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 00/34] libify mailinfo and call it directly from am

From
Ramsay Jones <ramsay@ramsayjones.plus.com>
Date
Oct 21, 2015, 15:51 UTC
Message-ID
<5627B496.7030102@ramsayjones.plus.com>
In-Reply-To
<xmqqh9ll6xo8.fsf@gitster.mtv.corp.google.com>
On 20/10/15 22:24, Junio C Hamano wrote:
Show 17 quoted lines
> Junio C Hamano <gitster@pobox.com> writes:
> 
>> During the discussion on the recent "git am" regression, I noticed
>> that the command reimplemented in C spawns one "mailsplit" and then
>> spawns "mailinfo" followed by "apply --index" to commit the changes
>> described in each message.  As there are platforms where spawning
>> subprocess via run_command() interface is heavy-weight, something
>> that is conceptually very simple like "mailinfo" is better called
>> directly inside the process---something that is lightweight and
>> frequently used is where the overhead of run_command() would be felt
>> most.
> 
> Although I still haven't seen any offer to help from those who work
> on the platforms that may benefit from this series the most, I have
> some numbers on my desktop (Dell T3500 2.66GHz Xeon X5650 with 12GB,
> running Ubuntu), where the cost of spawning is not as costly as
> elsewhere, making this series less pressing.

I suspect that I haven't tested exactly the same version as you, but I had a quick look at testing this on Cygwin today. I have included a complete transcript (below), so you can see what I did wrong! :-P

Show 14 quoted lines
> 
> Between 'master' and the version with this series (on 'jch'),
> applying this 34-patch series itself on top of 'master' using "git
> am", best of 5 numbers for running:
> 
>     time git am mbox >/dev/null
> 
> are
> 
>       (master)                 (with the series)
>     real    0m0.648s            real    0m0.537s
>     user    0m0.358s            user    0m0.338s
>     sys     0m0.172s            sys     0m0.154s
> 
The corresponding times for me were:
    (master)           (with the series)
  real	0m9.760s      real	0m5.744s
  user	0m0.531s      user	0m0.656s
  sys	0m5.726s      sys	0m3.520s
So, yes, a noticeable improvement! :)
HTH

ATB, Ramsay Jones

  $ uname -a
  CYGWIN_NT-6.3 satellite 2.2.1(0.289/5/3) 2015-08-20 11:42 x86_64 Cygwin
  $ pwd
  /home/ramsay/git
  $ git log --decorate --oneline -1
  74301d6 (HEAD -> master, origin/master, origin/HEAD) Sync with maint
  $ ./git version
  git version 2.6.2.280.g74301d6
  $ git format-patch --stdout 2a5ce7c^..896df93 >mailinfo.mbox
  $ git format-patch --stdout a4106a8^..559e247 >>mailinfo.mbox
  $ git checkout -b master-mailinfo master
  Switched to a new branch 'master-mailinfo'
  $ time ./git am mailinfo.mbox
  Applying: mailinfo: remove a no-op call convert_to_utf8(it, "")
  Applying: mailinfo: fold decode_header_bq() into decode_header()
  Applying: mailinfo: fix an off-by-one error in the boundary stack
  Applying: mailinfo: explicitly close file handle to the patch output
  Applying: mailinfo: move handle_boundary() lower
  Applying: mailinfo: get rid of function-local static states
  Applying: mailinfo: do not let handle_body() touch global "line" directly
  Applying: mailinfo: do not let handle_boundary() touch global "line" directly
  Applying: mailinfo: do not let find_boundary() touch global "line" directly
  Applying: mailinfo: move global "line" into mailinfo() function
  Applying: mailinfo: introduce "struct mailinfo" to hold globals
  Applying: mailinfo: move keep_subject & keep_non_patch_bracket to struct mailinfo
  Applying: mailinfo: move global "FILE *fin, *fout" to struct mailinfo
  Applying: mailinfo: move filter/header stage to struct mailinfo
  Applying: mailinfo: move patch_lines to struct mailinfo
  Applying: mailinfo: move add_message_id and message_id to struct mailinfo
  Applying: mailinfo: move use_scissors and use_inbody_headers to struct mailinfo
  Applying: mailinfo: move metainfo_charset to struct mailinfo
  Applying: mailinfo: move check for metainfo_charset to convert_to_utf8()
  Applying: mailinfo: move transfer_encoding to struct mailinfo
  Applying: mailinfo: move charset to struct mailinfo
  Applying: mailinfo: move cmitmsg and patchfile to struct mailinfo
  Applying: mailinfo: move [ps]_hdr_data to struct mailinfo
  Applying: mailinfo: move content/content_top to struct mailinfo
  Applying: mailinfo: handle_commit_msg() shouldn't be called after finding patchbreak
  Applying: mailinfo: keep the parsed log message in a strbuf
  Applying: mailinfo: move read_one_header_line() closer to its callers
  Applying: mailinfo: move check_header() after the helpers it uses
  Applying: mailinfo: move cleanup_space() before its users
  Applying: mailinfo: move definition of MAX_HDR_PARSED closer to its use
  Applying: mailinfo: libify
  Applying: mailinfo: handle charset conversion errors in the caller
  Applying: mailinfo: remove calls to exit() and die() deep in the callchain
  Applying: am: make direct call to mailinfo
  Applying: mailinfo: plug strbuf leak during continuation line handling
  
  real	0m9.760s
  user	0m0.531s
  sys	0m5.726s
  $ 
  
  $ make clean >/dev/null 2>&1
  $ make >out.mi 2>&1
  $ ./git version
  git version 2.6.2.315.g1e9f6ff
  $ git describe
  v2.6.2-315-g1e9f6ff
  $ git checkout -b new-mailinfo master
  Switched to a new branch 'new-mailinfo'
  $ time ./git am mailinfo.mbox
  Applying: mailinfo: remove a no-op call convert_to_utf8(it, "")
  Applying: mailinfo: fold decode_header_bq() into decode_header()
  Applying: mailinfo: fix an off-by-one error in the boundary stack
  Applying: mailinfo: explicitly close file handle to the patch output
  Applying: mailinfo: move handle_boundary() lower
  Applying: mailinfo: get rid of function-local static states
  Applying: mailinfo: do not let handle_body() touch global "line" directly
  Applying: mailinfo: do not let handle_boundary() touch global "line" directly
  Applying: mailinfo: do not let find_boundary() touch global "line" directly
  Applying: mailinfo: move global "line" into mailinfo() function
  Applying: mailinfo: introduce "struct mailinfo" to hold globals
  Applying: mailinfo: move keep_subject & keep_non_patch_bracket to struct mailinfo
  Applying: mailinfo: move global "FILE *fin, *fout" to struct mailinfo
  Applying: mailinfo: move filter/header stage to struct mailinfo
  Applying: mailinfo: move patch_lines to struct mailinfo
  Applying: mailinfo: move add_message_id and message_id to struct mailinfo
  Applying: mailinfo: move use_scissors and use_inbody_headers to struct mailinfo
  Applying: mailinfo: move metainfo_charset to struct mailinfo
  Applying: mailinfo: move check for metainfo_charset to convert_to_utf8()
  Applying: mailinfo: move transfer_encoding to struct mailinfo
  Applying: mailinfo: move charset to struct mailinfo
  Applying: mailinfo: move cmitmsg and patchfile to struct mailinfo
  Applying: mailinfo: move [ps]_hdr_data to struct mailinfo
  Applying: mailinfo: move content/content_top to struct mailinfo
  Applying: mailinfo: handle_commit_msg() shouldn't be called after finding patchbreak
  Applying: mailinfo: keep the parsed log message in a strbuf
  Applying: mailinfo: move read_one_header_line() closer to its callers
  Applying: mailinfo: move check_header() after the helpers it uses
  Applying: mailinfo: move cleanup_space() before its users
  Applying: mailinfo: move definition of MAX_HDR_PARSED closer to its use
  Applying: mailinfo: libify
  Applying: mailinfo: handle charset conversion errors in the caller
  Applying: mailinfo: remove calls to exit() and die() deep in the callchain
  Applying: am: make direct call to mailinfo
  Applying: mailinfo: plug strbuf leak during continuation line handling
  
  real	0m5.744s
  user	0m0.656s
  sys	0m3.520s
  $ 
  $ git rev-parse master-mailinfo^{tree} new-mailinfo^{tree}
  cebb1a110f3af9f4393bb66b942ae28e2743f233
  cebb1a110f3af9f4393bb66b942ae28e2743f233
  $
Previous: Stefan BellerNext: Johannes Sixt
Message 119 of 124 in “mailinfo libification”
  1. 00/26 mailinfo libificationJunio C Hamano, Oct 13, 2015
  2. 01/26 mailinfo: remove a no-op call convert_to_utf8(it, "")Junio C Hamano, Oct 13, 2015
  3. 02/26 mailinfo: fix for off-by-one error in boundary stackJunio C Hamano, Oct 13, 2015
  4. Stefan BellerOct 14, 2015
  5. Junio C HamanoOct 14, 2015
  6. 03/26 mailinfo: fold decode_header_bq() into decode_header()Junio C Hamano, Oct 13, 2015
  7. 04/26 mailinfo: move handle_boundary() lowerJunio C Hamano, Oct 13, 2015
  8. 05/26 mailinfo: get rid of function-local static statesJunio C Hamano, Oct 13, 2015
  9. 06/26 mailinfo: always pass "line" as an argumentJunio C Hamano, Oct 13, 2015
  10. Stefan BellerOct 14, 2015
  11. Junio C HamanoOct 14, 2015
  12. 07/26 mailinfo: move global "line" into mailinfo() functionJunio C Hamano, Oct 13, 2015
  13. Stefan BellerOct 14, 2015
  14. 08/26 mailinfo: introduce "struct mailinfo" to hold globalsJunio C Hamano, Oct 13, 2015
  15. 09/26 mailinfo: move keep_subject & keep_non_patch_bracket to struct mailinfoJunio C Hamano, Oct 13, 2015
  16. 10/26 mailinfo: move global "FILE *fin, *fout" to struct mailinfoJunio C Hamano, Oct 13, 2015
  17. 11/26 mailinfo: move filter/header stage to struct mailinfoJunio C Hamano, Oct 13, 2015
  18. 12/26 mailinfo: move patch_lines to struct mailinfoJunio C Hamano, Oct 13, 2015
  19. 13/26 mailinfo: move add_message_id and message_id to struct mailinfoJunio C Hamano, Oct 13, 2015
  20. 14/26 mailinfo: move use_scissors and use_inbody_headers to struct mailinfoJunio C Hamano, Oct 13, 2015
  21. 15/26 mailinfo: move metainfo_charset to struct mailinfoJunio C Hamano, Oct 13, 2015
  22. 16/26 mailinfo: move transfer_encoding to struct mailinfoJunio C Hamano, Oct 13, 2015
  23. 17/26 mailinfo: move charset to struct mailinfoJunio C Hamano, Oct 13, 2015
  24. 18/26 mailinfo: handle_commit_msg() shouldn't be called after finding patchbreakJunio C Hamano, Oct 13, 2015
  25. 19/26 mailinfo: move cmitmsg and patchfile to struct mailinfoJunio C Hamano, Oct 13, 2015
  26. 27/26 mailinfo: close patchfileJunio C Hamano, Oct 14, 2015
  27. 28/26 am: make direct call to mailinfoJunio C Hamano, Oct 14, 2015
  28. 20/26 mailinfo: move [ps]_hdr_data to struct mailinfoJunio C Hamano, Oct 13, 2015
  29. 21/26 mailinfo: keep the parsed log message in a strbufJunio C Hamano, Oct 13, 2015
  30. 22/26 mailinfo: move content/content_top to struct mailinfoJunio C Hamano, Oct 13, 2015
  31. 23/26 mailinfo: handle errors found in decode_header() betterJunio C Hamano, Oct 13, 2015
  32. 24/26 mailinfo: handle charset conversion errors in the callerJunio C Hamano, Oct 13, 2015
  33. 25/26 mailinfo: remove calls to exit() and die() deep in the callchainJunio C Hamano, Oct 13, 2015
  34. 26/26 mailinfo: libify the whole thingJunio C Hamano, Oct 13, 2015
  35. 00/31 libify mailinfo and call it directly from amJunio C Hamano, Oct 14, 2015
  36. 01/31 mailinfo: remove a no-op call convert_to_utf8(it, "")Junio C Hamano, Oct 14, 2015
  37. 02/31 mailinfo: fix for off-by-one error in boundary stackJunio C Hamano, Oct 14, 2015
  38. 03/31 mailinfo: explicitly close file handle to the patch outputJunio C Hamano, Oct 14, 2015
  39. 04/31 mailinfo: fold decode_header_bq() into decode_header()Junio C Hamano, Oct 14, 2015
  40. 05/31 mailinfo: move handle_boundary() lowerJunio C Hamano, Oct 14, 2015
  41. 06/31 mailinfo: get rid of function-local static statesJunio C Hamano, Oct 14, 2015
  42. 07/31 mailinfo: always pass "line" as an argumentJunio C Hamano, Oct 14, 2015
  43. 08/31 mailinfo: move global "line" into mailinfo() functionJunio C Hamano, Oct 14, 2015
  44. 09/31 mailinfo: introduce "struct mailinfo" to hold globalsJunio C Hamano, Oct 14, 2015
  45. 10/31 mailinfo: move keep_subject & keep_non_patch_bracket to struct mailinfoJunio C Hamano, Oct 14, 2015
  46. 11/31 mailinfo: move global "FILE *fin, *fout" to struct mailinfoJunio C Hamano, Oct 14, 2015
  47. 12/31 mailinfo: move filter/header stage to struct mailinfoJunio C Hamano, Oct 14, 2015
  48. 13/31 mailinfo: move patch_lines to struct mailinfoJunio C Hamano, Oct 14, 2015
  49. 14/31 mailinfo: move add_message_id and message_id to struct mailinfoJunio C Hamano, Oct 14, 2015
  50. 15/31 mailinfo: move use_scissors and use_inbody_headers to struct mailinfoJunio C Hamano, Oct 14, 2015
  51. 16/31 mailinfo: move metainfo_charset to struct mailinfoJunio C Hamano, Oct 14, 2015
  52. Eric SunshineOct 15, 2015
  53. 17/31 mailinfo: move transfer_encoding to struct mailinfoJunio C Hamano, Oct 14, 2015
  54. 18/31 mailinfo: move charset to struct mailinfoJunio C Hamano, Oct 14, 2015
  55. 19/31 mailinfo: handle_commit_msg() shouldn't be called after finding patchbreakJunio C Hamano, Oct 14, 2015
  56. 20/31 mailinfo: move cmitmsg and patchfile to struct mailinfoJunio C Hamano, Oct 14, 2015
  57. Junio C HamanoOct 14, 2015
  58. 21/31 mailinfo: move [ps]_hdr_data to struct mailinfoJunio C Hamano, Oct 14, 2015
  59. Junio C HamanoOct 14, 2015
  60. 22/31 mailinfo: keep the parsed log message in a strbufJunio C Hamano, Oct 14, 2015
  61. 23/31 mailinfo: move content/content_top to struct mailinfoJunio C Hamano, Oct 14, 2015
  62. 24/31 mailinfo: move read_one_header_line() closer to its callersJunio C Hamano, Oct 14, 2015
  63. 25/31 mailinfo: move check_header() after the helpers it usesJunio C Hamano, Oct 14, 2015
  64. 26/31 mailinfo: move cleanup_space() before its usersJunio C Hamano, Oct 14, 2015
  65. 27/31 mailinfo: move definition of MAX_HDR_PARSED to closer to its useJunio C Hamano, Oct 14, 2015
  66. 28/31 mailinfo: libifyJunio C Hamano, Oct 14, 2015
  67. 29/31 mailinfo: handle charset conversion errors in the callerJunio C Hamano, Oct 14, 2015
  68. 30/31 mailinfo: remove calls to exit() and die() deep in the callchainJunio C Hamano, Oct 14, 2015
  69. 31/31 am: make direct call to mailinfoJunio C Hamano, Oct 14, 2015
  70. 00/34 libify mailinfo and call it directly from amJunio C Hamano, Oct 19, 2015
  71. 01/34 mailinfo: remove a no-op call convert_to_utf8(it, "")Junio C Hamano, Oct 19, 2015
  72. 02/34 mailinfo: fold decode_header_bq() into decode_header()Junio C Hamano, Oct 19, 2015
  73. 03/34 mailinfo: fix an off-by-one error in the boundary stackJunio C Hamano, Oct 19, 2015
  74. 04/34 mailinfo: explicitly close file handle to the patch outputJunio C Hamano, Oct 19, 2015
  75. 05/34 mailinfo: move handle_boundary() lowerJunio C Hamano, Oct 19, 2015
  76. 06/34 mailinfo: get rid of function-local static statesJunio C Hamano, Oct 19, 2015
  77. 07/34 mailinfo: do not let handle_body() touch global "line" directlyJunio C Hamano, Oct 19, 2015
  78. 08/34 mailinfo: do not let handle_boundary() touch global "line" directlyJunio C Hamano, Oct 19, 2015
  79. 09/34 mailinfo: do not let find_boundary() touch global "line" directlyJunio C Hamano, Oct 19, 2015
  80. 10/34 mailinfo: move global "line" into mailinfo() functionJunio C Hamano, Oct 19, 2015
  81. Eric SunshineOct 19, 2015
  82. Junio C HamanoOct 20, 2015
  83. 11/34 mailinfo: introduce "struct mailinfo" to hold globalsJunio C Hamano, Oct 19, 2015
  84. 12/34 mailinfo: move keep_subject & keep_non_patch_bracket to struct mailinfoJunio C Hamano, Oct 19, 2015
  85. 13/34 mailinfo: move global "FILE *fin, *fout" to struct mailinfoJunio C Hamano, Oct 19, 2015
  86. 14/34 mailinfo: move filter/header stage to struct mailinfoJunio C Hamano, Oct 19, 2015
  87. Stefan BellerOct 21, 2015
  88. 15/34 mailinfo: move patch_lines to struct mailinfoJunio C Hamano, Oct 19, 2015
  89. 16/34 mailinfo: move add_message_id and message_id to struct mailinfoJunio C Hamano, Oct 19, 2015
  90. 17/34 mailinfo: move use_scissors and use_inbody_headers to struct mailinfoJunio C Hamano, Oct 19, 2015
  91. Stefan BellerOct 21, 2015
  92. Junio C HamanoOct 21, 2015
  93. 18/34 mailinfo: move metainfo_charset to struct mailinfoJunio C Hamano, Oct 19, 2015
  94. 19/34 mailinfo: move check for metainfo_charset to convert_to_utf8()Junio C Hamano, Oct 19, 2015
  95. Eric SunshineOct 19, 2015
  96. 20/34 mailinfo: move transfer_encoding to struct mailinfoJunio C Hamano, Oct 19, 2015
  97. 21/34 mailinfo: move charset to struct mailinfoJunio C Hamano, Oct 19, 2015
  98. 22/34 mailinfo: move cmitmsg and patchfile to struct mailinfoJunio C Hamano, Oct 19, 2015
  99. 23/34 mailinfo: move [ps]_hdr_data to struct mailinfoJunio C Hamano, Oct 19, 2015
  100. Stefan BellerOct 21, 2015
  101. 24/34 mailinfo: move content/content_top to struct mailinfoJunio C Hamano, Oct 19, 2015
  102. Stefan BellerOct 21, 2015
  103. Junio C HamanoOct 21, 2015
  104. Stefan BellerOct 21, 2015
  105. 25/34 mailinfo: handle_commit_msg() shouldn't be called after finding patchbreakJunio C Hamano, Oct 19, 2015
  106. 26/34 mailinfo: keep the parsed log message in a strbufJunio C Hamano, Oct 19, 2015
  107. 27/34 mailinfo: move read_one_header_line() closer to its callersJunio C Hamano, Oct 19, 2015
  108. 28/34 mailinfo: move check_header() after the helpers it usesJunio C Hamano, Oct 19, 2015
  109. 29/34 mailinfo: move cleanup_space() before its usersJunio C Hamano, Oct 19, 2015
  110. 30/34 mailinfo: move definition of MAX_HDR_PARSED closer to its useJunio C Hamano, Oct 19, 2015
  111. 31/34 mailinfo: libifyJunio C Hamano, Oct 19, 2015
  112. 32/34 mailinfo: handle charset conversion errors in the callerJunio C Hamano, Oct 19, 2015
  113. 33/34 mailinfo: remove calls to exit() and die() deep in the callchainJunio C Hamano, Oct 19, 2015
  114. 34/34 am: make direct call to mailinfoJunio C Hamano, Oct 19, 2015
  115. Junio C HamanoOct 20, 2015
  116. Stefan BellerOct 20, 2015
  117. Junio C HamanoOct 20, 2015
  118. Stefan BellerOct 20, 2015
  119. Ramsay JonesOct 21, 2015
  120. Johannes SixtOct 21, 2015
  121. Junio C HamanoOct 21, 2015
  122. Johannes SchindelinOct 26, 2015
  123. Junio C HamanoOct 26, 2015
  124. 00/35 libify mailinfo and call it directly from amJunio C Hamano, Oct 21, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.