git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v5 0/7] Fast git status via a file system watcher

From
Ben Peart <peartben@gmail.com>
Date
Jul 10, 2017, 14:40 UTC
Message-ID
<97342e22-c2ca-9c09-4db0-24cac014b689@gmail.com>
In-Reply-To
<9b1af00c-2451-501c-5315-7ed2b81f3f18@gmail.com>
On 7/10/2017 9:36 AM, Ben Peart wrote:
Show 56 quoted lines
> 
> 
> On 6/28/2017 1:11 AM, Christian Couder wrote:
>> On Sat, Jun 10, 2017 at 3:40 PM, Ben Peart <peartben@gmail.com> wrote:
>>> Changes from V4 include:
>> ...
>>
>> I took a look at this patch series except the last patch ([PATCH v5
>> 7/7] fsmonitor: add a performance test) as Junio reviewed it already,
>> and had only a few comments on patches 3/7 and 4/7.
>>
>> I am still not convinced by the discussions following v2
>> (http://public-inbox.org/git/20170518201333.13088-1-benpeart@microsoft.com/) 
>>
>> about using a hook instead of for example a "core.fsmonitorcommand".
>>
>> I think using a hook is not necessary and might not be a good match
>> for later optimizations. For example people might want to use a
>> library or some OS specific system calls to do what the hook does.
>>
>> AEvar previously reported some not so great performance numbers on
>> some big Booking.com boxes with a big monorepo and it seems that using
>> taskset for example to make sure that the hook is run on the same CPU
>> improves these numbers significantly. So avoiding to run a separate
>> process can be important in some cases.
>>
> 
> Using a hook is the only pattern I've seen in git that provides a way to 
> enable OS specific calls.  I used a hook so that different file 
> monitoring services could be plugged in depending on the OS or tools 
> available.  The Watchman integration script was mostly intended as a 
> sample that could be used where Watchman is available and works well.
> 
> I had not heard about the taskset issues you mention above.  If there is 
> something else that can be done to make this work better, please let me 
> know the details.
> 
> If I'm understanding you correctly, you are suggesting that someone 
> should be able to configure a setting (core.fsmonitorcommand) that gives 
> a custom command line that would be run instead of running the 
> query-fsmonitor hook.
> 
> I'm not entirely sure how that should work.  There are command line 
> options that need to be passed (currently the interface version as well 
> as the current clock in nanoseconds).  How would those passed when using 
> the custom command?
> 
> Is it OK to just append them to the given command line?  Does there need 
> to be some substitution token to indicate where they should be inserted 
> (ie "mycustomcommand --custom --options %version% --more-options 
> %timestamp%").  Are there any other tokens that should be supported (ie 
> PID or processor mask?).
> 
> Is there a design pattern already used somewhere in git that I can 
> follow or is this all blazing a new trail?
> 
My co-worker reminded me about git difftool - is this what you had in mind?
> Thanks for continuing to look into this. Feedback is good!
> 
> 
Previous: Ben PeartNext: Ben Peart
Message 24 of 137 in “Fast git status via a file system watcher”
  1. 0/7 Fast git status via a file system watcherBen Peart, Jun 10, 2017
  2. 2/7 dir: make lookup_untracked() available outside of dir.cBen Peart, Jun 10, 2017
  3. 1/7 bswap: add 64 bit endianness helper get_be64Ben Peart, Jun 10, 2017
  4. 4/7 fsmonitor: add test cases for fsmonitor extensionBen Peart, Jun 10, 2017
  5. Christian CouderJun 27, 2017
  6. Ben PeartJul 7, 2017
  7. 3/7 fsmonitor: teach git to optionally utilize a file system monitor to speed up detecting new or changed files.Ben Peart, Jun 10, 2017
  8. Christian CouderJun 27, 2017
  9. Ben PeartJul 3, 2017
  10. 6/7 fsmonitor: add a sample query-fsmonitor hook script for WatchmanBen Peart, Jun 10, 2017
  11. 7/7 fsmonitor: add a performance testBen Peart, Jun 10, 2017
  12. Ben PeartJun 10, 2017
  13. Junio C HamanoJun 12, 2017
  14. Ben PeartJun 14, 2017
  15. Junio C HamanoJun 14, 2017
  16. Ben PeartJul 7, 2017
  17. Junio C HamanoJul 7, 2017
  18. Ben PeartJul 7, 2017
  19. David TurnerJul 7, 2017
  20. Christian CouderJul 8, 2017
  21. 5/7 fsmonitor: add documentation for the fsmonitor extension.Ben Peart, Jun 10, 2017
  22. Christian CouderJun 28, 2017
  23. Ben PeartJul 10, 2017
  24. Ben PeartJul 10, 2017
  25. 00/12 Fast git status via a file system watcherBen Peart, Sep 15, 2017
  26. 01/12 bswap: add 64 bit endianness helper get_be64Ben Peart, Sep 15, 2017
  27. 04/12 fsmonitor: teach git to optionally utilize a file system monitor to speed up detecting new or changed files.Ben Peart, Sep 15, 2017
  28. David TurnerSep 15, 2017
  29. Ben PeartSep 18, 2017
  30. David TurnerSep 18, 2017
  31. Ben PeartSep 18, 2017
  32. 05/12 fsmonitor: add documentation for the fsmonitor extension.Ben Peart, Sep 15, 2017
  33. David TurnerSep 15, 2017
  34. Ben PeartSep 18, 2017
  35. Junio C HamanoSep 17, 2017
  36. Ben PeartSep 18, 2017
  37. 07/12 update-index: add fsmonitor support to update-indexBen Peart, Sep 15, 2017
  38. 06/12 ls-files: Add support in ls-files to display the fsmonitor valid bitBen Peart, Sep 15, 2017
  39. David TurnerSep 15, 2017
  40. 10/12 fsmonitor: add test cases for fsmonitor extensionBen Peart, Sep 15, 2017
  41. David TurnerSep 15, 2017
  42. David TurnerSep 19, 2017
  43. Ben PeartSep 19, 2017
  44. Torsten BögershausenSep 16, 2017
  45. 1/1 test-lint: echo -e (or -E) is not portabletboegi@web.de, Sep 17, 2017
  46. Jonathan NiederSep 19, 2017
  47. Torsten BögershausenSep 20, 2017
  48. Junio C HamanoSep 22, 2017
  49. Ben PeartSep 18, 2017
  50. Junio C HamanoSep 17, 2017
  51. Ben PeartSep 18, 2017
  52. Jonathan NiederSep 19, 2017
  53. 12/12 fsmonitor: add a performance testBen Peart, Sep 15, 2017
  54. David TurnerSep 15, 2017
  55. Johannes SchindelinSep 18, 2017
  56. Ben PeartSep 18, 2017
  57. Johannes SchindelinSep 19, 2017
  58. 11/12 fsmonitor: add a sample integration script for WatchmanBen Peart, Sep 15, 2017
  59. 08/12 fsmonitor: add a test tool to dump the index extensionBen Peart, Sep 15, 2017
  60. Junio C HamanoSep 17, 2017
  61. Ben PeartSep 18, 2017
  62. Torsten BögershausenSep 18, 2017
  63. Ben PeartSep 18, 2017
  64. Torsten BögershausenSep 19, 2017
  65. Ben PeartSep 19, 2017
  66. 09/12 split-index: disable the fsmonitor extension when running the split index testBen Peart, Sep 15, 2017
  67. Jonathan NiederSep 19, 2017
  68. Ben PeartSep 20, 2017
  69. Jonathan NiederSep 20, 2017
  70. Ben PeartSep 21, 2017
  71. 03/12 update-index: add a new --force-write-index optionBen Peart, Sep 15, 2017
  72. 02/12 preload-index: add override to enable testing preload-indexBen Peart, Sep 15, 2017
  73. 00/12 Fast git status via a file system watcherBen Peart, Sep 19, 2017
  74. 01/12 bswap: add 64 bit endianness helper get_be64Ben Peart, Sep 19, 2017
  75. 02/12 preload-index: add override to enable testing preload-indexBen Peart, Sep 19, 2017
  76. Stefan BellerSep 20, 2017
  77. Ben PeartSep 21, 2017
  78. Stefan BellerSep 21, 2017
  79. 05/12 fsmonitor: add documentation for the fsmonitor extension.Ben Peart, Sep 19, 2017
  80. Martin ÅgrenSep 20, 2017
  81. Ben PeartSep 20, 2017
  82. Martin ÅgrenSep 20, 2017
  83. 04/12 fsmonitor: teach git to optionally utilize a file system monitor to speed up detecting new or changed files.Ben Peart, Sep 19, 2017
  84. Junio C HamanoSep 20, 2017
  85. Ben PeartSep 20, 2017
  86. Junio C HamanoSep 21, 2017
  87. Ben PeartSep 21, 2017
  88. Ben PeartSep 21, 2017
  89. Junio C HamanoSep 22, 2017
  90. Junio C HamanoSep 20, 2017
  91. Ben PeartSep 20, 2017
  92. 08/12 fsmonitor: add a test tool to dump the index extensionBen Peart, Sep 19, 2017
  93. 09/12 split-index: disable the fsmonitor extension when running the split index testBen Peart, Sep 19, 2017
  94. 07/12 update-index: add fsmonitor support to update-indexBen Peart, Sep 19, 2017
  95. 10/12 fsmonitor: add test cases for fsmonitor extensionBen Peart, Sep 19, 2017
  96. 03/12 update-index: add a new --force-write-index optionBen Peart, Sep 19, 2017
  97. Junio C HamanoSep 20, 2017
  98. Ben PeartSep 20, 2017
  99. Junio C HamanoSep 21, 2017
  100. Ben PeartSep 21, 2017
  101. Junio C HamanoSep 21, 2017
  102. Junio C HamanoSep 21, 2017
  103. 12/12 fsmonitor: add a performance testBen Peart, Sep 19, 2017
  104. 11/12 fsmonitor: add a sample integration script for WatchmanBen Peart, Sep 19, 2017
  105. 06/12 ls-files: Add support in ls-files to display the fsmonitor valid bitBen Peart, Sep 19, 2017
  106. David TurnerSep 19, 2017
  107. Ben PeartSep 19, 2017
  108. David TurnerSep 19, 2017
  109. Ben PeartSep 19, 2017
  110. 00/12 Fast git status via a file system watcherBen Peart, Sep 22, 2017
  111. 02/12 preload-index: add override to enable testing preload-indexBen Peart, Sep 22, 2017
  112. 01/12 bswap: add 64 bit endianness helper get_be64Ben Peart, Sep 22, 2017
  113. Martin ÅgrenSep 22, 2017
  114. Ben PeartSep 23, 2017
  115. Jeff KingSep 24, 2017
  116. Junio C HamanoSep 24, 2017
  117. 03/12 update-index: add a new --force-write-index optionBen Peart, Sep 22, 2017
  118. 11/12 fsmonitor: add a sample integration script for WatchmanBen Peart, Sep 22, 2017
  119. 04/12 fsmonitor: teach git to optionally utilize a file system monitor to speed up detecting new or changed files.Ben Peart, Sep 22, 2017
  120. 10/12 fsmonitor: add test cases for fsmonitor extensionBen Peart, Sep 22, 2017
  121. 09/12 split-index: disable the fsmonitor extension when running the split index testBen Peart, Sep 22, 2017
  122. 12/12 fsmonitor: add a performance testBen Peart, Sep 22, 2017
  123. 06/12 ls-files: Add support in ls-files to display the fsmonitor valid bitBen Peart, Sep 22, 2017
  124. 05/12 fsmonitor: add documentation for the fsmonitor extension.Ben Peart, Sep 22, 2017
  125. 07/12 update-index: add fsmonitor support to update-indexBen Peart, Sep 22, 2017
  126. 08/12 fsmonitor: add a test tool to dump the index extensionBen Peart, Sep 22, 2017
  127. Martin ÅgrenSep 22, 2017
  128. Ben PeartSep 23, 2017
  129. Junio C HamanoSep 24, 2017
  130. Junio C HamanoSep 29, 2017
  131. Ben PeartSep 29, 2017
  132. Junio C HamanoOct 1, 2017
  133. Ben PeartOct 3, 2017
  134. Junio C HamanoOct 4, 2017
  135. Alex VandiverOct 4, 2017
  136. Ben PeartOct 4, 2017
  137. Ben PeartOct 4, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.