git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] Enable index-pack threading in msysgit.

From
SZStefan Zager <szager@chromium.org>
Date
Mar 19, 2014, 19:15 UTC
Message-ID
<CAHOQ7J9sJv-L2xeKOiq1YvG5HhUP_XWCEdrtJogfZW6-NaDmWA@mail.gmail.com>
In-Reply-To
<CAHOQ7J9c_ZfzYEmO861Oa64YZeArQQBMnah1yWAkChME7dA+TA@mail.gmail.com>
On Wed, Mar 19, 2014 at 9:57 AM, Stefan Zager <szager@chromium.org> wrote:
Show 24 quoted lines
>>
>> I still don't understand how compat/pread.c does not work with pack_fd
>> per thread. I don't have Windows to test, but I forced compat/pread.c
>> on on Linux with similar pack_fd changes and it worked fine, helgrind
>> only complained about progress.c.
>>
>> A pread() implementation that is thread-safe with condition sounds
>> like an invite for trouble later. And I don't think converting read()
>> to pread() is a good idea. Platforms that rely on pread() will hit
>> first because of more use of compat/pread.c. read() seeks while
>> pread() does not, so we have to audit more code..
>
> Using one fd per thread is all well and good for something like
> index-pack, which only accesses a single pack file.  But using that
> heuristic to add threading elsewhere is probably not going to work.
> For example, I have a patch in progress to add threading to checkout,
> and another one planned to add threading to status.  In both cases, we
> would need one fd per thread per pack file, which is pretty
> ridiculous.
>
> There really aren't very many calls to read() in the code.  I don't
> think it would be very difficult to eliminate the remaining ones.  The
> more interesting question, I think is: what platforms still don't have
> a thread-safe pread implementation?

I don't want to go too deep down the rabbit hole here. We don't have to solve the read() vs. pread() issue once for all right now; that can wait for another day. The pread() implementation in this patch is certainly no worse than the one in compat/pread.c.

Stefan
Previous: Stefan ZagerNext: Junio C Hamano
Message 6 of 19 in “Enable index-pack threading in msysgit.”
  1. Enable index-pack threading in msysgit.szager@chromium.org, Mar 19, 2014
  2. Duy NguyenMar 19, 2014
  3. Stefan ZagerMar 19, 2014
  4. Duy NguyenMar 19, 2014
  5. Stefan ZagerMar 19, 2014
  6. Stefan ZagerMar 19, 2014
  7. Junio C HamanoMar 19, 2014
  8. Karsten BleesMar 20, 2014
  9. Stefan ZagerMar 20, 2014
  10. Karsten BleesMar 20, 2014
  11. Stefan ZagerMar 20, 2014
  12. Duy NguyenMar 21, 2014
  13. Karsten BleesMar 21, 2014
  14. Duy NguyenMar 21, 2014
  15. Duy NguyenMar 21, 2014
  16. Stefan ZagerMar 21, 2014
  17. Karsten BleesMar 21, 2014
  18. index-pack: work around thread-unsafe pread()Nguyễn Thái Ngọc Duy, Mar 25, 2014
  19. Johannes SixtMar 26, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.