git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] read-cache: make the index write buffer size 128K

From
Junio C Hamano <gitster@pobox.com>
Date
Feb 25, 2021, 05:41 UTC
Message-ID
<xmqqmtvswvp7.fsf@gitster.g>
In-Reply-To
<CANQDOdfJApBOEm2gPMwtz9T0ETPoDk107mF7LYRGCmjFLi3Jxg@mail.gmail.com>
Neeraj Singh <nksingh85@gmail.com> writes:
Show 21 quoted lines
> If we think about doing the fastest possible memcpy, I think we want to aim for
> maximizing the use of the CPU cache.  A write buffer that's too big would result
> in most of the data being flushed to DRAM between when git writes it and the
> OS reads it.  L1 caches are typically ~32K and L2 caches are on the
> order of 256K.
> We probably don't want to exceed the size of the L2 cache, and we
> should actually
> leave some room for OS code and data, so 128K is a good number from
> that perspective.
>
> I collected data from an experiment with different buffer sizes on Windows on my
> 3.6Ghz Xeon W-2133 machine:
> https://docs.google.com/spreadsheets/d/1Bu6pjp53NPDK6AKQI_cry-hgxEqlicv27dptoXZYnwc/edit?usp=sharing
>
> The timing is pretty much in the noise after we pass 32K.  So I think
> 8K is too small, but
> given the flatness of the curve we can feel good about any value above
> 32K from a performance
> perspective.  I still think 128K is a decent number that won't likely
> need to be changed for
> some time.
Thanks for a supporting graph.

I can very well imagine that it would have been tempting to instead say "after we pass 128k" while explaining exactly the same graph, and doing so would have given a more coherent argument to support the choice of 128k the patch made. You knew that a "then perhaps we can reclaim 96k by sizing the buffer down a bit?" would become a reasonable response, but you still chose to be honest, which I kinda like ;-)

Previous: Neeraj SinghNext: Chris Torek
Message 7 of 11 in “read-cache: make the index write buffer size 128K”
  1. read-cache: make the index write buffer size 128KNeeraj K. Singh via GitGitGadget, Feb 18, 2021
  2. Jeff HostetlerFeb 19, 2021
  3. Junio C HamanoFeb 20, 2021
  4. Neeraj SinghFeb 20, 2021
  5. Junio C HamanoFeb 21, 2021
  6. Neeraj SinghFeb 24, 2021
  7. Junio C HamanoFeb 25, 2021
  8. Chris TorekFeb 25, 2021
  9. Junio C HamanoFeb 25, 2021
  10. Neeraj SinghFeb 25, 2021
  11. Chris TorekFeb 25, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.