{"thread":{"id":"55168","subject":"[PATCH] read-cache: make the index write buffer size 128K","startedAt":"2021-02-18T02:49:26Z","lastAt":"2021-02-25T07:58:35Z","messageCount":11,"participants":["Neeraj K. Singh via GitGitGadget","Jeff Hostetler","Junio C Hamano","Neeraj Singh","Chris Torek"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"417269","messageId":"pull.877.git.1613616506949.gitgitgadget@gmail.com","threadId":"55168","inReplyTo":null,"subject":"[PATCH] read-cache: make the index write buffer size 128K","fromName":"Neeraj K. Singh via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-18T02:48:26Z","receivedAt":"2021-02-18T02:49:26Z","isPatch":true,"sender":{"key":"name:Neeraj K. Singh","avatar":null},"body":"From: Neeraj Singh \n\nWriting an index 8K at a time invokes the OS filesystem and caching code\nvery frequently, introducing noticeable overhead while writing large\nindexes. When experimenting with different write buffer sizes on Windows\nwriting the Windows OS repo index (260MB), most of the benefit came by\nbumping the index write buffer size to 64K. I picked 128K to ensure that\nwe're past the knee of the curve.\n\nWith this change, the time under do_write_index for an index with 3M\nfiles goes from ~1.02s to ~0.72s.\n\nSigned-off-by: Neeraj Singh \n---\n read-cache: make the index write buffer size 128K\n \n Writing an index 8K at a time invokes the OS filesystem and caching code\n very frequently, introducing noticeable overhead while writing large\n indexes. When experimenting with different write buffer sizes on Windows\n writing the Windows OS repo index (260MB), most of the benefit came by\n bumping the index write buffer size to 64K. I picked 128K to ensure that\n we're past the knee of the curve.\n \n With this change, the time under do_write_index for an index with 3M\n files goes from ~1.02s to ~0.72s.\n \n Signed-off-by: Neeraj Singh neerajsi@ntdev.microsoft.com\n \n Note: This was previously discussed on the mailing list in 2016 at:\n https://lore.kernel.org/git/1458350341-12276-1-git-send-email-dturner@twopensource.com/.\n \n Since then, I believe we have a couple changes:\n \n * 'small' development platforms like raspberry pi have gotten larger\n (4GB RAM).\n * spectre and meltdown make individual system calls more expensive when\n mitigations are enabled\n * there have been many investments to make very large repos scale well\n in git, so huge repos are more common now.\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-877%2Fneerajsi-msft%2Fneerajsi%2Findex-buffer-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-877/neerajsi-msft/neerajsi/index-buffer-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/877\n\n read-cache.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/read-cache.c b/read-cache.c\nindex 29144cf879e7..a5b2779b9586 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -2447,7 +2447,7 @@ int repo_index_has_changes(struct repository *repo,\n \t}\n }\n \n-#define WRITE_BUFFER_SIZE 8192\n+#define WRITE_BUFFER_SIZE (128 * 1024)\n static unsigned char write_buffer[WRITE_BUFFER_SIZE];\n static unsigned long write_buffer_len;\n \n\nbase-commit: 45526154a57d15947cad7262230d0b935cedb9d3\n-- \ngitgitgadget\n"},{"id":"417417","messageId":"f52df30b-4ab0-fd6f-17f8-70daed81df39@jeffhostetler.com","threadId":"55168","inReplyTo":"pull.877.git.1613616506949.gitgitgadget@gmail.com","subject":"Re: [PATCH] read-cache: make the index write buffer size 128K","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2021-02-19T19:12:42Z","receivedAt":"2021-02-19T19:13:28Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 2/17/21 9:48 PM, Neeraj K. Singh via GitGitGadget wrote:\n> From: Neeraj Singh \n> \n> Writing an index 8K at a time invokes the OS filesystem and caching code\n> very frequently, introducing noticeable overhead while writing large\n> indexes. When experimenting with different write buffer sizes on Windows\n> writing the Windows OS repo index (260MB), most of the benefit came by\n> bumping the index write buffer size to 64K. I picked 128K to ensure that\n> we're past the knee of the curve.\n> \n> With this change, the time under do_write_index for an index with 3M\n> files goes from ~1.02s to ~0.72s.\n\n[...]\n\n> \n> -#define WRITE_BUFFER_SIZE 8192\n> +#define WRITE_BUFFER_SIZE (128 * 1024)\n> static unsigned char write_buffer[WRITE_BUFFER_SIZE];\n> static unsigned long write_buffer_len;\n\n[...]\n\nVery nice.\n\nI can confirm that this gives nice gains on Windows. (I'm using\nthe Office repo which has a 188MB index file (2.1M files at HEAD).\nRunning \"git status\" shows a gain of about 200ms.\n\nWe get a smaller gain on Mac of about 50ms (again, using the Office\nrepo).\n\nSo, you may add my sign-off or ACK to this.\n Signed-off-by: Jeff Hostetler \n\n\n\nFWIW, You might take a look at `t/perf/p0007-write-cache.sh`\nUpdate it as follows:\n\n```\ndiff --git a/t/perf/p0007-write-cache.sh b/t/perf/p0007-write-cache.sh\nindex 09595264f0..337280ff1c 100755\n--- a/t/perf/p0007-write-cache.sh\n+++ b/t/perf/p0007-write-cache.sh\n@@ -4,7 +4,8 @@ test_description=\"Tests performance of writing the index\"\n\n . ./perf-lib.sh\n\n-test_perf_default_repo\n+test_perf_large_repo\n\n test_expect_success \"setup repo\" '\n if git rev-parse --verify refs/heads/p0006-ballast^{commit}\n```\n\n\nThen you can run it like this:\n\n $ cd t/perf\n $ GIT_PERF_LARGE_REPO=/path/to/your/enlistment ./p0007-write-cache\n\nThen you can run it with the small and then with the large buffer and\nget times for essentially just the index write in isolation.\n\nHope this helps,\nJeff\n"},{"id":"417427","messageId":"xmqqv9ana05b.fsf@gitster.g","threadId":"55168","inReplyTo":"f52df30b-4ab0-fd6f-17f8-70daed81df39@jeffhostetler.com","subject":"Re: [PATCH] read-cache: make the index write buffer size 128K","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-20T03:28:00Z","receivedAt":"2021-02-20T03:29:04Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff Hostetler writes:\n\n> On 2/17/21 9:48 PM, Neeraj K. Singh via GitGitGadget wrote:\n>> From: Neeraj Singh \n>> Writing an index 8K at a time invokes the OS filesystem and caching\n>> code\n>> very frequently, introducing noticeable overhead while writing large\n>> indexes. When experimenting with different write buffer sizes on Windows\n>> writing the Windows OS repo index (260MB), most of the benefit came by\n>> bumping the index write buffer size to 64K. I picked 128K to ensure that\n>> we're past the knee of the curve.\n>> With this change, the time under do_write_index for an index with 3M\n>> files goes from ~1.02s to ~0.72s.\n>\n> [...]\n>\n>> -#define WRITE_BUFFER_SIZE 8192\n>> +#define WRITE_BUFFER_SIZE (128 * 1024)\n>> static unsigned char write_buffer[WRITE_BUFFER_SIZE];\n>> static unsigned long write_buffer_len;\n>\n> [...]\n>\n> Very nice.\n\nI wonder if we gain more by going say 4M buffer size or even larger?\n\nIs this something we can make the system auto-tune itself? This is\nnot about reading but writing, so we already have enough information\nto estimate how much we would need to write out.\n\nThanks.\n"},{"id":"417436","messageId":"CANQDOdeEd=JjWL4gb5CWHL_HkvJMnFumW74ew4DXJahh4BKvfQ@mail.gmail.com","threadId":"55168","inReplyTo":"xmqqv9ana05b.fsf@gitster.g","subject":"Re: [PATCH] read-cache: make the index write buffer size 128K","fromName":"Neeraj Singh","fromEmail":"nksingh85@gmail.com","sentAt":"2021-02-20T07:56:54Z","receivedAt":"2021-02-20T07:58:05Z","isPatch":true,"sender":{"key":"nksingh85@gmail.com","avatar":null},"body":"On Fri, Feb 19, 2021 at 11:46 PM Junio C Hamano wrote:\n>\n> Jeff Hostetler writes:\n>\n> > On 2/17/21 9:48 PM, Neeraj K. Singh via GitGitGadget wrote:\n> >> From: Neeraj Singh \n> >> Writing an index 8K at a time invokes the OS filesystem and caching\n> >> code\n> >> very frequently, introducing noticeable overhead while writing large\n> >> indexes. When experimenting with different write buffer sizes on Windows\n> >> writing the Windows OS repo index (260MB), most of the benefit came by\n> >> bumping the index write buffer size to 64K. I picked 128K to ensure that\n> >> we're past the knee of the curve.\n> >> With this change, the time under do_write_index for an index with 3M\n> >> files goes from ~1.02s to ~0.72s.\n> >\n> > [...]\n> >\n> >> -#define WRITE_BUFFER_SIZE 8192\n> >> +#define WRITE_BUFFER_SIZE (128 * 1024)\n> >> static unsigned char write_buffer[WRITE_BUFFER_SIZE];\n> >> static unsigned long write_buffer_len;\n> >\n> > [...]\n> >\n> > Very nice.\n>\n> I wonder if we gain more by going say 4M buffer size or even larger?\n>\n> Is this something we can make the system auto-tune itself? This is\n> not about reading but writing, so we already have enough information\n> to estimate how much we would need to write out.\n>\n> Thanks.\n>\n\nHi Junio,\nAt some point the cost of the memcpy into the filesystem cache begins to\ndominate the cost of the system call, so increasing the buffer size\nhas diminishing returns.\n\nAn alternate approach would be to mmap the index file we are trying to\nwrite and thereby\ncopy the data directly into the filesystem cache pages. That's a much\nmore difficult change to\nmake and verify, so I'd rather leave that as an exercise to the reader\nfor now :).\n\nThanks,\n-Neeraj\n"},{"id":"417454","messageId":"xmqqo8gd8tyr.fsf@gitster.g","threadId":"55168","inReplyTo":"CANQDOdeEd=JjWL4gb5CWHL_HkvJMnFumW74ew4DXJahh4BKvfQ@mail.gmail.com","subject":"Re: [PATCH] read-cache: make the index write buffer size 128K","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-21T12:51:24Z","receivedAt":"2021-02-21T12:52:25Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Neeraj Singh writes:\n\n>> >> -#define WRITE_BUFFER_SIZE 8192\n>> >> +#define WRITE_BUFFER_SIZE (128 * 1024)\n>> >> static unsigned char write_buffer[WRITE_BUFFER_SIZE];\n>> >> static unsigned long write_buffer_len;\n>> >\n>> > [...]\n>> >\n>> > Very nice.\n>>\n>> I wonder if we gain more by going say 4M buffer size or even larger?\n>>\n>> Is this something we can make the system auto-tune itself? This is\n>> not about reading but writing, so we already have enough information\n>> to estimate how much we would need to write out.\n>>\n>> Thanks.\n>>\n>\n> Hi Junio,\n> At some point the cost of the memcpy into the filesystem cache begins to\n> dominate the cost of the system call, so increasing the buffer size\n> has diminishing returns.\n\nYes, I know that kind of \"general principle\". \n\nIf I recall correctly, we used to pass too large a buffer to a\nsingle write(2) system call (I do not know if it was for the\nindex---I suspect it was for some other data), and found out that it\nmade response to ^C take too long, and tuned the buffer size down.\n\nI was asking where the sweet spot for this codepath would be, and if\nwe can take a measurement to make a better decision than \"8k feels\ntoo small and 128k turns out to be better than 8k\". It does not\ntell us if 128k would always do better than 64k or 256k, for\nexample.\n\nI suspect that the sweet spot would be dependent on many parameters\n(not just the operating system, but also relative speed among\nmemory, \"disk\", and cpu, and also the size of the index) and if we\ncan devise a way to auto-tune it so that we do not have to worry\nabout it.\n\nThanks.\n"},{"id":"417760","messageId":"CANQDOdfJApBOEm2gPMwtz9T0ETPoDk107mF7LYRGCmjFLi3Jxg@mail.gmail.com","threadId":"55168","inReplyTo":"xmqqo8gd8tyr.fsf@gitster.g","subject":"Re: [PATCH] read-cache: make the index write buffer size 128K","fromName":"Neeraj Singh","fromEmail":"nksingh85@gmail.com","sentAt":"2021-02-24T20:56:46Z","receivedAt":"2021-02-24T20:57:53Z","isPatch":true,"sender":{"key":"nksingh85@gmail.com","avatar":null},"body":"On Sun, Feb 21, 2021 at 4:51 AM Junio C Hamano wrote:\n>\n> Neeraj Singh writes:\n>\n> >> >> -#define WRITE_BUFFER_SIZE 8192\n> >> >> +#define WRITE_BUFFER_SIZE (128 * 1024)\n> >> >> static unsigned char write_buffer[WRITE_BUFFER_SIZE];\n> >> >> static unsigned long write_buffer_len;\n> >> >\n> >> > [...]\n> >> >\n> >> > Very nice.\n> >>\n> >> I wonder if we gain more by going say 4M buffer size or even larger?\n> >>\n> >> Is this something we can make the system auto-tune itself? This is\n> >> not about reading but writing, so we already have enough information\n> >> to estimate how much we would need to write out.\n> >>\n> >> Thanks.\n> >>\n> >\n> > Hi Junio,\n> > At some point the cost of the memcpy into the filesystem cache begins to\n> > dominate the cost of the system call, so increasing the buffer size\n> > has diminishing returns.\n>\n> Yes, I know that kind of \"general principle\".\n>\n> If I recall correctly, we used to pass too large a buffer to a\n> single write(2) system call (I do not know if it was for the\n> index---I suspect it was for some other data), and found out that it\n> made response to ^C take too long, and tuned the buffer size down.\n>\n> I was asking where the sweet spot for this codepath would be, and if\n> we can take a measurement to make a better decision than \"8k feels\n> too small and 128k turns out to be better than 8k\". It does not\n> tell us if 128k would always do better than 64k or 256k, for\n> example.\n>\n> I suspect that the sweet spot would be dependent on many parameters\n> (not just the operating system, but also relative speed among\n> memory, \"disk\", and cpu, and also the size of the index) and if we\n> can devise a way to auto-tune it so that we do not have to worry\n> about it.\n>\n> Thanks.\n\nI think the main concern on a reasonably-configured machine is the speed\nof memcpy and the cost of the code to get to that memcpy (syscall, file system\nfree space allocator, page allocator, mapping from file offset to cache page).\nDisk shouldn't matter, since we write the file with OS buffering and\nbuffer flushing\nwill happen asynchronously some time after the git command completes.\n\nIf we think about doing the fastest possible memcpy, I think we want to aim for\nmaximizing the use of the CPU cache. A write buffer that's too big would result\nin most of the data being flushed to DRAM between when git writes it and the\nOS reads it. L1 caches are typically ~32K and L2 caches are on the\norder of 256K.\nWe probably don't want to exceed the size of the L2 cache, and we\nshould actually\nleave some room for OS code and data, so 128K is a good number from\nthat perspective.\n\nI collected data from an experiment with different buffer sizes on Windows on my\n3.6Ghz Xeon W-2133 machine:\nhttps://docs.google.com/spreadsheets/d/1Bu6pjp53NPDK6AKQI_cry-hgxEqlicv27dptoXZYnwc/edit?usp=sharing\n\nThe timing is pretty much in the noise after we pass 32K. So I think\n8K is too small, but\ngiven the flatness of the curve we can feel good about any value above\n32K from a performance\nperspective. I still think 128K is a decent number that won't likely\nneed to be changed for\nsome time.\n\nThanks,\n-Neeraj\n"},{"id":"417787","messageId":"xmqqmtvswvp7.fsf@gitster.g","threadId":"55168","inReplyTo":"CANQDOdfJApBOEm2gPMwtz9T0ETPoDk107mF7LYRGCmjFLi3Jxg@mail.gmail.com","subject":"Re: [PATCH] read-cache: make the index write buffer size 128K","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-25T05:41:24Z","receivedAt":"2021-02-25T05:42:25Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Neeraj Singh writes:\n\n> If we think about doing the fastest possible memcpy, I think we want to aim for\n> maximizing the use of the CPU cache. A write buffer that's too big would result\n> in most of the data being flushed to DRAM between when git writes it and the\n> OS reads it. L1 caches are typically ~32K and L2 caches are on the\n> order of 256K.\n> We probably don't want to exceed the size of the L2 cache, and we\n> should actually\n> leave some room for OS code and data, so 128K is a good number from\n> that perspective.\n>\n> I collected data from an experiment with different buffer sizes on Windows on my\n> 3.6Ghz Xeon W-2133 machine:\n> https://docs.google.com/spreadsheets/d/1Bu6pjp53NPDK6AKQI_cry-hgxEqlicv27dptoXZYnwc/edit?usp=sharing\n>\n> The timing is pretty much in the noise after we pass 32K. So I think\n> 8K is too small, but\n> given the flatness of the curve we can feel good about any value above\n> 32K from a performance\n> perspective. I still think 128K is a decent number that won't likely\n> need to be changed for\n> some time.\n\nThanks for a supporting graph.\n\nI can very well imagine that it would have been tempting to instead\nsay \"after we pass 128k\" while explaining exactly the same graph,\nand doing so would have given a more coherent argument to support\nthe choice of 128k the patch made. You knew that a \"then perhaps we\ncan reclaim 96k by sizing the buffer down a bit?\" would become a\nreasonable response, but you still chose to be honest, which I kinda\nlike ;-)\n\n\n\n"},{"id":"417793","messageId":"CAPx1GvdA1prtO+y-bJ7yu8oZP6Lp9mHQ5gv-fXvS193NFospkA@mail.gmail.com","threadId":"55168","inReplyTo":"xmqqmtvswvp7.fsf@gitster.g","subject":"Re: [PATCH] read-cache: make the index write buffer size 128K","fromName":"Chris Torek","fromEmail":"chris.torek@gmail.com","sentAt":"2021-02-25T06:58:36Z","receivedAt":"2021-02-25T06:59:47Z","isPatch":true,"sender":{"key":"chris.torek@gmail.com","avatar":"https://avatars.githubusercontent.com/u/16826774?v=4"},"body":"> Neeraj Singh writes:\n> > I collected data from an experiment with different buffer sizes on Windows on my\n> > 3.6Ghz Xeon W-2133 machine:\n> > https://docs.google.com/spreadsheets/d/1Bu6pjp53NPDK6AKQI_cry-hgxEqlicv27dptoXZYnwc/edit?usp=sharing\n> >\n> > The timing is pretty much in the noise after we pass 32K. So I think\n> > 8K is too small, but\n> > given the flatness of the curve we can feel good about any value above\n> > 32K from a performance\n> > perspective. I still think 128K is a decent number that won't likely\n> > need to be changed for\n> > some time.\n\nLinux/BSD/etc `stat` system calls report st_blksize values to tell\nuser code the optimal size for read and write calls. Does Windows\nhave one? (It's not POSIX but is XSI.)\n\n(How *well* the OS reports `st_blksize` is another question\nentirely, but at least if the report says, say, 128k, and that's\nwrong, that's no longer Git's fault. :-) )\n\nOn Wed, Feb 24, 2021 at 10:46 PM Junio C Hamano wrote:\n> Thanks for a supporting graph.\n>\n> I can very well imagine that it would have been tempting to instead\n> say \"after we pass 128k\" while explaining exactly the same graph,\n> and doing so would have given a more coherent argument to support\n> the choice of 128k the patch made. You knew that a \"then perhaps we\n> can reclaim 96k by sizing the buffer down a bit?\" would become a\n> reasonable response, but you still chose to be honest, which I kinda\n> like ;-)\n\n128K is correct for ZFS; 64K is typically correct for UFS2; 8K is\nthe old UFS1 size. Anything under that has been too small for\na long time. :-)\n\nChris\n"},{"id":"417797","messageId":"xmqqeeh4wrad.fsf@gitster.g","threadId":"55168","inReplyTo":"CAPx1GvdA1prtO+y-bJ7yu8oZP6Lp9mHQ5gv-fXvS193NFospkA@mail.gmail.com","subject":"Re: [PATCH] read-cache: make the index write buffer size 128K","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-25T07:16:42Z","receivedAt":"2021-02-25T07:18:10Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Chris Torek writes:\n\n> Linux/BSD/etc `stat` system calls report st_blksize values to tell\n> user code the optimal size for read and write calls. Does Windows\n> have one? (It's not POSIX but is XSI.)\n>\n> (How *well* the OS reports `st_blksize` is another question\n> entirely, but at least if the report says, say, 128k, and that's\n> wrong, that's no longer Git's fault. :-) )\n> ...\n> 128K is correct for ZFS; 64K is typically correct for UFS2; 8K is\n> the old UFS1 size. Anything under that has been too small for\n> a long time. :-)\n\nThat's rather tempting. After opening a locked index to write\nthings out, the value is a single fstat() away...\n\n"},{"id":"417801","messageId":"CANQDOdd+keN3LyC4CHWDZ3JHYurEy_FLyw5GT5UKqg0RcTA+DA@mail.gmail.com","threadId":"55168","inReplyTo":"xmqqeeh4wrad.fsf@gitster.g","subject":"Re: [PATCH] read-cache: make the index write buffer size 128K","fromName":"Neeraj Singh","fromEmail":"nksingh85@gmail.com","sentAt":"2021-02-25T07:36:20Z","receivedAt":"2021-02-25T07:46:14Z","isPatch":true,"sender":{"key":"nksingh85@gmail.com","avatar":null},"body":"On Wed, Feb 24, 2021 at 11:16 PM Junio C Hamano wrote:\n>\n> Chris Torek writes:\n>\n> > Linux/BSD/etc `stat` system calls report st_blksize values to tell\n> > user code the optimal size for read and write calls. Does Windows\n> > have one? (It's not POSIX but is XSI.)\n> >\n> > (How *well* the OS reports `st_blksize` is another question\n> > entirely, but at least if the report says, say, 128k, and that's\n> > wrong, that's no longer Git's fault. :-) )\n> > ...\n> > 128K is correct for ZFS; 64K is typically correct for UFS2; 8K is\n> > the old UFS1 size. Anything under that has been too small for\n> > a long time. :-)\n>\n> That's rather tempting. After opening a locked index to write\n> things out, the value is a single fstat() away...\n>\n\nFrom a quick perusal of freebsd, st_blksize seems to be the system\nPAGE_SIZE by default (4k most of the time, I assume). The Windows\nequivalent of this value is really tuned to what you want to send down\nwhen bypassing the cache (to avoid partial cluster/stripe writes).\n\nhttps://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sys_stat.h.html\ndoesn't elicit much confidence. The units of st_blksize aren't even\ndefined.\n\nThanks,\nNeeraj\n"},{"id":"417804","messageId":"CAPx1GvcFMzrFFmwMvDLofj73UcEi6T3u_v+usvENkU2yYGLoaQ@mail.gmail.com","threadId":"55168","inReplyTo":"CANQDOdd+keN3LyC4CHWDZ3JHYurEy_FLyw5GT5UKqg0RcTA+DA@mail.gmail.com","subject":"Re: [PATCH] read-cache: make the index write buffer size 128K","fromName":"Chris Torek","fromEmail":"chris.torek@gmail.com","sentAt":"2021-02-25T07:57:36Z","receivedAt":"2021-02-25T07:58:35Z","isPatch":true,"sender":{"key":"chris.torek@gmail.com","avatar":"https://avatars.githubusercontent.com/u/16826774?v=4"},"body":"On Wed, Feb 24, 2021 at 11:36 PM Neeraj Singh wrote:\n> From a quick perusal of freebsd, st_blksize seems to be the system\n> PAGE_SIZE by default (4k most of the time, I assume). The Windows\n> equivalent of this value is really tuned to what you want to send down\n> when bypassing the cache (to avoid partial cluster/stripe writes).\n\nIt's page-size for pipes, sockets, etc., but for real files, it's based on\na report from the underlying file system. It's actually 8k on a typical\nancient UFS file system, 64K on UFS2, and 128K on ZFS, on FreeBSD.\n\n\n> https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sys_stat.h.html\n> doesn't elicit much confidence. The units of st_blksize aren't even\n> defined.\n\nDespite POSIX's rather obstreperous definition of st_blksize, the\nunits are actually just bytes, in practice.\n\nChris\n"}]}