git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 3/4] read-cache: use hashfile instead of git_hash_ctx

From
Junio C Hamano <gitster@pobox.com>
Date
May 17, 2021, 22:13 UTC
Message-ID
<xmqqfsyl57q2.fsf@gitster.g>
In-Reply-To
<b94172ccf5e91fe59a1d32774dbec23e624f1135.1621254292.git.gitgitgadget@gmail.com>
"Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:
> ...
> mult-pack-indexes, and commit-graphs. Therefore, it seems prudent to
multi-pack, I would say.
> There are still some remaining: the extension headers are hashed for use

some remaining what? I first read an unwritten word as "issues", but I think the answer is "uses of git_hash_ctx".

> in the End of Index Entries (EOIE) extension. This use of the
> git_hash_ctx is left as-is. There are multiple reasons to not use a
> hashfile here, including ...
Show 7 quoted lines
> In addition to the test suite passing, I computed indexes using the
> previous binaries and the binaries compiled after this change, and found
> the index data to be exactly equal. Finally, I did extensive performance
> testing of "git update-index --force-write" on repos of various sizes,
> including one with over 2 million paths at HEAD. These tests
> demonstrated less than 1% difference in behavior, so the performance
> should be considered identical.

Hmph, does that mean 128k buffer is overkill and if we wanted to unify the buffer sizes we should have used 8k instead?

Wait, the removal of fsync has made things faster in general, hasn't it? Did something else degrade performance to cancel that gain?

The patch looks an obvious improvement. What was open-coded in longhand is now a well structured series of API calls and the result is much easier to follow and maintain.

Thanks.
Previous: Derrick Stolee via GitGitGadgetNext: Derrick Stolee
Message 11 of 27 in “Convert index writes to use hashfile API”
  1. 0/3 Convert index writes to use hashfile APIDerrick Stolee via GitGitGadget, Mar 26, 2021
  2. 3/3 read-cache: delete unused hashing methodsDerrick Stolee via GitGitGadget, Mar 26, 2021
  3. 1/3 csum-file: add nested_hashfile()Derrick Stolee via GitGitGadget, Mar 26, 2021
  4. 2/3 read-cache: use hashfile instead of git_hash_ctxDerrick Stolee via GitGitGadget, Mar 26, 2021
  5. Derrick StoleeMar 29, 2021
  6. Derrick StoleeMar 29, 2021
  7. Derrick StoleeMar 26, 2021
  8. 0/4 Convert index writes to use hashfile APIDerrick Stolee via GitGitGadget, May 17, 2021
  9. 1/4 hashfile: use write_in_full()Derrick Stolee via GitGitGadget, May 17, 2021
  10. 3/4 read-cache: use hashfile instead of git_hash_ctxDerrick Stolee via GitGitGadget, May 17, 2021
  11. Junio C HamanoMay 17, 2021
  12. Derrick StoleeMay 18, 2021
  13. 2/4 csum-file.h: increase hashfile buffer sizeDerrick Stolee via GitGitGadget, May 17, 2021
  14. Junio C HamanoMay 17, 2021
  15. Jeff KingMay 18, 2021
  16. Derrick StoleeMay 18, 2021
  17. Jeff KingMay 18, 2021
  18. Jeff KingMay 18, 2021
  19. 4/4 read-cache: delete unused hashing methodsDerrick Stolee via GitGitGadget, May 17, 2021
  20. 0/4 Convert index writes to use hashfile APIDerrick Stolee via GitGitGadget, May 18, 2021
  21. 2/4 csum-file.h: increase hashfile buffer sizeDerrick Stolee via GitGitGadget, May 18, 2021
  22. t4216-log-bloom.sh fails with -v (but not --verbose-log)Ævar Arnfjörð Bjarmason, Nov 25, 2021
  23. Jeff KingNov 26, 2021
  24. Derrick StoleeNov 29, 2021
  25. 1/4 hashfile: use write_in_full()Derrick Stolee via GitGitGadget, May 18, 2021
  26. 3/4 read-cache: use hashfile instead of git_hash_ctxDerrick Stolee via GitGitGadget, May 18, 2021
  27. 4/4 read-cache: delete unused hashing methodsDerrick Stolee via GitGitGadget, May 18, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.