git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 2/2] sparse-index: update index read to consider index.sparse config

From
Derrick Stolee <stolee@gmail.com>
Date
Oct 17, 2021, 19:33 UTC
Message-ID
<8c148f7d-f175-7dc5-51a0-eef48aa98c3d@gmail.com>
In-Reply-To
<xmqq5ytw19ck.fsf@gitster.g>
On 10/17/2021 1:58 AM, Junio C Hamano wrote:
Show 35 quoted lines
> Derrick Stolee <stolee@gmail.com> writes:
> 
>> I think you are slightly mistaken here: If index.sparse=true, then a
>> full index will be converted to one on write, but not immediately upon
>> read. This means that subsequent commands will read a sparse index, and
>> they will either benefit from that or not depending on whether they are
>> integrated with the sparse index or not.
>>
>> The new behavior here is that if index.sparse=false, then we convert
>> a sparse index to a full one upon read, avoiding any chance that a
>> Git command is operating on a sparse index in-memory.
> 
> And if index.sparse=true, then we convert a full on-disk index to a
> sparse one in-core upon reading, right?  My comment was solely on
> that side of the picture, not on the "index.sparse is set to false
> so we automatically expand" case.
> 
>> The simplest summary I can say is here:
>>
>> * If index.sparse=false, then a sparse index will be converted to
>>   full upon read.
>>
>> * If index.sparse=true, then a full index will be converted to sparse
>>   on write.
> 
> Oh, I see, so yes I was very much misunderstanding what you guys are
> trying to do.  I somehow thought that sparse-to-full and
> full-to-sparse conversions (1) already happen on the write codepath,
> and (2) this patch makes them both happen also on the read codepath.
> 
> IOW:
> 
>     * If index.sparse=false, a sparse index will be written as full,
>       and if it is true, a non-sparse index will be written as
>       sparse, even before these patches.
This statement is true.
>     * In addition, with these patches, if index.sparse=false, a
>       sparse index will be expaned to full upon reading, and if it
>       is true, a non-sparse index will be shrunk to sparse upon
>       reading

This is only half true. If "index.sparse=false", then a sparse index will be expanded to full upon reading. If "index.sparse=true" then nothing special will happen to an index on reading.

> What your summary above is saying is very much different.

Yes, these are different things. It also gets around the concern that "if we have a non-integrated command, then index.sparse=true would convert a full index to a sparse one, only to be expanded again". That doesn't happen if we only convert from full to sparse on write.

Thanks, -Stolee

Previous: Junio C HamanoNext: Junio C Hamano
Message 7 of 44 in “sparse-index: expand/collapse based on 'index.sparse'”
  1. 0/2 sparse-index: expand/collapse based on 'index.sparse'Victoria Dye via GitGitGadget, Oct 15, 2021
  2. 1/2 test-read-cache.c: prepare_repo_settings after config initVictoria Dye via GitGitGadget, Oct 15, 2021
  3. 2/2 sparse-index: update index read to consider index.sparse configVictoria Dye via GitGitGadget, Oct 15, 2021
  4. Junio C HamanoOct 15, 2021
  5. Derrick StoleeOct 17, 2021
  6. Junio C HamanoOct 17, 2021
  7. Derrick StoleeOct 17, 2021
  8. Junio C HamanoOct 18, 2021
  9. Derrick StoleeOct 18, 2021
  10. Victoria DyeOct 18, 2021
  11. Derrick StoleeOct 21, 2021
  12. Junio C HamanoOct 18, 2021
  13. 0/3 sparse-index: expand/collapse based on 'index.sparse'Victoria Dye via GitGitGadget, Oct 21, 2021
  14. 1/3 test-read-cache.c: prepare_repo_settings after config initVictoria Dye via GitGitGadget, Oct 21, 2021
  15. 3/3 sparse-index: update do_read_index to ensure correct sparsityVictoria Dye via GitGitGadget, Oct 21, 2021
  16. 2/3 sparse-index: add ensure_correct_sparsity functionVictoria Dye via GitGitGadget, Oct 21, 2021
  17. Junio C HamanoOct 21, 2021
  18. Victoria DyeOct 27, 2021
  19. 0/3 sparse-index: expand/collapse based on 'index.sparse'Victoria Dye via GitGitGadget, Oct 27, 2021
  20. 1/3 test-read-cache.c: prepare_repo_settings after config initVictoria Dye via GitGitGadget, Oct 27, 2021
  21. 2/3 sparse-index: add ensure_correct_sparsity functionVictoria Dye via GitGitGadget, Oct 27, 2021
  22. Derrick StoleeOct 27, 2021
  23. Junio C HamanoOct 27, 2021
  24. Derrick StoleeOct 28, 2021
  25. Victoria DyeOct 29, 2021
  26. 3/3 sparse-index: update do_read_index to ensure correct sparsityVictoria Dye via GitGitGadget, Oct 27, 2021
  27. Derrick StoleeOct 27, 2021
  28. 0/4 sparse-index: expand/collapse based on 'index.sparse'Victoria Dye via GitGitGadget, Oct 29, 2021
  29. 1/4 test-read-cache.c: prepare_repo_settings after config initVictoria Dye via GitGitGadget, Oct 29, 2021
  30. 2/4 sparse-index: avoid unnecessary cache tree clearingVictoria Dye via GitGitGadget, Oct 29, 2021
  31. Junio C HamanoOct 29, 2021
  32. Derrick StoleeOct 29, 2021
  33. Junio C HamanoOct 29, 2021
  34. 4/4 sparse-index: update do_read_index to ensure correct sparsityVictoria Dye via GitGitGadget, Oct 29, 2021
  35. Elijah NewrenNov 22, 2021
  36. Victoria DyeNov 22, 2021
  37. 3/4 sparse-index: add ensure_correct_sparsity functionVictoria Dye via GitGitGadget, Oct 29, 2021
  38. Elijah NewrenNov 22, 2021
  39. 0/4 sparse-index: expand/collapse based on 'index.sparse'Victoria Dye via GitGitGadget, Nov 23, 2021
  40. 1/4 test-read-cache.c: prepare_repo_settings after config initVictoria Dye via GitGitGadget, Nov 23, 2021
  41. 2/4 sparse-index: avoid unnecessary cache tree clearingVictoria Dye via GitGitGadget, Nov 23, 2021
  42. 3/4 sparse-index: add ensure_correct_sparsity functionVictoria Dye via GitGitGadget, Nov 23, 2021
  43. 4/4 sparse-index: update do_read_index to ensure correct sparsityVictoria Dye via GitGitGadget, Nov 23, 2021
  44. Elijah NewrenNov 23, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.