git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 0/3] reftable/stack: use geometric table compaction

From
Patrick Steinhardt <ps@pks.im>
Date
Apr 3, 2024, 10:18 UTC
Message-ID
<Zg0tEa7DDEuub09Q@tanuki>
In-Reply-To
<CAOw_e7az+S4nkG1HKDcqXkK0BOhs6cBpkb=QEGBoZW8MSEEvGg@mail.gmail.com>
On Wed, Apr 03, 2024 at 12:13:42PM +0200, Han-Wen Nienhuys wrote:
Show 28 quoted lines
> On Fri, Mar 22, 2024 at 10:51 AM Patrick Steinhardt <ps@pks.im> wrote:
> > > Thanks for taking a look!
> >
> > Cc'ing Han-Wen and Josh for additional input. From my point of view the
> > new algorithm is simpler to understand and less fragile, but I do wonder
> > whether there is anything that we're missing.
> 
> Good spotting. I hadn't thought about alternating tables.
> 
> I have one minor criticism:
> 
> Environment variables are untyped global variables without any form of
> data protection, so I find them unsavoury, and have tried to avoid
> them throughout. (The whole reftable library only looks at $TMPDIR in
> tests). They're also accessible to end users, so it can become a
> feature that can inadvertently become a maintenance burden.
> 
> For testing, there is a stack->disable_auto_compact.
> 
> If you want to keep that style, I would elevate disable_auto_compact
> into reftable_write_options to make it API surface. This will let you
> use it in tests written in C, which can be unittests and therefore
> more precise and fine-grained. They also run more quickly, and are
> easier to instrument with asan/valgrind/etc. The test for tables with
> alternating sizes can be easily written in C.
> 
> If you really need it, you could initialize disable_auto_compact from
> the environment, but I would suggest avoiding it if possible.

That's actually a good point. I think keeping this as an environment variable isn't too bad as a stop-gap measure for now, and it should be obvious to users that it's not for general use due to the `GIT_TEST` prefix.

But I'm definitely supportive of lifting it out of the reftable library and into the reftable backend so that it is specific to Git, not to the reftable library.

Patrick
Previous: Han-Wen NienhuysNext: Justin Tobler
Message 14 of 52 in “reftable/stack: use geometric table compaction”
  1. reftable/stack: use geometric table compactionJustin Tobler via GitGitGadget, Mar 5, 2024
  2. Patrick SteinhardtMar 6, 2024
  3. Patrick SteinhardtMar 6, 2024
  4. Justin ToblerMar 21, 2024
  5. 0/3 reftable/stack: use geometric table compactionJustin Tobler via GitGitGadget, Mar 21, 2024
  6. 1/3 reftable/stack: add env to disable autocompactionJustin Tobler via GitGitGadget, Mar 21, 2024
  7. Patrick SteinhardtMar 22, 2024
  8. 2/3 reftable/stack: use geometric table compactionJustin Tobler via GitGitGadget, Mar 21, 2024
  9. Patrick SteinhardtMar 22, 2024
  10. Karthik NayakMar 27, 2024
  11. 3/3 reftable/segment: make segment end inclusiveJustin Tobler via GitGitGadget, Mar 21, 2024
  12. Patrick SteinhardtMar 22, 2024
  13. Han-Wen NienhuysApr 3, 2024
  14. Patrick SteinhardtApr 3, 2024
  15. Justin ToblerApr 3, 2024
  16. Junio C HamanoApr 3, 2024
  17. 0/3 reftable/stack: use geometric table compactionJustin Tobler via GitGitGadget, Mar 29, 2024
  18. 1/3 reftable/stack: add env to disable autocompactionJustin Tobler via GitGitGadget, Mar 29, 2024
  19. Junio C HamanoMar 29, 2024
  20. Junio C HamanoMar 29, 2024
  21. Patrick SteinhardtApr 2, 2024
  22. Junio C HamanoApr 2, 2024
  23. 3/3 reftable/stack: make segment end inclusiveJustin Tobler via GitGitGadget, Mar 29, 2024
  24. Junio C HamanoMar 29, 2024
  25. Patrick SteinhardtApr 2, 2024
  26. 2/3 reftable/stack: use geometric table compactionJustin Tobler via GitGitGadget, Mar 29, 2024
  27. Patrick SteinhardtApr 2, 2024
  28. 0/2 reftable/stack: use geometric table compactionJustin Tobler via GitGitGadget, Apr 3, 2024
  29. 1/2 reftable/stack: add env to disable autocompactionJustin Tobler via GitGitGadget, Apr 3, 2024
  30. 2/2 reftable/stack: use geometric table compactionJustin Tobler via GitGitGadget, Apr 3, 2024
  31. Patrick SteinhardtApr 3, 2024
  32. Karthik NayakApr 3, 2024
  33. Junio C HamanoApr 3, 2024
  34. 0/3 reftable/stack: use geometric table compactionJustin Tobler via GitGitGadget, Apr 4, 2024
  35. 1/3 reftable/stack: allow disabling of auto-compactionJustin Tobler via GitGitGadget, Apr 4, 2024
  36. Patrick SteinhardtApr 8, 2024
  37. 2/3 reftable/stack: add env to disable autocompactionJustin Tobler via GitGitGadget, Apr 4, 2024
  38. Patrick SteinhardtApr 8, 2024
  39. Junio C HamanoApr 8, 2024
  40. 3/3 reftable/stack: use geometric table compactionJustin Tobler via GitGitGadget, Apr 4, 2024
  41. Patrick SteinhardtApr 8, 2024
  42. Justin ToblerApr 8, 2024
  43. 0/3 reftable/stack: use geometric table compactionJustin Tobler via GitGitGadget, Apr 8, 2024
  44. 1/3 reftable/stack: expose option to disable auto-compactionJustin Tobler via GitGitGadget, Apr 8, 2024
  45. 2/3 reftable/stack: add env to disable autocompactionJustin Tobler via GitGitGadget, Apr 8, 2024
  46. 3/3 reftable/stack: use geometric table compactionJustin Tobler via GitGitGadget, Apr 8, 2024
  47. Patrick SteinhardtApr 8, 2024
  48. Junio C HamanoApr 8, 2024
  49. Junio C HamanoApr 3, 2024
  50. Patrick SteinhardtApr 3, 2024
  51. Patrick SteinhardtApr 4, 2024
  52. Justin ToblerApr 4, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.