git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 3/7] reftable/merged: skip comparison for records of the same subiter

From
Eric Sunshine <sunshine@sunshineco.com>
Date
Feb 1, 2024, 17:29 UTC
Message-ID
<CAPig+cQwgYdKCHnu1TUj2JYyQkRF6Ey4hw1MvAVpfhqcwjV04w@mail.gmail.com>
In-Reply-To
<0ca86eba710895f0e22fc15fe5221f5487031f64.1706782841.git.ps@pks.im>
On Thu, Feb 1, 2024 at 8:17 AM Patrick Steinhardt <ps@pks.im> wrote:
Show 8 quoted lines
> When retrieving the next entry of a merged iterator we need to drop all
> records of other sub-iterators that would be shadowed by the record that
> we are about to return. We do this by comparing record keys, dropping
> all keys that are smaller or equal to the key of the record we are about
> to return.
>
> There is an edge case here where we can skip that comparison: when the
> record in the priority queue comes from the same subiterator than the
s/than/as/
Show 28 quoted lines
> record we are about to return then we know that its key must be larger
> than the key of the record we are about to return. This property is
> guaranteed by the sub-iterators, and if it didn't hold then the whole
> merged iterator would return records in the wrong order, too.
>
> While this may seem like a very specific edge case it's in fact quite
> likely to happen. For most repositories out there you can assume that we
> will end up with one large table and several smaller ones on top of it.
> Thus, it is very likely that the next entry will sort towards the top of
> the priority queue.
>
> Special case this and break out of the loop in that case. The following
> benchmark uses git-show-ref(1) to print a single ref matching a pattern
> out of 1 million refs:
>
>   Benchmark 1: show-ref: single matching ref (revision = HEAD~)
>     Time (mean ± σ):     162.6 ms ±   4.5 ms    [User: 159.0 ms, System: 3.5 ms]
>     Range (min … max):   156.6 ms … 188.5 ms    1000 runs
>
>   Benchmark 2: show-ref: single matching ref (revision = HEAD)
>     Time (mean ± σ):     156.8 ms ±   4.7 ms    [User: 153.0 ms, System: 3.6 ms]
>     Range (min … max):   151.4 ms … 188.4 ms    1000 runs
>
>   Summary
>     show-ref: single matching ref (revision = HEAD) ran
>       1.04 ± 0.04 times faster than show-ref: single matching ref (revision = HEAD~)
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 6 of 23 in “reftable: improve ref iteration performance”
  1. 0/7 reftable: improve ref iteration performancePatrick Steinhardt, Feb 1, 2024
  2. 1/7 reftable/record: introduce function to compare records by keyPatrick Steinhardt, Feb 1, 2024
  3. Eric SunshineFeb 1, 2024
  4. 2/7 reftable/merged: allocation-less dropping of shadowed recordsPatrick Steinhardt, Feb 1, 2024
  5. 3/7 reftable/merged: skip comparison for records of the same subiterPatrick Steinhardt, Feb 1, 2024
  6. Eric SunshineFeb 1, 2024
  7. Patrick SteinhardtFeb 2, 2024
  8. 4/7 reftable/pq: allocation-less comparison of entry keysPatrick Steinhardt, Feb 1, 2024
  9. 5/7 reftable/block: swap buffers instead of copyingPatrick Steinhardt, Feb 1, 2024
  10. 6/7 reftable/record: don't try to reallocate ref record namePatrick Steinhardt, Feb 1, 2024
  11. 7/7 reftable/reader: add comments to `table_iter_next()`Patrick Steinhardt, Feb 1, 2024
  12. John CaiFeb 9, 2024
  13. Patrick SteinhardtFeb 12, 2024
  14. 0/7 reftable: improve ref iteration performancePatrick Steinhardt, Feb 12, 2024
  15. 1/7 reftable/record: introduce function to compare records by keyPatrick Steinhardt, Feb 12, 2024
  16. 2/7 reftable/merged: allocation-less dropping of shadowed recordsPatrick Steinhardt, Feb 12, 2024
  17. 3/7 reftable/merged: skip comparison for records of the same subiterPatrick Steinhardt, Feb 12, 2024
  18. 4/7 reftable/pq: allocation-less comparison of entry keysPatrick Steinhardt, Feb 12, 2024
  19. 5/7 reftable/block: swap buffers instead of copyingPatrick Steinhardt, Feb 12, 2024
  20. 6/7 reftable/record: don't try to reallocate ref record namePatrick Steinhardt, Feb 12, 2024
  21. 7/7 reftable/reader: add comments to `table_iter_next()`Patrick Steinhardt, Feb 12, 2024
  22. Junio C HamanoFeb 12, 2024
  23. Patrick SteinhardtFeb 13, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.