git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: What is the status of GSoC 2022 work on making Git use roaring bitmaps?

From
Han-Wen Nienhuys <hanwen@google.com>
Date
Aug 1, 2023, 11:26 UTC
Message-ID
<CAFQ2z_MzWauauzq_fKdcKTXahutLtADb7uHTh7ysinGNOx75nQ@mail.gmail.com>
In-Reply-To
<ZMgXBc5idN+sR3o1@nand.local>
On Mon, Jul 31, 2023 at 10:18 PM Taylor Blau <me@ttaylorr.com> wrote:
Show 9 quoted lines
> I haven't proved conclusively one way or the other where Roaring+Run is
> significantly faster than EWAH or vice-versa. There are some cases where
> the former is a clear winner, and other cases where it's the latter.
>
> In any event, my extremely WIP patches to make this mostly work are
> available here:
>
>   https://github.com/ttaylorr/git/compare/tb/roaring-bitmaps
>
thanks. For anyone reading along, the changes to JGit are here
https://git.eclipse.org/r/c/jgit/jgit/+/203448

I was looking into this because I was hoping that roaring might decrease peak memory usage.

I don't have firm evidence that it's better or worse, but I did observe that runtime and memory usage during GC (which is heavy on bitmap operations due to delta/xor encoding) was unchanged. That makes me pessimistic that there are significant gains to be had.

> One thing that I was able to do to produce slightly smaller Roaring+Run
Just for my edification: what is "Roaring + Run" ?

-- Han-Wen Nienhuys - Google Munich I work 80%. Don't expect answers from me on Fridays. -- Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich Registergericht und -nummer: Hamburg, HRB 86891 Sitz der Gesellschaft: Hamburg Geschäftsführer: Paul Manicle, Liana Sebastian

Previous: Taylor BlauNext: Jakub Narębski
Message 8 of 14 in “What is the status of GSoC 2022 work on making Git use roaring bitmaps?”
  1. Jakub NarębskiMar 23, 2023
  2. Taylor BlauMar 23, 2023
  3. Jakub NarębskiMar 23, 2023
  4. Abhradeep ChakrabortyMar 24, 2023
  5. Jakub NarębskiMar 25, 2023
  6. Han-Wen NienhuysJul 31, 2023
  7. Taylor BlauJul 31, 2023
  8. Han-Wen NienhuysAug 1, 2023
  9. Jakub NarębskiAug 1, 2023
  10. Han-Wen NienhuysAug 1, 2023
  11. Jakub NarębskiAug 1, 2023
  12. Taylor BlauAug 1, 2023
  13. Jakub NarębskiAug 1, 2023
  14. Taylor BlauAug 1, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.