git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: What is the status of GSoC 2022 work on making Git use roaring bitmaps?

From
Han-Wen Nienhuys <hanwen@google.com>
Date
Jul 31, 2023, 17:46 UTC
Message-ID
<CAFQ2z_P2HkT8grAFk=6Mr05rJRfsh_sXypVFPyHr0v5xkcjYTA@mail.gmail.com>
In-Reply-To
<85tty8afvy.fsf@gmail.com>
On Sat, Mar 25, 2023 at 6:40 PM Jakub Narębski <jnareb@gmail.com> wrote:
Show 6 quoted lines
> >>> Abhradeep promised[1] that he'd include some performance work in his
> >>> next version of that series. I think the main things we'd be interested
> >>> in are:
> >>>
> >>>   - Does using Roaring provide a file-size advantage over
> >>>     EWAH-compressed bitmaps?

I modified JGit to write Roaring bitmaps instead of EWAH bitmaps. The resulting difference in file sizes are small, and actually favor EWAH:

$ ls -l {ewah-repos,roaring-repos}/*.git/objects/pack/*.bitmap -r--r----- 1 hanwen primarygroup 26257386 Jul 31 15:04 ewah-repos/android-pfb.git/objects/pack/pack-b14c35ec7fc3bb20884abe51a81c832be5983fdc.bitmap -r--r----- 1 hanwen primarygroup 27621579 Jul 31 15:20 roaring-repos/android-pfb.git/objects/pack/pack-b14c35ec7fc3bb20884abe51a81c832be5983fdc.bitmap

-r--r----- 1 hanwen primarygroup 1037356 Jul 31 14:46 ewah-repos/gerrit.git/objects/pack/pack-fe46c7f96a2910f5775a2ff3bef7e4fa0e779f91.bitmap -r--r----- 1 hanwen primarygroup 1242608 Jul 31 14:45 roaring-repos/gerrit.git/objects/pack/pack-fe46c7f96a2910f5775a2ff3bef7e4fa0e779f91.bitmap

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--
Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschäftsführer: Paul Manicle, Liana Sebastian
Previous: Jakub NarębskiNext: Taylor Blau
Message 6 of 14 in “What is the status of GSoC 2022 work on making Git use roaring bitmaps?”
  1. Jakub NarębskiMar 23, 2023
  2. Taylor BlauMar 23, 2023
  3. Jakub NarębskiMar 23, 2023
  4. Abhradeep ChakrabortyMar 24, 2023
  5. Jakub NarębskiMar 25, 2023
  6. Han-Wen NienhuysJul 31, 2023
  7. Taylor BlauJul 31, 2023
  8. Han-Wen NienhuysAug 1, 2023
  9. Jakub NarębskiAug 1, 2023
  10. Han-Wen NienhuysAug 1, 2023
  11. Jakub NarębskiAug 1, 2023
  12. Taylor BlauAug 1, 2023
  13. Jakub NarębskiAug 1, 2023
  14. Taylor BlauAug 1, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.