git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 08/19] ewah: compressed bitmap implementation

From
Jeff King <peff@peff.net>
Date
Oct 25, 2013, 03:15 UTC
Message-ID
<20131025031528.GA26283@sigill.intra.peff.net>
In-Reply-To
<xmqqiowmkztz.fsf@gitster.dls.corp.google.com>
On Thu, Oct 24, 2013 at 04:34:48PM -0700, Junio C Hamano wrote:
Show 11 quoted lines
> Jeff King <peff@peff.net> writes:
> 
> > From: Vicent Marti <tanoku@gmail.com>
> >
> > EWAH is a word-aligned compressed variant of a bitset (i.e. a data
> > structure that acts as a 0-indexed boolean array for many entries).
> 
> I think I've already pointed out some when the original series was
> posted, but this does not seem to compile with -Wdecl-after-stmt
> (possibly other violations may exist; I haven't looked very closely
> yet).

Sorry about that. I typically compile with "-Wall -Werror" which does not catch this, and I didn't carefully go over the ewah code for style, as I considered it mostly a black-box import (though certainly it is worth fixing this case, as it's not just a style issue).

It looks like there is also some void pointer arithmetic...I _thought_ that was handled by -Wall, but I suppose not. Maybe I need to beef up my warning settings.

Note that there is also some use of cast-mmap-file-to-struct, only one instance of which uses __attribute__(packed). However, I notice that the regular pack code is also guilty of this (e.g., see check_packed_git_idx), so I don't know how careful we want to be (and whether we should be using the packed attribute when needed, or if it is sufficiently non-portable that we want to do it by hand in such cases).

-Peff
Previous: Junio C HamanoNext: Thomas Rast
Message 14 of 87 in “pack bitmaps”
  1. 0/19 pack bitmapsJeff King, Oct 24, 2013
  2. 01/19 sha1write: make buffer const-correctJeff King, Oct 24, 2013
  3. 02/19 revindex: Export new APIsJeff King, Oct 24, 2013
  4. 03/19 pack-objects: Refactor the packing listJeff King, Oct 24, 2013
  5. 04/19 pack-objects: factor out name_hashJeff King, Oct 24, 2013
  6. 05/19 revision: allow setting custom limiter functionJeff King, Oct 24, 2013
  7. 06/19 sha1_file: export `git_open_noatime`Jeff King, Oct 24, 2013
  8. 07/19 compat: add endianness helpersJeff King, Oct 24, 2013
  9. Thomas RastOct 26, 2013
  10. Jeff KingOct 30, 2013
  11. Vicent MartíOct 30, 2013
  12. 08/19 ewah: compressed bitmap implementationJeff King, Oct 24, 2013
  13. Junio C HamanoOct 24, 2013
  14. Jeff KingOct 25, 2013
  15. Thomas RastOct 26, 2013
  16. 09/19 documentation: add documentation for the bitmap formatJeff King, Oct 24, 2013
  17. Duy NguyenOct 25, 2013
  18. Jeff KingOct 25, 2013
  19. Duy NguyenOct 25, 2013
  20. Shawn PearceOct 25, 2013
  21. Jeff KingOct 30, 2013
  22. Shawn PearceOct 30, 2013
  23. Vicent MartiOct 30, 2013
  24. Vicent MartiOct 30, 2013
  25. 10/19 pack-bitmap: add support for bitmap indexesJeff King, Oct 24, 2013
  26. Shawn PearceOct 25, 2013
  27. Jeff KingOct 30, 2013
  28. Shawn PearceOct 30, 2013
  29. Vicent MartiOct 30, 2013
  30. Shawn PearceOct 30, 2013
  31. Jeff KingOct 30, 2013
  32. 11/19 pack-objects: use bitmaps when packing objectsJeff King, Oct 24, 2013
  33. Shawn PearceOct 25, 2013
  34. Jeff KingOct 30, 2013
  35. Shawn PearceOct 30, 2013
  36. Vicent MartiOct 30, 2013
  37. 12/19 rev-list: add bitmap mode to speed up object listsJeff King, Oct 24, 2013
  38. Shawn PearceOct 25, 2013
  39. Jeff KingOct 30, 2013
  40. 13/19 pack-objects: implement bitmap writingJeff King, Oct 24, 2013
  41. Duy NguyenOct 25, 2013
  42. Jeff KingOct 25, 2013
  43. 14/19 repack: stop using magic number for ARRAY_SIZE(exts)Jeff King, Oct 24, 2013
  44. 15/19 repack: turn exts array into array-of-structJeff King, Oct 24, 2013
  45. 16/19 repack: handle optional files created by pack-objectsJeff King, Oct 24, 2013
  46. 17/19 repack: consider bitmaps when performing repacksJeff King, Oct 24, 2013
  47. 18/19 t: add basic bitmap functionality testsJeff King, Oct 24, 2013
  48. 19/19 pack-bitmap: implement optional name_hash cacheJeff King, Oct 24, 2013
  49. Junio C HamanoOct 24, 2013
  50. Junio C HamanoOct 25, 2013
  51. 0/19 pack bitmapsJeff King, Oct 25, 2013
  52. 01/19 sha1write: make buffer const-correctJeff King, Oct 25, 2013
  53. 02/19 revindex: Export new APIsJeff King, Oct 25, 2013
  54. 03/19 pack-objects: Refactor the packing listJeff King, Oct 25, 2013
  55. 04/19 pack-objects: factor out name_hashJeff King, Oct 25, 2013
  56. 05/19 revision: allow setting custom limiter functionJeff King, Oct 25, 2013
  57. 06/19 sha1_file: export `git_open_noatime`Jeff King, Oct 25, 2013
  58. 07/19 compat: add endianness helpersJeff King, Oct 25, 2013
  59. 08/19 ewah: compressed bitmap implementationJeff King, Oct 25, 2013
  60. 09/19 documentation: add documentation for the bitmap formatJeff King, Oct 25, 2013
  61. 10/19 pack-bitmap: add support for bitmap indexesJeff King, Oct 25, 2013
  62. Junio C HamanoOct 25, 2013
  63. Jeff KingOct 26, 2013
  64. Jeff KingOct 26, 2013
  65. Junio C HamanoOct 28, 2013
  66. Jeff KingOct 30, 2013
  67. Duy NguyenOct 26, 2013
  68. Jeff KingOct 30, 2013
  69. 11/19 pack-objects: use bitmaps when packing objectsJeff King, Oct 25, 2013
  70. Duy NguyenOct 26, 2013
  71. Jeff KingOct 30, 2013
  72. Duy NguyenOct 30, 2013
  73. Jeff KingOct 30, 2013
  74. Duy NguyenOct 31, 2013
  75. 12/19 rev-list: add bitmap mode to speed up object listsJeff King, Oct 25, 2013
  76. 13/19 pack-objects: implement bitmap writingJeff King, Oct 25, 2013
  77. 14/19 repack: stop using magic number for ARRAY_SIZE(exts)Jeff King, Oct 25, 2013
  78. 15/19 repack: turn exts array into array-of-structJeff King, Oct 25, 2013
  79. 16/19 repack: handle optional files created by pack-objectsJeff King, Oct 25, 2013
  80. 17/19 repack: consider bitmaps when performing repacksJeff King, Oct 25, 2013
  81. 18/19 t: add basic bitmap functionality testsJeff King, Oct 25, 2013
  82. SZEDER GáborOct 28, 2013
  83. Jeff KingOct 30, 2013
  84. 19/19 pack-bitmap: implement optional name_hash cacheJeff King, Oct 25, 2013
  85. 20/19 count-objects: consider .bitmap without .pack/.idx pair garbageNguyễn Thái Ngọc Duy, Oct 26, 2013
  86. Jeff KingOct 30, 2013
  87. Junio C HamanoOct 30, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.