git/list[1] front-page[2] threads[3] people[4] search[5] about
wed 2026-10-07 18:03 UTC

Re: [PATCH v3 4/6] pack-bitmap: prepare to read lookup table extension

From
Taylor Blau <me@ttaylorr.com>
Date
Jul 15, 2022, 22:20 UTC
Message-ID
<YtHoJ90N6rmDmn6M@nand.local>
In-Reply-To
<CAPOJW5y+ywbiT2XBYYNN+y73+V98Ro33D1bgZQveQLTPfrgE_g@mail.gmail.com>
On Fri, Jul 15, 2022 at 10:08:17PM +0530, Abhradeep Chakraborty wrote:
Show 27 quoted lines
> On Fri, Jul 15, 2022 at 8:16 AM Taylor Blau <me@ttaylorr.com> wrote:
> >
> > On Mon, Jul 04, 2022 at 08:46:14AM +0000, Abhradeep Chakraborty via GitGitGadget wrote:
> > > +/*
> > > + * Searches for a matching triplet. `va` is a pointer
> > > + * to the wanted commit position value. `vb` points to
> > > + * a triplet in lookup table. The first 4 bytes of each
> > > + * triplet (pointed by `vb`) are compared with `*va`.
> > > + */
> > > +static int triplet_cmp(const void *va, const void *vb)
> > > +{
> > > +
> > > +     uint32_t a = *(uint32_t *)va;
> >
> > The comment you added is definitely helpful, but I still think that this
> > line is a little magical. `*va` isn't really a pointer to a `uint32_t`,
> > but a pointer to the start of a triplet, which just *happens* to have a
> > 4-byte integer at the beginning of it.
>
> Are you sure about this? As far as I know, the first parameter of such
> comparing functions is always a pointer to the given key that we need
> to search for and the second parameter points to each element of an
> array.
>
> I think "`va is a pointer to the wanted commit position value" is not
> that descriptive. Maybe "`va` is a pointer to the given key" is
> better. What do you think?

Yes, the first argument to the comparison function used in bsearch() is a pointer to some element in the array. I just meant that that array is the bitmap_git->table_lookup region, so each element isn't actually a uint32_t array, but the whole thing is an array of (uint32_t, uint64_t, uint32_t) triplets.

What you wrote here is fine, and I don't even think that the comment needs updating. If you did want to clarify, I think you could say something along the lines of what you wrote above ("`va` is a pointer to an array element") and add something along the lines of "where the array is the lookup table region of the .bitmap".

Show 27 quoted lines
> > > +             ALLOC_ARRAY(xor_items, xor_items_alloc);
> >
> > This ALLOC_ARRAY() looks great to me. I wonder if we could amortize the
> > cost of allocating in this (somewhat) hot function by treating the
> > `xor_items` array as a reusable static buffer where we reset
> > xor_items_nr to 0 when entering this function.
> >
> > > +             while (xor_row != 0xffffffff) {
> > > +                     struct object_id xor_oid;
> > > +
> > > +                     if (xor_items_nr + 1 >= bitmap_git->entry_count) {
> > > +                             free(xor_items);
> > > +                             error(_("corrupt bitmap lookup table: xor chain exceed entry count"));
> >
> > I think we can probably `die()` here, we're pretty much out of luck in
> > this case.
> > ...
> > > +                             error(_("corrupt bitmap lookup table: commit index %u out of range"),
> > > +                                     triplet.commit_pos);
> >
> > Same here.
>
> I didn't use `die()` here because I thought returning NULL would be a
> better idea. In that case, Git can still do its job by using the
> traditional approach  - traversing  between objects.
> `load_bitmap_entries_v1` also returns NULL if an error occurs. What do
> you think?

Ah, I wasn't aware that our callers are graceful enough to handle this like that. Yes, if we can fallback gracefully, we should, so I think just error()-ing here (and above) is the right choice. Thanks for saying so.

Thanks, Taylor

Previous: Taylor BlauNext: Abhradeep Chakraborty
Message 97 of 162 in “Documentation/technical: describe bitmap lookup table extension”
  1. 1/6 Documentation/technical: describe bitmap lookup table extensionAbhradeep Chakraborty via GitGitGadget, Jun 20, 2022
  2. 0/6 [GSoC] bitmap: integrate a lookup table extension to the bitmap formatAbhradeep Chakraborty via GitGitGadget, Jun 20, 2022
  3. 2/6 pack-bitmap: prepare to read lookup table extensionAbhradeep Chakraborty via GitGitGadget, Jun 20, 2022
  4. 3/6 pack-bitmap-write.c: write lookup table extensionAbhradeep Chakraborty via GitGitGadget, Jun 20, 2022
  5. 5/6 bitmap-commit-table: add tests for the bitmap lookup tableAbhradeep Chakraborty via GitGitGadget, Jun 20, 2022
  6. 4/6 builtin/pack-objects.c: learn pack.writeBitmapLookupTableTaylor Blau via GitGitGadget, Jun 20, 2022
  7. 6/6 bitmap-lookup-table: add performance testsAbhradeep Chakraborty via GitGitGadget, Jun 20, 2022
  8. Derrick StoleeJun 20, 2022
  9. Taylor BlauJun 20, 2022
  10. Taylor BlauJun 20, 2022
  11. Derrick StoleeJun 20, 2022
  12. Derrick StoleeJun 20, 2022
  13. Taylor BlauJun 20, 2022
  14. Taylor BlauJun 20, 2022
  15. Taylor BlauJun 20, 2022
  16. Abhradeep ChakrabortyJun 21, 2022
  17. Abhradeep ChakrabortyJun 21, 2022
  18. Abhradeep ChakrabortyJun 21, 2022
  19. Abhradeep ChakrabortyJun 21, 2022
  20. Abhradeep ChakrabortyJun 21, 2022
  21. Abhradeep ChakrabortyJun 21, 2022
  22. Abhradeep ChakrabortyJun 21, 2022
  23. Taylor BlauJun 22, 2022
  24. Taylor BlauJun 22, 2022
  25. Taylor BlauJun 22, 2022
  26. Abhradeep ChakrabortyJun 22, 2022
  27. Taylor BlauJun 22, 2022
  28. Taylor BlauJun 22, 2022
  29. Taylor BlauJun 22, 2022
  30. Taylor BlauJun 22, 2022
  31. Abhradeep ChakrabortyJun 22, 2022
  32. Taylor BlauJun 22, 2022
  33. 1/6 Documentation/technical: describe bitmap lookup table extensionAbhradeep Chakraborty via GitGitGadget, Jun 26, 2022
  34. 2/6 pack-bitmap-write.c: write lookup table extensionAbhradeep Chakraborty via GitGitGadget, Jun 26, 2022
  35. 3/6 pack-bitmap-write: learn pack.writeBitmapLookupTable and add testsAbhradeep Chakraborty via GitGitGadget, Jun 26, 2022
  36. 0/6 [GSoC] bitmap: integrate a lookup table extension to the bitmap formatAbhradeep Chakraborty via GitGitGadget, Jun 26, 2022
  37. 5/6 bitmap-lookup-table: add performance tests for lookup tableAbhradeep Chakraborty via GitGitGadget, Jun 26, 2022
  38. 6/6 p5310-pack-bitmaps.sh: enable pack.writeReverseIndex for testingAbhradeep Chakraborty via GitGitGadget, Jun 26, 2022
  39. 4/6 pack-bitmap: prepare to read lookup table extensionAbhradeep Chakraborty via GitGitGadget, Jun 26, 2022
  40. Derrick StoleeJun 27, 2022
  41. Derrick StoleeJun 27, 2022
  42. Derrick StoleeJun 27, 2022
  43. Derrick StoleeJun 27, 2022
  44. Taylor BlauJun 27, 2022
  45. Taylor BlauJun 27, 2022
  46. Taylor BlauJun 27, 2022
  47. Abhradeep ChakrabortyJun 27, 2022
  48. Abhradeep ChakrabortyJun 27, 2022
  49. 3/6 pack-bitmap-write: learn pack.writeBitmapLookupTable and add testsAbhradeep Chakraborty, Jun 27, 2022
  50. Taylor BlauJun 27, 2022
  51. Taylor BlauJun 27, 2022
  52. Abhradeep ChakrabortyJun 27, 2022
  53. Abhradeep ChakrabortyJun 27, 2022
  54. Derrick StoleeJun 27, 2022
  55. Abhradeep ChakrabortyJun 27, 2022
  56. Taylor BlauJun 27, 2022
  57. Taylor BlauJun 27, 2022
  58. Taylor BlauJun 27, 2022
  59. Taylor BlauJun 27, 2022
  60. Abhradeep ChakrabortyJun 28, 2022
  61. Abhradeep ChakrabortyJun 28, 2022
  62. Abhradeep ChakrabortyJun 28, 2022
  63. Abhradeep ChakrabortyJun 28, 2022
  64. Taylor BlauJun 29, 2022
  65. Taylor BlauJun 29, 2022
  66. Taylor BlauJun 29, 2022
  67. Taylor BlauJun 29, 2022
  68. Taylor BlauJun 29, 2022
  69. Abhradeep ChakrabortyJun 30, 2022
  70. Abhradeep ChakrabortyJun 30, 2022
  71. 2/6 pack-bitmap-write.c: write lookup table extensionAbhradeep Chakraborty via GitGitGadget, Jul 4, 2022
  72. 1/6 Documentation/technical: describe bitmap lookup table extensionAbhradeep Chakraborty via GitGitGadget, Jul 4, 2022
  73. 0/6 [GSoC] bitmap: integrate a lookup table extension to the bitmap formatAbhradeep Chakraborty via GitGitGadget, Jul 4, 2022
  74. 3/6 pack-bitmap-write: learn pack.writeBitmapLookupTable and add testsAbhradeep Chakraborty via GitGitGadget, Jul 4, 2022
  75. 4/6 pack-bitmap: prepare to read lookup table extensionAbhradeep Chakraborty via GitGitGadget, Jul 4, 2022
  76. 5/6 bitmap-lookup-table: add performance tests for lookup tableAbhradeep Chakraborty via GitGitGadget, Jul 4, 2022
  77. 6/6 p5310-pack-bitmaps.sh: remove pack.writeReverseIndexAbhradeep Chakraborty via GitGitGadget, Jul 4, 2022
  78. Abhradeep ChakrabortyJul 4, 2022
  79. Junio C HamanoJul 6, 2022
  80. Abhradeep ChakrabortyJul 7, 2022
  81. Kaartic SivaraamJul 7, 2022
  82. Abhradeep ChakrabortyJul 7, 2022
  83. Philip OakleyJul 8, 2022
  84. Abhradeep ChakrabortyJul 9, 2022
  85. Philip OakleyJul 10, 2022
  86. Taylor BlauJul 14, 2022
  87. Taylor BlauJul 14, 2022
  88. Taylor BlauJul 15, 2022
  89. Taylor BlauJul 15, 2022
  90. Taylor BlauJul 15, 2022
  91. Philip OakleyJul 15, 2022
  92. Abhradeep ChakrabortyJul 15, 2022
  93. Abhradeep ChakrabortyJul 15, 2022
  94. Abhradeep ChakrabortyJul 15, 2022
  95. Abhradeep ChakrabortyJul 15, 2022
  96. Taylor BlauJul 15, 2022
  97. Taylor BlauJul 15, 2022
  98. Abhradeep ChakrabortyJul 16, 2022
  99. Martin ÅgrenJul 18, 2022
  100. Martin ÅgrenJul 18, 2022
  101. Abhradeep ChakrabortyJul 18, 2022
  102. Martin ÅgrenJul 18, 2022
  103. 1/6 Documentation/technical: describe bitmap lookup table extensionAbhradeep Chakraborty via GitGitGadget, Jul 20, 2022
  104. 2/6 pack-bitmap-write.c: write lookup table extensionAbhradeep Chakraborty via GitGitGadget, Jul 20, 2022
  105. 0/6 [GSoC] bitmap: integrate a lookup table extension to the bitmap formatAbhradeep Chakraborty via GitGitGadget, Jul 20, 2022
  106. 4/6 pack-bitmap: prepare to read lookup table extensionAbhradeep Chakraborty via GitGitGadget, Jul 20, 2022
  107. 5/6 p5310-pack-bitmaps.sh: enable `pack.writeReverseIndex`Abhradeep Chakraborty via GitGitGadget, Jul 20, 2022
  108. 3/6 pack-bitmap-write: learn pack.writeBitmapLookupTable and add testsAbhradeep Chakraborty via GitGitGadget, Jul 20, 2022
  109. 6/6 bitmap-lookup-table: add performance tests for lookup tableAbhradeep Chakraborty via GitGitGadget, Jul 20, 2022
  110. 0/6 [GSoC] bitmap: integrate a lookup table extension to the bitmap formatAbhradeep Chakraborty via GitGitGadget, Jul 20, 2022
  111. 1/6 Documentation/technical: describe bitmap lookup table extensionAbhradeep Chakraborty via GitGitGadget, Jul 20, 2022
  112. 2/6 pack-bitmap-write.c: write lookup table extensionAbhradeep Chakraborty via GitGitGadget, Jul 20, 2022
  113. 4/6 pack-bitmap: prepare to read lookup table extensionAbhradeep Chakraborty via GitGitGadget, Jul 20, 2022
  114. 5/6 p5310-pack-bitmaps.sh: enable `pack.writeReverseIndex`Abhradeep Chakraborty via GitGitGadget, Jul 20, 2022
  115. 6/6 bitmap-lookup-table: add performance tests for lookup tableAbhradeep Chakraborty via GitGitGadget, Jul 20, 2022
  116. 3/6 pack-bitmap-write: learn pack.writeBitmapLookupTable and add testsAbhradeep Chakraborty via GitGitGadget, Jul 20, 2022
  117. Taylor BlauJul 26, 2022
  118. Taylor BlauJul 26, 2022
  119. Taylor BlauJul 26, 2022
  120. Taylor BlauJul 26, 2022
  121. Taylor BlauJul 26, 2022
  122. Ævar Arnfjörð BjarmasonJul 26, 2022
  123. Derrick StoleeJul 26, 2022
  124. Ævar Arnfjörð BjarmasonJul 26, 2022
  125. Abhradeep ChakrabortyJul 26, 2022
  126. Abhradeep ChakrabortyJul 26, 2022
  127. Abhradeep ChakrabortyJul 26, 2022
  128. Eric SunshineJul 26, 2022
  129. Johannes SchindelinJul 28, 2022
  130. Abhradeep ChakrabortyAug 2, 2022
  131. Johannes SchindelinAug 2, 2022
  132. Abhradeep ChakrabortyAug 2, 2022
  133. Johannes SchindelinAug 8, 2022
  134. Abhradeep ChakrabortyAug 8, 2022
  135. Johannes SchindelinAug 9, 2022
  136. Abhradeep ChakrabortyAug 9, 2022
  137. Abhradeep ChakrabortyAug 9, 2022
  138. Johannes SchindelinAug 10, 2022
  139. Johannes SchindelinAug 10, 2022
  140. Abhradeep ChakrabortyAug 10, 2022
  141. Derrick StoleeAug 10, 2022
  142. Abhradeep ChakrabortyAug 12, 2022
  143. Derrick StoleeAug 12, 2022
  144. Abhradeep ChakrabortyAug 13, 2022
  145. Abhradeep ChakrabortyAug 13, 2022
  146. 2/6 bitmap: move `get commit positions` code to `bitmap_writer_finish`Abhradeep Chakraborty via GitGitGadget, Aug 14, 2022
  147. 0/6 [GSoC] bitmap: integrate a lookup table extension to the bitmap formatAbhradeep Chakraborty via GitGitGadget, Aug 14, 2022
  148. 1/6 Documentation/technical: describe bitmap lookup table extensionAbhradeep Chakraborty via GitGitGadget, Aug 14, 2022
  149. 3/6 pack-bitmap-write.c: write lookup table extensionAbhradeep Chakraborty via GitGitGadget, Aug 14, 2022
  150. 5/6 pack-bitmap: prepare to read lookup table extensionAbhradeep Chakraborty via GitGitGadget, Aug 14, 2022
  151. 4/6 pack-bitmap-write: learn pack.writeBitmapLookupTable and add testsAbhradeep Chakraborty via GitGitGadget, Aug 14, 2022
  152. 6/6 bitmap-lookup-table: add performance tests for lookup tableAbhradeep Chakraborty via GitGitGadget, Aug 14, 2022
  153. Taylor BlauAug 16, 2022
  154. Taylor BlauAug 16, 2022
  155. Abhradeep ChakrabortyAug 17, 2022
  156. Taylor BlauAug 17, 2022
  157. Junio C HamanoAug 19, 2022
  158. Taylor BlauAug 19, 2022
  159. Johannes SchindelinAug 22, 2022
  160. Taylor BlauAug 22, 2022
  161. Taylor BlauAug 25, 2022
  162. Junio C HamanoAug 26, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.