git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 00/24] pack-objects: multi-pack verbatim reuse

From
Taylor Blau <me@ttaylorr.com>
Date
Dec 1, 2023, 20:02 UTC
Message-ID
<ZWo7zjs+q8SZ5o1o@nand.local>
In-Reply-To
<ZWmZ0pk6fr-8dmF1@tanuki>
On Fri, Dec 01, 2023 at 09:31:14AM +0100, Patrick Steinhardt wrote:
Show 17 quoted lines
> > But
> > I still find it relatively unsatisfying for a couple of reasons:
> >
> >   - With the exception of the last group of patches, none of the earlier
> >     series enable any new, useful behavior on their own. IOW, if we just
> >     merged the first three series and then forgot about this topic, we
> >     wouldn't have done anything useful ;-).
>
> Well, sometimes I wish we'd buy more into the iterative style of working
> in the Git project, where it's fine to land patch series that only work
> into the direction of a specific topic but don't yet do anything
> interesting by themselves. The benefits would be both that individual
> pieces land faster while also ensuring that the review load is kept at
> bay.
>
> But there's of course also downsides to this, especially in an open
> source project like Git:

I tend to agree with the downsides you list. My biggest concern with this series in particular is that we're trying to break down an all-or-nothing change into smaller components. So if we landed four out of the five of those series, it would be better to have landed none of them, since the first four aren't really all that useful on their own.

I suppose if we're relatively confident that the last series will be merged eventually, then that seems like less of a concern. But I'm not sure that we're at that point yet.

Show 5 quoted lines
> I wonder whether a compromise would be to continue sending complete
> patch series, but explicitly point out "break points" in your patch
> series. These break points could be an indicator to the maintainer that
> it's fine to merge everything up to it while still keeping out the
> remainder of the patch series.

I think that's a reasonable alternative. It does create a minor headache for the maintainer[^1], though, so I'd like to avoid it if possible.

Show 8 quoted lines
> >   - The fourth series (which actually implements multi-pack reuse) still
> >     remains the most complicated, and would likely be the most difficult
> >     to review. So I think you'd still have one difficult series to
> >     review, plus four other series which are slightly less difficult to
> >     review ;-).
>
> That's fine in my opinion, there's no surprise here that a complicated
> topic is demanding for both the author and the reviewer.

My preference is to avoid splitting the series if we can help it. But if you feel strongly, or others feel similarly, I'm happy to take another crack at breaking it up. Thanks for all of your review so far!

Thanks, Taylor

Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 69 of 107 in “pack-objects: multi-pack verbatim reuse”
  1. 00/24 pack-objects: multi-pack verbatim reuseTaylor Blau, Nov 28, 2023
  2. 01/24 pack-objects: free packing_data in more placesTaylor Blau, Nov 28, 2023
  3. Patrick SteinhardtNov 30, 2023
  4. Taylor BlauNov 30, 2023
  5. 02/24 pack-bitmap-write: deep-clear the `bb_commit` slabTaylor Blau, Nov 28, 2023
  6. Patrick SteinhardtNov 30, 2023
  7. Taylor BlauNov 30, 2023
  8. Jeff KingDec 12, 2023
  9. Taylor BlauDec 12, 2023
  10. 03/24 pack-bitmap: plug leak in find_objects()Taylor Blau, Nov 28, 2023
  11. Jeff KingDec 12, 2023
  12. 04/24 midx: factor out `fill_pack_info()`Taylor Blau, Nov 28, 2023
  13. Patrick SteinhardtNov 30, 2023
  14. Taylor BlauNov 30, 2023
  15. 05/24 midx: implement `DISP` chunkTaylor Blau, Nov 28, 2023
  16. Patrick SteinhardtNov 30, 2023
  17. Taylor BlauNov 30, 2023
  18. Junio C HamanoDec 3, 2023
  19. Taylor BlauDec 5, 2023
  20. Junio C HamanoDec 9, 2023
  21. Taylor BlauDec 9, 2023
  22. Jeff KingDec 12, 2023
  23. Taylor BlauDec 13, 2023
  24. Junio C HamanoDec 13, 2023
  25. 06/24 midx: implement `midx_locate_pack()`Taylor Blau, Nov 28, 2023
  26. 07/24 midx: implement `--retain-disjoint` modeTaylor Blau, Nov 28, 2023
  27. Patrick SteinhardtNov 30, 2023
  28. Taylor BlauNov 30, 2023
  29. Patrick SteinhardtDec 1, 2023
  30. 08/24 pack-objects: implement `--ignore-disjoint` modeTaylor Blau, Nov 28, 2023
  31. Patrick SteinhardtNov 30, 2023
  32. Taylor BlauNov 30, 2023
  33. Patrick SteinhardtDec 1, 2023
  34. Taylor BlauDec 1, 2023
  35. 09/24 repack: implement `--extend-disjoint` modeTaylor Blau, Nov 28, 2023
  36. Patrick SteinhardtDec 7, 2023
  37. Taylor BlauDec 7, 2023
  38. Patrick SteinhardtDec 8, 2023
  39. Taylor BlauDec 8, 2023
  40. Patrick SteinhardtDec 11, 2023
  41. Taylor BlauDec 11, 2023
  42. 10/24 pack-bitmap: pass `bitmapped_pack` struct to pack-reuse functionsTaylor Blau, Nov 28, 2023
  43. Patrick SteinhardtDec 7, 2023
  44. Taylor BlauDec 7, 2023
  45. Patrick SteinhardtDec 8, 2023
  46. 11/24 pack-bitmap: simplify `reuse_partial_packfile_from_bitmap()` signatureTaylor Blau, Nov 28, 2023
  47. Patrick SteinhardtDec 7, 2023
  48. Taylor BlauDec 7, 2023
  49. 12/24 pack-bitmap: return multiple packs via `reuse_partial_packfile_from_bitmap()`Taylor Blau, Nov 28, 2023
  50. 13/24 pack-objects: parameterize pack-reuse routines over a single packTaylor Blau, Nov 28, 2023
  51. 14/24 pack-objects: keep track of `pack_start` for each reuse packTaylor Blau, Nov 28, 2023
  52. Patrick SteinhardtDec 7, 2023
  53. Taylor BlauDec 7, 2023
  54. 15/24 pack-objects: pass `bitmapped_pack`'s to pack-reuse functionsTaylor Blau, Nov 28, 2023
  55. 16/24 pack-objects: prepare `write_reused_pack()` for multi-pack reuseTaylor Blau, Nov 28, 2023
  56. Patrick SteinhardtDec 7, 2023
  57. Taylor BlauDec 7, 2023
  58. 17/24 pack-objects: prepare `write_reused_pack_verbatim()` for multi-pack reuseTaylor Blau, Nov 28, 2023
  59. 18/24 pack-objects: include number of packs reused in outputTaylor Blau, Nov 28, 2023
  60. 19/24 pack-bitmap: prepare to mark objects from multiple packs for reuseTaylor Blau, Nov 28, 2023
  61. 20/24 pack-objects: add tracing for various packfile metricsTaylor Blau, Nov 28, 2023
  62. 21/24 t/test-lib-functions.sh: implement `test_trace2_data` helperTaylor Blau, Nov 28, 2023
  63. 22/24 pack-objects: allow setting `pack.allowPackReuse` to "single"Taylor Blau, Nov 28, 2023
  64. 23/24 pack-bitmap: reuse objects from all disjoint packsTaylor Blau, Nov 28, 2023
  65. 24/24 t/perf: add performance tests for multi-pack reuseTaylor Blau, Nov 28, 2023
  66. Patrick SteinhardtNov 30, 2023
  67. Taylor BlauNov 30, 2023
  68. Patrick SteinhardtDec 1, 2023
  69. Taylor BlauDec 1, 2023
  70. Patrick SteinhardtDec 4, 2023
  71. Jeff KingDec 12, 2023
  72. Taylor BlauDec 15, 2023
  73. Jeff KingDec 21, 2023
  74. Taylor BlauJan 4, 2024
  75. 00/26 pack-objects: multi-pack verbatim reuseTaylor Blau, Dec 14, 2023
  76. 01/26 pack-objects: free packing_data in more placesTaylor Blau, Dec 14, 2023
  77. 02/26 pack-bitmap-write: deep-clear the `bb_commit` slabTaylor Blau, Dec 14, 2023
  78. 03/26 pack-bitmap: plug leak in find_objects()Taylor Blau, Dec 14, 2023
  79. 04/26 midx: factor out `fill_pack_info()`Taylor Blau, Dec 14, 2023
  80. 05/26 midx: implement `BTMP` chunkTaylor Blau, Dec 14, 2023
  81. 06/26 midx: implement `midx_locate_pack()`Taylor Blau, Dec 14, 2023
  82. 07/26 pack-bitmap: pass `bitmapped_pack` struct to pack-reuse functionsTaylor Blau, Dec 14, 2023
  83. 08/26 ewah: implement `bitmap_is_empty()`Taylor Blau, Dec 14, 2023
  84. 09/26 pack-bitmap: simplify `reuse_partial_packfile_from_bitmap()` signatureTaylor Blau, Dec 14, 2023
  85. 10/26 pack-bitmap: return multiple packs via `reuse_partial_packfile_from_bitmap()`Taylor Blau, Dec 14, 2023
  86. 11/26 pack-objects: parameterize pack-reuse routines over a single packTaylor Blau, Dec 14, 2023
  87. 12/26 pack-objects: keep track of `pack_start` for each reuse packTaylor Blau, Dec 14, 2023
  88. 13/26 pack-objects: pass `bitmapped_pack`'s to pack-reuse functionsTaylor Blau, Dec 14, 2023
  89. 14/26 pack-objects: prepare `write_reused_pack()` for multi-pack reuseTaylor Blau, Dec 14, 2023
  90. 15/26 pack-objects: prepare `write_reused_pack_verbatim()` for multi-pack reuseTaylor Blau, Dec 14, 2023
  91. 16/26 pack-objects: include number of packs reused in outputTaylor Blau, Dec 14, 2023
  92. 17/26 git-compat-util.h: implement checked size_t to uint32_t conversionTaylor Blau, Dec 14, 2023
  93. 18/26 midx: implement `midx_preferred_pack()`Taylor Blau, Dec 14, 2023
  94. 19/26 pack-revindex: factor out `midx_key_to_pack_pos()` helperTaylor Blau, Dec 14, 2023
  95. 20/26 pack-revindex: implement `midx_pair_to_pack_pos()`Taylor Blau, Dec 14, 2023
  96. 21/26 pack-bitmap: prepare to mark objects from multiple packs for reuseTaylor Blau, Dec 14, 2023
  97. 22/26 pack-objects: add tracing for various packfile metricsTaylor Blau, Dec 14, 2023
  98. 23/26 t/test-lib-functions.sh: implement `test_trace2_data` helperTaylor Blau, Dec 14, 2023
  99. 24/26 pack-objects: allow setting `pack.allowPackReuse` to "single"Taylor Blau, Dec 14, 2023
  100. 25/26 pack-bitmap: enable reuse from all bitmapped packsTaylor Blau, Dec 14, 2023
  101. 26/26 t/perf: add performance tests for multi-pack reuseTaylor Blau, Dec 14, 2023
  102. Junio C HamanoDec 15, 2023
  103. Taylor BlauDec 15, 2023
  104. Junio C HamanoDec 15, 2023
  105. Taylor BlauDec 15, 2023
  106. Jeff KingDec 21, 2023
  107. Taylor BlauJan 4, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.