git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 00/11] pack-bitmap: convert offset to ref deltas where possible

From
Taylor Blau <me@ttaylorr.com>
Date
Oct 10, 2024, 20:32 UTC
Message-ID
<Zwg583RghrWdgqO1@nand.local>
In-Reply-To
<xmqq8quvk8w9.fsf@gitster.g>
On Thu, Oct 10, 2024 at 01:20:06PM -0700, Junio C Hamano wrote:
Show 26 quoted lines
> Taylor Blau <me@ttaylorr.com> writes:
>
> >> So when you pick the copy of Y out of another pack, what's so
> >> different?  After emitting Y to the resulting pack stream (and
> >> remembering where in the packstream you did so), when it is X's turn
> >> to be emitted, shouldn't you be able to compute the distance in the
> >> resulting packstream to represent X as an ofs-delta against Y, which
> >> should already be happening when you had both X and Y in the same
> >> original pack?
> >
> > Good question. The difference is that if you're reusing X and Y from
> > same pack, you know that Y occurs some number of bytes *before* X in the
> > resulting pack.
> >
> > But if Y comes from a different pack, it may get pushed further back in
> > the MIDX pseudo-pack order. So in that case the assembled pack may list
> > X before Y, in which case X cannot be an OFS_DELTA of Y, since offset
> > deltas require that the base object appears first.
>
> That is what we have always done even before we started bitmap based
> optimization.  If we happen to write Y before X, we consider doing
> ofs-delta for X, but otherwise we do ref-delta for X.  We do reorder
> fairly late in the pipeline when we notice that X that we are about
> to write out depends on Y that we haven't emitted to avoid this,
> though.  All of that the bitmap-based optimization code path should
> be able to imitate, I would think.

I see..., so yeah: before this patch series we would have not processed X via the bitmap-powered pack reuse mechanism, and after this series we will.

Thanks, Taylor

Previous: Junio C HamanoNext: Jeff King
Message 24 of 31 in “pack-bitmap: convert offset to ref deltas where possible”
  1. 00/11 pack-bitmap: convert offset to ref deltas where possibleTaylor Blau, Oct 9, 2024
  2. 01/11 pack-bitmap.c: do not pass `pack_pos` to `try_partial_reuse()`Taylor Blau, Oct 9, 2024
  3. 02/11 pack-bitmap.c: avoid unnecessary `offset_to_pack_pos()`Taylor Blau, Oct 9, 2024
  4. 03/11 pack-bitmap.c: delay calling 'offset_to_pack_pos()'Taylor Blau, Oct 9, 2024
  5. 04/11 pack-bitmap.c: compare `base_offset` to `delta_obj_offset`Taylor Blau, Oct 9, 2024
  6. 05/11 pack-bitmap.c: extract `find_base_bitmap_pos()`Taylor Blau, Oct 9, 2024
  7. 06/11 pack-bitmap: drop `from_midx` field from `bitmapped_pack`Taylor Blau, Oct 9, 2024
  8. 07/11 write_reused_pack_one(): translate bit positions directlyTaylor Blau, Oct 9, 2024
  9. Jeff KingOct 11, 2024
  10. Taylor BlauNov 4, 2024
  11. 08/11 t5332: enable OFS_DELTAs via test_pack_objects_reusedTaylor Blau, Oct 9, 2024
  12. Jeff KingOct 11, 2024
  13. Taylor BlauNov 4, 2024
  14. 09/11 pack-bitmap: enable cross-pack delta reuseTaylor Blau, Oct 9, 2024
  15. Jeff KingOct 11, 2024
  16. Taylor BlauNov 4, 2024
  17. 10/11 pack-bitmap.c: record whether the result was filteredTaylor Blau, Oct 9, 2024
  18. Jeff KingOct 11, 2024
  19. Taylor BlauNov 4, 2024
  20. 11/11 pack-bitmap: enable reusing deltas with base objects in 'haves' bitmapTaylor Blau, Oct 9, 2024
  21. Junio C HamanoOct 10, 2024
  22. Taylor BlauOct 10, 2024
  23. Junio C HamanoOct 10, 2024
  24. Taylor BlauOct 10, 2024
  25. Jeff KingOct 11, 2024
  26. Jeff KingOct 11, 2024
  27. Junio C HamanoOct 11, 2024
  28. Jeff KingOct 11, 2024
  29. Taylor BlauNov 19, 2024
  30. Taylor BlauNov 19, 2024
  31. Jeff KingDec 18, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.