git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 1/4] midx: access pack names through `nth_midxed_pack_name()`

From
Taylor Blau <me@ttaylorr.com>
Date
Jun 3, 2025, 22:22 UTC
Message-ID
<aD91ivFa2x1VBGRo@nand.local>
In-Reply-To
<xmqqr007jg9b.fsf@gitster.g>
On Thu, May 29, 2025 at 01:47:44PM -0700, Junio C Hamano wrote:
Show 29 quoted lines
> > diff --git a/pack-bitmap.c b/pack-bitmap.c
> > index b9f1d86604..8ddc150778 100644
> > --- a/pack-bitmap.c
> > +++ b/pack-bitmap.c
> > @@ -490,7 +490,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,
> >  	for (i = 0; i < bitmap_git->midx->num_packs + bitmap_git->midx->num_packs_in_base; i++) {
> >  		if (prepare_midx_pack(bitmap_repo(bitmap_git), bitmap_git->midx, i)) {
> >  			warning(_("could not open pack %s"),
> > -				bitmap_git->midx->pack_names[i]);
> > +				nth_midxed_pack_name(bitmap_git->midx, i));
>
> This loop runs from 0 to (num_packs + num_packs_in_base).  I
> understand if it runs from num_packs_in_base to (num_packs +
> num_packs_in_base), iterating only on this layer, but probably this
> just tries to open everything (i.e. in addition to num_packs we
> have, we know num_packs_in_base packs are there in our base layer(s),
> so we iterate from 0 to that number).
>
> The updated code converts the global 'i', which runs from 0 to
> "everything under us" num_packs + num_packs_in_base, to
> corresponding layer midx plus offset in it, so it looks good, but
> then, is the original reference to bitmap_git->midx->pack_names[i]
> even correct?  If we have a base, i can run larger than
> bitmap_git->midx->num_packs, which is the size of the array
> bitmap_git->midx->pack_names[].
>
> Or, unlike how the proposed log message portrayed this change as
> (i.e. code clean up), does this patch fix real bugs that manifest
> only when midx files are chained?

Right; the original code was buggy if we had a failure opening a MIDX'd pack outside of the base layer in an incremental MIDX bitmap. Reading the proposed log message again, I see what you're saying. I am happy to clarify that this is indeed a bugfix, not just a cleanup.

Show 14 quoted lines
> > @@ -2469,7 +2469,7 @@ void reuse_partial_packfile_from_bitmap(struct bitmap_index *bitmap_git,
> >  			struct bitmapped_pack pack;
> >  			if (nth_bitmapped_pack(r, bitmap_git->midx, &pack, i) < 0) {
> >  				warning(_("unable to load pack: '%s', disabling pack-reuse"),
> > -					bitmap_git->midx->pack_names[i]);
> > +					nth_midxed_pack_name(bitmap_git->midx, i));
> >  				free(packs);
> >  				return;
> >  			}
>
> Similar to the above, this is also in a loop that runs from 0 to
> num_packs+num_packs_in_base.  Is the array access to find the name
> for the error message in the original even correct when midx are
> chained?
Right; this spot suffers from the same bug as the previous hunk.
Show 20 quoted lines
> > diff --git a/t/helper/test-read-midx.c b/t/helper/test-read-midx.c
> > index ac81390899..fbed0f6919 100644
> > --- a/t/helper/test-read-midx.c
> > +++ b/t/helper/test-read-midx.c
> > @@ -53,8 +53,9 @@ static int read_midx_file(const char *object_dir, const char *checksum,
> >  	printf("\nnum_objects: %d\n", m->num_objects);
> >
> >  	printf("packs:\n");
> > -	for (i = 0; i < m->num_packs; i++)
> > -		printf("%s\n", m->pack_names[i]);
> > +	for (i = m->num_packs_in_base; i < m->num_packs + m->num_packs_in_base;
> > +	     i++)
> > +		printf("%s\n", nth_midxed_pack_name(m, i));
>
> OK.  This used to iterate from 0 to num_packs using the local
> offset.  Now it iterates from num_packs_in_base to num_packs_in_base+num_packs,
> meaning we iterate over packs in the given midx.  No change in
> behaviour, as accesses to m->pack_names[i] using the local offset in
> the original was correct, and the updated code iterates using the
> global offset.  This is not a bugfix but is a code cleanup.
Right.
Show 8 quoted lines
> > @@ -108,7 +109,7 @@ static int read_midx_preferred_pack(const char *object_dir)
> >  		return 1;
> >  	}
> >
> > -	printf("%s\n", midx->pack_names[preferred_pack]);
> > +	printf("%s\n", nth_midxed_pack_name(midx, preferred_pack));
>
> Again, is the original buggy when midx are chained?
This is also a bugfix.

Thanks, Taylor

Previous: Junio C HamanoNext: Junio C Hamano
Message 38 of 53 in “packfile: avoid access(3p) calls for missing packs”
  1. packfile: avoid access(3p) calls for missing packsPatrick Steinhardt, May 16, 2025
  2. Junio C HamanoMay 16, 2025
  3. Jeff KingMay 19, 2025
  4. Junio C HamanoMay 19, 2025
  5. Patrick SteinhardtMay 20, 2025
  6. Jeff KingMay 22, 2025
  7. Taylor BlauMay 23, 2025
  8. Jeff KingMay 23, 2025
  9. 0/2 packfile: avoid access(3p) calls for missing packsPatrick Steinhardt, May 20, 2025
  10. 1/2 packfile: explain ordering of how we look up auxiliary pack filesPatrick Steinhardt, May 20, 2025
  11. Taylor BlauMay 23, 2025
  12. 2/2 midx: stop repeatedly looking up nonexistent packfilesPatrick Steinhardt, May 20, 2025
  13. Jeff KingMay 22, 2025
  14. Junio C HamanoMay 22, 2025
  15. Jeff KingMay 22, 2025
  16. Junio C HamanoMay 22, 2025
  17. Taylor BlauMay 23, 2025
  18. Jeff KingMay 23, 2025
  19. Taylor BlauMay 23, 2025
  20. 0/5 midx: improve prepare_midx_pack() ergonomicsTaylor Blau, May 25, 2025
  21. 1/5 pack-bitmap.c: fix broken warning() when missing MIDX'd packTaylor Blau, May 25, 2025
  22. Patrick SteinhardtMay 26, 2025
  23. Taylor BlauMay 28, 2025
  24. 2/5 midx-write.c: guard against incremental MIDXs in want_included_pack()Taylor Blau, May 25, 2025
  25. Patrick SteinhardtMay 26, 2025
  26. Taylor BlauMay 28, 2025
  27. 3/5 midx-write.c: simplify fill_packs_from_midx()Taylor Blau, May 25, 2025
  28. Patrick SteinhardtMay 26, 2025
  29. Taylor BlauMay 28, 2025
  30. 4/5 midx-write.c: extract inner loop from fill_packs_from_midx()Taylor Blau, May 25, 2025
  31. 5/5 midx: return a `packed_git` pointer from `prepare_midx_pack()`Taylor Blau, May 25, 2025
  32. Patrick SteinhardtMay 26, 2025
  33. Taylor BlauMay 28, 2025
  34. Patrick SteinhardtMay 28, 2025
  35. 0/4 midx: improve prepare_midx_pack() ergonomicsTaylor Blau, May 28, 2025
  36. 1/4 midx: access pack names through `nth_midxed_pack_name()`Taylor Blau, May 28, 2025
  37. Junio C HamanoMay 29, 2025
  38. Taylor BlauJun 3, 2025
  39. Junio C HamanoMay 29, 2025
  40. Taylor BlauJun 3, 2025
  41. 2/4 midx-write.c: guard against incremental MIDXs in want_included_pack()Taylor Blau, May 28, 2025
  42. 3/4 midx-write.c: extract inner loop from fill_packs_from_midx()Taylor Blau, May 28, 2025
  43. 4/4 midx: return a `packed_git` pointer from `prepare_midx_pack()`Taylor Blau, May 28, 2025
  44. Jeff KingMay 30, 2025
  45. Taylor BlauJun 3, 2025
  46. Junio C HamanoAug 28, 2025
  47. Taylor BlauMay 23, 2025
  48. Jeff KingMay 23, 2025
  49. Junio C HamanoMay 21, 2025
  50. 0/2 packfile: avoid access(3p) calls for missing packsPatrick Steinhardt, May 28, 2025
  51. 2/2 midx: stop repeatedly looking up nonexistent packfilesPatrick Steinhardt, May 28, 2025
  52. 1/2 packfile: explain ordering of how we look up auxiliary pack filesPatrick Steinhardt, May 28, 2025
  53. Jeff KingMay 30, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.