git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] repack: fix geometric repacking with gitalternates

From
Patrick Steinhardt <ps@pks.im>
Date
Apr 11, 2023, 17:13 UTC
Message-ID
<ZDWVSuFJLU54txwc@ncase>
In-Reply-To
<ZDSgbYa+j/5c5t8j@nand.local>
On Mon, Apr 10, 2023 at 07:49:01PM -0400, Taylor Blau wrote:
> On Mon, Apr 10, 2023 at 11:06:49AM -0400, Derrick Stolee wrote:
[snip]
Show 47 quoted lines
> > Perhaps this method could include a step to create a new, "local"
> > geometry containing only the packs within the local object dir. We can
> > then skip the --preferred-pack option and bitmap if this is different
> > than the original geometry (perhaps with a warning message to help
> > users who did this accidentally).
> 
> It would be nice to not have to juggle multiple pack geometry structs,
> since the logic of what gets repacked, what gets thrown away, and what
> gets kept is already fairly complicated (at least to me) and pretty
> fragile.
> 
> I think if I were sketching this out, I'd start out by doing something
> like:
> 
> --- 8< ---
> diff --git a/builtin/repack.c b/builtin/repack.c
> index df4d8e0f0b..eab5f58444 100644
> --- a/builtin/repack.c
> +++ b/builtin/repack.c
> @@ -558,6 +558,10 @@ static void midx_included_packs(struct string_list *include,
>  		for (i = geometry->split; i < geometry->pack_nr; i++) {
>  			struct packed_git *p = geometry->pack[i];
> 
> +			/* MIDXs cannot refer to non-local packs */
> +			if (!p->pack_local)
> +				continue;
> +
>  			strbuf_addstr(&buf, pack_basename(p));
>  			strbuf_strip_suffix(&buf, ".pack");
>  			strbuf_addstr(&buf, ".idx");
> --- >8 ---
> 
> ...and I actually think that might do it, since:
> 
> 	- existing_nonkept_packs is populated by calling readdir() on the local
> 		repository's $GIT_DIR/objects/pack directory, so it will never contain
> 		any non-local packs.
> 
> 	- existing_kept_packs is also OK for the same reason
> 
> 	- names (which tracks the packs that we just wrote) will never contain a
> 		non-local pack, since we never write packs outside of our local pack
> 		directory
> 
> So that would cause you to write a MIDX containing only local packs (as
> desired) regardless of whether or not the caller passed --[no]-local or
> not.

I think this doesn't actually do anything. The reason is that the `--stdin-packs` option in git-multi-pack-index(1) does not actually treat the provided packs as packs that are to be included in the multi-pack-index. Instead, it uses it as a filter for `get_all_packs()`. And as `get_all_packs()` returns packfiles paths prefixed with the alternate directory's name, but we pass in basenames only, we would already be filtering out any non-local packfiles.

So ultimately, we would only ever write an MIDX containing only local packs already. It rather feels like this is only by chance though, so I think it is good to include your patch regardless of whether it actually does something or not. Better be explicit here, also as documentation to the reader.

I'll include it as part of my patch series that I'm to send out later this week.

Patrick
Previous: Taylor BlauNext: Taylor Blau
Message 7 of 67 in “repack: fix geometric repacking with gitalternates”
  1. repack: fix geometric repacking with gitalternatesPatrick Steinhardt, Apr 4, 2023
  2. Taylor BlauApr 4, 2023
  3. Taylor BlauApr 4, 2023
  4. Patrick SteinhardtApr 5, 2023
  5. Derrick StoleeApr 10, 2023
  6. Taylor BlauApr 10, 2023
  7. Patrick SteinhardtApr 11, 2023
  8. Taylor BlauApr 11, 2023
  9. Patrick SteinhardtApr 12, 2023
  10. Patrick SteinhardtApr 11, 2023
  11. Patrick SteinhardtApr 11, 2023
  12. Taylor BlauApr 11, 2023
  13. Taylor BlauApr 10, 2023
  14. 0/8 repack: fix geometric repacking with gitalternatesPatrick Steinhardt, Apr 12, 2023
  15. 1/8 midx: fix segfault with no packs and invalid preferred packPatrick Steinhardt, Apr 12, 2023
  16. Taylor BlauApr 12, 2023
  17. Patrick SteinhardtApr 13, 2023
  18. 2/8 repack: fix trying to use preferred pack in alternatesPatrick Steinhardt, Apr 12, 2023
  19. Taylor BlauApr 12, 2023
  20. Patrick SteinhardtApr 13, 2023
  21. 3/8 repack: fix generating multi-pack-index with only non-local packsPatrick Steinhardt, Apr 12, 2023
  22. Taylor BlauApr 12, 2023
  23. 4/8 pack-objects: fix error when packing same pack twicePatrick Steinhardt, Apr 12, 2023
  24. Taylor BlauApr 12, 2023
  25. 5/8 pack-objects: fix error when same packfile is included and excludedPatrick Steinhardt, Apr 12, 2023
  26. Taylor BlauApr 12, 2023
  27. 6/8 pack-objects: extend test coverage of `--stdin-packs` with alternatesPatrick Steinhardt, Apr 12, 2023
  28. 7/8 repack: honor `-l` when calculating pack geometryPatrick Steinhardt, Apr 12, 2023
  29. Junio C HamanoApr 12, 2023
  30. Junio C HamanoApr 13, 2023
  31. Patrick SteinhardtApr 13, 2023
  32. 8/8 repack: disable writing bitmaps when doing a local geometric repackPatrick Steinhardt, Apr 12, 2023
  33. Taylor BlauApr 12, 2023
  34. Patrick SteinhardtApr 13, 2023
  35. Patrick SteinhardtApr 13, 2023
  36. Taylor BlauApr 12, 2023
  37. 00/10 repack: fix geometric repacking with gitalternatesPatrick Steinhardt, Apr 13, 2023
  38. 01/10 midx: fix segfault with no packs and invalid preferred packPatrick Steinhardt, Apr 13, 2023
  39. Derrick StoleeApr 13, 2023
  40. 02/10 repack: fix trying to use preferred pack in alternatesPatrick Steinhardt, Apr 13, 2023
  41. 03/10 repack: fix generating multi-pack-index with only non-local packsPatrick Steinhardt, Apr 13, 2023
  42. 04/10 pack-objects: split out `--stdin-packs` tests into separate filePatrick Steinhardt, Apr 13, 2023
  43. 05/10 pack-objects: fix error when packing same pack twicePatrick Steinhardt, Apr 13, 2023
  44. 06/10 pack-objects: fix error when same packfile is included and excludedPatrick Steinhardt, Apr 13, 2023
  45. 07/10 pack-objects: extend test coverage of `--stdin-packs` with alternatesPatrick Steinhardt, Apr 13, 2023
  46. 08/10 t/helper: allow chmtime to print verbosely without modifying mtimePatrick Steinhardt, Apr 13, 2023
  47. 09/10 repack: honor `-l` when calculating pack geometryPatrick Steinhardt, Apr 13, 2023
  48. Derrick StoleeApr 13, 2023
  49. Patrick SteinhardtApr 13, 2023
  50. Junio C HamanoApr 13, 2023
  51. 10/10 repack: disable writing bitmaps when doing a local repackPatrick Steinhardt, Apr 13, 2023
  52. Derrick StoleeApr 13, 2023
  53. Junio C HamanoApr 14, 2023
  54. Patrick SteinhardtApr 14, 2023
  55. 00/10 repack: fix geometric repacking with gitalternatesPatrick Steinhardt, Apr 14, 2023
  56. 01/10 midx: fix segfault with no packs and invalid preferred packPatrick Steinhardt, Apr 14, 2023
  57. 03/10 repack: fix generating multi-pack-index with only non-local packsPatrick Steinhardt, Apr 14, 2023
  58. 02/10 repack: fix trying to use preferred pack in alternatesPatrick Steinhardt, Apr 14, 2023
  59. 04/10 pack-objects: split out `--stdin-packs` tests into separate filePatrick Steinhardt, Apr 14, 2023
  60. 05/10 pack-objects: fix error when packing same pack twicePatrick Steinhardt, Apr 14, 2023
  61. 06/10 pack-objects: fix error when same packfile is included and excludedPatrick Steinhardt, Apr 14, 2023
  62. 07/10 pack-objects: extend test coverage of `--stdin-packs` with alternatesPatrick Steinhardt, Apr 14, 2023
  63. 08/10 t/helper: allow chmtime to print verbosely without modifying mtimePatrick Steinhardt, Apr 14, 2023
  64. 09/10 repack: honor `-l` when calculating pack geometryPatrick Steinhardt, Apr 14, 2023
  65. 10/10 repack: disable writing bitmaps when doing a local repackPatrick Steinhardt, Apr 14, 2023
  66. Derrick StoleeApr 14, 2023
  67. Junio C HamanoApr 14, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.