git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] repack: fix geometric repacking with gitalternates

From
Patrick Steinhardt <ps@pks.im>
Date
Apr 11, 2023, 17:26 UTC
Message-ID
<ZDWYLPw4Hk0-12Fe@ncase>
In-Reply-To
<ZDWTp2RfFTbiE9I9@ncase>
On Tue, Apr 11, 2023 at 07:06:48PM +0200, Patrick Steinhardt wrote:
Show 35 quoted lines
> On Mon, Apr 10, 2023 at 11:06:49AM -0400, Derrick Stolee wrote:
> > On 4/5/2023 3:08 AM, Patrick Steinhardt wrote:
> > > On Tue, Apr 04, 2023 at 02:55:50PM -0400, Taylor Blau wrote:
> > >> On Tue, Apr 04, 2023 at 01:08:33PM +0200, Patrick Steinhardt wrote:
> [snip]
> > > I'd personally be fine to start honoring the `po_args.local` flag so
> > > that we skip over any non-local packfiles there while ignoring the
> > > larger problem of non-local geometric repacks breaking in certain
> > > scenarios. It would at least improve the status quo as users now have a
> > > way out in case they ever happen to hit that error. And it allows us to
> > > use geometric repacks in alternated repositories. But are we okay with
> > > punting on the larger issue for the time being?
> > 
> > I think the real bug is isolated in write_midx_included_packs() in how
> > it may specify packs that the multi-pack-index cannot track. It should
> > be worth the time exploring if there is an easy fix there, and then the
> > po_args.local version can be used as a backup/performance tweak on top
> > of that.
> 
> The problems are quite multi-faceted, but taken on their own most of the
> problems actually seem quite easy to fix. I've got a local patch series
> that addresses almost all of the pain points I have found until now:
> 
> - A segfault in git-multi-pack-index(1) when passed no packfiles and a
>   preferred packfile that it cannot find.
> 
> - The issue that git-repack(1) asks git-multi-pack-index(1) to write an
>   MIDX with packs that it cannot actually track because they are not
>   local.
> 
> - An issue with git-pack-objects(1) that keeps it from packing objects
>   that are non-local due to the way `--stdin-packs` is only referring to
>   the packfile basenames.
> 
> - The fact that we don't honor `-l` when doing geometric repacking.

Ah, one more thing I forgot: add a safety mechanism that disables writing bitmaps when we see non-local packs.

Patrick
Previous: Patrick SteinhardtNext: Taylor Blau
Message 11 of 67 in “repack: fix geometric repacking with gitalternates”
  1. repack: fix geometric repacking with gitalternatesPatrick Steinhardt, Apr 4, 2023
  2. Taylor BlauApr 4, 2023
  3. Taylor BlauApr 4, 2023
  4. Patrick SteinhardtApr 5, 2023
  5. Derrick StoleeApr 10, 2023
  6. Taylor BlauApr 10, 2023
  7. Patrick SteinhardtApr 11, 2023
  8. Taylor BlauApr 11, 2023
  9. Patrick SteinhardtApr 12, 2023
  10. Patrick SteinhardtApr 11, 2023
  11. Patrick SteinhardtApr 11, 2023
  12. Taylor BlauApr 11, 2023
  13. Taylor BlauApr 10, 2023
  14. 0/8 repack: fix geometric repacking with gitalternatesPatrick Steinhardt, Apr 12, 2023
  15. 1/8 midx: fix segfault with no packs and invalid preferred packPatrick Steinhardt, Apr 12, 2023
  16. Taylor BlauApr 12, 2023
  17. Patrick SteinhardtApr 13, 2023
  18. 2/8 repack: fix trying to use preferred pack in alternatesPatrick Steinhardt, Apr 12, 2023
  19. Taylor BlauApr 12, 2023
  20. Patrick SteinhardtApr 13, 2023
  21. 3/8 repack: fix generating multi-pack-index with only non-local packsPatrick Steinhardt, Apr 12, 2023
  22. Taylor BlauApr 12, 2023
  23. 4/8 pack-objects: fix error when packing same pack twicePatrick Steinhardt, Apr 12, 2023
  24. Taylor BlauApr 12, 2023
  25. 5/8 pack-objects: fix error when same packfile is included and excludedPatrick Steinhardt, Apr 12, 2023
  26. Taylor BlauApr 12, 2023
  27. 6/8 pack-objects: extend test coverage of `--stdin-packs` with alternatesPatrick Steinhardt, Apr 12, 2023
  28. 7/8 repack: honor `-l` when calculating pack geometryPatrick Steinhardt, Apr 12, 2023
  29. Junio C HamanoApr 12, 2023
  30. Junio C HamanoApr 13, 2023
  31. Patrick SteinhardtApr 13, 2023
  32. 8/8 repack: disable writing bitmaps when doing a local geometric repackPatrick Steinhardt, Apr 12, 2023
  33. Taylor BlauApr 12, 2023
  34. Patrick SteinhardtApr 13, 2023
  35. Patrick SteinhardtApr 13, 2023
  36. Taylor BlauApr 12, 2023
  37. 00/10 repack: fix geometric repacking with gitalternatesPatrick Steinhardt, Apr 13, 2023
  38. 01/10 midx: fix segfault with no packs and invalid preferred packPatrick Steinhardt, Apr 13, 2023
  39. Derrick StoleeApr 13, 2023
  40. 02/10 repack: fix trying to use preferred pack in alternatesPatrick Steinhardt, Apr 13, 2023
  41. 03/10 repack: fix generating multi-pack-index with only non-local packsPatrick Steinhardt, Apr 13, 2023
  42. 04/10 pack-objects: split out `--stdin-packs` tests into separate filePatrick Steinhardt, Apr 13, 2023
  43. 05/10 pack-objects: fix error when packing same pack twicePatrick Steinhardt, Apr 13, 2023
  44. 06/10 pack-objects: fix error when same packfile is included and excludedPatrick Steinhardt, Apr 13, 2023
  45. 07/10 pack-objects: extend test coverage of `--stdin-packs` with alternatesPatrick Steinhardt, Apr 13, 2023
  46. 08/10 t/helper: allow chmtime to print verbosely without modifying mtimePatrick Steinhardt, Apr 13, 2023
  47. 09/10 repack: honor `-l` when calculating pack geometryPatrick Steinhardt, Apr 13, 2023
  48. Derrick StoleeApr 13, 2023
  49. Patrick SteinhardtApr 13, 2023
  50. Junio C HamanoApr 13, 2023
  51. 10/10 repack: disable writing bitmaps when doing a local repackPatrick Steinhardt, Apr 13, 2023
  52. Derrick StoleeApr 13, 2023
  53. Junio C HamanoApr 14, 2023
  54. Patrick SteinhardtApr 14, 2023
  55. 00/10 repack: fix geometric repacking with gitalternatesPatrick Steinhardt, Apr 14, 2023
  56. 01/10 midx: fix segfault with no packs and invalid preferred packPatrick Steinhardt, Apr 14, 2023
  57. 03/10 repack: fix generating multi-pack-index with only non-local packsPatrick Steinhardt, Apr 14, 2023
  58. 02/10 repack: fix trying to use preferred pack in alternatesPatrick Steinhardt, Apr 14, 2023
  59. 04/10 pack-objects: split out `--stdin-packs` tests into separate filePatrick Steinhardt, Apr 14, 2023
  60. 05/10 pack-objects: fix error when packing same pack twicePatrick Steinhardt, Apr 14, 2023
  61. 06/10 pack-objects: fix error when same packfile is included and excludedPatrick Steinhardt, Apr 14, 2023
  62. 07/10 pack-objects: extend test coverage of `--stdin-packs` with alternatesPatrick Steinhardt, Apr 14, 2023
  63. 08/10 t/helper: allow chmtime to print verbosely without modifying mtimePatrick Steinhardt, Apr 14, 2023
  64. 09/10 repack: honor `-l` when calculating pack geometryPatrick Steinhardt, Apr 14, 2023
  65. 10/10 repack: disable writing bitmaps when doing a local repackPatrick Steinhardt, Apr 14, 2023
  66. Derrick StoleeApr 14, 2023
  67. Junio C HamanoApr 14, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.