git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] builtin/repack.c: invalidate MIDX only when necessary

From
Taylor Blau <me@ttaylorr.com>
Date
Aug 25, 2020, 17:34 UTC
Message-ID
<20200825173425.GA16844@syl.lan>
In-Reply-To
<20200825172901.GD1414394@coredump.intra.peff.net>
On Tue, Aug 25, 2020 at 01:29:01PM -0400, Jeff King wrote:
Show 27 quoted lines
> On Tue, Aug 25, 2020 at 01:10:13PM -0400, Derrick Stolee wrote:
>
> > > If we do care, though, that implies some cooperation between the
> > > deletion process and the midx code. Perhaps it even argues that repack
> > > should refuse to delete such a single pack at all, since it _isn't_
> > > redundant. It's part of a midx, and the caller should rewrite the midx
> > > first itself, and _then_ look for redundant packs.
> >
> > It is worth noting that we _do_ have a way to integrate the delete and
> > write code using 'git multi-pack-index expire'. One way to resolve this
> > atomicity would be to do the following inside the repack command:
> >
> >  1. Create and index the new pack.
> >  2. git multi-pack-index write
> >  3. git multi-pack-index expire
>
> Given that discussion elsewhere points to git-repack only really
> deleting packs in all-in-one mode (and not ever a single pack), it seems
> like we can really be much simpler here. If we're not deleting packs via
> all-in-one, there's no need to touch the midx at all. If we are, then
> it's reasonable to delete the midx immediately (after having written our
> new pack but before deleting) since our new single pack idx is as good
> or better.
>
> I.e., drop step 2 above, and make step 3 just clear_midx_file(). Which
> is roughly what the code does now, isn't it? Or is there some reason
> that "expire" is more interesting than just clearing?

It's not clear to me whether you're talking about my patch, or what a more full integration with 'git repack' looks like.

If you are talking about my patch, I disagree: checking that the MIDX doesn't know about a pack we're dropping *is* useful even without all-in-one, because of '.keep' packs (as demonstrated by the new test in my patch).

To me, this patch seems like an incremental step in the direction that we ultimately want to be going, but it's hard to untangle whether the ensuing discussion is targeted at my patch, or the ultimate goal.

Show 5 quoted lines
> And if anybody does want to drop single packs, etc, they can do so by
> generating a sensible midx separately from the repack operation (and
> probably doing so before dropping the packs for atomicity).
>
> -Peff

Thanks, Taylor

Previous: Jeff KingNext: Jeff King
Message 77 of 78 in “builtin/repack.c: invalidate MIDX only when necessary”
  1. builtin/repack.c: invalidate MIDX only when necessaryTaylor Blau, Aug 25, 2020
  2. Jeff KingAug 25, 2020
  3. Taylor BlauAug 25, 2020
  4. Derrick StoleeAug 25, 2020
  5. Taylor BlauAug 25, 2020
  6. Derrick StoleeAug 25, 2020
  7. Taylor BlauAug 25, 2020
  8. Jeff KingAug 25, 2020
  9. Junio C HamanoAug 25, 2020
  10. Taylor BlauAug 25, 2020
  11. Derrick StoleeAug 25, 2020
  12. Jeff KingAug 25, 2020
  13. Jeff KingAug 25, 2020
  14. Junio C HamanoAug 25, 2020
  15. Jeff KingAug 25, 2020
  16. pack-redundant: gauge the usage before proposing its removalJunio C Hamano, Aug 25, 2020
  17. Taylor BlauAug 25, 2020
  18. Junio C HamanoAug 25, 2020
  19. 0/3 War on dashed-gitJunio C Hamano, Aug 26, 2020
  20. 1/3 transport-helper: do not run git-remote-ext etc. in dashed formJunio C Hamano, Aug 26, 2020
  21. Eric SunshineAug 26, 2020
  22. Johannes SchindelinAug 26, 2020
  23. Junio C HamanoAug 26, 2020
  24. 2/3 cvsexportcommit: do not run git programs in dashed formJunio C Hamano, Aug 26, 2020
  25. Eric SunshineAug 26, 2020
  26. Junio C HamanoAug 26, 2020
  27. Junio C HamanoAug 26, 2020
  28. Junio C HamanoAug 26, 2020
  29. Johannes SchindelinAug 26, 2020
  30. 3/3 git: catch an attempt to run "git-foo"Junio C Hamano, Aug 26, 2020
  31. Junio C HamanoAug 26, 2020
  32. Johannes SchindelinAug 26, 2020
  33. Junio C HamanoAug 26, 2020
  34. Johannes SchindelinAug 28, 2020
  35. Junio C HamanoAug 28, 2020
  36. Johannes SchindelinAug 31, 2020
  37. Junio C HamanoAug 31, 2020
  38. Johannes SchindelinDec 20, 2020
  39. Junio C HamanoDec 21, 2020
  40. Johannes SchindelinDec 30, 2020
  41. Johannes SchindelinAug 26, 2020
  42. Junio C HamanoAug 26, 2020
  43. 0/2 avoid running "git-subcmd" in the dashed formJunio C Hamano, Aug 26, 2020
  44. 1/2 transport-helper: do not run git-remote-ext etc. in dashed formJunio C Hamano, Aug 26, 2020
  45. 2/2 cvsexportcommit: do not run git programs in dashed formJunio C Hamano, Aug 26, 2020
  46. 3/2 credential-cache: use child_process.argsJunio C Hamano, Aug 26, 2020
  47. run_command: teach API users to use embedded 'args' moreJunio C Hamano, Aug 26, 2020
  48. Jeff KingAug 27, 2020
  49. Junio C HamanoAug 27, 2020
  50. Eric SunshineAug 27, 2020
  51. Jeff KingAug 27, 2020
  52. Eric SunshineAug 27, 2020
  53. worktree: fix leak in check_clean_worktree()Jeff King, Aug 27, 2020
  54. Eric SunshineAug 27, 2020
  55. Junio C HamanoAug 27, 2020
  56. Jeff KingAug 27, 2020
  57. Jeff KingAug 27, 2020
  58. Junio C HamanoAug 27, 2020
  59. Jeff KingAug 27, 2020
  60. Junio C HamanoAug 27, 2020
  61. Junio C HamanoAug 31, 2020
  62. Jeff KingSep 1, 2020
  63. Junio C HamanoSep 1, 2020
  64. Derrick StoleeAug 27, 2020
  65. Junio C HamanoAug 27, 2020
  66. Jeff KingAug 28, 2020
  67. Junio C HamanoAug 28, 2020
  68. Son Luong NgocAug 25, 2020
  69. Derrick StoleeAug 25, 2020
  70. Taylor BlauAug 25, 2020
  71. builtin/repack.c: invalidate MIDX only when necessaryTaylor Blau, Aug 25, 2020
  72. Derrick StoleeAug 26, 2020
  73. Junio C HamanoAug 26, 2020
  74. Jeff KingAug 25, 2020
  75. Derrick StoleeAug 25, 2020
  76. Jeff KingAug 25, 2020
  77. Taylor BlauAug 25, 2020
  78. Jeff KingAug 25, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.