git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 3/4] builtin/pack-objects.c: ensure included `--stdin-packs` exist

From
Taylor Blau <me@ttaylorr.com>
Date
May 25, 2022, 00:14 UTC
Message-ID
<Yo10/KlseGJeY6uQ@nand.local>
In-Reply-To
<xmqq1qwifv1c.fsf@gitster.g>
On Tue, May 24, 2022 at 03:03:11PM -0700, Junio C Hamano wrote:
Show 7 quoted lines
> Taylor Blau <me@ttaylorr.com> writes:
>
> > Calling `is_pack_valid()` early on makes it substantially less likely
> > that we will have to deal with a pack going away, since we'll have an
> > open file descriptor on its contents much earlier.
>
> Sorry for asking a stupid question (or two), but I am confused.

No such thing as a stupid question, so your apology is not necessary in the slightest :).

Show 20 quoted lines
> This does make sure that we can read and use the contents of the
> packfile even when somebody else removes it from the disk by
> ensuring that
>
>  (1) we have an open file descriptor to it, so that we could open
>      mmap window into it at will; or
>
>  (2) we have a mmap window that covers all of it (this should be the
>      norm on platforms with vast address space); or
>
>  (3) we are in the same state as (1) by opening the packfile to
>      validate the pack right now.
>
> and during the pack-object we are running (aka "repack"), we can
> continue to read from that pack that may have already disappeared
> from the disk.
>
> But is that sufficient?  Are we writing the resulting new pack(s)
> out in such a way that the repository is healthy without the pack
> we noticed is disappearing?  How do we ensure that?

It's sufficient in the sense that we're writing out all of the objects we were asked to (from pack-objects's perspective). Of course, if the "simultaneous writer" is just removing packs right after they are opened, that will produce an obviously-broken state. But assuming that repack isn't removing objects it shouldn't (which I think is a safe assumption from pack-objects' perspective, since all it cares about is writing packs that contain the desired set of objects), then we are OK.

Thanks, Taylor

Previous: Junio C HamanoNext: Victoria Dye
Message 24 of 29 in “pack-objects: fix a pair of MIDX bitmap-related races”
  1. 0/2 pack-objects: fix a pair of MIDX bitmap-related racesTaylor Blau, May 13, 2022
  2. 1/2 pack-bitmap: check preferred pack validity when opening MIDX bitmapTaylor Blau, May 13, 2022
  3. Junio C HamanoMay 13, 2022
  4. Taylor BlauMay 13, 2022
  5. 2/2 builtin/pack-objects.c: ensure pack validity from MIDX bitmap objectsTaylor Blau, May 13, 2022
  6. Jonathan TanMay 13, 2022
  7. Taylor BlauMay 14, 2022
  8. Jonathan TanMay 16, 2022
  9. Taylor BlauMay 14, 2022
  10. Jonathan TanMay 16, 2022
  11. 0/4 pack-objects: fix a pair of MIDX bitmap-related racesTaylor Blau, May 24, 2022
  12. 1/4 pack-bitmap.c: check preferred pack validity when opening MIDX bitmapTaylor Blau, May 24, 2022
  13. Ævar Arnfjörð BjarmasonMay 24, 2022
  14. Taylor BlauMay 24, 2022
  15. Ævar Arnfjörð BjarmasonMay 24, 2022
  16. 2/4 builtin/pack-objects.c: avoid redundant NULL checkTaylor Blau, May 24, 2022
  17. Junio C HamanoMay 24, 2022
  18. Taylor BlauMay 25, 2022
  19. 3/4 builtin/pack-objects.c: ensure included `--stdin-packs` existTaylor Blau, May 24, 2022
  20. Ævar Arnfjörð BjarmasonMay 24, 2022
  21. Taylor BlauMay 24, 2022
  22. Ævar Arnfjörð BjarmasonMay 24, 2022
  23. Junio C HamanoMay 24, 2022
  24. Taylor BlauMay 25, 2022
  25. Victoria DyeMay 26, 2022
  26. Taylor BlauMay 26, 2022
  27. 4/4 builtin/pack-objects.c: ensure pack validity from MIDX bitmap objectsTaylor Blau, May 24, 2022
  28. Junio C HamanoMay 24, 2022
  29. Taylor BlauMay 25, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.