git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 0/4] align the behavior when opening "packed-refs"

From
shejialuo <shejialuo@gmail.com>
Date
May 9, 2025, 15:26 UTC
Message-ID
<aB4ewBsxnq0Yv3Fd@ArchLinux>
In-Reply-To
<20250508203350.GG18229@coredump.intra.peff.net>
On Thu, May 08, 2025 at 04:33:50PM -0400, Jeff King wrote:
Show 14 quoted lines
> On Thu, May 08, 2025 at 01:20:54PM -0700, Junio C Hamano wrote:
> 
> > > I left a few comments that I think bear addressing (or at least some
> > > discussion).
> > 
> > I found both of your points very good ones, especially the "why are
> > we making an in-core copy anyway later?"
> > 
> > Which may probably mean that munmap_temporary_snapshot() is not the
> > helper function we want in the code path, so one of the preliminary
> > refactoring patches can be removed.
> 
> Yep.
> 
Right, I will remove this patch in the next version.
Show 22 quoted lines
> > Even on mmap-incapable platforms, we have enough emulation in
> > git_mmap() and git_munmap(), and this code path that wants to read a
> > packed-refs file just mmap(), do its thing, and then munmap(),
> > without worrying anything about "ah, temporary, so we need to make
> > an in-core copy for ourselves".
> 
> Hmm, yeah. I had thought that somebody could explicitly set
> mmap_strategy themselves via config, in which case we'd want to avoid
> calling git_mmap() on systems where it actually does mmap. But it
> doesn't look like we have any mechanism for setting the config. So
> MMAP_NONE happens only when NO_MMAP is set.
> 
>   I briefly wondered if the existing code could be simplified to get rid
>   of MMAP_NONE, since it could also rely on git_mmap() to make an
>   in-memory copy. But it gets weird with MMAP_PREVENTS_DELETE and
>   NO_MMAP combined, since then we make a pointless extra copy (one to
>   fake-mmap, and one into the new buffer). OTOH, I'd expect those to be
>   mutually exclusive.
> 
>   I kind of wonder with MMAP_TEMPORARY why we bother mapping at all and
>   not just reading into a buffer. Maybe it's more efficient?
> 

I am also confused about this. If we eventually would allocate a buffer, it is wired that we map in the first place.

>   Probably not worth revisiting all of this old code, though. And
>   anyway, because of the SMALL_FILE_SIZE check, we couldn't even
>   simplify away the read_in_full() code.
> 

Right, I will try to clean the code later. At now, I think we need to concentrate on using `mmap`. I will add this into my TODOs. Thanks for the suggestion.

> -Peff

Thanks, Jialuo

Previous: Jeff KingNext: shejialuo
Message 33 of 67 in “align the behavior when opening "packed-refs"”
  1. 0/4 align the behavior when opening "packed-refs"shejialuo, May 6, 2025
  2. 1/4 packed-backend: skip checking consistency of empty packed-refs fileshejialuo, May 6, 2025
  3. Junio C HamanoMay 6, 2025
  4. shejialuoMay 7, 2025
  5. Junio C HamanoMay 6, 2025
  6. shejialuoMay 7, 2025
  7. 2/4 packed-backend: extract snapshot allocation in `load_contents`shejialuo, May 6, 2025
  8. Junio C HamanoMay 6, 2025
  9. 3/4 packed-backend: extract munmap operation for `MMAP_TEMPORARY`shejialuo, May 6, 2025
  10. Junio C HamanoMay 6, 2025
  11. Junio C HamanoMay 6, 2025
  12. shejialuoMay 7, 2025
  13. 4/4 packed-backend: use mmap when opening large "packed-refs" fileshejialuo, May 6, 2025
  14. Junio C HamanoMay 6, 2025
  15. Junio C HamanoMay 6, 2025
  16. shejialuoMay 7, 2025
  17. 0/4 align the behavior when opening "packed-refs"shejialuo, May 7, 2025
  18. 1/4 packed-backend: fsck should allow an empty "packed-refs" fileshejialuo, May 7, 2025
  19. 2/4 packed-backend: extract snapshot allocation in `load_contents`shejialuo, May 7, 2025
  20. 3/4 packed-backend: extract munmap operation for `MMAP_TEMPORARY`shejialuo, May 7, 2025
  21. Jeff KingMay 8, 2025
  22. Junio C HamanoMay 8, 2025
  23. shejialuoMay 9, 2025
  24. 4/4 packed-backend: mmap large "packed-refs" file during fsckshejialuo, May 7, 2025
  25. Jeff KingMay 8, 2025
  26. shejialuoMay 9, 2025
  27. Jeff KingMay 9, 2025
  28. shejialuoMay 9, 2025
  29. Junio C HamanoMay 7, 2025
  30. Jeff KingMay 8, 2025
  31. Junio C HamanoMay 8, 2025
  32. Jeff KingMay 8, 2025
  33. shejialuoMay 9, 2025
  34. 0/3 align the behavior when opening "packed-refs"shejialuo, May 11, 2025
  35. 1/3 packed-backend: fsck should allow an empty "packed-refs" fileshejialuo, May 11, 2025
  36. Patrick SteinhardtMay 12, 2025
  37. shejialuoMay 12, 2025
  38. Patrick SteinhardtMay 12, 2025
  39. Jeff KingMay 12, 2025
  40. Junio C HamanoMay 12, 2025
  41. Patrick SteinhardtMay 13, 2025
  42. shejialuoMay 13, 2025
  43. 2/3 packed-backend: extract snapshot allocation in `load_contents`shejialuo, May 11, 2025
  44. Patrick SteinhardtMay 12, 2025
  45. shejialuoMay 12, 2025
  46. Patrick SteinhardtMay 12, 2025
  47. Jeff KingMay 12, 2025
  48. shejialuoMay 13, 2025
  49. 3/3 packed-backend: mmap large "packed-refs" file during fsckshejialuo, May 11, 2025
  50. Jeff KingMay 12, 2025
  51. 0/3 align the behavior when opening "packed-refs"shejialuo, May 13, 2025
  52. 1/3 packed-backend: fsck should warn when "packed-refs" file is emptyshejialuo, May 13, 2025
  53. Junio C HamanoMay 13, 2025
  54. shejialuoMay 14, 2025
  55. 3/3 packed-backend: mmap large "packed-refs" file during fsckshejialuo, May 13, 2025
  56. Junio C HamanoMay 13, 2025
  57. shejialuoMay 14, 2025
  58. 2/3 packed-backend: extract snapshot allocation in `load_contents`shejialuo, May 13, 2025
  59. 0/3 align the behavior when opening "packed-refs"shejialuo, May 14, 2025
  60. 2/3 packed-backend: extract snapshot allocation in `load_contents`shejialuo, May 14, 2025
  61. 1/3 packed-backend: fsck should warn when "packed-refs" file is emptyshejialuo, May 14, 2025
  62. 3/3 packed-backend: mmap large "packed-refs" file during fsckshejialuo, May 14, 2025
  63. Junio C HamanoMay 15, 2025
  64. Junio C HamanoMay 21, 2025
  65. Jeff KingMay 22, 2025
  66. Patrick SteinhardtMay 23, 2025
  67. Junio C HamanoMay 23, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.