git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 3/3] packed-backend: mmap large "packed-refs" file during fsck

From
Jeff King <peff@peff.net>
Date
May 12, 2025, 13:08 UTC
Message-ID
<20250512130815.GC1191360@coredump.intra.peff.net>
In-Reply-To
<aCCt1zJ2yviOz--l@ArchLinux>
On Sun, May 11, 2025 at 10:01:59PM +0800, shejialuo wrote:
Show 22 quoted lines
> @@ -2113,24 +2113,19 @@ static int packed_fsck(struct ref_store *ref_store,
>  		goto cleanup;
>  	}
>  
> -	if (!st.st_size)
> +	if (!allocate_snapshot_buffer(&snapshot, fd, &st))
>  		goto cleanup;
>  
> -	if (strbuf_read(&packed_ref_content, fd, 0) < 0) {
> -		ret = error_errno(_("unable to read '%s'"), refs->path);
> -		goto cleanup;
> -	}
> -
> -	ret = packed_fsck_ref_content(o, ref_store, &sorted, packed_ref_content.buf,
> -				      packed_ref_content.buf + packed_ref_content.len);
> +	ret = packed_fsck_ref_content(o, ref_store, &sorted, snapshot.start,
> +				      snapshot.eof);
>  	if (!ret && sorted)
> -		ret = packed_fsck_ref_sorted(o, ref_store, packed_ref_content.buf,
> -					     packed_ref_content.buf + packed_ref_content.len);
> +		ret = packed_fsck_ref_sorted(o, ref_store, snapshot.start,
> +					     snapshot.eof);

OK, so we still use allocate_snapshot_buffer(), but then we hold on to the snapshot. Good.

I do think just using xmmap() would have been sufficient (and not needed patch 2 then), but I'm OK with this direction under the logic that we may end up sharing more code between the normal and fsck code paths eventually.

-Peff
Previous: shejialuoNext: shejialuo
Message 50 of 67 in “align the behavior when opening "packed-refs"”
  1. 0/4 align the behavior when opening "packed-refs"shejialuo, May 6, 2025
  2. 1/4 packed-backend: skip checking consistency of empty packed-refs fileshejialuo, May 6, 2025
  3. Junio C HamanoMay 6, 2025
  4. shejialuoMay 7, 2025
  5. Junio C HamanoMay 6, 2025
  6. shejialuoMay 7, 2025
  7. 2/4 packed-backend: extract snapshot allocation in `load_contents`shejialuo, May 6, 2025
  8. Junio C HamanoMay 6, 2025
  9. 3/4 packed-backend: extract munmap operation for `MMAP_TEMPORARY`shejialuo, May 6, 2025
  10. Junio C HamanoMay 6, 2025
  11. Junio C HamanoMay 6, 2025
  12. shejialuoMay 7, 2025
  13. 4/4 packed-backend: use mmap when opening large "packed-refs" fileshejialuo, May 6, 2025
  14. Junio C HamanoMay 6, 2025
  15. Junio C HamanoMay 6, 2025
  16. shejialuoMay 7, 2025
  17. 0/4 align the behavior when opening "packed-refs"shejialuo, May 7, 2025
  18. 1/4 packed-backend: fsck should allow an empty "packed-refs" fileshejialuo, May 7, 2025
  19. 2/4 packed-backend: extract snapshot allocation in `load_contents`shejialuo, May 7, 2025
  20. 3/4 packed-backend: extract munmap operation for `MMAP_TEMPORARY`shejialuo, May 7, 2025
  21. Jeff KingMay 8, 2025
  22. Junio C HamanoMay 8, 2025
  23. shejialuoMay 9, 2025
  24. 4/4 packed-backend: mmap large "packed-refs" file during fsckshejialuo, May 7, 2025
  25. Jeff KingMay 8, 2025
  26. shejialuoMay 9, 2025
  27. Jeff KingMay 9, 2025
  28. shejialuoMay 9, 2025
  29. Junio C HamanoMay 7, 2025
  30. Jeff KingMay 8, 2025
  31. Junio C HamanoMay 8, 2025
  32. Jeff KingMay 8, 2025
  33. shejialuoMay 9, 2025
  34. 0/3 align the behavior when opening "packed-refs"shejialuo, May 11, 2025
  35. 1/3 packed-backend: fsck should allow an empty "packed-refs" fileshejialuo, May 11, 2025
  36. Patrick SteinhardtMay 12, 2025
  37. shejialuoMay 12, 2025
  38. Patrick SteinhardtMay 12, 2025
  39. Jeff KingMay 12, 2025
  40. Junio C HamanoMay 12, 2025
  41. Patrick SteinhardtMay 13, 2025
  42. shejialuoMay 13, 2025
  43. 2/3 packed-backend: extract snapshot allocation in `load_contents`shejialuo, May 11, 2025
  44. Patrick SteinhardtMay 12, 2025
  45. shejialuoMay 12, 2025
  46. Patrick SteinhardtMay 12, 2025
  47. Jeff KingMay 12, 2025
  48. shejialuoMay 13, 2025
  49. 3/3 packed-backend: mmap large "packed-refs" file during fsckshejialuo, May 11, 2025
  50. Jeff KingMay 12, 2025
  51. 0/3 align the behavior when opening "packed-refs"shejialuo, May 13, 2025
  52. 1/3 packed-backend: fsck should warn when "packed-refs" file is emptyshejialuo, May 13, 2025
  53. Junio C HamanoMay 13, 2025
  54. shejialuoMay 14, 2025
  55. 3/3 packed-backend: mmap large "packed-refs" file during fsckshejialuo, May 13, 2025
  56. Junio C HamanoMay 13, 2025
  57. shejialuoMay 14, 2025
  58. 2/3 packed-backend: extract snapshot allocation in `load_contents`shejialuo, May 13, 2025
  59. 0/3 align the behavior when opening "packed-refs"shejialuo, May 14, 2025
  60. 2/3 packed-backend: extract snapshot allocation in `load_contents`shejialuo, May 14, 2025
  61. 1/3 packed-backend: fsck should warn when "packed-refs" file is emptyshejialuo, May 14, 2025
  62. 3/3 packed-backend: mmap large "packed-refs" file during fsckshejialuo, May 14, 2025
  63. Junio C HamanoMay 15, 2025
  64. Junio C HamanoMay 21, 2025
  65. Jeff KingMay 22, 2025
  66. Patrick SteinhardtMay 23, 2025
  67. Junio C HamanoMay 23, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.