git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 0/4] loose: fix migrating "loose-object-idx" with batch fsync

From
Junio C Hamano <gitster@pobox.com>
Date
Oct 29, 2026, 22:22 UTC
Message-ID
<xmqq8q3gywny.fsf@gitster.g>
In-Reply-To
<20261029002413.3291184-1-jltobler@gmail.com>
Justin Tobler <jltobler@gmail.com> writes:
Show 29 quoted lines
> Greetings,
>
> While working on a separate patch series to convert the "files" ODB
> transaction to be eagerly created instead of lazily, I encountered this
> bug that causes an ODB transaction to fail on commit when a
> compatibility hash is also configured. This occurs because the temporary
> directory migration code does not know how to handle the loose object
> index file.
>
> This patch addresses updating the migration logic to explicitly merge
> mapping entries in the transaction and atomically write a new
> "loose-object-idx" file.
>
> Some other approaches I considered:
>   - Storing the uncommitted object mapping entries in the transaction
>     without writing a file and only writing on transaction commit. I
>     chose to keep writing the temp index into the transaction so that
>     child processes that write to the tempdir could still append entries.
>   - Also writing mapping entires to the main ODB loose object index.
>     This also didn't seem like a good apprach because it write entries
>     that may not ultimately get committed.
>
> Changes since V1:
>   - The loose object index write after the merging maps is now atomic.
>   - Added a fixed timeout to all loose object index writers.
>   - Duplicate entries are now dropped when writing the merged mapping file.
>
> Thanks for the review,
> -Justin
Thanks.

Merging this topic to 'seen' had a funny interaction with other topics in flight. repo_write_loose_object_map() function, from which loose_object_map_write() helper function was extracted out of, loses all callers, and will disappear in the merge. But the new helper, which gained a caller loose_object_idx_merge() in this series, remains.

Previous: Justin ToblerNext: Justin Tobler
Message 18 of 33 in “tmp-objdir: merge loose object index when migrating tmpdir”
  1. tmp-objdir: merge loose object index when migrating tmpdirJustin Tobler, Oct 22, 2026
  2. Patrick SteinhardtOct 23, 2026
  3. Justin ToblerOct 23, 2026
  4. Patrick SteinhardtOct 26, 2026
  5. Justin ToblerOct 26, 2026
  6. 0/4 loose: fix migrating "loose-object-idx" with batch fsyncJustin Tobler, Oct 29, 2026
  7. 1/4 loose: use finite timeout when locking "loose-object-idx" fileJustin Tobler, Oct 29, 2026
  8. Patrick SteinhardtOct 29, 2026
  9. 2/4 loose: lift loose object index parsing logic into helperJustin Tobler, Oct 29, 2026
  10. Patrick SteinhardtOct 29, 2026
  11. Justin ToblerOct 29, 2026
  12. 3/4 loose: lift loose object index writing logic into helperJustin Tobler, Oct 29, 2026
  13. Patrick SteinhardtOct 29, 2026
  14. Justin ToblerOct 29, 2026
  15. 4/4 tmp-objdir: merge loose object index when migrating tmpdirJustin Tobler, Oct 29, 2026
  16. Patrick SteinhardtOct 29, 2026
  17. Justin ToblerOct 29, 2026
  18. Junio C HamanoOct 29, 2026
  19. Justin ToblerOct 29, 2026
  20. 0/4 loose: fix migrating "loose-object-idx" with batch fsyncJustin Tobler, Oct 29, 2026
  21. 1/4 loose: use finite timeout when locking "loose-object-idx" fileJustin Tobler, Oct 29, 2026
  22. Karthik NayakOct 30, 2026
  23. 2/4 loose: lift loose object index parsing logic into helperJustin Tobler, Oct 29, 2026
  24. Karthik NayakOct 30, 2026
  25. Justin ToblerOct 30, 2026
  26. 3/4 loose: lift loose object index writing logic into helperJustin Tobler, Oct 29, 2026
  27. 4/4 tmp-objdir: merge loose object index when migrating tmpdirJustin Tobler, Oct 29, 2026
  28. Karthik NayakOct 30, 2026
  29. Justin ToblerOct 30, 2026
  30. Patrick SteinhardtOct 30, 2026
  31. Justin ToblerOct 30, 2026
  32. Karthik NayakOct 30, 2026
  33. Justin ToblerOct 30, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.