git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 5/8] midx: load multi-pack indices via their source

From
Karthik Nayak <karthik.188@gmail.com>
Date
Aug 7, 2025, 08:49 UTC
Message-ID
<CAOLa=ZTA_3po038J7DGxk=gEM3JkkH5afBeRqTJLcKNM87hmxQ@mail.gmail.com>
In-Reply-To
<20250729-b4-pks-midx-deduplicate-source-info-v1-5-748db2eda3b5@pks.im>
Patrick Steinhardt <ps@pks.im> writes:
Show 24 quoted lines
> To load a multi-pack index the caller is expected to pass both the
> repository and the object directory where the multi-pack index is
> located. While this works, this layout has a couple of downsides:
>
>   - We need to pass in information reduntant with the owning source,
>     namely its object directory and whether the source is local or not.
>
>   - We don't have access to the source when loading the multi-pack
>     index. If we had that access, we could store a pointer to the owning
>     source in the MIDX and thus deduplicate some information.
>
>   - Multi-pack indices are inherently specific to the object source and
>     its format. With the goal of pluggable object backends in mind we
>     will eventually want the backends to own the logic of reading and
>     writing multi-pack indices. Making the logic work on top of object
>     sources is a step into that direction.
>
> Refactor loading of multi-pack indices accordingly.
>
> This surfaces one small problem though: git-multi-pack-index(1) and our
> MIDX test helper both know to read and write multi-pack-indices located
> in a different object directory. This issue is addressed by adding the
> user-provided object directory as an in-memory alternate.
>

Doesn't this commit only fix the 'read' side of things i.e. 'cmd_multi_pack_index_verify'.

Shouldn't we squash the next commit into this? Otherwise, writing midx present in a different object directory is broken as of this commit no? For e.g. in 'cmd_multi_pack_index_expire' we call 'handle_object_dir_option' which would add it as an alternate obd, but we don't use the return value at all.

[snip]
Previous: Patrick SteinhardtNext: Karthik Nayak
Message 13 of 56 in “midx: stop deduplicating info redundant with their sources”
  1. 0/8 midx: stop deduplicating info redundant with their sourcesPatrick Steinhardt, Jul 29, 2025
  2. 1/8 odb: store locality in object database sourcesPatrick Steinhardt, Jul 29, 2025
  3. Toon ClaesAug 6, 2025
  4. Patrick SteinhardtAug 7, 2025
  5. Karthik NayakAug 7, 2025
  6. 2/8 odb: allow `odb_find_source()` to failPatrick Steinhardt, Jul 29, 2025
  7. 3/8 odb: return newly created in-memory sourcesPatrick Steinhardt, Jul 29, 2025
  8. Toon ClaesAug 6, 2025
  9. Patrick SteinhardtAug 7, 2025
  10. Karthik NayakAug 7, 2025
  11. 4/8 midx: drop redundant `struct repository` parameterPatrick Steinhardt, Jul 29, 2025
  12. 5/8 midx: load multi-pack indices via their sourcePatrick Steinhardt, Jul 29, 2025
  13. Karthik NayakAug 7, 2025
  14. Karthik NayakAug 7, 2025
  15. 6/8 midx: write multi-pack indices via their sourcePatrick Steinhardt, Jul 29, 2025
  16. Karthik NayakAug 7, 2025
  17. 7/8 midx: stop duplicating info redundant with its owning sourcePatrick Steinhardt, Jul 29, 2025
  18. 8/8 midx: compute paths via their sourcePatrick Steinhardt, Jul 29, 2025
  19. Junio C HamanoJul 29, 2025
  20. Patrick SteinhardtJul 30, 2025
  21. 0/9 midx: stop duplicating info redundant with their sourcesPatrick Steinhardt, Aug 7, 2025
  22. 1/9 odb: store locality in object database sourcesPatrick Steinhardt, Aug 7, 2025
  23. Taylor BlauAug 7, 2025
  24. 2/9 odb: allow `odb_find_source()` to failPatrick Steinhardt, Aug 7, 2025
  25. Taylor BlauAug 7, 2025
  26. Patrick SteinhardtAug 11, 2025
  27. 3/9 odb: return newly created in-memory sourcesPatrick Steinhardt, Aug 7, 2025
  28. Taylor BlauAug 7, 2025
  29. Patrick SteinhardtAug 11, 2025
  30. 4/9 odb: simplify calling `link_alt_odb_entry()`Patrick Steinhardt, Aug 7, 2025
  31. Taylor BlauAug 7, 2025
  32. 5/9 midx: drop redundant `struct repository` parameterPatrick Steinhardt, Aug 7, 2025
  33. 6/9 midx: load multi-pack indices via their sourcePatrick Steinhardt, Aug 7, 2025
  34. Taylor BlauAug 7, 2025
  35. 7/9 midx: write multi-pack indices via their sourcePatrick Steinhardt, Aug 7, 2025
  36. Taylor BlauAug 7, 2025
  37. 8/9 midx: stop duplicating info redundant with its owning sourcePatrick Steinhardt, Aug 7, 2025
  38. 9/9 midx: compute paths via their sourcePatrick Steinhardt, Aug 7, 2025
  39. Taylor BlauAug 7, 2025
  40. Patrick SteinhardtAug 11, 2025
  41. Karthik NayakAug 7, 2025
  42. 00/10 midx: stop duplicating info redundant with their sourcesPatrick Steinhardt, Aug 11, 2025
  43. 01/10 odb: store locality in object database sourcesPatrick Steinhardt, Aug 11, 2025
  44. 02/10 odb: allow `odb_find_source()` to failPatrick Steinhardt, Aug 11, 2025
  45. 03/10 odb: consistently use "dir" to refer to alternate's directoryPatrick Steinhardt, Aug 11, 2025
  46. 04/10 odb: return newly created in-memory sourcesPatrick Steinhardt, Aug 11, 2025
  47. 05/10 odb: simplify calling `link_alt_odb_entry()`Patrick Steinhardt, Aug 11, 2025
  48. 06/10 midx: drop redundant `struct repository` parameterPatrick Steinhardt, Aug 11, 2025
  49. 07/10 midx: load multi-pack indices via their sourcePatrick Steinhardt, Aug 11, 2025
  50. 08/10 midx: write multi-pack indices via their sourcePatrick Steinhardt, Aug 11, 2025
  51. 09/10 midx: stop duplicating info redundant with its owning sourcePatrick Steinhardt, Aug 11, 2025
  52. 10/10 midx: compute paths via their sourcePatrick Steinhardt, Aug 11, 2025
  53. Junio C HamanoAug 28, 2025
  54. Taylor BlauAug 29, 2025
  55. Derrick StoleeAug 30, 2025
  56. Patrick SteinhardtSep 2, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.