git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 2/5] setup: detangle loading of loose object maps

From
Justin Tobler <jltobler@gmail.com>
Date
Jul 28, 2026, 20:32 UTC
Message-ID
<amkOb3rvWFUpnT28@denethor>
In-Reply-To
<20260724-pks-odb-create-on-disk-v1-2-3b3d265d979b@pks.im>
On 26/07/24 05:48AM, Patrick Steinhardt wrote:
Show 12 quoted lines
> When a repository is configured to use a compatibility hash function
> then we load the loose object map when we initialize the repository.
> This object map provides the mappings between the canonical object hash
> and the compatibility object hash.
> 
> Loading the object map happens in `repo_set_compat_hash_algo()`, which
> calls `repo_read_loose_object_map()` in case the compatibility object
> hash is non-zero. This setup sequence has two major downsides:
> 
>   - We assume that the primary object database is the "files" object
>     database so that we can extract its "loose" backend. This stops
>     working with pluggable object databases.

So IIUC, does this mean that `repo_set_compat_hash_algo()` is directly reaching into the loose object source to load the compatibility object map? I suppose it should be the responsibility of the respective ODB backend to handle object compatibility.

>   - We require the object database to already have been initialized when
>     configuring the object database. This means that we must intermix
>     configuration of the repository and initialization of its
>     sub-structures in a weird way.

If there any reason we need to eagerly load compatibility object mappings?

> Refactor the logic so that we instead load the loose object map via the
> "loose" backend, which fixes both of the above issues.
Sounds reasonable.
Show 28 quoted lines
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  loose.c            | 11 +++++------
>  loose.h            |  1 +
>  odb/source-loose.c |  2 ++
>  repository.c       |  2 --
>  setup.c            |  5 +++--
>  5 files changed, 11 insertions(+), 10 deletions(-)
> 
> diff --git a/loose.c b/loose.c
> index 9dad75373b..a3b2dcedc2 100644
> --- a/loose.c
> +++ b/loose.c
> @@ -61,7 +61,7 @@ static int insert_loose_map(struct odb_source_loose *loose,
>  	return inserted;
>  }
>  
> -static int load_one_loose_object_map(struct odb_source_loose *loose)
> +int loose_object_map_load(struct odb_source_loose *loose)
>  {
>  	struct repository *repo = loose->base.odb->repo;
>  	struct strbuf buf = STRBUF_INIT;
> @@ -69,6 +69,9 @@ static int load_one_loose_object_map(struct odb_source_loose *loose)
>  	FILE *fp;
>  	int ret = -1;
>  
> +	if (!should_use_loose_object_map(repo))
> +		return 0;

Previously the above condition has asserted in `repo_read_loose_object_map()` which calls `loose_object_map_load()` for each source. Do we expect each source to potentially answer differently though?

Show 41 quoted lines
> +
>  	if (!loose->map)
>  		loose_object_map_init(&loose->map);
>  	if (!loose->cache) {
> @@ -112,14 +115,10 @@ int repo_read_loose_object_map(struct repository *repo)
>  {
>  	struct odb_source *source;
>  
> -	if (!should_use_loose_object_map(repo))
> -		return 0;
> -
>  	odb_prepare_alternates(repo->objects);
> -
>  	for (source = repo->objects->sources; source; source = source->next) {
>  		struct odb_source_files *files = odb_source_files_downcast(source);
> -		if (load_one_loose_object_map(files->loose) < 0)
> +		if (loose_object_map_load(files->loose) < 0)
>  			return -1;
>  	}
>  
> diff --git a/loose.h b/loose.h
> index 6c9b3f4571..ed663ac550 100644
> --- a/loose.h
> +++ b/loose.h
> @@ -13,6 +13,7 @@ struct loose_object_map {
>  
>  void loose_object_map_init(struct loose_object_map **map);
>  void loose_object_map_clear(struct loose_object_map **map);
> +int loose_object_map_load(struct odb_source_loose *loose);
>  int repo_loose_object_map_oid(struct repository *repo,
>  			      const struct object_id *src,
>  			      const struct git_hash_algo *dest_algo,
> diff --git a/odb/source-loose.c b/odb/source-loose.c
> index 3f7d04a56e..812ca1c138 100644
> --- a/odb/source-loose.c
> +++ b/odb/source-loose.c
> @@ -727,5 +727,7 @@ struct odb_source_loose *odb_source_loose_new(struct object_database *odb,
>  	if (!is_absolute_path(loose->base.path))
>  		chdir_notify_register(NULL, odb_source_loose_reparent, loose);
>  
> +	loose_object_map_load(loose);

Now we load the loose object map for the specific source when its created.

Show 13 quoted lines
> +
>  	return loose;
>  }
> diff --git a/repository.c b/repository.c
> index 2ef0778846..6d633002b4 100644
> --- a/repository.c
> +++ b/repository.c
> @@ -201,8 +201,6 @@ void repo_set_compat_hash_algo(struct repository *repo MAYBE_UNUSED, uint32_t al
>  	if (hash_algo_by_ptr(repo->hash_algo) == algo)
>  		BUG("hash_algo and compat_hash_algo match");
>  	repo->compat_hash_algo = algo ? &hash_algos[algo] : NULL;
> -	if (repo->compat_hash_algo)
> -		repo_read_loose_object_map(repo);
The loose object map is no longer read eagerly.
Show 22 quoted lines
>  #else
>  	if (algo)
>  		die(_("compatibility hash algorithm support requires Rust"));
> diff --git a/setup.c b/setup.c
> index d31808130b..825572f5f1 100644
> --- a/setup.c
> +++ b/setup.c
> @@ -1788,8 +1788,6 @@ int apply_repository_format(struct repository *repo,
>  
>  	repo->bare_cfg = format->is_bare;
>  	repo_set_hash_algo(repo, format->hash_algo);
> -	repo->objects = odb_new(repo, object_directory,
> -				alternate_object_directories);
>  	repo_set_compat_hash_algo(repo, format->compat_hash_algo);
>  	repo_set_ref_storage_format(repo,
>  				    format->ref_storage_format,
> @@ -1805,6 +1803,9 @@ int apply_repository_format(struct repository *repo,
>  	repo->repository_format_precious_objects =
>  		format->precious_objects;
>  
> +	repo->objects = odb_new(repo, object_directory,
> +				alternate_object_directories);

We now defer creating the ODB until after the compat hash is configured. Makes sense.

-Justin
Previous: Patrick SteinhardtNext: Toon Claes
Message 10 of 68 in “odb: make creation of object database pluggable”
  1. 0/5 odb: make creation of object database pluggablePatrick Steinhardt, Jul 24, 2026
  2. 1/5 loose: load loose object map for the correct sourcePatrick Steinhardt, Jul 24, 2026
  3. Junio C HamanoJul 24, 2026
  4. Justin ToblerJul 28, 2026
  5. Toon ClaesJul 30, 2026
  6. Patrick SteinhardtAug 4, 2026
  7. 2/5 setup: detangle loading of loose object mapsPatrick Steinhardt, Jul 24, 2026
  8. Junio C HamanoJul 24, 2026
  9. Patrick SteinhardtAug 4, 2026
  10. Justin ToblerJul 28, 2026
  11. Toon ClaesJul 30, 2026
  12. Patrick SteinhardtAug 4, 2026
  13. 3/5 setup: defer object database creationPatrick Steinhardt, Jul 24, 2026
  14. Junio C HamanoJul 24, 2026
  15. Patrick SteinhardtAug 4, 2026
  16. Justin ToblerJul 28, 2026
  17. Patrick SteinhardtAug 4, 2026
  18. Patrick SteinhardtAug 4, 2026
  19. 4/5 odb/source: introduce function to map source type to namePatrick Steinhardt, Jul 24, 2026
  20. Junio C HamanoJul 26, 2026
  21. Patrick SteinhardtAug 4, 2026
  22. 5/5 odb: make creation of on-disk structures pluggablePatrick Steinhardt, Jul 24, 2026
  23. Junio C HamanoJul 26, 2026
  24. Patrick SteinhardtAug 4, 2026
  25. Justin ToblerJul 28, 2026
  26. 0/5 odb: make creation of object database pluggablePatrick Steinhardt, Aug 4, 2026
  27. 1/5 loose: load loose object map for the correct sourcePatrick Steinhardt, Aug 4, 2026
  28. 2/5 setup: detangle loading of loose object mapsPatrick Steinhardt, Aug 4, 2026
  29. 3/5 setup: defer object database creationPatrick Steinhardt, Aug 4, 2026
  30. Toon ClaesAug 4, 2026
  31. Patrick SteinhardtAug 5, 2026
  32. 4/5 odb/source: introduce function to map source type to namePatrick Steinhardt, Aug 4, 2026
  33. 5/5 odb: make creation of on-disk structures pluggablePatrick Steinhardt, Aug 4, 2026
  34. Justin ToblerAug 4, 2026
  35. 0/6 odb: make creation of object database pluggablePatrick Steinhardt, Aug 5, 2026
  36. 1/6 loose: load loose object map for the correct sourcePatrick Steinhardt, Aug 5, 2026
  37. 2/6 setup: detangle loading of loose object mapsPatrick Steinhardt, Aug 5, 2026
  38. 3/6 setup: handle ODB-related environment variables in `odb_new()`Patrick Steinhardt, Aug 5, 2026
  39. Toon ClaesAug 5, 2026
  40. Patrick SteinhardtAug 6, 2026
  41. 4/6 setup: defer object database creationPatrick Steinhardt, Aug 5, 2026
  42. Toon ClaesAug 5, 2026
  43. Patrick SteinhardtAug 6, 2026
  44. 5/6 odb/source: introduce function to map source type to namePatrick Steinhardt, Aug 5, 2026
  45. 6/6 odb: make creation of on-disk structures pluggablePatrick Steinhardt, Aug 5, 2026
  46. Toon ClaesAug 5, 2026
  47. 0/6 odb: make creation of object database pluggablePatrick Steinhardt, Aug 6, 2026
  48. 1/6 loose: load loose object map for the correct sourcePatrick Steinhardt, Aug 6, 2026
  49. 2/6 setup: detangle loading of loose object mapsPatrick Steinhardt, Aug 6, 2026
  50. 3/6 setup: handle ODB-related environment variables in `odb_new()`Patrick Steinhardt, Aug 6, 2026
  51. 4/6 setup: defer object database creationPatrick Steinhardt, Aug 6, 2026
  52. Toon ClaesAug 6, 2026
  53. Patrick SteinhardtAug 6, 2026
  54. Junio C HamanoAug 6, 2026
  55. 5/6 odb/source: introduce function to map source type to namePatrick Steinhardt, Aug 6, 2026
  56. 6/6 odb: make creation of on-disk structures pluggablePatrick Steinhardt, Aug 6, 2026
  57. Toon ClaesAug 6, 2026
  58. 0/6 odb: make creation of object database pluggablePatrick Steinhardt, Aug 7, 2026
  59. 1/6 loose: load loose object map for the correct sourcePatrick Steinhardt, Aug 7, 2026
  60. 2/6 setup: detangle loading of loose object mapsPatrick Steinhardt, Aug 7, 2026
  61. 3/6 setup: handle ODB-related environment variables in `odb_new()`Patrick Steinhardt, Aug 7, 2026
  62. 4/6 setup: defer object database creationPatrick Steinhardt, Aug 7, 2026
  63. Junio C HamanoAug 7, 2026
  64. Toon ClaesAug 7, 2026
  65. 5/6 odb/source: introduce function to map source type to namePatrick Steinhardt, Aug 7, 2026
  66. 6/6 odb: make creation of on-disk structures pluggablePatrick Steinhardt, Aug 7, 2026
  67. Toon ClaesAug 7, 2026
  68. Patrick SteinhardtAug 7, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.