git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 1/8] odb: store locality in object database sources

From
Karthik Nayak <karthik.188@gmail.com>
Date
Aug 7, 2025, 08:12 UTC
Message-ID
<CAOLa=ZTr6pkYHu05xBKJwY+6RL215o06c=qgZ1b=4MTMxyV6cg@mail.gmail.com>
In-Reply-To
<20250729-b4-pks-midx-deduplicate-source-info-v1-1-748db2eda3b5@pks.im>
Patrick Steinhardt <ps@pks.im> writes:
Show 88 quoted lines
> Object database sources are classified either as:
>
>   - Local, which means that the source is the repository's primary
>     source. This is typically ".git/objects".
>
>   - Non-local, which is everything else. Most importantly this includes
>     alternates and quarantine directories.
>
> This locality is often computed ad-hoc by checking whether a given
> object source is the first one. This works, but it is quite roundabout.
>
> Refactor the code so that we store locality when creating the sources in
> the first place. This makes it both more accessible and robust.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  midx.c       | 5 +++--
>  midx.h       | 2 +-
>  odb.c        | 1 +
>  odb.h        | 8 ++++++++
>  packfile.c   | 9 ++++-----
>  repository.c | 1 +
>  6 files changed, 18 insertions(+), 8 deletions(-)
>
> diff --git a/midx.c b/midx.c
> index 7d407682e6..b9ca0915a6 100644
> --- a/midx.c
> +++ b/midx.c
> @@ -723,7 +723,7 @@ int midx_preferred_pack(struct multi_pack_index *m, uint32_t *pack_int_id)
>  	return 0;
>  }
>
> -int prepare_multi_pack_index_one(struct odb_source *source, int local)
> +int prepare_multi_pack_index_one(struct odb_source *source)
>  {
>  	struct repository *r = source->odb->repo;
>
> @@ -734,7 +734,8 @@ int prepare_multi_pack_index_one(struct odb_source *source, int local)
>  	if (source->midx)
>  		return 1;
>
> -	source->midx = load_multi_pack_index(r, source->path, local);
> +	source->midx = load_multi_pack_index(r, source->path,
> +					     source->local);
>
>  	return !!source->midx;
>  }
> diff --git a/midx.h b/midx.h
> index 076382de8a..28c426a823 100644
> --- a/midx.h
> +++ b/midx.h
> @@ -122,7 +122,7 @@ int fill_midx_entry(struct repository *r, const struct object_id *oid, struct pa
>  int midx_contains_pack(struct multi_pack_index *m,
>  		       const char *idx_or_pack_name);
>  int midx_preferred_pack(struct multi_pack_index *m, uint32_t *pack_int_id);
> -int prepare_multi_pack_index_one(struct odb_source *source, int local);
> +int prepare_multi_pack_index_one(struct odb_source *source);
>
>  /*
>   * Variant of write_midx_file which writes a MIDX containing only the packs
> diff --git a/odb.c b/odb.c
> index 1f48a0448e..1761a50840 100644
> --- a/odb.c
> +++ b/odb.c
> @@ -176,6 +176,7 @@ static int link_alt_odb_entry(struct object_database *odb,
>
>  	CALLOC_ARRAY(alternate, 1);
>  	alternate->odb = odb;
> +	alternate->local = false;
>  	/* pathbuf.buf is already in r->objects->source_by_path */
>  	alternate->path = strbuf_detach(&pathbuf, NULL);
>
> diff --git a/odb.h b/odb.h
> index 09177bf430..d9f4dcf79f 100644
> --- a/odb.h
> +++ b/odb.h
> @@ -63,6 +63,14 @@ struct odb_source {
>  	 */
>  	struct multi_pack_index *midx;
>
> +	/*
> +	 * Figure out whether this is the local alternate of the owning
> +	 * repository, which would typically be its ".git/objects" directory.
> +	 * This local object directory is usually where objects would be
> +	 * written to.
> +	 */
> +	bool local;
> +

So now, each 'obd_source' has a 'local' field, which would be false by default. But when the local 'odb_source' is setup, we set it to true.

Show 22 quoted lines
>  	/*
>  	 * This is a temporary object store created by the tmp_objdir
>  	 * facility. Disable ref updates since the objects in the store
> diff --git a/packfile.c b/packfile.c
> index 5d73932f50..a38544b87b 100644
> --- a/packfile.c
> +++ b/packfile.c
> @@ -935,14 +935,14 @@ static void prepare_pack(const char *full_name, size_t full_name_len,
>  		report_garbage(PACKDIR_FILE_GARBAGE, full_name);
>  }
>
> -static void prepare_packed_git_one(struct odb_source *source, int local)
> +static void prepare_packed_git_one(struct odb_source *source)
>  {
>  	struct string_list garbage = STRING_LIST_INIT_DUP;
>  	struct prepare_pack_data data = {
>  		.m = source->midx,
>  		.r = source->odb->repo,
>  		.garbage = &garbage,
> -		.local = local,
> +		.local = source->local,
>

The 'prepare_pack_data' already has a 'local' field, so we can now simply set it using the 'odb_source's field of the same name. Nice.

Show 28 quoted lines
>  	};
>
>  	for_each_file_in_pack_dir(source->path, prepare_pack, &data);
> @@ -1037,9 +1037,8 @@ static void prepare_packed_git(struct repository *r)
>
>  	odb_prepare_alternates(r->objects);
>  	for (source = r->objects->sources; source; source = source->next) {
> -		int local = (source == r->objects->sources);
> -		prepare_multi_pack_index_one(source, local);
> -		prepare_packed_git_one(source, local);
> +		prepare_multi_pack_index_one(source);
> +		prepare_packed_git_one(source);
>  	}
>  	rearrange_packed_git(r);
>
> diff --git a/repository.c b/repository.c
> index ecd691181f..97f0578381 100644
> --- a/repository.c
> +++ b/repository.c
> @@ -168,6 +168,7 @@ void repo_set_gitdir(struct repository *repo,
>  	if (!repo->objects->sources) {
>  		CALLOC_ARRAY(repo->objects->sources, 1);
>  		repo->objects->sources->odb = repo->objects;
> +		repo->objects->sources->local = true;
>  		repo->objects->sources_tail = &repo->objects->sources->next;
>  	}
>  	expand_base_dir(&repo->objects->sources->path, o->object_dir,
>
So this sets up the local 'odb_source' and as such sets 'local' to true.
Looks good.
> --
> 2.50.1.619.g074bbf1d35.dirty
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 5 of 56 in “midx: stop deduplicating info redundant with their sources”
  1. 0/8 midx: stop deduplicating info redundant with their sourcesPatrick Steinhardt, Jul 29, 2025
  2. 1/8 odb: store locality in object database sourcesPatrick Steinhardt, Jul 29, 2025
  3. Toon ClaesAug 6, 2025
  4. Patrick SteinhardtAug 7, 2025
  5. Karthik NayakAug 7, 2025
  6. 2/8 odb: allow `odb_find_source()` to failPatrick Steinhardt, Jul 29, 2025
  7. 3/8 odb: return newly created in-memory sourcesPatrick Steinhardt, Jul 29, 2025
  8. Toon ClaesAug 6, 2025
  9. Patrick SteinhardtAug 7, 2025
  10. Karthik NayakAug 7, 2025
  11. 4/8 midx: drop redundant `struct repository` parameterPatrick Steinhardt, Jul 29, 2025
  12. 5/8 midx: load multi-pack indices via their sourcePatrick Steinhardt, Jul 29, 2025
  13. Karthik NayakAug 7, 2025
  14. Karthik NayakAug 7, 2025
  15. 6/8 midx: write multi-pack indices via their sourcePatrick Steinhardt, Jul 29, 2025
  16. Karthik NayakAug 7, 2025
  17. 7/8 midx: stop duplicating info redundant with its owning sourcePatrick Steinhardt, Jul 29, 2025
  18. 8/8 midx: compute paths via their sourcePatrick Steinhardt, Jul 29, 2025
  19. Junio C HamanoJul 29, 2025
  20. Patrick SteinhardtJul 30, 2025
  21. 0/9 midx: stop duplicating info redundant with their sourcesPatrick Steinhardt, Aug 7, 2025
  22. 1/9 odb: store locality in object database sourcesPatrick Steinhardt, Aug 7, 2025
  23. Taylor BlauAug 7, 2025
  24. 2/9 odb: allow `odb_find_source()` to failPatrick Steinhardt, Aug 7, 2025
  25. Taylor BlauAug 7, 2025
  26. Patrick SteinhardtAug 11, 2025
  27. 3/9 odb: return newly created in-memory sourcesPatrick Steinhardt, Aug 7, 2025
  28. Taylor BlauAug 7, 2025
  29. Patrick SteinhardtAug 11, 2025
  30. 4/9 odb: simplify calling `link_alt_odb_entry()`Patrick Steinhardt, Aug 7, 2025
  31. Taylor BlauAug 7, 2025
  32. 5/9 midx: drop redundant `struct repository` parameterPatrick Steinhardt, Aug 7, 2025
  33. 6/9 midx: load multi-pack indices via their sourcePatrick Steinhardt, Aug 7, 2025
  34. Taylor BlauAug 7, 2025
  35. 7/9 midx: write multi-pack indices via their sourcePatrick Steinhardt, Aug 7, 2025
  36. Taylor BlauAug 7, 2025
  37. 8/9 midx: stop duplicating info redundant with its owning sourcePatrick Steinhardt, Aug 7, 2025
  38. 9/9 midx: compute paths via their sourcePatrick Steinhardt, Aug 7, 2025
  39. Taylor BlauAug 7, 2025
  40. Patrick SteinhardtAug 11, 2025
  41. Karthik NayakAug 7, 2025
  42. 00/10 midx: stop duplicating info redundant with their sourcesPatrick Steinhardt, Aug 11, 2025
  43. 01/10 odb: store locality in object database sourcesPatrick Steinhardt, Aug 11, 2025
  44. 02/10 odb: allow `odb_find_source()` to failPatrick Steinhardt, Aug 11, 2025
  45. 03/10 odb: consistently use "dir" to refer to alternate's directoryPatrick Steinhardt, Aug 11, 2025
  46. 04/10 odb: return newly created in-memory sourcesPatrick Steinhardt, Aug 11, 2025
  47. 05/10 odb: simplify calling `link_alt_odb_entry()`Patrick Steinhardt, Aug 11, 2025
  48. 06/10 midx: drop redundant `struct repository` parameterPatrick Steinhardt, Aug 11, 2025
  49. 07/10 midx: load multi-pack indices via their sourcePatrick Steinhardt, Aug 11, 2025
  50. 08/10 midx: write multi-pack indices via their sourcePatrick Steinhardt, Aug 11, 2025
  51. 09/10 midx: stop duplicating info redundant with its owning sourcePatrick Steinhardt, Aug 11, 2025
  52. 10/10 midx: compute paths via their sourcePatrick Steinhardt, Aug 11, 2025
  53. Junio C HamanoAug 28, 2025
  54. Taylor BlauAug 29, 2025
  55. Derrick StoleeAug 30, 2025
  56. Patrick SteinhardtSep 2, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.