Re: [PATCH v2 0/4] odb: eagerly load alternates
Patrick Steinhardt <ps@pks.im> writes:
Show 37 quoted lines
> Hi,
>
> when initializing the object database we only eagerly initialize the
> primary object database source. If the primary source has alternates,
> those alternates are only initialized the first time we really access
> the object database.
>
> When introduced in ace1534d6f (Introduce SHA1_FILE_DIRECTORIES to
> support multiple object databases., 2005-05-07), alternates were
> originally only loaded when a given object wasn't found in the primary
> object database. This was also reinforced by later optimization, for
> example in 693d2bc625 (Attempt to delay prepare_alt_odb during get_sha1,
> 2007-05-26), where we tried to avoid loading alternates in even more
> cases. But as Git has evolved, we eventually started to eagerly parse
> alternates all over the codebase, including on every single object
> lookup, and consequently deferring this operation does not really buy us
> much anymore.
>
> The result of this is that we have calls to `odb_prepare_alternates()`
> cluttered all over the code base. This is somewhat awkward, and as
> almost every Git command ends up reading objects at it doesn't even buy
> us anything.
>
> This patch series thus gets rid of the lazy-loading. Besides simplifying
> the codebase a bit, it also prepares us for moving alternates into the
> "files" backend as discussed in [1].
>
> The series is built on top of 010afd3166 (The 12th batch, 2026-08-07)
> with ps/odb-make-creation-pluggable at e927cfeb21 (odb: make creation of
> on-disk structures pluggable, 2026-08-07) merged into it.
>
> Changes in v2:
> - Add a missing word to a commit message.
> - Explain why we don't have to handle GIT_ALTERNATE_OBJECT_DIRECTORIES
> when re-preparing the object database.
> - Link to v1: https://patch.msgid.link/20260810-pks-odb-eagerly-prepare-alternates-v1-0-f0fa4a4004e1@pks.im
>
V2 looks good, I have one nit, but it's not work re-rolling :)
Show 72 quoted lines
> Thanks!
>
> Patrick
>
> [1]: <amLgMqkqxR8mKIbT@pks.im>
>
> ---
> Patrick Steinhardt (4):
> odb: decouple source path comparisons from `the_repository`
> odb: eagerly initialize alternates
> odb: drop `loaded_alternates` field
> odb: drop `alternates_db` field
>
> builtin/fsck.c | 3 --
> builtin/pack-objects.c | 3 --
> commit-graph.c | 4 --
> loose.c | 1 -
> object-name.c | 1 -
> odb.c | 109 ++++++++++++++++++++++++-------------------------
> odb.h | 22 +++++-----
> odb/source.h | 7 ++++
> odb/streaming.c | 1 -
> pack-bitmap.c | 2 -
> packfile.c | 1 -
> packfile.h | 2 -
> 12 files changed, 70 insertions(+), 86 deletions(-)
>
> Range-diff versus v1:
>
> 1: 25802adffa = 1: 721907c60d odb: decouple source path comparisons from `the_repository`
> 2: 1e73b730d8 ! 2: 3b2c23566c odb: eagerly initialize alternates
> @@ Commit message
> many calls to `odb_prepare_alternates()` cluttered around the code base
> whenever we are about to iterate through the sources.
>
> - This lazy loading doesn't really add much value: the moment where read
> - any object we _have_ to load the alternates anyway. So given that most
> - of our commands would access the object database this optimization is
> - not really buying us much in the first place. Quite on the contrary, it
> - makes the code harder to understand and is a potential source of bugs in
> - case any callsite forgot to prepare alternates before we iterate through
> - the sources.
> + This lazy loading doesn't really add much value: the moment where we
> + read any object we _have_ to load the alternates anyway. So given that
> + most of our commands would access the object database this optimization
> + is not really buying us much in the first place. Quite on the contrary,
> + it makes the code harder to understand and is a potential source of bugs
> + in case any callsite forgot to prepare alternates before we iterate
> + through the sources.
>
> Historically though there was a reason why we deferred lazy-loading: it
> may happen that the repository has "core.ignoreCase" configured, and we
> 3: 2ca1aa2a37 = 3: df5d7df91d odb: drop `loaded_alternates` field
> 4: 1e97c93bdf ! 4: 50a37ef385 odb: drop `alternates_db` field
> @@ odb.c: void odb_free(struct object_database *o)
> pthread_mutex_destroy(&o->replace_mutex);
>
> @@ odb.c: void odb_prepare(struct object_database *o, enum odb_prepare_flags flags)
> - * the lifetime of the process.
> + * Reprepare alt odbs, in case the alternates file was modified
> + * during the course of this process. This only _adds_ odbs to
> + * the linked list, so existing odbs will continue to exist for
> +- * the lifetime of the process.
> ++ * the lifetime of the process. Consequently, we don't have to
> ++ * reprocess GIT_ALTERNATE_OBJECT_DIRECTORIES here.
> */
> if (flags & ODB_PREPARE_FLUSH_CACHES) {
> - odb_prepare_alternates(o);
>
> ---
> base-commit: f6ad67a7977439ad8351d42e6ccfd11f714db765
> change-id: 20260804-pks-odb-eagerly-prepare-alternates-3efb0a38e0dd