git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 1/8] builtin/clone: defer setup of the object database

From
Justin Tobler <jltobler@gmail.com>
Date
Sep 6, 2026, 16:42 UTC
Message-ID
<ap2URrS4i-nHV5cB@denethor>
In-Reply-To
<20260831-pks-odb-write-alternates-at-creation-time-v2-1-aecd2382ba1c@pks.im>
On 26/08/31 12:02PM, Patrick Steinhardt wrote:
Show 10 quoted lines
> When cloning a repository we defer initialization of the reference
> database. This is because we don't yet know all details required for us
> to initialize the refdb in the first place. Most importantly, what we
> are missing is information about the object hash.
> 
> We don't do the same thing for the object database yet, but here we
> essentially have the same problem. While the "files" database does not
> need any information about the object format at creation time, alternate
> backends are likely to require that information so that they can
> properly set up their data structures.

Ok, so if we want to move the writing of alternates to happen at ODB creation time, this information will be needed ahead of time.

Show 22 quoted lines
> Besides this forward-looking future proofing though, we also have a
> second use case for deferring initialization of the object database,
> namely alternates. When initializing the object database we do not yet
> know whether we'll need alternates or not because this depends on the
> repository we're about to clone from. If it is a local repository and
> the user has passed "--refernce{,-if-able}", then we will end up writing
> alternates into the object database.
> 
> The ugly part though is that we cannot determine where the repository is
> getting cloned from before it has been initialized. While we of course
> already have access to the user-provided URI, that URI can be very well
> rewritten via "url.<base>.insteadOf". We can of course read the global-
> and system-level configuration to resolve it. But we explicitly resolve
> the URI a second time after we have initialized the repository because
> it can happen that we copy a ".git/config" over from our templates, and
> that file may cause us to rewrite the path.
> 
> In a subsequent commit though we'll start to write alternates as part of
> the repository initialization, so we'll need to have the URI properly
> resolved before we can initialize the object database. This is ugly, but
> as mentioned above it makes sense for us to defer its initialization
> anyway so that we also know about the object hash already.
Ok.
Show 28 quoted lines
> Introduce a new flag that makes `init_db()` skip initializing the object
> database. Expose `create_object_database()` and make use of it after we
> have resolved the URI.
> 
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  builtin/clone.c | 12 ++++++++----
>  setup.c         |  8 +++++---
>  setup.h         | 15 ++++++++++-----
>  3 files changed, 23 insertions(+), 12 deletions(-)
> 
> diff --git a/builtin/clone.c b/builtin/clone.c
> index 5b25cca510..0a67492ebd 100644
> --- a/builtin/clone.c
> +++ b/builtin/clone.c
> @@ -1184,11 +1184,14 @@ int cmd_clone(int argc,
>  	 * database. We do not yet know about the object format of the
>  	 * repository, and reference backends may persist that information into
>  	 * their on-disk data structures.
> +	 *
> +	 * Furthermore, we skip initializing the object database so that we can
> +	 * first resolve potential alternates before creating it.
>  	 */
>  	init_db(the_repository, git_dir, real_git_dir, work_tree, option_template,
>  		GIT_HASH_UNKNOWN, ref_storage_format, NULL,
>  		do_not_override_repo_unix_permissions,
> -		INIT_DB_QUIET | INIT_DB_SKIP_REFDB);
> +		INIT_DB_QUIET | INIT_DB_SKIP_REFDB | INIT_DB_SKIP_ODB);

Ok, now we skip the initializing the ODB during init_db in favor of delaying it to after we have the required config info. This makes sense to me, but IMO the `init_db()` interface has grown quite awkward with these "skip" flags. It appears that there are only two callers of `init_db()` which makes me wonder if it would be simpler to just require them to explicitly set up the ref DB and ODB.

Show 17 quoted lines
>  	if (real_git_dir) {
>  		free((char *)git_dir);
> @@ -1311,9 +1314,6 @@ int cmd_clone(int argc,
>  		strbuf_reset(&key);
>  	}
>  
> -	if (option_required_reference.nr || option_optional_reference.nr)
> -		setup_reference();
> -
>  	remote = remote_get_early(remote_name);
>  
>  	if (!option_rev)
> @@ -1342,6 +1342,10 @@ int cmd_clone(int argc,
>  	if (option_local > 0 && !is_local)
>  		warning(_("--local is ignored"));
>  
> +	create_object_database(the_repository);
We now explicitly create the object database here.
> +	if (option_required_reference.nr || option_optional_reference.nr)
> +		setup_reference();
Any reason the reference setup is also further deferred here?
-Justin
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 22 of 85 in “odb: write alternates at creation time”
  1. 0/8 odb: write alternates at creation timePatrick Steinhardt, Aug 25, 2026
  2. 1/8 builtin/clone: defer setup of the object databasePatrick Steinhardt, Aug 25, 2026
  3. 2/8 builtin/clone: move around `setup_reference()`Patrick Steinhardt, Aug 25, 2026
  4. 3/8 builtin/clone: refactor handling of "--reference{,-if-able}"Patrick Steinhardt, Aug 25, 2026
  5. Toon ClaesAug 28, 2026
  6. 4/8 builtin/clone: move setup of alternates for shared local clonesPatrick Steinhardt, Aug 25, 2026
  7. Toon ClaesAug 28, 2026
  8. 5/8 builtin/clone: move setup of alternates for non-shared local clonesPatrick Steinhardt, Aug 25, 2026
  9. Toon ClaesAug 28, 2026
  10. Patrick SteinhardtAug 31, 2026
  11. 6/8 odb/source: support writing alternates when creating the databasePatrick Steinhardt, Aug 25, 2026
  12. Toon ClaesAug 28, 2026
  13. Patrick SteinhardtAug 31, 2026
  14. Junio C HamanoAug 28, 2026
  15. Patrick SteinhardtAug 31, 2026
  16. 7/8 builtin/clone: write alternates via `odb_create_on_disk()`Patrick Steinhardt, Aug 25, 2026
  17. 8/8 odb/source: remove the ability to write alternatesPatrick Steinhardt, Aug 25, 2026
  18. Toon ClaesAug 28, 2026
  19. Patrick SteinhardtAug 31, 2026
  20. 0/8 odb: write alternates at creation timePatrick Steinhardt, Aug 31, 2026
  21. 1/8 builtin/clone: defer setup of the object databasePatrick Steinhardt, Aug 31, 2026
  22. Justin ToblerSep 6, 2026
  23. Patrick SteinhardtSep 7, 2026
  24. 2/8 builtin/clone: move around `setup_reference()`Patrick Steinhardt, Aug 31, 2026
  25. 3/8 builtin/clone: refactor handling of "--reference{,-if-able}"Patrick Steinhardt, Aug 31, 2026
  26. 4/8 builtin/clone: move setup of alternates for shared local clonesPatrick Steinhardt, Aug 31, 2026
  27. 5/8 builtin/clone: move setup of alternates for non-shared local clonesPatrick Steinhardt, Aug 31, 2026
  28. 6/8 odb/source: support writing alternates when creating the databasePatrick Steinhardt, Aug 31, 2026
  29. Justin ToblerSep 6, 2026
  30. Patrick SteinhardtSep 7, 2026
  31. 7/8 builtin/clone: write alternates via `odb_create_on_disk()`Patrick Steinhardt, Aug 31, 2026
  32. Justin ToblerSep 6, 2026
  33. 8/8 odb/source: remove the ability to write alternatesPatrick Steinhardt, Aug 31, 2026
  34. Toon ClaesSep 1, 2026
  35. 0/9 odb: write alternates at creation timePatrick Steinhardt, Sep 7, 2026
  36. 1/9 setup: split up concerns of `init_db()`Patrick Steinhardt, Sep 7, 2026
  37. Justin ToblerSep 8, 2026
  38. Patrick SteinhardtSep 9, 2026
  39. 2/9 builtin/clone: defer setup of the object databasePatrick Steinhardt, Sep 7, 2026
  40. Justin ToblerSep 8, 2026
  41. Patrick SteinhardtSep 9, 2026
  42. 3/9 builtin/clone: move around `setup_reference()`Patrick Steinhardt, Sep 7, 2026
  43. 4/9 builtin/clone: refactor handling of "--reference{,-if-able}"Patrick Steinhardt, Sep 7, 2026
  44. 5/9 builtin/clone: move setup of alternates for shared local clonesPatrick Steinhardt, Sep 7, 2026
  45. Justin ToblerSep 8, 2026
  46. Justin ToblerSep 8, 2026
  47. 6/9 builtin/clone: move setup of alternates for non-shared local clonesPatrick Steinhardt, Sep 7, 2026
  48. 7/9 odb/source: support writing alternates when creating the databasePatrick Steinhardt, Sep 7, 2026
  49. 8/9 builtin/clone: write alternates via `odb_create_on_disk()`Patrick Steinhardt, Sep 7, 2026
  50. 9/9 odb/source: remove the ability to write alternatesPatrick Steinhardt, Sep 7, 2026
  51. 0/9 odb: write alternates at creation timePatrick Steinhardt, Sep 9, 2026
  52. 1/9 setup: split up concerns of `init_db()`Patrick Steinhardt, Sep 9, 2026
  53. Karthik NayakSep 10, 2026
  54. 2/9 builtin/clone: defer setup of the object databasePatrick Steinhardt, Sep 9, 2026
  55. Karthik NayakSep 10, 2026
  56. 3/9 builtin/clone: move around `setup_reference()`Patrick Steinhardt, Sep 9, 2026
  57. Karthik NayakSep 10, 2026
  58. 4/9 builtin/clone: refactor handling of "--reference{,-if-able}"Patrick Steinhardt, Sep 9, 2026
  59. Karthik NayakSep 10, 2026
  60. Patrick SteinhardtSep 10, 2026
  61. 5/9 builtin/clone: move setup of alternates for shared local clonesPatrick Steinhardt, Sep 9, 2026
  62. Karthik NayakSep 10, 2026
  63. Karthik NayakSep 10, 2026
  64. Patrick SteinhardtSep 10, 2026
  65. 6/9 builtin/clone: move setup of alternates for non-shared local clonesPatrick Steinhardt, Sep 9, 2026
  66. Karthik NayakSep 10, 2026
  67. 7/9 odb/source: support writing alternates when creating the databasePatrick Steinhardt, Sep 9, 2026
  68. Karthik NayakSep 10, 2026
  69. Patrick SteinhardtSep 11, 2026
  70. 8/9 builtin/clone: write alternates via `odb_create_on_disk()`Patrick Steinhardt, Sep 9, 2026
  71. 9/9 odb/source: remove the ability to write alternatesPatrick Steinhardt, Sep 9, 2026
  72. Justin ToblerSep 9, 2026
  73. 0/9 odb: write alternates at creation timePatrick Steinhardt, Sep 10, 2026
  74. 1/9 setup: split up concerns of `init_db()`Patrick Steinhardt, Sep 10, 2026
  75. 2/9 builtin/clone: defer setup of the object databasePatrick Steinhardt, Sep 10, 2026
  76. 3/9 builtin/clone: move around `setup_reference()`Patrick Steinhardt, Sep 10, 2026
  77. 4/9 builtin/clone: refactor handling of "--reference{,-if-able}"Patrick Steinhardt, Sep 10, 2026
  78. 5/9 builtin/clone: move setup of alternates for shared local clonesPatrick Steinhardt, Sep 10, 2026
  79. 6/9 builtin/clone: move setup of alternates for non-shared local clonesPatrick Steinhardt, Sep 10, 2026
  80. 7/9 odb/source: support writing alternates when creating the databasePatrick Steinhardt, Sep 10, 2026
  81. 8/9 builtin/clone: write alternates via `odb_create_on_disk()`Patrick Steinhardt, Sep 10, 2026
  82. 9/9 odb/source: remove the ability to write alternatesPatrick Steinhardt, Sep 10, 2026
  83. Karthik NayakSep 10, 2026
  84. Patrick SteinhardtSep 11, 2026
  85. Karthik NayakSep 14, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.