{"thread":{"id":"60639","subject":"[PATCH 00/12] Introduce `refStorage` extension","startedAt":"2023-12-20T10:54:55Z","lastAt":"2023-12-29T08:44:14Z","messageCount":60,"participants":["Patrick Steinhardt","Junio C Hamano","Karthik Nayak","Eric Sunshine"],"isPatch":true,"patchVersion":1,"patchTotal":12},"messages":[{"id":"485849","messageId":"cover.1703067989.git.ps@pks.im","threadId":"60639","inReplyTo":null,"subject":"[PATCH 00/12] Introduce `refStorage` extension","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-20T10:54:50Z","receivedAt":"2023-12-20T10:54:55Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Hi,\n\nthis patch series introduces a new `refStorage` extension and related\ntooling. While there is only a single user-visible ref backend for now,\nthis extension will eventually allow us to determine which backend shall\nbe used for a repository. Furthermore, the series introduces tooling so\nthat the ref storage format becomes more accessible.\n\nThe series is structured as follows:\n\n - Patches 1 - 3: preliminary refactorings that prepare for the\n introduction of the ref storage format.\n\n - Patches 4 - 6: wire up the ref format when setting up the repository\n and creating a new one.\n\n - Patches 7 - 8: introduction of envvars to control the ref format\n globally.\n\n - Patches 9 - 11: amend our tooling to know how to access and set the\n ref storage format.\n\n - Patch 12: small patch for t9500 to make it handle ref storage\n extensions in the future.\n\nI've been kind of torn regarding how to name this in the user-facing\nparts. Even though the extension is called \"refStorage\", I rather opted\nto use \"ref format\" in the user-facing parts, which is similar in nature\nto the \"object format\". Changing the extension to be called \"refFormat\"\nwould cause issues for JGit though, so it's likely not an option to\nchange it now.\n\nAnyway, I'd appreciate your thoughts and proposals and am happy to\nrename the user-facing parts if we get to a preferable agreement.\n\nThis series depends on 18c9cb7524 (builtin/clone: create the refdb with\nthe correct object format, 2023-12-12) because it relies on the more\ncareful setup of the repository's refdb during clones.\n\nThanks!\n\nPatrick\n\nPatrick Steinhardt (12):\n t: introduce DEFAULT_REPO_FORMAT prereq\n worktree: skip reading HEAD when repairing worktrees\n refs: refactor logic to look up storage backends\n setup: start tracking ref storage format when\n setup: set repository's formats on init\n setup: introduce \"extensions.refStorage\" extension\n setup: introduce GIT_DEFAULT_REF_FORMAT envvar\n t: introduce GIT_TEST_DEFAULT_REF_FORMAT envvar\n builtin/rev-parse: introduce `--show-ref-format` flag\n builtin/init: introduce `--ref-format=` value flag\n builtin/clone: introduce `--ref-format=` value flag\n t9500: write \"extensions.refstorage\" into config\n\n Documentation/config/extensions.txt | 11 +++\n Documentation/git-clone.txt | 6 ++\n Documentation/git-init.txt | 7 ++\n Documentation/git-rev-parse.txt | 3 +\n Documentation/git.txt | 5 ++\n Documentation/ref-storage-format.txt | 1 +\n .../technical/repository-version.txt | 5 ++\n builtin/clone.c | 17 ++++-\n builtin/init-db.c | 15 +++-\n builtin/rev-parse.c | 4 ++\n refs.c | 34 ++++++---\n refs.h | 4 ++\n refs/debug.c | 1 -\n refs/files-backend.c | 1 -\n refs/packed-backend.c | 1 -\n refs/refs-internal.h | 1 -\n repository.c | 1 +\n repository.h | 6 ++\n setup.c | 63 +++++++++++++++--\n setup.h | 9 ++-\n t/README | 3 +\n t/t0001-init.sh | 70 +++++++++++++++++++\n t/t1500-rev-parse.sh | 17 +++++\n t/t3200-branch.sh | 2 +-\n t/t5601-clone.sh | 17 +++++\n t/t9500-gitweb-standalone-no-errors.sh | 5 ++\n t/test-lib-functions.sh | 5 ++\n t/test-lib.sh | 15 +++-\n worktree.c | 24 ++++---\n 29 files changed, 318 insertions(+), 35 deletions(-)\n create mode 100644 Documentation/ref-storage-format.txt\n\n-- \n2.43.GIT\n\n"},{"id":"485850","messageId":"239ca38efdbe3195e6319be4423b17d2b42e0b9f.1703067989.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703067989.git.ps@pks.im","subject":"[PATCH 01/12] t: introduce DEFAULT_REPO_FORMAT prereq","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-20T10:54:54Z","receivedAt":"2023-12-20T10:55:05Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"A limited number of tests require repositories to have the default\nrepository format or otherwise they would fail to run, e.g. because they\nfail to detect the correct hash function. While the hash function is the\nonly extension right now that creates problems like this, we are about\nto add a second extensions for the ref format.\n\nIntroduce a new DEFAULT_REPO_FORMAT prereq that can easily be amended\nwhenever we add new format extensions. Next to making any such changes\neasier on us, the prerequisite's name should also help to clarify the\nintent better.\n\nSigned-off-by: Patrick Steinhardt \n---\n t/t3200-branch.sh | 2 +-\n t/test-lib.sh | 4 ++++\n 2 files changed, 5 insertions(+), 1 deletion(-)\n\ndiff --git a/t/t3200-branch.sh b/t/t3200-branch.sh\nindex 6a316f081e..de7d3014e4 100755\n--- a/t/t3200-branch.sh\n+++ b/t/t3200-branch.sh\n@@ -519,7 +519,7 @@ EOF\n \n mv .git/config .git/config-saved\n \n-test_expect_success SHA1 'git branch -m q q2 without config should succeed' '\n+test_expect_success DEFAULT_REPO_FORMAT 'git branch -m q q2 without config should succeed' '\n \tgit branch -m q q2 &&\n \tgit branch -m q2 q\n '\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 876b99562a..dc03f06b8e 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1936,6 +1936,10 @@ test_lazy_prereq SHA1 '\n \tesac\n '\n \n+test_lazy_prereq DEFAULT_REPO_FORMAT '\n+\ttest_have_prereq SHA1\n+'\n+\n # Ensure that no test accidentally triggers a Git command\n # that runs the actual maintenance scheduler, affecting a user's\n # system permanently.\n-- \n2.43.GIT\n\n"},{"id":"485851","messageId":"e895091025e6786e0a831e4fb9b092eb74bd4379.1703067989.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703067989.git.ps@pks.im","subject":"[PATCH 02/12] worktree: skip reading HEAD when repairing worktrees","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-20T10:54:58Z","receivedAt":"2023-12-20T10:55:05Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"When calling `git init --separate-git-dir=` on a preexisting\nrepository, then we move the Git directory of that repository to the new\npath specified by the user. If there are worktrees present in the Git\nrepository, we need to repair the worktrees so that their gitlinks point\nto the new location of the repository.\n\nThis repair logic will load repositories via `get_worktrees()`, which\nwill enumerate up and initialize all worktrees. Part of initialization\nis logic that we resolve their respective worktree HEADs, even though\nthat information may not actually be needed in the end by all callers.\n\nIn the context of git-init(1) this is about to become a problem, because\nwe do not have a repository that was set up via `setup_git_directory()`\nor friends. Consequentially, it is not yet fully initialized at the time\nof calling `repair_worktrees()`, and properly setting up all parts of\nthe repository in `init_db()` before we repair worktrees is not an easy\nthing to do. While this is okay right now where we only have a single\nreference backend in Git, once we gain a second one we would be trying\nto look up the worktree HEADs before we have figured out the reference\nformat, which does not work.\n\nWe do not require the worktree HEADs at all to repair worktrees. So\nlet's fix issue this by skipping over the step that reads them.\n\nSigned-off-by: Patrick Steinhardt \n---\n worktree.c | 24 ++++++++++++++++--------\n 1 file changed, 16 insertions(+), 8 deletions(-)\n\ndiff --git a/worktree.c b/worktree.c\nindex a56a6c2a3d..9702ed0308 100644\n--- a/worktree.c\n+++ b/worktree.c\n@@ -51,7 +51,7 @@ static void add_head_info(struct worktree *wt)\n /**\n * get the main worktree\n */\n-static struct worktree *get_main_worktree(void)\n+static struct worktree *get_main_worktree(int skip_reading_head)\n {\n \tstruct worktree *worktree = NULL;\n \tstruct strbuf worktree_path = STRBUF_INIT;\n@@ -70,11 +70,13 @@ static struct worktree *get_main_worktree(void)\n \t */\n \tworktree->is_bare = (is_bare_repository_cfg == 1) ||\n \t\tis_bare_repository();\n-\tadd_head_info(worktree);\n+\tif (!skip_reading_head)\n+\t\tadd_head_info(worktree);\n \treturn worktree;\n }\n \n-static struct worktree *get_linked_worktree(const char *id)\n+static struct worktree *get_linked_worktree(const char *id,\n+\t\t\t\t\t int skip_reading_head)\n {\n \tstruct worktree *worktree = NULL;\n \tstruct strbuf path = STRBUF_INIT;\n@@ -93,7 +95,8 @@ static struct worktree *get_linked_worktree(const char *id)\n \tCALLOC_ARRAY(worktree, 1);\n \tworktree->path = strbuf_detach(&worktree_path, NULL);\n \tworktree->id = xstrdup(id);\n-\tadd_head_info(worktree);\n+\tif (!skip_reading_head)\n+\t\tadd_head_info(worktree);\n \n done:\n \tstrbuf_release(&path);\n@@ -118,7 +121,7 @@ static void mark_current_worktree(struct worktree **worktrees)\n \tfree(git_dir);\n }\n \n-struct worktree **get_worktrees(void)\n+static struct worktree **get_worktrees_internal(int skip_reading_head)\n {\n \tstruct worktree **list = NULL;\n \tstruct strbuf path = STRBUF_INIT;\n@@ -128,7 +131,7 @@ struct worktree **get_worktrees(void)\n \n \tALLOC_ARRAY(list, alloc);\n \n-\tlist[counter++] = get_main_worktree();\n+\tlist[counter++] = get_main_worktree(skip_reading_head);\n \n \tstrbuf_addf(&path, \"%s/worktrees\", get_git_common_dir());\n \tdir = opendir(path.buf);\n@@ -137,7 +140,7 @@ struct worktree **get_worktrees(void)\n \t\twhile ((d = readdir_skip_dot_and_dotdot(dir)) != NULL) {\n \t\t\tstruct worktree *linked = NULL;\n \n-\t\t\tif ((linked = get_linked_worktree(d->d_name))) {\n+\t\t\tif ((linked = get_linked_worktree(d->d_name, skip_reading_head))) {\n \t\t\t\tALLOC_GROW(list, counter + 1, alloc);\n \t\t\t\tlist[counter++] = linked;\n \t\t\t}\n@@ -151,6 +154,11 @@ struct worktree **get_worktrees(void)\n \treturn list;\n }\n \n+struct worktree **get_worktrees(void)\n+{\n+\treturn get_worktrees_internal(0);\n+}\n+\n const char *get_worktree_git_dir(const struct worktree *wt)\n {\n \tif (!wt)\n@@ -591,7 +599,7 @@ static void repair_noop(int iserr UNUSED,\n \n void repair_worktrees(worktree_repair_fn fn, void *cb_data)\n {\n-\tstruct worktree **worktrees = get_worktrees();\n+\tstruct worktree **worktrees = get_worktrees_internal(1);\n \tstruct worktree **wt = worktrees + 1; /* +1 skips main worktree */\n \n \tif (!fn)\n-- \n2.43.GIT\n\n"},{"id":"485852","messageId":"f712d5ef5bc5bed423cf50a0d0489095ba103df7.1703067989.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703067989.git.ps@pks.im","subject":"[PATCH 03/12] refs: refactor logic to look up storage backends","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-20T10:55:03Z","receivedAt":"2023-12-20T10:55:06Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"In order to look up ref storage backends, we're currently using a linked\nlist of backends, where each backend is expected to set up its `next`\npointer to the next ref storage backend. This is kind of a weird setup\nas backends need to be aware of other backends without much of a reason.\n\nRefactor the code so that the array of backends is centrally defined in\n\"refs.c\", where each backend is now identified by an integer constant.\nExpose functions to translate from those integer constants to the name\nand vice versa, which will be required by subsequent patches.\n\nSigned-off-by: Patrick Steinhardt \n---\n refs.c | 34 +++++++++++++++++++++++++---------\n refs.h | 3 +++\n refs/debug.c | 1 -\n refs/files-backend.c | 1 -\n refs/packed-backend.c | 1 -\n refs/refs-internal.h | 1 -\n repository.h | 3 +++\n 7 files changed, 31 insertions(+), 13 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 16bfa21df7..e87c85897d 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -33,17 +33,33 @@\n /*\n * List of all available backends\n */\n-static struct ref_storage_be *refs_backends = &refs_be_files;\n+static const struct ref_storage_be *refs_backends[] = {\n+\t[REF_STORAGE_FORMAT_FILES] = &refs_be_files,\n+};\n \n-static struct ref_storage_be *find_ref_storage_backend(const char *name)\n+static const struct ref_storage_be *find_ref_storage_backend(int ref_storage_format)\n {\n-\tstruct ref_storage_be *be;\n-\tfor (be = refs_backends; be; be = be->next)\n-\t\tif (!strcmp(be->name, name))\n-\t\t\treturn be;\n+\tif (ref_storage_format && ref_storage_format < ARRAY_SIZE(refs_backends))\n+\t\treturn refs_backends[ref_storage_format];\n \treturn NULL;\n }\n \n+int ref_storage_format_by_name(const char *name)\n+{\n+\tfor (int i = 0; i < ARRAY_SIZE(refs_backends); i++)\n+\t\tif (refs_backends[i] && !strcmp(refs_backends[i]->name, name))\n+\t\t\treturn i;\n+\treturn REF_STORAGE_FORMAT_UNKNOWN;\n+}\n+\n+const char *ref_storage_format_to_name(int ref_storage_format)\n+{\n+\tconst struct ref_storage_be *be = find_ref_storage_backend(ref_storage_format);\n+\tif (!be)\n+\t\treturn \"unknown\";\n+\treturn be->name;\n+}\n+\n /*\n * How to handle various characters in refnames:\n * 0: An acceptable character for refs\n@@ -2029,12 +2045,12 @@ static struct ref_store *ref_store_init(struct repository *repo,\n \t\t\t\t\tconst char *gitdir,\n \t\t\t\t\tunsigned int flags)\n {\n-\tconst char *be_name = \"files\";\n-\tstruct ref_storage_be *be = find_ref_storage_backend(be_name);\n+\tint format = REF_STORAGE_FORMAT_FILES;\n+\tconst struct ref_storage_be *be = find_ref_storage_backend(format);\n \tstruct ref_store *refs;\n \n \tif (!be)\n-\t\tBUG(\"reference backend %s is unknown\", be_name);\n+\t\tBUG(\"reference backend %s is unknown\", ref_storage_format_to_name(format));\n \n \trefs = be->init(repo, gitdir, flags);\n \treturn refs;\ndiff --git a/refs.h b/refs.h\nindex 23211a5ea1..c6571bcf6c 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -11,6 +11,9 @@ struct string_list;\n struct string_list_item;\n struct worktree;\n \n+int ref_storage_format_by_name(const char *name);\n+const char *ref_storage_format_to_name(int ref_storage_format);\n+\n /*\n * Resolve a reference, recursively following symbolic refererences.\n *\ndiff --git a/refs/debug.c b/refs/debug.c\nindex 83b7a0ba65..b9775f2c37 100644\n--- a/refs/debug.c\n+++ b/refs/debug.c\n@@ -426,7 +426,6 @@ static int debug_reflog_expire(struct ref_store *ref_store, const char *refname,\n }\n \n struct ref_storage_be refs_be_debug = {\n-\t.next = NULL,\n \t.name = \"debug\",\n \t.init = NULL,\n \t.init_db = debug_init_db,\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex ad8b1d143f..43fd0ac760 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3241,7 +3241,6 @@ static int files_init_db(struct ref_store *ref_store, struct strbuf *err UNUSED)\n }\n \n struct ref_storage_be refs_be_files = {\n-\t.next = NULL,\n \t.name = \"files\",\n \t.init = files_ref_store_create,\n \t.init_db = files_init_db,\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex b9fa097a29..8d1090e284 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1705,7 +1705,6 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n }\n \n struct ref_storage_be refs_be_packed = {\n-\t.next = NULL,\n \t.name = \"packed\",\n \t.init = packed_ref_store_create,\n \t.init_db = packed_init_db,\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 4af83bf9a5..8e9f04cc67 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -663,7 +663,6 @@ typedef int read_symbolic_ref_fn(struct ref_store *ref_store, const char *refnam\n \t\t\t\t struct strbuf *referent);\n \n struct ref_storage_be {\n-\tstruct ref_storage_be *next;\n \tconst char *name;\n \tref_store_init_fn *init;\n \tref_init_db_fn *init_db;\ndiff --git a/repository.h b/repository.h\nindex 5f18486f64..ea4c488b81 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -24,6 +24,9 @@ enum fetch_negotiation_setting {\n \tFETCH_NEGOTIATION_NOOP,\n };\n \n+#define REF_STORAGE_FORMAT_UNKNOWN 0\n+#define REF_STORAGE_FORMAT_FILES 1\n+\n struct repo_settings {\n \tint initialized;\n \n-- \n2.43.GIT\n\n"},{"id":"485853","messageId":"6564659d403de098799ddb8101b74c2803a655d4.1703067989.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703067989.git.ps@pks.im","subject":"[PATCH 04/12] setup: start tracking ref storage format when","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-20T10:55:07Z","receivedAt":"2023-12-20T10:55:11Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"In order to discern which ref storage format a repository is supposed to\nuse we need to start setting up and/or discovering the format. This\nneeds to happen in two separate code paths.\n\n - The first path is when we create a repository via `init_db()`. When\n we are re-initializing a preexisting repository we need to retain\n the previously used ref storage format -- if the user asked for a\n different format then this indicates an erorr and we error out.\n Otherwise we either initialize the repository with the format asked\n for by the user or the default format, which currently is the\n \"files\" backend.\n\n - The second path is when discovering repositories, where we need to\n read the config of that repository. There is not yet any way to\n configure something other than the \"files\" backend, so we can just\n blindly set the ref storage format to this backend.\n\nWire up this logic so that we have the ref storage format always readily\navailable when needed. As there is only a single backend and because it\nis not configurable we cannot yet verify that this tracking works as\nexpected via tests, but tests will be added in subsequent commits. To\ncountermand this ommission now though, raise a BUG() in case the ref\nstorage format is not set up properly in `ref_store_init()`.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/clone.c | 5 +++--\n builtin/init-db.c | 4 +++-\n refs.c | 6 +++---\n refs.h | 1 +\n repository.c | 1 +\n repository.h | 3 +++\n setup.c | 26 +++++++++++++++++++++++---\n setup.h | 6 +++++-\n 8 files changed, 42 insertions(+), 10 deletions(-)\n\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex 339af10c10..6840064b59 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -1105,7 +1105,8 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \t * repository, and reference backends may persist that information into\n \t * their on-disk data structures.\n \t */\n-\tinit_db(git_dir, real_git_dir, option_template, GIT_HASH_UNKNOWN, NULL,\n+\tinit_db(git_dir, real_git_dir, option_template, GIT_HASH_UNKNOWN,\n+\t\tREF_STORAGE_FORMAT_UNKNOWN, NULL,\n \t\tdo_not_override_repo_unix_permissions, INIT_DB_QUIET | INIT_DB_SKIP_REFDB);\n \n \tif (real_git_dir) {\n@@ -1290,7 +1291,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \thash_algo = hash_algo_by_ptr(transport_get_hash_algo(transport));\n \tinitialize_repository_version(hash_algo, 1);\n \trepo_set_hash_algo(the_repository, hash_algo);\n-\tcreate_reference_database(NULL, 1);\n+\tcreate_reference_database(the_repository->ref_storage_format, NULL, 1);\n \n \t/*\n \t * Before fetching from the remote, download and install bundle\ndiff --git a/builtin/init-db.c b/builtin/init-db.c\nindex cb727c826f..b6e80feab6 100644\n--- a/builtin/init-db.c\n+++ b/builtin/init-db.c\n@@ -11,6 +11,7 @@\n #include \"object-file.h\"\n #include \"parse-options.h\"\n #include \"path.h\"\n+#include \"refs.h\"\n #include \"setup.h\"\n #include \"strbuf.h\"\n \n@@ -236,5 +237,6 @@ int cmd_init_db(int argc, const char **argv, const char *prefix)\n \n \tflags |= INIT_DB_EXIST_OK;\n \treturn init_db(git_dir, real_git_dir, template_dir, hash_algo,\n-\t\t initial_branch, init_shared_repository, flags);\n+\t\t REF_STORAGE_FORMAT_UNKNOWN, initial_branch,\n+\t\t init_shared_repository, flags);\n }\ndiff --git a/refs.c b/refs.c\nindex e87c85897d..d289a5e175 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -2045,12 +2045,12 @@ static struct ref_store *ref_store_init(struct repository *repo,\n \t\t\t\t\tconst char *gitdir,\n \t\t\t\t\tunsigned int flags)\n {\n-\tint format = REF_STORAGE_FORMAT_FILES;\n-\tconst struct ref_storage_be *be = find_ref_storage_backend(format);\n+\tconst struct ref_storage_be *be;\n \tstruct ref_store *refs;\n \n+\tbe = find_ref_storage_backend(repo->ref_storage_format);\n \tif (!be)\n-\t\tBUG(\"reference backend %s is unknown\", ref_storage_format_to_name(format));\n+\t\tBUG(\"reference backend is unknown\");\n \n \trefs = be->init(repo, gitdir, flags);\n \treturn refs;\ndiff --git a/refs.h b/refs.h\nindex c6571bcf6c..944a67ac1b 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -11,6 +11,7 @@ struct string_list;\n struct string_list_item;\n struct worktree;\n \n+int default_ref_storage_format(void);\n int ref_storage_format_by_name(const char *name);\n const char *ref_storage_format_to_name(int ref_storage_format);\n \ndiff --git a/repository.c b/repository.c\nindex a7679ceeaa..a75c1598b0 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -184,6 +184,7 @@ int repo_init(struct repository *repo,\n \t\tgoto error;\n \n \trepo_set_hash_algo(repo, format.hash_algo);\n+\trepo->ref_storage_format = format.ref_storage_format;\n \trepo->repository_format_worktree_config = format.worktree_config;\n \n \t/* take ownership of format.partial_clone */\ndiff --git a/repository.h b/repository.h\nindex ea4c488b81..22c30cdbc9 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -163,6 +163,9 @@ struct repository {\n \t/* Repository's current hash algorithm, as serialized on disk. */\n \tconst struct git_hash_algo *hash_algo;\n \n+\t/* Repository's reference storage format, as serialized on disk. */\n+\tint ref_storage_format;\n+\n \t/* A unique-id for tracing purposes. */\n \tint trace2_repo_id;\n \ndiff --git a/setup.c b/setup.c\nindex 155fe13f70..c1bf106379 100644\n--- a/setup.c\n+++ b/setup.c\n@@ -1564,6 +1564,8 @@ const char *setup_git_directory_gently(int *nongit_ok)\n \t\t}\n \t\tif (startup_info->have_repository) {\n \t\t\trepo_set_hash_algo(the_repository, repo_fmt.hash_algo);\n+\t\t\tthe_repository->ref_storage_format =\n+\t\t\t\trepo_fmt.ref_storage_format;\n \t\t\tthe_repository->repository_format_worktree_config =\n \t\t\t\trepo_fmt.worktree_config;\n \t\t\t/* take ownership of repo_fmt.partial_clone */\n@@ -1657,6 +1659,8 @@ void check_repository_format(struct repository_format *fmt)\n \tcheck_repository_format_gently(get_git_dir(), fmt, NULL);\n \tstartup_info->have_repository = 1;\n \trepo_set_hash_algo(the_repository, fmt->hash_algo);\n+\tthe_repository->ref_storage_format =\n+\t\tfmt->ref_storage_format;\n \tthe_repository->repository_format_worktree_config =\n \t\tfmt->worktree_config;\n \tthe_repository->repository_format_partial_clone =\n@@ -1897,7 +1901,8 @@ static int is_reinit(void)\n \treturn ret;\n }\n \n-void create_reference_database(const char *initial_branch, int quiet)\n+void create_reference_database(int ref_storage_format,\n+\t\t\t const char *initial_branch, int quiet)\n {\n \tstruct strbuf err = STRBUF_INIT;\n \tint reinit = is_reinit();\n@@ -1917,6 +1922,7 @@ void create_reference_database(const char *initial_branch, int quiet)\n \tsafe_create_dir(git_path(\"refs\"), 1);\n \tadjust_shared_perm(git_path(\"refs\"));\n \n+\tthe_repository->ref_storage_format = ref_storage_format;\n \tif (refs_init_db(&err))\n \t\tdie(\"failed to set up refs db: %s\", err.buf);\n \n@@ -2135,8 +2141,20 @@ static void validate_hash_algorithm(struct repository_format *repo_fmt, int hash\n \t}\n }\n \n+static void validate_ref_storage_format(struct repository_format *repo_fmt, int format)\n+{\n+\tif (repo_fmt->version >= 0 &&\n+\t format != REF_STORAGE_FORMAT_UNKNOWN &&\n+\t format != repo_fmt->ref_storage_format) {\n+\t\tdie(_(\"attempt to reinitialize repository with different reference storage format\"));\n+\t} else if (format != REF_STORAGE_FORMAT_UNKNOWN) {\n+\t\trepo_fmt->ref_storage_format = format;\n+\t}\n+}\n+\n int init_db(const char *git_dir, const char *real_git_dir,\n-\t const char *template_dir, int hash, const char *initial_branch,\n+\t const char *template_dir, int hash,\n+\t int ref_storage_format, const char *initial_branch,\n \t int init_shared_repository, unsigned int flags)\n {\n \tint reinit;\n@@ -2179,13 +2197,15 @@ int init_db(const char *git_dir, const char *real_git_dir,\n \tcheck_repository_format(&repo_fmt);\n \n \tvalidate_hash_algorithm(&repo_fmt, hash);\n+\tvalidate_ref_storage_format(&repo_fmt, ref_storage_format);\n \n \treinit = create_default_files(template_dir, original_git_dir,\n \t\t\t\t &repo_fmt, prev_bare_repository,\n \t\t\t\t init_shared_repository);\n \n \tif (!(flags & INIT_DB_SKIP_REFDB))\n-\t\tcreate_reference_database(initial_branch, flags & INIT_DB_QUIET);\n+\t\tcreate_reference_database(repo_fmt.ref_storage_format,\n+\t\t\t\t\t initial_branch, flags & INIT_DB_QUIET);\n \tcreate_object_directory();\n \n \tif (get_shared_repository()) {\ndiff --git a/setup.h b/setup.h\nindex 3f0f17c351..4927abf2cf 100644\n--- a/setup.h\n+++ b/setup.h\n@@ -115,6 +115,7 @@ struct repository_format {\n \tint worktree_config;\n \tint is_bare;\n \tint hash_algo;\n+\tint ref_storage_format;\n \tint sparse_index;\n \tchar *work_tree;\n \tstruct string_list unknown_extensions;\n@@ -131,6 +132,7 @@ struct repository_format {\n \t.version = -1, \\\n \t.is_bare = -1, \\\n \t.hash_algo = GIT_HASH_SHA1, \\\n+\t.ref_storage_format = REF_STORAGE_FORMAT_FILES, \\\n \t.unknown_extensions = STRING_LIST_INIT_DUP, \\\n \t.v1_only_extensions = STRING_LIST_INIT_DUP, \\\n }\n@@ -175,10 +177,12 @@ void check_repository_format(struct repository_format *fmt);\n \n int init_db(const char *git_dir, const char *real_git_dir,\n \t const char *template_dir, int hash_algo,\n+\t int ref_storage_format,\n \t const char *initial_branch, int init_shared_repository,\n \t unsigned int flags);\n void initialize_repository_version(int hash_algo, int reinit);\n-void create_reference_database(const char *initial_branch, int quiet);\n+void create_reference_database(int ref_storage_format,\n+\t\t\t const char *initial_branch, int quiet);\n \n /*\n * NOTE NOTE NOTE!!\n-- \n2.43.GIT\n\n"},{"id":"485854","messageId":"f90a63d63cfa2f0b9e19cdec4790a77d6a4b9b3b.1703067989.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703067989.git.ps@pks.im","subject":"[PATCH 05/12] setup: set repository's formats on init","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-20T10:55:12Z","receivedAt":"2023-12-20T10:55:15Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The proper hash algorithm and ref storage format that will be used for a\nnewly initialized repository will be figured out in `init_db()` via\n`validate_hash_algorithm()` and `validate_ref_storage_format()`. Until\nnow though, we never set up the hash algorithm or ref storage format of\n`the_repository` accordingly.\n\nThere are only two callsites of `init_db()`, one in git-init(1) and one\nin git-clone(1). The former function doesn't care for the formats to be\nset up properly because it never access the repository after calling the\nfunction in the first place.\n\nFor git-clone(1) it's a different story though, as we call `init_db()`\nbefore listing remote refs. While we do indeed have the wrong hash\nfunction in `the_repository` when `init_db()` sets up a non-default\nobject format for the repository, it never mattered because we adjust\nthe hash after learning about the remote's hash function via the listed\nrefs.\n\nSo the current state is correct for the hash algo, but it's not for the\nref storage format because git-clone(1) wouldn't know to set it up\nproperly. But instead of adjusting only the `ref_storage_format`, set\nboth the hash algo and the ref storage format so that `the_repository`\nis in the correct state when `init_db()` exits. This is fine as we will\nadjust the hash later on anyway and makes it easier to reason about the\nend state of `the_repository`.\n\nSigned-off-by: Patrick Steinhardt \n---\n setup.c | 7 +++++++\n 1 file changed, 7 insertions(+)\n\ndiff --git a/setup.c b/setup.c\nindex c1bf106379..5dffea6e2f 100644\n--- a/setup.c\n+++ b/setup.c\n@@ -2203,6 +2203,13 @@ int init_db(const char *git_dir, const char *real_git_dir,\n \t\t\t\t &repo_fmt, prev_bare_repository,\n \t\t\t\t init_shared_repository);\n \n+\t/*\n+\t * Now that we have set up both the hash algorithm and the ref storage\n+\t * format we can update the repository's settings accordingly.\n+\t */\n+\tthe_repository->hash_algo = &hash_algos[repo_fmt.hash_algo];\n+\tthe_repository->ref_storage_format = repo_fmt.ref_storage_format;\n+\n \tif (!(flags & INIT_DB_SKIP_REFDB))\n \t\tcreate_reference_database(repo_fmt.ref_storage_format,\n \t\t\t\t\t initial_branch, flags & INIT_DB_QUIET);\n-- \n2.43.GIT\n\n"},{"id":"485855","messageId":"beeb182f2844bd8a9e1f2a2767838282f705e0a2.1703067989.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703067989.git.ps@pks.im","subject":"[PATCH 06/12] setup: introduce \"extensions.refStorage\" extension","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-20T10:55:16Z","receivedAt":"2023-12-20T10:55:18Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Introduce a new \"extensions.refStorage\" extension that allows us to\nspecify the ref storage format used by a repository. For now, the only\nsupported format is the \"files\" format, but this list will likely soon\nbe extended to also support the upcoming \"reftable\" format.\n\nThere have been discussions on the Git mailing list in the past around\nhow exactly this extension should look like. One alternative [1] that\nwas discussed was whether it would make sense to model the extension in\nsuch a way that backends are arbitrarily stackable. This would allow for\na combined value of e.g. \"loose,packed-refs\" or \"loose,reftable\", which\nindicates that new refs would be written via \"loose\" files backend and\ncompressed into \"packed-refs\" or \"reftable\" backends, respectively.\n\nIt is arguable though whether this flexibility and the complexity that\nit brings with it is really required for now. It is not foreseeable that\nthere will be a proliferation of backends in the near-term future, and\nthe current set of existing formats and formats which are on the horizon\ncan easily be configured with the much simpler proposal where we have a\nsingle value, only.\n\nFurthermore, if we ever see that we indeed want to gain the ability to\narbitrarily stack the ref formats, then we can adapt the current\nextension rather easily. Given that Git clients will refuse any unknown\nvalue for the \"extensions.refStorage\" extension they would also know to\nignore a stacked \"loose,packed-refs\" in the future.\n\nSo let's stick with the easy proposal for the time being and wire up the\nextension.\n\n[1]: \n\nSigned-off-by: Patrick Steinhardt \n---\n Documentation/config/extensions.txt | 11 ++++++++\n Documentation/ref-storage-format.txt | 1 +\n .../technical/repository-version.txt | 5 ++++\n builtin/clone.c | 2 +-\n setup.c | 23 +++++++++++++---\n setup.h | 3 ++-\n t/t0001-init.sh | 26 +++++++++++++++++++\n t/test-lib.sh | 2 +-\n 8 files changed, 67 insertions(+), 6 deletions(-)\n create mode 100644 Documentation/ref-storage-format.txt\n\ndiff --git a/Documentation/config/extensions.txt b/Documentation/config/extensions.txt\nindex bccaec7a96..66db0e15da 100644\n--- a/Documentation/config/extensions.txt\n+++ b/Documentation/config/extensions.txt\n@@ -7,6 +7,17 @@ Note that this setting should only be set by linkgit:git-init[1] or\n linkgit:git-clone[1]. Trying to change it after initialization will not\n work and will produce hard-to-diagnose issues.\n \n+extensions.refStorage::\n+\tSpecify the ref storage format to use. The acceptable values are:\n++\n+include::../ref-storage-format.txt[]\n++\n+It is an error to specify this key unless `core.repositoryFormatVersion` is 1.\n++\n+Note that this setting should only be set by linkgit:git-init[1] or\n+linkgit:git-clone[1]. Trying to change it after initialization will not\n+work and will produce hard-to-diagnose issues.\n+\n extensions.worktreeConfig::\n \tIf enabled, then worktrees will load config settings from the\n \t`$GIT_DIR/config.worktree` file in addition to the\ndiff --git a/Documentation/ref-storage-format.txt b/Documentation/ref-storage-format.txt\nnew file mode 100644\nindex 0000000000..1a65cac468\n--- /dev/null\n+++ b/Documentation/ref-storage-format.txt\n@@ -0,0 +1 @@\n+* `files` for loose files with packed-refs. This is the default.\ndiff --git a/Documentation/technical/repository-version.txt b/Documentation/technical/repository-version.txt\nindex 045a76756f..27be3741e6 100644\n--- a/Documentation/technical/repository-version.txt\n+++ b/Documentation/technical/repository-version.txt\n@@ -100,3 +100,8 @@ If set, by default \"git config\" reads from both \"config\" and\n multiple working directory mode, \"config\" file is shared while\n \"config.worktree\" is per-working directory (i.e., it's in\n GIT_COMMON_DIR/worktrees//config.worktree)\n+\n+==== `refStorage`\n+\n+Specifies the file format for the ref database. The only valid value\n+is `files` (loose references with a packed-refs file).\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex 6840064b59..20c161e94a 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -1289,7 +1289,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \t * ours to the same thing.\n \t */\n \thash_algo = hash_algo_by_ptr(transport_get_hash_algo(transport));\n-\tinitialize_repository_version(hash_algo, 1);\n+\tinitialize_repository_version(hash_algo, the_repository->ref_storage_format, 1);\n \trepo_set_hash_algo(the_repository, hash_algo);\n \tcreate_reference_database(the_repository->ref_storage_format, NULL, 1);\n \ndiff --git a/setup.c b/setup.c\nindex 5dffea6e2f..7eb4ae8ea0 100644\n--- a/setup.c\n+++ b/setup.c\n@@ -590,6 +590,17 @@ static enum extension_result handle_extension(const char *var,\n \t\t\t\t \"extensions.objectformat\", value);\n \t\tdata->hash_algo = format;\n \t\treturn EXTENSION_OK;\n+\t} else if (!strcmp(ext, \"refstorage\")) {\n+\t\tint format;\n+\n+\t\tif (!value)\n+\t\t\treturn config_error_nonbool(var);\n+\t\tformat = ref_storage_format_by_name(value);\n+\t\tif (format == REF_STORAGE_FORMAT_UNKNOWN)\n+\t\t\treturn error(_(\"invalid value for '%s': '%s'\"),\n+\t\t\t\t \"extensions.refstorage\", value);\n+\t\tdata->ref_storage_format = format;\n+\t\treturn EXTENSION_OK;\n \t}\n \treturn EXTENSION_UNKNOWN;\n }\n@@ -1869,12 +1880,14 @@ static int needs_work_tree_config(const char *git_dir, const char *work_tree)\n \treturn 1;\n }\n \n-void initialize_repository_version(int hash_algo, int reinit)\n+void initialize_repository_version(int hash_algo, int ref_storage_format,\n+\t\t\t\t int reinit)\n {\n \tchar repo_version_string[10];\n \tint repo_version = GIT_REPO_VERSION;\n \n-\tif (hash_algo != GIT_HASH_SHA1)\n+\tif (hash_algo != GIT_HASH_SHA1 ||\n+\t ref_storage_format != REF_STORAGE_FORMAT_FILES)\n \t\trepo_version = GIT_REPO_VERSION_READ;\n \n \t/* This forces creation of new config file */\n@@ -1887,6 +1900,10 @@ void initialize_repository_version(int hash_algo, int reinit)\n \t\t\t hash_algos[hash_algo].name);\n \telse if (reinit)\n \t\tgit_config_set_gently(\"extensions.objectformat\", NULL);\n+\n+\tif (ref_storage_format != REF_STORAGE_FORMAT_FILES)\n+\t\tgit_config_set(\"extensions.refstorage\",\n+\t\t\t ref_storage_format_to_name(ref_storage_format));\n }\n \n static int is_reinit(void)\n@@ -2028,7 +2045,7 @@ static int create_default_files(const char *template_path,\n \t\tadjust_shared_perm(get_git_dir());\n \t}\n \n-\tinitialize_repository_version(fmt->hash_algo, 0);\n+\tinitialize_repository_version(fmt->hash_algo, fmt->ref_storage_format, 0);\n \n \t/* Check filemode trustability */\n \tpath = git_path_buf(&buf, \"config\");\ndiff --git a/setup.h b/setup.h\nindex 4927abf2cf..89033ae1d9 100644\n--- a/setup.h\n+++ b/setup.h\n@@ -180,7 +180,8 @@ int init_db(const char *git_dir, const char *real_git_dir,\n \t int ref_storage_format,\n \t const char *initial_branch, int init_shared_repository,\n \t unsigned int flags);\n-void initialize_repository_version(int hash_algo, int reinit);\n+void initialize_repository_version(int hash_algo, int ref_storage_format,\n+\t\t\t\t int reinit);\n void create_reference_database(int ref_storage_format,\n \t\t\t const char *initial_branch, int quiet);\n \ndiff --git a/t/t0001-init.sh b/t/t0001-init.sh\nindex 2b78e3be47..38b3e4c39e 100755\n--- a/t/t0001-init.sh\n+++ b/t/t0001-init.sh\n@@ -532,6 +532,32 @@ test_expect_success 'init rejects attempts to initialize with different hash' '\n \ttest_must_fail git -C sha256 init --object-format=sha1\n '\n \n+test_expect_success DEFAULT_REPO_FORMAT 'extensions.refStorage is not allowed with repo version 0' '\n+\ttest_when_finished \"rm -rf refstorage\" &&\n+\tgit init refstorage &&\n+\tgit -C refstorage config extensions.refStorage files &&\n+\ttest_must_fail git -C refstorage rev-parse 2>err &&\n+\tgrep \"repo version is 0, but v1-only extension found\" err\n+'\n+\n+test_expect_success DEFAULT_REPO_FORMAT 'extensions.refStorage with files backend' '\n+\ttest_when_finished \"rm -rf refstorage\" &&\n+\tgit init refstorage &&\n+\tgit -C refstorage config core.repositoryformatversion 1 &&\n+\tgit -C refstorage config extensions.refStorage files &&\n+\ttest_commit -C refstorage A &&\n+\tgit -C refstorage rev-parse --verify HEAD\n+'\n+\n+test_expect_success DEFAULT_REPO_FORMAT 'extensions.refStorage with unknown backend' '\n+\ttest_when_finished \"rm -rf refstorage\" &&\n+\tgit init refstorage &&\n+\tgit -C refstorage config core.repositoryformatversion 1 &&\n+\tgit -C refstorage config extensions.refStorage garbage &&\n+\ttest_must_fail git -C refstorage rev-parse 2>err &&\n+\tgrep \"invalid value for ${SQ}extensions.refstorage${SQ}: ${SQ}garbage${SQ}\" err\n+'\n+\n test_expect_success MINGW 'core.hidedotfiles = false' '\n \tgit config --global core.hidedotfiles false &&\n \trm -rf newdir &&\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex dc03f06b8e..4685cc3d48 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1937,7 +1937,7 @@ test_lazy_prereq SHA1 '\n '\n \n test_lazy_prereq DEFAULT_REPO_FORMAT '\n-\ttest_have_prereq SHA1\n+\ttest_have_prereq SHA1,REFFILES\n '\n \n # Ensure that no test accidentally triggers a Git command\n-- \n2.43.GIT\n\n"},{"id":"485856","messageId":"dd91a75da49b167f983d11819c6f7330cdf4c11c.1703067989.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703067989.git.ps@pks.im","subject":"[PATCH 07/12] setup: introduce GIT_DEFAULT_REF_FORMAT envvar","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-20T10:55:20Z","receivedAt":"2023-12-20T10:55:23Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Introduce a new GIT_DEFAULT_REF_FORMAT environment variable that lets\nusers control the default ref format used by both git-init(1) and\ngit-clone(1). This is modeled after GIT_DEFAULT_OBJECT_FORMAT, which\ndoes the same thing for the repository's object format.\n\nSigned-off-by: Patrick Steinhardt \n---\n Documentation/git.txt | 5 +++++\n setup.c | 7 +++++++\n t/t0001-init.sh | 18 ++++++++++++++++++\n 3 files changed, 30 insertions(+)\n\ndiff --git a/Documentation/git.txt b/Documentation/git.txt\nindex 4698d7a42b..0ab19eef3b 100644\n--- a/Documentation/git.txt\n+++ b/Documentation/git.txt\n@@ -556,6 +556,11 @@ double-quotes and respecting backslash escapes. E.g., the value\n \tis always used. The default is \"sha1\".\n \tSee `--object-format` in linkgit:git-init[1].\n \n+`GIT_DEFAULT_REF_FORMAT`::\n+\tIf this variable is set, the default reference backend format for new\n+\trepositories will be set to this value. The default is \"files\".\n+\tSee `--ref-format` in linkgit:git-init[1].\n+\n Git Commits\n ~~~~~~~~~~~\n `GIT_AUTHOR_NAME`::\ndiff --git a/setup.c b/setup.c\nindex 7eb4ae8ea0..8913e30974 100644\n--- a/setup.c\n+++ b/setup.c\n@@ -2160,12 +2160,19 @@ static void validate_hash_algorithm(struct repository_format *repo_fmt, int hash\n \n static void validate_ref_storage_format(struct repository_format *repo_fmt, int format)\n {\n+\tconst char *name = getenv(\"GIT_DEFAULT_REF_FORMAT\");\n+\n \tif (repo_fmt->version >= 0 &&\n \t format != REF_STORAGE_FORMAT_UNKNOWN &&\n \t format != repo_fmt->ref_storage_format) {\n \t\tdie(_(\"attempt to reinitialize repository with different reference storage format\"));\n \t} else if (format != REF_STORAGE_FORMAT_UNKNOWN) {\n \t\trepo_fmt->ref_storage_format = format;\n+\t} else if (name) {\n+\t\tformat = ref_storage_format_by_name(name);\n+\t\tif (format == REF_STORAGE_FORMAT_UNKNOWN)\n+\t\t\tdie(_(\"unknown ref storage format '%s'\"), name);\n+\t\trepo_fmt->ref_storage_format = format;\n \t}\n }\n \ndiff --git a/t/t0001-init.sh b/t/t0001-init.sh\nindex 38b3e4c39e..30ce752cc1 100755\n--- a/t/t0001-init.sh\n+++ b/t/t0001-init.sh\n@@ -558,6 +558,24 @@ test_expect_success DEFAULT_REPO_FORMAT 'extensions.refStorage with unknown back\n \tgrep \"invalid value for ${SQ}extensions.refstorage${SQ}: ${SQ}garbage${SQ}\" err\n '\n \n+test_expect_success DEFAULT_REPO_FORMAT 'init with GIT_DEFAULT_REF_FORMAT=files' '\n+\ttest_when_finished \"rm -rf refformat\" &&\n+\tGIT_DEFAULT_REF_FORMAT=files git init refformat &&\n+\techo 0 >expect &&\n+\tgit -C refformat config core.repositoryformatversion >actual &&\n+\ttest_cmp expect actual &&\n+\ttest_must_fail git -C refformat config extensions.refstorage\n+'\n+\n+test_expect_success 'init with GIT_DEFAULT_REF_FORMAT=garbage' '\n+\ttest_when_finished \"rm -rf refformat\" &&\n+\tcat >expect <<-EOF &&\n+\tfatal: unknown ref storage format ${SQ}garbage${SQ}\n+\tEOF\n+\ttest_must_fail env GIT_DEFAULT_REF_FORMAT=garbage git init refformat 2>err &&\n+\ttest_cmp expect err\n+'\n+\n test_expect_success MINGW 'core.hidedotfiles = false' '\n \tgit config --global core.hidedotfiles false &&\n \trm -rf newdir &&\n-- \n2.43.GIT\n\n"},{"id":"485857","messageId":"ed3bf008cdb043fa51bb6c1e211760f0e83a813c.1703067989.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703067989.git.ps@pks.im","subject":"[PATCH 08/12] t: introduce GIT_TEST_DEFAULT_REF_FORMAT envvar","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-20T10:55:24Z","receivedAt":"2023-12-20T10:55:27Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Introduce a new GIT_TEST_DEFAULT_REF_FORMAT environment variable that\nlets developers run the test suite with a different default ref format\nwithout impacting the ref format used by non-test Git invocations. This\nis modeled after GIT_TEST_DEFAULT_OBJECT_FORMAT, which does the same\nthing for the repository's object format.\n\nAdapt the setup of the `REFFILES` test prerequisite to be conditionally\nset based on the default ref format.\n\nSigned-off-by: Patrick Steinhardt \n---\n t/README | 3 +++\n t/test-lib-functions.sh | 5 +++++\n t/test-lib.sh | 11 ++++++++++-\n 3 files changed, 18 insertions(+), 1 deletion(-)\n\ndiff --git a/t/README b/t/README\nindex 36463d0742..621d3b8c09 100644\n--- a/t/README\n+++ b/t/README\n@@ -479,6 +479,9 @@ GIT_TEST_DEFAULT_HASH= specifies which hash algorithm to\n use in the test scripts. Recognized values for are \"sha1\"\n and \"sha256\".\n \n+GIT_TEST_DEFAULT_REF_FORMAT= specifies which ref storage format\n+to use in the test scripts. Recognized values for are \"files\".\n+\n GIT_TEST_NO_WRITE_REV_INDEX=, when true disables the\n 'pack.writeReverseIndex' setting.\n \ndiff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh\nindex 03292602fb..61871b2a3b 100644\n--- a/t/test-lib-functions.sh\n+++ b/t/test-lib-functions.sh\n@@ -1659,6 +1659,11 @@ test_detect_hash () {\n \ttest_hash_algo=\"${GIT_TEST_DEFAULT_HASH:-sha1}\"\n }\n \n+# Detect the hash algorithm in use.\n+test_detect_ref_format () {\n+\techo \"${GIT_TEST_DEFAULT_REF_FORMAT:-files}\"\n+}\n+\n # Load common hash metadata and common placeholder object IDs for use with\n # test_oid.\n test_oid_init () {\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 4685cc3d48..fc93aa57e6 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -542,6 +542,8 @@ export EDITOR\n \n GIT_DEFAULT_HASH=\"${GIT_TEST_DEFAULT_HASH:-sha1}\"\n export GIT_DEFAULT_HASH\n+GIT_DEFAULT_REF_FORMAT=\"${GIT_TEST_DEFAULT_REF_FORMAT:-files}\"\n+export GIT_DEFAULT_REF_FORMAT\n GIT_TEST_MERGE_ALGORITHM=\"${GIT_TEST_MERGE_ALGORITHM:-ort}\"\n export GIT_TEST_MERGE_ALGORITHM\n \n@@ -1745,7 +1747,14 @@ parisc* | hppa*)\n \t;;\n esac\n \n-test_set_prereq REFFILES\n+case \"$GIT_DEFAULT_REF_FORMAT\" in\n+files)\n+\ttest_set_prereq REFFILES;;\n+*)\n+\techo 2>&1 \"error: unknown ref format $GIT_DEFAULT_REF_FORMAT\"\n+\texit 1\n+\t;;\n+esac\n \n ( COLUMNS=1 && test $COLUMNS = 1 ) && test_set_prereq COLUMNS_CAN_BE_1\n test -z \"$NO_CURL\" && test_set_prereq LIBCURL\n-- \n2.43.GIT\n\n"},{"id":"485858","messageId":"8a3d950d692e6c86f980b79768719b3f0f622403.1703067989.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703067989.git.ps@pks.im","subject":"[PATCH 09/12] builtin/rev-parse: introduce `--show-ref-format` flag","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-20T10:55:28Z","receivedAt":"2023-12-20T10:55:31Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Introduce a new `--show-ref-format` to git-rev-parse(1) that causes it\nto print the ref format used by a repository.\n\nSigned-off-by: Patrick Steinhardt \n---\n Documentation/git-rev-parse.txt | 3 +++\n builtin/rev-parse.c | 4 ++++\n t/t1500-rev-parse.sh | 17 +++++++++++++++++\n 3 files changed, 24 insertions(+)\n\ndiff --git a/Documentation/git-rev-parse.txt b/Documentation/git-rev-parse.txt\nindex 912fab9f5e..546faf9017 100644\n--- a/Documentation/git-rev-parse.txt\n+++ b/Documentation/git-rev-parse.txt\n@@ -307,6 +307,9 @@ The following options are unaffected by `--path-format`:\n \tinput, multiple algorithms may be printed, space-separated.\n \tIf not specified, the default is \"storage\".\n \n+--show-ref-format::\n+\tShow the reference storage format used for the repository.\n+\n \n Other Options\n ~~~~~~~~~~~~~\ndiff --git a/builtin/rev-parse.c b/builtin/rev-parse.c\nindex fde8861ca4..99725a6ff1 100644\n--- a/builtin/rev-parse.c\n+++ b/builtin/rev-parse.c\n@@ -1059,6 +1059,10 @@ int cmd_rev_parse(int argc, const char **argv, const char *prefix)\n \t\t\t\tputs(the_hash_algo->name);\n \t\t\t\tcontinue;\n \t\t\t}\n+\t\t\tif (!strcmp(arg, \"--show-ref-format\")) {\n+\t\t\t\tputs(ref_storage_format_to_name(the_repository->ref_storage_format));\n+\t\t\t\tcontinue;\n+\t\t\t}\n \t\t\tif (!strcmp(arg, \"--end-of-options\")) {\n \t\t\t\tseen_end_of_options = 1;\n \t\t\t\tif (filter & (DO_FLAGS | DO_REVS))\ndiff --git a/t/t1500-rev-parse.sh b/t/t1500-rev-parse.sh\nindex 3f9e7f62e4..a669e592f1 100755\n--- a/t/t1500-rev-parse.sh\n+++ b/t/t1500-rev-parse.sh\n@@ -208,6 +208,23 @@ test_expect_success 'rev-parse --show-object-format in repo' '\n \tgrep \"unknown mode for --show-object-format: squeamish-ossifrage\" err\n '\n \n+test_expect_success 'rev-parse --show-ref-format' '\n+\ttest_detect_ref_format >expect &&\n+\tgit rev-parse --show-ref-format >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'rev-parse --show-ref-format with invalid storage' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\tgit config extensions.refstorage broken &&\n+\t\ttest_must_fail git rev-parse --show-ref-format 2>err &&\n+\t\tgrep \"error: invalid value for ${SQ}extensions.refstorage${SQ}: ${SQ}broken${SQ}\" err\n+\t)\n+'\n+\n test_expect_success '--show-toplevel from subdir of working tree' '\n \tpwd >expect &&\n \tgit -C sub/dir rev-parse --show-toplevel >actual &&\n-- \n2.43.GIT\n\n"},{"id":"485859","messageId":"4d98b5355310448e6a95a3312bf46450247def89.1703067989.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703067989.git.ps@pks.im","subject":"[PATCH 10/12] builtin/init: introduce `--ref-format=` value flag","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-20T10:55:33Z","receivedAt":"2023-12-20T10:55:37Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Introduce a new `--ref-format` value flag for git-init(1) that allows\nthe user to specify the ref format that is to be used for a newly\ninitialized repository.\n\nSigned-off-by: Patrick Steinhardt \n---\n Documentation/git-init.txt | 7 +++++++\n builtin/init-db.c | 13 ++++++++++++-\n t/t0001-init.sh | 26 ++++++++++++++++++++++++++\n 3 files changed, 45 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-init.txt b/Documentation/git-init.txt\nindex 6f0d2973bf..e8dc645bb5 100644\n--- a/Documentation/git-init.txt\n+++ b/Documentation/git-init.txt\n@@ -11,6 +11,7 @@ SYNOPSIS\n [verse]\n 'git init' [-q | --quiet] [--bare] [--template=]\n \t [--separate-git-dir ] [--object-format=]\n+\t [--ref-format=]\n \t [-b | --initial-branch=]\n \t [--shared[=]] []\n \n@@ -57,6 +58,12 @@ values are 'sha1' and (if enabled) 'sha256'. 'sha1' is the default.\n +\n include::object-format-disclaimer.txt[]\n \n+--ref-format=::\n+\n+Specify the given ref storage format for the repository. The valid values are:\n++\n+include::ref-storage-format.txt[]\n+\n --template=::\n \n Specify the directory from which templates will be used. (See the \"TEMPLATE\ndiff --git a/builtin/init-db.c b/builtin/init-db.c\nindex b6e80feab6..770b2822fe 100644\n--- a/builtin/init-db.c\n+++ b/builtin/init-db.c\n@@ -58,6 +58,7 @@ static int shared_callback(const struct option *opt, const char *arg, int unset)\n static const char *const init_db_usage[] = {\n \tN_(\"git init [-q | --quiet] [--bare] [--template=]\\n\"\n \t \" [--separate-git-dir ] [--object-format=]\\n\"\n+\t \" [--ref-format=]\\n\"\n \t \" [-b | --initial-branch=]\\n\"\n \t \" [--shared[=]] []\"),\n \tNULL\n@@ -77,8 +78,10 @@ int cmd_init_db(int argc, const char **argv, const char *prefix)\n \tconst char *template_dir = NULL;\n \tunsigned int flags = 0;\n \tconst char *object_format = NULL;\n+\tconst char *ref_format = NULL;\n \tconst char *initial_branch = NULL;\n \tint hash_algo = GIT_HASH_UNKNOWN;\n+\tint ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN;\n \tint init_shared_repository = -1;\n \tconst struct option init_db_options[] = {\n \t\tOPT_STRING(0, \"template\", &template_dir, N_(\"template-directory\"),\n@@ -96,6 +99,8 @@ int cmd_init_db(int argc, const char **argv, const char *prefix)\n \t\t\t N_(\"override the name of the initial branch\")),\n \t\tOPT_STRING(0, \"object-format\", &object_format, N_(\"hash\"),\n \t\t\t N_(\"specify the hash algorithm to use\")),\n+\t\tOPT_STRING(0, \"ref-format\", &ref_format, N_(\"format\"),\n+\t\t\t N_(\"specify the reference format to use\")),\n \t\tOPT_END()\n \t};\n \n@@ -159,6 +164,12 @@ int cmd_init_db(int argc, const char **argv, const char *prefix)\n \t\t\tdie(_(\"unknown hash algorithm '%s'\"), object_format);\n \t}\n \n+\tif (ref_format) {\n+\t\tref_storage_format = ref_storage_format_by_name(ref_format);\n+\t\tif (ref_storage_format == REF_STORAGE_FORMAT_UNKNOWN)\n+\t\t\tdie(_(\"unknown ref storage format '%s'\"), ref_format);\n+\t}\n+\n \tif (init_shared_repository != -1)\n \t\tset_shared_repository(init_shared_repository);\n \n@@ -237,6 +248,6 @@ int cmd_init_db(int argc, const char **argv, const char *prefix)\n \n \tflags |= INIT_DB_EXIST_OK;\n \treturn init_db(git_dir, real_git_dir, template_dir, hash_algo,\n-\t\t REF_STORAGE_FORMAT_UNKNOWN, initial_branch,\n+\t\t ref_storage_format, initial_branch,\n \t\t init_shared_repository, flags);\n }\ndiff --git a/t/t0001-init.sh b/t/t0001-init.sh\nindex 30ce752cc1..b131d665db 100755\n--- a/t/t0001-init.sh\n+++ b/t/t0001-init.sh\n@@ -576,6 +576,32 @@ test_expect_success 'init with GIT_DEFAULT_REF_FORMAT=garbage' '\n \ttest_cmp expect err\n '\n \n+test_expect_success 'init with --ref-format=files' '\n+\ttest_when_finished \"rm -rf refformat\" &&\n+\tgit init --ref-format=files refformat &&\n+\techo files >expect &&\n+\tgit -C refformat rev-parse --show-ref-format >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 're-init with same format' '\n+\ttest_when_finished \"rm -rf refformat\" &&\n+\tgit init --ref-format=files refformat &&\n+\tgit init --ref-format=files refformat &&\n+\techo files >expect &&\n+\tgit -C refformat rev-parse --show-ref-format >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'init with --ref-format=garbage' '\n+\ttest_when_finished \"rm -rf refformat\" &&\n+\tcat >expect <<-EOF &&\n+\tfatal: unknown ref storage format ${SQ}garbage${SQ}\n+\tEOF\n+\ttest_must_fail git init --ref-format=garbage refformat 2>err &&\n+\ttest_cmp expect err\n+'\n+\n test_expect_success MINGW 'core.hidedotfiles = false' '\n \tgit config --global core.hidedotfiles false &&\n \trm -rf newdir &&\n-- \n2.43.GIT\n\n"},{"id":"485860","messageId":"71cf0ce82711189d8370ac47880f9e6cba0e06fd.1703067989.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703067989.git.ps@pks.im","subject":"[PATCH 11/12] builtin/clone: introduce `--ref-format=` value flag","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-20T10:55:38Z","receivedAt":"2023-12-20T10:55:41Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Introduce a new `--ref-format` value flag for git-clone(1) that allows\nthe user to specify the ref format that is to be used for a newly\ninitialized repository.\n\nSigned-off-by: Patrick Steinhardt \n---\n Documentation/git-clone.txt | 6 ++++++\n builtin/clone.c | 12 +++++++++++-\n t/t5601-clone.sh | 17 +++++++++++++++++\n 3 files changed, 34 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt\nindex c37c4a37f7..6e43eb9c20 100644\n--- a/Documentation/git-clone.txt\n+++ b/Documentation/git-clone.txt\n@@ -311,6 +311,12 @@ or `--mirror` is given)\n \tThe result is Git repository can be separated from working\n \ttree.\n \n+--ref-format=::\n --jobs ::\n \tThe number of submodules fetched at the same time.\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex 20c161e94a..b635bbcb43 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -72,6 +72,7 @@ static char *remote_name = NULL;\n static char *option_branch = NULL;\n static struct string_list option_not = STRING_LIST_INIT_NODUP;\n static const char *real_git_dir;\n+static const char *ref_format;\n static char *option_upload_pack = \"git-upload-pack\";\n static int option_verbosity;\n static int option_progress = -1;\n@@ -157,6 +158,8 @@ static struct option builtin_clone_options[] = {\n \t\t N_(\"any cloned submodules will be shallow\")),\n \tOPT_STRING(0, \"separate-git-dir\", &real_git_dir, N_(\"gitdir\"),\n \t\t N_(\"separate git dir from working tree\")),\n+\tOPT_STRING(0, \"ref-format\", &ref_format, N_(\"format\"),\n+\t\t N_(\"specify the reference format to use\")),\n \tOPT_STRING_LIST('c', \"config\", &option_config, N_(\"key=value\"),\n \t\t\tN_(\"set config inside the new repository\")),\n \tOPT_STRING_LIST(0, \"server-option\", &server_options,\n@@ -930,6 +933,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \tint submodule_progress;\n \tint filter_submodules = 0;\n \tint hash_algo;\n+\tint ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN;\n \tconst int do_not_override_repo_unix_permissions = -1;\n \n \tstruct transport_ls_refs_options transport_ls_refs_options =\n@@ -955,6 +959,12 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \tif (option_single_branch == -1)\n \t\toption_single_branch = deepen ? 1 : 0;\n \n+\tif (ref_format) {\n+\t\tref_storage_format = ref_storage_format_by_name(ref_format);\n+\t\tif (ref_storage_format == REF_STORAGE_FORMAT_UNKNOWN)\n+\t\t\tdie(_(\"unknown ref storage format '%s'\"), ref_format);\n+\t}\n+\n \tif (option_mirror)\n \t\toption_bare = 1;\n \n@@ -1106,7 +1116,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \t * their on-disk data structures.\n \t */\n \tinit_db(git_dir, real_git_dir, option_template, GIT_HASH_UNKNOWN,\n-\t\tREF_STORAGE_FORMAT_UNKNOWN, NULL,\n+\t\tref_storage_format, NULL,\n \t\tdo_not_override_repo_unix_permissions, INIT_DB_QUIET | INIT_DB_SKIP_REFDB);\n \n \tif (real_git_dir) {\ndiff --git a/t/t5601-clone.sh b/t/t5601-clone.sh\nindex 47eae641f0..fb1b9c686d 100755\n--- a/t/t5601-clone.sh\n+++ b/t/t5601-clone.sh\n@@ -157,6 +157,23 @@ test_expect_success 'clone --mirror does not repeat tags' '\n \n '\n \n+test_expect_success 'clone with files ref format' '\n+\ttest_when_finished \"rm -rf ref-storage\" &&\n+\tgit clone --ref-format=files --mirror src ref-storage &&\n+\techo files >expect &&\n+\tgit -C ref-storage rev-parse --show-ref-format >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'clone with garbage ref format' '\n+\tcat >expect <<-EOF &&\n+\tfatal: unknown ref storage format ${SQ}garbage${SQ}\n+\tEOF\n+\ttest_must_fail git clone --ref-format=garbage --mirror src ref-storage 2>err &&\n+\ttest_cmp expect err &&\n+\ttest_path_is_missing ref-storage\n+'\n+\n test_expect_success 'clone to destination with trailing /' '\n \n \tgit clone src target-1/ &&\n-- \n2.43.GIT\n\n"},{"id":"485861","messageId":"bbe2fbb15495ad6c8eb0824b4b4aaa7f3e6e2537.1703067989.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703067989.git.ps@pks.im","subject":"[PATCH 12/12] t9500: write \"extensions.refstorage\" into config","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-20T10:55:42Z","receivedAt":"2023-12-20T10:55:45Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"In t9500 we're writing a custom configuration that sets up gitweb. This\nrequires us manually ensure that the repository format is configured as\nrequired, including both the repository format version and extensions.\nWith the introduction of the \"extensions.refStorage\" extension we need\nto update the test to also write this new one.\n\nSigned-off-by: Patrick Steinhardt \n---\n t/t9500-gitweb-standalone-no-errors.sh | 5 +++++\n 1 file changed, 5 insertions(+)\n\ndiff --git a/t/t9500-gitweb-standalone-no-errors.sh b/t/t9500-gitweb-standalone-no-errors.sh\nindex 0333065d4d..7679780fb8 100755\n--- a/t/t9500-gitweb-standalone-no-errors.sh\n+++ b/t/t9500-gitweb-standalone-no-errors.sh\n@@ -627,6 +627,7 @@ test_expect_success \\\n test_expect_success 'setup' '\n \tversion=$(git config core.repositoryformatversion) &&\n \talgo=$(test_might_fail git config extensions.objectformat) &&\n+\trefstorage=$(test_might_fail git config extensions.refstorage) &&\n \tcat >.git/config <<-\\EOF &&\n \t# testing noval and alternate separator\n \t[gitweb]\n@@ -637,6 +638,10 @@ test_expect_success 'setup' '\n \tif test -n \"$algo\"\n \tthen\n \t\tgit config extensions.objectformat \"$algo\"\n+\tfi &&\n+\tif test -n \"$refstorage\"\n+\tthen\n+\t\tgit config extensions.refstorage \"$refstorage\"\n \tfi\n '\n \n-- \n2.43.GIT\n\n"},{"id":"485881","messageId":"xmqqv88ssp4r.fsf@gitster.g","threadId":"60639","inReplyTo":"6564659d403de098799ddb8101b74c2803a655d4.1703067989.git.ps@pks.im","subject":"Re: [PATCH 04/12] setup: start tracking ref storage format when","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-12-20T18:30:28Z","receivedAt":"2023-12-20T18:30:37Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"There is a topic in-flight that introduces the .compat_hash_algo\nmember to the repo_fmt structure. Seeing a conflict resolution like\nthe attached (there are many others that are similar in spirit), I\nhave to wonder if we want to add repo_set_ref_storage_format()\nhelper function. There are many assignments to .ref_storage_format\nmember after this series is applied.\n\nNote that I haven't read the series in full, so take the above with\na grain of salt---it might turn out to be that direct assignment is\nmore desirable, I dunno.\n\nThanks.\n\ndiff --cc setup.c\nindex 2f4571c12a,5038e78cdd..0000000000\n--- i/setup.c\n+++ w/setup.c\n@@@ ...\n@@@ -1584,8 -1577,8 +1595,10 @@@ const char *setup_git_directory_gently(\n \t\t}\n \t\tif (startup_info->have_repository) {\n \t\t\trepo_set_hash_algo(the_repository, repo_fmt.hash_algo);\n +\t\t\trepo_set_compat_hash_algo(the_repository,\n +\t\t\t\t\t\t repo_fmt.compat_hash_algo);\n+ \t\t\tthe_repository->ref_storage_format =\n+ \t\t\t\trepo_fmt.ref_storage_format;\n \t\t\tthe_repository->repository_format_worktree_config =\n \t\t\t\trepo_fmt.worktree_config;\n \t\t\t/* take ownership of repo_fmt.partial_clone */\n\n"},{"id":"485972","messageId":"CAOLa=ZSY-FBUMOx9SHkJqf7XteQC=hVoYxo73q=E1wL3QOBYtA@mail.gmail.com","threadId":"60639","inReplyTo":"239ca38efdbe3195e6319be4423b17d2b42e0b9f.1703067989.git.ps@pks.im","subject":"Re: [PATCH 01/12] t: introduce DEFAULT_REPO_FORMAT prereq","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2023-12-22T11:41:10Z","receivedAt":"2023-12-22T11:41:12Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt writes:\n\n> A limited number of tests require repositories to have the default\n> repository format or otherwise they would fail to run, e.g. because they\n> fail to detect the correct hash function. While the hash function is the\n> only extension right now that creates problems like this, we are about\n> to add a second extensions for the ref format.\n>\n\nNit: s/extensions/extension\n\n> -test_expect_success SHA1 'git branch -m q q2 without config should succeed' '\n> +test_expect_success DEFAULT_REPO_FORMAT 'git branch -m q q2 without config should succeed' '\n> \tgit branch -m q q2 &&\n> \tgit branch -m q2 q\n> '\n\nMy only concern is whether we'll end up blindly adding\nDEFAULT_REPO_FORMAT for tests where only SHA1 is a prereq or only where\nthe new format extension is a prereq.\n"},{"id":"485973","messageId":"CAOLa=ZTfrgNq5bDqtetBURWOOy0GeyTKGnmRR30yaDs0_-ehzQ@mail.gmail.com","threadId":"60639","inReplyTo":"e895091025e6786e0a831e4fb9b092eb74bd4379.1703067989.git.ps@pks.im","subject":"Re: [PATCH 02/12] worktree: skip reading HEAD when repairing worktrees","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2023-12-22T12:23:08Z","receivedAt":"2023-12-22T12:23:10Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt writes:\n\n> When calling `git init --separate-git-dir=` on a preexisting\n> repository, then we move the Git directory of that repository to the new\n> path specified by the user. If there are worktrees present in the Git\n> repository, we need to repair the worktrees so that their gitlinks point\n> to the new location of the repository.\n>\n\nNit: s/then we/we\n"},{"id":"485974","messageId":"CAOLa=ZR1Bc+Acwjma9OM0SvyLg4-HB-S3Pxr66VCKTb0d_tdnw@mail.gmail.com","threadId":"60639","inReplyTo":"f712d5ef5bc5bed423cf50a0d0489095ba103df7.1703067989.git.ps@pks.im","subject":"Re: [PATCH 03/12] refs: refactor logic to look up storage backends","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2023-12-22T12:38:30Z","receivedAt":"2023-12-22T12:38:32Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt writes:\n\n> +\n> +const char *ref_storage_format_to_name(int ref_storage_format)\n> +{\n> +\tconst struct ref_storage_be *be = find_ref_storage_backend(ref_storage_format);\n> +\tif (!be)\n> +\t\treturn \"unknown\";\n> +\treturn be->name;\n> +}\n>\n\nWould it make more sense to return NULL here?\n\n> +\n> /*\n> * How to handle various characters in refnames:\n> * 0: An acceptable character for refs\n> @@ -2029,12 +2045,12 @@ static struct ref_store *ref_store_init(struct repository *repo,\n> \t\t\t\t\tconst char *gitdir,\n> \t\t\t\t\tunsigned int flags)\n> {\n> -\tconst char *be_name = \"files\";\n> -\tstruct ref_storage_be *be = find_ref_storage_backend(be_name);\n> +\tint format = REF_STORAGE_FORMAT_FILES;\n> +\tconst struct ref_storage_be *be = find_ref_storage_backend(format);\n> \tstruct ref_store *refs;\n>\n> \tif (!be)\n> -\t\tBUG(\"reference backend %s is unknown\", be_name);\n> +\t\tBUG(\"reference backend %s is unknown\", ref_storage_format_to_name(format));\n>\n\nThis doesn't really get us more information, since be == NULL, calling\n`ref_storage_format_to_name` will again call `find_ref_storage_backend`,\nwhich will be NULL. I wonder if its better to:\n\n@@ -2029,12 +2045,12 @@ static struct ref_store *ref_store_init(struct\nrepository *repo,\n const char *gitdir,\n unsigned int flags)\n {\n -\tconst char *be_name = \"files\";\n -\tstruct ref_storage_be *be = find_ref_storage_backend(be_name);\n +\tint format = REF_STORAGE_FORMAT_FILES;\n +\tconst struct ref_storage_be *be = find_ref_storage_backend(format);\n struct ref_store *refs;\n\n if (!be)\n -\t\tBUG(\"reference backend %s is unknown\", be_name);\n +\t\tBUG(\"reference backend is unknown\");\n"},{"id":"485975","messageId":"CAOLa=ZQLLqj=78dJN0y51EFf3_XsHnTBYNjkB0cAAGf3dXJ9KA@mail.gmail.com","threadId":"60639","inReplyTo":"6564659d403de098799ddb8101b74c2803a655d4.1703067989.git.ps@pks.im","subject":"Re: [PATCH 04/12] setup: start tracking ref storage format when","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2023-12-22T13:09:37Z","receivedAt":"2023-12-22T13:09:38Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt writes:\n\n> In order to discern which ref storage format a repository is supposed to\n> use we need to start setting up and/or discovering the format. This\n> needs to happen in two separate code paths.\n>\n> - The first path is when we create a repository via `init_db()`. When\n> we are re-initializing a preexisting repository we need to retain\n> the previously used ref storage format -- if the user asked for a\n> different format then this indicates an erorr and we error out.\n\nNit: s/erorr/error\n\n> diff --git a/refs.c b/refs.c\n> index e87c85897d..d289a5e175 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -2045,12 +2045,12 @@ static struct ref_store *ref_store_init(struct repository *repo,\n> \t\t\t\t\tconst char *gitdir,\n> \t\t\t\t\tunsigned int flags)\n> {\n> -\tint format = REF_STORAGE_FORMAT_FILES;\n> -\tconst struct ref_storage_be *be = find_ref_storage_backend(format);\n> +\tconst struct ref_storage_be *be;\n> \tstruct ref_store *refs;\n>\n> +\tbe = find_ref_storage_backend(repo->ref_storage_format);\n> \tif (!be)\n> -\t\tBUG(\"reference backend %s is unknown\", ref_storage_format_to_name(format));\n> +\t\tBUG(\"reference backend is unknown\");\n>\n> \trefs = be->init(repo, gitdir, flags);\n> \treturn refs;\n> diff --git a/refs.h b/refs.h\n> index c6571bcf6c..944a67ac1b 100644\n> --- a/refs.h\n> +++ b/refs.h\n> @@ -11,6 +11,7 @@ struct string_list;\n> struct string_list_item;\n> struct worktree;\n>\n> +int default_ref_storage_format(void);\n>\n\nIs this used/defined somewhere?\n"},{"id":"485976","messageId":"CAOLa=ZSoW5NhUOLE_qYNTvPwdUi9eCwamtJOgTtyrvsg-LcytA@mail.gmail.com","threadId":"60639","inReplyTo":"bbe2fbb15495ad6c8eb0824b4b4aaa7f3e6e2537.1703067989.git.ps@pks.im","subject":"Re: [PATCH 12/12] t9500: write \"extensions.refstorage\" into config","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2023-12-22T13:43:01Z","receivedAt":"2023-12-22T13:43:02Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt writes:\n\n> In t9500 we're writing a custom configuration that sets up gitweb. This\n> requires us manually ensure that the repository format is configured as\n>\n\nNit: s/requires us/requires us to/\n"},{"id":"485977","messageId":"CAOLa=ZR0mM_rVBa3LG-etpgYGCpiXinCZ83hcWLVeZVUhKH3UA@mail.gmail.com","threadId":"60639","inReplyTo":"cover.1703067989.git.ps@pks.im","subject":"Re: [PATCH 00/12] Introduce `refStorage` extension","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2023-12-22T13:43:42Z","receivedAt":"2023-12-22T13:43:44Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Hello,\n\nPatrick Steinhardt writes:\n> Hi,\n>\n> this patch series introduces a new `refStorage` extension and related\n> tooling. While there is only a single user-visible ref backend for now,\n> this extension will eventually allow us to determine which backend shall\n> be used for a repository. Furthermore, the series introduces tooling so\n> that the ref storage format becomes more accessible.\n\nApart from small nits, the series looks good to me, thanks!\n"},{"id":"486040","messageId":"xmqqy1dgfzta.fsf@gitster.g","threadId":"60639","inReplyTo":"CAOLa=ZR0mM_rVBa3LG-etpgYGCpiXinCZ83hcWLVeZVUhKH3UA@mail.gmail.com","subject":"Re: [PATCH 00/12] Introduce `refStorage` extension","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-12-26T20:56:17Z","receivedAt":"2023-12-26T20:56:20Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Karthik Nayak writes:\n\n> Hello,\n>\n> Patrick Steinhardt writes:\n>> Hi,\n>>\n>> this patch series introduces a new `refStorage` extension and related\n>> tooling. While there is only a single user-visible ref backend for now,\n>> this extension will eventually allow us to determine which backend shall\n>> be used for a repository. Furthermore, the series introduces tooling so\n>> that the ref storage format becomes more accessible.\n>\n> Apart from small nits, the series looks good to me, thanks!\n\nThanks. Hopefully we can merge it down after a quick, small, and\nfinal reroll?\n"},{"id":"486090","messageId":"ZY04G_p1W_4EDeSX@tanuki","threadId":"60639","inReplyTo":"CAOLa=ZSY-FBUMOx9SHkJqf7XteQC=hVoYxo73q=E1wL3QOBYtA@mail.gmail.com","subject":"Re: [PATCH 01/12] t: introduce DEFAULT_REPO_FORMAT prereq","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-28T08:55:55Z","receivedAt":"2023-12-28T08:56:00Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Dec 22, 2023 at 03:41:10AM -0800, Karthik Nayak wrote:\n> Patrick Steinhardt writes:\n[snip]\n> > -test_expect_success SHA1 'git branch -m q q2 without config should succeed' '\n> > +test_expect_success DEFAULT_REPO_FORMAT 'git branch -m q q2 without config should succeed' '\n> > \tgit branch -m q q2 &&\n> > \tgit branch -m q2 q\n> > '\n> \n> My only concern is whether we'll end up blindly adding\n> DEFAULT_REPO_FORMAT for tests where only SHA1 is a prereq or only where\n> the new format extension is a prereq.\n\nYeah, this is of course a possibility with this new prereq. I don't\nreally know what to do about it though, and think that it does serve a\nsensible purpose. So... dunno.\n\nPatrick\n"},{"id":"486091","messageId":"ZY04KePUMJEv_QQM@tanuki","threadId":"60639","inReplyTo":"CAOLa=ZR1Bc+Acwjma9OM0SvyLg4-HB-S3Pxr66VCKTb0d_tdnw@mail.gmail.com","subject":"Re: [PATCH 03/12] refs: refactor logic to look up storage backends","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-28T08:56:09Z","receivedAt":"2023-12-28T08:56:13Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Dec 22, 2023 at 04:38:30AM -0800, Karthik Nayak wrote:\n> Patrick Steinhardt writes:\n> \n> > +\n> > +const char *ref_storage_format_to_name(int ref_storage_format)\n> > +{\n> > +\tconst struct ref_storage_be *be = find_ref_storage_backend(ref_storage_format);\n> > +\tif (!be)\n> > +\t\treturn \"unknown\";\n> > +\treturn be->name;\n> > +}\n> >\n> \n> Would it make more sense to return NULL here?\n\nThe only purpose of this function will be to print error messages, so\n\"unknown\" seems like the better choice.\n\n> > /*\n> > * How to handle various characters in refnames:\n> > * 0: An acceptable character for refs\n> > @@ -2029,12 +2045,12 @@ static struct ref_store *ref_store_init(struct repository *repo,\n> > \t\t\t\t\tconst char *gitdir,\n> > \t\t\t\t\tunsigned int flags)\n> > {\n> > -\tconst char *be_name = \"files\";\n> > -\tstruct ref_storage_be *be = find_ref_storage_backend(be_name);\n> > +\tint format = REF_STORAGE_FORMAT_FILES;\n> > +\tconst struct ref_storage_be *be = find_ref_storage_backend(format);\n> > \tstruct ref_store *refs;\n> >\n> > \tif (!be)\n> > -\t\tBUG(\"reference backend %s is unknown\", be_name);\n> > +\t\tBUG(\"reference backend %s is unknown\", ref_storage_format_to_name(format));\n> >\n> \n> This doesn't really get us more information, since be == NULL, calling\n> `ref_storage_format_to_name` will again call `find_ref_storage_backend`,\n> which will be NULL. I wonder if its better to:\n> \n> @@ -2029,12 +2045,12 @@ static struct ref_store *ref_store_init(struct\n> repository *repo,\n> const char *gitdir,\n> unsigned int flags)\n> {\n> -\tconst char *be_name = \"files\";\n> -\tstruct ref_storage_be *be = find_ref_storage_backend(be_name);\n> +\tint format = REF_STORAGE_FORMAT_FILES;\n> +\tconst struct ref_storage_be *be = find_ref_storage_backend(format);\n> struct ref_store *refs;\n> \n> if (!be)\n> -\t\tBUG(\"reference backend %s is unknown\", be_name);\n> +\t\tBUG(\"reference backend is unknown\");\n\nGood point, will change.\n\nPatrick\n"},{"id":"486092","messageId":"ZY04MuIQtuxNFwIw@tanuki","threadId":"60639","inReplyTo":"CAOLa=ZQLLqj=78dJN0y51EFf3_XsHnTBYNjkB0cAAGf3dXJ9KA@mail.gmail.com","subject":"Re: [PATCH 04/12] setup: start tracking ref storage format when","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-28T08:56:18Z","receivedAt":"2023-12-28T08:56:22Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Dec 22, 2023 at 05:09:37AM -0800, Karthik Nayak wrote:\n> Patrick Steinhardt writes:\n> > diff --git a/refs.c b/refs.c\n> > index e87c85897d..d289a5e175 100644\n> > --- a/refs.c\n> > +++ b/refs.c\n> > @@ -2045,12 +2045,12 @@ static struct ref_store *ref_store_init(struct repository *repo,\n> > \t\t\t\t\tconst char *gitdir,\n> > \t\t\t\t\tunsigned int flags)\n> > {\n> > -\tint format = REF_STORAGE_FORMAT_FILES;\n> > -\tconst struct ref_storage_be *be = find_ref_storage_backend(format);\n> > +\tconst struct ref_storage_be *be;\n> > \tstruct ref_store *refs;\n> >\n> > +\tbe = find_ref_storage_backend(repo->ref_storage_format);\n> > \tif (!be)\n> > -\t\tBUG(\"reference backend %s is unknown\", ref_storage_format_to_name(format));\n> > +\t\tBUG(\"reference backend is unknown\");\n> >\n> > \trefs = be->init(repo, gitdir, flags);\n> > \treturn refs;\n> > diff --git a/refs.h b/refs.h\n> > index c6571bcf6c..944a67ac1b 100644\n> > --- a/refs.h\n> > +++ b/refs.h\n> > @@ -11,6 +11,7 @@ struct string_list;\n> > struct string_list_item;\n> > struct worktree;\n> >\n> > +int default_ref_storage_format(void);\n> >\n> \n> Is this used/defined somewhere?\n\nNo, it's a leftover from a previous iteration. Will drop.\n\nPatrick\n"},{"id":"486093","messageId":"ZY04OlTNUEZs5T-T@tanuki","threadId":"60639","inReplyTo":"xmqqv88ssp4r.fsf@gitster.g","subject":"Re: [PATCH 04/12] setup: start tracking ref storage format when","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-28T08:56:26Z","receivedAt":"2023-12-28T08:56:29Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Dec 20, 2023 at 10:30:28AM -0800, Junio C Hamano wrote:\n> There is a topic in-flight that introduces the .compat_hash_algo\n> member to the repo_fmt structure. Seeing a conflict resolution like\n> the attached (there are many others that are similar in spirit), I\n> have to wonder if we want to add repo_set_ref_storage_format()\n> helper function. There are many assignments to .ref_storage_format\n> member after this series is applied.\n> \n> Note that I haven't read the series in full, so take the above with\n> a grain of salt---it might turn out to be that direct assignment is\n> more desirable, I dunno.\n> \n> Thanks.\n\nWrapping this in a `repo_set_ref_storage_format()` doesn't add much for\nnow, but on the other hand it also doesn't hurt and is more in line with\nhow we set the other formats. This could also be useful in the future to\nhave a central place for additional sanity checks, if required. So yeah,\nlet's do it.\n\nMakes me wonder whether we should then also add the following diff to\n\"setup: set repository's format on init\" when both topics are being\nmerged together:\n\ndiff --git a/setup.c b/setup.c\nindex 3d980814bc..3d35c78c68 100644\n--- a/setup.c\n+++ b/setup.c\n@@ -2210,6 +2210,7 @@ int init_db(const char *git_dir, const char *real_git_dir,\n \t * format we can update the repository's settings accordingly.\n \t */\n \trepo_set_hash_algo(the_repository, repo_fmt.hash_algo);\n+\trepo_set_compat_hash_algo(the_repository, repo_fmt.compat_hash_algo);\n \trepo_set_ref_storage_format(the_repository, repo_fmt.ref_storage_format);\n \n \tif (!(flags & INIT_DB_SKIP_REFDB))\n\nPatrick\n"},{"id":"486094","messageId":"cover.1703753910.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703067989.git.ps@pks.im","subject":"[PATCH v2 00/12] Introduce `refStorage` extension","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-28T09:57:23Z","receivedAt":"2023-12-28T09:57:28Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Hi,\n\nthis is the second version of my patch series that introduces the new\n`refStorage` extension. This extension will be used for the upcoming\nreftable backend.\n\nChanges compared to v2:\n\n - Fixed various typos in commit messages.\n\n - Fixed redundant information when the refstorage extension's value\n isn't understood.\n\n - Introduced `repo_set_ref_storage_format()`.\n\nThanks for the feedback so far!\n\nPatrick\n\nPatrick Steinhardt (12):\n t: introduce DEFAULT_REPO_FORMAT prereq\n worktree: skip reading HEAD when repairing worktrees\n refs: refactor logic to look up storage backends\n setup: start tracking ref storage format\n setup: set repository's formats on init\n setup: introduce \"extensions.refStorage\" extension\n setup: introduce GIT_DEFAULT_REF_FORMAT envvar\n t: introduce GIT_TEST_DEFAULT_REF_FORMAT envvar\n builtin/rev-parse: introduce `--show-ref-format` flag\n builtin/init: introduce `--ref-format=` value flag\n builtin/clone: introduce `--ref-format=` value flag\n t9500: write \"extensions.refstorage\" into config\n\n Documentation/config/extensions.txt | 11 +++\n Documentation/git-clone.txt | 6 ++\n Documentation/git-init.txt | 7 ++\n Documentation/git-rev-parse.txt | 3 +\n Documentation/git.txt | 5 ++\n Documentation/ref-storage-format.txt | 1 +\n .../technical/repository-version.txt | 5 ++\n builtin/clone.c | 17 ++++-\n builtin/init-db.c | 15 +++-\n builtin/rev-parse.c | 4 ++\n refs.c | 34 ++++++---\n refs.h | 3 +\n refs/debug.c | 1 -\n refs/files-backend.c | 1 -\n refs/packed-backend.c | 1 -\n refs/refs-internal.h | 1 -\n repository.c | 6 ++\n repository.h | 7 ++\n setup.c | 63 +++++++++++++++--\n setup.h | 9 ++-\n t/README | 3 +\n t/t0001-init.sh | 70 +++++++++++++++++++\n t/t1500-rev-parse.sh | 17 +++++\n t/t3200-branch.sh | 2 +-\n t/t5601-clone.sh | 17 +++++\n t/t9500-gitweb-standalone-no-errors.sh | 5 ++\n t/test-lib-functions.sh | 5 ++\n t/test-lib.sh | 15 +++-\n worktree.c | 24 ++++---\n 29 files changed, 323 insertions(+), 35 deletions(-)\n create mode 100644 Documentation/ref-storage-format.txt\n\nRange-diff against v1:\n 1: 239ca38efd ! 1: 3613439cb7 t: introduce DEFAULT_REPO_FORMAT prereq\n @@ Commit message\n repository format or otherwise they would fail to run, e.g. because they\n fail to detect the correct hash function. While the hash function is the\n only extension right now that creates problems like this, we are about\n - to add a second extensions for the ref format.\n + to add a second extension for the ref format.\n \n Introduce a new DEFAULT_REPO_FORMAT prereq that can easily be amended\n whenever we add new format extensions. Next to making any such changes\n 2: e895091025 ! 2: ecf4f1ddee worktree: skip reading HEAD when repairing worktrees\n @@ Commit message\n worktree: skip reading HEAD when repairing worktrees\n \n When calling `git init --separate-git-dir=` on a preexisting\n - repository, then we move the Git directory of that repository to the new\n - path specified by the user. If there are worktrees present in the Git\n - repository, we need to repair the worktrees so that their gitlinks point\n - to the new location of the repository.\n + repository, we move the Git directory of that repository to the new path\n + specified by the user. If there are worktrees present in the repository,\n + we need to repair the worktrees so that their gitlinks point to the new\n + location of the repository.\n \n This repair logic will load repositories via `get_worktrees()`, which\n will enumerate up and initialize all worktrees. Part of initialization\n @@ Commit message\n format, which does not work.\n \n We do not require the worktree HEADs at all to repair worktrees. So\n - let's fix issue this by skipping over the step that reads them.\n + let's fix this issue by skipping over the step that reads them.\n \n Signed-off-by: Patrick Steinhardt \n \n 3: f712d5ef5b ! 3: 12329b99b7 refs: refactor logic to look up storage backends\n @@ refs.c\n -\tfor (be = refs_backends; be; be = be->next)\n -\t\tif (!strcmp(be->name, name))\n -\t\t\treturn be;\n -+\tif (ref_storage_format && ref_storage_format < ARRAY_SIZE(refs_backends))\n ++\tif (ref_storage_format < ARRAY_SIZE(refs_backends))\n +\t\treturn refs_backends[ref_storage_format];\n \treturn NULL;\n }\n @@ refs.c: static struct ref_store *ref_store_init(struct repository *repo,\n \n \tif (!be)\n -\t\tBUG(\"reference backend %s is unknown\", be_name);\n -+\t\tBUG(\"reference backend %s is unknown\", ref_storage_format_to_name(format));\n ++\t\tBUG(\"reference backend is unknown\");\n \n \trefs = be->init(repo, gitdir, flags);\n \treturn refs;\n 4: 6564659d40 ! 4: ddd099fbaf setup: start tracking ref storage format when\n @@ Metadata\n Author: Patrick Steinhardt \n \n ## Commit message ##\n - setup: start tracking ref storage format when\n + setup: start tracking ref storage format\n \n In order to discern which ref storage format a repository is supposed to\n use we need to start setting up and/or discovering the format. This\n @@ Commit message\n - The first path is when we create a repository via `init_db()`. When\n we are re-initializing a preexisting repository we need to retain\n the previously used ref storage format -- if the user asked for a\n - different format then this indicates an erorr and we error out.\n + different format then this indicates an error and we error out.\n Otherwise we either initialize the repository with the format asked\n for by the user or the default format, which currently is the\n \"files\" backend.\n @@ refs.c: static struct ref_store *ref_store_init(struct repository *repo,\n \n +\tbe = find_ref_storage_backend(repo->ref_storage_format);\n \tif (!be)\n --\t\tBUG(\"reference backend %s is unknown\", ref_storage_format_to_name(format));\n -+\t\tBUG(\"reference backend is unknown\");\n - \n - \trefs = be->init(repo, gitdir, flags);\n - \treturn refs;\n -\n - ## refs.h ##\n -@@ refs.h: struct string_list;\n - struct string_list_item;\n - struct worktree;\n - \n -+int default_ref_storage_format(void);\n - int ref_storage_format_by_name(const char *name);\n - const char *ref_storage_format_to_name(int ref_storage_format);\n + \t\tBUG(\"reference backend is unknown\");\n \n \n ## repository.c ##\n +@@ repository.c: void repo_set_hash_algo(struct repository *repo, int hash_algo)\n + \trepo->hash_algo = &hash_algos[hash_algo];\n + }\n + \n ++void repo_set_ref_storage_format(struct repository *repo, int format)\n ++{\n ++\trepo->ref_storage_format = format;\n ++}\n ++\n + /*\n + * Attempt to resolve and set the provided 'gitdir' for repository 'repo'.\n + * Return 0 upon success and a non-zero value upon failure.\n @@ repository.c: int repo_init(struct repository *repo,\n \t\tgoto error;\n \n \trepo_set_hash_algo(repo, format.hash_algo);\n -+\trepo->ref_storage_format = format.ref_storage_format;\n ++\trepo_set_ref_storage_format(repo, format.ref_storage_format);\n \trepo->repository_format_worktree_config = format.worktree_config;\n \n \t/* take ownership of format.partial_clone */\n @@ repository.h: struct repository {\n \t/* A unique-id for tracing purposes. */\n \tint trace2_repo_id;\n \n +@@ repository.h: void repo_set_gitdir(struct repository *repo, const char *root,\n + \t\t const struct set_gitdir_args *extra_args);\n + void repo_set_worktree(struct repository *repo, const char *path);\n + void repo_set_hash_algo(struct repository *repo, int algo);\n ++void repo_set_ref_storage_format(struct repository *repo, int format);\n + void initialize_the_repository(void);\n + RESULT_MUST_BE_USED\n + int repo_init(struct repository *r, const char *gitdir, const char *worktree);\n \n ## setup.c ##\n @@ setup.c: const char *setup_git_directory_gently(int *nongit_ok)\n \t\t}\n \t\tif (startup_info->have_repository) {\n \t\t\trepo_set_hash_algo(the_repository, repo_fmt.hash_algo);\n -+\t\t\tthe_repository->ref_storage_format =\n -+\t\t\t\trepo_fmt.ref_storage_format;\n ++\t\t\trepo_set_ref_storage_format(the_repository,\n ++\t\t\t\t\t\t repo_fmt.ref_storage_format);\n \t\t\tthe_repository->repository_format_worktree_config =\n \t\t\t\trepo_fmt.worktree_config;\n \t\t\t/* take ownership of repo_fmt.partial_clone */\n @@ setup.c: void check_repository_format(struct repository_format *fmt)\n \tcheck_repository_format_gently(get_git_dir(), fmt, NULL);\n \tstartup_info->have_repository = 1;\n \trepo_set_hash_algo(the_repository, fmt->hash_algo);\n -+\tthe_repository->ref_storage_format =\n -+\t\tfmt->ref_storage_format;\n ++\trepo_set_ref_storage_format(the_repository,\n ++\t\t\t\t fmt->ref_storage_format);\n \tthe_repository->repository_format_worktree_config =\n \t\tfmt->worktree_config;\n \tthe_repository->repository_format_partial_clone =\n @@ setup.c: void create_reference_database(const char *initial_branch, int quiet)\n \tsafe_create_dir(git_path(\"refs\"), 1);\n \tadjust_shared_perm(git_path(\"refs\"));\n \n -+\tthe_repository->ref_storage_format = ref_storage_format;\n ++\trepo_set_ref_storage_format(the_repository, ref_storage_format);\n \tif (refs_init_db(&err))\n \t\tdie(\"failed to set up refs db: %s\", err.buf);\n \n 5: f90a63d63c ! 5: 01a1e58a97 setup: set repository's formats on init\n @@ setup.c: int init_db(const char *git_dir, const char *real_git_dir,\n +\t * Now that we have set up both the hash algorithm and the ref storage\n +\t * format we can update the repository's settings accordingly.\n +\t */\n -+\tthe_repository->hash_algo = &hash_algos[repo_fmt.hash_algo];\n -+\tthe_repository->ref_storage_format = repo_fmt.ref_storage_format;\n ++\trepo_set_hash_algo(the_repository, repo_fmt.hash_algo);\n ++\trepo_set_ref_storage_format(the_repository, repo_fmt.ref_storage_format);\n +\n \tif (!(flags & INIT_DB_SKIP_REFDB))\n \t\tcreate_reference_database(repo_fmt.ref_storage_format,\n 6: beeb182f28 = 6: 0a586fa648 setup: introduce \"extensions.refStorage\" extension\n 7: dd91a75da4 = 7: 6d8754f73a setup: introduce GIT_DEFAULT_REF_FORMAT envvar\n 8: ed3bf008cd = 8: c645932f3d t: introduce GIT_TEST_DEFAULT_REF_FORMAT envvar\n 9: 8a3d950d69 = 9: 761d647770 builtin/rev-parse: introduce `--show-ref-format` flag\n10: 4d98b53553 = 10: e382b5bf08 builtin/init: introduce `--ref-format=` value flag\n11: 71cf0ce827 = 11: 257233658d builtin/clone: introduce `--ref-format=` value flag\n12: bbe2fbb154 ! 12: b8cd06ec53 t9500: write \"extensions.refstorage\" into config\n @@ Commit message\n t9500: write \"extensions.refstorage\" into config\n \n In t9500 we're writing a custom configuration that sets up gitweb. This\n - requires us manually ensure that the repository format is configured as\n - required, including both the repository format version and extensions.\n - With the introduction of the \"extensions.refStorage\" extension we need\n - to update the test to also write this new one.\n + requires us to manually ensure that the repository format is configured\n + as required, including both the repository format version and\n + extensions. With the introduction of the \"extensions.refStorage\"\n + extension we need to update the test to also write this new one.\n \n Signed-off-by: Patrick Steinhardt \n \n\nbase-commit: e79552d19784ee7f4bbce278fe25f93fbda196fa\n-- \n2.43.GIT\n\n"},{"id":"486095","messageId":"3613439cb7c40054111998da9fb8dfb2d5c46a72.1703753910.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703753910.git.ps@pks.im","subject":"[PATCH v2 01/12] t: introduce DEFAULT_REPO_FORMAT prereq","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-28T09:57:27Z","receivedAt":"2023-12-28T09:57:38Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"A limited number of tests require repositories to have the default\nrepository format or otherwise they would fail to run, e.g. because they\nfail to detect the correct hash function. While the hash function is the\nonly extension right now that creates problems like this, we are about\nto add a second extension for the ref format.\n\nIntroduce a new DEFAULT_REPO_FORMAT prereq that can easily be amended\nwhenever we add new format extensions. Next to making any such changes\neasier on us, the prerequisite's name should also help to clarify the\nintent better.\n\nSigned-off-by: Patrick Steinhardt \n---\n t/t3200-branch.sh | 2 +-\n t/test-lib.sh | 4 ++++\n 2 files changed, 5 insertions(+), 1 deletion(-)\n\ndiff --git a/t/t3200-branch.sh b/t/t3200-branch.sh\nindex 6a316f081e..de7d3014e4 100755\n--- a/t/t3200-branch.sh\n+++ b/t/t3200-branch.sh\n@@ -519,7 +519,7 @@ EOF\n \n mv .git/config .git/config-saved\n \n-test_expect_success SHA1 'git branch -m q q2 without config should succeed' '\n+test_expect_success DEFAULT_REPO_FORMAT 'git branch -m q q2 without config should succeed' '\n \tgit branch -m q q2 &&\n \tgit branch -m q2 q\n '\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 876b99562a..dc03f06b8e 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1936,6 +1936,10 @@ test_lazy_prereq SHA1 '\n \tesac\n '\n \n+test_lazy_prereq DEFAULT_REPO_FORMAT '\n+\ttest_have_prereq SHA1\n+'\n+\n # Ensure that no test accidentally triggers a Git command\n # that runs the actual maintenance scheduler, affecting a user's\n # system permanently.\n-- \n2.43.GIT\n\n"},{"id":"486096","messageId":"ecf4f1ddee36643f0ff7e3d40b9aa7c7e6e6ce43.1703753910.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703753910.git.ps@pks.im","subject":"[PATCH v2 02/12] worktree: skip reading HEAD when repairing worktrees","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-28T09:57:31Z","receivedAt":"2023-12-28T09:57:39Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"When calling `git init --separate-git-dir=` on a preexisting\nrepository, we move the Git directory of that repository to the new path\nspecified by the user. If there are worktrees present in the repository,\nwe need to repair the worktrees so that their gitlinks point to the new\nlocation of the repository.\n\nThis repair logic will load repositories via `get_worktrees()`, which\nwill enumerate up and initialize all worktrees. Part of initialization\nis logic that we resolve their respective worktree HEADs, even though\nthat information may not actually be needed in the end by all callers.\n\nIn the context of git-init(1) this is about to become a problem, because\nwe do not have a repository that was set up via `setup_git_directory()`\nor friends. Consequentially, it is not yet fully initialized at the time\nof calling `repair_worktrees()`, and properly setting up all parts of\nthe repository in `init_db()` before we repair worktrees is not an easy\nthing to do. While this is okay right now where we only have a single\nreference backend in Git, once we gain a second one we would be trying\nto look up the worktree HEADs before we have figured out the reference\nformat, which does not work.\n\nWe do not require the worktree HEADs at all to repair worktrees. So\nlet's fix this issue by skipping over the step that reads them.\n\nSigned-off-by: Patrick Steinhardt \n---\n worktree.c | 24 ++++++++++++++++--------\n 1 file changed, 16 insertions(+), 8 deletions(-)\n\ndiff --git a/worktree.c b/worktree.c\nindex a56a6c2a3d..9702ed0308 100644\n--- a/worktree.c\n+++ b/worktree.c\n@@ -51,7 +51,7 @@ static void add_head_info(struct worktree *wt)\n /**\n * get the main worktree\n */\n-static struct worktree *get_main_worktree(void)\n+static struct worktree *get_main_worktree(int skip_reading_head)\n {\n \tstruct worktree *worktree = NULL;\n \tstruct strbuf worktree_path = STRBUF_INIT;\n@@ -70,11 +70,13 @@ static struct worktree *get_main_worktree(void)\n \t */\n \tworktree->is_bare = (is_bare_repository_cfg == 1) ||\n \t\tis_bare_repository();\n-\tadd_head_info(worktree);\n+\tif (!skip_reading_head)\n+\t\tadd_head_info(worktree);\n \treturn worktree;\n }\n \n-static struct worktree *get_linked_worktree(const char *id)\n+static struct worktree *get_linked_worktree(const char *id,\n+\t\t\t\t\t int skip_reading_head)\n {\n \tstruct worktree *worktree = NULL;\n \tstruct strbuf path = STRBUF_INIT;\n@@ -93,7 +95,8 @@ static struct worktree *get_linked_worktree(const char *id)\n \tCALLOC_ARRAY(worktree, 1);\n \tworktree->path = strbuf_detach(&worktree_path, NULL);\n \tworktree->id = xstrdup(id);\n-\tadd_head_info(worktree);\n+\tif (!skip_reading_head)\n+\t\tadd_head_info(worktree);\n \n done:\n \tstrbuf_release(&path);\n@@ -118,7 +121,7 @@ static void mark_current_worktree(struct worktree **worktrees)\n \tfree(git_dir);\n }\n \n-struct worktree **get_worktrees(void)\n+static struct worktree **get_worktrees_internal(int skip_reading_head)\n {\n \tstruct worktree **list = NULL;\n \tstruct strbuf path = STRBUF_INIT;\n@@ -128,7 +131,7 @@ struct worktree **get_worktrees(void)\n \n \tALLOC_ARRAY(list, alloc);\n \n-\tlist[counter++] = get_main_worktree();\n+\tlist[counter++] = get_main_worktree(skip_reading_head);\n \n \tstrbuf_addf(&path, \"%s/worktrees\", get_git_common_dir());\n \tdir = opendir(path.buf);\n@@ -137,7 +140,7 @@ struct worktree **get_worktrees(void)\n \t\twhile ((d = readdir_skip_dot_and_dotdot(dir)) != NULL) {\n \t\t\tstruct worktree *linked = NULL;\n \n-\t\t\tif ((linked = get_linked_worktree(d->d_name))) {\n+\t\t\tif ((linked = get_linked_worktree(d->d_name, skip_reading_head))) {\n \t\t\t\tALLOC_GROW(list, counter + 1, alloc);\n \t\t\t\tlist[counter++] = linked;\n \t\t\t}\n@@ -151,6 +154,11 @@ struct worktree **get_worktrees(void)\n \treturn list;\n }\n \n+struct worktree **get_worktrees(void)\n+{\n+\treturn get_worktrees_internal(0);\n+}\n+\n const char *get_worktree_git_dir(const struct worktree *wt)\n {\n \tif (!wt)\n@@ -591,7 +599,7 @@ static void repair_noop(int iserr UNUSED,\n \n void repair_worktrees(worktree_repair_fn fn, void *cb_data)\n {\n-\tstruct worktree **worktrees = get_worktrees();\n+\tstruct worktree **worktrees = get_worktrees_internal(1);\n \tstruct worktree **wt = worktrees + 1; /* +1 skips main worktree */\n \n \tif (!fn)\n-- \n2.43.GIT\n\n"},{"id":"486097","messageId":"12329b99b753f79fe93fe017e71b08227d213c1e.1703753910.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703753910.git.ps@pks.im","subject":"[PATCH v2 03/12] refs: refactor logic to look up storage backends","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-28T09:57:35Z","receivedAt":"2023-12-28T09:57:40Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"In order to look up ref storage backends, we're currently using a linked\nlist of backends, where each backend is expected to set up its `next`\npointer to the next ref storage backend. This is kind of a weird setup\nas backends need to be aware of other backends without much of a reason.\n\nRefactor the code so that the array of backends is centrally defined in\n\"refs.c\", where each backend is now identified by an integer constant.\nExpose functions to translate from those integer constants to the name\nand vice versa, which will be required by subsequent patches.\n\nSigned-off-by: Patrick Steinhardt \n---\n refs.c | 34 +++++++++++++++++++++++++---------\n refs.h | 3 +++\n refs/debug.c | 1 -\n refs/files-backend.c | 1 -\n refs/packed-backend.c | 1 -\n refs/refs-internal.h | 1 -\n repository.h | 3 +++\n 7 files changed, 31 insertions(+), 13 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 16bfa21df7..ab14634731 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -33,17 +33,33 @@\n /*\n * List of all available backends\n */\n-static struct ref_storage_be *refs_backends = &refs_be_files;\n+static const struct ref_storage_be *refs_backends[] = {\n+\t[REF_STORAGE_FORMAT_FILES] = &refs_be_files,\n+};\n \n-static struct ref_storage_be *find_ref_storage_backend(const char *name)\n+static const struct ref_storage_be *find_ref_storage_backend(int ref_storage_format)\n {\n-\tstruct ref_storage_be *be;\n-\tfor (be = refs_backends; be; be = be->next)\n-\t\tif (!strcmp(be->name, name))\n-\t\t\treturn be;\n+\tif (ref_storage_format < ARRAY_SIZE(refs_backends))\n+\t\treturn refs_backends[ref_storage_format];\n \treturn NULL;\n }\n \n+int ref_storage_format_by_name(const char *name)\n+{\n+\tfor (int i = 0; i < ARRAY_SIZE(refs_backends); i++)\n+\t\tif (refs_backends[i] && !strcmp(refs_backends[i]->name, name))\n+\t\t\treturn i;\n+\treturn REF_STORAGE_FORMAT_UNKNOWN;\n+}\n+\n+const char *ref_storage_format_to_name(int ref_storage_format)\n+{\n+\tconst struct ref_storage_be *be = find_ref_storage_backend(ref_storage_format);\n+\tif (!be)\n+\t\treturn \"unknown\";\n+\treturn be->name;\n+}\n+\n /*\n * How to handle various characters in refnames:\n * 0: An acceptable character for refs\n@@ -2029,12 +2045,12 @@ static struct ref_store *ref_store_init(struct repository *repo,\n \t\t\t\t\tconst char *gitdir,\n \t\t\t\t\tunsigned int flags)\n {\n-\tconst char *be_name = \"files\";\n-\tstruct ref_storage_be *be = find_ref_storage_backend(be_name);\n+\tint format = REF_STORAGE_FORMAT_FILES;\n+\tconst struct ref_storage_be *be = find_ref_storage_backend(format);\n \tstruct ref_store *refs;\n \n \tif (!be)\n-\t\tBUG(\"reference backend %s is unknown\", be_name);\n+\t\tBUG(\"reference backend is unknown\");\n \n \trefs = be->init(repo, gitdir, flags);\n \treturn refs;\ndiff --git a/refs.h b/refs.h\nindex ff113bb12a..e2098ea51b 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -11,6 +11,9 @@ struct string_list;\n struct string_list_item;\n struct worktree;\n \n+int ref_storage_format_by_name(const char *name);\n+const char *ref_storage_format_to_name(int ref_storage_format);\n+\n /*\n * Resolve a reference, recursively following symbolic refererences.\n *\ndiff --git a/refs/debug.c b/refs/debug.c\nindex 83b7a0ba65..b9775f2c37 100644\n--- a/refs/debug.c\n+++ b/refs/debug.c\n@@ -426,7 +426,6 @@ static int debug_reflog_expire(struct ref_store *ref_store, const char *refname,\n }\n \n struct ref_storage_be refs_be_debug = {\n-\t.next = NULL,\n \t.name = \"debug\",\n \t.init = NULL,\n \t.init_db = debug_init_db,\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex ad8b1d143f..43fd0ac760 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3241,7 +3241,6 @@ static int files_init_db(struct ref_store *ref_store, struct strbuf *err UNUSED)\n }\n \n struct ref_storage_be refs_be_files = {\n-\t.next = NULL,\n \t.name = \"files\",\n \t.init = files_ref_store_create,\n \t.init_db = files_init_db,\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex b9fa097a29..8d1090e284 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1705,7 +1705,6 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n }\n \n struct ref_storage_be refs_be_packed = {\n-\t.next = NULL,\n \t.name = \"packed\",\n \t.init = packed_ref_store_create,\n \t.init_db = packed_init_db,\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 4af83bf9a5..8e9f04cc67 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -663,7 +663,6 @@ typedef int read_symbolic_ref_fn(struct ref_store *ref_store, const char *refnam\n \t\t\t\t struct strbuf *referent);\n \n struct ref_storage_be {\n-\tstruct ref_storage_be *next;\n \tconst char *name;\n \tref_store_init_fn *init;\n \tref_init_db_fn *init_db;\ndiff --git a/repository.h b/repository.h\nindex 5f18486f64..ea4c488b81 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -24,6 +24,9 @@ enum fetch_negotiation_setting {\n \tFETCH_NEGOTIATION_NOOP,\n };\n \n+#define REF_STORAGE_FORMAT_UNKNOWN 0\n+#define REF_STORAGE_FORMAT_FILES 1\n+\n struct repo_settings {\n \tint initialized;\n \n-- \n2.43.GIT\n\n"},{"id":"486098","messageId":"ddd099fbaf3d8f4b6b6276048d0fcbc814f9e59c.1703753910.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703753910.git.ps@pks.im","subject":"[PATCH v2 04/12] setup: start tracking ref storage format","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-28T09:57:40Z","receivedAt":"2023-12-28T09:57:44Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"In order to discern which ref storage format a repository is supposed to\nuse we need to start setting up and/or discovering the format. This\nneeds to happen in two separate code paths.\n\n - The first path is when we create a repository via `init_db()`. When\n we are re-initializing a preexisting repository we need to retain\n the previously used ref storage format -- if the user asked for a\n different format then this indicates an error and we error out.\n Otherwise we either initialize the repository with the format asked\n for by the user or the default format, which currently is the\n \"files\" backend.\n\n - The second path is when discovering repositories, where we need to\n read the config of that repository. There is not yet any way to\n configure something other than the \"files\" backend, so we can just\n blindly set the ref storage format to this backend.\n\nWire up this logic so that we have the ref storage format always readily\navailable when needed. As there is only a single backend and because it\nis not configurable we cannot yet verify that this tracking works as\nexpected via tests, but tests will be added in subsequent commits. To\ncountermand this ommission now though, raise a BUG() in case the ref\nstorage format is not set up properly in `ref_store_init()`.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/clone.c | 5 +++--\n builtin/init-db.c | 4 +++-\n refs.c | 4 ++--\n repository.c | 6 ++++++\n repository.h | 4 ++++\n setup.c | 26 +++++++++++++++++++++++---\n setup.h | 6 +++++-\n 7 files changed, 46 insertions(+), 9 deletions(-)\n\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex 343f536cf8..48aeb1b90b 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -1107,7 +1107,8 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \t * repository, and reference backends may persist that information into\n \t * their on-disk data structures.\n \t */\n-\tinit_db(git_dir, real_git_dir, option_template, GIT_HASH_UNKNOWN, NULL,\n+\tinit_db(git_dir, real_git_dir, option_template, GIT_HASH_UNKNOWN,\n+\t\tREF_STORAGE_FORMAT_UNKNOWN, NULL,\n \t\tdo_not_override_repo_unix_permissions, INIT_DB_QUIET | INIT_DB_SKIP_REFDB);\n \n \tif (real_git_dir) {\n@@ -1292,7 +1293,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \thash_algo = hash_algo_by_ptr(transport_get_hash_algo(transport));\n \tinitialize_repository_version(hash_algo, 1);\n \trepo_set_hash_algo(the_repository, hash_algo);\n-\tcreate_reference_database(NULL, 1);\n+\tcreate_reference_database(the_repository->ref_storage_format, NULL, 1);\n \n \t/*\n \t * Before fetching from the remote, download and install bundle\ndiff --git a/builtin/init-db.c b/builtin/init-db.c\nindex cb727c826f..b6e80feab6 100644\n--- a/builtin/init-db.c\n+++ b/builtin/init-db.c\n@@ -11,6 +11,7 @@\n #include \"object-file.h\"\n #include \"parse-options.h\"\n #include \"path.h\"\n+#include \"refs.h\"\n #include \"setup.h\"\n #include \"strbuf.h\"\n \n@@ -236,5 +237,6 @@ int cmd_init_db(int argc, const char **argv, const char *prefix)\n \n \tflags |= INIT_DB_EXIST_OK;\n \treturn init_db(git_dir, real_git_dir, template_dir, hash_algo,\n-\t\t initial_branch, init_shared_repository, flags);\n+\t\t REF_STORAGE_FORMAT_UNKNOWN, initial_branch,\n+\t\t init_shared_repository, flags);\n }\ndiff --git a/refs.c b/refs.c\nindex ab14634731..be8fcabde0 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -2045,10 +2045,10 @@ static struct ref_store *ref_store_init(struct repository *repo,\n \t\t\t\t\tconst char *gitdir,\n \t\t\t\t\tunsigned int flags)\n {\n-\tint format = REF_STORAGE_FORMAT_FILES;\n-\tconst struct ref_storage_be *be = find_ref_storage_backend(format);\n+\tconst struct ref_storage_be *be;\n \tstruct ref_store *refs;\n \n+\tbe = find_ref_storage_backend(repo->ref_storage_format);\n \tif (!be)\n \t\tBUG(\"reference backend is unknown\");\n \ndiff --git a/repository.c b/repository.c\nindex a7679ceeaa..691cabf45b 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -104,6 +104,11 @@ void repo_set_hash_algo(struct repository *repo, int hash_algo)\n \trepo->hash_algo = &hash_algos[hash_algo];\n }\n \n+void repo_set_ref_storage_format(struct repository *repo, int format)\n+{\n+\trepo->ref_storage_format = format;\n+}\n+\n /*\n * Attempt to resolve and set the provided 'gitdir' for repository 'repo'.\n * Return 0 upon success and a non-zero value upon failure.\n@@ -184,6 +189,7 @@ int repo_init(struct repository *repo,\n \t\tgoto error;\n \n \trepo_set_hash_algo(repo, format.hash_algo);\n+\trepo_set_ref_storage_format(repo, format.ref_storage_format);\n \trepo->repository_format_worktree_config = format.worktree_config;\n \n \t/* take ownership of format.partial_clone */\ndiff --git a/repository.h b/repository.h\nindex ea4c488b81..d3a24da4d6 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -163,6 +163,9 @@ struct repository {\n \t/* Repository's current hash algorithm, as serialized on disk. */\n \tconst struct git_hash_algo *hash_algo;\n \n+\t/* Repository's reference storage format, as serialized on disk. */\n+\tint ref_storage_format;\n+\n \t/* A unique-id for tracing purposes. */\n \tint trace2_repo_id;\n \n@@ -202,6 +205,7 @@ void repo_set_gitdir(struct repository *repo, const char *root,\n \t\t const struct set_gitdir_args *extra_args);\n void repo_set_worktree(struct repository *repo, const char *path);\n void repo_set_hash_algo(struct repository *repo, int algo);\n+void repo_set_ref_storage_format(struct repository *repo, int format);\n void initialize_the_repository(void);\n RESULT_MUST_BE_USED\n int repo_init(struct repository *r, const char *gitdir, const char *worktree);\ndiff --git a/setup.c b/setup.c\nindex bc90bbd033..e58ab7e786 100644\n--- a/setup.c\n+++ b/setup.c\n@@ -1566,6 +1566,8 @@ const char *setup_git_directory_gently(int *nongit_ok)\n \t\t}\n \t\tif (startup_info->have_repository) {\n \t\t\trepo_set_hash_algo(the_repository, repo_fmt.hash_algo);\n+\t\t\trepo_set_ref_storage_format(the_repository,\n+\t\t\t\t\t\t repo_fmt.ref_storage_format);\n \t\t\tthe_repository->repository_format_worktree_config =\n \t\t\t\trepo_fmt.worktree_config;\n \t\t\t/* take ownership of repo_fmt.partial_clone */\n@@ -1659,6 +1661,8 @@ void check_repository_format(struct repository_format *fmt)\n \tcheck_repository_format_gently(get_git_dir(), fmt, NULL);\n \tstartup_info->have_repository = 1;\n \trepo_set_hash_algo(the_repository, fmt->hash_algo);\n+\trepo_set_ref_storage_format(the_repository,\n+\t\t\t\t fmt->ref_storage_format);\n \tthe_repository->repository_format_worktree_config =\n \t\tfmt->worktree_config;\n \tthe_repository->repository_format_partial_clone =\n@@ -1899,7 +1903,8 @@ static int is_reinit(void)\n \treturn ret;\n }\n \n-void create_reference_database(const char *initial_branch, int quiet)\n+void create_reference_database(int ref_storage_format,\n+\t\t\t const char *initial_branch, int quiet)\n {\n \tstruct strbuf err = STRBUF_INIT;\n \tint reinit = is_reinit();\n@@ -1919,6 +1924,7 @@ void create_reference_database(const char *initial_branch, int quiet)\n \tsafe_create_dir(git_path(\"refs\"), 1);\n \tadjust_shared_perm(git_path(\"refs\"));\n \n+\trepo_set_ref_storage_format(the_repository, ref_storage_format);\n \tif (refs_init_db(&err))\n \t\tdie(\"failed to set up refs db: %s\", err.buf);\n \n@@ -2137,8 +2143,20 @@ static void validate_hash_algorithm(struct repository_format *repo_fmt, int hash\n \t}\n }\n \n+static void validate_ref_storage_format(struct repository_format *repo_fmt, int format)\n+{\n+\tif (repo_fmt->version >= 0 &&\n+\t format != REF_STORAGE_FORMAT_UNKNOWN &&\n+\t format != repo_fmt->ref_storage_format) {\n+\t\tdie(_(\"attempt to reinitialize repository with different reference storage format\"));\n+\t} else if (format != REF_STORAGE_FORMAT_UNKNOWN) {\n+\t\trepo_fmt->ref_storage_format = format;\n+\t}\n+}\n+\n int init_db(const char *git_dir, const char *real_git_dir,\n-\t const char *template_dir, int hash, const char *initial_branch,\n+\t const char *template_dir, int hash,\n+\t int ref_storage_format, const char *initial_branch,\n \t int init_shared_repository, unsigned int flags)\n {\n \tint reinit;\n@@ -2181,13 +2199,15 @@ int init_db(const char *git_dir, const char *real_git_dir,\n \tcheck_repository_format(&repo_fmt);\n \n \tvalidate_hash_algorithm(&repo_fmt, hash);\n+\tvalidate_ref_storage_format(&repo_fmt, ref_storage_format);\n \n \treinit = create_default_files(template_dir, original_git_dir,\n \t\t\t\t &repo_fmt, prev_bare_repository,\n \t\t\t\t init_shared_repository);\n \n \tif (!(flags & INIT_DB_SKIP_REFDB))\n-\t\tcreate_reference_database(initial_branch, flags & INIT_DB_QUIET);\n+\t\tcreate_reference_database(repo_fmt.ref_storage_format,\n+\t\t\t\t\t initial_branch, flags & INIT_DB_QUIET);\n \tcreate_object_directory();\n \n \tif (get_shared_repository()) {\ndiff --git a/setup.h b/setup.h\nindex 3f0f17c351..4927abf2cf 100644\n--- a/setup.h\n+++ b/setup.h\n@@ -115,6 +115,7 @@ struct repository_format {\n \tint worktree_config;\n \tint is_bare;\n \tint hash_algo;\n+\tint ref_storage_format;\n \tint sparse_index;\n \tchar *work_tree;\n \tstruct string_list unknown_extensions;\n@@ -131,6 +132,7 @@ struct repository_format {\n \t.version = -1, \\\n \t.is_bare = -1, \\\n \t.hash_algo = GIT_HASH_SHA1, \\\n+\t.ref_storage_format = REF_STORAGE_FORMAT_FILES, \\\n \t.unknown_extensions = STRING_LIST_INIT_DUP, \\\n \t.v1_only_extensions = STRING_LIST_INIT_DUP, \\\n }\n@@ -175,10 +177,12 @@ void check_repository_format(struct repository_format *fmt);\n \n int init_db(const char *git_dir, const char *real_git_dir,\n \t const char *template_dir, int hash_algo,\n+\t int ref_storage_format,\n \t const char *initial_branch, int init_shared_repository,\n \t unsigned int flags);\n void initialize_repository_version(int hash_algo, int reinit);\n-void create_reference_database(const char *initial_branch, int quiet);\n+void create_reference_database(int ref_storage_format,\n+\t\t\t const char *initial_branch, int quiet);\n \n /*\n * NOTE NOTE NOTE!!\n-- \n2.43.GIT\n\n"},{"id":"486099","messageId":"01a1e58a97b4d1935d4f7c7f49eaa56ccedab3b1.1703753910.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703753910.git.ps@pks.im","subject":"[PATCH v2 05/12] setup: set repository's formats on init","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-28T09:57:44Z","receivedAt":"2023-12-28T09:57:48Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The proper hash algorithm and ref storage format that will be used for a\nnewly initialized repository will be figured out in `init_db()` via\n`validate_hash_algorithm()` and `validate_ref_storage_format()`. Until\nnow though, we never set up the hash algorithm or ref storage format of\n`the_repository` accordingly.\n\nThere are only two callsites of `init_db()`, one in git-init(1) and one\nin git-clone(1). The former function doesn't care for the formats to be\nset up properly because it never access the repository after calling the\nfunction in the first place.\n\nFor git-clone(1) it's a different story though, as we call `init_db()`\nbefore listing remote refs. While we do indeed have the wrong hash\nfunction in `the_repository` when `init_db()` sets up a non-default\nobject format for the repository, it never mattered because we adjust\nthe hash after learning about the remote's hash function via the listed\nrefs.\n\nSo the current state is correct for the hash algo, but it's not for the\nref storage format because git-clone(1) wouldn't know to set it up\nproperly. But instead of adjusting only the `ref_storage_format`, set\nboth the hash algo and the ref storage format so that `the_repository`\nis in the correct state when `init_db()` exits. This is fine as we will\nadjust the hash later on anyway and makes it easier to reason about the\nend state of `the_repository`.\n\nSigned-off-by: Patrick Steinhardt \n---\n setup.c | 7 +++++++\n 1 file changed, 7 insertions(+)\n\ndiff --git a/setup.c b/setup.c\nindex e58ab7e786..3d980814bc 100644\n--- a/setup.c\n+++ b/setup.c\n@@ -2205,6 +2205,13 @@ int init_db(const char *git_dir, const char *real_git_dir,\n \t\t\t\t &repo_fmt, prev_bare_repository,\n \t\t\t\t init_shared_repository);\n \n+\t/*\n+\t * Now that we have set up both the hash algorithm and the ref storage\n+\t * format we can update the repository's settings accordingly.\n+\t */\n+\trepo_set_hash_algo(the_repository, repo_fmt.hash_algo);\n+\trepo_set_ref_storage_format(the_repository, repo_fmt.ref_storage_format);\n+\n \tif (!(flags & INIT_DB_SKIP_REFDB))\n \t\tcreate_reference_database(repo_fmt.ref_storage_format,\n \t\t\t\t\t initial_branch, flags & INIT_DB_QUIET);\n-- \n2.43.GIT\n\n"},{"id":"486100","messageId":"0a586fa648ba618de394d0006d13a117574a260a.1703753910.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703753910.git.ps@pks.im","subject":"[PATCH v2 06/12] setup: introduce \"extensions.refStorage\" extension","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-28T09:57:49Z","receivedAt":"2023-12-28T09:57:52Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Introduce a new \"extensions.refStorage\" extension that allows us to\nspecify the ref storage format used by a repository. For now, the only\nsupported format is the \"files\" format, but this list will likely soon\nbe extended to also support the upcoming \"reftable\" format.\n\nThere have been discussions on the Git mailing list in the past around\nhow exactly this extension should look like. One alternative [1] that\nwas discussed was whether it would make sense to model the extension in\nsuch a way that backends are arbitrarily stackable. This would allow for\na combined value of e.g. \"loose,packed-refs\" or \"loose,reftable\", which\nindicates that new refs would be written via \"loose\" files backend and\ncompressed into \"packed-refs\" or \"reftable\" backends, respectively.\n\nIt is arguable though whether this flexibility and the complexity that\nit brings with it is really required for now. It is not foreseeable that\nthere will be a proliferation of backends in the near-term future, and\nthe current set of existing formats and formats which are on the horizon\ncan easily be configured with the much simpler proposal where we have a\nsingle value, only.\n\nFurthermore, if we ever see that we indeed want to gain the ability to\narbitrarily stack the ref formats, then we can adapt the current\nextension rather easily. Given that Git clients will refuse any unknown\nvalue for the \"extensions.refStorage\" extension they would also know to\nignore a stacked \"loose,packed-refs\" in the future.\n\nSo let's stick with the easy proposal for the time being and wire up the\nextension.\n\n[1]: \n\nSigned-off-by: Patrick Steinhardt \n---\n Documentation/config/extensions.txt | 11 ++++++++\n Documentation/ref-storage-format.txt | 1 +\n .../technical/repository-version.txt | 5 ++++\n builtin/clone.c | 2 +-\n setup.c | 23 +++++++++++++---\n setup.h | 3 ++-\n t/t0001-init.sh | 26 +++++++++++++++++++\n t/test-lib.sh | 2 +-\n 8 files changed, 67 insertions(+), 6 deletions(-)\n create mode 100644 Documentation/ref-storage-format.txt\n\ndiff --git a/Documentation/config/extensions.txt b/Documentation/config/extensions.txt\nindex bccaec7a96..66db0e15da 100644\n--- a/Documentation/config/extensions.txt\n+++ b/Documentation/config/extensions.txt\n@@ -7,6 +7,17 @@ Note that this setting should only be set by linkgit:git-init[1] or\n linkgit:git-clone[1]. Trying to change it after initialization will not\n work and will produce hard-to-diagnose issues.\n \n+extensions.refStorage::\n+\tSpecify the ref storage format to use. The acceptable values are:\n++\n+include::../ref-storage-format.txt[]\n++\n+It is an error to specify this key unless `core.repositoryFormatVersion` is 1.\n++\n+Note that this setting should only be set by linkgit:git-init[1] or\n+linkgit:git-clone[1]. Trying to change it after initialization will not\n+work and will produce hard-to-diagnose issues.\n+\n extensions.worktreeConfig::\n \tIf enabled, then worktrees will load config settings from the\n \t`$GIT_DIR/config.worktree` file in addition to the\ndiff --git a/Documentation/ref-storage-format.txt b/Documentation/ref-storage-format.txt\nnew file mode 100644\nindex 0000000000..1a65cac468\n--- /dev/null\n+++ b/Documentation/ref-storage-format.txt\n@@ -0,0 +1 @@\n+* `files` for loose files with packed-refs. This is the default.\ndiff --git a/Documentation/technical/repository-version.txt b/Documentation/technical/repository-version.txt\nindex 045a76756f..27be3741e6 100644\n--- a/Documentation/technical/repository-version.txt\n+++ b/Documentation/technical/repository-version.txt\n@@ -100,3 +100,8 @@ If set, by default \"git config\" reads from both \"config\" and\n multiple working directory mode, \"config\" file is shared while\n \"config.worktree\" is per-working directory (i.e., it's in\n GIT_COMMON_DIR/worktrees//config.worktree)\n+\n+==== `refStorage`\n+\n+Specifies the file format for the ref database. The only valid value\n+is `files` (loose references with a packed-refs file).\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex 48aeb1b90b..0fb3816d0c 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -1291,7 +1291,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \t * ours to the same thing.\n \t */\n \thash_algo = hash_algo_by_ptr(transport_get_hash_algo(transport));\n-\tinitialize_repository_version(hash_algo, 1);\n+\tinitialize_repository_version(hash_algo, the_repository->ref_storage_format, 1);\n \trepo_set_hash_algo(the_repository, hash_algo);\n \tcreate_reference_database(the_repository->ref_storage_format, NULL, 1);\n \ndiff --git a/setup.c b/setup.c\nindex 3d980814bc..3aac8c01cd 100644\n--- a/setup.c\n+++ b/setup.c\n@@ -592,6 +592,17 @@ static enum extension_result handle_extension(const char *var,\n \t\t\t\t \"extensions.objectformat\", value);\n \t\tdata->hash_algo = format;\n \t\treturn EXTENSION_OK;\n+\t} else if (!strcmp(ext, \"refstorage\")) {\n+\t\tint format;\n+\n+\t\tif (!value)\n+\t\t\treturn config_error_nonbool(var);\n+\t\tformat = ref_storage_format_by_name(value);\n+\t\tif (format == REF_STORAGE_FORMAT_UNKNOWN)\n+\t\t\treturn error(_(\"invalid value for '%s': '%s'\"),\n+\t\t\t\t \"extensions.refstorage\", value);\n+\t\tdata->ref_storage_format = format;\n+\t\treturn EXTENSION_OK;\n \t}\n \treturn EXTENSION_UNKNOWN;\n }\n@@ -1871,12 +1882,14 @@ static int needs_work_tree_config(const char *git_dir, const char *work_tree)\n \treturn 1;\n }\n \n-void initialize_repository_version(int hash_algo, int reinit)\n+void initialize_repository_version(int hash_algo, int ref_storage_format,\n+\t\t\t\t int reinit)\n {\n \tchar repo_version_string[10];\n \tint repo_version = GIT_REPO_VERSION;\n \n-\tif (hash_algo != GIT_HASH_SHA1)\n+\tif (hash_algo != GIT_HASH_SHA1 ||\n+\t ref_storage_format != REF_STORAGE_FORMAT_FILES)\n \t\trepo_version = GIT_REPO_VERSION_READ;\n \n \t/* This forces creation of new config file */\n@@ -1889,6 +1902,10 @@ void initialize_repository_version(int hash_algo, int reinit)\n \t\t\t hash_algos[hash_algo].name);\n \telse if (reinit)\n \t\tgit_config_set_gently(\"extensions.objectformat\", NULL);\n+\n+\tif (ref_storage_format != REF_STORAGE_FORMAT_FILES)\n+\t\tgit_config_set(\"extensions.refstorage\",\n+\t\t\t ref_storage_format_to_name(ref_storage_format));\n }\n \n static int is_reinit(void)\n@@ -2030,7 +2047,7 @@ static int create_default_files(const char *template_path,\n \t\tadjust_shared_perm(get_git_dir());\n \t}\n \n-\tinitialize_repository_version(fmt->hash_algo, 0);\n+\tinitialize_repository_version(fmt->hash_algo, fmt->ref_storage_format, 0);\n \n \t/* Check filemode trustability */\n \tpath = git_path_buf(&buf, \"config\");\ndiff --git a/setup.h b/setup.h\nindex 4927abf2cf..89033ae1d9 100644\n--- a/setup.h\n+++ b/setup.h\n@@ -180,7 +180,8 @@ int init_db(const char *git_dir, const char *real_git_dir,\n \t int ref_storage_format,\n \t const char *initial_branch, int init_shared_repository,\n \t unsigned int flags);\n-void initialize_repository_version(int hash_algo, int reinit);\n+void initialize_repository_version(int hash_algo, int ref_storage_format,\n+\t\t\t\t int reinit);\n void create_reference_database(int ref_storage_format,\n \t\t\t const char *initial_branch, int quiet);\n \ndiff --git a/t/t0001-init.sh b/t/t0001-init.sh\nindex 2b78e3be47..38b3e4c39e 100755\n--- a/t/t0001-init.sh\n+++ b/t/t0001-init.sh\n@@ -532,6 +532,32 @@ test_expect_success 'init rejects attempts to initialize with different hash' '\n \ttest_must_fail git -C sha256 init --object-format=sha1\n '\n \n+test_expect_success DEFAULT_REPO_FORMAT 'extensions.refStorage is not allowed with repo version 0' '\n+\ttest_when_finished \"rm -rf refstorage\" &&\n+\tgit init refstorage &&\n+\tgit -C refstorage config extensions.refStorage files &&\n+\ttest_must_fail git -C refstorage rev-parse 2>err &&\n+\tgrep \"repo version is 0, but v1-only extension found\" err\n+'\n+\n+test_expect_success DEFAULT_REPO_FORMAT 'extensions.refStorage with files backend' '\n+\ttest_when_finished \"rm -rf refstorage\" &&\n+\tgit init refstorage &&\n+\tgit -C refstorage config core.repositoryformatversion 1 &&\n+\tgit -C refstorage config extensions.refStorage files &&\n+\ttest_commit -C refstorage A &&\n+\tgit -C refstorage rev-parse --verify HEAD\n+'\n+\n+test_expect_success DEFAULT_REPO_FORMAT 'extensions.refStorage with unknown backend' '\n+\ttest_when_finished \"rm -rf refstorage\" &&\n+\tgit init refstorage &&\n+\tgit -C refstorage config core.repositoryformatversion 1 &&\n+\tgit -C refstorage config extensions.refStorage garbage &&\n+\ttest_must_fail git -C refstorage rev-parse 2>err &&\n+\tgrep \"invalid value for ${SQ}extensions.refstorage${SQ}: ${SQ}garbage${SQ}\" err\n+'\n+\n test_expect_success MINGW 'core.hidedotfiles = false' '\n \tgit config --global core.hidedotfiles false &&\n \trm -rf newdir &&\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex dc03f06b8e..4685cc3d48 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1937,7 +1937,7 @@ test_lazy_prereq SHA1 '\n '\n \n test_lazy_prereq DEFAULT_REPO_FORMAT '\n-\ttest_have_prereq SHA1\n+\ttest_have_prereq SHA1,REFFILES\n '\n \n # Ensure that no test accidentally triggers a Git command\n-- \n2.43.GIT\n\n"},{"id":"486101","messageId":"6d8754f73af769d94a77474e6f4b7d5447be84a7.1703753910.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703753910.git.ps@pks.im","subject":"[PATCH v2 07/12] setup: introduce GIT_DEFAULT_REF_FORMAT envvar","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-28T09:57:53Z","receivedAt":"2023-12-28T09:57:57Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Introduce a new GIT_DEFAULT_REF_FORMAT environment variable that lets\nusers control the default ref format used by both git-init(1) and\ngit-clone(1). This is modeled after GIT_DEFAULT_OBJECT_FORMAT, which\ndoes the same thing for the repository's object format.\n\nSigned-off-by: Patrick Steinhardt \n---\n Documentation/git.txt | 5 +++++\n setup.c | 7 +++++++\n t/t0001-init.sh | 18 ++++++++++++++++++\n 3 files changed, 30 insertions(+)\n\ndiff --git a/Documentation/git.txt b/Documentation/git.txt\nindex bf9e6af695..88e4ed4bd6 100644\n--- a/Documentation/git.txt\n+++ b/Documentation/git.txt\n@@ -556,6 +556,11 @@ double-quotes and respecting backslash escapes. E.g., the value\n \tis always used. The default is \"sha1\".\n \tSee `--object-format` in linkgit:git-init[1].\n \n+`GIT_DEFAULT_REF_FORMAT`::\n+\tIf this variable is set, the default reference backend format for new\n+\trepositories will be set to this value. The default is \"files\".\n+\tSee `--ref-format` in linkgit:git-init[1].\n+\n Git Commits\n ~~~~~~~~~~~\n `GIT_AUTHOR_NAME`::\ndiff --git a/setup.c b/setup.c\nindex 3aac8c01cd..4712bba6f8 100644\n--- a/setup.c\n+++ b/setup.c\n@@ -2162,12 +2162,19 @@ static void validate_hash_algorithm(struct repository_format *repo_fmt, int hash\n \n static void validate_ref_storage_format(struct repository_format *repo_fmt, int format)\n {\n+\tconst char *name = getenv(\"GIT_DEFAULT_REF_FORMAT\");\n+\n \tif (repo_fmt->version >= 0 &&\n \t format != REF_STORAGE_FORMAT_UNKNOWN &&\n \t format != repo_fmt->ref_storage_format) {\n \t\tdie(_(\"attempt to reinitialize repository with different reference storage format\"));\n \t} else if (format != REF_STORAGE_FORMAT_UNKNOWN) {\n \t\trepo_fmt->ref_storage_format = format;\n+\t} else if (name) {\n+\t\tformat = ref_storage_format_by_name(name);\n+\t\tif (format == REF_STORAGE_FORMAT_UNKNOWN)\n+\t\t\tdie(_(\"unknown ref storage format '%s'\"), name);\n+\t\trepo_fmt->ref_storage_format = format;\n \t}\n }\n \ndiff --git a/t/t0001-init.sh b/t/t0001-init.sh\nindex 38b3e4c39e..30ce752cc1 100755\n--- a/t/t0001-init.sh\n+++ b/t/t0001-init.sh\n@@ -558,6 +558,24 @@ test_expect_success DEFAULT_REPO_FORMAT 'extensions.refStorage with unknown back\n \tgrep \"invalid value for ${SQ}extensions.refstorage${SQ}: ${SQ}garbage${SQ}\" err\n '\n \n+test_expect_success DEFAULT_REPO_FORMAT 'init with GIT_DEFAULT_REF_FORMAT=files' '\n+\ttest_when_finished \"rm -rf refformat\" &&\n+\tGIT_DEFAULT_REF_FORMAT=files git init refformat &&\n+\techo 0 >expect &&\n+\tgit -C refformat config core.repositoryformatversion >actual &&\n+\ttest_cmp expect actual &&\n+\ttest_must_fail git -C refformat config extensions.refstorage\n+'\n+\n+test_expect_success 'init with GIT_DEFAULT_REF_FORMAT=garbage' '\n+\ttest_when_finished \"rm -rf refformat\" &&\n+\tcat >expect <<-EOF &&\n+\tfatal: unknown ref storage format ${SQ}garbage${SQ}\n+\tEOF\n+\ttest_must_fail env GIT_DEFAULT_REF_FORMAT=garbage git init refformat 2>err &&\n+\ttest_cmp expect err\n+'\n+\n test_expect_success MINGW 'core.hidedotfiles = false' '\n \tgit config --global core.hidedotfiles false &&\n \trm -rf newdir &&\n-- \n2.43.GIT\n\n"},{"id":"486102","messageId":"c645932f3da2395b7fdb3c8cdb28a341f8a6ae47.1703753910.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703753910.git.ps@pks.im","subject":"[PATCH v2 08/12] t: introduce GIT_TEST_DEFAULT_REF_FORMAT envvar","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-28T09:57:58Z","receivedAt":"2023-12-28T09:58:01Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Introduce a new GIT_TEST_DEFAULT_REF_FORMAT environment variable that\nlets developers run the test suite with a different default ref format\nwithout impacting the ref format used by non-test Git invocations. This\nis modeled after GIT_TEST_DEFAULT_OBJECT_FORMAT, which does the same\nthing for the repository's object format.\n\nAdapt the setup of the `REFFILES` test prerequisite to be conditionally\nset based on the default ref format.\n\nSigned-off-by: Patrick Steinhardt \n---\n t/README | 3 +++\n t/test-lib-functions.sh | 5 +++++\n t/test-lib.sh | 11 ++++++++++-\n 3 files changed, 18 insertions(+), 1 deletion(-)\n\ndiff --git a/t/README b/t/README\nindex 36463d0742..621d3b8c09 100644\n--- a/t/README\n+++ b/t/README\n@@ -479,6 +479,9 @@ GIT_TEST_DEFAULT_HASH= specifies which hash algorithm to\n use in the test scripts. Recognized values for are \"sha1\"\n and \"sha256\".\n \n+GIT_TEST_DEFAULT_REF_FORMAT= specifies which ref storage format\n+to use in the test scripts. Recognized values for are \"files\".\n+\n GIT_TEST_NO_WRITE_REV_INDEX=, when true disables the\n 'pack.writeReverseIndex' setting.\n \ndiff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh\nindex 5eb57914ab..a3a51ea9e8 100644\n--- a/t/test-lib-functions.sh\n+++ b/t/test-lib-functions.sh\n@@ -1659,6 +1659,11 @@ test_detect_hash () {\n \ttest_hash_algo=\"${GIT_TEST_DEFAULT_HASH:-sha1}\"\n }\n \n+# Detect the hash algorithm in use.\n+test_detect_ref_format () {\n+\techo \"${GIT_TEST_DEFAULT_REF_FORMAT:-files}\"\n+}\n+\n # Load common hash metadata and common placeholder object IDs for use with\n # test_oid.\n test_oid_init () {\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 4685cc3d48..fc93aa57e6 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -542,6 +542,8 @@ export EDITOR\n \n GIT_DEFAULT_HASH=\"${GIT_TEST_DEFAULT_HASH:-sha1}\"\n export GIT_DEFAULT_HASH\n+GIT_DEFAULT_REF_FORMAT=\"${GIT_TEST_DEFAULT_REF_FORMAT:-files}\"\n+export GIT_DEFAULT_REF_FORMAT\n GIT_TEST_MERGE_ALGORITHM=\"${GIT_TEST_MERGE_ALGORITHM:-ort}\"\n export GIT_TEST_MERGE_ALGORITHM\n \n@@ -1745,7 +1747,14 @@ parisc* | hppa*)\n \t;;\n esac\n \n-test_set_prereq REFFILES\n+case \"$GIT_DEFAULT_REF_FORMAT\" in\n+files)\n+\ttest_set_prereq REFFILES;;\n+*)\n+\techo 2>&1 \"error: unknown ref format $GIT_DEFAULT_REF_FORMAT\"\n+\texit 1\n+\t;;\n+esac\n \n ( COLUMNS=1 && test $COLUMNS = 1 ) && test_set_prereq COLUMNS_CAN_BE_1\n test -z \"$NO_CURL\" && test_set_prereq LIBCURL\n-- \n2.43.GIT\n\n"},{"id":"486103","messageId":"761d647770845320682e3a89fe1a92cb3bd96cc9.1703753910.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703753910.git.ps@pks.im","subject":"[PATCH v2 09/12] builtin/rev-parse: introduce `--show-ref-format` flag","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-28T09:58:02Z","receivedAt":"2023-12-28T09:58:05Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Introduce a new `--show-ref-format` to git-rev-parse(1) that causes it\nto print the ref format used by a repository.\n\nSigned-off-by: Patrick Steinhardt \n---\n Documentation/git-rev-parse.txt | 3 +++\n builtin/rev-parse.c | 4 ++++\n t/t1500-rev-parse.sh | 17 +++++++++++++++++\n 3 files changed, 24 insertions(+)\n\ndiff --git a/Documentation/git-rev-parse.txt b/Documentation/git-rev-parse.txt\nindex 912fab9f5e..546faf9017 100644\n--- a/Documentation/git-rev-parse.txt\n+++ b/Documentation/git-rev-parse.txt\n@@ -307,6 +307,9 @@ The following options are unaffected by `--path-format`:\n \tinput, multiple algorithms may be printed, space-separated.\n \tIf not specified, the default is \"storage\".\n \n+--show-ref-format::\n+\tShow the reference storage format used for the repository.\n+\n \n Other Options\n ~~~~~~~~~~~~~\ndiff --git a/builtin/rev-parse.c b/builtin/rev-parse.c\nindex 917f122440..d08987646a 100644\n--- a/builtin/rev-parse.c\n+++ b/builtin/rev-parse.c\n@@ -1062,6 +1062,10 @@ int cmd_rev_parse(int argc, const char **argv, const char *prefix)\n \t\t\t\tputs(the_hash_algo->name);\n \t\t\t\tcontinue;\n \t\t\t}\n+\t\t\tif (!strcmp(arg, \"--show-ref-format\")) {\n+\t\t\t\tputs(ref_storage_format_to_name(the_repository->ref_storage_format));\n+\t\t\t\tcontinue;\n+\t\t\t}\n \t\t\tif (!strcmp(arg, \"--end-of-options\")) {\n \t\t\t\tseen_end_of_options = 1;\n \t\t\t\tif (filter & (DO_FLAGS | DO_REVS))\ndiff --git a/t/t1500-rev-parse.sh b/t/t1500-rev-parse.sh\nindex 3f9e7f62e4..a669e592f1 100755\n--- a/t/t1500-rev-parse.sh\n+++ b/t/t1500-rev-parse.sh\n@@ -208,6 +208,23 @@ test_expect_success 'rev-parse --show-object-format in repo' '\n \tgrep \"unknown mode for --show-object-format: squeamish-ossifrage\" err\n '\n \n+test_expect_success 'rev-parse --show-ref-format' '\n+\ttest_detect_ref_format >expect &&\n+\tgit rev-parse --show-ref-format >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'rev-parse --show-ref-format with invalid storage' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\tgit config extensions.refstorage broken &&\n+\t\ttest_must_fail git rev-parse --show-ref-format 2>err &&\n+\t\tgrep \"error: invalid value for ${SQ}extensions.refstorage${SQ}: ${SQ}broken${SQ}\" err\n+\t)\n+'\n+\n test_expect_success '--show-toplevel from subdir of working tree' '\n \tpwd >expect &&\n \tgit -C sub/dir rev-parse --show-toplevel >actual &&\n-- \n2.43.GIT\n\n"},{"id":"486104","messageId":"e382b5bf0853726169ade0f657bd70ad9cf57710.1703753910.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703753910.git.ps@pks.im","subject":"[PATCH v2 10/12] builtin/init: introduce `--ref-format=` value flag","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-28T09:58:06Z","receivedAt":"2023-12-28T09:58:10Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Introduce a new `--ref-format` value flag for git-init(1) that allows\nthe user to specify the ref format that is to be used for a newly\ninitialized repository.\n\nSigned-off-by: Patrick Steinhardt \n---\n Documentation/git-init.txt | 7 +++++++\n builtin/init-db.c | 13 ++++++++++++-\n t/t0001-init.sh | 26 ++++++++++++++++++++++++++\n 3 files changed, 45 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-init.txt b/Documentation/git-init.txt\nindex 6f0d2973bf..e8dc645bb5 100644\n--- a/Documentation/git-init.txt\n+++ b/Documentation/git-init.txt\n@@ -11,6 +11,7 @@ SYNOPSIS\n [verse]\n 'git init' [-q | --quiet] [--bare] [--template=]\n \t [--separate-git-dir ] [--object-format=]\n+\t [--ref-format=]\n \t [-b | --initial-branch=]\n \t [--shared[=]] []\n \n@@ -57,6 +58,12 @@ values are 'sha1' and (if enabled) 'sha256'. 'sha1' is the default.\n +\n include::object-format-disclaimer.txt[]\n \n+--ref-format=::\n+\n+Specify the given ref storage format for the repository. The valid values are:\n++\n+include::ref-storage-format.txt[]\n+\n --template=::\n \n Specify the directory from which templates will be used. (See the \"TEMPLATE\ndiff --git a/builtin/init-db.c b/builtin/init-db.c\nindex b6e80feab6..770b2822fe 100644\n--- a/builtin/init-db.c\n+++ b/builtin/init-db.c\n@@ -58,6 +58,7 @@ static int shared_callback(const struct option *opt, const char *arg, int unset)\n static const char *const init_db_usage[] = {\n \tN_(\"git init [-q | --quiet] [--bare] [--template=]\\n\"\n \t \" [--separate-git-dir ] [--object-format=]\\n\"\n+\t \" [--ref-format=]\\n\"\n \t \" [-b | --initial-branch=]\\n\"\n \t \" [--shared[=]] []\"),\n \tNULL\n@@ -77,8 +78,10 @@ int cmd_init_db(int argc, const char **argv, const char *prefix)\n \tconst char *template_dir = NULL;\n \tunsigned int flags = 0;\n \tconst char *object_format = NULL;\n+\tconst char *ref_format = NULL;\n \tconst char *initial_branch = NULL;\n \tint hash_algo = GIT_HASH_UNKNOWN;\n+\tint ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN;\n \tint init_shared_repository = -1;\n \tconst struct option init_db_options[] = {\n \t\tOPT_STRING(0, \"template\", &template_dir, N_(\"template-directory\"),\n@@ -96,6 +99,8 @@ int cmd_init_db(int argc, const char **argv, const char *prefix)\n \t\t\t N_(\"override the name of the initial branch\")),\n \t\tOPT_STRING(0, \"object-format\", &object_format, N_(\"hash\"),\n \t\t\t N_(\"specify the hash algorithm to use\")),\n+\t\tOPT_STRING(0, \"ref-format\", &ref_format, N_(\"format\"),\n+\t\t\t N_(\"specify the reference format to use\")),\n \t\tOPT_END()\n \t};\n \n@@ -159,6 +164,12 @@ int cmd_init_db(int argc, const char **argv, const char *prefix)\n \t\t\tdie(_(\"unknown hash algorithm '%s'\"), object_format);\n \t}\n \n+\tif (ref_format) {\n+\t\tref_storage_format = ref_storage_format_by_name(ref_format);\n+\t\tif (ref_storage_format == REF_STORAGE_FORMAT_UNKNOWN)\n+\t\t\tdie(_(\"unknown ref storage format '%s'\"), ref_format);\n+\t}\n+\n \tif (init_shared_repository != -1)\n \t\tset_shared_repository(init_shared_repository);\n \n@@ -237,6 +248,6 @@ int cmd_init_db(int argc, const char **argv, const char *prefix)\n \n \tflags |= INIT_DB_EXIST_OK;\n \treturn init_db(git_dir, real_git_dir, template_dir, hash_algo,\n-\t\t REF_STORAGE_FORMAT_UNKNOWN, initial_branch,\n+\t\t ref_storage_format, initial_branch,\n \t\t init_shared_repository, flags);\n }\ndiff --git a/t/t0001-init.sh b/t/t0001-init.sh\nindex 30ce752cc1..b131d665db 100755\n--- a/t/t0001-init.sh\n+++ b/t/t0001-init.sh\n@@ -576,6 +576,32 @@ test_expect_success 'init with GIT_DEFAULT_REF_FORMAT=garbage' '\n \ttest_cmp expect err\n '\n \n+test_expect_success 'init with --ref-format=files' '\n+\ttest_when_finished \"rm -rf refformat\" &&\n+\tgit init --ref-format=files refformat &&\n+\techo files >expect &&\n+\tgit -C refformat rev-parse --show-ref-format >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 're-init with same format' '\n+\ttest_when_finished \"rm -rf refformat\" &&\n+\tgit init --ref-format=files refformat &&\n+\tgit init --ref-format=files refformat &&\n+\techo files >expect &&\n+\tgit -C refformat rev-parse --show-ref-format >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'init with --ref-format=garbage' '\n+\ttest_when_finished \"rm -rf refformat\" &&\n+\tcat >expect <<-EOF &&\n+\tfatal: unknown ref storage format ${SQ}garbage${SQ}\n+\tEOF\n+\ttest_must_fail git init --ref-format=garbage refformat 2>err &&\n+\ttest_cmp expect err\n+'\n+\n test_expect_success MINGW 'core.hidedotfiles = false' '\n \tgit config --global core.hidedotfiles false &&\n \trm -rf newdir &&\n-- \n2.43.GIT\n\n"},{"id":"486105","messageId":"257233658de252252f2493593f6af958947c2aa2.1703753910.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703753910.git.ps@pks.im","subject":"[PATCH v2 11/12] builtin/clone: introduce `--ref-format=` value flag","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-28T09:58:11Z","receivedAt":"2023-12-28T09:58:14Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Introduce a new `--ref-format` value flag for git-clone(1) that allows\nthe user to specify the ref format that is to be used for a newly\ninitialized repository.\n\nSigned-off-by: Patrick Steinhardt \n---\n Documentation/git-clone.txt | 6 ++++++\n builtin/clone.c | 12 +++++++++++-\n t/t5601-clone.sh | 17 +++++++++++++++++\n 3 files changed, 34 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt\nindex c37c4a37f7..6e43eb9c20 100644\n--- a/Documentation/git-clone.txt\n+++ b/Documentation/git-clone.txt\n@@ -311,6 +311,12 @@ or `--mirror` is given)\n \tThe result is Git repository can be separated from working\n \ttree.\n \n+--ref-format=::\n --jobs ::\n \tThe number of submodules fetched at the same time.\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex 0fb3816d0c..18093d5d16 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -72,6 +72,7 @@ static char *remote_name = NULL;\n static char *option_branch = NULL;\n static struct string_list option_not = STRING_LIST_INIT_NODUP;\n static const char *real_git_dir;\n+static const char *ref_format;\n static char *option_upload_pack = \"git-upload-pack\";\n static int option_verbosity;\n static int option_progress = -1;\n@@ -157,6 +158,8 @@ static struct option builtin_clone_options[] = {\n \t\t N_(\"any cloned submodules will be shallow\")),\n \tOPT_STRING(0, \"separate-git-dir\", &real_git_dir, N_(\"gitdir\"),\n \t\t N_(\"separate git dir from working tree\")),\n+\tOPT_STRING(0, \"ref-format\", &ref_format, N_(\"format\"),\n+\t\t N_(\"specify the reference format to use\")),\n \tOPT_STRING_LIST('c', \"config\", &option_config, N_(\"key=value\"),\n \t\t\tN_(\"set config inside the new repository\")),\n \tOPT_STRING_LIST(0, \"server-option\", &server_options,\n@@ -932,6 +935,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \tint submodule_progress;\n \tint filter_submodules = 0;\n \tint hash_algo;\n+\tint ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN;\n \tconst int do_not_override_repo_unix_permissions = -1;\n \n \tstruct transport_ls_refs_options transport_ls_refs_options =\n@@ -957,6 +961,12 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \tif (option_single_branch == -1)\n \t\toption_single_branch = deepen ? 1 : 0;\n \n+\tif (ref_format) {\n+\t\tref_storage_format = ref_storage_format_by_name(ref_format);\n+\t\tif (ref_storage_format == REF_STORAGE_FORMAT_UNKNOWN)\n+\t\t\tdie(_(\"unknown ref storage format '%s'\"), ref_format);\n+\t}\n+\n \tif (option_mirror)\n \t\toption_bare = 1;\n \n@@ -1108,7 +1118,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \t * their on-disk data structures.\n \t */\n \tinit_db(git_dir, real_git_dir, option_template, GIT_HASH_UNKNOWN,\n-\t\tREF_STORAGE_FORMAT_UNKNOWN, NULL,\n+\t\tref_storage_format, NULL,\n \t\tdo_not_override_repo_unix_permissions, INIT_DB_QUIET | INIT_DB_SKIP_REFDB);\n \n \tif (real_git_dir) {\ndiff --git a/t/t5601-clone.sh b/t/t5601-clone.sh\nindex 47eae641f0..fb1b9c686d 100755\n--- a/t/t5601-clone.sh\n+++ b/t/t5601-clone.sh\n@@ -157,6 +157,23 @@ test_expect_success 'clone --mirror does not repeat tags' '\n \n '\n \n+test_expect_success 'clone with files ref format' '\n+\ttest_when_finished \"rm -rf ref-storage\" &&\n+\tgit clone --ref-format=files --mirror src ref-storage &&\n+\techo files >expect &&\n+\tgit -C ref-storage rev-parse --show-ref-format >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'clone with garbage ref format' '\n+\tcat >expect <<-EOF &&\n+\tfatal: unknown ref storage format ${SQ}garbage${SQ}\n+\tEOF\n+\ttest_must_fail git clone --ref-format=garbage --mirror src ref-storage 2>err &&\n+\ttest_cmp expect err &&\n+\ttest_path_is_missing ref-storage\n+'\n+\n test_expect_success 'clone to destination with trailing /' '\n \n \tgit clone src target-1/ &&\n-- \n2.43.GIT\n\n"},{"id":"486106","messageId":"b8cd06ec53091b124273f621db8de13cdb38835e.1703753910.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703753910.git.ps@pks.im","subject":"[PATCH v2 12/12] t9500: write \"extensions.refstorage\" into config","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-28T09:58:15Z","receivedAt":"2023-12-28T09:58:18Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"In t9500 we're writing a custom configuration that sets up gitweb. This\nrequires us to manually ensure that the repository format is configured\nas required, including both the repository format version and\nextensions. With the introduction of the \"extensions.refStorage\"\nextension we need to update the test to also write this new one.\n\nSigned-off-by: Patrick Steinhardt \n---\n t/t9500-gitweb-standalone-no-errors.sh | 5 +++++\n 1 file changed, 5 insertions(+)\n\ndiff --git a/t/t9500-gitweb-standalone-no-errors.sh b/t/t9500-gitweb-standalone-no-errors.sh\nindex 0333065d4d..7679780fb8 100755\n--- a/t/t9500-gitweb-standalone-no-errors.sh\n+++ b/t/t9500-gitweb-standalone-no-errors.sh\n@@ -627,6 +627,7 @@ test_expect_success \\\n test_expect_success 'setup' '\n \tversion=$(git config core.repositoryformatversion) &&\n \talgo=$(test_might_fail git config extensions.objectformat) &&\n+\trefstorage=$(test_might_fail git config extensions.refstorage) &&\n \tcat >.git/config <<-\\EOF &&\n \t# testing noval and alternate separator\n \t[gitweb]\n@@ -637,6 +638,10 @@ test_expect_success 'setup' '\n \tif test -n \"$algo\"\n \tthen\n \t\tgit config extensions.objectformat \"$algo\"\n+\tfi &&\n+\tif test -n \"$refstorage\"\n+\tthen\n+\t\tgit config extensions.refstorage \"$refstorage\"\n \tfi\n '\n \n-- \n2.43.GIT\n\n"},{"id":"486129","messageId":"xmqqplyqgsem.fsf@gitster.g","threadId":"60639","inReplyTo":"ZY04OlTNUEZs5T-T@tanuki","subject":"Re: [PATCH 04/12] setup: start tracking ref storage format when","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-12-28T17:15:29Z","receivedAt":"2023-12-28T17:15:32Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> Makes me wonder whether we should then also add the following diff to\n> \"setup: set repository's format on init\" when both topics are being\n> merged together:\n>\n> diff --git a/setup.c b/setup.c\n> index 3d980814bc..3d35c78c68 100644\n> --- a/setup.c\n> +++ b/setup.c\n> @@ -2210,6 +2210,7 @@ int init_db(const char *git_dir, const char *real_git_dir,\n> \t * format we can update the repository's settings accordingly.\n> \t */\n> \trepo_set_hash_algo(the_repository, repo_fmt.hash_algo);\n> +\trepo_set_compat_hash_algo(the_repository, repo_fmt.compat_hash_algo);\n> \trepo_set_ref_storage_format(the_repository, repo_fmt.ref_storage_format);\n> \n> \tif (!(flags & INIT_DB_SKIP_REFDB))\n\nShouldn't that come from the series that wants .compat_hash_algo in\nthe repo_fmt structure, whichever it is, not added by an evil merge?\n"},{"id":"486130","messageId":"xmqqjzoygrx8.fsf@gitster.g","threadId":"60639","inReplyTo":"12329b99b753f79fe93fe017e71b08227d213c1e.1703753910.git.ps@pks.im","subject":"Re: [PATCH v2 03/12] refs: refactor logic to look up storage backends","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-12-28T17:25:55Z","receivedAt":"2023-12-28T17:26:04Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> In order to look up ref storage backends, we're currently using a linked\n> list of backends, where each backend is expected to set up its `next`\n> pointer to the next ref storage backend. This is kind of a weird setup\n> as backends need to be aware of other backends without much of a reason.\n>\n> Refactor the code so that the array of backends is centrally defined in\n> \"refs.c\", where each backend is now identified by an integer constant.\n> Expose functions to translate from those integer constants to the name\n> and vice versa, which will be required by subsequent patches.\n\nA small question. Does this have to be \"int\", or is \"unsigned\" (or\neven an enum, rewrittenfrom the \"REF_STORAGE_FORMAT_*\" family of CPP\nmacro constants) good enough? I am only wondering what happens when\nyou clal find_ref_storage_backend() with a negative index.\n\nFor that matter, how REF_STORAGE_FORMAT_UNKNOWN (whose value is 0)\nis handled by the function also gets curious. The caller may have\nto find that the backend hasn't been specified by receiving an\nelement in the refs_backends[] array that corresponds to it, but the\nerror behaviour of this function is also to return NULL, so it has\nto be prepared to handle both cases?\n\n> +static const struct ref_storage_be *refs_backends[] = {\n> +\t[REF_STORAGE_FORMAT_FILES] = &refs_be_files,\n> +};\n> ...\n> +static const struct ref_storage_be *find_ref_storage_backend(int ref_storage_format)\n> {\n> +\tif (ref_storage_format < ARRAY_SIZE(refs_backends))\n> +\t\treturn refs_backends[ref_storage_format];\n> \treturn NULL;\n> }\n"},{"id":"486131","messageId":"CAPig+cT6mRyJijL1qo2g56Yny-JxkDYjjmGpAncyS_4Hcpaz6Q@mail.gmail.com","threadId":"60639","inReplyTo":"ecf4f1ddee36643f0ff7e3d40b9aa7c7e6e6ce43.1703753910.git.ps@pks.im","subject":"Re: [PATCH v2 02/12] worktree: skip reading HEAD when repairing worktrees","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2023-12-28T18:08:24Z","receivedAt":"2023-12-28T18:08:36Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Thu, Dec 28, 2023 at 4:57 AM Patrick Steinhardt wrote:\n> When calling `git init --separate-git-dir=` on a preexisting\n> repository, we move the Git directory of that repository to the new path\n> specified by the user. If there are worktrees present in the repository,\n> we need to repair the worktrees so that their gitlinks point to the new\n> location of the repository.\n>\n> This repair logic will load repositories via `get_worktrees()`, which\n> will enumerate up and initialize all worktrees. Part of initialization\n> is logic that we resolve their respective worktree HEADs, even though\n> that information may not actually be needed in the end by all callers.\n>\n> In the context of git-init(1) this is about to become a problem, because\n> we do not have a repository that was set up via `setup_git_directory()`\n> or friends. Consequentially, it is not yet fully initialized at the time\n> of calling `repair_worktrees()`, and properly setting up all parts of\n> the repository in `init_db()` before we repair worktrees is not an easy\n> thing to do. While this is okay right now where we only have a single\n> reference backend in Git, once we gain a second one we would be trying\n> to look up the worktree HEADs before we have figured out the reference\n> format, which does not work.\n\ns/Consequentially/Consequently/\n\nI found it difficult to digest this paragraph with its foreshadowing\nphrase \"about to become a problem\" since it wasn't apparent until the\nvery final sentence in the paragraph what the actual problem would be.\nPerhaps if you mention early on that the reftable backend will have\ntrouble with the current code, it would be easier to grasp. Maybe\nsomething like this:\n\n Although not a problem presently with the file-based reference\n backend, it will become a problem with the upcoming reftable\n backend. In the context of git-init(1) a fully-materialized\n repository set up via `setup_git_directory()` or friends is not\n yet present. Consequently, it is not yet fully initialized at the\n time `repair_worktrees()` is called, and properly setting up all\n parts of the repository in `init_db()` before we repair worktrees\n is not an easy task. With introduction of the reftable backend,\n it would try to look up the worktree HEADs before we have figured\n out the reference format, thus would not work.\n\n> We do not require the worktree HEADs at all to repair worktrees. So\n> let's fix this issue by skipping over the step that reads them.\n>\n> Signed-off-by: Patrick Steinhardt \n> ---\n> diff --git a/worktree.c b/worktree.c\n> @@ -51,7 +51,7 @@ static void add_head_info(struct worktree *wt)\n> -static struct worktree *get_main_worktree(void)\n> +static struct worktree *get_main_worktree(int skip_reading_head)\n> {\n> - add_head_info(worktree);\n> + if (!skip_reading_head)\n> + add_head_info(worktree);\n\nThis is so special-case that it feels more than a little dirty.\n\n> @@ -591,7 +599,7 @@ static void repair_noop(int iserr UNUSED,\n> void repair_worktrees(worktree_repair_fn fn, void *cb_data)\n> {\n> - struct worktree **worktrees = get_worktrees();\n> + struct worktree **worktrees = get_worktrees_internal(1);\n\nIn an ideal world, a repair function should not be calling\nget_worktrees() at all since get_worktrees() is not tolerant of\ncorruption of the worktree administrative files. (Plus, as you note,\nit does more work than necessary for the current set of repairs\nperformed by `git worktree repair`.)\n\nEven as I was implementing the worktree repair code, I wavered back\nand forth multiple times between calling get_worktrees() and writing a\ncustom corruption-tolerant function to retrieve worktree\nadministrative information. In the end, I opted for get_worktrees()\nfor the pragmatic reason that it allowed me to narrow the scope of the\npatches to the types of repairs which were the current focus without\ngetting mired down in the involved details of writing a\ncorruption-tolerant function for retrieving worktree metadata.\nHowever, that decision was made with the understanding that the\npragmatic choice of the moment would not rule out the possibility of\nreturning later and implementing the more correct approach of having a\ncorruption-tolerant function for retrieving worktree metadata.\n\nThe special-case ugliness of this patch suggests strongly in favor of\nimplementing the earlier-envisioned corruption-tolerant function for\nretrieving worktree metadata rather than the band-aid approach taken\nby this patch. The generic name get_worktrees_internal() isn't helpful\neither; it doesn't do a good job of conveying any particular meaning\nto the reader.\n\nHaving said all that, I'm not overly opposed to this patch, especially\nsince your main focus is on getting the reftable backend integrated,\nand because the changes (and ugliness) introduced by this patch are\nentirely self-contained and private to worktree.c, so are not a\nshow-stopper by any means. Rather, I wanted to get down to writing\nwhat I think would be a better future approach if someone gets around\nto tackling it. (There is no pressing need at the moment, and that\nsomeone doesn't have to be you.)\n"},{"id":"486133","messageId":"CAPig+cSKpzOCOzC_mtNoA4yYmHCtMxB-Ujsd7YYHK-SPJvgt8w@mail.gmail.com","threadId":"60639","inReplyTo":"CAPig+cT6mRyJijL1qo2g56Yny-JxkDYjjmGpAncyS_4Hcpaz6Q@mail.gmail.com","subject":"Re: [PATCH v2 02/12] worktree: skip reading HEAD when repairing worktrees","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2023-12-28T18:13:04Z","receivedAt":"2023-12-28T18:13:16Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Thu, Dec 28, 2023 at 1:08 PM Eric Sunshine wrote:\n> Having said all that, I'm not overly opposed to this patch, especially\n> since your main focus is on getting the reftable backend integrated,\n> and because the changes (and ugliness) introduced by this patch are\n> entirely self-contained and private to worktree.c, so are not a\n> show-stopper by any means. Rather, I wanted to get down to writing\n> what I think would be a better future approach if someone gets around\n> to tackling it. (There is no pressing need at the moment, and that\n> someone doesn't have to be you.)\n\nI forgot to mention that, if you reroll for some reason, the\nget_worktrees()/get_worktrees_internal() dance might deserve an\nin-source NEEDSWORK comment explaining that get_worktrees_internal()\nexists to work around the shortcoming that a corruption-tolerant\nfunction for retrieving worktree metadata (for use by the \"repair\"\nfunction) does not yet exist.\n"},{"id":"486140","messageId":"ZY3UMwe0X4gVHyAm@framework","threadId":"60639","inReplyTo":"xmqqplyqgsem.fsf@gitster.g","subject":"Re: [PATCH 04/12] setup: start tracking ref storage format when","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-28T20:01:55Z","receivedAt":"2023-12-28T20:01:59Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Dec 28, 2023 at 09:15:29AM -0800, Junio C Hamano wrote:\n> Patrick Steinhardt writes:\n> \n> > Makes me wonder whether we should then also add the following diff to\n> > \"setup: set repository's format on init\" when both topics are being\n> > merged together:\n> >\n> > diff --git a/setup.c b/setup.c\n> > index 3d980814bc..3d35c78c68 100644\n> > --- a/setup.c\n> > +++ b/setup.c\n> > @@ -2210,6 +2210,7 @@ int init_db(const char *git_dir, const char *real_git_dir,\n> > \t * format we can update the repository's settings accordingly.\n> > \t */\n> > \trepo_set_hash_algo(the_repository, repo_fmt.hash_algo);\n> > +\trepo_set_compat_hash_algo(the_repository, repo_fmt.compat_hash_algo);\n> > \trepo_set_ref_storage_format(the_repository, repo_fmt.ref_storage_format);\n> > \n> > \tif (!(flags & INIT_DB_SKIP_REFDB))\n> \n> Shouldn't that come from the series that wants .compat_hash_algo in\n> the repo_fmt structure, whichever it is, not added by an evil merge?\n\nWell, the above code is newly added by my series to ensure that\n`init_db()` results in a properly initialized repo upon return. So the\ncompat hash algo series cannot yet call `repo_set_compat_hash_algo()`\nbecause the code site doesn't exist, whereas my series cannot yet add\nthe call because there is no compat hash algo yet.\n\nSo depending on which series lands first we'll either have to adapt the\nrespective other series or do an evil merge.\n\nPatrick\n"},{"id":"486141","messageId":"ZY3Wcua6dtzO2jG4@framework","threadId":"60639","inReplyTo":"xmqqjzoygrx8.fsf@gitster.g","subject":"Re: [PATCH v2 03/12] refs: refactor logic to look up storage backends","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-28T20:11:30Z","receivedAt":"2023-12-28T20:11:36Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Dec 28, 2023 at 09:25:55AM -0800, Junio C Hamano wrote:\n> Patrick Steinhardt writes:\n> \n> > In order to look up ref storage backends, we're currently using a linked\n> > list of backends, where each backend is expected to set up its `next`\n> > pointer to the next ref storage backend. This is kind of a weird setup\n> > as backends need to be aware of other backends without much of a reason.\n> >\n> > Refactor the code so that the array of backends is centrally defined in\n> > \"refs.c\", where each backend is now identified by an integer constant.\n> > Expose functions to translate from those integer constants to the name\n> > and vice versa, which will be required by subsequent patches.\n> \n> A small question. Does this have to be \"int\", or is \"unsigned\" (or\n> even an enum, rewrittenfrom the \"REF_STORAGE_FORMAT_*\" family of CPP\n> macro constants) good enough? I am only wondering what happens when\n> you clal find_ref_storage_backend() with a negative index.\n\nNo, it does not have to be an `int`, and handling a negative index would\nbe a bug. I tried to stick to what we have with `GIT_HASH_UNKNOWN`,\n`GIT_HASH_SHA1` etc, which is exactly similar in spirit. Whether it's\nthe perfect way to handle this... probably not. Without the context I\nwould've used an `enum`, but instead I opted for consistency.\n\n> For that matter, how REF_STORAGE_FORMAT_UNKNOWN (whose value is 0)\n> is handled by the function also gets curious. The caller may have\n> to find that the backend hasn't been specified by receiving an\n> element in the refs_backends[] array that corresponds to it, but the\n> error behaviour of this function is also to return NULL, so it has\n> to be prepared to handle both cases?\n\nYeah, we do not really discern those two cases for now and instead just\nreturn `NULL` both for any unknown ref storage format. All callers know\nto handle `NULL`, but the error handling will only report a generic\n\"unknown\" backend error.\n\nThe easiest way to discern those cases would be to `BUG()` when being\npassed an invalid ref storage format smaller than 0 or larger than the\nnumber of known backends. Because ultimately it is just that, a bug that\nshouldn't ever occur.\n\nNot sure whether this is worth a reroll?\n\nPatrick\n\n> > +static const struct ref_storage_be *refs_backends[] = {\n> > +\t[REF_STORAGE_FORMAT_FILES] = &refs_be_files,\n> > +};\n> > ...\n> > +static const struct ref_storage_be *find_ref_storage_backend(int ref_storage_format)\n> > {\n> > +\tif (ref_storage_format < ARRAY_SIZE(refs_backends))\n> > +\t\treturn refs_backends[ref_storage_format];\n> > \treturn NULL;\n> > }\n"},{"id":"486142","messageId":"ZY3YAyybyZaxctRp@framework","threadId":"60639","inReplyTo":"CAPig+cSKpzOCOzC_mtNoA4yYmHCtMxB-Ujsd7YYHK-SPJvgt8w@mail.gmail.com","subject":"Re: [PATCH v2 02/12] worktree: skip reading HEAD when repairing worktrees","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-28T20:18:11Z","receivedAt":"2023-12-28T20:18:16Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Dec 28, 2023 at 01:13:04PM -0500, Eric Sunshine wrote:\n> On Thu, Dec 28, 2023 at 1:08 PM Eric Sunshine wrote:\n> > Having said all that, I'm not overly opposed to this patch, especially\n> > since your main focus is on getting the reftable backend integrated,\n> > and because the changes (and ugliness) introduced by this patch are\n> > entirely self-contained and private to worktree.c, so are not a\n> > show-stopper by any means. Rather, I wanted to get down to writing\n> > what I think would be a better future approach if someone gets around\n> > to tackling it. (There is no pressing need at the moment, and that\n> > someone doesn't have to be you.)\n> \n> I forgot to mention that, if you reroll for some reason, the\n> get_worktrees()/get_worktrees_internal() dance might deserve an\n> in-source NEEDSWORK comment explaining that get_worktrees_internal()\n> exists to work around the shortcoming that a corruption-tolerant\n> function for retrieving worktree metadata (for use by the \"repair\"\n> function) does not yet exist.\n\nThanks for sharing your thoughts, will do.\n\nPatrick\n"},{"id":"486144","messageId":"xmqq4jg2gitt.fsf@gitster.g","threadId":"60639","inReplyTo":"ZY3Wcua6dtzO2jG4@framework","subject":"Re: [PATCH v2 03/12] refs: refactor logic to look up storage backends","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-12-28T20:42:22Z","receivedAt":"2023-12-28T20:42:27Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> Yeah, we do not really discern those two cases for now and instead just\n> return `NULL` both for any unknown ref storage format. All callers know\n> to handle `NULL`, but the error handling will only report a generic\n> \"unknown\" backend error.\n>\n> The easiest way to discern those cases would be to `BUG()` when being\n> passed an invalid ref storage format smaller than 0 or larger than the\n> number of known backends. Because ultimately it is just that, a bug that\n> shouldn't ever occur.\n>\n> Not sure whether this is worth a reroll?\n\nBy using an unsigned type, you no longer have to worry about getting\nhanded a negative index, as the \"must be smaller than ARRAY_SIZE()\"\ncheck will be sufficient to catch anybody who passes \"-1\" (casted to\nunsigned by parameter passing). So I would say that would be a good\nenough reason to reroll, whether we differentiate 0 and an index\nthat is larger than refs_backends[] (or a negative one) with an\nexplicit BUG(), or just leave it to the caller by returning NULL.\nAs to the error handling, I suspect it is sufficient to return NULL\nand let the caller handle it.\n\nThanks.\n\n\n>\n> Patrick\n>\n>> > +static const struct ref_storage_be *refs_backends[] = {\n>> > +\t[REF_STORAGE_FORMAT_FILES] = &refs_be_files,\n>> > +};\n>> > ...\n>> > +static const struct ref_storage_be *find_ref_storage_backend(int ref_storage_format)\n>> > {\n>> > +\tif (ref_storage_format < ARRAY_SIZE(refs_backends))\n>> > +\t\treturn refs_backends[ref_storage_format];\n>> > \treturn NULL;\n>> > }\n"},{"id":"486147","messageId":"cover.1703833818.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703067989.git.ps@pks.im","subject":"[PATCH v3 00/12] Introduce `refStorage` extension","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-29T07:26:22Z","receivedAt":"2023-12-29T07:26:28Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Hi,\n\nthis is the third version of my patch series that introduces the new\n`refStorage` extension. This extension will be used for the upcoming\nreftable backend.\n\nChanges compared to v3:\n\n - The `ref_storage_format` is now tracked as an `unsigned int`,\n proposed by Junio.\n\n - Reworded the commit message in patch 2, proposed by Eric.\n\n - Added a NEEDSWORK comment to `get_worktrees_internal()`, propose by\n Eric.\n\nThanks for your reviews!\n\nPatrick\n\nPatrick Steinhardt (12):\n t: introduce DEFAULT_REPO_FORMAT prereq\n worktree: skip reading HEAD when repairing worktrees\n refs: refactor logic to look up storage backends\n setup: start tracking ref storage format\n setup: set repository's formats on init\n setup: introduce \"extensions.refStorage\" extension\n setup: introduce GIT_DEFAULT_REF_FORMAT envvar\n t: introduce GIT_TEST_DEFAULT_REF_FORMAT envvar\n builtin/rev-parse: introduce `--show-ref-format` flag\n builtin/init: introduce `--ref-format=` value flag\n builtin/clone: introduce `--ref-format=` value flag\n t9500: write \"extensions.refstorage\" into config\n\n Documentation/config/extensions.txt | 11 +++\n Documentation/git-clone.txt | 6 ++\n Documentation/git-init.txt | 7 ++\n Documentation/git-rev-parse.txt | 3 +\n Documentation/git.txt | 5 ++\n Documentation/ref-storage-format.txt | 1 +\n .../technical/repository-version.txt | 5 ++\n builtin/clone.c | 17 ++++-\n builtin/init-db.c | 15 +++-\n builtin/rev-parse.c | 4 ++\n refs.c | 34 ++++++---\n refs.h | 3 +\n refs/debug.c | 1 -\n refs/files-backend.c | 1 -\n refs/packed-backend.c | 1 -\n refs/refs-internal.h | 1 -\n repository.c | 6 ++\n repository.h | 7 ++\n setup.c | 66 +++++++++++++++--\n setup.h | 10 ++-\n t/README | 3 +\n t/t0001-init.sh | 70 +++++++++++++++++++\n t/t1500-rev-parse.sh | 17 +++++\n t/t3200-branch.sh | 2 +-\n t/t5601-clone.sh | 17 +++++\n t/t9500-gitweb-standalone-no-errors.sh | 5 ++\n t/test-lib-functions.sh | 5 ++\n t/test-lib.sh | 15 +++-\n worktree.c | 31 +++++---\n 29 files changed, 334 insertions(+), 35 deletions(-)\n create mode 100644 Documentation/ref-storage-format.txt\n\nRange-diff against v2:\n 1: 3613439cb7 = 1: 578deaabcf t: introduce DEFAULT_REPO_FORMAT prereq\n 2: ecf4f1ddee ! 2: 77c7213c66 worktree: skip reading HEAD when repairing worktrees\n @@ Commit message\n is logic that we resolve their respective worktree HEADs, even though\n that information may not actually be needed in the end by all callers.\n \n - In the context of git-init(1) this is about to become a problem, because\n - we do not have a repository that was set up via `setup_git_directory()`\n - or friends. Consequentially, it is not yet fully initialized at the time\n - of calling `repair_worktrees()`, and properly setting up all parts of\n - the repository in `init_db()` before we repair worktrees is not an easy\n - thing to do. While this is okay right now where we only have a single\n - reference backend in Git, once we gain a second one we would be trying\n - to look up the worktree HEADs before we have figured out the reference\n - format, which does not work.\n + Although not a problem presently with the file-based reference backend,\n + it will become a problem with the upcoming reftable backend. In the\n + context of git-init(1) we do not have a fully-initialized repository set\n + up via `setup_git_directory()` or friends. Consequently, we do not know\n + about the repository format when `repair_worktrees()` is called, and\n + properly setting up all parts of the repositroy in `init_db()` before we\n + try to repair worktrees is not an easy task. With the introduction of\n + the reftable backend, we would ultimately try to look up the worktree\n + HEADs before we have figured out the reference format, which does not\n + work.\n \n We do not require the worktree HEADs at all to repair worktrees. So\n let's fix this issue by skipping over the step that reads them.\n @@ worktree.c: static void mark_current_worktree(struct worktree **worktrees)\n }\n \n -struct worktree **get_worktrees(void)\n ++/*\n ++ * NEEDSWORK: This function exists so that we can look up metadata of a\n ++ * worktree without trying to access any of its internals like the refdb. It\n ++ * would be preferable to instead have a corruption-tolerant function for\n ++ * retrieving worktree metadata that could be used when the worktree is known\n ++ * to not be in a healthy state, e.g. when creating or repairing it.\n ++ */\n +static struct worktree **get_worktrees_internal(int skip_reading_head)\n {\n \tstruct worktree **list = NULL;\n 3: 12329b99b7 ! 3: 47649570bf refs: refactor logic to look up storage backends\n @@ refs.c\n +};\n \n -static struct ref_storage_be *find_ref_storage_backend(const char *name)\n -+static const struct ref_storage_be *find_ref_storage_backend(int ref_storage_format)\n ++static const struct ref_storage_be *find_ref_storage_backend(unsigned int ref_storage_format)\n {\n -\tstruct ref_storage_be *be;\n -\tfor (be = refs_backends; be; be = be->next)\n @@ refs.c\n \treturn NULL;\n }\n \n -+int ref_storage_format_by_name(const char *name)\n ++unsigned int ref_storage_format_by_name(const char *name)\n +{\n -+\tfor (int i = 0; i < ARRAY_SIZE(refs_backends); i++)\n ++\tfor (unsigned int i = 0; i < ARRAY_SIZE(refs_backends); i++)\n +\t\tif (refs_backends[i] && !strcmp(refs_backends[i]->name, name))\n +\t\t\treturn i;\n +\treturn REF_STORAGE_FORMAT_UNKNOWN;\n +}\n +\n -+const char *ref_storage_format_to_name(int ref_storage_format)\n ++const char *ref_storage_format_to_name(unsigned int ref_storage_format)\n +{\n +\tconst struct ref_storage_be *be = find_ref_storage_backend(ref_storage_format);\n +\tif (!be)\n @@ refs.c: static struct ref_store *ref_store_init(struct repository *repo,\n {\n -\tconst char *be_name = \"files\";\n -\tstruct ref_storage_be *be = find_ref_storage_backend(be_name);\n -+\tint format = REF_STORAGE_FORMAT_FILES;\n ++\tunsigned int format = REF_STORAGE_FORMAT_FILES;\n +\tconst struct ref_storage_be *be = find_ref_storage_backend(format);\n \tstruct ref_store *refs;\n \n @@ refs.h: struct string_list;\n struct string_list_item;\n struct worktree;\n \n -+int ref_storage_format_by_name(const char *name);\n -+const char *ref_storage_format_to_name(int ref_storage_format);\n ++unsigned int ref_storage_format_by_name(const char *name);\n ++const char *ref_storage_format_to_name(unsigned int ref_storage_format);\n +\n /*\n * Resolve a reference, recursively following symbolic refererences.\n 4: ddd099fbaf ! 4: 837764d0b5 setup: start tracking ref storage format\n @@ refs.c: static struct ref_store *ref_store_init(struct repository *repo,\n \t\t\t\t\tconst char *gitdir,\n \t\t\t\t\tunsigned int flags)\n {\n --\tint format = REF_STORAGE_FORMAT_FILES;\n +-\tunsigned int format = REF_STORAGE_FORMAT_FILES;\n -\tconst struct ref_storage_be *be = find_ref_storage_backend(format);\n +\tconst struct ref_storage_be *be;\n \tstruct ref_store *refs;\n @@ repository.c: void repo_set_hash_algo(struct repository *repo, int hash_algo)\n \trepo->hash_algo = &hash_algos[hash_algo];\n }\n \n -+void repo_set_ref_storage_format(struct repository *repo, int format)\n ++void repo_set_ref_storage_format(struct repository *repo, unsigned int format)\n +{\n +\trepo->ref_storage_format = format;\n +}\n @@ repository.h: struct repository {\n \tconst struct git_hash_algo *hash_algo;\n \n +\t/* Repository's reference storage format, as serialized on disk. */\n -+\tint ref_storage_format;\n ++\tunsigned int ref_storage_format;\n +\n \t/* A unique-id for tracing purposes. */\n \tint trace2_repo_id;\n @@ repository.h: void repo_set_gitdir(struct repository *repo, const char *root,\n \t\t const struct set_gitdir_args *extra_args);\n void repo_set_worktree(struct repository *repo, const char *path);\n void repo_set_hash_algo(struct repository *repo, int algo);\n -+void repo_set_ref_storage_format(struct repository *repo, int format);\n ++void repo_set_ref_storage_format(struct repository *repo, unsigned int format);\n void initialize_the_repository(void);\n RESULT_MUST_BE_USED\n int repo_init(struct repository *r, const char *gitdir, const char *worktree);\n @@ setup.c: static int is_reinit(void)\n }\n \n -void create_reference_database(const char *initial_branch, int quiet)\n -+void create_reference_database(int ref_storage_format,\n ++void create_reference_database(unsigned int ref_storage_format,\n +\t\t\t const char *initial_branch, int quiet)\n {\n \tstruct strbuf err = STRBUF_INIT;\n @@ setup.c: static void validate_hash_algorithm(struct repository_format *repo_fmt,\n \t}\n }\n \n -+static void validate_ref_storage_format(struct repository_format *repo_fmt, int format)\n ++static void validate_ref_storage_format(struct repository_format *repo_fmt,\n ++\t\t\t\t\tunsigned int format)\n +{\n +\tif (repo_fmt->version >= 0 &&\n +\t format != REF_STORAGE_FORMAT_UNKNOWN &&\n @@ setup.c: static void validate_hash_algorithm(struct repository_format *repo_fmt,\n int init_db(const char *git_dir, const char *real_git_dir,\n -\t const char *template_dir, int hash, const char *initial_branch,\n +\t const char *template_dir, int hash,\n -+\t int ref_storage_format, const char *initial_branch,\n ++\t unsigned int ref_storage_format,\n ++\t const char *initial_branch,\n \t int init_shared_repository, unsigned int flags)\n {\n \tint reinit;\n @@ setup.h: struct repository_format {\n \tint worktree_config;\n \tint is_bare;\n \tint hash_algo;\n -+\tint ref_storage_format;\n ++\tunsigned int ref_storage_format;\n \tint sparse_index;\n \tchar *work_tree;\n \tstruct string_list unknown_extensions;\n @@ setup.h: void check_repository_format(struct repository_format *fmt);\n \n int init_db(const char *git_dir, const char *real_git_dir,\n \t const char *template_dir, int hash_algo,\n -+\t int ref_storage_format,\n ++\t unsigned int ref_storage_format,\n \t const char *initial_branch, int init_shared_repository,\n \t unsigned int flags);\n void initialize_repository_version(int hash_algo, int reinit);\n -void create_reference_database(const char *initial_branch, int quiet);\n -+void create_reference_database(int ref_storage_format,\n ++void create_reference_database(unsigned int ref_storage_format,\n +\t\t\t const char *initial_branch, int quiet);\n \n /*\n 5: 01a1e58a97 = 5: a51da56d9b setup: set repository's formats on init\n 6: 0a586fa648 ! 6: a1e03e4392 setup: introduce \"extensions.refStorage\" extension\n @@ setup.c: static enum extension_result handle_extension(const char *var,\n \t\tdata->hash_algo = format;\n \t\treturn EXTENSION_OK;\n +\t} else if (!strcmp(ext, \"refstorage\")) {\n -+\t\tint format;\n ++\t\tunsigned int format;\n +\n +\t\tif (!value)\n +\t\t\treturn config_error_nonbool(var);\n @@ setup.c: static int needs_work_tree_config(const char *git_dir, const char *work\n }\n \n -void initialize_repository_version(int hash_algo, int reinit)\n -+void initialize_repository_version(int hash_algo, int ref_storage_format,\n ++void initialize_repository_version(int hash_algo,\n ++\t\t\t\t unsigned int ref_storage_format,\n +\t\t\t\t int reinit)\n {\n \tchar repo_version_string[10];\n @@ setup.c: static int create_default_files(const char *template_path,\n \n ## setup.h ##\n @@ setup.h: int init_db(const char *git_dir, const char *real_git_dir,\n - \t int ref_storage_format,\n + \t unsigned int ref_storage_format,\n \t const char *initial_branch, int init_shared_repository,\n \t unsigned int flags);\n -void initialize_repository_version(int hash_algo, int reinit);\n -+void initialize_repository_version(int hash_algo, int ref_storage_format,\n ++void initialize_repository_version(int hash_algo,\n ++\t\t\t\t unsigned int ref_storage_format,\n +\t\t\t\t int reinit);\n - void create_reference_database(int ref_storage_format,\n + void create_reference_database(unsigned int ref_storage_format,\n \t\t\t const char *initial_branch, int quiet);\n \n \n 7: 6d8754f73a ! 7: 5ffc70e9be setup: introduce GIT_DEFAULT_REF_FORMAT envvar\n @@ Documentation/git.txt: double-quotes and respecting backslash escapes. E.g., the\n \n ## setup.c ##\n @@ setup.c: static void validate_hash_algorithm(struct repository_format *repo_fmt, int hash\n - \n - static void validate_ref_storage_format(struct repository_format *repo_fmt, int format)\n + static void validate_ref_storage_format(struct repository_format *repo_fmt,\n + \t\t\t\t\tunsigned int format)\n {\n +\tconst char *name = getenv(\"GIT_DEFAULT_REF_FORMAT\");\n +\n 8: c645932f3d = 8: 13c074acdf t: introduce GIT_TEST_DEFAULT_REF_FORMAT envvar\n 9: 761d647770 = 9: 4ee3c9a2d1 builtin/rev-parse: introduce `--show-ref-format` flag\n10: e382b5bf08 ! 10: 25773e3560 builtin/init: introduce `--ref-format=` value flag\n @@ builtin/init-db.c: int cmd_init_db(int argc, const char **argv, const char *pref\n +\tconst char *ref_format = NULL;\n \tconst char *initial_branch = NULL;\n \tint hash_algo = GIT_HASH_UNKNOWN;\n -+\tint ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN;\n ++\tunsigned int ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN;\n \tint init_shared_repository = -1;\n \tconst struct option init_db_options[] = {\n \t\tOPT_STRING(0, \"template\", &template_dir, N_(\"template-directory\"),\n11: 257233658d ! 11: 3f1cb6b9e5 builtin/clone: introduce `--ref-format=` value flag\n @@ builtin/clone.c: int cmd_clone(int argc, const char **argv, const char *prefix)\n \tint submodule_progress;\n \tint filter_submodules = 0;\n \tint hash_algo;\n -+\tint ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN;\n ++\tunsigned int ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN;\n \tconst int do_not_override_repo_unix_permissions = -1;\n \n \tstruct transport_ls_refs_options transport_ls_refs_options =\n12: b8cd06ec53 = 12: 2e7682b2f3 t9500: write \"extensions.refstorage\" into config\n\nbase-commit: e79552d19784ee7f4bbce278fe25f93fbda196fa\n-- \n2.43.GIT\n\n"},{"id":"486148","messageId":"578deaabcf5fa608a62993dd7586d7eec272c4e2.1703833819.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703833818.git.ps@pks.im","subject":"[PATCH v3 01/12] t: introduce DEFAULT_REPO_FORMAT prereq","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-29T07:26:26Z","receivedAt":"2023-12-29T07:26:30Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"A limited number of tests require repositories to have the default\nrepository format or otherwise they would fail to run, e.g. because they\nfail to detect the correct hash function. While the hash function is the\nonly extension right now that creates problems like this, we are about\nto add a second extension for the ref format.\n\nIntroduce a new DEFAULT_REPO_FORMAT prereq that can easily be amended\nwhenever we add new format extensions. Next to making any such changes\neasier on us, the prerequisite's name should also help to clarify the\nintent better.\n\nSigned-off-by: Patrick Steinhardt \n---\n t/t3200-branch.sh | 2 +-\n t/test-lib.sh | 4 ++++\n 2 files changed, 5 insertions(+), 1 deletion(-)\n\ndiff --git a/t/t3200-branch.sh b/t/t3200-branch.sh\nindex 6a316f081e..de7d3014e4 100755\n--- a/t/t3200-branch.sh\n+++ b/t/t3200-branch.sh\n@@ -519,7 +519,7 @@ EOF\n \n mv .git/config .git/config-saved\n \n-test_expect_success SHA1 'git branch -m q q2 without config should succeed' '\n+test_expect_success DEFAULT_REPO_FORMAT 'git branch -m q q2 without config should succeed' '\n \tgit branch -m q q2 &&\n \tgit branch -m q2 q\n '\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 876b99562a..dc03f06b8e 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1936,6 +1936,10 @@ test_lazy_prereq SHA1 '\n \tesac\n '\n \n+test_lazy_prereq DEFAULT_REPO_FORMAT '\n+\ttest_have_prereq SHA1\n+'\n+\n # Ensure that no test accidentally triggers a Git command\n # that runs the actual maintenance scheduler, affecting a user's\n # system permanently.\n-- \n2.43.GIT\n\n"},{"id":"486149","messageId":"77c7213c66d6b5caf2c24e17578d3e42f7b745a7.1703833819.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703833818.git.ps@pks.im","subject":"[PATCH v3 02/12] worktree: skip reading HEAD when repairing worktrees","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-29T07:26:30Z","receivedAt":"2023-12-29T07:26:33Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"When calling `git init --separate-git-dir=` on a preexisting\nrepository, we move the Git directory of that repository to the new path\nspecified by the user. If there are worktrees present in the repository,\nwe need to repair the worktrees so that their gitlinks point to the new\nlocation of the repository.\n\nThis repair logic will load repositories via `get_worktrees()`, which\nwill enumerate up and initialize all worktrees. Part of initialization\nis logic that we resolve their respective worktree HEADs, even though\nthat information may not actually be needed in the end by all callers.\n\nAlthough not a problem presently with the file-based reference backend,\nit will become a problem with the upcoming reftable backend. In the\ncontext of git-init(1) we do not have a fully-initialized repository set\nup via `setup_git_directory()` or friends. Consequently, we do not know\nabout the repository format when `repair_worktrees()` is called, and\nproperly setting up all parts of the repositroy in `init_db()` before we\ntry to repair worktrees is not an easy task. With the introduction of\nthe reftable backend, we would ultimately try to look up the worktree\nHEADs before we have figured out the reference format, which does not\nwork.\n\nWe do not require the worktree HEADs at all to repair worktrees. So\nlet's fix this issue by skipping over the step that reads them.\n\nSigned-off-by: Patrick Steinhardt \n---\n worktree.c | 31 +++++++++++++++++++++++--------\n 1 file changed, 23 insertions(+), 8 deletions(-)\n\ndiff --git a/worktree.c b/worktree.c\nindex a56a6c2a3d..cc34a3419b 100644\n--- a/worktree.c\n+++ b/worktree.c\n@@ -51,7 +51,7 @@ static void add_head_info(struct worktree *wt)\n /**\n * get the main worktree\n */\n-static struct worktree *get_main_worktree(void)\n+static struct worktree *get_main_worktree(int skip_reading_head)\n {\n \tstruct worktree *worktree = NULL;\n \tstruct strbuf worktree_path = STRBUF_INIT;\n@@ -70,11 +70,13 @@ static struct worktree *get_main_worktree(void)\n \t */\n \tworktree->is_bare = (is_bare_repository_cfg == 1) ||\n \t\tis_bare_repository();\n-\tadd_head_info(worktree);\n+\tif (!skip_reading_head)\n+\t\tadd_head_info(worktree);\n \treturn worktree;\n }\n \n-static struct worktree *get_linked_worktree(const char *id)\n+static struct worktree *get_linked_worktree(const char *id,\n+\t\t\t\t\t int skip_reading_head)\n {\n \tstruct worktree *worktree = NULL;\n \tstruct strbuf path = STRBUF_INIT;\n@@ -93,7 +95,8 @@ static struct worktree *get_linked_worktree(const char *id)\n \tCALLOC_ARRAY(worktree, 1);\n \tworktree->path = strbuf_detach(&worktree_path, NULL);\n \tworktree->id = xstrdup(id);\n-\tadd_head_info(worktree);\n+\tif (!skip_reading_head)\n+\t\tadd_head_info(worktree);\n \n done:\n \tstrbuf_release(&path);\n@@ -118,7 +121,14 @@ static void mark_current_worktree(struct worktree **worktrees)\n \tfree(git_dir);\n }\n \n-struct worktree **get_worktrees(void)\n+/*\n+ * NEEDSWORK: This function exists so that we can look up metadata of a\n+ * worktree without trying to access any of its internals like the refdb. It\n+ * would be preferable to instead have a corruption-tolerant function for\n+ * retrieving worktree metadata that could be used when the worktree is known\n+ * to not be in a healthy state, e.g. when creating or repairing it.\n+ */\n+static struct worktree **get_worktrees_internal(int skip_reading_head)\n {\n \tstruct worktree **list = NULL;\n \tstruct strbuf path = STRBUF_INIT;\n@@ -128,7 +138,7 @@ struct worktree **get_worktrees(void)\n \n \tALLOC_ARRAY(list, alloc);\n \n-\tlist[counter++] = get_main_worktree();\n+\tlist[counter++] = get_main_worktree(skip_reading_head);\n \n \tstrbuf_addf(&path, \"%s/worktrees\", get_git_common_dir());\n \tdir = opendir(path.buf);\n@@ -137,7 +147,7 @@ struct worktree **get_worktrees(void)\n \t\twhile ((d = readdir_skip_dot_and_dotdot(dir)) != NULL) {\n \t\t\tstruct worktree *linked = NULL;\n \n-\t\t\tif ((linked = get_linked_worktree(d->d_name))) {\n+\t\t\tif ((linked = get_linked_worktree(d->d_name, skip_reading_head))) {\n \t\t\t\tALLOC_GROW(list, counter + 1, alloc);\n \t\t\t\tlist[counter++] = linked;\n \t\t\t}\n@@ -151,6 +161,11 @@ struct worktree **get_worktrees(void)\n \treturn list;\n }\n \n+struct worktree **get_worktrees(void)\n+{\n+\treturn get_worktrees_internal(0);\n+}\n+\n const char *get_worktree_git_dir(const struct worktree *wt)\n {\n \tif (!wt)\n@@ -591,7 +606,7 @@ static void repair_noop(int iserr UNUSED,\n \n void repair_worktrees(worktree_repair_fn fn, void *cb_data)\n {\n-\tstruct worktree **worktrees = get_worktrees();\n+\tstruct worktree **worktrees = get_worktrees_internal(1);\n \tstruct worktree **wt = worktrees + 1; /* +1 skips main worktree */\n \n \tif (!fn)\n-- \n2.43.GIT\n\n"},{"id":"486150","messageId":"47649570bffe2d3b9c6bc0322f23c549827bce1d.1703833819.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703833818.git.ps@pks.im","subject":"[PATCH v3 03/12] refs: refactor logic to look up storage backends","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-29T07:26:34Z","receivedAt":"2023-12-29T07:26:38Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"In order to look up ref storage backends, we're currently using a linked\nlist of backends, where each backend is expected to set up its `next`\npointer to the next ref storage backend. This is kind of a weird setup\nas backends need to be aware of other backends without much of a reason.\n\nRefactor the code so that the array of backends is centrally defined in\n\"refs.c\", where each backend is now identified by an integer constant.\nExpose functions to translate from those integer constants to the name\nand vice versa, which will be required by subsequent patches.\n\nSigned-off-by: Patrick Steinhardt \n---\n refs.c | 34 +++++++++++++++++++++++++---------\n refs.h | 3 +++\n refs/debug.c | 1 -\n refs/files-backend.c | 1 -\n refs/packed-backend.c | 1 -\n refs/refs-internal.h | 1 -\n repository.h | 3 +++\n 7 files changed, 31 insertions(+), 13 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 16bfa21df7..dea3d5c9a0 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -33,17 +33,33 @@\n /*\n * List of all available backends\n */\n-static struct ref_storage_be *refs_backends = &refs_be_files;\n+static const struct ref_storage_be *refs_backends[] = {\n+\t[REF_STORAGE_FORMAT_FILES] = &refs_be_files,\n+};\n \n-static struct ref_storage_be *find_ref_storage_backend(const char *name)\n+static const struct ref_storage_be *find_ref_storage_backend(unsigned int ref_storage_format)\n {\n-\tstruct ref_storage_be *be;\n-\tfor (be = refs_backends; be; be = be->next)\n-\t\tif (!strcmp(be->name, name))\n-\t\t\treturn be;\n+\tif (ref_storage_format < ARRAY_SIZE(refs_backends))\n+\t\treturn refs_backends[ref_storage_format];\n \treturn NULL;\n }\n \n+unsigned int ref_storage_format_by_name(const char *name)\n+{\n+\tfor (unsigned int i = 0; i < ARRAY_SIZE(refs_backends); i++)\n+\t\tif (refs_backends[i] && !strcmp(refs_backends[i]->name, name))\n+\t\t\treturn i;\n+\treturn REF_STORAGE_FORMAT_UNKNOWN;\n+}\n+\n+const char *ref_storage_format_to_name(unsigned int ref_storage_format)\n+{\n+\tconst struct ref_storage_be *be = find_ref_storage_backend(ref_storage_format);\n+\tif (!be)\n+\t\treturn \"unknown\";\n+\treturn be->name;\n+}\n+\n /*\n * How to handle various characters in refnames:\n * 0: An acceptable character for refs\n@@ -2029,12 +2045,12 @@ static struct ref_store *ref_store_init(struct repository *repo,\n \t\t\t\t\tconst char *gitdir,\n \t\t\t\t\tunsigned int flags)\n {\n-\tconst char *be_name = \"files\";\n-\tstruct ref_storage_be *be = find_ref_storage_backend(be_name);\n+\tunsigned int format = REF_STORAGE_FORMAT_FILES;\n+\tconst struct ref_storage_be *be = find_ref_storage_backend(format);\n \tstruct ref_store *refs;\n \n \tif (!be)\n-\t\tBUG(\"reference backend %s is unknown\", be_name);\n+\t\tBUG(\"reference backend is unknown\");\n \n \trefs = be->init(repo, gitdir, flags);\n \treturn refs;\ndiff --git a/refs.h b/refs.h\nindex ff113bb12a..11b3b6ccea 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -11,6 +11,9 @@ struct string_list;\n struct string_list_item;\n struct worktree;\n \n+unsigned int ref_storage_format_by_name(const char *name);\n+const char *ref_storage_format_to_name(unsigned int ref_storage_format);\n+\n /*\n * Resolve a reference, recursively following symbolic refererences.\n *\ndiff --git a/refs/debug.c b/refs/debug.c\nindex 83b7a0ba65..b9775f2c37 100644\n--- a/refs/debug.c\n+++ b/refs/debug.c\n@@ -426,7 +426,6 @@ static int debug_reflog_expire(struct ref_store *ref_store, const char *refname,\n }\n \n struct ref_storage_be refs_be_debug = {\n-\t.next = NULL,\n \t.name = \"debug\",\n \t.init = NULL,\n \t.init_db = debug_init_db,\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex ad8b1d143f..43fd0ac760 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3241,7 +3241,6 @@ static int files_init_db(struct ref_store *ref_store, struct strbuf *err UNUSED)\n }\n \n struct ref_storage_be refs_be_files = {\n-\t.next = NULL,\n \t.name = \"files\",\n \t.init = files_ref_store_create,\n \t.init_db = files_init_db,\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex b9fa097a29..8d1090e284 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1705,7 +1705,6 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n }\n \n struct ref_storage_be refs_be_packed = {\n-\t.next = NULL,\n \t.name = \"packed\",\n \t.init = packed_ref_store_create,\n \t.init_db = packed_init_db,\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 4af83bf9a5..8e9f04cc67 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -663,7 +663,6 @@ typedef int read_symbolic_ref_fn(struct ref_store *ref_store, const char *refnam\n \t\t\t\t struct strbuf *referent);\n \n struct ref_storage_be {\n-\tstruct ref_storage_be *next;\n \tconst char *name;\n \tref_store_init_fn *init;\n \tref_init_db_fn *init_db;\ndiff --git a/repository.h b/repository.h\nindex 5f18486f64..ea4c488b81 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -24,6 +24,9 @@ enum fetch_negotiation_setting {\n \tFETCH_NEGOTIATION_NOOP,\n };\n \n+#define REF_STORAGE_FORMAT_UNKNOWN 0\n+#define REF_STORAGE_FORMAT_FILES 1\n+\n struct repo_settings {\n \tint initialized;\n \n-- \n2.43.GIT\n\n"},{"id":"486151","messageId":"837764d0b5ff1236f2a8ab263e7eb622be46f1a7.1703833819.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703833818.git.ps@pks.im","subject":"[PATCH v3 04/12] setup: start tracking ref storage format","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-29T07:26:39Z","receivedAt":"2023-12-29T07:26:42Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"In order to discern which ref storage format a repository is supposed to\nuse we need to start setting up and/or discovering the format. This\nneeds to happen in two separate code paths.\n\n - The first path is when we create a repository via `init_db()`. When\n we are re-initializing a preexisting repository we need to retain\n the previously used ref storage format -- if the user asked for a\n different format then this indicates an error and we error out.\n Otherwise we either initialize the repository with the format asked\n for by the user or the default format, which currently is the\n \"files\" backend.\n\n - The second path is when discovering repositories, where we need to\n read the config of that repository. There is not yet any way to\n configure something other than the \"files\" backend, so we can just\n blindly set the ref storage format to this backend.\n\nWire up this logic so that we have the ref storage format always readily\navailable when needed. As there is only a single backend and because it\nis not configurable we cannot yet verify that this tracking works as\nexpected via tests, but tests will be added in subsequent commits. To\ncountermand this ommission now though, raise a BUG() in case the ref\nstorage format is not set up properly in `ref_store_init()`.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/clone.c | 5 +++--\n builtin/init-db.c | 4 +++-\n refs.c | 4 ++--\n repository.c | 6 ++++++\n repository.h | 4 ++++\n setup.c | 28 +++++++++++++++++++++++++---\n setup.h | 6 +++++-\n 7 files changed, 48 insertions(+), 9 deletions(-)\n\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex 343f536cf8..48aeb1b90b 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -1107,7 +1107,8 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \t * repository, and reference backends may persist that information into\n \t * their on-disk data structures.\n \t */\n-\tinit_db(git_dir, real_git_dir, option_template, GIT_HASH_UNKNOWN, NULL,\n+\tinit_db(git_dir, real_git_dir, option_template, GIT_HASH_UNKNOWN,\n+\t\tREF_STORAGE_FORMAT_UNKNOWN, NULL,\n \t\tdo_not_override_repo_unix_permissions, INIT_DB_QUIET | INIT_DB_SKIP_REFDB);\n \n \tif (real_git_dir) {\n@@ -1292,7 +1293,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \thash_algo = hash_algo_by_ptr(transport_get_hash_algo(transport));\n \tinitialize_repository_version(hash_algo, 1);\n \trepo_set_hash_algo(the_repository, hash_algo);\n-\tcreate_reference_database(NULL, 1);\n+\tcreate_reference_database(the_repository->ref_storage_format, NULL, 1);\n \n \t/*\n \t * Before fetching from the remote, download and install bundle\ndiff --git a/builtin/init-db.c b/builtin/init-db.c\nindex cb727c826f..b6e80feab6 100644\n--- a/builtin/init-db.c\n+++ b/builtin/init-db.c\n@@ -11,6 +11,7 @@\n #include \"object-file.h\"\n #include \"parse-options.h\"\n #include \"path.h\"\n+#include \"refs.h\"\n #include \"setup.h\"\n #include \"strbuf.h\"\n \n@@ -236,5 +237,6 @@ int cmd_init_db(int argc, const char **argv, const char *prefix)\n \n \tflags |= INIT_DB_EXIST_OK;\n \treturn init_db(git_dir, real_git_dir, template_dir, hash_algo,\n-\t\t initial_branch, init_shared_repository, flags);\n+\t\t REF_STORAGE_FORMAT_UNKNOWN, initial_branch,\n+\t\t init_shared_repository, flags);\n }\ndiff --git a/refs.c b/refs.c\nindex dea3d5c9a0..fdbf5f4cb1 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -2045,10 +2045,10 @@ static struct ref_store *ref_store_init(struct repository *repo,\n \t\t\t\t\tconst char *gitdir,\n \t\t\t\t\tunsigned int flags)\n {\n-\tunsigned int format = REF_STORAGE_FORMAT_FILES;\n-\tconst struct ref_storage_be *be = find_ref_storage_backend(format);\n+\tconst struct ref_storage_be *be;\n \tstruct ref_store *refs;\n \n+\tbe = find_ref_storage_backend(repo->ref_storage_format);\n \tif (!be)\n \t\tBUG(\"reference backend is unknown\");\n \ndiff --git a/repository.c b/repository.c\nindex a7679ceeaa..d7d24d416a 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -104,6 +104,11 @@ void repo_set_hash_algo(struct repository *repo, int hash_algo)\n \trepo->hash_algo = &hash_algos[hash_algo];\n }\n \n+void repo_set_ref_storage_format(struct repository *repo, unsigned int format)\n+{\n+\trepo->ref_storage_format = format;\n+}\n+\n /*\n * Attempt to resolve and set the provided 'gitdir' for repository 'repo'.\n * Return 0 upon success and a non-zero value upon failure.\n@@ -184,6 +189,7 @@ int repo_init(struct repository *repo,\n \t\tgoto error;\n \n \trepo_set_hash_algo(repo, format.hash_algo);\n+\trepo_set_ref_storage_format(repo, format.ref_storage_format);\n \trepo->repository_format_worktree_config = format.worktree_config;\n \n \t/* take ownership of format.partial_clone */\ndiff --git a/repository.h b/repository.h\nindex ea4c488b81..f5269b3730 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -163,6 +163,9 @@ struct repository {\n \t/* Repository's current hash algorithm, as serialized on disk. */\n \tconst struct git_hash_algo *hash_algo;\n \n+\t/* Repository's reference storage format, as serialized on disk. */\n+\tunsigned int ref_storage_format;\n+\n \t/* A unique-id for tracing purposes. */\n \tint trace2_repo_id;\n \n@@ -202,6 +205,7 @@ void repo_set_gitdir(struct repository *repo, const char *root,\n \t\t const struct set_gitdir_args *extra_args);\n void repo_set_worktree(struct repository *repo, const char *path);\n void repo_set_hash_algo(struct repository *repo, int algo);\n+void repo_set_ref_storage_format(struct repository *repo, unsigned int format);\n void initialize_the_repository(void);\n RESULT_MUST_BE_USED\n int repo_init(struct repository *r, const char *gitdir, const char *worktree);\ndiff --git a/setup.c b/setup.c\nindex bc90bbd033..9c9a167f52 100644\n--- a/setup.c\n+++ b/setup.c\n@@ -1566,6 +1566,8 @@ const char *setup_git_directory_gently(int *nongit_ok)\n \t\t}\n \t\tif (startup_info->have_repository) {\n \t\t\trepo_set_hash_algo(the_repository, repo_fmt.hash_algo);\n+\t\t\trepo_set_ref_storage_format(the_repository,\n+\t\t\t\t\t\t repo_fmt.ref_storage_format);\n \t\t\tthe_repository->repository_format_worktree_config =\n \t\t\t\trepo_fmt.worktree_config;\n \t\t\t/* take ownership of repo_fmt.partial_clone */\n@@ -1659,6 +1661,8 @@ void check_repository_format(struct repository_format *fmt)\n \tcheck_repository_format_gently(get_git_dir(), fmt, NULL);\n \tstartup_info->have_repository = 1;\n \trepo_set_hash_algo(the_repository, fmt->hash_algo);\n+\trepo_set_ref_storage_format(the_repository,\n+\t\t\t\t fmt->ref_storage_format);\n \tthe_repository->repository_format_worktree_config =\n \t\tfmt->worktree_config;\n \tthe_repository->repository_format_partial_clone =\n@@ -1899,7 +1903,8 @@ static int is_reinit(void)\n \treturn ret;\n }\n \n-void create_reference_database(const char *initial_branch, int quiet)\n+void create_reference_database(unsigned int ref_storage_format,\n+\t\t\t const char *initial_branch, int quiet)\n {\n \tstruct strbuf err = STRBUF_INIT;\n \tint reinit = is_reinit();\n@@ -1919,6 +1924,7 @@ void create_reference_database(const char *initial_branch, int quiet)\n \tsafe_create_dir(git_path(\"refs\"), 1);\n \tadjust_shared_perm(git_path(\"refs\"));\n \n+\trepo_set_ref_storage_format(the_repository, ref_storage_format);\n \tif (refs_init_db(&err))\n \t\tdie(\"failed to set up refs db: %s\", err.buf);\n \n@@ -2137,8 +2143,22 @@ static void validate_hash_algorithm(struct repository_format *repo_fmt, int hash\n \t}\n }\n \n+static void validate_ref_storage_format(struct repository_format *repo_fmt,\n+\t\t\t\t\tunsigned int format)\n+{\n+\tif (repo_fmt->version >= 0 &&\n+\t format != REF_STORAGE_FORMAT_UNKNOWN &&\n+\t format != repo_fmt->ref_storage_format) {\n+\t\tdie(_(\"attempt to reinitialize repository with different reference storage format\"));\n+\t} else if (format != REF_STORAGE_FORMAT_UNKNOWN) {\n+\t\trepo_fmt->ref_storage_format = format;\n+\t}\n+}\n+\n int init_db(const char *git_dir, const char *real_git_dir,\n-\t const char *template_dir, int hash, const char *initial_branch,\n+\t const char *template_dir, int hash,\n+\t unsigned int ref_storage_format,\n+\t const char *initial_branch,\n \t int init_shared_repository, unsigned int flags)\n {\n \tint reinit;\n@@ -2181,13 +2201,15 @@ int init_db(const char *git_dir, const char *real_git_dir,\n \tcheck_repository_format(&repo_fmt);\n \n \tvalidate_hash_algorithm(&repo_fmt, hash);\n+\tvalidate_ref_storage_format(&repo_fmt, ref_storage_format);\n \n \treinit = create_default_files(template_dir, original_git_dir,\n \t\t\t\t &repo_fmt, prev_bare_repository,\n \t\t\t\t init_shared_repository);\n \n \tif (!(flags & INIT_DB_SKIP_REFDB))\n-\t\tcreate_reference_database(initial_branch, flags & INIT_DB_QUIET);\n+\t\tcreate_reference_database(repo_fmt.ref_storage_format,\n+\t\t\t\t\t initial_branch, flags & INIT_DB_QUIET);\n \tcreate_object_directory();\n \n \tif (get_shared_repository()) {\ndiff --git a/setup.h b/setup.h\nindex 3f0f17c351..3d3eda7967 100644\n--- a/setup.h\n+++ b/setup.h\n@@ -115,6 +115,7 @@ struct repository_format {\n \tint worktree_config;\n \tint is_bare;\n \tint hash_algo;\n+\tunsigned int ref_storage_format;\n \tint sparse_index;\n \tchar *work_tree;\n \tstruct string_list unknown_extensions;\n@@ -131,6 +132,7 @@ struct repository_format {\n \t.version = -1, \\\n \t.is_bare = -1, \\\n \t.hash_algo = GIT_HASH_SHA1, \\\n+\t.ref_storage_format = REF_STORAGE_FORMAT_FILES, \\\n \t.unknown_extensions = STRING_LIST_INIT_DUP, \\\n \t.v1_only_extensions = STRING_LIST_INIT_DUP, \\\n }\n@@ -175,10 +177,12 @@ void check_repository_format(struct repository_format *fmt);\n \n int init_db(const char *git_dir, const char *real_git_dir,\n \t const char *template_dir, int hash_algo,\n+\t unsigned int ref_storage_format,\n \t const char *initial_branch, int init_shared_repository,\n \t unsigned int flags);\n void initialize_repository_version(int hash_algo, int reinit);\n-void create_reference_database(const char *initial_branch, int quiet);\n+void create_reference_database(unsigned int ref_storage_format,\n+\t\t\t const char *initial_branch, int quiet);\n \n /*\n * NOTE NOTE NOTE!!\n-- \n2.43.GIT\n\n"},{"id":"486152","messageId":"a51da56d9b843078b3c8d013902c38c5b65728f5.1703833819.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703833818.git.ps@pks.im","subject":"[PATCH v3 05/12] setup: set repository's formats on init","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-29T07:26:43Z","receivedAt":"2023-12-29T07:26:47Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The proper hash algorithm and ref storage format that will be used for a\nnewly initialized repository will be figured out in `init_db()` via\n`validate_hash_algorithm()` and `validate_ref_storage_format()`. Until\nnow though, we never set up the hash algorithm or ref storage format of\n`the_repository` accordingly.\n\nThere are only two callsites of `init_db()`, one in git-init(1) and one\nin git-clone(1). The former function doesn't care for the formats to be\nset up properly because it never access the repository after calling the\nfunction in the first place.\n\nFor git-clone(1) it's a different story though, as we call `init_db()`\nbefore listing remote refs. While we do indeed have the wrong hash\nfunction in `the_repository` when `init_db()` sets up a non-default\nobject format for the repository, it never mattered because we adjust\nthe hash after learning about the remote's hash function via the listed\nrefs.\n\nSo the current state is correct for the hash algo, but it's not for the\nref storage format because git-clone(1) wouldn't know to set it up\nproperly. But instead of adjusting only the `ref_storage_format`, set\nboth the hash algo and the ref storage format so that `the_repository`\nis in the correct state when `init_db()` exits. This is fine as we will\nadjust the hash later on anyway and makes it easier to reason about the\nend state of `the_repository`.\n\nSigned-off-by: Patrick Steinhardt \n---\n setup.c | 7 +++++++\n 1 file changed, 7 insertions(+)\n\ndiff --git a/setup.c b/setup.c\nindex 9c9a167f52..49570e6b3a 100644\n--- a/setup.c\n+++ b/setup.c\n@@ -2207,6 +2207,13 @@ int init_db(const char *git_dir, const char *real_git_dir,\n \t\t\t\t &repo_fmt, prev_bare_repository,\n \t\t\t\t init_shared_repository);\n \n+\t/*\n+\t * Now that we have set up both the hash algorithm and the ref storage\n+\t * format we can update the repository's settings accordingly.\n+\t */\n+\trepo_set_hash_algo(the_repository, repo_fmt.hash_algo);\n+\trepo_set_ref_storage_format(the_repository, repo_fmt.ref_storage_format);\n+\n \tif (!(flags & INIT_DB_SKIP_REFDB))\n \t\tcreate_reference_database(repo_fmt.ref_storage_format,\n \t\t\t\t\t initial_branch, flags & INIT_DB_QUIET);\n-- \n2.43.GIT\n\n"},{"id":"486153","messageId":"a1e03e4392d77113f5a8b82f68956b71d37cdd26.1703833819.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703833818.git.ps@pks.im","subject":"[PATCH v3 06/12] setup: introduce \"extensions.refStorage\" extension","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-29T07:26:47Z","receivedAt":"2023-12-29T07:26:51Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Introduce a new \"extensions.refStorage\" extension that allows us to\nspecify the ref storage format used by a repository. For now, the only\nsupported format is the \"files\" format, but this list will likely soon\nbe extended to also support the upcoming \"reftable\" format.\n\nThere have been discussions on the Git mailing list in the past around\nhow exactly this extension should look like. One alternative [1] that\nwas discussed was whether it would make sense to model the extension in\nsuch a way that backends are arbitrarily stackable. This would allow for\na combined value of e.g. \"loose,packed-refs\" or \"loose,reftable\", which\nindicates that new refs would be written via \"loose\" files backend and\ncompressed into \"packed-refs\" or \"reftable\" backends, respectively.\n\nIt is arguable though whether this flexibility and the complexity that\nit brings with it is really required for now. It is not foreseeable that\nthere will be a proliferation of backends in the near-term future, and\nthe current set of existing formats and formats which are on the horizon\ncan easily be configured with the much simpler proposal where we have a\nsingle value, only.\n\nFurthermore, if we ever see that we indeed want to gain the ability to\narbitrarily stack the ref formats, then we can adapt the current\nextension rather easily. Given that Git clients will refuse any unknown\nvalue for the \"extensions.refStorage\" extension they would also know to\nignore a stacked \"loose,packed-refs\" in the future.\n\nSo let's stick with the easy proposal for the time being and wire up the\nextension.\n\n[1]: \n\nSigned-off-by: Patrick Steinhardt \n---\n Documentation/config/extensions.txt | 11 ++++++++\n Documentation/ref-storage-format.txt | 1 +\n .../technical/repository-version.txt | 5 ++++\n builtin/clone.c | 2 +-\n setup.c | 24 ++++++++++++++---\n setup.h | 4 ++-\n t/t0001-init.sh | 26 +++++++++++++++++++\n t/test-lib.sh | 2 +-\n 8 files changed, 69 insertions(+), 6 deletions(-)\n create mode 100644 Documentation/ref-storage-format.txt\n\ndiff --git a/Documentation/config/extensions.txt b/Documentation/config/extensions.txt\nindex bccaec7a96..66db0e15da 100644\n--- a/Documentation/config/extensions.txt\n+++ b/Documentation/config/extensions.txt\n@@ -7,6 +7,17 @@ Note that this setting should only be set by linkgit:git-init[1] or\n linkgit:git-clone[1]. Trying to change it after initialization will not\n work and will produce hard-to-diagnose issues.\n \n+extensions.refStorage::\n+\tSpecify the ref storage format to use. The acceptable values are:\n++\n+include::../ref-storage-format.txt[]\n++\n+It is an error to specify this key unless `core.repositoryFormatVersion` is 1.\n++\n+Note that this setting should only be set by linkgit:git-init[1] or\n+linkgit:git-clone[1]. Trying to change it after initialization will not\n+work and will produce hard-to-diagnose issues.\n+\n extensions.worktreeConfig::\n \tIf enabled, then worktrees will load config settings from the\n \t`$GIT_DIR/config.worktree` file in addition to the\ndiff --git a/Documentation/ref-storage-format.txt b/Documentation/ref-storage-format.txt\nnew file mode 100644\nindex 0000000000..1a65cac468\n--- /dev/null\n+++ b/Documentation/ref-storage-format.txt\n@@ -0,0 +1 @@\n+* `files` for loose files with packed-refs. This is the default.\ndiff --git a/Documentation/technical/repository-version.txt b/Documentation/technical/repository-version.txt\nindex 045a76756f..27be3741e6 100644\n--- a/Documentation/technical/repository-version.txt\n+++ b/Documentation/technical/repository-version.txt\n@@ -100,3 +100,8 @@ If set, by default \"git config\" reads from both \"config\" and\n multiple working directory mode, \"config\" file is shared while\n \"config.worktree\" is per-working directory (i.e., it's in\n GIT_COMMON_DIR/worktrees//config.worktree)\n+\n+==== `refStorage`\n+\n+Specifies the file format for the ref database. The only valid value\n+is `files` (loose references with a packed-refs file).\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex 48aeb1b90b..0fb3816d0c 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -1291,7 +1291,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \t * ours to the same thing.\n \t */\n \thash_algo = hash_algo_by_ptr(transport_get_hash_algo(transport));\n-\tinitialize_repository_version(hash_algo, 1);\n+\tinitialize_repository_version(hash_algo, the_repository->ref_storage_format, 1);\n \trepo_set_hash_algo(the_repository, hash_algo);\n \tcreate_reference_database(the_repository->ref_storage_format, NULL, 1);\n \ndiff --git a/setup.c b/setup.c\nindex 49570e6b3a..fb1413cabd 100644\n--- a/setup.c\n+++ b/setup.c\n@@ -592,6 +592,17 @@ static enum extension_result handle_extension(const char *var,\n \t\t\t\t \"extensions.objectformat\", value);\n \t\tdata->hash_algo = format;\n \t\treturn EXTENSION_OK;\n+\t} else if (!strcmp(ext, \"refstorage\")) {\n+\t\tunsigned int format;\n+\n+\t\tif (!value)\n+\t\t\treturn config_error_nonbool(var);\n+\t\tformat = ref_storage_format_by_name(value);\n+\t\tif (format == REF_STORAGE_FORMAT_UNKNOWN)\n+\t\t\treturn error(_(\"invalid value for '%s': '%s'\"),\n+\t\t\t\t \"extensions.refstorage\", value);\n+\t\tdata->ref_storage_format = format;\n+\t\treturn EXTENSION_OK;\n \t}\n \treturn EXTENSION_UNKNOWN;\n }\n@@ -1871,12 +1882,15 @@ static int needs_work_tree_config(const char *git_dir, const char *work_tree)\n \treturn 1;\n }\n \n-void initialize_repository_version(int hash_algo, int reinit)\n+void initialize_repository_version(int hash_algo,\n+\t\t\t\t unsigned int ref_storage_format,\n+\t\t\t\t int reinit)\n {\n \tchar repo_version_string[10];\n \tint repo_version = GIT_REPO_VERSION;\n \n-\tif (hash_algo != GIT_HASH_SHA1)\n+\tif (hash_algo != GIT_HASH_SHA1 ||\n+\t ref_storage_format != REF_STORAGE_FORMAT_FILES)\n \t\trepo_version = GIT_REPO_VERSION_READ;\n \n \t/* This forces creation of new config file */\n@@ -1889,6 +1903,10 @@ void initialize_repository_version(int hash_algo, int reinit)\n \t\t\t hash_algos[hash_algo].name);\n \telse if (reinit)\n \t\tgit_config_set_gently(\"extensions.objectformat\", NULL);\n+\n+\tif (ref_storage_format != REF_STORAGE_FORMAT_FILES)\n+\t\tgit_config_set(\"extensions.refstorage\",\n+\t\t\t ref_storage_format_to_name(ref_storage_format));\n }\n \n static int is_reinit(void)\n@@ -2030,7 +2048,7 @@ static int create_default_files(const char *template_path,\n \t\tadjust_shared_perm(get_git_dir());\n \t}\n \n-\tinitialize_repository_version(fmt->hash_algo, 0);\n+\tinitialize_repository_version(fmt->hash_algo, fmt->ref_storage_format, 0);\n \n \t/* Check filemode trustability */\n \tpath = git_path_buf(&buf, \"config\");\ndiff --git a/setup.h b/setup.h\nindex 3d3eda7967..3599aec93c 100644\n--- a/setup.h\n+++ b/setup.h\n@@ -180,7 +180,9 @@ int init_db(const char *git_dir, const char *real_git_dir,\n \t unsigned int ref_storage_format,\n \t const char *initial_branch, int init_shared_repository,\n \t unsigned int flags);\n-void initialize_repository_version(int hash_algo, int reinit);\n+void initialize_repository_version(int hash_algo,\n+\t\t\t\t unsigned int ref_storage_format,\n+\t\t\t\t int reinit);\n void create_reference_database(unsigned int ref_storage_format,\n \t\t\t const char *initial_branch, int quiet);\n \ndiff --git a/t/t0001-init.sh b/t/t0001-init.sh\nindex 2b78e3be47..38b3e4c39e 100755\n--- a/t/t0001-init.sh\n+++ b/t/t0001-init.sh\n@@ -532,6 +532,32 @@ test_expect_success 'init rejects attempts to initialize with different hash' '\n \ttest_must_fail git -C sha256 init --object-format=sha1\n '\n \n+test_expect_success DEFAULT_REPO_FORMAT 'extensions.refStorage is not allowed with repo version 0' '\n+\ttest_when_finished \"rm -rf refstorage\" &&\n+\tgit init refstorage &&\n+\tgit -C refstorage config extensions.refStorage files &&\n+\ttest_must_fail git -C refstorage rev-parse 2>err &&\n+\tgrep \"repo version is 0, but v1-only extension found\" err\n+'\n+\n+test_expect_success DEFAULT_REPO_FORMAT 'extensions.refStorage with files backend' '\n+\ttest_when_finished \"rm -rf refstorage\" &&\n+\tgit init refstorage &&\n+\tgit -C refstorage config core.repositoryformatversion 1 &&\n+\tgit -C refstorage config extensions.refStorage files &&\n+\ttest_commit -C refstorage A &&\n+\tgit -C refstorage rev-parse --verify HEAD\n+'\n+\n+test_expect_success DEFAULT_REPO_FORMAT 'extensions.refStorage with unknown backend' '\n+\ttest_when_finished \"rm -rf refstorage\" &&\n+\tgit init refstorage &&\n+\tgit -C refstorage config core.repositoryformatversion 1 &&\n+\tgit -C refstorage config extensions.refStorage garbage &&\n+\ttest_must_fail git -C refstorage rev-parse 2>err &&\n+\tgrep \"invalid value for ${SQ}extensions.refstorage${SQ}: ${SQ}garbage${SQ}\" err\n+'\n+\n test_expect_success MINGW 'core.hidedotfiles = false' '\n \tgit config --global core.hidedotfiles false &&\n \trm -rf newdir &&\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex dc03f06b8e..4685cc3d48 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1937,7 +1937,7 @@ test_lazy_prereq SHA1 '\n '\n \n test_lazy_prereq DEFAULT_REPO_FORMAT '\n-\ttest_have_prereq SHA1\n+\ttest_have_prereq SHA1,REFFILES\n '\n \n # Ensure that no test accidentally triggers a Git command\n-- \n2.43.GIT\n\n"},{"id":"486154","messageId":"5ffc70e9be3cd3a4d6300dc0497c5cc0f12c0967.1703833819.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703833818.git.ps@pks.im","subject":"[PATCH v3 07/12] setup: introduce GIT_DEFAULT_REF_FORMAT envvar","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-29T07:26:52Z","receivedAt":"2023-12-29T07:26:55Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Introduce a new GIT_DEFAULT_REF_FORMAT environment variable that lets\nusers control the default ref format used by both git-init(1) and\ngit-clone(1). This is modeled after GIT_DEFAULT_OBJECT_FORMAT, which\ndoes the same thing for the repository's object format.\n\nSigned-off-by: Patrick Steinhardt \n---\n Documentation/git.txt | 5 +++++\n setup.c | 7 +++++++\n t/t0001-init.sh | 18 ++++++++++++++++++\n 3 files changed, 30 insertions(+)\n\ndiff --git a/Documentation/git.txt b/Documentation/git.txt\nindex bf9e6af695..88e4ed4bd6 100644\n--- a/Documentation/git.txt\n+++ b/Documentation/git.txt\n@@ -556,6 +556,11 @@ double-quotes and respecting backslash escapes. E.g., the value\n \tis always used. The default is \"sha1\".\n \tSee `--object-format` in linkgit:git-init[1].\n \n+`GIT_DEFAULT_REF_FORMAT`::\n+\tIf this variable is set, the default reference backend format for new\n+\trepositories will be set to this value. The default is \"files\".\n+\tSee `--ref-format` in linkgit:git-init[1].\n+\n Git Commits\n ~~~~~~~~~~~\n `GIT_AUTHOR_NAME`::\ndiff --git a/setup.c b/setup.c\nindex fb1413cabd..1ab1a66bcb 100644\n--- a/setup.c\n+++ b/setup.c\n@@ -2164,12 +2164,19 @@ static void validate_hash_algorithm(struct repository_format *repo_fmt, int hash\n static void validate_ref_storage_format(struct repository_format *repo_fmt,\n \t\t\t\t\tunsigned int format)\n {\n+\tconst char *name = getenv(\"GIT_DEFAULT_REF_FORMAT\");\n+\n \tif (repo_fmt->version >= 0 &&\n \t format != REF_STORAGE_FORMAT_UNKNOWN &&\n \t format != repo_fmt->ref_storage_format) {\n \t\tdie(_(\"attempt to reinitialize repository with different reference storage format\"));\n \t} else if (format != REF_STORAGE_FORMAT_UNKNOWN) {\n \t\trepo_fmt->ref_storage_format = format;\n+\t} else if (name) {\n+\t\tformat = ref_storage_format_by_name(name);\n+\t\tif (format == REF_STORAGE_FORMAT_UNKNOWN)\n+\t\t\tdie(_(\"unknown ref storage format '%s'\"), name);\n+\t\trepo_fmt->ref_storage_format = format;\n \t}\n }\n \ndiff --git a/t/t0001-init.sh b/t/t0001-init.sh\nindex 38b3e4c39e..30ce752cc1 100755\n--- a/t/t0001-init.sh\n+++ b/t/t0001-init.sh\n@@ -558,6 +558,24 @@ test_expect_success DEFAULT_REPO_FORMAT 'extensions.refStorage with unknown back\n \tgrep \"invalid value for ${SQ}extensions.refstorage${SQ}: ${SQ}garbage${SQ}\" err\n '\n \n+test_expect_success DEFAULT_REPO_FORMAT 'init with GIT_DEFAULT_REF_FORMAT=files' '\n+\ttest_when_finished \"rm -rf refformat\" &&\n+\tGIT_DEFAULT_REF_FORMAT=files git init refformat &&\n+\techo 0 >expect &&\n+\tgit -C refformat config core.repositoryformatversion >actual &&\n+\ttest_cmp expect actual &&\n+\ttest_must_fail git -C refformat config extensions.refstorage\n+'\n+\n+test_expect_success 'init with GIT_DEFAULT_REF_FORMAT=garbage' '\n+\ttest_when_finished \"rm -rf refformat\" &&\n+\tcat >expect <<-EOF &&\n+\tfatal: unknown ref storage format ${SQ}garbage${SQ}\n+\tEOF\n+\ttest_must_fail env GIT_DEFAULT_REF_FORMAT=garbage git init refformat 2>err &&\n+\ttest_cmp expect err\n+'\n+\n test_expect_success MINGW 'core.hidedotfiles = false' '\n \tgit config --global core.hidedotfiles false &&\n \trm -rf newdir &&\n-- \n2.43.GIT\n\n"},{"id":"486155","messageId":"13c074acdfbb4699d99b5d20a1620153b4a5c87d.1703833819.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703833818.git.ps@pks.im","subject":"[PATCH v3 08/12] t: introduce GIT_TEST_DEFAULT_REF_FORMAT envvar","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-29T07:26:56Z","receivedAt":"2023-12-29T07:26:59Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Introduce a new GIT_TEST_DEFAULT_REF_FORMAT environment variable that\nlets developers run the test suite with a different default ref format\nwithout impacting the ref format used by non-test Git invocations. This\nis modeled after GIT_TEST_DEFAULT_OBJECT_FORMAT, which does the same\nthing for the repository's object format.\n\nAdapt the setup of the `REFFILES` test prerequisite to be conditionally\nset based on the default ref format.\n\nSigned-off-by: Patrick Steinhardt \n---\n t/README | 3 +++\n t/test-lib-functions.sh | 5 +++++\n t/test-lib.sh | 11 ++++++++++-\n 3 files changed, 18 insertions(+), 1 deletion(-)\n\ndiff --git a/t/README b/t/README\nindex 36463d0742..621d3b8c09 100644\n--- a/t/README\n+++ b/t/README\n@@ -479,6 +479,9 @@ GIT_TEST_DEFAULT_HASH= specifies which hash algorithm to\n use in the test scripts. Recognized values for are \"sha1\"\n and \"sha256\".\n \n+GIT_TEST_DEFAULT_REF_FORMAT= specifies which ref storage format\n+to use in the test scripts. Recognized values for are \"files\".\n+\n GIT_TEST_NO_WRITE_REV_INDEX=, when true disables the\n 'pack.writeReverseIndex' setting.\n \ndiff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh\nindex 5eb57914ab..a3a51ea9e8 100644\n--- a/t/test-lib-functions.sh\n+++ b/t/test-lib-functions.sh\n@@ -1659,6 +1659,11 @@ test_detect_hash () {\n \ttest_hash_algo=\"${GIT_TEST_DEFAULT_HASH:-sha1}\"\n }\n \n+# Detect the hash algorithm in use.\n+test_detect_ref_format () {\n+\techo \"${GIT_TEST_DEFAULT_REF_FORMAT:-files}\"\n+}\n+\n # Load common hash metadata and common placeholder object IDs for use with\n # test_oid.\n test_oid_init () {\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 4685cc3d48..fc93aa57e6 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -542,6 +542,8 @@ export EDITOR\n \n GIT_DEFAULT_HASH=\"${GIT_TEST_DEFAULT_HASH:-sha1}\"\n export GIT_DEFAULT_HASH\n+GIT_DEFAULT_REF_FORMAT=\"${GIT_TEST_DEFAULT_REF_FORMAT:-files}\"\n+export GIT_DEFAULT_REF_FORMAT\n GIT_TEST_MERGE_ALGORITHM=\"${GIT_TEST_MERGE_ALGORITHM:-ort}\"\n export GIT_TEST_MERGE_ALGORITHM\n \n@@ -1745,7 +1747,14 @@ parisc* | hppa*)\n \t;;\n esac\n \n-test_set_prereq REFFILES\n+case \"$GIT_DEFAULT_REF_FORMAT\" in\n+files)\n+\ttest_set_prereq REFFILES;;\n+*)\n+\techo 2>&1 \"error: unknown ref format $GIT_DEFAULT_REF_FORMAT\"\n+\texit 1\n+\t;;\n+esac\n \n ( COLUMNS=1 && test $COLUMNS = 1 ) && test_set_prereq COLUMNS_CAN_BE_1\n test -z \"$NO_CURL\" && test_set_prereq LIBCURL\n-- \n2.43.GIT\n\n"},{"id":"486156","messageId":"4ee3c9a2d169e5a8fa657531919f9b2833f2a1ac.1703833819.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703833818.git.ps@pks.im","subject":"[PATCH v3 09/12] builtin/rev-parse: introduce `--show-ref-format` flag","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-29T07:27:00Z","receivedAt":"2023-12-29T07:27:04Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Introduce a new `--show-ref-format` to git-rev-parse(1) that causes it\nto print the ref format used by a repository.\n\nSigned-off-by: Patrick Steinhardt \n---\n Documentation/git-rev-parse.txt | 3 +++\n builtin/rev-parse.c | 4 ++++\n t/t1500-rev-parse.sh | 17 +++++++++++++++++\n 3 files changed, 24 insertions(+)\n\ndiff --git a/Documentation/git-rev-parse.txt b/Documentation/git-rev-parse.txt\nindex 912fab9f5e..546faf9017 100644\n--- a/Documentation/git-rev-parse.txt\n+++ b/Documentation/git-rev-parse.txt\n@@ -307,6 +307,9 @@ The following options are unaffected by `--path-format`:\n \tinput, multiple algorithms may be printed, space-separated.\n \tIf not specified, the default is \"storage\".\n \n+--show-ref-format::\n+\tShow the reference storage format used for the repository.\n+\n \n Other Options\n ~~~~~~~~~~~~~\ndiff --git a/builtin/rev-parse.c b/builtin/rev-parse.c\nindex 917f122440..d08987646a 100644\n--- a/builtin/rev-parse.c\n+++ b/builtin/rev-parse.c\n@@ -1062,6 +1062,10 @@ int cmd_rev_parse(int argc, const char **argv, const char *prefix)\n \t\t\t\tputs(the_hash_algo->name);\n \t\t\t\tcontinue;\n \t\t\t}\n+\t\t\tif (!strcmp(arg, \"--show-ref-format\")) {\n+\t\t\t\tputs(ref_storage_format_to_name(the_repository->ref_storage_format));\n+\t\t\t\tcontinue;\n+\t\t\t}\n \t\t\tif (!strcmp(arg, \"--end-of-options\")) {\n \t\t\t\tseen_end_of_options = 1;\n \t\t\t\tif (filter & (DO_FLAGS | DO_REVS))\ndiff --git a/t/t1500-rev-parse.sh b/t/t1500-rev-parse.sh\nindex 3f9e7f62e4..a669e592f1 100755\n--- a/t/t1500-rev-parse.sh\n+++ b/t/t1500-rev-parse.sh\n@@ -208,6 +208,23 @@ test_expect_success 'rev-parse --show-object-format in repo' '\n \tgrep \"unknown mode for --show-object-format: squeamish-ossifrage\" err\n '\n \n+test_expect_success 'rev-parse --show-ref-format' '\n+\ttest_detect_ref_format >expect &&\n+\tgit rev-parse --show-ref-format >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'rev-parse --show-ref-format with invalid storage' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\tgit config extensions.refstorage broken &&\n+\t\ttest_must_fail git rev-parse --show-ref-format 2>err &&\n+\t\tgrep \"error: invalid value for ${SQ}extensions.refstorage${SQ}: ${SQ}broken${SQ}\" err\n+\t)\n+'\n+\n test_expect_success '--show-toplevel from subdir of working tree' '\n \tpwd >expect &&\n \tgit -C sub/dir rev-parse --show-toplevel >actual &&\n-- \n2.43.GIT\n\n"},{"id":"486157","messageId":"25773e356063e5c109c238105c148b1112cc9672.1703833819.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703833818.git.ps@pks.im","subject":"[PATCH v3 10/12] builtin/init: introduce `--ref-format=` value flag","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-29T07:27:04Z","receivedAt":"2023-12-29T07:27:08Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Introduce a new `--ref-format` value flag for git-init(1) that allows\nthe user to specify the ref format that is to be used for a newly\ninitialized repository.\n\nSigned-off-by: Patrick Steinhardt \n---\n Documentation/git-init.txt | 7 +++++++\n builtin/init-db.c | 13 ++++++++++++-\n t/t0001-init.sh | 26 ++++++++++++++++++++++++++\n 3 files changed, 45 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-init.txt b/Documentation/git-init.txt\nindex 6f0d2973bf..e8dc645bb5 100644\n--- a/Documentation/git-init.txt\n+++ b/Documentation/git-init.txt\n@@ -11,6 +11,7 @@ SYNOPSIS\n [verse]\n 'git init' [-q | --quiet] [--bare] [--template=]\n \t [--separate-git-dir ] [--object-format=]\n+\t [--ref-format=]\n \t [-b | --initial-branch=]\n \t [--shared[=]] []\n \n@@ -57,6 +58,12 @@ values are 'sha1' and (if enabled) 'sha256'. 'sha1' is the default.\n +\n include::object-format-disclaimer.txt[]\n \n+--ref-format=::\n+\n+Specify the given ref storage format for the repository. The valid values are:\n++\n+include::ref-storage-format.txt[]\n+\n --template=::\n \n Specify the directory from which templates will be used. (See the \"TEMPLATE\ndiff --git a/builtin/init-db.c b/builtin/init-db.c\nindex b6e80feab6..a4f81e2af5 100644\n--- a/builtin/init-db.c\n+++ b/builtin/init-db.c\n@@ -58,6 +58,7 @@ static int shared_callback(const struct option *opt, const char *arg, int unset)\n static const char *const init_db_usage[] = {\n \tN_(\"git init [-q | --quiet] [--bare] [--template=]\\n\"\n \t \" [--separate-git-dir ] [--object-format=]\\n\"\n+\t \" [--ref-format=]\\n\"\n \t \" [-b | --initial-branch=]\\n\"\n \t \" [--shared[=]] []\"),\n \tNULL\n@@ -77,8 +78,10 @@ int cmd_init_db(int argc, const char **argv, const char *prefix)\n \tconst char *template_dir = NULL;\n \tunsigned int flags = 0;\n \tconst char *object_format = NULL;\n+\tconst char *ref_format = NULL;\n \tconst char *initial_branch = NULL;\n \tint hash_algo = GIT_HASH_UNKNOWN;\n+\tunsigned int ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN;\n \tint init_shared_repository = -1;\n \tconst struct option init_db_options[] = {\n \t\tOPT_STRING(0, \"template\", &template_dir, N_(\"template-directory\"),\n@@ -96,6 +99,8 @@ int cmd_init_db(int argc, const char **argv, const char *prefix)\n \t\t\t N_(\"override the name of the initial branch\")),\n \t\tOPT_STRING(0, \"object-format\", &object_format, N_(\"hash\"),\n \t\t\t N_(\"specify the hash algorithm to use\")),\n+\t\tOPT_STRING(0, \"ref-format\", &ref_format, N_(\"format\"),\n+\t\t\t N_(\"specify the reference format to use\")),\n \t\tOPT_END()\n \t};\n \n@@ -159,6 +164,12 @@ int cmd_init_db(int argc, const char **argv, const char *prefix)\n \t\t\tdie(_(\"unknown hash algorithm '%s'\"), object_format);\n \t}\n \n+\tif (ref_format) {\n+\t\tref_storage_format = ref_storage_format_by_name(ref_format);\n+\t\tif (ref_storage_format == REF_STORAGE_FORMAT_UNKNOWN)\n+\t\t\tdie(_(\"unknown ref storage format '%s'\"), ref_format);\n+\t}\n+\n \tif (init_shared_repository != -1)\n \t\tset_shared_repository(init_shared_repository);\n \n@@ -237,6 +248,6 @@ int cmd_init_db(int argc, const char **argv, const char *prefix)\n \n \tflags |= INIT_DB_EXIST_OK;\n \treturn init_db(git_dir, real_git_dir, template_dir, hash_algo,\n-\t\t REF_STORAGE_FORMAT_UNKNOWN, initial_branch,\n+\t\t ref_storage_format, initial_branch,\n \t\t init_shared_repository, flags);\n }\ndiff --git a/t/t0001-init.sh b/t/t0001-init.sh\nindex 30ce752cc1..b131d665db 100755\n--- a/t/t0001-init.sh\n+++ b/t/t0001-init.sh\n@@ -576,6 +576,32 @@ test_expect_success 'init with GIT_DEFAULT_REF_FORMAT=garbage' '\n \ttest_cmp expect err\n '\n \n+test_expect_success 'init with --ref-format=files' '\n+\ttest_when_finished \"rm -rf refformat\" &&\n+\tgit init --ref-format=files refformat &&\n+\techo files >expect &&\n+\tgit -C refformat rev-parse --show-ref-format >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 're-init with same format' '\n+\ttest_when_finished \"rm -rf refformat\" &&\n+\tgit init --ref-format=files refformat &&\n+\tgit init --ref-format=files refformat &&\n+\techo files >expect &&\n+\tgit -C refformat rev-parse --show-ref-format >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'init with --ref-format=garbage' '\n+\ttest_when_finished \"rm -rf refformat\" &&\n+\tcat >expect <<-EOF &&\n+\tfatal: unknown ref storage format ${SQ}garbage${SQ}\n+\tEOF\n+\ttest_must_fail git init --ref-format=garbage refformat 2>err &&\n+\ttest_cmp expect err\n+'\n+\n test_expect_success MINGW 'core.hidedotfiles = false' '\n \tgit config --global core.hidedotfiles false &&\n \trm -rf newdir &&\n-- \n2.43.GIT\n\n"},{"id":"486158","messageId":"3f1cb6b9e57f4e164d2070913dad2cd3dfc1f3f1.1703833819.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703833818.git.ps@pks.im","subject":"[PATCH v3 11/12] builtin/clone: introduce `--ref-format=` value flag","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-29T07:27:09Z","receivedAt":"2023-12-29T07:27:12Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Introduce a new `--ref-format` value flag for git-clone(1) that allows\nthe user to specify the ref format that is to be used for a newly\ninitialized repository.\n\nSigned-off-by: Patrick Steinhardt \n---\n Documentation/git-clone.txt | 6 ++++++\n builtin/clone.c | 12 +++++++++++-\n t/t5601-clone.sh | 17 +++++++++++++++++\n 3 files changed, 34 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt\nindex c37c4a37f7..6e43eb9c20 100644\n--- a/Documentation/git-clone.txt\n+++ b/Documentation/git-clone.txt\n@@ -311,6 +311,12 @@ or `--mirror` is given)\n \tThe result is Git repository can be separated from working\n \ttree.\n \n+--ref-format=::\n --jobs ::\n \tThe number of submodules fetched at the same time.\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex 0fb3816d0c..f1635e0e8c 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -72,6 +72,7 @@ static char *remote_name = NULL;\n static char *option_branch = NULL;\n static struct string_list option_not = STRING_LIST_INIT_NODUP;\n static const char *real_git_dir;\n+static const char *ref_format;\n static char *option_upload_pack = \"git-upload-pack\";\n static int option_verbosity;\n static int option_progress = -1;\n@@ -157,6 +158,8 @@ static struct option builtin_clone_options[] = {\n \t\t N_(\"any cloned submodules will be shallow\")),\n \tOPT_STRING(0, \"separate-git-dir\", &real_git_dir, N_(\"gitdir\"),\n \t\t N_(\"separate git dir from working tree\")),\n+\tOPT_STRING(0, \"ref-format\", &ref_format, N_(\"format\"),\n+\t\t N_(\"specify the reference format to use\")),\n \tOPT_STRING_LIST('c', \"config\", &option_config, N_(\"key=value\"),\n \t\t\tN_(\"set config inside the new repository\")),\n \tOPT_STRING_LIST(0, \"server-option\", &server_options,\n@@ -932,6 +935,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \tint submodule_progress;\n \tint filter_submodules = 0;\n \tint hash_algo;\n+\tunsigned int ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN;\n \tconst int do_not_override_repo_unix_permissions = -1;\n \n \tstruct transport_ls_refs_options transport_ls_refs_options =\n@@ -957,6 +961,12 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \tif (option_single_branch == -1)\n \t\toption_single_branch = deepen ? 1 : 0;\n \n+\tif (ref_format) {\n+\t\tref_storage_format = ref_storage_format_by_name(ref_format);\n+\t\tif (ref_storage_format == REF_STORAGE_FORMAT_UNKNOWN)\n+\t\t\tdie(_(\"unknown ref storage format '%s'\"), ref_format);\n+\t}\n+\n \tif (option_mirror)\n \t\toption_bare = 1;\n \n@@ -1108,7 +1118,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \t * their on-disk data structures.\n \t */\n \tinit_db(git_dir, real_git_dir, option_template, GIT_HASH_UNKNOWN,\n-\t\tREF_STORAGE_FORMAT_UNKNOWN, NULL,\n+\t\tref_storage_format, NULL,\n \t\tdo_not_override_repo_unix_permissions, INIT_DB_QUIET | INIT_DB_SKIP_REFDB);\n \n \tif (real_git_dir) {\ndiff --git a/t/t5601-clone.sh b/t/t5601-clone.sh\nindex 47eae641f0..fb1b9c686d 100755\n--- a/t/t5601-clone.sh\n+++ b/t/t5601-clone.sh\n@@ -157,6 +157,23 @@ test_expect_success 'clone --mirror does not repeat tags' '\n \n '\n \n+test_expect_success 'clone with files ref format' '\n+\ttest_when_finished \"rm -rf ref-storage\" &&\n+\tgit clone --ref-format=files --mirror src ref-storage &&\n+\techo files >expect &&\n+\tgit -C ref-storage rev-parse --show-ref-format >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'clone with garbage ref format' '\n+\tcat >expect <<-EOF &&\n+\tfatal: unknown ref storage format ${SQ}garbage${SQ}\n+\tEOF\n+\ttest_must_fail git clone --ref-format=garbage --mirror src ref-storage 2>err &&\n+\ttest_cmp expect err &&\n+\ttest_path_is_missing ref-storage\n+'\n+\n test_expect_success 'clone to destination with trailing /' '\n \n \tgit clone src target-1/ &&\n-- \n2.43.GIT\n\n"},{"id":"486159","messageId":"2e7682b2f3b066b7e122b2ce3fcd32df0604a427.1703833819.git.ps@pks.im","threadId":"60639","inReplyTo":"cover.1703833818.git.ps@pks.im","subject":"[PATCH v3 12/12] t9500: write \"extensions.refstorage\" into config","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2023-12-29T07:27:13Z","receivedAt":"2023-12-29T07:27:17Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"In t9500 we're writing a custom configuration that sets up gitweb. This\nrequires us to manually ensure that the repository format is configured\nas required, including both the repository format version and\nextensions. With the introduction of the \"extensions.refStorage\"\nextension we need to update the test to also write this new one.\n\nSigned-off-by: Patrick Steinhardt \n---\n t/t9500-gitweb-standalone-no-errors.sh | 5 +++++\n 1 file changed, 5 insertions(+)\n\ndiff --git a/t/t9500-gitweb-standalone-no-errors.sh b/t/t9500-gitweb-standalone-no-errors.sh\nindex 0333065d4d..7679780fb8 100755\n--- a/t/t9500-gitweb-standalone-no-errors.sh\n+++ b/t/t9500-gitweb-standalone-no-errors.sh\n@@ -627,6 +627,7 @@ test_expect_success \\\n test_expect_success 'setup' '\n \tversion=$(git config core.repositoryformatversion) &&\n \talgo=$(test_might_fail git config extensions.objectformat) &&\n+\trefstorage=$(test_might_fail git config extensions.refstorage) &&\n \tcat >.git/config <<-\\EOF &&\n \t# testing noval and alternate separator\n \t[gitweb]\n@@ -637,6 +638,10 @@ test_expect_success 'setup' '\n \tif test -n \"$algo\"\n \tthen\n \t\tgit config extensions.objectformat \"$algo\"\n+\tfi &&\n+\tif test -n \"$refstorage\"\n+\tthen\n+\t\tgit config extensions.refstorage \"$refstorage\"\n \tfi\n '\n \n-- \n2.43.GIT\n\n"},{"id":"486160","messageId":"CAPig+cRmgkEQwWdN_7BT6taJ_2uKKaX0AO_43K_q6+mo9jtuEg@mail.gmail.com","threadId":"60639","inReplyTo":"cover.1703833818.git.ps@pks.im","subject":"Re: [PATCH v3 00/12] Introduce `refStorage` extension","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2023-12-29T08:44:02Z","receivedAt":"2023-12-29T08:44:14Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Fri, Dec 29, 2023 at 2:26 AM Patrick Steinhardt wrote:\n> this is the third version of my patch series that introduces the new\n> `refStorage` extension. This extension will be used for the upcoming\n> reftable backend.\n>\n> Changes compared to v3:\n> - Reworded the commit message in patch 2, proposed by Eric.\n> - Added a NEEDSWORK comment to `get_worktrees_internal()`, propose by\n> Eric.\n\nThanks for addressing my minor review comments.\n"}]}