[PATCH v3 00/12] Introduce `refStorage` extension
- From
Patrick Steinhardt <ps@pks.im>
- Date
- Dec 29, 2023, 07:26 UTC
- Message-ID
- <cover.1703833818.git.ps@pks.im>
- In-Reply-To
- <cover.1703067989.git.ps@pks.im>
Hi,
this is the third version of my patch series that introduces the new `refStorage` extension. This extension will be used for the upcoming reftable backend.
Changes compared to v3:
- The `ref_storage_format` is now tracked as an `unsigned int`,
proposed by Junio.- Reworded the commit message in patch 2, proposed by Eric.
- Added a NEEDSWORK comment to `get_worktrees_internal()`, propose by
Eric.Thanks for your reviews!
Patrick
Patrick Steinhardt (12): t: introduce DEFAULT_REPO_FORMAT prereq worktree: skip reading HEAD when repairing worktrees refs: refactor logic to look up storage backends setup: start tracking ref storage format setup: set repository's formats on init setup: introduce "extensions.refStorage" extension setup: introduce GIT_DEFAULT_REF_FORMAT envvar t: introduce GIT_TEST_DEFAULT_REF_FORMAT envvar builtin/rev-parse: introduce `--show-ref-format` flag builtin/init: introduce `--ref-format=` value flag builtin/clone: introduce `--ref-format=` value flag t9500: write "extensions.refstorage" into config
Documentation/config/extensions.txt | 11 +++ Documentation/git-clone.txt | 6 ++ Documentation/git-init.txt | 7 ++ Documentation/git-rev-parse.txt | 3 + Documentation/git.txt | 5 ++ Documentation/ref-storage-format.txt | 1 + .../technical/repository-version.txt | 5 ++ builtin/clone.c | 17 ++++- builtin/init-db.c | 15 +++- builtin/rev-parse.c | 4 ++ refs.c | 34 ++++++--- refs.h | 3 + refs/debug.c | 1 - refs/files-backend.c | 1 - refs/packed-backend.c | 1 - refs/refs-internal.h | 1 - repository.c | 6 ++ repository.h | 7 ++ setup.c | 66 +++++++++++++++-- setup.h | 10 ++- t/README | 3 + t/t0001-init.sh | 70 +++++++++++++++++++ t/t1500-rev-parse.sh | 17 +++++ t/t3200-branch.sh | 2 +- t/t5601-clone.sh | 17 +++++ t/t9500-gitweb-standalone-no-errors.sh | 5 ++ t/test-lib-functions.sh | 5 ++ t/test-lib.sh | 15 +++- worktree.c | 31 +++++--- 29 files changed, 334 insertions(+), 35 deletions(-) create mode 100644 Documentation/ref-storage-format.txt
Range-diff against v2:
1: 3613439cb7 = 1: 578deaabcf t: introduce DEFAULT_REPO_FORMAT prereq
2: ecf4f1ddee ! 2: 77c7213c66 worktree: skip reading HEAD when repairing worktrees
@@ Commit message
is logic that we resolve their respective worktree HEADs, even though
that information may not actually be needed in the end by all callers.
- In the context of git-init(1) this is about to become a problem, because
- we do not have a repository that was set up via `setup_git_directory()`
- or friends. Consequentially, it is not yet fully initialized at the time
- of calling `repair_worktrees()`, and properly setting up all parts of
- the repository in `init_db()` before we repair worktrees is not an easy
- thing to do. While this is okay right now where we only have a single
- reference backend in Git, once we gain a second one we would be trying
- to look up the worktree HEADs before we have figured out the reference
- format, which does not work.
+ Although not a problem presently with the file-based reference backend,
+ it will become a problem with the upcoming reftable backend. In the
+ context of git-init(1) we do not have a fully-initialized repository set
+ up via `setup_git_directory()` or friends. Consequently, we do not know
+ about the repository format when `repair_worktrees()` is called, and
+ properly setting up all parts of the repositroy in `init_db()` before we
+ try to repair worktrees is not an easy task. With the introduction of
+ the reftable backend, we would ultimately try to look up the worktree
+ HEADs before we have figured out the reference format, which does not
+ work.
We do not require the worktree HEADs at all to repair worktrees. So
let's fix this issue by skipping over the step that reads them.
@@ worktree.c: static void mark_current_worktree(struct worktree **worktrees)
}
-struct worktree **get_worktrees(void)
++/*
++ * NEEDSWORK: This function exists so that we can look up metadata of a
++ * worktree without trying to access any of its internals like the refdb. It
++ * would be preferable to instead have a corruption-tolerant function for
++ * retrieving worktree metadata that could be used when the worktree is known
++ * to not be in a healthy state, e.g. when creating or repairing it.
++ */
+static struct worktree **get_worktrees_internal(int skip_reading_head)
{
struct worktree **list = NULL;
3: 12329b99b7 ! 3: 47649570bf refs: refactor logic to look up storage backends
@@ refs.c
+};
-static struct ref_storage_be *find_ref_storage_backend(const char *name)
-+static const struct ref_storage_be *find_ref_storage_backend(int ref_storage_format)
++static const struct ref_storage_be *find_ref_storage_backend(unsigned int ref_storage_format)
{
- struct ref_storage_be *be;
- for (be = refs_backends; be; be = be->next)
@@ refs.c
return NULL;
}
-+int ref_storage_format_by_name(const char *name)
++unsigned int ref_storage_format_by_name(const char *name)
+{
-+ for (int i = 0; i < ARRAY_SIZE(refs_backends); i++)
++ for (unsigned int i = 0; i < ARRAY_SIZE(refs_backends); i++)
+ if (refs_backends[i] && !strcmp(refs_backends[i]->name, name))
+ return i;
+ return REF_STORAGE_FORMAT_UNKNOWN;
+}
+
-+const char *ref_storage_format_to_name(int ref_storage_format)
++const char *ref_storage_format_to_name(unsigned int ref_storage_format)
+{
+ const struct ref_storage_be *be = find_ref_storage_backend(ref_storage_format);
+ if (!be)
@@ refs.c: static struct ref_store *ref_store_init(struct repository *repo,
{
- const char *be_name = "files";
- struct ref_storage_be *be = find_ref_storage_backend(be_name);
-+ int format = REF_STORAGE_FORMAT_FILES;
++ unsigned int format = REF_STORAGE_FORMAT_FILES;
+ const struct ref_storage_be *be = find_ref_storage_backend(format);
struct ref_store *refs;
@@ refs.h: struct string_list;
struct string_list_item;
struct worktree;
-+int ref_storage_format_by_name(const char *name);
-+const char *ref_storage_format_to_name(int ref_storage_format);
++unsigned int ref_storage_format_by_name(const char *name);
++const char *ref_storage_format_to_name(unsigned int ref_storage_format);
+
/*
* Resolve a reference, recursively following symbolic refererences.
4: ddd099fbaf ! 4: 837764d0b5 setup: start tracking ref storage format
@@ refs.c: static struct ref_store *ref_store_init(struct repository *repo,
const char *gitdir,
unsigned int flags)
{
-- int format = REF_STORAGE_FORMAT_FILES;
+- unsigned int format = REF_STORAGE_FORMAT_FILES;
- const struct ref_storage_be *be = find_ref_storage_backend(format);
+ const struct ref_storage_be *be;
struct ref_store *refs;
@@ repository.c: void repo_set_hash_algo(struct repository *repo, int hash_algo)
repo->hash_algo = &hash_algos[hash_algo];
}
-+void repo_set_ref_storage_format(struct repository *repo, int format)
++void repo_set_ref_storage_format(struct repository *repo, unsigned int format)
+{
+ repo->ref_storage_format = format;
+}
@@ repository.h: struct repository {
const struct git_hash_algo *hash_algo;
+ /* Repository's reference storage format, as serialized on disk. */
-+ int ref_storage_format;
++ unsigned int ref_storage_format;
+
/* A unique-id for tracing purposes. */
int trace2_repo_id;
@@ repository.h: void repo_set_gitdir(struct repository *repo, const char *root,
const struct set_gitdir_args *extra_args);
void repo_set_worktree(struct repository *repo, const char *path);
void repo_set_hash_algo(struct repository *repo, int algo);
-+void repo_set_ref_storage_format(struct repository *repo, int format);
++void repo_set_ref_storage_format(struct repository *repo, unsigned int format);
void initialize_the_repository(void);
RESULT_MUST_BE_USED
int repo_init(struct repository *r, const char *gitdir, const char *worktree);
@@ setup.c: static int is_reinit(void)
}
-void create_reference_database(const char *initial_branch, int quiet)
-+void create_reference_database(int ref_storage_format,
++void create_reference_database(unsigned int ref_storage_format,
+ const char *initial_branch, int quiet)
{
struct strbuf err = STRBUF_INIT;
@@ setup.c: static void validate_hash_algorithm(struct repository_format *repo_fmt,
}
}
-+static void validate_ref_storage_format(struct repository_format *repo_fmt, int format)
++static void validate_ref_storage_format(struct repository_format *repo_fmt,
++ unsigned int format)
+{
+ if (repo_fmt->version >= 0 &&
+ format != REF_STORAGE_FORMAT_UNKNOWN &&
@@ setup.c: static void validate_hash_algorithm(struct repository_format *repo_fmt,
int init_db(const char *git_dir, const char *real_git_dir,
- const char *template_dir, int hash, const char *initial_branch,
+ const char *template_dir, int hash,
-+ int ref_storage_format, const char *initial_branch,
++ unsigned int ref_storage_format,
++ const char *initial_branch,
int init_shared_repository, unsigned int flags)
{
int reinit;
@@ setup.h: struct repository_format {
int worktree_config;
int is_bare;
int hash_algo;
-+ int ref_storage_format;
++ unsigned int ref_storage_format;
int sparse_index;
char *work_tree;
struct string_list unknown_extensions;
@@ setup.h: void check_repository_format(struct repository_format *fmt);
int init_db(const char *git_dir, const char *real_git_dir,
const char *template_dir, int hash_algo,
-+ int ref_storage_format,
++ unsigned int ref_storage_format,
const char *initial_branch, int init_shared_repository,
unsigned int flags);
void initialize_repository_version(int hash_algo, int reinit);
-void create_reference_database(const char *initial_branch, int quiet);
-+void create_reference_database(int ref_storage_format,
++void create_reference_database(unsigned int ref_storage_format,
+ const char *initial_branch, int quiet);
/*
5: 01a1e58a97 = 5: a51da56d9b setup: set repository's formats on init
6: 0a586fa648 ! 6: a1e03e4392 setup: introduce "extensions.refStorage" extension
@@ setup.c: static enum extension_result handle_extension(const char *var,
data->hash_algo = format;
return EXTENSION_OK;
+ } else if (!strcmp(ext, "refstorage")) {
-+ int format;
++ unsigned int format;
+
+ if (!value)
+ return config_error_nonbool(var);
@@ setup.c: static int needs_work_tree_config(const char *git_dir, const char *work
}
-void initialize_repository_version(int hash_algo, int reinit)
-+void initialize_repository_version(int hash_algo, int ref_storage_format,
++void initialize_repository_version(int hash_algo,
++ unsigned int ref_storage_format,
+ int reinit)
{
char repo_version_string[10];
@@ setup.c: static int create_default_files(const char *template_path,
## setup.h ##
@@ setup.h: int init_db(const char *git_dir, const char *real_git_dir,
- int ref_storage_format,
+ unsigned int ref_storage_format,
const char *initial_branch, int init_shared_repository,
unsigned int flags);
-void initialize_repository_version(int hash_algo, int reinit);
-+void initialize_repository_version(int hash_algo, int ref_storage_format,
++void initialize_repository_version(int hash_algo,
++ unsigned int ref_storage_format,
+ int reinit);
- void create_reference_database(int ref_storage_format,
+ void create_reference_database(unsigned int ref_storage_format,
const char *initial_branch, int quiet);
7: 6d8754f73a ! 7: 5ffc70e9be setup: introduce GIT_DEFAULT_REF_FORMAT envvar
@@ Documentation/git.txt: double-quotes and respecting backslash escapes. E.g., the
## setup.c ##
@@ setup.c: static void validate_hash_algorithm(struct repository_format *repo_fmt, int hash
-
- static void validate_ref_storage_format(struct repository_format *repo_fmt, int format)
+ static void validate_ref_storage_format(struct repository_format *repo_fmt,
+ unsigned int format)
{
+ const char *name = getenv("GIT_DEFAULT_REF_FORMAT");
+
8: c645932f3d = 8: 13c074acdf t: introduce GIT_TEST_DEFAULT_REF_FORMAT envvar
9: 761d647770 = 9: 4ee3c9a2d1 builtin/rev-parse: introduce `--show-ref-format` flag
10: e382b5bf08 ! 10: 25773e3560 builtin/init: introduce `--ref-format=` value flag
@@ builtin/init-db.c: int cmd_init_db(int argc, const char **argv, const char *pref
+ const char *ref_format = NULL;
const char *initial_branch = NULL;
int hash_algo = GIT_HASH_UNKNOWN;
-+ int ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN;
++ unsigned int ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN;
int init_shared_repository = -1;
const struct option init_db_options[] = {
OPT_STRING(0, "template", &template_dir, N_("template-directory"),
11: 257233658d ! 11: 3f1cb6b9e5 builtin/clone: introduce `--ref-format=` value flag
@@ builtin/clone.c: int cmd_clone(int argc, const char **argv, const char *prefix)
int submodule_progress;
int filter_submodules = 0;
int hash_algo;
-+ int ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN;
++ unsigned int ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN;
const int do_not_override_repo_unix_permissions = -1;
struct transport_ls_refs_options transport_ls_refs_options =
12: b8cd06ec53 = 12: 2e7682b2f3 t9500: write "extensions.refstorage" into configbase-commit: e79552d19784ee7f4bbce278fe25f93fbda196fa
-- 2.43.GIT