{"thread":{"id":"57386","subject":"[PATCH] refs: RFC: Reftable support for git-core","startedAt":"2022-02-08T22:25:35Z","lastAt":"2022-02-08T22:25:35Z","messageCount":1,"participants":["Han-Wen Nienhuys via GitGitGadget"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"448006","messageId":"pull.1215.git.git.1644351400761.gitgitgadget@gmail.com","threadId":"57386","inReplyTo":null,"subject":"[PATCH] refs: RFC: Reftable support for git-core","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-02-08T20:16:40Z","receivedAt":"2022-02-08T22:25:35Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nFor background, see Documentation/technical/reftable.txt.\n\nThis introduces the file refs/reftable-backend.c containing a\nreftable-powered ref storage backend.\n\nIt can be activated by setting GIT_TEST_REFTABLE in the environment.\nWhen GIT_TEST_REFTABLE is set, the test prerequisite !REFFILES is set.\n\nThere is no option to git-init for now, as the test suite still shows\nfailures with GIT_TEST_REFTABLE=1.\n\nExample use: see t/t0031-reftable.sh\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Johannes Schindelin \nHelped-by: Johannes Schindelin \nHelped-by: Junio Hamano \nHelped-by: Patrick Steinhardt \nCo-authored-by: Jeff King \n---\n RFC: Reftable backend current version\n \n In\n https://lore.kernel.org/git/220203.867dab6dmp.gmgdl@evledraar.gmail.com/\n Ævar considered the option of merging the reftable without 100% of tests\n passing. In my development branch, I am down to 26 test failures.\n \n As input to this discussion, I'm proffering the latest version of the\n reftable support.\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1215%2Fhanwen%2Freftable-rfc-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1215/hanwen/reftable-rfc-v1\nPull-Request: https://github.com/git/git/pull/1215\n\n Documentation/config/extensions.txt | 9 +\n .../technical/repository-version.txt | 7 +\n Makefile | 1 +\n builtin/clone.c | 5 +-\n builtin/init-db.c | 41 +-\n builtin/stash.c | 8 +-\n builtin/worktree.c | 27 +-\n cache.h | 8 +-\n config.mak.uname | 2 +-\n contrib/workdir/git-new-workdir | 2 +-\n refs.c | 29 +-\n refs.h | 3 +\n refs/refs-internal.h | 1 +\n refs/reftable-backend.c | 1898 +++++++++++++++++\n repository.c | 2 +\n repository.h | 3 +\n setup.c | 8 +\n t/t0031-reftable.sh | 380 ++++\n t/t1409-avoid-packing-refs.sh | 6 +\n t/t1450-fsck.sh | 6 +\n t/t3210-pack-refs.sh | 6 +\n t/test-lib.sh | 7 +-\n 22 files changed, 2429 insertions(+), 30 deletions(-)\n create mode 100644 refs/reftable-backend.c\n create mode 100755 t/t0031-reftable.sh\n\ndiff --git a/Documentation/config/extensions.txt b/Documentation/config/extensions.txt\nindex 5999dcb2a1f..9ea3e99f2d0 100644\n--- a/Documentation/config/extensions.txt\n+++ b/Documentation/config/extensions.txt\n@@ -37,3 +37,12 @@ details.\n +\n For historical reasons, `extensions.worktreeConfig` is respected\n regardless of the `core.repositoryFormatVersion` setting.\n++\n+extensions.refStorage::\n+\tSpecify the ref storage mechanism to use. The acceptable values are `files` and\n+\t`reftable`. If not specified, `files` is assumed. It is an error to specify\n+\tthis key unless `core.repositoryFormatVersion` is 1.\n++\n+Note that this setting should only be set by linkgit:git-init[1] or\n+linkgit:git-clone[1]. Trying to change it after initialization will not\n+work and will produce hard-to-diagnose issues.\ndiff --git a/Documentation/technical/repository-version.txt b/Documentation/technical/repository-version.txt\nindex 7844ef30ffd..72576235833 100644\n--- a/Documentation/technical/repository-version.txt\n+++ b/Documentation/technical/repository-version.txt\n@@ -100,3 +100,10 @@ If set, by default \"git config\" reads from both \"config\" and\n multiple working directory mode, \"config\" file is shared while\n \"config.worktree\" is per-working directory (i.e., it's in\n GIT_COMMON_DIR/worktrees//config.worktree)\n+\n+==== `refStorage`\n+\n+Specifies the file format for the ref database. Values are `files`\n+(for the traditional packed + loose ref format) and `reftable` for the\n+binary reftable format. See https://github.com/google/reftable for\n+more information.\ndiff --git a/Makefile b/Makefile\nindex daa21bed6c3..550e08782a5 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1000,6 +1000,7 @@ LIB_OBJS += reflog-walk.o\n LIB_OBJS += refs.o\n LIB_OBJS += refs/debug.o\n LIB_OBJS += refs/files-backend.o\n+LIB_OBJS += refs/reftable-backend.o\n LIB_OBJS += refs/iterator.o\n LIB_OBJS += refs/packed-backend.o\n LIB_OBJS += refs/ref-cache.o\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex f81f085ee87..d48580a7367 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -1043,7 +1043,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \t}\n \n \tinit_db(git_dir, real_git_dir, option_template, GIT_HASH_UNKNOWN, NULL,\n-\t\tINIT_DB_QUIET);\n+\t\tdefault_ref_storage(), INIT_DB_QUIET);\n \n \tif (real_git_dir) {\n \t\tfree((char *)git_dir);\n@@ -1199,7 +1199,8 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \t\t * Now that we know what algorithm the remote side is using,\n \t\t * let's set ours to the same thing.\n \t\t */\n-\t\tinitialize_repository_version(hash_algo, 1);\n+\t\tinitialize_repository_version(hash_algo, 1,\n+\t\t\t\t\t default_ref_storage());\n \t\trepo_set_hash_algo(the_repository, hash_algo);\n \t\t/*\n \t\t * transport_get_remote_refs() may return refs with null sha-1\ndiff --git a/builtin/init-db.c b/builtin/init-db.c\nindex 546f9c595e7..37dd3dcca75 100644\n--- a/builtin/init-db.c\n+++ b/builtin/init-db.c\n@@ -167,12 +167,14 @@ static int needs_work_tree_config(const char *git_dir, const char *work_tree)\n \treturn 1;\n }\n \n-void initialize_repository_version(int hash_algo, int reinit)\n+void initialize_repository_version(int hash_algo, int reinit,\n+\t\t\t\t const char *ref_storage_format)\n {\n \tchar repo_version_string[10];\n \tint repo_version = GIT_REPO_VERSION;\n \n-\tif (hash_algo != GIT_HASH_SHA1)\n+\tif (hash_algo != GIT_HASH_SHA1 ||\n+\t !strcmp(ref_storage_format, \"reftable\"))\n \t\trepo_version = GIT_REPO_VERSION_READ;\n \n \t/* This forces creation of new config file */\n@@ -226,6 +228,7 @@ static int create_default_files(const char *template_path,\n \tis_bare_repository_cfg = init_is_bare_repository || !work_tree;\n \tif (init_shared_repository != -1)\n \t\tset_shared_repository(init_shared_repository);\n+\tthe_repository->ref_storage_format = xstrdup(fmt->ref_storage);\n \n \t/*\n \t * We would have created the above under user's umask -- under\n@@ -235,6 +238,24 @@ static int create_default_files(const char *template_path,\n \t\tadjust_shared_perm(get_git_dir());\n \t}\n \n+\t/*\n+\t * Check to see if .git/HEAD exists; this must happen before\n+\t * initializing the ref db, because we want to see if there is an\n+\t * existing HEAD.\n+\t */\n+\tpath = git_path_buf(&buf, \"HEAD\");\n+\treinit = (!access(path, R_OK) ||\n+\t\t readlink(path, junk, sizeof(junk) - 1) != -1);\n+\n+\t/*\n+\t * refs/heads is a file when using reftable. We can't reinitialize with\n+\t * a reftable because it will overwrite HEAD\n+\t */\n+\tif (reinit && (!strcmp(fmt->ref_storage, \"reftable\")) ==\n+\t\t\t is_directory(git_path_buf(&buf, \"refs/heads\"))) {\n+\t\tdie(\"cannot switch ref storage format.\");\n+\t}\n+\n \t/*\n \t * We need to create a \"refs\" dir in any case so that older\n \t * versions of git can tell that this is a repository.\n@@ -242,6 +263,8 @@ static int create_default_files(const char *template_path,\n \tsafe_create_dir(git_path(\"refs\"), 1);\n \tadjust_shared_perm(git_path(\"refs\"));\n \n+\t/* XXX hack. What is the right place to initialize this? */\n+\tthe_repository->hash_algo = &hash_algos[fmt->hash_algo];\n \tif (refs_init_db(&err))\n \t\tdie(\"failed to set up refs db: %s\", err.buf);\n \n@@ -249,9 +272,6 @@ static int create_default_files(const char *template_path,\n \t * Point the HEAD symref to the initial branch with if HEAD does\n \t * not yet exist.\n \t */\n-\tpath = git_path_buf(&buf, \"HEAD\");\n-\treinit = (!access(path, R_OK)\n-\t\t || readlink(path, junk, sizeof(junk)-1) != -1);\n \tif (!reinit) {\n \t\tchar *ref;\n \n@@ -268,7 +288,7 @@ static int create_default_files(const char *template_path,\n \t\tfree(ref);\n \t}\n \n-\tinitialize_repository_version(fmt->hash_algo, 0);\n+\tinitialize_repository_version(fmt->hash_algo, 0, fmt->ref_storage);\n \n \t/* Check filemode trustability */\n \tpath = git_path_buf(&buf, \"config\");\n@@ -383,7 +403,7 @@ static void validate_hash_algorithm(struct repository_format *repo_fmt, int hash\n \n int init_db(const char *git_dir, const char *real_git_dir,\n \t const char *template_dir, int hash, const char *initial_branch,\n-\t unsigned int flags)\n+\t const char *ref_storage_format, unsigned int flags)\n {\n \tint reinit;\n \tint exist_ok = flags & INIT_DB_EXIST_OK;\n@@ -422,6 +442,7 @@ int init_db(const char *git_dir, const char *real_git_dir,\n \t * is an attempt to reinitialize new repository with an old tool.\n \t */\n \tcheck_repository_format(&repo_fmt);\n+\trepo_fmt.ref_storage = xstrdup(ref_storage_format);\n \n \tvalidate_hash_algorithm(&repo_fmt, hash);\n \n@@ -455,6 +476,9 @@ int init_db(const char *git_dir, const char *real_git_dir,\n \t\tgit_config_set(\"receive.denyNonFastforwards\", \"true\");\n \t}\n \n+\tif (!strcmp(ref_storage_format, \"reftable\"))\n+\t\tgit_config_set(\"extensions.refStorage\", ref_storage_format);\n+\n \tif (!(flags & INIT_DB_QUIET)) {\n \t\tint len = strlen(git_dir);\n \n@@ -528,6 +552,7 @@ static const char *const init_db_usage[] = {\n int cmd_init_db(int argc, const char **argv, const char *prefix)\n {\n \tconst char *git_dir;\n+\tconst char *ref_storage_format = default_ref_storage();\n \tconst char *real_git_dir = NULL;\n \tconst char *work_tree;\n \tconst char *template_dir = NULL;\n@@ -692,5 +717,5 @@ int cmd_init_db(int argc, const char **argv, const char *prefix)\n \n \tflags |= INIT_DB_EXIST_OK;\n \treturn init_db(git_dir, real_git_dir, template_dir, hash_algo,\n-\t\t initial_branch, flags);\n+\t\t initial_branch, ref_storage_format, flags);\n }\ndiff --git a/builtin/stash.c b/builtin/stash.c\nindex 5897febfbec..e60b063d133 100644\n--- a/builtin/stash.c\n+++ b/builtin/stash.c\n@@ -207,10 +207,16 @@ static int get_stash_info(struct stash_info *info, int argc, const char **argv)\n static int do_clear_stash(void)\n {\n \tstruct object_id obj;\n+\tint result;\n \tif (get_oid(ref_stash, &obj))\n \t\treturn 0;\n \n-\treturn delete_ref(NULL, ref_stash, &obj, 0);\n+\tresult = delete_ref(NULL, ref_stash, &obj, 0);\n+\n+\t/* Ignore error; this is necessary for reftable, which keeps reflogs\n+\t * even when refs are deleted. */\n+\tdelete_reflog(ref_stash);\n+\treturn result;\n }\n \n static int clear_stash(int argc, const char **argv, const char *prefix)\ndiff --git a/builtin/worktree.c b/builtin/worktree.c\nindex 31a8a2a4e8a..75db3a2cb77 100644\n--- a/builtin/worktree.c\n+++ b/builtin/worktree.c\n@@ -14,6 +14,7 @@\n #include \"utf8.h\"\n #include \"worktree.h\"\n #include \"quote.h\"\n+#include \"../refs/refs-internal.h\"\n \n static const char * const worktree_usage[] = {\n \tN_(\"git worktree add [] []\"),\n@@ -329,9 +330,29 @@ static int add_worktree(const char *path, const char *refname,\n \t * worktree.\n \t */\n \tstrbuf_reset(&sb);\n-\tstrbuf_addf(&sb, \"%s/HEAD\", sb_repo.buf);\n-\twrite_file(sb.buf, \"%s\", oid_to_hex(null_oid()));\n-\tstrbuf_reset(&sb);\n+\tif (get_main_ref_store(the_repository)->be == &refs_be_reftable) {\n+\t\t/* XXX this is cut & paste from reftable_init_db. */\n+\t\tstrbuf_addf(&sb, \"%s/HEAD\", sb_repo.buf);\n+\t\twrite_file(sb.buf, \"%s\", \"ref: refs/heads/.invalid\\n\");\n+\t\tstrbuf_reset(&sb);\n+\n+\t\tstrbuf_addf(&sb, \"%s/refs\", sb_repo.buf);\n+\t\tsafe_create_dir(sb.buf, 1);\n+\t\tstrbuf_reset(&sb);\n+\n+\t\tstrbuf_addf(&sb, \"%s/refs/heads\", sb_repo.buf);\n+\t\twrite_file(sb.buf, \"this repository uses the reftable format\");\n+\t\tstrbuf_reset(&sb);\n+\n+\t\tstrbuf_addf(&sb, \"%s/reftable\", sb_repo.buf);\n+\t\tsafe_create_dir(sb.buf, 1);\n+\t\tstrbuf_reset(&sb);\n+\t} else {\n+\t\tstrbuf_addf(&sb, \"%s/HEAD\", sb_repo.buf);\n+\t\twrite_file(sb.buf, \"%s\", oid_to_hex(null_oid()));\n+\t\tstrbuf_reset(&sb);\n+\t}\n+\n \tstrbuf_addf(&sb, \"%s/commondir\", sb_repo.buf);\n \twrite_file(sb.buf, \"../..\");\n \ndiff --git a/cache.h b/cache.h\nindex 7cc7b29e6ec..c63d2152232 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -653,9 +653,10 @@ int path_inside_repo(const char *prefix, const char *path);\n #define INIT_DB_EXIST_OK 0x0002\n \n int init_db(const char *git_dir, const char *real_git_dir,\n-\t const char *template_dir, int hash_algo,\n-\t const char *initial_branch, unsigned int flags);\n-void initialize_repository_version(int hash_algo, int reinit);\n+\t const char *template_dir, int hash_algo, const char *initial_branch,\n+\t const char *ref_storage_format, unsigned int flags);\n+void initialize_repository_version(int hash_algo, int reinit,\n+\t\t\t\t const char *ref_storage_format);\n \n void sanitize_stdfds(void);\n int daemonize(void);\n@@ -1076,6 +1077,7 @@ struct repository_format {\n \tint hash_algo;\n \tint sparse_index;\n \tchar *work_tree;\n+\tchar *ref_storage;\n \tstruct string_list unknown_extensions;\n \tstruct string_list v1_only_extensions;\n };\ndiff --git a/config.mak.uname b/config.mak.uname\nindex 501970902da..dae90ae515f 100644\n--- a/config.mak.uname\n+++ b/config.mak.uname\n@@ -749,7 +749,7 @@ vcxproj:\n \t# Make .vcxproj files and add them\n \tunset QUIET_GEN QUIET_BUILT_IN; \\\n \tperl contrib/buildsystems/generate -g Vcxproj\n-\tgit add -f git.sln {*,*/lib,t/helper/*}/*.vcxproj\n+\tgit add -f git.sln {*,*/lib,*/libreftable,t/helper/*}/*.vcxproj\n \n \t# Generate the LinkOrCopyBuiltins.targets and LinkOrCopyRemoteHttp.targets file\n \t(echo '' && \\\ndiff --git a/contrib/workdir/git-new-workdir b/contrib/workdir/git-new-workdir\nindex 888c34a5215..989197aace0 100755\n--- a/contrib/workdir/git-new-workdir\n+++ b/contrib/workdir/git-new-workdir\n@@ -79,7 +79,7 @@ trap cleanup $siglist\n # create the links to the original repo. explicitly exclude index, HEAD and\n # logs/HEAD from the list since they are purely related to the current working\n # directory, and should not be shared.\n-for x in config refs logs/refs objects info hooks packed-refs remotes rr-cache svn\n+for x in config refs logs/refs objects info hooks packed-refs remotes rr-cache svn reftable\n do\n \t# create a containing directory if needed\n \tcase $x in\ndiff --git a/refs.c b/refs.c\nindex d680de3bc04..ed900148d1c 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -20,10 +20,15 @@\n #include \"repository.h\"\n #include \"sigchain.h\"\n \n+const char *default_ref_storage(void)\n+{\n+\treturn git_env_bool(\"GIT_TEST_REFTABLE\", 0) ? \"reftable\" : \"files\";\n+}\n+\n /*\n * List of all available backends\n */\n-static struct ref_storage_be *refs_backends = &refs_be_files;\n+static struct ref_storage_be *refs_backends = &refs_be_reftable;\n \n static struct ref_storage_be *find_ref_storage_backend(const char *name)\n {\n@@ -1860,13 +1865,13 @@ static struct ref_store *lookup_ref_store_map(struct hashmap *map,\n * gitdir.\n */\n static struct ref_store *ref_store_init(struct repository *repo,\n-\t\t\t\t\tconst char *gitdir,\n+\t\t\t\t\tconst char *gitdir, const char *be_name,\n \t\t\t\t\tunsigned int flags)\n {\n-\tconst char *be_name = \"files\";\n-\tstruct ref_storage_be *be = find_ref_storage_backend(be_name);\n+\tstruct ref_storage_be *be;\n \tstruct ref_store *refs;\n \n+\tbe = find_ref_storage_backend(be_name);\n \tif (!be)\n \t\tBUG(\"reference backend %s is unknown\", be_name);\n \n@@ -1882,7 +1887,11 @@ struct ref_store *get_main_ref_store(struct repository *r)\n \tif (!r->gitdir)\n \t\tBUG(\"attempting to get main_ref_store outside of repository\");\n \n-\tr->refs_private = ref_store_init(r, r->gitdir, REF_STORE_ALL_CAPS);\n+\tr->refs_private = ref_store_init(r, r->gitdir,\n+\t\t\t\t\t r->ref_storage_format ?\n+\t\t\t\t\t\t r->ref_storage_format :\n+\t\t\t\t\t\t default_ref_storage(),\n+\t\t\t\t\t REF_STORE_ALL_CAPS);\n \tr->refs_private = maybe_debug_wrap_ref_store(r->gitdir, r->refs_private);\n \treturn r->refs_private;\n }\n@@ -1950,7 +1959,7 @@ struct ref_store *get_submodule_ref_store(const char *submodule)\n \t\tfree(subrepo);\n \t\tgoto done;\n \t}\n-\trefs = ref_store_init(subrepo, submodule_sb.buf,\n+\trefs = ref_store_init(subrepo, submodule_sb.buf, default_ref_storage(),\n \t\t\t REF_STORE_READ | REF_STORE_ODB);\n \tregister_ref_store_map(&submodule_ref_stores, \"submodule\",\n \t\t\t refs, submodule);\n@@ -1964,6 +1973,7 @@ done:\n \n struct ref_store *get_worktree_ref_store(const struct worktree *wt)\n {\n+\tconst char *format = default_ref_storage();\n \tstruct ref_store *refs;\n \tconst char *id;\n \n@@ -1978,11 +1988,10 @@ struct ref_store *get_worktree_ref_store(const struct worktree *wt)\n \tif (wt->id)\n \t\trefs = ref_store_init(the_repository,\n \t\t\t\t git_common_path(\"worktrees/%s\", wt->id),\n-\t\t\t\t REF_STORE_ALL_CAPS);\n+\t\t\t\t format, REF_STORE_ALL_CAPS);\n \telse\n-\t\trefs = ref_store_init(the_repository,\n-\t\t\t\t get_git_common_dir(),\n-\t\t\t\t REF_STORE_ALL_CAPS);\n+\t\trefs = ref_store_init(the_repository, get_git_common_dir(),\n+\t\t\t\t format, REF_STORE_ALL_CAPS);\n \n \tif (refs)\n \t\tregister_ref_store_map(&worktree_ref_stores, \"worktree\",\ndiff --git a/refs.h b/refs.h\nindex ff859d59513..42a35621fa6 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -11,6 +11,9 @@ struct string_list;\n struct string_list_item;\n struct worktree;\n \n+/* Returns the ref storage backend to use by default. */\n+const char *default_ref_storage(void);\n+\n /*\n * Resolve a reference, recursively following symbolic refererences.\n *\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 6e15db3ca4e..012820f7b41 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -679,6 +679,7 @@ struct ref_storage_be {\n };\n \n extern struct ref_storage_be refs_be_files;\n+extern struct ref_storage_be refs_be_reftable;\n extern struct ref_storage_be refs_be_packed;\n \n /*\ndiff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\nnew file mode 100644\nindex 00000000000..81bfc522338\n--- /dev/null\n+++ b/refs/reftable-backend.c\n@@ -0,0 +1,1898 @@\n+#include \"../cache.h\"\n+#include \"../chdir-notify.h\"\n+#include \"../config.h\"\n+#include \"../iterator.h\"\n+#include \"../lockfile.h\"\n+#include \"../object.h\"\n+#include \"../refs.h\"\n+#include \"../reftable/reftable-stack.h\"\n+#include \"../reftable/reftable-record.h\"\n+#include \"../reftable/reftable-error.h\"\n+#include \"../reftable/reftable-blocksource.h\"\n+#include \"../reftable/reftable-reader.h\"\n+#include \"../reftable/reftable-iterator.h\"\n+#include \"../reftable/reftable-merged.h\"\n+#include \"../reftable/reftable-generic.h\"\n+#include \"../worktree.h\"\n+#include \"refs-internal.h\"\n+\n+extern struct ref_storage_be refs_be_reftable;\n+\n+struct git_reftable_ref_store {\n+\tstruct ref_store base;\n+\tunsigned int store_flags;\n+\n+\tint err;\n+\tchar *repo_dir;\n+\tchar *reftable_dir;\n+\n+\tstruct reftable_stack *main_stack;\n+\tstruct reftable_stack *worktree_stack;\n+\n+\tstruct reftable_write_options write_options;\n+};\n+\n+/*\n+ * Some refs are global to the repository (refs/heads/{*}), while others are\n+ * local to the worktree (eg. HEAD, refs/bisect/{*}). We solve this by having\n+ * two separate databases (ie. two reftable/ directories), one for the\n+ * repository, and one for the worktree. For reading, we merge the view (see\n+ * git_reftable_iterator) of both, when necessary.\n+ *\n+ * Unfortunately, the worktrees can also be selected by specifying a magic\n+ * refname (eg. worktree/BLA/refname, even if BLA isn't the current worktree.)\n+ */\n+static struct reftable_stack *stack_for(struct git_reftable_ref_store *store,\n+\t\t\t\t\tconst char *refname)\n+{\n+\tconst char *wtname = refname;\n+\tint wtname_len = 0;\n+\tconst char *wtref = refname;\n+\n+\tif (refname == NULL)\n+\t\treturn store->main_stack;\n+\n+\tif (!parse_worktree_ref(refname, &wtname, &wtname_len, &wtref) &&\n+\t wtname_len) {\n+\t\t/* Woe you if you try to access worktree/BLA/REF and the current\n+\t\t * worktree from the same process.\n+\t\t */\n+\t\tstruct strbuf wt_dir = STRBUF_INIT;\n+\n+\t\tstrbuf_addstr(&wt_dir, store->base.gitdir);\n+\t\tstrbuf_addstr(&wt_dir, \"/worktrees/\");\n+\t\tstrbuf_add(&wt_dir, wtname, wtname_len);\n+\t\tstrbuf_addstr(&wt_dir, \"/reftable\");\n+\n+\t\tif (store->worktree_stack)\n+\t\t\treftable_stack_destroy(store->worktree_stack);\n+\t\tstore->err = reftable_new_stack(&store->worktree_stack,\n+\t\t\t\t\t\twt_dir.buf,\n+\t\t\t\t\t\tstore->write_options);\n+\t\tassert(store->err != REFTABLE_API_ERROR);\n+\n+\t\treturn store->worktree_stack;\n+\t}\n+\n+\tif (store->worktree_stack == NULL)\n+\t\treturn store->main_stack;\n+\n+\tswitch (ref_type(refname)) {\n+\tcase REF_TYPE_PER_WORKTREE:\n+\tcase REF_TYPE_PSEUDOREF:\n+\tcase REF_TYPE_OTHER_PSEUDOREF:\n+\t\treturn store->worktree_stack;\n+\tdefault:\n+\tcase REF_TYPE_MAIN_PSEUDOREF:\n+\tcase REF_TYPE_NORMAL:\n+\t\treturn store->main_stack;\n+\t}\n+}\n+\n+static int should_log(const char *refname)\n+{\n+\treturn log_all_ref_updates != LOG_REFS_NONE &&\n+\t (log_all_ref_updates == LOG_REFS_ALWAYS ||\n+\t\tlog_all_ref_updates == LOG_REFS_UNSET ||\n+\t\tshould_autocreate_reflog(refname));\n+}\n+\n+static const char *bare_ref_name(const char *ref)\n+{\n+\tconst char *out = ref;\n+\tint name_len = 0;\n+\tif (skip_prefix(ref, \"main-worktree/\", &out))\n+\t\treturn out;\n+\n+\tif (!parse_worktree_ref(ref, NULL, &name_len, &out) && name_len) {\n+\t\treturn out;\n+\t}\n+\n+\treturn ref;\n+}\n+\n+static int git_reftable_read_raw_ref(struct ref_store *ref_store,\n+\t\t\t\t const char *refname, struct object_id *oid,\n+\t\t\t\t struct strbuf *referent,\n+\t\t\t\t unsigned int *type, int *failure_errno);\n+\n+static void clear_reftable_log_record(struct reftable_log_record *log)\n+{\n+\tswitch (log->value_type) {\n+\tcase REFTABLE_LOG_UPDATE:\n+\t\t/* when we write log records, the hashes are owned by a struct\n+\t\t * oid */\n+\t\tlog->value.update.old_hash = NULL;\n+\t\tlog->value.update.new_hash = NULL;\n+\t\tbreak;\n+\tcase REFTABLE_LOG_DELETION:\n+\t\tbreak;\n+\t}\n+\treftable_log_record_release(log);\n+}\n+\n+static void fill_reftable_log_record(struct reftable_log_record *log)\n+{\n+\tconst char *info = git_committer_info(0);\n+\tstruct ident_split split = { NULL };\n+\tint result = split_ident_line(&split, info, strlen(info));\n+\tint sign = 1;\n+\tassert(0 == result);\n+\n+\treftable_log_record_release(log);\n+\tlog->value_type = REFTABLE_LOG_UPDATE;\n+\tlog->value.update.name =\n+\t\txstrndup(split.name_begin, split.name_end - split.name_begin);\n+\tlog->value.update.email =\n+\t\txstrndup(split.mail_begin, split.mail_end - split.mail_begin);\n+\tlog->value.update.time = atol(split.date_begin);\n+\tif (*split.tz_begin == '-') {\n+\t\tsign = -1;\n+\t\tsplit.tz_begin++;\n+\t}\n+\tif (*split.tz_begin == '+') {\n+\t\tsign = 1;\n+\t\tsplit.tz_begin++;\n+\t}\n+\n+\tlog->value.update.tz_offset = sign * atoi(split.tz_begin);\n+}\n+\n+static int has_suffix(struct strbuf *b, const char *suffix)\n+{\n+\tsize_t len = strlen(suffix);\n+\n+\tif (len > b->len) {\n+\t\treturn 0;\n+\t}\n+\n+\treturn 0 == strncmp(b->buf + b->len - len, suffix, len);\n+}\n+\n+/* trims the last path component of b. Returns -1 if it is not\n+ * present, or 0 on success\n+ */\n+static int trim_component(struct strbuf *b)\n+{\n+\tchar *last;\n+\tlast = strrchr(b->buf, '/');\n+\tif (!last)\n+\t\treturn -1;\n+\tstrbuf_setlen(b, last - b->buf);\n+\treturn 0;\n+}\n+\n+/* Returns whether `b` is a worktree path. Mutates its arg, trimming it to the\n+ * gitdir\n+ */\n+static int is_worktree(struct strbuf *b)\n+{\n+\tif (trim_component(b) < 0) {\n+\t\treturn 0;\n+\t}\n+\tif (!has_suffix(b, \"/worktrees\")) {\n+\t\treturn 0;\n+\t}\n+\ttrim_component(b);\n+\treturn 1;\n+}\n+\n+static struct ref_store *git_reftable_ref_store_create(struct repository *repo,\n+\t\t\t\t\t\t const char *path,\n+\t\t\t\t\t\t unsigned int store_flags)\n+{\n+\tstruct git_reftable_ref_store *refs = xcalloc(1, sizeof(*refs));\n+\tstruct ref_store *ref_store = (struct ref_store *)refs;\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tstruct strbuf gitdir = STRBUF_INIT;\n+\tstruct strbuf wt_buf = STRBUF_INIT;\n+\tint wt = 0;\n+\tint shared = get_shared_repository();\n+\tif (shared < 0)\n+\t\tshared = -shared;\n+\n+\trefs->write_options.block_size = 4096;\n+\trefs->write_options.hash_id = the_hash_algo->format_id;\n+\tif (shared && (shared & 0600))\n+\t\trefs->write_options.default_permissions = shared;\n+\n+\tstrbuf_realpath(&wt_buf, path, /*die_on_error=*/0);\n+\n+\t/* this is clumsy, but the official worktree functions (eg.\n+\t * get_worktrees()) function will try to initialize a ref storage\n+\t * backend, leading to infinite recursion. */\n+\twt = is_worktree(&wt_buf);\n+\tif (wt) {\n+\t\tstrbuf_addbuf(&gitdir, &wt_buf);\n+\t} else {\n+\t\tstrbuf_realpath(&gitdir, path, /*die_on_error=*/0);\n+\t}\n+\n+\t/* XXX should this use `path` or `gitdir.buf` ? */\n+\tbase_ref_store_init(ref_store, repo, path, &refs_be_reftable);\n+\trefs->store_flags = store_flags;\n+\tstrbuf_addf(&sb, \"%s/reftable\", gitdir.buf);\n+\trefs->reftable_dir = xstrdup(sb.buf);\n+\trefs->base.repo = repo;\n+\tstrbuf_reset(&sb);\n+\n+\trefs->err = reftable_new_stack(&refs->main_stack, refs->reftable_dir,\n+\t\t\t\t refs->write_options);\n+\tassert(refs->err != REFTABLE_API_ERROR);\n+\n+\tif (refs->err == 0 && wt) {\n+\t\tstrbuf_addf(&sb, \"%s/reftable\", path);\n+\n+\t\trefs->err = reftable_new_stack(&refs->worktree_stack, sb.buf,\n+\t\t\t\t\t refs->write_options);\n+\t\tassert(refs->err != REFTABLE_API_ERROR);\n+\t}\n+\n+\tstrbuf_release(&sb);\n+\tstrbuf_release(&wt_buf);\n+\tstrbuf_release(&gitdir);\n+\treturn ref_store;\n+}\n+\n+static int git_reftable_init_db(struct ref_store *ref_store, struct strbuf *err)\n+{\n+\tstruct git_reftable_ref_store *refs =\n+\t\t(struct git_reftable_ref_store *)ref_store;\n+\tstruct strbuf sb = STRBUF_INIT;\n+\n+\tsafe_create_dir(refs->reftable_dir, 1);\n+\n+\tstrbuf_addf(&sb, \"%s/HEAD\", refs->base.gitdir);\n+\twrite_file(sb.buf, \"ref: refs/heads/.invalid\");\n+\tadjust_shared_perm(sb.buf);\n+\tstrbuf_reset(&sb);\n+\n+\tstrbuf_addf(&sb, \"%s/refs\", refs->base.gitdir);\n+\tsafe_create_dir(sb.buf, 1);\n+\tstrbuf_reset(&sb);\n+\n+\tstrbuf_addf(&sb, \"%s/refs/heads\", refs->base.gitdir);\n+\twrite_file(sb.buf, \"this repository uses the reftable format\");\n+\tadjust_shared_perm(sb.buf);\n+\n+\tstrbuf_release(&sb);\n+\treturn 0;\n+}\n+\n+struct git_reftable_iterator {\n+\tstruct ref_iterator base;\n+\tstruct reftable_iterator iter;\n+\tstruct reftable_ref_record ref;\n+\tstruct object_id oid;\n+\tstruct ref_store *ref_store;\n+\n+\t/* In case we must iterate over 2 stacks, this is non-null. */\n+\tstruct reftable_merged_table *merged;\n+\tunsigned int flags;\n+\tint err;\n+\tconst char *prefix;\n+};\n+\n+static int reftable_ref_iterator_advance(struct ref_iterator *ref_iterator)\n+{\n+\tstruct git_reftable_iterator *ri =\n+\t\t(struct git_reftable_iterator *)ref_iterator;\n+\tstruct git_reftable_ref_store *refs =\n+\t\t(struct git_reftable_ref_store *)ri->ref_store;\n+\n+\twhile (ri->err == 0) {\n+\t\tint signed_flags = 0;\n+\t\tri->err = reftable_iterator_next_ref(&ri->iter, &ri->ref);\n+\t\tif (ri->err) {\n+\t\t\tbreak;\n+\t\t}\n+\n+\t\tri->base.flags = 0;\n+\n+\t\tif (ref_type(ri->ref.refname) == REF_TYPE_PSEUDOREF) {\n+\t\t\t/*\n+\t\t\t pseudorefs, eg. HEAD, FETCH_HEAD should not be\n+\t\t\t produced, by default.\n+\t\t\t */\n+\t\t\tcontinue;\n+\t\t}\n+\t\tri->base.refname = ri->ref.refname;\n+\t\tif (ri->prefix != NULL &&\n+\t\t strncmp(ri->prefix, ri->ref.refname, strlen(ri->prefix))) {\n+\t\t\tri->err = 1;\n+\t\t\tbreak;\n+\t\t}\n+\t\tif (ri->flags & DO_FOR_EACH_PER_WORKTREE_ONLY &&\n+\t\t ref_type(ri->base.refname) != REF_TYPE_PER_WORKTREE)\n+\t\t\tcontinue;\n+\n+\t\tif (ri->flags & DO_FOR_EACH_INCLUDE_BROKEN &&\n+\t\t check_refname_format(ri->base.refname,\n+\t\t\t\t\t REFNAME_ALLOW_ONELEVEL)) {\n+\t\t\t/* This is odd, as REF_BAD_NAME and REF_ISBROKEN are\n+\t\t\t orthogonal, but it's what the spec says and the\n+\t\t\t files-backend does. */\n+\t\t\tri->base.flags |= REF_BAD_NAME | REF_ISBROKEN;\n+\t\t\tbreak;\n+\t\t}\n+\n+\t\tswitch (ri->ref.value_type) {\n+\t\tcase REFTABLE_REF_VAL1:\n+\t\t\toidread(&ri->oid, ri->ref.value.val1);\n+\t\t\tbreak;\n+\t\tcase REFTABLE_REF_VAL2:\n+\t\t\toidread(&ri->oid, ri->ref.value.val2.value);\n+\t\t\tbreak;\n+\t\tcase REFTABLE_REF_SYMREF:\n+\t\t\tri->base.flags = REF_ISSYMREF;\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\tabort();\n+\t\t}\n+\n+\t\tri->base.oid = &ri->oid;\n+\t\tif (!(ri->flags & DO_FOR_EACH_INCLUDE_BROKEN) &&\n+\t\t !ref_resolves_to_object(ri->base.refname, refs->base.repo,\n+\t\t\t\t\t ri->base.oid, ri->base.flags)) {\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\t/* Arguably, resolving recursively following symlinks should be\n+\t\t * lifted to refs.c because it is shared between reftable and\n+\t\t * the files backend, but it's here now.\n+\t\t */\n+\t\tif (!refs_resolve_ref_unsafe(ri->ref_store, ri->ref.refname,\n+\t\t\t\t\t RESOLVE_REF_READING, &ri->oid,\n+\t\t\t\t\t &signed_flags)) {\n+\t\t\tri->base.flags = signed_flags;\n+\t\t\tif (ri->ref.value_type == REFTABLE_REF_SYMREF &&\n+\t\t\t ri->flags & DO_FOR_EACH_OMIT_DANGLING_SYMREFS)\n+\t\t\t\tcontinue;\n+\n+\t\t\tif (ri->ref.value_type == REFTABLE_REF_SYMREF &&\n+\t\t\t !(ri->flags & DO_FOR_EACH_INCLUDE_BROKEN) &&\n+\t\t\t (ri->base.flags & REF_ISBROKEN)) {\n+\t\t\t\tcontinue;\n+\t\t\t}\n+\n+\t\t\tif (is_null_oid(&ri->oid)) {\n+\t\t\t\toidclr(&ri->oid);\n+\t\t\t\tri->base.flags |= REF_ISBROKEN;\n+\t\t\t}\n+\t\t}\n+\t\tbreak;\n+\t}\n+\n+\tif (ri->err > 0) {\n+\t\treturn ITER_DONE;\n+\t}\n+\tif (ri->err < 0) {\n+\t\treturn ITER_ERROR;\n+\t}\n+\n+\treturn ITER_OK;\n+}\n+\n+static int reftable_ref_iterator_peel(struct ref_iterator *ref_iterator,\n+\t\t\t\t struct object_id *peeled)\n+{\n+\tstruct git_reftable_iterator *ri =\n+\t\t(struct git_reftable_iterator *)ref_iterator;\n+\tif (ri->ref.value_type == REFTABLE_REF_VAL2) {\n+\t\toidread(peeled, ri->ref.value.val2.target_value);\n+\t\treturn 0;\n+\t}\n+\n+\treturn -1;\n+}\n+\n+static int reftable_ref_iterator_abort(struct ref_iterator *ref_iterator)\n+{\n+\tstruct git_reftable_iterator *ri =\n+\t\t(struct git_reftable_iterator *)ref_iterator;\n+\treftable_ref_record_release(&ri->ref);\n+\treftable_iterator_destroy(&ri->iter);\n+\tif (ri->merged) {\n+\t\treftable_merged_table_free(ri->merged);\n+\t}\n+\treturn 0;\n+}\n+\n+static struct ref_iterator_vtable reftable_ref_iterator_vtable = {\n+\treftable_ref_iterator_advance, reftable_ref_iterator_peel,\n+\treftable_ref_iterator_abort\n+};\n+\n+static struct ref_iterator *\n+git_reftable_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,\n+\t\t\t\tunsigned int flags)\n+{\n+\tstruct git_reftable_ref_store *refs =\n+\t\t(struct git_reftable_ref_store *)ref_store;\n+\tstruct git_reftable_iterator *ri = xcalloc(1, sizeof(*ri));\n+\n+\tif (refs->err < 0) {\n+\t\tri->err = refs->err;\n+\t} else if (refs->worktree_stack == NULL) {\n+\t\tstruct reftable_merged_table *mt =\n+\t\t\treftable_stack_merged_table(refs->main_stack);\n+\t\tri->err = reftable_merged_table_seek_ref(mt, &ri->iter, prefix);\n+\t} else {\n+\t\tstruct reftable_merged_table *mt1 =\n+\t\t\treftable_stack_merged_table(refs->main_stack);\n+\t\tstruct reftable_merged_table *mt2 =\n+\t\t\treftable_stack_merged_table(refs->worktree_stack);\n+\t\tstruct reftable_table *tabs =\n+\t\t\txcalloc(2, sizeof(struct reftable_table));\n+\t\treftable_table_from_merged_table(&tabs[0], mt1);\n+\t\treftable_table_from_merged_table(&tabs[1], mt2);\n+\n+\t\t/* XXX this isn't correct. This will merge reftables, which\n+\t\t * gives precedence to the most recently updated refs. We should\n+\t\t * give precedence to refs from the worktree / main stack\n+\t\t * depending on where we are\n+\t\t */\n+\t\tri->err = reftable_new_merged_table(&ri->merged, tabs, 2,\n+\t\t\t\t\t\t the_hash_algo->format_id);\n+\t\tif (ri->err == 0)\n+\t\t\tri->err = reftable_merged_table_seek_ref(\n+\t\t\t\tri->merged, &ri->iter, prefix);\n+\t}\n+\n+\tbase_ref_iterator_init(&ri->base, &reftable_ref_iterator_vtable, 1);\n+\tri->prefix = prefix;\n+\tri->base.oid = &ri->oid;\n+\tri->flags = flags;\n+\tri->ref_store = ref_store;\n+\treturn &ri->base;\n+}\n+\n+static int fixup_symrefs(struct ref_store *ref_store,\n+\t\t\t struct ref_transaction *transaction)\n+{\n+\tstruct strbuf referent = STRBUF_INIT;\n+\tint i = 0;\n+\tint err = 0;\n+\n+\tfor (i = 0; i < transaction->nr; i++) {\n+\t\tstruct ref_update *update = transaction->updates[i];\n+\t\tstruct object_id old_oid;\n+\t\tint failure_errno;\n+\n+\t\terr = git_reftable_read_raw_ref(ref_store, update->refname,\n+\t\t\t\t\t\t&old_oid, &referent,\n+\t\t\t\t\t\t/* mutate input, like\n+\t\t\t\t\t\t files-backend.c */\n+\t\t\t\t\t\t&update->type, &failure_errno);\n+\t\tif (err < 0 && failure_errno == ENOENT &&\n+\t\t is_null_oid(&update->old_oid)) {\n+\t\t\terr = 0;\n+\t\t}\n+\t\tif (err < 0)\n+\t\t\tgoto done;\n+\n+\t\tif (!(update->type & REF_ISSYMREF))\n+\t\t\tcontinue;\n+\n+\t\tif (update->flags & REF_NO_DEREF) {\n+\t\t\t/* what should happen here? See files-backend.c\n+\t\t\t * lock_ref_for_update. */\n+\t\t} else {\n+\t\t\t/*\n+\t\t\t If we are updating a symref (eg. HEAD), we should also\n+\t\t\t update the branch that the symref points to.\n+\n+\t\t\t This is generic functionality, and would be better\n+\t\t\t done in refs.c, but the current implementation is\n+\t\t\t intertwined with the locking in files-backend.c.\n+\t\t\t*/\n+\t\t\tint new_flags = update->flags;\n+\t\t\tstruct ref_update *new_update = NULL;\n+\n+\t\t\t/* if this is an update for HEAD, should also record a\n+\t\t\t log entry for HEAD? See files-backend.c,\n+\t\t\t split_head_update()\n+\t\t\t*/\n+\t\t\tnew_update = ref_transaction_add_update(\n+\t\t\t\ttransaction, referent.buf, new_flags,\n+\t\t\t\t&update->new_oid, &update->old_oid,\n+\t\t\t\tupdate->msg);\n+\t\t\tnew_update->parent_update = update;\n+\n+\t\t\t/* files-backend sets REF_LOG_ONLY here. */\n+\t\t\tupdate->flags |= REF_NO_DEREF | REF_LOG_ONLY;\n+\t\t\tupdate->flags &= ~REF_HAVE_OLD;\n+\t\t}\n+\t}\n+\n+done:\n+\tassert(err != REFTABLE_API_ERROR);\n+\tstrbuf_release(&referent);\n+\treturn err;\n+}\n+\n+static int git_reftable_transaction_prepare(struct ref_store *ref_store,\n+\t\t\t\t\t struct ref_transaction *transaction,\n+\t\t\t\t\t struct strbuf *errbuf)\n+{\n+\tstruct git_reftable_ref_store *refs =\n+\t\t(struct git_reftable_ref_store *)ref_store;\n+\tstruct reftable_addition *add = NULL;\n+\tstruct reftable_stack *stack = stack_for(\n+\t\trefs,\n+\t\ttransaction->nr ? transaction->updates[0]->refname : NULL);\n+\tint i;\n+\n+\tint err = refs->err;\n+\tif (err < 0) {\n+\t\tgoto done;\n+\t}\n+\n+\terr = reftable_stack_reload(stack);\n+\tif (err) {\n+\t\tgoto done;\n+\t}\n+\n+\terr = reftable_stack_new_addition(&add, stack);\n+\tif (err) {\n+\t\tgoto done;\n+\t}\n+\n+\tfor (i = 0; i < transaction->nr; i++) {\n+\t\tstruct ref_update *u = transaction->updates[i];\n+\t\tif ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&\n+\t\t !(u->flags & REF_SKIP_OID_VERIFICATION) &&\n+\t\t !(u->flags & REF_LOG_ONLY)) {\n+\t\t\tstruct object *o =\n+\t\t\t\tparse_object(refs->base.repo, &u->new_oid);\n+\t\t\tif (!o) {\n+\t\t\t\tstrbuf_addf(\n+\t\t\t\t\terrbuf,\n+\t\t\t\t\t\"trying to write ref '%s' with nonexistent object %s\",\n+\t\t\t\t\tu->refname, oid_to_hex(&u->new_oid));\n+\t\t\t\terr = -1;\n+\t\t\t\tgoto done;\n+\t\t\t}\n+\t\t}\n+\t}\n+\n+\terr = fixup_symrefs(ref_store, transaction);\n+\tif (err) {\n+\t\tgoto done;\n+\t}\n+\n+\ttransaction->backend_data = add;\n+\ttransaction->state = REF_TRANSACTION_PREPARED;\n+\n+done:\n+\tassert(err != REFTABLE_API_ERROR);\n+\tif (err < 0) {\n+\t\tif (add) {\n+\t\t\treftable_addition_destroy(add);\n+\t\t\tadd = NULL;\n+\t\t}\n+\t\ttransaction->state = REF_TRANSACTION_CLOSED;\n+\t\tif (!errbuf->len)\n+\t\t\tstrbuf_addf(errbuf, \"reftable: transaction prepare: %s\",\n+\t\t\t\t reftable_error_str(err));\n+\t}\n+\n+\treturn err;\n+}\n+\n+static int git_reftable_transaction_abort(struct ref_store *ref_store,\n+\t\t\t\t\t struct ref_transaction *transaction,\n+\t\t\t\t\t struct strbuf *err)\n+{\n+\tstruct reftable_addition *add =\n+\t\t(struct reftable_addition *)transaction->backend_data;\n+\treftable_addition_destroy(add);\n+\ttransaction->backend_data = NULL;\n+\n+\t/* XXX. Shouldn't this be handled generically in refs.c? */\n+\ttransaction->state = REF_TRANSACTION_CLOSED;\n+\treturn 0;\n+}\n+\n+static int reftable_check_old_oid(struct ref_store *refs, const char *refname,\n+\t\t\t\t struct object_id *want_oid)\n+{\n+\tstruct object_id out_oid;\n+\tint out_flags = 0;\n+\tconst char *resolved = refs_resolve_ref_unsafe(\n+\t\trefs, refname, RESOLVE_REF_READING, &out_oid, &out_flags);\n+\tif (is_null_oid(want_oid) != (resolved == NULL)) {\n+\t\treturn REFTABLE_LOCK_ERROR;\n+\t}\n+\n+\tif (resolved != NULL && !oideq(&out_oid, want_oid)) {\n+\t\treturn REFTABLE_LOCK_ERROR;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static int ref_update_cmp(const void *a, const void *b)\n+{\n+\treturn strcmp((*(struct ref_update **)a)->refname,\n+\t\t (*(struct ref_update **)b)->refname);\n+}\n+\n+static int write_transaction_table(struct reftable_writer *writer, void *arg)\n+{\n+\tstruct ref_transaction *transaction = (struct ref_transaction *)arg;\n+\tstruct git_reftable_ref_store *refs =\n+\t\t(struct git_reftable_ref_store *)transaction->ref_store;\n+\tstruct reftable_stack *stack =\n+\t\tstack_for(refs, transaction->updates[0]->refname);\n+\tuint64_t ts = reftable_stack_next_update_index(stack);\n+\tint err = 0;\n+\tint i = 0;\n+\tint log_count = 0;\n+\tstruct reftable_log_record *logs =\n+\t\tcalloc(transaction->nr, sizeof(*logs));\n+\tstruct ref_update **sorted =\n+\t\tmalloc(transaction->nr * sizeof(struct ref_update *));\n+\tstruct reftable_merged_table *mt = reftable_stack_merged_table(stack);\n+\tstruct reftable_table tab = { NULL };\n+\tstruct reftable_ref_record ref = { NULL };\n+\treftable_table_from_merged_table(&tab, mt);\n+\tCOPY_ARRAY(sorted, transaction->updates, transaction->nr);\n+\tQSORT(sorted, transaction->nr, ref_update_cmp);\n+\treftable_writer_set_limits(writer, ts, ts);\n+\n+\tfor (i = 0; i < transaction->nr; i++) {\n+\t\tstruct ref_update *u = sorted[i];\n+\t\tstruct reftable_log_record *log = &logs[log_count];\n+\t\tstruct object_id old_id = *null_oid();\n+\n+\t\tlog->value.update.new_hash = NULL;\n+\t\tlog->value.update.old_hash = NULL;\n+\t\tif ((u->flags & REF_FORCE_CREATE_REFLOG) ||\n+\t\t should_log(u->refname))\n+\t\t\tlog_count++;\n+\t\tfill_reftable_log_record(log);\n+\n+\t\tlog->update_index = ts;\n+\t\tlog->value_type = REFTABLE_LOG_UPDATE;\n+\t\tlog->refname = xstrdup(u->refname);\n+\t\tlog->value.update.new_hash = u->new_oid.hash;\n+\t\tlog->value.update.message =\n+\t\t\txstrndup(u->msg, refs->write_options.block_size / 2);\n+\n+\t\terr = reftable_table_read_ref(&tab, u->refname, &ref);\n+\t\tif (err < 0)\n+\t\t\tgoto done;\n+\t\telse if (err > 0) {\n+\t\t\terr = 0;\n+\t\t}\n+\n+\t\t/* XXX if this is a symref (say, HEAD), should we deref the\n+\t\t * symref and check the update.old_hash against the referent? */\n+\t\tif (ref.value_type == REFTABLE_REF_VAL2 ||\n+\t\t ref.value_type == REFTABLE_REF_VAL1)\n+\t\t\toidread(&old_id, ref.value.val1);\n+\n+\t\t/* XXX fold together with the old_id check below? */\n+\t\tlog->value.update.old_hash = old_id.hash;\n+\t\tif (u->flags & REF_LOG_ONLY) {\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tif (u->flags & REF_HAVE_NEW) {\n+\t\t\tstruct reftable_ref_record ref = { NULL };\n+\t\t\tstruct object_id peeled;\n+\n+\t\t\tint peel_error = peel_object(&u->new_oid, &peeled);\n+\t\t\tref.refname = (char *)u->refname;\n+\t\t\tref.update_index = ts;\n+\n+\t\t\tif (!peel_error) {\n+\t\t\t\tref.value_type = REFTABLE_REF_VAL2;\n+\t\t\t\tref.value.val2.target_value = peeled.hash;\n+\t\t\t\tref.value.val2.value = u->new_oid.hash;\n+\t\t\t} else if (!is_null_oid(&u->new_oid)) {\n+\t\t\t\tref.value_type = REFTABLE_REF_VAL1;\n+\t\t\t\tref.value.val1 = u->new_oid.hash;\n+\t\t\t}\n+\n+\t\t\terr = reftable_writer_add_ref(writer, &ref);\n+\t\t\tif (err < 0) {\n+\t\t\t\tgoto done;\n+\t\t\t}\n+\t\t}\n+\t}\n+\n+\tfor (i = 0; i < log_count; i++) {\n+\t\terr = reftable_writer_add_log(writer, &logs[i]);\n+\t\tlogs[i].value.update.new_hash = NULL;\n+\t\tlogs[i].value.update.old_hash = NULL;\n+\t\tclear_reftable_log_record(&logs[i]);\n+\t\tif (err < 0) {\n+\t\t\tgoto done;\n+\t\t}\n+\t}\n+\n+done:\n+\tassert(err != REFTABLE_API_ERROR);\n+\treftable_ref_record_release(&ref);\n+\tfree(logs);\n+\tfree(sorted);\n+\treturn err;\n+}\n+\n+static int git_reftable_transaction_finish(struct ref_store *ref_store,\n+\t\t\t\t\t struct ref_transaction *transaction,\n+\t\t\t\t\t struct strbuf *errmsg)\n+{\n+\tstruct reftable_addition *add =\n+\t\t(struct reftable_addition *)transaction->backend_data;\n+\tint err = 0;\n+\tint i;\n+\n+\tfor (i = 0; i < transaction->nr; i++) {\n+\t\tstruct ref_update *u = transaction->updates[i];\n+\t\tif (u->flags & REF_HAVE_OLD) {\n+\t\t\terr = reftable_check_old_oid(transaction->ref_store,\n+\t\t\t\t\t\t u->refname, &u->old_oid);\n+\t\t\tif (err < 0) {\n+\t\t\t\tgoto done;\n+\t\t\t}\n+\t\t}\n+\t}\n+\tif (transaction->nr) {\n+\t\terr = reftable_addition_add(add, &write_transaction_table,\n+\t\t\t\t\t transaction);\n+\t\tif (err < 0) {\n+\t\t\tgoto done;\n+\t\t}\n+\t}\n+\n+\terr = reftable_addition_commit(add);\n+\n+done:\n+\tassert(err != REFTABLE_API_ERROR);\n+\treftable_addition_destroy(add);\n+\ttransaction->state = REF_TRANSACTION_CLOSED;\n+\ttransaction->backend_data = NULL;\n+\tif (err) {\n+\t\tstrbuf_addf(errmsg, \"reftable: transaction failure: %s\",\n+\t\t\t reftable_error_str(err));\n+\t\treturn -1;\n+\t}\n+\treturn err;\n+}\n+\n+static int\n+git_reftable_transaction_initial_commit(struct ref_store *ref_store,\n+\t\t\t\t\tstruct ref_transaction *transaction,\n+\t\t\t\t\tstruct strbuf *errmsg)\n+{\n+\tint err = git_reftable_transaction_prepare(ref_store, transaction,\n+\t\t\t\t\t\t errmsg);\n+\tif (err)\n+\t\treturn err;\n+\n+\treturn git_reftable_transaction_finish(ref_store, transaction, errmsg);\n+}\n+\n+struct write_delete_refs_arg {\n+\tstruct git_reftable_ref_store *refs;\n+\tstruct reftable_stack *stack;\n+\tstruct string_list *refnames;\n+\tconst char *logmsg;\n+\tunsigned int flags;\n+};\n+\n+static int write_delete_refs_table(struct reftable_writer *writer, void *argv)\n+{\n+\tstruct write_delete_refs_arg *arg =\n+\t\t(struct write_delete_refs_arg *)argv;\n+\tuint64_t ts = reftable_stack_next_update_index(arg->stack);\n+\tint err = 0;\n+\tint i = 0;\n+\n+\treftable_writer_set_limits(writer, ts, ts);\n+\tfor (i = 0; i < arg->refnames->nr; i++) {\n+\t\tstruct reftable_ref_record ref = {\n+\t\t\t.refname = (char *)arg->refnames->items[i].string,\n+\t\t\t.value_type = REFTABLE_REF_DELETION,\n+\t\t\t.update_index = ts,\n+\t\t};\n+\t\terr = reftable_writer_add_ref(writer, &ref);\n+\t\tif (err < 0) {\n+\t\t\treturn err;\n+\t\t}\n+\t}\n+\n+\tfor (i = 0; i < arg->refnames->nr; i++) {\n+\t\tstruct reftable_log_record log = {\n+\t\t\t.update_index = ts,\n+\t\t};\n+\t\tstruct reftable_ref_record current = { NULL };\n+\t\tfill_reftable_log_record(&log);\n+\t\tlog.update_index = ts;\n+\t\tlog.refname = xstrdup(arg->refnames->items[i].string);\n+\t\tif (!should_log(log.refname)) {\n+\t\t\tcontinue;\n+\t\t}\n+\t\tlog.value.update.message = xstrndup(\n+\t\t\targ->logmsg, arg->refs->write_options.block_size / 2);\n+\t\tlog.value.update.new_hash = NULL;\n+\t\tlog.value.update.old_hash = NULL;\n+\t\tif (reftable_stack_read_ref(arg->stack, log.refname,\n+\t\t\t\t\t ¤t) == 0) {\n+\t\t\tlog.value.update.old_hash =\n+\t\t\t\treftable_ref_record_val1(¤t);\n+\t\t}\n+\t\terr = reftable_writer_add_log(writer, &log);\n+\t\tlog.value.update.old_hash = NULL;\n+\t\treftable_ref_record_release(¤t);\n+\n+\t\tclear_reftable_log_record(&log);\n+\t\tif (err < 0) {\n+\t\t\treturn err;\n+\t\t}\n+\t}\n+\treturn 0;\n+}\n+\n+static int git_reftable_delete_refs(struct ref_store *ref_store,\n+\t\t\t\t const char *msg,\n+\t\t\t\t struct string_list *refnames,\n+\t\t\t\t unsigned int flags)\n+{\n+\tstruct git_reftable_ref_store *refs =\n+\t\t(struct git_reftable_ref_store *)ref_store;\n+\tstruct reftable_stack *stack = stack_for(\n+\t\trefs, refnames->nr ? refnames->items[0].string : NULL);\n+\tstruct write_delete_refs_arg arg = {\n+\t\t.refs = refs,\n+\t\t.stack = stack,\n+\t\t.refnames = refnames,\n+\t\t.logmsg = msg,\n+\t\t.flags = flags,\n+\t};\n+\tint err = refs->err;\n+\tif (err < 0) {\n+\t\tgoto done;\n+\t}\n+\n+\tstring_list_sort(refnames);\n+\terr = reftable_stack_reload(stack);\n+\tif (err) {\n+\t\tgoto done;\n+\t}\n+\terr = reftable_stack_add(stack, &write_delete_refs_table, &arg);\n+done:\n+\tassert(err != REFTABLE_API_ERROR);\n+\treturn err;\n+}\n+\n+static int git_reftable_pack_refs(struct ref_store *ref_store,\n+\t\t\t\t unsigned int flags)\n+{\n+\tstruct git_reftable_ref_store *refs =\n+\t\t(struct git_reftable_ref_store *)ref_store;\n+\tint err = refs->err;\n+\tif (err < 0) {\n+\t\treturn err;\n+\t}\n+\terr = reftable_stack_compact_all(refs->main_stack, NULL);\n+\tif (err == 0 && refs->worktree_stack != NULL)\n+\t\terr = reftable_stack_compact_all(refs->worktree_stack, NULL);\n+\tif (err == 0)\n+\t\terr = reftable_stack_clean(refs->main_stack);\n+\tif (err == 0 && refs->worktree_stack != NULL)\n+\t\terr = reftable_stack_clean(refs->worktree_stack);\n+\n+\treturn err;\n+}\n+\n+struct write_create_symref_arg {\n+\tstruct git_reftable_ref_store *refs;\n+\tstruct reftable_stack *stack;\n+\tconst char *refname;\n+\tconst char *target;\n+\tconst char *logmsg;\n+};\n+\n+static int write_create_symref_table(struct reftable_writer *writer, void *arg)\n+{\n+\tstruct write_create_symref_arg *create =\n+\t\t(struct write_create_symref_arg *)arg;\n+\tuint64_t ts = reftable_stack_next_update_index(create->stack);\n+\tint err = 0;\n+\n+\tstruct reftable_ref_record ref = {\n+\t\t.refname = (char *)create->refname,\n+\t\t.value_type = REFTABLE_REF_SYMREF,\n+\t\t.value.symref = (char *)create->target,\n+\t\t.update_index = ts,\n+\t};\n+\treftable_writer_set_limits(writer, ts, ts);\n+\terr = reftable_writer_add_ref(writer, &ref);\n+\tif (err == 0) {\n+\t\tstruct reftable_log_record log = { NULL };\n+\t\tstruct object_id new_oid;\n+\t\tstruct object_id old_oid;\n+\n+\t\tfill_reftable_log_record(&log);\n+\t\tlog.refname = xstrdup(create->refname);\n+\t\tif (!should_log(log.refname)) {\n+\t\t\treturn err;\n+\t\t}\n+\t\tlog.update_index = ts;\n+\t\tlog.value.update.message =\n+\t\t\txstrndup(create->logmsg,\n+\t\t\t\t create->refs->write_options.block_size / 2);\n+\t\tif (refs_resolve_ref_unsafe(\n+\t\t\t (struct ref_store *)create->refs, create->refname,\n+\t\t\t RESOLVE_REF_READING, &old_oid, NULL) != NULL) {\n+\t\t\tlog.value.update.old_hash = old_oid.hash;\n+\t\t}\n+\n+\t\tif (refs_resolve_ref_unsafe((struct ref_store *)create->refs,\n+\t\t\t\t\t create->target, RESOLVE_REF_READING,\n+\t\t\t\t\t &new_oid, NULL) != NULL) {\n+\t\t\tlog.value.update.new_hash = new_oid.hash;\n+\t\t}\n+\n+\t\tif (log.value.update.old_hash != NULL ||\n+\t\t log.value.update.new_hash != NULL) {\n+\t\t\terr = reftable_writer_add_log(writer, &log);\n+\t\t}\n+\t\tlog.refname = NULL;\n+\t\tlog.value.update.message = NULL;\n+\t\tlog.value.update.old_hash = NULL;\n+\t\tlog.value.update.new_hash = NULL;\n+\t\tclear_reftable_log_record(&log);\n+\t}\n+\treturn err;\n+}\n+\n+static int git_reftable_create_symref(struct ref_store *ref_store,\n+\t\t\t\t const char *refname, const char *target,\n+\t\t\t\t const char *logmsg)\n+{\n+\tstruct git_reftable_ref_store *refs =\n+\t\t(struct git_reftable_ref_store *)ref_store;\n+\tstruct reftable_stack *stack = stack_for(refs, refname);\n+\tstruct write_create_symref_arg arg = { .refs = refs,\n+\t\t\t\t\t .stack = stack,\n+\t\t\t\t\t .refname = refname,\n+\t\t\t\t\t .target = target,\n+\t\t\t\t\t .logmsg = logmsg };\n+\tint err = refs->err;\n+\tif (err < 0) {\n+\t\tgoto done;\n+\t}\n+\terr = reftable_stack_reload(stack);\n+\tif (err) {\n+\t\tgoto done;\n+\t}\n+\terr = reftable_stack_add(stack, &write_create_symref_table, &arg);\n+done:\n+\tassert(err != REFTABLE_API_ERROR);\n+\treturn err;\n+}\n+\n+struct write_rename_arg {\n+\tstruct git_reftable_ref_store *refs;\n+\tstruct reftable_stack *stack;\n+\tconst char *oldname;\n+\tconst char *newname;\n+\tconst char *logmsg;\n+};\n+\n+static int write_rename_table(struct reftable_writer *writer, void *argv)\n+{\n+\tstruct write_rename_arg *arg = (struct write_rename_arg *)argv;\n+\tuint64_t ts = reftable_stack_next_update_index(arg->stack);\n+\tstruct reftable_ref_record old_ref = { NULL };\n+\tstruct reftable_ref_record new_ref = { NULL };\n+\tint err = reftable_stack_read_ref(arg->stack, arg->oldname, &old_ref);\n+\tstruct reftable_ref_record todo[2] = {\n+\t\t{\n+\t\t\t.refname = (char *)arg->oldname,\n+\t\t\t.update_index = ts,\n+\t\t\t.value_type = REFTABLE_REF_DELETION,\n+\t\t},\n+\t\told_ref,\n+\t};\n+\n+\tif (err) {\n+\t\tgoto done;\n+\t}\n+\n+\t/* git-branch supports a --force, but the check is not atomic. */\n+\tif (!reftable_stack_read_ref(arg->stack, arg->newname, &new_ref)) {\n+\t\tgoto done;\n+\t}\n+\n+\treftable_writer_set_limits(writer, ts, ts);\n+\n+\ttodo[1].update_index = ts;\n+\ttodo[1].refname = (char *)arg->newname;\n+\n+\terr = reftable_writer_add_refs(writer, todo, 2);\n+\tif (err < 0) {\n+\t\tgoto done;\n+\t}\n+\n+\tif (reftable_ref_record_val1(&old_ref)) {\n+\t\tuint8_t *val1 = reftable_ref_record_val1(&old_ref);\n+\t\tstruct reftable_log_record todo[2] = { { NULL } };\n+\t\tint firstlog = 0;\n+\t\tint lastlog = 2;\n+\t\tchar *msg = xstrndup(arg->logmsg,\n+\t\t\t\t arg->refs->write_options.block_size / 2);\n+\t\tfill_reftable_log_record(&todo[0]);\n+\t\tfill_reftable_log_record(&todo[1]);\n+\n+\t\ttodo[0].refname = xstrdup(arg->oldname);\n+\t\ttodo[0].update_index = ts;\n+\t\ttodo[0].value.update.message = msg;\n+\t\ttodo[0].value.update.old_hash = val1;\n+\t\ttodo[0].value.update.new_hash = NULL;\n+\n+\t\ttodo[1].refname = xstrdup(arg->newname);\n+\t\ttodo[1].update_index = ts;\n+\t\ttodo[1].value.update.old_hash = NULL;\n+\t\ttodo[1].value.update.new_hash = val1;\n+\t\ttodo[1].value.update.message = xstrdup(msg);\n+\n+\t\tif (!should_log(todo[1].refname)) {\n+\t\t\tlastlog--;\n+\t\t}\n+\t\tif (!should_log(todo[0].refname)) {\n+\t\t\tfirstlog++;\n+\t\t}\n+\t\terr = reftable_writer_add_logs(writer, &todo[firstlog],\n+\t\t\t\t\t lastlog - firstlog);\n+\n+\t\tclear_reftable_log_record(&todo[0]);\n+\t\tclear_reftable_log_record(&todo[1]);\n+\t\tif (err < 0) {\n+\t\t\tgoto done;\n+\t\t}\n+\n+\t} else {\n+\t\t/* XXX what should we write into the reflog if we rename a\n+\t\t * symref? */\n+\t}\n+\n+done:\n+\tassert(err != REFTABLE_API_ERROR);\n+\treftable_ref_record_release(&new_ref);\n+\treftable_ref_record_release(&old_ref);\n+\treturn err;\n+}\n+\n+static int write_copy_table(struct reftable_writer *writer, void *argv)\n+{\n+\tstruct write_rename_arg *arg = (struct write_rename_arg *)argv;\n+\tuint64_t ts = reftable_stack_next_update_index(arg->stack);\n+\tstruct reftable_ref_record old_ref = { NULL };\n+\tstruct reftable_ref_record new_ref = { NULL };\n+\tstruct reftable_log_record log = { NULL };\n+\tstruct reftable_iterator it = { NULL };\n+\tint err = reftable_stack_read_ref(arg->stack, arg->oldname, &old_ref);\n+\tif (err) {\n+\t\tgoto done;\n+\t}\n+\n+\t/* git-branch supports a --force, but the check is not atomic. */\n+\tif (reftable_stack_read_ref(arg->stack, arg->newname, &new_ref) == 0) {\n+\t\tgoto done;\n+\t}\n+\n+\treftable_writer_set_limits(writer, ts, ts);\n+\n+\tFREE_AND_NULL(old_ref.refname);\n+\told_ref.refname = xstrdup(arg->newname);\n+\told_ref.update_index = ts;\n+\terr = reftable_writer_add_ref(writer, &old_ref);\n+\tif (err < 0) {\n+\t\tgoto done;\n+\t}\n+\n+\t/* XXX this copies the entire reflog history. Is this the right\n+\t * semantics? should clear out existing reflog entries for oldname? */\n+\tif (!should_log(arg->newname))\n+\t\tgoto done;\n+\n+\terr = reftable_merged_table_seek_log(\n+\t\treftable_stack_merged_table(arg->stack), &it, arg->oldname);\n+\tif (err < 0) {\n+\t\tgoto done;\n+\t}\n+\twhile (1) {\n+\t\tint err = reftable_iterator_next_log(&it, &log);\n+\t\tif (err < 0) {\n+\t\t\tgoto done;\n+\t\t}\n+\n+\t\tif (err > 0 || strcmp(log.refname, arg->oldname)) {\n+\t\t\tbreak;\n+\t\t}\n+\t\tFREE_AND_NULL(log.refname);\n+\t\tlog.refname = xstrdup(arg->newname);\n+\t\treftable_writer_add_log(writer, &log);\n+\t\treftable_log_record_release(&log);\n+\t}\n+\n+done:\n+\tassert(err != REFTABLE_API_ERROR);\n+\treftable_ref_record_release(&new_ref);\n+\treftable_ref_record_release(&old_ref);\n+\treftable_log_record_release(&log);\n+\treftable_iterator_destroy(&it);\n+\treturn err;\n+}\n+\n+static int git_reftable_rename_ref(struct ref_store *ref_store,\n+\t\t\t\t const char *oldrefname,\n+\t\t\t\t const char *newrefname, const char *logmsg)\n+{\n+\tstruct git_reftable_ref_store *refs =\n+\t\t(struct git_reftable_ref_store *)ref_store;\n+\tstruct reftable_stack *stack = stack_for(refs, newrefname);\n+\tstruct write_rename_arg arg = {\n+\t\t.refs = refs,\n+\t\t.stack = stack,\n+\t\t.oldname = oldrefname,\n+\t\t.newname = newrefname,\n+\t\t.logmsg = logmsg,\n+\t};\n+\tint err = refs->err;\n+\tif (err < 0) {\n+\t\tgoto done;\n+\t}\n+\terr = reftable_stack_reload(stack);\n+\tif (err) {\n+\t\tgoto done;\n+\t}\n+\n+\terr = reftable_stack_add(stack, &write_rename_table, &arg);\n+done:\n+\tassert(err != REFTABLE_API_ERROR);\n+\treturn err;\n+}\n+\n+static int git_reftable_copy_ref(struct ref_store *ref_store,\n+\t\t\t\t const char *oldrefname, const char *newrefname,\n+\t\t\t\t const char *logmsg)\n+{\n+\tstruct git_reftable_ref_store *refs =\n+\t\t(struct git_reftable_ref_store *)ref_store;\n+\tstruct reftable_stack *stack = stack_for(refs, newrefname);\n+\tstruct write_rename_arg arg = {\n+\t\t.refs = refs,\n+\t\t.stack = stack,\n+\t\t.oldname = oldrefname,\n+\t\t.newname = newrefname,\n+\t\t.logmsg = logmsg,\n+\t};\n+\tint err = refs->err;\n+\tif (err < 0) {\n+\t\tgoto done;\n+\t}\n+\terr = reftable_stack_reload(stack);\n+\tif (err) {\n+\t\tgoto done;\n+\t}\n+\n+\terr = reftable_stack_add(stack, &write_copy_table, &arg);\n+done:\n+\tassert(err != REFTABLE_API_ERROR);\n+\treturn err;\n+}\n+\n+struct git_reftable_reflog_ref_iterator {\n+\tstruct ref_iterator base;\n+\tstruct reftable_iterator iter;\n+\tstruct reftable_log_record log;\n+\tstruct object_id oid;\n+\tstruct git_reftable_ref_store *refs;\n+\n+\t/* Used when iterating over worktree & main */\n+\tstruct reftable_merged_table *merged;\n+\tchar *last_name;\n+};\n+\n+static int\n+git_reftable_reflog_ref_iterator_advance(struct ref_iterator *ref_iterator)\n+{\n+\tstruct git_reftable_reflog_ref_iterator *ri =\n+\t\t(struct git_reftable_reflog_ref_iterator *)ref_iterator;\n+\n+\twhile (1) {\n+\t\tint flags = 0;\n+\t\tint err = reftable_iterator_next_log(&ri->iter, &ri->log);\n+\n+\t\tif (err > 0) {\n+\t\t\treturn ITER_DONE;\n+\t\t}\n+\t\tif (err < 0) {\n+\t\t\treturn ITER_ERROR;\n+\t\t}\n+\n+\t\tri->base.refname = ri->log.refname;\n+\t\tif (ri->last_name != NULL &&\n+\t\t !strcmp(ri->log.refname, ri->last_name)) {\n+\t\t\t/* we want the refnames that we have reflogs for, so we\n+\t\t\t * skip if we've already produced this name. This could\n+\t\t\t * be faster by seeking directly to\n+\t\t\t * reflog@update_index==0.\n+\t\t\t */\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tif (!refs_resolve_ref_unsafe(&ri->refs->base, ri->log.refname,\n+\t\t\t\t\t 0, &ri->oid, &flags)) {\n+\t\t\terror(\"bad ref for %s\", ri->log.refname);\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tfree(ri->last_name);\n+\t\tri->last_name = xstrdup(ri->log.refname);\n+\t\tri->base.oid = &ri->oid;\n+\t\tri->base.flags = flags;\n+\t\treturn ITER_OK;\n+\t}\n+}\n+\n+static int\n+git_reftable_reflog_ref_iterator_peel(struct ref_iterator *ref_iterator,\n+\t\t\t\t struct object_id *peeled)\n+{\n+\tBUG(\"not supported.\");\n+\treturn -1;\n+}\n+\n+static int\n+git_reftable_reflog_ref_iterator_abort(struct ref_iterator *ref_iterator)\n+{\n+\tstruct git_reftable_reflog_ref_iterator *ri =\n+\t\t(struct git_reftable_reflog_ref_iterator *)ref_iterator;\n+\treftable_log_record_release(&ri->log);\n+\treftable_iterator_destroy(&ri->iter);\n+\tif (ri->merged)\n+\t\treftable_merged_table_free(ri->merged);\n+\treturn 0;\n+}\n+\n+static struct ref_iterator_vtable git_reftable_reflog_ref_iterator_vtable = {\n+\tgit_reftable_reflog_ref_iterator_advance,\n+\tgit_reftable_reflog_ref_iterator_peel,\n+\tgit_reftable_reflog_ref_iterator_abort\n+};\n+\n+static struct ref_iterator *\n+git_reftable_reflog_iterator_begin(struct ref_store *ref_store)\n+{\n+\tstruct git_reftable_reflog_ref_iterator *ri = xcalloc(1, sizeof(*ri));\n+\tstruct git_reftable_ref_store *refs =\n+\t\t(struct git_reftable_ref_store *)ref_store;\n+\n+\tri->refs = refs;\n+\tif (refs->worktree_stack == NULL) {\n+\t\tstruct reftable_stack *stack = refs->main_stack;\n+\t\tstruct reftable_merged_table *mt =\n+\t\t\treftable_stack_merged_table(stack);\n+\t\tint err = reftable_merged_table_seek_log(mt, &ri->iter, \"\");\n+\t\tif (err < 0) {\n+\t\t\tfree(ri);\n+\t\t\t/* XXX how to handle errors in iterator_begin()? */\n+\t\t\treturn NULL;\n+\t\t}\n+\t} else {\n+\t\tstruct reftable_merged_table *mt1 =\n+\t\t\treftable_stack_merged_table(refs->main_stack);\n+\t\tstruct reftable_merged_table *mt2 =\n+\t\t\treftable_stack_merged_table(refs->worktree_stack);\n+\t\tstruct reftable_table *tabs =\n+\t\t\txcalloc(2, sizeof(struct reftable_table));\n+\t\tint err = 0;\n+\t\treftable_table_from_merged_table(&tabs[0], mt1);\n+\t\treftable_table_from_merged_table(&tabs[1], mt2);\n+\t\terr = reftable_new_merged_table(&ri->merged, tabs, 2,\n+\t\t\t\t\t\tthe_hash_algo->format_id);\n+\t\tif (err < 0) {\n+\t\t\tfree(tabs);\n+\t\t\t/* XXX idem. */\n+\t\t\treturn NULL;\n+\t\t}\n+\t\terr = reftable_merged_table_seek_log(ri->merged, &ri->iter, \"\");\n+\t\tif (err < 0) {\n+\t\t\treturn NULL;\n+\t\t}\n+\t}\n+\tbase_ref_iterator_init(&ri->base,\n+\t\t\t &git_reftable_reflog_ref_iterator_vtable, 1);\n+\tri->base.oid = &ri->oid;\n+\n+\treturn (struct ref_iterator *)ri;\n+}\n+\n+static int git_reftable_for_each_reflog_ent_newest_first(\n+\tstruct ref_store *ref_store, const char *refname, each_reflog_ent_fn fn,\n+\tvoid *cb_data)\n+{\n+\tstruct reftable_iterator it = { NULL };\n+\tstruct git_reftable_ref_store *refs =\n+\t\t(struct git_reftable_ref_store *)ref_store;\n+\tstruct reftable_stack *stack = stack_for(refs, refname);\n+\tstruct reftable_merged_table *mt = NULL;\n+\tint err = 0;\n+\tstruct reftable_log_record log = { NULL };\n+\n+\tif (refs->err < 0) {\n+\t\treturn refs->err;\n+\t}\n+\trefname = bare_ref_name(refname);\n+\n+\tmt = reftable_stack_merged_table(stack);\n+\terr = reftable_merged_table_seek_log(mt, &it, refname);\n+\twhile (err == 0) {\n+\t\tstruct object_id old_oid;\n+\t\tstruct object_id new_oid;\n+\t\tconst char *full_committer = \"\";\n+\n+\t\terr = reftable_iterator_next_log(&it, &log);\n+\t\tif (err > 0) {\n+\t\t\terr = 0;\n+\t\t\tbreak;\n+\t\t}\n+\t\tif (err < 0) {\n+\t\t\tbreak;\n+\t\t}\n+\n+\t\tif (strcmp(log.refname, refname)) {\n+\t\t\tbreak;\n+\t\t}\n+\n+\t\toidread(&old_oid, log.value.update.old_hash);\n+\t\toidread(&new_oid, log.value.update.new_hash);\n+\n+\t\tif (is_null_oid(&old_oid) && is_null_oid(&new_oid)) {\n+\t\t\t/* placeholder for existence. */\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tfull_committer = fmt_ident(log.value.update.name,\n+\t\t\t\t\t log.value.update.email,\n+\t\t\t\t\t WANT_COMMITTER_IDENT,\n+\t\t\t\t\t /*date*/ NULL, IDENT_NO_DATE);\n+\t\terr = fn(&old_oid, &new_oid, full_committer,\n+\t\t\t log.value.update.time, log.value.update.tz_offset,\n+\t\t\t log.value.update.message, cb_data);\n+\t\tif (err)\n+\t\t\tbreak;\n+\t}\n+\n+\treftable_log_record_release(&log);\n+\treftable_iterator_destroy(&it);\n+\treturn err;\n+}\n+\n+static int git_reftable_for_each_reflog_ent_oldest_first(\n+\tstruct ref_store *ref_store, const char *refname, each_reflog_ent_fn fn,\n+\tvoid *cb_data)\n+{\n+\tstruct reftable_iterator it = { NULL };\n+\tstruct git_reftable_ref_store *refs =\n+\t\t(struct git_reftable_ref_store *)ref_store;\n+\tstruct reftable_stack *stack = stack_for(refs, refname);\n+\tstruct reftable_merged_table *mt = NULL;\n+\tstruct reftable_log_record *logs = NULL;\n+\tint cap = 0;\n+\tint len = 0;\n+\tint err = 0;\n+\tint i = 0;\n+\n+\tif (refs->err < 0) {\n+\t\treturn refs->err;\n+\t}\n+\trefname = bare_ref_name(refname);\n+\tmt = reftable_stack_merged_table(stack);\n+\terr = reftable_merged_table_seek_log(mt, &it, refname);\n+\n+\twhile (err == 0) {\n+\t\tstruct reftable_log_record log = { NULL };\n+\t\terr = reftable_iterator_next_log(&it, &log);\n+\t\tif (err > 0) {\n+\t\t\terr = 0;\n+\t\t\tbreak;\n+\t\t}\n+\t\tif (err < 0) {\n+\t\t\tbreak;\n+\t\t}\n+\n+\t\tif (strcmp(log.refname, refname)) {\n+\t\t\tbreak;\n+\t\t}\n+\n+\t\tif (len == cap) {\n+\t\t\tcap = 2 * cap + 1;\n+\t\t\tlogs = realloc(logs, cap * sizeof(*logs));\n+\t\t}\n+\n+\t\tlogs[len++] = log;\n+\t}\n+\n+\tfor (i = len; i--;) {\n+\t\tstruct reftable_log_record *log = &logs[i];\n+\t\tstruct object_id old_oid;\n+\t\tstruct object_id new_oid;\n+\t\tconst char *full_committer = \"\";\n+\n+\t\toidread(&old_oid, log->value.update.old_hash);\n+\t\toidread(&new_oid, log->value.update.new_hash);\n+\n+\t\tif (is_null_oid(&old_oid) && is_null_oid(&new_oid)) {\n+\t\t\t/* placeholder for existence. */\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tfull_committer = fmt_ident(log->value.update.name,\n+\t\t\t\t\t log->value.update.email,\n+\t\t\t\t\t WANT_COMMITTER_IDENT, NULL,\n+\t\t\t\t\t IDENT_NO_DATE);\n+\t\terr = fn(&old_oid, &new_oid, full_committer,\n+\t\t\t log->value.update.time, log->value.update.tz_offset,\n+\t\t\t log->value.update.message, cb_data);\n+\t\tif (err) {\n+\t\t\tbreak;\n+\t\t}\n+\t}\n+\n+\tfor (i = 0; i < len; i++) {\n+\t\treftable_log_record_release(&logs[i]);\n+\t}\n+\tfree(logs);\n+\n+\treftable_iterator_destroy(&it);\n+\treturn err;\n+}\n+\n+static int git_reftable_reflog_exists(struct ref_store *ref_store,\n+\t\t\t\t const char *refname)\n+{\n+\tstruct reftable_iterator it = { NULL };\n+\tstruct git_reftable_ref_store *refs =\n+\t\t(struct git_reftable_ref_store *)ref_store;\n+\tstruct reftable_stack *stack = stack_for(refs, refname);\n+\tstruct reftable_merged_table *mt = reftable_stack_merged_table(stack);\n+\tstruct reftable_log_record log = { NULL };\n+\tint err = refs->err;\n+\n+\tif (err < 0) {\n+\t\tgoto done;\n+\t}\n+\n+\trefname = bare_ref_name(refname);\n+\terr = reftable_merged_table_seek_log(mt, &it, refname);\n+\tif (err) {\n+\t\tgoto done;\n+\t}\n+\terr = reftable_iterator_next_log(&it, &log);\n+\tif (err) {\n+\t\tgoto done;\n+\t}\n+\n+\tif (strcmp(log.refname, refname)) {\n+\t\terr = 1;\n+\t}\n+\n+done:\n+\treftable_iterator_destroy(&it);\n+\treftable_log_record_release(&log);\n+\treturn !err;\n+}\n+\n+struct write_reflog_existence_arg {\n+\tstruct git_reftable_ref_store *refs;\n+\tconst char *refname;\n+\tstruct reftable_stack *stack;\n+};\n+\n+static int write_reflog_existence_table(struct reftable_writer *writer,\n+\t\t\t\t\tvoid *argv)\n+{\n+\tstruct write_reflog_existence_arg *arg =\n+\t\t(struct write_reflog_existence_arg *)argv;\n+\tuint64_t ts = reftable_stack_next_update_index(arg->stack);\n+\tstruct reftable_log_record log = { NULL };\n+\n+\tint err = reftable_stack_read_log(arg->stack, arg->refname, &log);\n+\tif (err <= 0) {\n+\t\tgoto done;\n+\t}\n+\n+\treftable_writer_set_limits(writer, ts, ts);\n+\n+\tlog.refname = (char *)arg->refname;\n+\tlog.update_index = ts;\n+\tlog.value_type = REFTABLE_LOG_UPDATE;\n+\terr = reftable_writer_add_log(writer, &log);\n+\n+\t/* field is not malloced */\n+\tlog.refname = NULL;\n+\n+done:\n+\tassert(err != REFTABLE_API_ERROR);\n+\treftable_log_record_release(&log);\n+\treturn err;\n+}\n+\n+static int git_reftable_create_reflog(struct ref_store *ref_store,\n+\t\t\t\t const char *refname,\n+\t\t\t\t struct strbuf *errmsg)\n+{\n+\tstruct git_reftable_ref_store *refs =\n+\t\t(struct git_reftable_ref_store *)ref_store;\n+\tstruct reftable_stack *stack = stack_for(refs, refname);\n+\tstruct write_reflog_existence_arg arg = {\n+\t\t.refs = refs,\n+\t\t.stack = stack,\n+\t\t.refname = refname,\n+\t};\n+\tint err = refs->err;\n+\tif (err < 0) {\n+\t\tgoto done;\n+\t}\n+\n+\terr = reftable_stack_reload(stack);\n+\tif (err) {\n+\t\tgoto done;\n+\t}\n+\n+\terr = reftable_stack_add(stack, &write_reflog_existence_table, &arg);\n+\n+done:\n+\treturn err;\n+}\n+\n+struct write_reflog_delete_arg {\n+\tstruct reftable_stack *stack;\n+\tconst char *refname;\n+};\n+\n+static int write_reflog_delete_table(struct reftable_writer *writer, void *argv)\n+{\n+\tstruct write_reflog_delete_arg *arg = argv;\n+\tstruct reftable_merged_table *mt =\n+\t\treftable_stack_merged_table(arg->stack);\n+\tstruct reftable_log_record log = { NULL };\n+\tstruct reftable_iterator it = { NULL };\n+\tuint64_t ts = reftable_stack_next_update_index(arg->stack);\n+\tint err = reftable_merged_table_seek_log(mt, &it, arg->refname);\n+\n+\treftable_writer_set_limits(writer, ts, ts);\n+\twhile (err == 0) {\n+\t\tstruct reftable_log_record tombstone = {\n+\t\t\t.refname = (char *)arg->refname,\n+\t\t\t.update_index = REFTABLE_LOG_DELETION,\n+\t\t};\n+\t\terr = reftable_iterator_next_log(&it, &log);\n+\t\tif (err > 0) {\n+\t\t\terr = 0;\n+\t\t\tbreak;\n+\t\t}\n+\n+\t\tif (err < 0 || strcmp(log.refname, arg->refname)) {\n+\t\t\tbreak;\n+\t\t}\n+\t\tif (log.value_type == REFTABLE_LOG_DELETION)\n+\t\t\tcontinue;\n+\n+\t\ttombstone.update_index = log.update_index;\n+\t\terr = reftable_writer_add_log(writer, &tombstone);\n+\t}\n+\n+\treftable_log_record_release(&log);\n+\treturn err;\n+}\n+\n+static int git_reftable_delete_reflog(struct ref_store *ref_store,\n+\t\t\t\t const char *refname)\n+{\n+\tstruct git_reftable_ref_store *refs =\n+\t\t(struct git_reftable_ref_store *)ref_store;\n+\tstruct reftable_stack *stack = stack_for(refs, refname);\n+\tstruct write_reflog_delete_arg arg = {\n+\t\t.stack = stack,\n+\t\t.refname = refname,\n+\t};\n+\tint err = reftable_stack_add(stack, &write_reflog_delete_table, &arg);\n+\tassert(err != REFTABLE_API_ERROR);\n+\treturn err;\n+}\n+\n+struct reflog_expiry_arg {\n+\tstruct reftable_stack *stack;\n+\tstruct reftable_log_record *records;\n+\tint len;\n+\tconst char *refname;\n+};\n+\n+static int write_reflog_expiry_table(struct reftable_writer *writer, void *argv)\n+{\n+\tstruct reflog_expiry_arg *arg = (struct reflog_expiry_arg *)argv;\n+\tuint64_t ts = reftable_stack_next_update_index(arg->stack);\n+\tint i = 0;\n+\tint live_records = 0;\n+\tuint64_t max_ts = 0;\n+\tfor (i = 0; i < arg->len; i++) {\n+\t\tif (arg->records[i].value_type == REFTABLE_LOG_UPDATE)\n+\t\t\tlive_records++;\n+\n+\t\tif (max_ts < arg->records[i].update_index)\n+\t\t\tmax_ts = arg->records[i].update_index;\n+\t}\n+\n+\treftable_writer_set_limits(writer, ts, ts);\n+\tif (live_records == 0) {\n+\t\tstruct reftable_log_record log = {\n+\t\t\t.refname = (char *)arg->refname,\n+\t\t\t.update_index = max_ts + 1,\n+\t\t\t.value_type = REFTABLE_LOG_UPDATE,\n+\t\t\t/* existence dummy has null new/old oid */\n+\t\t};\n+\t\tint err;\n+\t\tif (log.update_index < ts)\n+\t\t\tlog.update_index = ts;\n+\n+\t\terr = reftable_writer_add_log(writer, &log);\n+\t\tif (err) {\n+\t\t\treturn err;\n+\t\t}\n+\t}\n+\n+\tfor (i = 0; i < arg->len; i++) {\n+\t\tint err = reftable_writer_add_log(writer, &arg->records[i]);\n+\t\tif (err) {\n+\t\t\treturn err;\n+\t\t}\n+\t}\n+\treturn 0;\n+}\n+\n+static int git_reftable_reflog_expire(\n+\tstruct ref_store *ref_store, const char *refname, unsigned int flags,\n+\treflog_expiry_prepare_fn prepare_fn,\n+\treflog_expiry_should_prune_fn should_prune_fn,\n+\treflog_expiry_cleanup_fn cleanup_fn, void *policy_cb_data)\n+{\n+\t/*\n+\t For log expiry, we write tombstones in place of the expired entries,\n+\t This means that the entries are still retrievable by delving into the\n+\t stack, and expiring entries paradoxically takes extra memory.\n+\n+\t This memory is only reclaimed when some operation issues a\n+\t git_reftable_pack_refs(), which will compact the entire stack and get\n+\t rid of deletion entries.\n+\n+\t It would be better if the refs backend supported an API that sets a\n+\t criterion for all refs, passing the criterion to pack_refs().\n+\n+\t On the plus side, because we do the expiration per ref, we can easily\n+\t insert the reflog existence dummies.\n+\t*/\n+\tstruct git_reftable_ref_store *refs =\n+\t\t(struct git_reftable_ref_store *)ref_store;\n+\tstruct reftable_stack *stack = stack_for(refs, refname);\n+\tstruct reftable_merged_table *mt = NULL;\n+\tstruct reflog_expiry_arg arg = {\n+\t\t.stack = stack,\n+\t\t.refname = refname,\n+\t};\n+\tstruct reftable_log_record *logs = NULL;\n+\tstruct reftable_log_record *rewritten = NULL;\n+\tstruct reftable_ref_record ref_record = { NULL };\n+\tint logs_len = 0;\n+\tint logs_cap = 0;\n+\tint i = 0;\n+\tuint8_t *last_hash = NULL;\n+\tstruct reftable_iterator it = { NULL };\n+\tstruct reftable_addition *add = NULL;\n+\tint err = 0;\n+\tstruct object_id oid = { 0 };\n+\tif (refs->err < 0) {\n+\t\treturn refs->err;\n+\t}\n+\terr = reftable_stack_reload(stack);\n+\tif (err) {\n+\t\tgoto done;\n+\t}\n+\n+\tmt = reftable_stack_merged_table(stack);\n+\terr = reftable_merged_table_seek_log(mt, &it, refname);\n+\tif (err < 0) {\n+\t\tgoto done;\n+\t}\n+\n+\terr = reftable_stack_new_addition(&add, stack);\n+\tif (err) {\n+\t\tgoto done;\n+\t}\n+\tif (!reftable_stack_read_ref(stack, refname, &ref_record)) {\n+\t\tuint8_t *hash = reftable_ref_record_val1(&ref_record);\n+\t\tif (hash)\n+\t\t\toidread(&oid, hash);\n+\t}\n+\n+\tprepare_fn(refname, &oid, policy_cb_data);\n+\twhile (1) {\n+\t\tstruct reftable_log_record log = { NULL };\n+\t\tint err = reftable_iterator_next_log(&it, &log);\n+\t\tif (err < 0) {\n+\t\t\tgoto done;\n+\t\t}\n+\n+\t\tif (err > 0 || strcmp(log.refname, refname)) {\n+\t\t\tbreak;\n+\t\t}\n+\n+\t\tif (logs_len >= logs_cap) {\n+\t\t\tint new_cap = logs_cap * 2 + 1;\n+\t\t\tlogs = realloc(logs, new_cap * sizeof(*logs));\n+\t\t\tlogs_cap = new_cap;\n+\t\t}\n+\t\tlogs[logs_len++] = log;\n+\t}\n+\n+\trewritten = calloc(logs_len, sizeof(*rewritten));\n+\tfor (i = logs_len - 1; i >= 0; i--) {\n+\t\tstruct object_id ooid;\n+\t\tstruct object_id noid;\n+\t\tstruct reftable_log_record *dest = &rewritten[i];\n+\n+\t\t*dest = logs[i];\n+\t\toidread(&ooid, logs[i].value.update.old_hash);\n+\t\toidread(&noid, logs[i].value.update.new_hash);\n+\n+\t\tif (should_prune_fn(&ooid, &noid, logs[i].value.update.email,\n+\t\t\t\t (timestamp_t)logs[i].value.update.time,\n+\t\t\t\t logs[i].value.update.tz_offset,\n+\t\t\t\t logs[i].value.update.message,\n+\t\t\t\t policy_cb_data)) {\n+\t\t\tdest->value_type = REFTABLE_LOG_DELETION;\n+\t\t} else {\n+\t\t\tif ((flags & EXPIRE_REFLOGS_REWRITE) &&\n+\t\t\t last_hash != NULL) {\n+\t\t\t\tdest->value.update.old_hash = last_hash;\n+\t\t\t}\n+\t\t\tlast_hash = logs[i].value.update.new_hash;\n+\t\t}\n+\t}\n+\n+\targ.records = rewritten;\n+\targ.len = logs_len;\n+\terr = reftable_addition_add(add, &write_reflog_expiry_table, &arg);\n+\tif (err < 0) {\n+\t\tgoto done;\n+\t}\n+\n+\tif (!(flags & EXPIRE_REFLOGS_DRY_RUN)) {\n+\t\t/* future improvement: we could skip writing records that were\n+\t\t * not changed. */\n+\t\terr = reftable_addition_commit(add);\n+\t}\n+\n+done:\n+\tif (add) {\n+\t\tcleanup_fn(policy_cb_data);\n+\t}\n+\tassert(err != REFTABLE_API_ERROR);\n+\treftable_addition_destroy(add);\n+\tfor (i = 0; i < logs_len; i++)\n+\t\treftable_log_record_release(&logs[i]);\n+\tfree(logs);\n+\tfree(rewritten);\n+\treftable_iterator_destroy(&it);\n+\treturn err;\n+}\n+\n+static int git_reftable_read_raw_ref(struct ref_store *ref_store,\n+\t\t\t\t const char *refname, struct object_id *oid,\n+\t\t\t\t struct strbuf *referent,\n+\t\t\t\t unsigned int *type, int *failure_errno)\n+{\n+\tstruct git_reftable_ref_store *refs =\n+\t\t(struct git_reftable_ref_store *)ref_store;\n+\tstruct reftable_stack *stack = stack_for(refs, refname);\n+\tstruct reftable_ref_record ref = { NULL };\n+\tint err = 0;\n+\n+\trefname = bare_ref_name(refname); /* XXX - in which other cases should\n+\t\t\t\t\t we do this? */\n+\tif (refs->err < 0) {\n+\t\treturn refs->err;\n+\t}\n+\n+\t/* This is usually not needed, but Git doesn't signal to ref backend if\n+\t a subprocess updated the ref DB. So we always check.\n+\t*/\n+\terr = reftable_stack_reload(stack);\n+\tif (err) {\n+\t\tgoto done;\n+\t}\n+\n+\terr = reftable_stack_read_ref(stack, refname, &ref);\n+\tif (err > 0) {\n+\t\t*failure_errno = ENOENT;\n+\t\terr = -1;\n+\t\tgoto done;\n+\t}\n+\tif (err < 0) {\n+\t\tgoto done;\n+\t}\n+\n+\tif (ref.value_type == REFTABLE_REF_SYMREF) {\n+\t\tstrbuf_reset(referent);\n+\t\tstrbuf_addstr(referent, ref.value.symref);\n+\t\t*type |= REF_ISSYMREF;\n+\t} else if (reftable_ref_record_val1(&ref) != NULL) {\n+\t\toidread(oid, reftable_ref_record_val1(&ref));\n+\t} else {\n+\t\t/* We got a tombstone, which should not happen. */\n+\t\tBUG(\"Got reftable_ref_record with value type %d\",\n+\t\t ref.value_type);\n+\t}\n+\n+done:\n+\tassert(err != REFTABLE_API_ERROR);\n+\treftable_ref_record_release(&ref);\n+\treturn err;\n+}\n+\n+struct ref_storage_be refs_be_reftable = {\n+\t&refs_be_files,\n+\t\"reftable\",\n+\tgit_reftable_ref_store_create,\n+\tgit_reftable_init_db,\n+\tgit_reftable_transaction_prepare,\n+\tgit_reftable_transaction_finish,\n+\tgit_reftable_transaction_abort,\n+\tgit_reftable_transaction_initial_commit,\n+\n+\tgit_reftable_pack_refs,\n+\tgit_reftable_create_symref,\n+\tgit_reftable_delete_refs,\n+\tgit_reftable_rename_ref,\n+\tgit_reftable_copy_ref,\n+\n+\tgit_reftable_ref_iterator_begin,\n+\tgit_reftable_read_raw_ref,\n+\n+\tgit_reftable_reflog_iterator_begin,\n+\tgit_reftable_for_each_reflog_ent_oldest_first,\n+\tgit_reftable_for_each_reflog_ent_newest_first,\n+\tgit_reftable_reflog_exists,\n+\tgit_reftable_create_reflog,\n+\tgit_reftable_delete_reflog,\n+\tgit_reftable_reflog_expire,\n+};\ndiff --git a/repository.c b/repository.c\nindex dddee32258f..3aaae1f43cd 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -185,6 +185,8 @@ int repo_init(struct repository *repo,\n \tif (worktree)\n \t\trepo_set_worktree(repo, worktree);\n \n+\trepo->ref_storage_format = xstrdup_or_null(format.ref_storage);\n+\n \tclear_repository_format(&format);\n \treturn 0;\n \ndiff --git a/repository.h b/repository.h\nindex 4e46ecdca56..ceb7f55a2c7 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -82,6 +82,9 @@ struct repository {\n \t */\n \tstruct ref_store *refs_private;\n \n+\t/* The format to use for the ref database. */\n+\tchar *ref_storage_format;\n+\n \t/*\n \t * Contains path to often used file names.\n \t */\ndiff --git a/setup.c b/setup.c\nindex 04ce33cdcd4..79f725eae8b 100644\n--- a/setup.c\n+++ b/setup.c\n@@ -563,6 +563,9 @@ static enum extension_result handle_extension(const char *var,\n \t\t\t\t \"extensions.objectformat\", value);\n \t\tdata->hash_algo = format;\n \t\treturn EXTENSION_OK;\n+\t} else if (!strcmp(ext, \"refstorage\")) {\n+\t\tdata->ref_storage = xstrdup(value);\n+\t\treturn EXTENSION_OK;\n \t}\n \treturn EXTENSION_UNKNOWN;\n }\n@@ -713,6 +716,7 @@ void clear_repository_format(struct repository_format *format)\n \tstring_list_clear(&format->v1_only_extensions, 0);\n \tfree(format->work_tree);\n \tfree(format->partial_clone);\n+\tfree(format->ref_storage);\n \tinit_repository_format(format);\n }\n \n@@ -1377,6 +1381,8 @@ const char *setup_git_directory_gently(int *nongit_ok)\n \t\t\tthe_repository->repository_format_partial_clone =\n \t\t\t\trepo_fmt.partial_clone;\n \t\t\trepo_fmt.partial_clone = NULL;\n+\t\t\tthe_repository->ref_storage_format =\n+\t\t\t\txstrdup_or_null(repo_fmt.ref_storage);\n \t\t}\n \t}\n \t/*\n@@ -1465,6 +1471,8 @@ void check_repository_format(struct repository_format *fmt)\n \trepo_set_hash_algo(the_repository, fmt->hash_algo);\n \tthe_repository->repository_format_partial_clone =\n \t\txstrdup_or_null(fmt->partial_clone);\n+\t/* XXX why is repo->ref_storage_format set in multiple places?! */\n+\tthe_repository->ref_storage_format = xstrdup_or_null(fmt->ref_storage);\n \tclear_repository_format(&repo_fmt);\n }\n \ndiff --git a/t/t0031-reftable.sh b/t/t0031-reftable.sh\nnew file mode 100755\nindex 00000000000..12cd46eec85\n--- /dev/null\n+++ b/t/t0031-reftable.sh\n@@ -0,0 +1,380 @@\n+#!/bin/sh\n+#\n+# Copyright (c) 2020 Google LLC\n+#\n+\n+test_description='reftable basics'\n+\n+. ./test-lib.sh\n+\n+INVALID_SHA1=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\n+\n+git_init () {\n+\tgit init -b primary \"$@\"\n+}\n+\n+initialize () {\n+\trm -rf .git &&\n+\t(GIT_TEST_REFTABLE=1; export GIT_TEST_REFTABLE; git_init) &&\n+\tmv .git/hooks .git/hooks-disabled\n+}\n+\n+write_script fake_editor <<\\EOF\n+echo \"$MSG\" >\"$1\"\n+echo \"$MSG\" >&2\n+EOF\n+GIT_EDITOR=./fake_editor\n+export GIT_EDITOR\n+\n+\n+test_expect_success 'using reftable' '\n+\tinitialize &&\n+\ttest -d .git/reftable &&\n+\ttest -f .git/reftable/tables.list\n+'\n+\n+test_expect_success 'read existing old OID if REF_HAVE_OLD is not set' '\n+\tinitialize &&\n+\ttest_commit 1st &&\n+\ttest_commit 2nd &&\n+\tMSG=b4 git notes add &&\n+\tMSG=b3 git notes edit &&\n+\techo b4 >expect &&\n+\tgit notes --ref commits@{1} show >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'git reflog delete' '\n+\tinitialize &&\n+\ttest_commit file &&\n+\ttest_commit file2 &&\n+\ttest_commit file3 &&\n+\ttest_commit file4 &&\n+\tgit reflog delete HEAD@{1} &&\n+\tgit reflog > output &&\n+\t! grep file3 output\n+'\n+\n+test_expect_success 'branch -D delete nonexistent branch' '\n+\tinitialize &&\n+\ttest_commit file &&\n+\ttest_must_fail git branch -D ../../my-private-file\n+'\n+\n+test_expect_success 'branch copy' '\n+\tinitialize &&\n+\ttest_commit file1 &&\n+\ttest_commit file2 &&\n+\tgit branch src &&\n+\tgit reflog src > expect &&\n+\tgit branch -c src dst &&\n+\tgit reflog dst | sed \"s/dst/src/g\" > actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'update-ref on corrupted data' '\n+\tinitialize &&\n+\ttest_commit file1 &&\n+\tOLD_SHA1=$(git rev-parse HEAD) &&\n+\ttest_commit file2 &&\n+\tls -l .git/reftable &&\n+\tfor f in .git/reftable/*.ref\n+\tdo\n+\t\t>$f\n+\tdone &&\n+\ttest_must_fail git update-ref refs/heads/main $OLD_SHA1\n+'\n+\n+test_expect_success 'git stash' '\n+\tinitialize &&\n+\ttest_commit file &&\n+\ttouch actual expected &&\n+\tgit -c status.showStash=true status >expected &&\n+\techo hoi >> file.t &&\n+\tgit stash push -m stashed &&\n+\tgit stash clear &&\n+\tgit -c status.showStash=true status >actual &&\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'rename branch' '\n+\tinitialize &&\n+\tgit symbolic-ref HEAD refs/heads/before &&\n+\ttest_commit file &&\n+\tgit show-ref | sed s/before/after/g > expected &&\n+\tgit branch -M after &&\n+\tgit show-ref > actual &&\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'SHA256 support, env' '\n+\trm -rf .git &&\n+\t(GIT_DEFAULT_HASH=sha256 && export GIT_DEFAULT_HASH &&\n+\t\t\t\t GIT_TEST_REFTABLE=1 git_init) &&\n+\tmv .git/hooks .git/hooks-disabled &&\n+\ttest_commit file\n+'\n+\n+test_expect_success 'SHA256 support, option' '\n+\trm -rf .git &&\n+\t(GIT_TEST_REFTABLE=1 git_init --object-format=sha256) &&\n+\tmv .git/hooks .git/hooks-disabled &&\n+\ttest_commit file\n+'\n+\n+test_expect_success 'delete ref' '\n+\tinitialize &&\n+\ttest_commit file &&\n+\tSHA=$(git show-ref -s --verify HEAD) &&\n+\ttest_write_lines \"$SHA refs/heads/primary\" \"$SHA refs/tags/file\" >expect &&\n+\tgit show-ref >actual &&\n+\t! git update-ref -d refs/tags/file $INVALID_SHA1 &&\n+\ttest_cmp expect actual &&\n+\tgit update-ref -d refs/tags/file $SHA &&\n+\ttest_write_lines \"$SHA refs/heads/primary\" >expect &&\n+\tgit show-ref >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'clone calls transaction_initial_commit' '\n+\ttest_commit message1 file1 &&\n+\tgit clone . cloned &&\n+\t(test -f cloned/file1 || echo \"Fixme.\")\n+'\n+\n+test_expect_success 'basic operation of reftable storage: commit, show-ref' '\n+\tinitialize &&\n+\ttest_commit file &&\n+\ttest_write_lines refs/heads/primary refs/tags/file >expect &&\n+\tgit show-ref &&\n+\tgit show-ref | cut -f2 -d\" \" >actual &&\n+\ttest_cmp actual expect\n+'\n+\n+test_expect_success 'reflog, repack' '\n+\tinitialize &&\n+\tfor count in $(test_seq 1 10)\n+\tdo\n+\t\ttest_commit \"number $count\" file.t $count number-$count ||\n+\t\treturn 1\n+\tdone &&\n+\tgit pack-refs &&\n+\tls -1 .git/reftable >table-files &&\n+\ttest_line_count = 2 table-files &&\n+\tgit reflog refs/heads/primary >output &&\n+\ttest_line_count = 10 output &&\n+\tgrep \"commit (initial): number 1\" output &&\n+\tgrep \"commit: number 10\" output &&\n+\tgit gc &&\n+\tgit reflog refs/heads/primary >output &&\n+\ttest_line_count = 0 output\n+'\n+\n+test_expect_success 'branch switch in reflog output' '\n+\tinitialize &&\n+\ttest_commit file1 &&\n+\tgit checkout -b branch1 &&\n+\ttest_commit file2 &&\n+\tgit checkout -b branch2 &&\n+\tgit switch - &&\n+\tgit rev-parse --symbolic-full-name HEAD >actual &&\n+\techo refs/heads/branch1 >expect &&\n+\ttest_cmp actual expect\n+'\n+\n+\n+# This matches show-ref's output\n+print_ref() {\n+\techo \"$(git rev-parse \"$1\") $1\"\n+}\n+\n+test_expect_success 'peeled tags are stored' '\n+\tinitialize &&\n+\ttest_commit file &&\n+\tgit tag -m \"annotated tag\" test_tag HEAD &&\n+\t{\n+\t\tprint_ref \"refs/heads/primary\" &&\n+\t\tprint_ref \"refs/tags/file\" &&\n+\t\tprint_ref \"refs/tags/test_tag\" &&\n+\t\tprint_ref \"refs/tags/test_tag^{}\"\n+\t} >expect &&\n+\tgit show-ref -d >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'show-ref works on fresh repo' '\n+\tinitialize &&\n+\trm -rf .git &&\n+\t(GIT_TEST_REFTABLE=1 git_init) &&\n+\t>expect &&\n+\t! git show-ref >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'checkout unborn branch' '\n+\tinitialize &&\n+\tgit checkout -b primary\n+'\n+\n+\n+test_expect_success 'dir/file conflict' '\n+\tinitialize &&\n+\ttest_commit file &&\n+\t! git branch primary/forbidden\n+'\n+\n+\n+test_expect_success 'do not clobber existing repo' '\n+\trm -rf .git &&\n+\tgit_init &&\n+\tcat .git/HEAD >expect &&\n+\ttest_commit file &&\n+\t(GIT_TEST_REFTABLE=1 git_init || true) &&\n+\tcat .git/HEAD >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+# cherry-pick uses a pseudo ref.\n+test_expect_success 'pseudo refs' '\n+\tinitialize &&\n+\ttest_commit message1 file1 &&\n+\ttest_commit message2 file2 &&\n+\tgit branch source &&\n+\tgit checkout HEAD^ &&\n+\ttest_commit message3 file3 &&\n+\tgit cherry-pick source &&\n+\ttest -f file2\n+'\n+\n+# cherry-pick uses a pseudo ref.\n+test_expect_success 'rebase' '\n+\tinitialize &&\n+\ttest_commit message1 file1 &&\n+\ttest_commit message2 file2 &&\n+\tgit branch source &&\n+\tgit checkout HEAD^ &&\n+\ttest_commit message3 file3 &&\n+\tgit rebase source &&\n+\ttest -f file2\n+'\n+\n+test_expect_success 'worktrees' '\n+\t(GIT_TEST_REFTABLE=1 git_init start) &&\n+\t(cd start && test_commit file1 && git checkout -b branch1 &&\n+\tgit checkout -b branch2 &&\n+\tgit worktree add ../wt\n+\t) &&\n+\tcd wt &&\n+\tgit checkout branch1 &&\n+\tgit branch\n+'\n+\n+test_expect_success 'worktrees 2' '\n+\tinitialize &&\n+\ttest_commit file1 &&\n+\tmkdir existing_empty &&\n+\tgit worktree add --detach existing_empty primary\n+'\n+\n+test_expect_success 'FETCH_HEAD' '\n+\tinitialize &&\n+\ttest_commit one &&\n+\t(git_init sub && cd sub && test_commit two) &&\n+\tgit --git-dir sub/.git rev-parse HEAD >expect &&\n+\tgit fetch sub &&\n+\tgit checkout FETCH_HEAD &&\n+\tgit rev-parse HEAD >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+. \"$TEST_DIRECTORY\"/lib-httpd.sh\n+start_httpd\n+\n+REPO=\"$HTTPD_DOCUMENT_ROOT_PATH/repo\"\n+\n+test_expect_success 'serving ls-remote' '\n+\tGIT_TEST_REFTABLE=1 git init -b main \"$REPO\" &&\n+\tcd \"$REPO\" &&\n+\ttest_commit m1 &&\n+\t>.git/git-daemon-export-ok &&\n+\tgit ls-remote \"http://127.0.0.1:$LIB_HTTPD_PORT/smart/repo\" | cut -f 2-2 -d \"\t\" >actual &&\n+\tcat << EOF >expect &&\n+HEAD\n+refs/heads/main\n+refs/tags/m1\n+EOF\n+\ttest_cmp actual expect\n+'\n+\n+test_expect_success 'large reflog message' '\n+\tinitialize &&\n+\tperl -e \"\n+\t\tprint \\\"this is a long commit message\\\" x 50000\n+\t\" >commit-msg &&\n+\tgit commit --allow-empty -F commit-msg\n+'\n+\n+test_expect_success 'update-ref OID verification' '\n+\tinitialize &&\n+\ttest_must_fail test-tool ref-store main update-ref msg refs/heads/bla $INVALID_SHA1 $ZERO_OID 0 &&\n+\ttest-tool ref-store main update-ref msg refs/heads/bla $INVALID_SHA1 $ZERO_OID REF_SKIP_OID_VERIFICATION\n+'\n+\n+test_expect_success 'bogus names are detected in iteration flags' '\n+\tinitialize &&\n+\ttest_commit file &&\n+\ttest-tool ref-store main update-ref msg \"refs/heads/bogus..name\" $(git rev-parse HEAD) $ZERO_OID REF_SKIP_REFNAME_VERIFICATION &&\n+\ttest-tool ref-store main for-each-ref \"\" > actual &&\n+\tgrep \"0000000000000000000000000000000000000000 refs/heads/bogus..name 0xc\" actual\n+'\n+\n+test_expect_success 'bogus OIDs are not detected in iteration flags' '\n+\tinitialize &&\n+\ttest_commit file &&\n+\ttest-tool ref-store main update-ref msg \"refs/heads/broken-hash\" $INVALID_SHA1 $ZERO_OID REF_SKIP_OID_VERIFICATION &&\n+\ttest-tool ref-store main for-each-ref \"\" > actual &&\n+cat actual &&\n+\tgrep \"$INVALID_SHA1 refs/heads/broken-hash 0x0\" actual\n+'\n+\n+test_expect_success 'reftable supports reflog existence' '\n+\tinitialize &&\n+\ttest_must_fail test-tool ref-store main reflog-exists refs/heads/reflogtest &&\n+\ttest-tool ref-store main create-reflog refs/heads/reflogtest &&\n+\ttest-tool ref-store main reflog-exists refs/heads/reflogtest &&\n+\t>expect &&\n+\ttest-tool ref-store main for-each-reflog-ent-reverse refs/heads/reflogtest >actual &&\n+\ttest_cmp expect actual &&\n+\ttest_commit file0 &&\n+\tgit checkout -b reflogtest &&\n+\ttest_commit fileA &&\n+\ttest_commit fileB &&\n+cat << EOF > expect &&\n+branch: Created from HEAD\n+commit: fileA\n+commit: fileB\n+EOF\n+\ttest-tool ref-store main for-each-reflog-ent refs/heads/reflogtest | sed \"s/.*\t//\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit reflog expire reflogtest --expire=all &&\n+\t>expect &&\n+\ttest-tool ref-store main for-each-reflog-ent refs/heads/reflogtest >actual &&\n+\ttest_cmp expect actual &&\n+\ttest-tool ref-store main reflog-exists refs/heads/reflogtest &&\n+\ttest-tool ref-store main delete-reflog refs/heads/reflogtest &&\n+\ttest_must_fail test-tool ref-store main reflog-exists refs/heads/reflogtest\n+'\n+\n+test_expect_success 'for-each-reflog-ent symref' '\n+\tinitialize &&\n+\ttest_commit file &&\n+\tgit branch &&\n+\tgit symbolic-ref refs/heads/sym refs/heads/primary &&\n+cat >expected <actual &&\n+\ttest_cmp expected actual\n+'\n+\n+test_done\ndiff --git a/t/t1409-avoid-packing-refs.sh b/t/t1409-avoid-packing-refs.sh\nindex be12fb63506..cdc21bf2dcb 100755\n--- a/t/t1409-avoid-packing-refs.sh\n+++ b/t/t1409-avoid-packing-refs.sh\n@@ -4,6 +4,12 @@ test_description='avoid rewriting packed-refs unnecessarily'\n \n . ./test-lib.sh\n \n+if test_have_prereq !REFFILES\n+then\n+ skip_all='skipping pack-refs tests; need files backend'\n+ test_done\n+fi\n+\n # Add an identifying mark to the packed-refs file header line. This\n # shouldn't upset readers, and it should be omitted if the file is\n # ever rewritten.\ndiff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh\nindex de50c0ea018..397f1edac9a 100755\n--- a/t/t1450-fsck.sh\n+++ b/t/t1450-fsck.sh\n@@ -8,6 +8,12 @@ test_description='git fsck random collection of tests\n \n . ./test-lib.sh\n \n+if test_have_prereq !REFFILES\n+then\n+ skip_all='skipping tests; incompatible with reftable'\n+ test_done\n+fi\n+\n test_expect_success setup '\n \tgit config gc.auto 0 &&\n \tgit config i18n.commitencoding ISO-8859-1 &&\ndiff --git a/t/t3210-pack-refs.sh b/t/t3210-pack-refs.sh\nindex 577f32dc71f..e523c3dd624 100755\n--- a/t/t3210-pack-refs.sh\n+++ b/t/t3210-pack-refs.sh\n@@ -14,6 +14,12 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n \n . ./test-lib.sh\n \n+if test_have_prereq !REFFILES\n+then\n+ skip_all='skipping pack-refs tests; requires files ref backend'\n+ test_done\n+fi\n+\n test_expect_success 'enable reflogs' '\n \tgit config core.logallrefupdates true\n '\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 46cd596e7f5..0fd021f3177 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1604,7 +1604,12 @@ parisc* | hppa*)\n \t;;\n esac\n \n-test_set_prereq REFFILES\n+if test -n \"$GIT_TEST_REFTABLE\"\n+then\n+ test_set_prereq !REFFILES\n+else\n+ test_set_prereq REFFILES\n+fi\n \n ( COLUMNS=1 && test $COLUMNS = 1 ) && test_set_prereq COLUMNS_CAN_BE_1\n test -z \"$NO_PERL\" && test_set_prereq PERL\n\nbase-commit: a9cd6a0c5981a7d3c9865cf1966243c82f93e58e\n-- \ngitgitgadget\n"}]}