{"thread":{"id":"41574","subject":"[PATCH v7 01/33] setup: call setup_git_directory_gently before accessing refs","startedAt":"2016-03-01T00:52:33Z","lastAt":"2016-04-25T16:46:54Z","messageCount":57,"participants":["David Turner","Duy Nguyen","Jeff King","Ramsay Jones","Michael Haggerty"],"isPatch":true,"patchVersion":7,"patchTotal":33},"messages":[{"id":"279889","messageId":"1456793586-22082-1-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":null,"subject":"[PATCH v7 00/33] refs backend","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:33Z","receivedAt":"2016-03-01T00:52:33Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"This one has suggestions from Peff, SZEDER Gábor, Duy Nguyen, and\nfixups from Junio.\n\nThe major changes are:\n\n* The new patch \"call setup_git_directory_gently before accessing\n refs\" -- this is necessary in order to move \"setup: configure ref\n storage config on startup\" from config to setup.\n\n* \"setup: configure ref storage config on startup\" is now much shorter.\n\nIn addition, there are some minor fixups to remove variable shadowing\nin the lmdb code and to improve the design of the\nset_ref_storage_backend family of functions.\n\nDavid Turner (30):\n setup: call setup_git_directory_gently before accessing refs\n refs: move head_ref{,_submodule} to the common code\n refs: move for_each_*ref* functions into common code\n files-backend: break out ref reading\n refs: move resolve_ref_unsafe into common code\n refs: add method for do_for_each_ref\n refs: add do_for_each_per_worktree_ref\n refs: add methods for reflog\n refs: add method for initial ref transaction commit\n refs: add method for delete_refs\n refs: add methods to init refs db\n refs: add method to rename refs\n refs: handle non-normal ref renames\n refs: make lock generic\n refs: move duplicate check to common code\n refs: allow log-only updates\n refs: don't dereference on rename\n refs: on symref reflog expire, lock symref not referrent\n refs: resolve symbolic refs first\n refs: always handle non-normal refs in files backend\n init: allow alternate ref strorage to be set for new repos\n refs: check submodules' ref storage config\n clone: allow ref storage backend to be set for clone\n svn: learn ref-storage argument\n refs: register ref storage backends\n setup: configure ref storage config on startup\n refs: break out resolve_ref_unsafe_submodule\n refs: add LMDB refs storage backend\n refs: tests for lmdb backend\n tests: add ref-storage argument\n\nRamsay Jones (1):\n refs: reduce the visibility of do_for_each_ref()\n\nRonnie Sahlberg (2):\n refs: add a backend method structure with transaction functions\n refs: add methods for misc ref operations\n\n .gitignore | 1 +\n Documentation/config.txt | 9 +\n Documentation/git-clone.txt | 6 +\n Documentation/git-init-db.txt | 2 +-\n Documentation/git-init.txt | 8 +-\n Documentation/technical/refs-lmdb-backend.txt | 61 +\n Documentation/technical/repository-version.txt | 7 +\n Makefile | 12 +\n builtin/clone.c | 5 +\n builtin/grep.c | 1 +\n builtin/init-db.c | 55 +-\n builtin/log.c | 2 +-\n builtin/shortlog.c | 7 +-\n builtin/submodule--helper.c | 2 +-\n cache.h | 2 +\n config.c | 1 +\n configure.ac | 33 +\n contrib/completion/git-completion.bash | 6 +-\n contrib/workdir/git-new-workdir | 3 +\n git-submodule.sh | 13 +\n git-svn.perl | 6 +-\n git.c | 2 +-\n path.c | 30 +-\n refs.c | 631 +++++++-\n refs.h | 16 +\n refs/files-backend.c | 686 ++++-----\n refs/lmdb-backend.c | 1886 ++++++++++++++++++++++++\n refs/refs-internal.h | 123 +-\n setup.c | 29 +\n shortlog.h | 2 +-\n t/README | 6 +\n t/lib-submodule-update.sh | 15 +-\n t/lib-t6000.sh | 7 +-\n t/t0001-init.sh | 25 +\n t/t0008-ignores.sh | 2 +-\n t/t0062-revision-walking.sh | 6 +\n t/t1021-rerere-in-workdir.sh | 6 +\n t/t1200-tutorial.sh | 8 +-\n t/t1302-repo-version.sh | 6 +\n t/t1305-config-include.sh | 17 +-\n t/t1400-update-ref.sh | 6 +\n t/t1401-symbolic-ref.sh | 17 +-\n t/t1404-update-ref-df-conflicts.sh | 8 +-\n t/t1410-reflog.sh | 16 +\n t/t1430-bad-ref-name.sh | 6 +\n t/t1450-fsck.sh | 12 +-\n t/t1460-refs-lmdb-backend.sh | 1109 ++++++++++++++\n t/t1470-refs-lmdb-backend-reflog.sh | 359 +++++\n t/t1480-refs-lmdb-submodule.sh | 85 ++\n t/t1506-rev-parse-diagnosis.sh | 4 +-\n t/t2013-checkout-submodule.sh | 2 +-\n t/t2105-update-index-gitfile.sh | 4 +-\n t/t2107-update-index-basic.sh | 6 +-\n t/t2201-add-update-typechange.sh | 4 +-\n t/t3001-ls-files-others-exclude.sh | 2 +-\n t/t3010-ls-files-killed-modified.sh | 4 +-\n t/t3040-subprojects-basic.sh | 4 +-\n t/t3050-subprojects-fetch.sh | 2 +-\n t/t3200-branch.sh | 84 +-\n t/t3210-pack-refs.sh | 7 +\n t/t3211-peel-ref.sh | 6 +\n t/t3308-notes-merge.sh | 2 +-\n t/t3404-rebase-interactive.sh | 2 +-\n t/t3600-rm.sh | 2 +-\n t/t3800-mktag.sh | 4 +-\n t/t3903-stash.sh | 2 +-\n t/t4010-diff-pathspec.sh | 2 +-\n t/t4020-diff-external.sh | 2 +-\n t/t4027-diff-submodule.sh | 2 +-\n t/t4035-diff-quiet.sh | 2 +-\n t/t4255-am-submodule.sh | 2 +-\n t/t5000-tar-tree.sh | 3 +-\n t/t5304-prune.sh | 2 +-\n t/t5312-prune-corruption.sh | 11 +-\n t/t5500-fetch-pack.sh | 10 +-\n t/t5510-fetch.sh | 30 +-\n t/t5526-fetch-submodules.sh | 4 +-\n t/t5527-fetch-odd-refs.sh | 7 +\n t/t5537-fetch-shallow.sh | 7 +\n t/t5700-clone-reference.sh | 42 +-\n t/t6001-rev-list-graft.sh | 3 +-\n t/t6010-merge-base.sh | 2 +-\n t/t6050-replace.sh | 4 +-\n t/t6120-describe.sh | 6 +-\n t/t6301-for-each-ref-errors.sh | 12 +-\n t/t7201-co.sh | 2 +-\n t/t7300-clean.sh | 25 +-\n t/t7400-submodule-basic.sh | 22 +-\n t/t7402-submodule-rebase.sh | 2 +-\n t/t7405-submodule-merge.sh | 10 +-\n t/t9104-git-svn-follow-parent.sh | 3 +-\n t/t9115-git-svn-dcommit-funky-renames.sh | 2 +-\n t/t9350-fast-export.sh | 6 +-\n t/t9902-completion.sh | 4 +-\n t/t9903-bash-prompt.sh | 2 +-\n t/test-lib-functions.sh | 53 +-\n t/test-lib.sh | 12 +\n test-match-trees.c | 2 +\n test-refs-lmdb-backend.c | 66 +\n test-revision-walking.c | 2 +\n 100 files changed, 5265 insertions(+), 605 deletions(-)\n create mode 100644 Documentation/technical/refs-lmdb-backend.txt\n create mode 100644 refs/lmdb-backend.c\n create mode 100755 t/t1460-refs-lmdb-backend.sh\n create mode 100755 t/t1470-refs-lmdb-backend-reflog.sh\n create mode 100755 t/t1480-refs-lmdb-submodule.sh\n create mode 100644 test-refs-lmdb-backend.c\n\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279885","messageId":"1456793586-22082-2-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 01/33] setup: call setup_git_directory_gently before accessing refs","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:34Z","receivedAt":"2016-03-01T00:52:34Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"Usually, git calls some form of setup_git_directory at startup. But\nsometimes, it doesn't. Usually, that's OK because it's not really\nusing the repository. But in some cases, it is using the repo. In\nthose cases, either setup_git_directory_gently must be called, or the\nrepository (e.g. the refs) must not be accessed.\n\nIn every case except grep and shortlog, we fix this problem by making\nthe call.\n\nIn grep, in the --no-index mode, we don't want to access repository,\nso we set a flag which prevents this.\n\nIn shortlog, we only want to skip accessing the repository when\nrunning without a repo (in stdin mode), so we check that before\ncalling the only repo-dependent function that doesn't do its own\nsetup.\n\nSuggested-by: Jeff King \nSigned-off-by: David Turner \n---\n builtin/grep.c | 1 +\n builtin/log.c | 2 +-\n builtin/shortlog.c | 7 ++++---\n git.c | 2 +-\n shortlog.h | 2 +-\n test-match-trees.c | 2 ++\n test-revision-walking.c | 2 ++\n 7 files changed, 12 insertions(+), 6 deletions(-)\n\ndiff --git a/builtin/grep.c b/builtin/grep.c\nindex 9e3f1cf..1e36b52 100644\n--- a/builtin/grep.c\n+++ b/builtin/grep.c\n@@ -531,6 +531,7 @@ static int grep_directory(struct grep_opt *opt, const struct pathspec *pathspec,\n \tif (exc_std)\n \t\tsetup_standard_excludes(&dir);\n \n+\tdir.flags |= DIR_NO_GITLINKS;\n \tfill_directory(&dir, pathspec);\n \tfor (i = 0; i < dir.nr; i++) {\n \t\tif (!dir_path_match(dir.entries[i], pathspec, 0, NULL))\ndiff --git a/builtin/log.c b/builtin/log.c\nindex 0d738d6..1d0e43e 100644\n--- a/builtin/log.c\n+++ b/builtin/log.c\n@@ -975,7 +975,7 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,\n \n \tstrbuf_release(&sb);\n \n-\tshortlog_init(&log);\n+\tshortlog_init(&log, 0);\n \tlog.wrap_lines = 1;\n \tlog.wrap = 72;\n \tlog.in1 = 2;\ndiff --git a/builtin/shortlog.c b/builtin/shortlog.c\nindex bfc082e..ab4305b 100644\n--- a/builtin/shortlog.c\n+++ b/builtin/shortlog.c\n@@ -219,11 +219,12 @@ static int parse_wrap_args(const struct option *opt, const char *arg, int unset)\n \treturn 0;\n }\n \n-void shortlog_init(struct shortlog *log)\n+void shortlog_init(struct shortlog *log, int nongit)\n {\n \tmemset(log, 0, sizeof(*log));\n \n-\tread_mailmap(&log->mailmap, &log->common_repo_prefix);\n+\tif (!nongit)\n+\t\tread_mailmap(&log->mailmap, &log->common_repo_prefix);\n \n \tlog->list.strdup_strings = 1;\n \tlog->wrap = DEFAULT_WRAPLEN;\n@@ -252,7 +253,7 @@ int cmd_shortlog(int argc, const char **argv, const char *prefix)\n \tstruct parse_opt_ctx_t ctx;\n \n \tgit_config(git_default_config, NULL);\n-\tshortlog_init(&log);\n+\tshortlog_init(&log, nongit);\n \tinit_revisions(&rev, prefix);\n \tparse_options_start(&ctx, argc, argv, prefix, options,\n \t\t\t PARSE_OPT_KEEP_DASHDASH | PARSE_OPT_KEEP_ARGV0);\ndiff --git a/git.c b/git.c\nindex 6cc0c07..51e0508 100644\n--- a/git.c\n+++ b/git.c\n@@ -376,7 +376,7 @@ static struct cmd_struct commands[] = {\n \t{ \"am\", cmd_am, RUN_SETUP | NEED_WORK_TREE },\n \t{ \"annotate\", cmd_annotate, RUN_SETUP },\n \t{ \"apply\", cmd_apply, RUN_SETUP_GENTLY },\n-\t{ \"archive\", cmd_archive },\n+\t{ \"archive\", cmd_archive, RUN_SETUP_GENTLY },\n \t{ \"bisect--helper\", cmd_bisect__helper, RUN_SETUP },\n \t{ \"blame\", cmd_blame, RUN_SETUP },\n \t{ \"branch\", cmd_branch, RUN_SETUP },\ndiff --git a/shortlog.h b/shortlog.h\nindex de4f86f..ed1fbca 100644\n--- a/shortlog.h\n+++ b/shortlog.h\n@@ -19,7 +19,7 @@ struct shortlog {\n \tstruct string_list mailmap;\n };\n \n-void shortlog_init(struct shortlog *log);\n+void shortlog_init(struct shortlog *log, int nongit);\n \n void shortlog_add_commit(struct shortlog *log, struct commit *commit);\n \ndiff --git a/test-match-trees.c b/test-match-trees.c\nindex 109f03e..4dad709 100644\n--- a/test-match-trees.c\n+++ b/test-match-trees.c\n@@ -6,6 +6,8 @@ int main(int ac, char **av)\n \tunsigned char hash1[20], hash2[20], shifted[20];\n \tstruct tree *one, *two;\n \n+\tsetup_git_directory();\n+\n \tif (get_sha1(av[1], hash1))\n \t\tdie(\"cannot parse %s as an object name\", av[1]);\n \tif (get_sha1(av[2], hash2))\ndiff --git a/test-revision-walking.c b/test-revision-walking.c\nindex 285f06b..3d03133 100644\n--- a/test-revision-walking.c\n+++ b/test-revision-walking.c\n@@ -50,6 +50,8 @@ int main(int argc, char **argv)\n \tif (argc < 2)\n \t\treturn 1;\n \n+\tsetup_git_directory();\n+\n \tif (!strcmp(argv[1], \"run-twice\")) {\n \t\tprintf(\"1st\\n\");\n \t\tif (!run_revision_walk())\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279891","messageId":"1456793586-22082-3-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 02/33] refs: move head_ref{,_submodule} to the common code","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:35Z","receivedAt":"2016-03-01T00:52:35Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"These don't use any backend-specific functions. These were previously\ndefined in terms of the do_head_ref helper function, but since they\nare otherwise identical, we don't need that function.\n\nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n refs.c | 23 +++++++++++++++++++++++\n refs/files-backend.c | 28 ----------------------------\n 2 files changed, 23 insertions(+), 28 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 0dadfea..beac899 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1086,3 +1086,26 @@ int rename_ref_available(const char *oldname, const char *newname)\n \tstrbuf_release(&err);\n \treturn ret;\n }\n+\n+int head_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)\n+{\n+\tstruct object_id oid;\n+\tint flag;\n+\n+\tif (submodule) {\n+\t\tif (resolve_gitlink_ref(submodule, \"HEAD\", oid.hash) == 0)\n+\t\t\treturn fn(\"HEAD\", &oid, 0, cb_data);\n+\n+\t\treturn 0;\n+\t}\n+\n+\tif (!read_ref_full(\"HEAD\", RESOLVE_REF_READING, oid.hash, &flag))\n+\t\treturn fn(\"HEAD\", &oid, flag, cb_data);\n+\n+\treturn 0;\n+}\n+\n+int head_ref(each_ref_fn fn, void *cb_data)\n+{\n+\treturn head_ref_submodule(NULL, fn, cb_data);\n+}\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 81f68f8..c07dc41 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1745,34 +1745,6 @@ static int do_for_each_ref(struct ref_cache *refs, const char *base,\n \treturn do_for_each_entry(refs, base, do_one_ref, &data);\n }\n \n-static int do_head_ref(const char *submodule, each_ref_fn fn, void *cb_data)\n-{\n-\tstruct object_id oid;\n-\tint flag;\n-\n-\tif (submodule) {\n-\t\tif (resolve_gitlink_ref(submodule, \"HEAD\", oid.hash) == 0)\n-\t\t\treturn fn(\"HEAD\", &oid, 0, cb_data);\n-\n-\t\treturn 0;\n-\t}\n-\n-\tif (!read_ref_full(\"HEAD\", RESOLVE_REF_READING, oid.hash, &flag))\n-\t\treturn fn(\"HEAD\", &oid, flag, cb_data);\n-\n-\treturn 0;\n-}\n-\n-int head_ref(each_ref_fn fn, void *cb_data)\n-{\n-\treturn do_head_ref(NULL, fn, cb_data);\n-}\n-\n-int head_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)\n-{\n-\treturn do_head_ref(submodule, fn, cb_data);\n-}\n-\n int for_each_ref(each_ref_fn fn, void *cb_data)\n {\n \treturn do_for_each_ref(&ref_cache, \"\", fn, 0, 0, cb_data);\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279887","messageId":"1456793586-22082-4-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 03/33] refs: move for_each_*ref* functions into common code","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:36Z","receivedAt":"2016-03-01T00:52:36Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"Make do_for_each_ref take a submodule as an argument instead of a\nref_cache. Since all for_each_*ref* functions are defined in terms of\ndo_for_each_ref, we can then move them into the common code.\n\nLater, we can simply make do_for_each_ref into a backend function.\n\nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n refs.c | 52 +++++++++++++++++++++++++++++++++++++++++++\n refs/files-backend.c | 62 +++++-----------------------------------------------\n refs/refs-internal.h | 9 ++++++++\n 3 files changed, 66 insertions(+), 57 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex beac899..b49c077 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1109,3 +1109,55 @@ int head_ref(each_ref_fn fn, void *cb_data)\n {\n \treturn head_ref_submodule(NULL, fn, cb_data);\n }\n+\n+int for_each_ref(each_ref_fn fn, void *cb_data)\n+{\n+\treturn do_for_each_ref(NULL, \"\", fn, 0, 0, cb_data);\n+}\n+\n+int for_each_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)\n+{\n+\treturn do_for_each_ref(submodule, \"\", fn, 0, 0, cb_data);\n+}\n+\n+int for_each_ref_in(const char *prefix, each_ref_fn fn, void *cb_data)\n+{\n+\treturn do_for_each_ref(NULL, prefix, fn, strlen(prefix), 0, cb_data);\n+}\n+\n+int for_each_fullref_in(const char *prefix, each_ref_fn fn, void *cb_data, unsigned int broken)\n+{\n+\tunsigned int flag = 0;\n+\n+\tif (broken)\n+\t\tflag = DO_FOR_EACH_INCLUDE_BROKEN;\n+\treturn do_for_each_ref(NULL, prefix, fn, 0, flag, cb_data);\n+}\n+\n+int for_each_ref_in_submodule(const char *submodule, const char *prefix,\n+\t\teach_ref_fn fn, void *cb_data)\n+{\n+\treturn do_for_each_ref(submodule, prefix, fn, strlen(prefix), 0, cb_data);\n+}\n+\n+int for_each_replace_ref(each_ref_fn fn, void *cb_data)\n+{\n+\treturn do_for_each_ref(NULL, git_replace_ref_base, fn,\n+\t\t\t strlen(git_replace_ref_base), 0, cb_data);\n+}\n+\n+int for_each_namespaced_ref(each_ref_fn fn, void *cb_data)\n+{\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tint ret;\n+\tstrbuf_addf(&buf, \"%srefs/\", get_git_namespace());\n+\tret = do_for_each_ref(NULL, buf.buf, fn, 0, 0, cb_data);\n+\tstrbuf_release(&buf);\n+\treturn ret;\n+}\n+\n+int for_each_rawref(each_ref_fn fn, void *cb_data)\n+{\n+\treturn do_for_each_ref(NULL, \"\", fn, 0,\n+\t\t\t DO_FOR_EACH_INCLUDE_BROKEN, cb_data);\n+}\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex c07dc41..9676ec2 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -513,9 +513,6 @@ static void sort_ref_dir(struct ref_dir *dir)\n \tdir->sorted = dir->nr = i;\n }\n \n-/* Include broken references in a do_for_each_ref*() iteration: */\n-#define DO_FOR_EACH_INCLUDE_BROKEN 0x01\n-\n /*\n * Return true iff the reference described by entry can be resolved to\n * an object in the database. Emit a warning if the referred-to\n@@ -1727,10 +1724,13 @@ static int do_for_each_entry(struct ref_cache *refs, const char *base,\n * value, stop the iteration and return that value; otherwise, return\n * 0.\n */\n-static int do_for_each_ref(struct ref_cache *refs, const char *base,\n-\t\t\t each_ref_fn fn, int trim, int flags, void *cb_data)\n+int do_for_each_ref(const char *submodule, const char *base,\n+\t\t each_ref_fn fn, int trim, int flags, void *cb_data)\n {\n \tstruct ref_entry_cb data;\n+\tstruct ref_cache *refs;\n+\n+\trefs = get_ref_cache(submodule);\n \tdata.base = base;\n \tdata.trim = trim;\n \tdata.flags = flags;\n@@ -1745,58 +1745,6 @@ static int do_for_each_ref(struct ref_cache *refs, const char *base,\n \treturn do_for_each_entry(refs, base, do_one_ref, &data);\n }\n \n-int for_each_ref(each_ref_fn fn, void *cb_data)\n-{\n-\treturn do_for_each_ref(&ref_cache, \"\", fn, 0, 0, cb_data);\n-}\n-\n-int for_each_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)\n-{\n-\treturn do_for_each_ref(get_ref_cache(submodule), \"\", fn, 0, 0, cb_data);\n-}\n-\n-int for_each_ref_in(const char *prefix, each_ref_fn fn, void *cb_data)\n-{\n-\treturn do_for_each_ref(&ref_cache, prefix, fn, strlen(prefix), 0, cb_data);\n-}\n-\n-int for_each_fullref_in(const char *prefix, each_ref_fn fn, void *cb_data, unsigned int broken)\n-{\n-\tunsigned int flag = 0;\n-\n-\tif (broken)\n-\t\tflag = DO_FOR_EACH_INCLUDE_BROKEN;\n-\treturn do_for_each_ref(&ref_cache, prefix, fn, 0, flag, cb_data);\n-}\n-\n-int for_each_ref_in_submodule(const char *submodule, const char *prefix,\n-\t\teach_ref_fn fn, void *cb_data)\n-{\n-\treturn do_for_each_ref(get_ref_cache(submodule), prefix, fn, strlen(prefix), 0, cb_data);\n-}\n-\n-int for_each_replace_ref(each_ref_fn fn, void *cb_data)\n-{\n-\treturn do_for_each_ref(&ref_cache, git_replace_ref_base, fn,\n-\t\t\t strlen(git_replace_ref_base), 0, cb_data);\n-}\n-\n-int for_each_namespaced_ref(each_ref_fn fn, void *cb_data)\n-{\n-\tstruct strbuf buf = STRBUF_INIT;\n-\tint ret;\n-\tstrbuf_addf(&buf, \"%srefs/\", get_git_namespace());\n-\tret = do_for_each_ref(&ref_cache, buf.buf, fn, 0, 0, cb_data);\n-\tstrbuf_release(&buf);\n-\treturn ret;\n-}\n-\n-int for_each_rawref(each_ref_fn fn, void *cb_data)\n-{\n-\treturn do_for_each_ref(&ref_cache, \"\", fn, 0,\n-\t\t\t DO_FOR_EACH_INCLUDE_BROKEN, cb_data);\n-}\n-\n static void unlock_ref(struct ref_lock *lock)\n {\n \t/* Do not free lock->lk -- atexit() still looks at them */\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex c7dded3..92aae80 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -197,4 +197,13 @@ const char *find_descendant_ref(const char *dirname,\n \n int rename_ref_available(const char *oldname, const char *newname);\n \n+\n+/* Include broken references in a do_for_each_ref*() iteration: */\n+#define DO_FOR_EACH_INCLUDE_BROKEN 0x01\n+\n+/*\n+ * The common backend for the for_each_*ref* functions\n+ */\n+int do_for_each_ref(const char *submodule, const char *base,\n+\t\t each_ref_fn fn, int trim, int flags, void *cb_data);\n #endif /* REFS_REFS_INTERNAL_H */\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279911","messageId":"1456793586-22082-5-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 04/33] files-backend: break out ref reading","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:37Z","receivedAt":"2016-03-01T00:52:37Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"Refactor resolve_ref_1 in terms of a new function read_raw_ref, which\nis responsible for reading ref data from the ref storage.\n\nLater, we will make read_raw_ref a pluggable backend function, and make\nresolve_ref_unsafe common.\n\nTesting done: Hacked in code to run both old and new version of\nresolve_ref_1 and compare all outputs, failing dramatically if outputs\ndiffered. Ran test suite.\n\nSigned-off-by: David Turner \nHelped-by: Duy Nguyen \nSigned-off-by: Junio C Hamano \n---\n refs/files-backend.c | 265 ++++++++++++++++++++++++++++++---------------------\n 1 file changed, 159 insertions(+), 106 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 9676ec2..8c6a58e 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1369,12 +1369,11 @@ static struct ref_entry *get_packed_ref(const char *refname)\n \n /*\n * A loose ref file doesn't exist; check for a packed ref. The\n- * options are forwarded from resolve_safe_unsafe().\n+ * options are forwarded from resolve_ref_unsafe().\n */\n static int resolve_missing_loose_ref(const char *refname,\n-\t\t\t\t int resolve_flags,\n \t\t\t\t unsigned char *sha1,\n-\t\t\t\t int *flags)\n+\t\t\t\t unsigned int *flags)\n {\n \tstruct ref_entry *entry;\n \n@@ -1390,64 +1389,48 @@ static int resolve_missing_loose_ref(const char *refname,\n \t\treturn 0;\n \t}\n \t/* The reference is not a packed reference, either. */\n-\tif (resolve_flags & RESOLVE_REF_READING) {\n-\t\terrno = ENOENT;\n-\t\treturn -1;\n-\t} else {\n-\t\thashclr(sha1);\n-\t\treturn 0;\n-\t}\n+\terrno = ENOENT;\n+\treturn -1;\n }\n \n-/* This function needs to return a meaningful errno on failure */\n-static const char *resolve_ref_1(const char *refname,\n-\t\t\t\t int resolve_flags,\n-\t\t\t\t unsigned char *sha1,\n-\t\t\t\t int *flags,\n-\t\t\t\t struct strbuf *sb_refname,\n-\t\t\t\t struct strbuf *sb_path,\n-\t\t\t\t struct strbuf *sb_contents)\n+/*\n+ * Read a raw ref from the filesystem or packed refs file.\n+ *\n+ * If the ref is a sha1, fill in sha1 and return 0.\n+ *\n+ * If the ref is symbolic, fill in *symref with the referrent\n+ * (e.g. \"refs/heads/master\") and return 0. The caller is responsible\n+ * for validating the referrent. Set REF_ISSYMREF in flags.\n+ *\n+ * If the ref is neither a symbolic ref nor a sha1, it is broken. Set\n+ * REF_ISBROKEN in flags, set errno to EINVAL, and return -1.\n+ *\n+ * If the ref doesn't exist, set errno to ENOENT and return -1.\n+ *\n+ * If there is another error reading the ref, set errno appropriately and\n+ * return -1.\n+ *\n+ * Backend-specific flags might be set in flags as well, regardless of\n+ * outcome.\n+ *\n+ * sb_path is workspace: the caller should allocate and free it.\n+ */\n+static int read_raw_ref(const char *refname, unsigned char *sha1,\n+\t\t\tstruct strbuf *symref, struct strbuf *sb_path,\n+\t\t\tunsigned int *flags)\n {\n-\tint depth = MAXDEPTH;\n-\tint bad_name = 0;\n-\n-\tif (flags)\n-\t\t*flags = 0;\n+\tstruct strbuf sb_contents = STRBUF_INIT;\n+\tint ret = -1;\n+\tconst char *path;\n+\tconst char *buf;\n \n-\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n-\t\tif (flags)\n-\t\t\t*flags |= REF_BAD_NAME;\n+\tstrbuf_reset(sb_path);\n+\tstrbuf_git_path(sb_path, \"%s\", refname);\n+\tpath = sb_path->buf;\n \n-\t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n-\t\t !refname_is_safe(refname)) {\n-\t\t\terrno = EINVAL;\n-\t\t\treturn NULL;\n-\t\t}\n-\t\t/*\n-\t\t * dwim_ref() uses REF_ISBROKEN to distinguish between\n-\t\t * missing refs and refs that were present but invalid,\n-\t\t * to complain about the latter to stderr.\n-\t\t *\n-\t\t * We don't know whether the ref exists, so don't set\n-\t\t * REF_ISBROKEN yet.\n-\t\t */\n-\t\tbad_name = 1;\n-\t}\n \tfor (;;) {\n-\t\tconst char *path;\n \t\tstruct stat st;\n-\t\tchar *buf;\n \t\tint fd;\n-\n-\t\tif (--depth < 0) {\n-\t\t\terrno = ELOOP;\n-\t\t\treturn NULL;\n-\t\t}\n-\n-\t\tstrbuf_reset(sb_path);\n-\t\tstrbuf_git_path(sb_path, \"%s\", refname);\n-\t\tpath = sb_path->buf;\n-\n \t\t/*\n \t\t * We might have to loop back here to avoid a race\n \t\t * condition: first we lstat() the file, then we try\n@@ -1457,49 +1440,45 @@ static const char *resolve_ref_1(const char *refname,\n \t\t * we don't want to report that as an error but rather\n \t\t * try again starting with the lstat().\n \t\t */\n-\tstat_ref:\n+\n \t\tif (lstat(path, &st) < 0) {\n \t\t\tif (errno != ENOENT)\n-\t\t\t\treturn NULL;\n-\t\t\tif (resolve_missing_loose_ref(refname, resolve_flags,\n-\t\t\t\t\t\t sha1, flags))\n-\t\t\t\treturn NULL;\n-\t\t\tif (bad_name) {\n-\t\t\t\thashclr(sha1);\n-\t\t\t\tif (flags)\n-\t\t\t\t\t*flags |= REF_ISBROKEN;\n-\t\t\t}\n-\t\t\treturn refname;\n+\t\t\t\tbreak;\n+\t\t\tif (resolve_missing_loose_ref(refname, sha1, flags))\n+\t\t\t\tbreak;\n+\t\t\tret = 0;\n+\t\t\tbreak;\n \t\t}\n \n \t\t/* Follow \"normalized\" - ie \"refs/..\" symlinks by hand */\n \t\tif (S_ISLNK(st.st_mode)) {\n-\t\t\tstrbuf_reset(sb_contents);\n-\t\t\tif (strbuf_readlink(sb_contents, path, 0) < 0) {\n+\t\t\tstrbuf_reset(&sb_contents);\n+\t\t\tif (strbuf_readlink(&sb_contents, path, 0) < 0) {\n \t\t\t\tif (errno == ENOENT || errno == EINVAL)\n \t\t\t\t\t/* inconsistent with lstat; retry */\n-\t\t\t\t\tgoto stat_ref;\n+\t\t\t\t\tcontinue;\n \t\t\t\telse\n-\t\t\t\t\treturn NULL;\n+\t\t\t\t\tbreak;\n \t\t\t}\n-\t\t\tif (starts_with(sb_contents->buf, \"refs/\") &&\n-\t\t\t !check_refname_format(sb_contents->buf, 0)) {\n-\t\t\t\tstrbuf_swap(sb_refname, sb_contents);\n-\t\t\t\trefname = sb_refname->buf;\n+\t\t\tif (starts_with(sb_contents.buf, \"refs/\") &&\n+\t\t\t !check_refname_format(sb_contents.buf, 0)) {\n+\t\t\t\tstrbuf_swap(&sb_contents, symref);\n \t\t\t\tif (flags)\n \t\t\t\t\t*flags |= REF_ISSYMREF;\n-\t\t\t\tif (resolve_flags & RESOLVE_REF_NO_RECURSE) {\n-\t\t\t\t\thashclr(sha1);\n-\t\t\t\t\treturn refname;\n-\t\t\t\t}\n-\t\t\t\tcontinue;\n+\t\t\t\tret = 0;\n+\t\t\t\tbreak;\n+\t\t\t} else {\n+\t\t\t\t/* bogus symlink ref */\n+\t\t\t\tif (flags)\n+\t\t\t\t\t*flags |= REF_ISBROKEN;\n+\t\t\t\tbreak;\n \t\t\t}\n \t\t}\n \n \t\t/* Is it a directory? */\n \t\tif (S_ISDIR(st.st_mode)) {\n \t\t\terrno = EISDIR;\n-\t\t\treturn NULL;\n+\t\t\tbreak;\n \t\t}\n \n \t\t/*\n@@ -1510,35 +1489,110 @@ static const char *resolve_ref_1(const char *refname,\n \t\tif (fd < 0) {\n \t\t\tif (errno == ENOENT)\n \t\t\t\t/* inconsistent with lstat; retry */\n-\t\t\t\tgoto stat_ref;\n+\t\t\t\tcontinue;\n \t\t\telse\n-\t\t\t\treturn NULL;\n+\t\t\t\tbreak;\n \t\t}\n-\t\tstrbuf_reset(sb_contents);\n-\t\tif (strbuf_read(sb_contents, fd, 256) < 0) {\n+\t\tstrbuf_reset(&sb_contents);\n+\t\tif (strbuf_read(&sb_contents, fd, 256) < 0) {\n \t\t\tint save_errno = errno;\n \t\t\tclose(fd);\n \t\t\terrno = save_errno;\n-\t\t\treturn NULL;\n+\t\t\tbreak;\n \t\t}\n \t\tclose(fd);\n-\t\tstrbuf_rtrim(sb_contents);\n+\t\tstrbuf_rtrim(&sb_contents);\n+\t\tbuf = sb_contents.buf;\n+\t\tif (starts_with(buf, \"ref:\")) {\n+\t\t\tbuf += 4;\n+\t\t\twhile (isspace(*buf))\n+\t\t\t\tbuf++;\n+\n+\t\t\tstrbuf_reset(symref);\n+\t\t\tstrbuf_addstr(symref, buf);\n+\t\t\tif (flags)\n+\t\t\t\t*flags |= REF_ISSYMREF;\n+\t\t\tret = 0;\n+\t\t\tbreak;\n+\t\t}\n \n \t\t/*\n-\t\t * Is it a symbolic ref?\n+\t\t * Please note that FETCH_HEAD has additional\n+\t\t * data after the sha.\n \t\t */\n-\t\tif (!starts_with(sb_contents->buf, \"ref:\")) {\n-\t\t\t/*\n-\t\t\t * Please note that FETCH_HEAD has a second\n-\t\t\t * line containing other data.\n-\t\t\t */\n-\t\t\tif (get_sha1_hex(sb_contents->buf, sha1) ||\n-\t\t\t (sb_contents->buf[40] != '\\0' && !isspace(sb_contents->buf[40]))) {\n+\t\tif (get_sha1_hex(buf, sha1) ||\n+\t\t (buf[40] != '\\0' && !isspace(buf[40]))) {\n+\t\t\tif (flags)\n+\t\t\t\t*flags |= REF_ISBROKEN;\n+\t\t\terrno = EINVAL;\n+\t\t\tbreak;\n+\t\t}\n+\t\tret = 0;\n+\t\tbreak;\n+\t}\n+\n+\tstrbuf_release(&sb_contents);\n+\treturn ret;\n+}\n+\n+/* This function needs to return a meaningful errno on failure */\n+static const char *resolve_ref_1(const char *refname,\n+\t\t\t\t int resolve_flags,\n+\t\t\t\t unsigned char *sha1,\n+\t\t\t\t int *flags,\n+\t\t\t\t struct strbuf *sb_refname,\n+\t\t\t\t struct strbuf *sb_path)\n+{\n+\tint bad_name = 0;\n+\tint symref_count;\n+\n+\tif (flags)\n+\t\t*flags = 0;\n+\n+\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n+\t\tif (flags)\n+\t\t\t*flags |= REF_BAD_NAME;\n+\n+\t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n+\t\t !refname_is_safe(refname)) {\n+\t\t\terrno = EINVAL;\n+\t\t\treturn NULL;\n+\t\t}\n+\t\t/*\n+\t\t * dwim_ref() uses REF_ISBROKEN to distinguish between\n+\t\t * missing refs and refs that were present but invalid,\n+\t\t * to complain about the latter to stderr.\n+\t\t *\n+\t\t * We don't know whether the ref exists, so don't set\n+\t\t * REF_ISBROKEN yet.\n+\t\t */\n+\t\tbad_name = 1;\n+\t}\n+\n+\tfor (symref_count = 0; symref_count < MAXDEPTH; symref_count++) {\n+\t\tunsigned int read_flags = 0;\n+\n+\t\tif (read_raw_ref(refname, sha1, sb_refname, sb_path, &read_flags)) {\n+\t\t\tint saved_errno = errno;\n+\t\t\tif (flags)\n+\t\t\t\t*flags |= read_flags;\n+\t\t\terrno = saved_errno;\n+\t\t\tif (bad_name) {\n+\t\t\t\thashclr(sha1);\n \t\t\t\tif (flags)\n \t\t\t\t\t*flags |= REF_ISBROKEN;\n-\t\t\t\terrno = EINVAL;\n+\t\t\t}\n+\t\t\tif (resolve_flags & RESOLVE_REF_READING || errno != ENOENT) {\n \t\t\t\treturn NULL;\n+\t\t\t} else {\n+\t\t\t\thashclr(sha1);\n+\t\t\t\treturn refname;\n \t\t\t}\n+\t\t}\n+\t\tif (flags)\n+\t\t\t*flags |= read_flags;\n+\n+\t\tif (!(read_flags & REF_ISSYMREF)) {\n \t\t\tif (bad_name) {\n \t\t\t\thashclr(sha1);\n \t\t\t\tif (flags)\n@@ -1546,44 +1600,43 @@ static const char *resolve_ref_1(const char *refname,\n \t\t\t}\n \t\t\treturn refname;\n \t\t}\n-\t\tif (flags)\n-\t\t\t*flags |= REF_ISSYMREF;\n-\t\tbuf = sb_contents->buf + 4;\n-\t\twhile (isspace(*buf))\n-\t\t\tbuf++;\n-\t\tstrbuf_reset(sb_refname);\n-\t\tstrbuf_addstr(sb_refname, buf);\n+\n \t\trefname = sb_refname->buf;\n \t\tif (resolve_flags & RESOLVE_REF_NO_RECURSE) {\n \t\t\thashclr(sha1);\n+\t\t\tif (bad_name && flags)\n+\t\t\t\t*flags |= REF_ISBROKEN;\n \t\t\treturn refname;\n \t\t}\n-\t\tif (check_refname_format(buf, REFNAME_ALLOW_ONELEVEL)) {\n+\n+\t\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\t\tif (flags)\n \t\t\t\t*flags |= REF_ISBROKEN;\n-\n \t\t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n-\t\t\t !refname_is_safe(buf)) {\n+\t\t\t !refname_is_safe(refname)) {\n \t\t\t\terrno = EINVAL;\n \t\t\t\treturn NULL;\n \t\t\t}\n \t\t\tbad_name = 1;\n \t\t}\n \t}\n+\n+\tif (flags)\n+\t\t*flags |= REF_ISBROKEN;\n+\treturn NULL;\n }\n \n const char *resolve_ref_unsafe(const char *refname, int resolve_flags,\n \t\t\t unsigned char *sha1, int *flags)\n {\n \tstatic struct strbuf sb_refname = STRBUF_INIT;\n-\tstruct strbuf sb_contents = STRBUF_INIT;\n \tstruct strbuf sb_path = STRBUF_INIT;\n \tconst char *ret;\n \n \tret = resolve_ref_1(refname, resolve_flags, sha1, flags,\n-\t\t\t &sb_refname, &sb_path, &sb_contents);\n+\t\t\t &sb_refname, &sb_path);\n+\n \tstrbuf_release(&sb_path);\n-\tstrbuf_release(&sb_contents);\n \treturn ret;\n }\n \n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279886","messageId":"1456793586-22082-6-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 05/33] refs: move resolve_ref_unsafe into common code","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:38Z","receivedAt":"2016-03-01T00:52:38Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"Now that resolve_ref_unsafe's only interaction with the backend is\nthrough read_raw_ref, we can move it into the common code. Later,\nwe'll replace read_raw_ref with a backend function.\n\nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n refs.c | 104 ++++++++++++++++++++++++++++++++++++++++++++++\n refs/files-backend.c | 115 ++-------------------------------------------------\n refs/refs-internal.h | 6 +++\n 3 files changed, 114 insertions(+), 111 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex b49c077..15aabc6 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1161,3 +1161,107 @@ int for_each_rawref(each_ref_fn fn, void *cb_data)\n \treturn do_for_each_ref(NULL, \"\", fn, 0,\n \t\t\t DO_FOR_EACH_INCLUDE_BROKEN, cb_data);\n }\n+/* This function needs to return a meaningful errno on failure */\n+static const char *resolve_ref_1(const char *refname,\n+\t\t\t\t int resolve_flags,\n+\t\t\t\t unsigned char *sha1,\n+\t\t\t\t int *flags,\n+\t\t\t\t struct strbuf *sb_refname,\n+\t\t\t\t struct strbuf *sb_path)\n+{\n+\tint bad_name = 0;\n+\tint symref_count;\n+\n+\tif (flags)\n+\t\t*flags = 0;\n+\n+\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n+\t\tif (flags)\n+\t\t\t*flags |= REF_BAD_NAME;\n+\n+\t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n+\t\t !refname_is_safe(refname)) {\n+\t\t\terrno = EINVAL;\n+\t\t\treturn NULL;\n+\t\t}\n+\t\t/*\n+\t\t * dwim_ref() uses REF_ISBROKEN to distinguish between\n+\t\t * missing refs and refs that were present but invalid,\n+\t\t * to complain about the latter to stderr.\n+\t\t *\n+\t\t * We don't know whether the ref exists, so don't set\n+\t\t * REF_ISBROKEN yet.\n+\t\t */\n+\t\tbad_name = 1;\n+\t}\n+\n+\tfor (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {\n+\t\tunsigned int read_flags = 0;\n+\n+\t\tif (read_raw_ref(refname, sha1, sb_refname, sb_path, &read_flags)) {\n+\t\t\tint saved_errno = errno;\n+\t\t\tif (flags)\n+\t\t\t\t*flags |= read_flags;\n+\t\t\terrno = saved_errno;\n+\t\t\tif (bad_name) {\n+\t\t\t\thashclr(sha1);\n+\t\t\t\tif (flags)\n+\t\t\t\t\t*flags |= REF_ISBROKEN;\n+\t\t\t}\n+\t\t\tif (resolve_flags & RESOLVE_REF_READING || errno != ENOENT) {\n+\t\t\t\treturn NULL;\n+\t\t\t} else {\n+\t\t\t\thashclr(sha1);\n+\t\t\t\treturn refname;\n+\t\t\t}\n+\t\t}\n+\t\tif (flags)\n+\t\t\t*flags |= read_flags;\n+\n+\t\tif (!(read_flags & REF_ISSYMREF)) {\n+\t\t\tif (bad_name) {\n+\t\t\t\thashclr(sha1);\n+\t\t\t\tif (flags)\n+\t\t\t\t\t*flags |= REF_ISBROKEN;\n+\t\t\t}\n+\t\t\treturn refname;\n+\t\t}\n+\n+\t\trefname = sb_refname->buf;\n+\t\tif (resolve_flags & RESOLVE_REF_NO_RECURSE) {\n+\t\t\thashclr(sha1);\n+\t\t\tif (bad_name && flags)\n+\t\t\t\t*flags |= REF_ISBROKEN;\n+\t\t\treturn refname;\n+\t\t}\n+\n+\t\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n+\t\t\tif (flags)\n+\t\t\t\t*flags |= REF_ISBROKEN;\n+\t\t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n+\t\t\t !refname_is_safe(refname)) {\n+\t\t\t\terrno = EINVAL;\n+\t\t\t\treturn NULL;\n+\t\t\t}\n+\t\t\tbad_name = 1;\n+\t\t}\n+\t}\n+\n+\tif (flags)\n+\t\t*flags |= REF_ISBROKEN;\n+\treturn NULL;\n+}\n+\n+const char *resolve_ref_unsafe(const char *refname, int resolve_flags,\n+\t\t\t unsigned char *sha1, int *flags)\n+{\n+\tstatic struct strbuf sb_refname = STRBUF_INIT;\n+\tstruct strbuf sb_path = STRBUF_INIT;\n+\tconst char *ret;\n+\n+\tret = resolve_ref_1(refname, resolve_flags, sha1, flags,\n+\t\t\t &sb_refname, &sb_path);\n+\n+\tstrbuf_release(&sb_path);\n+\treturn ret;\n+}\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 8c6a58e..2fb7714 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1269,8 +1269,6 @@ static struct ref_dir *get_loose_refs(struct ref_cache *refs)\n \treturn get_ref_dir(refs->loose);\n }\n \n-/* We allow \"recursive\" symbolic refs. Only within reason, though */\n-#define MAXDEPTH 5\n #define MAXREFLEN (1024)\n \n /*\n@@ -1300,7 +1298,7 @@ static int resolve_gitlink_ref_recursive(struct ref_cache *refs,\n \tchar buffer[128], *p;\n \tchar *path;\n \n-\tif (recursion > MAXDEPTH || strlen(refname) > MAXREFLEN)\n+\tif (recursion > SYMREF_MAXDEPTH || strlen(refname) > MAXREFLEN)\n \t\treturn -1;\n \tpath = *refs->name\n \t\t? git_pathdup_submodule(refs->name, \"%s\", refname)\n@@ -1415,9 +1413,9 @@ static int resolve_missing_loose_ref(const char *refname,\n *\n * sb_path is workspace: the caller should allocate and free it.\n */\n-static int read_raw_ref(const char *refname, unsigned char *sha1,\n-\t\t\tstruct strbuf *symref, struct strbuf *sb_path,\n-\t\t\tunsigned int *flags)\n+int read_raw_ref(const char *refname, unsigned char *sha1,\n+\t\t struct strbuf *symref, struct strbuf *sb_path,\n+\t\t unsigned int *flags)\n {\n \tstruct strbuf sb_contents = STRBUF_INIT;\n \tint ret = -1;\n@@ -1535,111 +1533,6 @@ static int read_raw_ref(const char *refname, unsigned char *sha1,\n \treturn ret;\n }\n \n-/* This function needs to return a meaningful errno on failure */\n-static const char *resolve_ref_1(const char *refname,\n-\t\t\t\t int resolve_flags,\n-\t\t\t\t unsigned char *sha1,\n-\t\t\t\t int *flags,\n-\t\t\t\t struct strbuf *sb_refname,\n-\t\t\t\t struct strbuf *sb_path)\n-{\n-\tint bad_name = 0;\n-\tint symref_count;\n-\n-\tif (flags)\n-\t\t*flags = 0;\n-\n-\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n-\t\tif (flags)\n-\t\t\t*flags |= REF_BAD_NAME;\n-\n-\t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n-\t\t !refname_is_safe(refname)) {\n-\t\t\terrno = EINVAL;\n-\t\t\treturn NULL;\n-\t\t}\n-\t\t/*\n-\t\t * dwim_ref() uses REF_ISBROKEN to distinguish between\n-\t\t * missing refs and refs that were present but invalid,\n-\t\t * to complain about the latter to stderr.\n-\t\t *\n-\t\t * We don't know whether the ref exists, so don't set\n-\t\t * REF_ISBROKEN yet.\n-\t\t */\n-\t\tbad_name = 1;\n-\t}\n-\n-\tfor (symref_count = 0; symref_count < MAXDEPTH; symref_count++) {\n-\t\tunsigned int read_flags = 0;\n-\n-\t\tif (read_raw_ref(refname, sha1, sb_refname, sb_path, &read_flags)) {\n-\t\t\tint saved_errno = errno;\n-\t\t\tif (flags)\n-\t\t\t\t*flags |= read_flags;\n-\t\t\terrno = saved_errno;\n-\t\t\tif (bad_name) {\n-\t\t\t\thashclr(sha1);\n-\t\t\t\tif (flags)\n-\t\t\t\t\t*flags |= REF_ISBROKEN;\n-\t\t\t}\n-\t\t\tif (resolve_flags & RESOLVE_REF_READING || errno != ENOENT) {\n-\t\t\t\treturn NULL;\n-\t\t\t} else {\n-\t\t\t\thashclr(sha1);\n-\t\t\t\treturn refname;\n-\t\t\t}\n-\t\t}\n-\t\tif (flags)\n-\t\t\t*flags |= read_flags;\n-\n-\t\tif (!(read_flags & REF_ISSYMREF)) {\n-\t\t\tif (bad_name) {\n-\t\t\t\thashclr(sha1);\n-\t\t\t\tif (flags)\n-\t\t\t\t\t*flags |= REF_ISBROKEN;\n-\t\t\t}\n-\t\t\treturn refname;\n-\t\t}\n-\n-\t\trefname = sb_refname->buf;\n-\t\tif (resolve_flags & RESOLVE_REF_NO_RECURSE) {\n-\t\t\thashclr(sha1);\n-\t\t\tif (bad_name && flags)\n-\t\t\t\t*flags |= REF_ISBROKEN;\n-\t\t\treturn refname;\n-\t\t}\n-\n-\t\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n-\t\t\tif (flags)\n-\t\t\t\t*flags |= REF_ISBROKEN;\n-\t\t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n-\t\t\t !refname_is_safe(refname)) {\n-\t\t\t\terrno = EINVAL;\n-\t\t\t\treturn NULL;\n-\t\t\t}\n-\t\t\tbad_name = 1;\n-\t\t}\n-\t}\n-\n-\tif (flags)\n-\t\t*flags |= REF_ISBROKEN;\n-\treturn NULL;\n-}\n-\n-const char *resolve_ref_unsafe(const char *refname, int resolve_flags,\n-\t\t\t unsigned char *sha1, int *flags)\n-{\n-\tstatic struct strbuf sb_refname = STRBUF_INIT;\n-\tstruct strbuf sb_path = STRBUF_INIT;\n-\tconst char *ret;\n-\n-\tret = resolve_ref_1(refname, resolve_flags, sha1, flags,\n-\t\t\t &sb_refname, &sb_path);\n-\n-\tstrbuf_release(&sb_path);\n-\treturn ret;\n-}\n-\n /*\n * Peel the entry (if possible) and return its new peel_status. If\n * repeel is true, re-peel the entry even if there is an old peeled\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 92aae80..979a136 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -197,6 +197,8 @@ const char *find_descendant_ref(const char *dirname,\n \n int rename_ref_available(const char *oldname, const char *newname);\n \n+/* We allow \"recursive\" symbolic refs. Only within reason, though */\n+#define SYMREF_MAXDEPTH 5\n \n /* Include broken references in a do_for_each_ref*() iteration: */\n #define DO_FOR_EACH_INCLUDE_BROKEN 0x01\n@@ -206,4 +208,8 @@ int rename_ref_available(const char *oldname, const char *newname);\n */\n int do_for_each_ref(const char *submodule, const char *base,\n \t\t each_ref_fn fn, int trim, int flags, void *cb_data);\n+\n+int read_raw_ref(const char *refname, unsigned char *sha1,\n+\t\t struct strbuf *symref, struct strbuf *sb_path,\n+\t\t unsigned int *flags);\n #endif /* REFS_REFS_INTERNAL_H */\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279917","messageId":"1456793586-22082-7-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 06/33] refs: add a backend method structure with transaction functions","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:39Z","receivedAt":"2016-03-01T00:52:39Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"From: Ronnie Sahlberg \n\nAdd a ref structure for storage backend methods. Start by adding a\nmethod pointer for the transaction commit function.\n\nAdd a function set_refs_backend to switch between storage\nbackends. The files based storage backend is the default.\n\nSigned-off-by: Ronnie Sahlberg \nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \nSigned-off-by: Jeff King \n---\n refs.c | 40 ++++++++++++++++++++++++++++++++++++++++\n refs.h | 7 +++++++\n refs/files-backend.c | 10 ++++++++--\n refs/refs-internal.h | 12 ++++++++++++\n 4 files changed, 67 insertions(+), 2 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 15aabc6..2e8efa9 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -10,6 +10,38 @@\n #include \"tag.h\"\n \n /*\n+ * We always have a files backend and it is the default.\n+ */\n+static struct ref_storage_be *the_refs_backend = &refs_be_files;\n+/*\n+ * List of all available backends\n+ */\n+static struct ref_storage_be *refs_backends = &refs_be_files;\n+\n+static struct ref_storage_be *find_ref_storage_backend(const char *name)\n+{\n+\tstruct ref_storage_be *be;\n+\tfor (be = refs_backends; be; be = be->next)\n+\t\tif (!strcmp(be->name, name))\n+\t\t\treturn be;\n+\treturn NULL;\n+}\n+\n+int set_ref_storage_backend(const char *name)\n+{\n+\tstruct ref_storage_be *be = find_ref_storage_backend(name);\n+\tif (!be)\n+\t\treturn -1;\n+\tthe_refs_backend = be;\n+\treturn 0;\n+}\n+\n+int ref_storage_backend_exists(const char *name)\n+{\n+\treturn find_ref_storage_backend(name) != NULL;\n+}\n+\n+/*\n * How to handle various characters in refnames:\n * 0: An acceptable character for refs\n * 1: End-of-component\n@@ -1161,6 +1193,7 @@ int for_each_rawref(each_ref_fn fn, void *cb_data)\n \treturn do_for_each_ref(NULL, \"\", fn, 0,\n \t\t\t DO_FOR_EACH_INCLUDE_BROKEN, cb_data);\n }\n+\n /* This function needs to return a meaningful errno on failure */\n static const char *resolve_ref_1(const char *refname,\n \t\t\t\t int resolve_flags,\n@@ -1265,3 +1298,10 @@ const char *resolve_ref_unsafe(const char *refname, int resolve_flags,\n \tstrbuf_release(&sb_path);\n \treturn ret;\n }\n+\n+/* backend functions */\n+int ref_transaction_commit(struct ref_transaction *transaction,\n+\t\t\t struct strbuf *err)\n+{\n+\treturn the_refs_backend->transaction_commit(transaction, err);\n+}\ndiff --git a/refs.h b/refs.h\nindex 31a2fa6..3405842 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -509,4 +509,11 @@ extern int reflog_expire(const char *refname, const unsigned char *sha1,\n \t\t\t reflog_expiry_cleanup_fn cleanup_fn,\n \t\t\t void *policy_cb_data);\n \n+/*\n+ * Switch to an alternate ref storage backend.\n+ */\n+int set_ref_storage_backend(const char *name);\n+\n+int ref_storage_backend_exists(const char *name);\n+\n #endif /* REFS_H */\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 2fb7714..a509240 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3003,8 +3003,8 @@ static int ref_update_reject_duplicates(struct string_list *refnames,\n \treturn 0;\n }\n \n-int ref_transaction_commit(struct ref_transaction *transaction,\n-\t\t\t struct strbuf *err)\n+static int files_transaction_commit(struct ref_transaction *transaction,\n+\t\t\t\t struct strbuf *err)\n {\n \tint ret = 0, i;\n \tint n = transaction->nr;\n@@ -3390,3 +3390,9 @@ int reflog_expire(const char *refname, const unsigned char *sha1,\n \tunlock_ref(lock);\n \treturn -1;\n }\n+\n+struct ref_storage_be refs_be_files = {\n+\tNULL,\n+\t\"files\",\n+\tfiles_transaction_commit,\n+};\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 979a136..62ba0c0 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -212,4 +212,16 @@ int do_for_each_ref(const char *submodule, const char *base,\n int read_raw_ref(const char *refname, unsigned char *sha1,\n \t\t struct strbuf *symref, struct strbuf *sb_path,\n \t\t unsigned int *flags);\n+\n+/* refs backends */\n+typedef int ref_transaction_commit_fn(struct ref_transaction *transaction,\n+\t\t\t\t struct strbuf *err);\n+\n+struct ref_storage_be {\n+\tstruct ref_storage_be *next;\n+\tconst char *name;\n+\tref_transaction_commit_fn *transaction_commit;\n+};\n+\n+extern struct ref_storage_be refs_be_files;\n #endif /* REFS_REFS_INTERNAL_H */\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279912","messageId":"1456793586-22082-8-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 07/33] refs: add methods for misc ref operations","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:40Z","receivedAt":"2016-03-01T00:52:40Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"From: Ronnie Sahlberg \n\nAdd ref backend methods for:\nread_raw_ref, verify_refname_available, pack_refs, peel_ref,\ncreate_symref, resolve_gitlink_ref.\n\nread_raw_ref becomes static because it's not used outside refs.c\n\nSigned-off-by: Ronnie Sahlberg \nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n refs.c | 37 +++++++++++++++++++++++++++++++++++++\n refs/files-backend.c | 33 ++++++++++++++++++++++-----------\n refs/refs-internal.h | 27 +++++++++++++++++++++++----\n 3 files changed, 82 insertions(+), 15 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 2e8efa9..f4873d6 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1194,6 +1194,14 @@ int for_each_rawref(each_ref_fn fn, void *cb_data)\n \t\t\t DO_FOR_EACH_INCLUDE_BROKEN, cb_data);\n }\n \n+static int read_raw_ref(const char *refname, unsigned char *sha1,\n+\t\t\tstruct strbuf *symref, struct strbuf *sb_path,\n+\t\t\tunsigned int *flags)\n+{\n+\treturn the_refs_backend->read_raw_ref(refname, sha1, symref, sb_path,\n+\t\t\t\t\t flags);\n+}\n+\n /* This function needs to return a meaningful errno on failure */\n static const char *resolve_ref_1(const char *refname,\n \t\t\t\t int resolve_flags,\n@@ -1305,3 +1313,32 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n {\n \treturn the_refs_backend->transaction_commit(transaction, err);\n }\n+\n+int verify_refname_available(const char *refname, struct string_list *extra,\n+\t\t\t struct string_list *skip, struct strbuf *err)\n+{\n+\treturn the_refs_backend->verify_refname_available(refname, extra, skip, err);\n+}\n+\n+int pack_refs(unsigned int flags)\n+{\n+\treturn the_refs_backend->pack_refs(flags);\n+}\n+\n+int peel_ref(const char *refname, unsigned char *sha1)\n+{\n+\treturn the_refs_backend->peel_ref(refname, sha1);\n+}\n+\n+int create_symref(const char *ref_target, const char *refs_heads_master,\n+\t\t const char *logmsg)\n+{\n+\treturn the_refs_backend->create_symref(ref_target, refs_heads_master,\n+\t\t\t\t\t logmsg);\n+}\n+\n+int resolve_gitlink_ref(const char *path, const char *refname,\n+\t\t\tunsigned char *sha1)\n+{\n+\treturn the_refs_backend->resolve_gitlink_ref(path, refname, sha1);\n+}\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex a509240..96be63c 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1330,7 +1330,8 @@ static int resolve_gitlink_ref_recursive(struct ref_cache *refs,\n \treturn resolve_gitlink_ref_recursive(refs, p, sha1, recursion+1);\n }\n \n-int resolve_gitlink_ref(const char *path, const char *refname, unsigned char *sha1)\n+static int files_resolve_gitlink_ref(const char *path, const char *refname,\n+\t\t\t\t unsigned char *sha1)\n {\n \tint len = strlen(path), retval;\n \tstruct strbuf submodule = STRBUF_INIT;\n@@ -1413,9 +1414,9 @@ static int resolve_missing_loose_ref(const char *refname,\n *\n * sb_path is workspace: the caller should allocate and free it.\n */\n-int read_raw_ref(const char *refname, unsigned char *sha1,\n-\t\t struct strbuf *symref, struct strbuf *sb_path,\n-\t\t unsigned int *flags)\n+static int files_read_raw_ref(const char *refname, unsigned char *sha1,\n+\t\t\t struct strbuf *symref, struct strbuf *sb_path,\n+\t\t\t unsigned int *flags)\n {\n \tstruct strbuf sb_contents = STRBUF_INIT;\n \tint ret = -1;\n@@ -1568,7 +1569,7 @@ static enum peel_status peel_entry(struct ref_entry *entry, int repeel)\n \treturn status;\n }\n \n-int peel_ref(const char *refname, unsigned char *sha1)\n+static int files_peel_ref(const char *refname, unsigned char *sha1)\n {\n \tint flag;\n \tunsigned char base[20];\n@@ -2127,7 +2128,7 @@ static void prune_refs(struct ref_to_prune *r)\n \t}\n }\n \n-int pack_refs(unsigned int flags)\n+static int files_pack_refs(unsigned int flags)\n {\n \tstruct pack_refs_cb_data cbdata;\n \n@@ -2318,10 +2319,10 @@ out:\n \treturn ret;\n }\n \n-int verify_refname_available(const char *newname,\n-\t\t\t struct string_list *extras,\n-\t\t\t struct string_list *skip,\n-\t\t\t struct strbuf *err)\n+static int files_verify_refname_available(const char *newname,\n+\t\t\t\t\t struct string_list *extras,\n+\t\t\t\t\t struct string_list *skip,\n+\t\t\t\t\t struct strbuf *err)\n {\n \tstruct ref_dir *packed_refs = get_packed_refs(&ref_cache);\n \tstruct ref_dir *loose_refs = get_loose_refs(&ref_cache);\n@@ -2741,7 +2742,9 @@ static int create_symref_locked(struct ref_lock *lock, const char *refname,\n \treturn 0;\n }\n \n-int create_symref(const char *refname, const char *target, const char *logmsg)\n+static int files_create_symref(const char *refname,\n+\t\t\t const char *target,\n+\t\t\t const char *logmsg)\n {\n \tstruct strbuf err = STRBUF_INIT;\n \tstruct ref_lock *lock;\n@@ -3395,4 +3398,12 @@ struct ref_storage_be refs_be_files = {\n \tNULL,\n \t\"files\",\n \tfiles_transaction_commit,\n+\n+\tfiles_pack_refs,\n+\tfiles_peel_ref,\n+\tfiles_create_symref,\n+\n+\tfiles_read_raw_ref,\n+\tfiles_verify_refname_available,\n+\tfiles_resolve_gitlink_ref,\n };\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 62ba0c0..c5f5ef7 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -209,18 +209,37 @@ int rename_ref_available(const char *oldname, const char *newname);\n int do_for_each_ref(const char *submodule, const char *base,\n \t\t each_ref_fn fn, int trim, int flags, void *cb_data);\n \n-int read_raw_ref(const char *refname, unsigned char *sha1,\n-\t\t struct strbuf *symref, struct strbuf *sb_path,\n-\t\t unsigned int *flags);\n-\n /* refs backends */\n typedef int ref_transaction_commit_fn(struct ref_transaction *transaction,\n \t\t\t\t struct strbuf *err);\n \n+/* misc methods */\n+typedef int pack_refs_fn(unsigned int flags);\n+typedef int peel_ref_fn(const char *refname, unsigned char *sha1);\n+typedef int create_symref_fn(const char *ref_target,\n+\t\t\t const char *refs_heads_master,\n+\t\t\t const char *logmsg);\n+\n+/* resolution methods */\n+typedef int read_raw_ref_fn(const char *refname, unsigned char *sha1,\n+\t\t\t struct strbuf *symref, struct strbuf *sb_path,\n+\t\t\t unsigned int *flags);\n+typedef int verify_refname_available_fn(const char *refname, struct string_list *extra, struct string_list *skip, struct strbuf *err);\n+typedef int resolve_gitlink_ref_fn(const char *path, const char *refname,\n+\t\t\t\t unsigned char *sha1);\n+\n struct ref_storage_be {\n \tstruct ref_storage_be *next;\n \tconst char *name;\n \tref_transaction_commit_fn *transaction_commit;\n+\n+\tpack_refs_fn *pack_refs;\n+\tpeel_ref_fn *peel_ref;\n+\tcreate_symref_fn *create_symref;\n+\n+\tread_raw_ref_fn *read_raw_ref;\n+\tverify_refname_available_fn *verify_refname_available;\n+\tresolve_gitlink_ref_fn *resolve_gitlink_ref;\n };\n \n extern struct ref_storage_be refs_be_files;\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279888","messageId":"1456793586-22082-9-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 08/33] refs: add method for do_for_each_ref","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:41Z","receivedAt":"2016-03-01T00:52:41Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"Add a ref backend method for do_for_each_ref.\n\nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n refs.c | 8 ++++++++\n refs/files-backend.c | 7 +++++--\n refs/refs-internal.h | 5 +++++\n 3 files changed, 18 insertions(+), 2 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex f4873d6..dc5682a 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1342,3 +1342,11 @@ int resolve_gitlink_ref(const char *path, const char *refname,\n {\n \treturn the_refs_backend->resolve_gitlink_ref(path, refname, sha1);\n }\n+\n+int do_for_each_ref(const char *submodule, const char *base,\n+\t\t each_ref_fn fn, int trim, int flags,\n+\t\t void *cb_data)\n+{\n+\treturn the_refs_backend->do_for_each_ref(submodule, base, fn, trim,\n+\t\t\t\t\t\t flags, cb_data);\n+}\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 96be63c..f3d857a 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1671,8 +1671,9 @@ static int do_for_each_entry(struct ref_cache *refs, const char *base,\n * value, stop the iteration and return that value; otherwise, return\n * 0.\n */\n-int do_for_each_ref(const char *submodule, const char *base,\n-\t\t each_ref_fn fn, int trim, int flags, void *cb_data)\n+static int files_do_for_each_ref(const char *submodule, const char *base,\n+\t\t\t\t each_ref_fn fn, int trim, int flags,\n+\t\t\t\t void *cb_data)\n {\n \tstruct ref_entry_cb data;\n \tstruct ref_cache *refs;\n@@ -3406,4 +3407,6 @@ struct ref_storage_be refs_be_files = {\n \tfiles_read_raw_ref,\n \tfiles_verify_refname_available,\n \tfiles_resolve_gitlink_ref,\n+\n+\tfiles_do_for_each_ref,\n };\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex c5f5ef7..c9b6745 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -227,6 +227,9 @@ typedef int read_raw_ref_fn(const char *refname, unsigned char *sha1,\n typedef int verify_refname_available_fn(const char *refname, struct string_list *extra, struct string_list *skip, struct strbuf *err);\n typedef int resolve_gitlink_ref_fn(const char *path, const char *refname,\n \t\t\t\t unsigned char *sha1);\n+typedef int do_for_each_ref_fn(const char *submodule, const char *base,\n+\t\t\t each_ref_fn fn, int trim, int flags,\n+\t\t\t void *cb_data);\n \n struct ref_storage_be {\n \tstruct ref_storage_be *next;\n@@ -240,6 +243,8 @@ struct ref_storage_be {\n \tread_raw_ref_fn *read_raw_ref;\n \tverify_refname_available_fn *verify_refname_available;\n \tresolve_gitlink_ref_fn *resolve_gitlink_ref;\n+\n+\tdo_for_each_ref_fn *do_for_each_ref;\n };\n \n extern struct ref_storage_be refs_be_files;\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279913","messageId":"1456793586-22082-10-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 09/33] refs: reduce the visibility of do_for_each_ref()","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:42Z","receivedAt":"2016-03-01T00:52:42Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"From: Ramsay Jones \n\nNow that we have moved do_for_each_ref into refs.c, it no longer needs\nto be public.\n\nSigned-off-by: Ramsay Jones \nSigned-off-by: Junio C Hamano \n---\n refs.c | 19 +++++++++++--------\n refs/refs-internal.h | 6 ------\n 2 files changed, 11 insertions(+), 14 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex dc5682a..cea5997 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1142,6 +1142,17 @@ int head_ref(each_ref_fn fn, void *cb_data)\n \treturn head_ref_submodule(NULL, fn, cb_data);\n }\n \n+/*\n+ * The common backend for the for_each_*ref* functions\n+ */\n+static int do_for_each_ref(const char *submodule, const char *base,\n+\t\t each_ref_fn fn, int trim, int flags,\n+\t\t void *cb_data)\n+{\n+\treturn the_refs_backend->do_for_each_ref(submodule, base, fn, trim,\n+\t\t\t\t\t\t flags, cb_data);\n+}\n+\n int for_each_ref(each_ref_fn fn, void *cb_data)\n {\n \treturn do_for_each_ref(NULL, \"\", fn, 0, 0, cb_data);\n@@ -1342,11 +1353,3 @@ int resolve_gitlink_ref(const char *path, const char *refname,\n {\n \treturn the_refs_backend->resolve_gitlink_ref(path, refname, sha1);\n }\n-\n-int do_for_each_ref(const char *submodule, const char *base,\n-\t\t each_ref_fn fn, int trim, int flags,\n-\t\t void *cb_data)\n-{\n-\treturn the_refs_backend->do_for_each_ref(submodule, base, fn, trim,\n-\t\t\t\t\t\t flags, cb_data);\n-}\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex c9b6745..3702737 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -203,12 +203,6 @@ int rename_ref_available(const char *oldname, const char *newname);\n /* Include broken references in a do_for_each_ref*() iteration: */\n #define DO_FOR_EACH_INCLUDE_BROKEN 0x01\n \n-/*\n- * The common backend for the for_each_*ref* functions\n- */\n-int do_for_each_ref(const char *submodule, const char *base,\n-\t\t each_ref_fn fn, int trim, int flags, void *cb_data);\n-\n /* refs backends */\n typedef int ref_transaction_commit_fn(struct ref_transaction *transaction,\n \t\t\t\t struct strbuf *err);\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279890","messageId":"1456793586-22082-11-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 10/33] refs: add do_for_each_per_worktree_ref","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:43Z","receivedAt":"2016-03-01T00:52:43Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"Alternate refs backends might still use files to store per-worktree\nrefs. So the files backend's ref-loading infrastructure should be\navailable to those backends, just for use on per-worktree refs. Add\ndo_for_each_per_worktree_ref, which iterates over per-worktree refs.\n\nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n refs/files-backend.c | 16 ++++++++++++++++\n refs/refs-internal.h | 7 +++++++\n 2 files changed, 23 insertions(+)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex f3d857a..31f38d0 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -560,6 +560,10 @@ static int do_one_ref(struct ref_entry *entry, void *cb_data)\n \tstruct ref_entry *old_current_ref;\n \tint retval;\n \n+\tif (data->flags & DO_FOR_EACH_PER_WORKTREE_ONLY &&\n+\t ref_type(entry->name) != REF_TYPE_PER_WORKTREE)\n+\t\treturn 0;\n+\n \tif (!starts_with(entry->name, data->base))\n \t\treturn 0;\n \n@@ -1693,6 +1697,18 @@ static int files_do_for_each_ref(const char *submodule, const char *base,\n \treturn do_for_each_entry(refs, base, do_one_ref, &data);\n }\n \n+int do_for_each_per_worktree_ref(const char *submodule, const char *base,\n+\t\t\t\t each_ref_fn fn, int trim, int flags,\n+\t\t\t\t void *cb_data)\n+{\n+\t/*\n+\t * It's important that this one use the files backend, since\n+\t * that's what controls the per-worktree refs\n+\t */\n+\treturn files_do_for_each_ref(submodule, base, fn, trim,\n+\t\t\t\t flags | DO_FOR_EACH_PER_WORKTREE_ONLY, cb_data);\n+}\n+\n static void unlock_ref(struct ref_lock *lock)\n {\n \t/* Do not free lock->lk -- atexit() still looks at them */\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 3702737..bb0d588 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -203,6 +203,13 @@ int rename_ref_available(const char *oldname, const char *newname);\n /* Include broken references in a do_for_each_ref*() iteration: */\n #define DO_FOR_EACH_INCLUDE_BROKEN 0x01\n \n+/* Only include per-worktree refs in a do_for_each_ref*() iteration */\n+#define DO_FOR_EACH_PER_WORKTREE_ONLY 0x02\n+\n+int do_for_each_per_worktree_ref(const char *submodule, const char *base,\n+\t\t\t\t each_ref_fn fn, int trim, int flags,\n+\t\t\t\t void *cb_data);\n+\n /* refs backends */\n typedef int ref_transaction_commit_fn(struct ref_transaction *transaction,\n \t\t\t\t struct strbuf *err);\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279915","messageId":"1456793586-22082-12-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 11/33] refs: add methods for reflog","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:44Z","receivedAt":"2016-03-01T00:52:44Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"In the file-based backend, the reflog piggybacks on the ref lock.\nSince other backends won't have the same sort of ref lock, ref backends\nmust also handle reflogs.\n\nSigned-off-by: Ronnie Sahlberg \nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n refs.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++\n refs/files-backend.c | 36 ++++++++++++++++++++++++------------\n refs/refs-internal.h | 27 +++++++++++++++++++++++++++\n 3 files changed, 97 insertions(+), 12 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex cea5997..637df71 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1353,3 +1353,49 @@ int resolve_gitlink_ref(const char *path, const char *refname,\n {\n \treturn the_refs_backend->resolve_gitlink_ref(path, refname, sha1);\n }\n+\n+int for_each_reflog_ent_reverse(const char *refname, each_reflog_ent_fn fn,\n+\t\t\t\tvoid *cb_data)\n+{\n+\treturn the_refs_backend->for_each_reflog_ent_reverse(refname, fn,\n+\t\t\t\t\t\t\t cb_data);\n+}\n+\n+int for_each_reflog_ent(const char *refname, each_reflog_ent_fn fn,\n+\t\t\tvoid *cb_data)\n+{\n+\treturn the_refs_backend->for_each_reflog_ent(refname, fn, cb_data);\n+}\n+\n+int for_each_reflog(each_ref_fn fn, void *cb_data)\n+{\n+\treturn the_refs_backend->for_each_reflog(fn, cb_data);\n+}\n+\n+int reflog_exists(const char *refname)\n+{\n+\treturn the_refs_backend->reflog_exists(refname);\n+}\n+\n+int safe_create_reflog(const char *refname, int force_create,\n+\t\t struct strbuf *err)\n+{\n+\treturn the_refs_backend->create_reflog(refname, force_create, err);\n+}\n+\n+int delete_reflog(const char *refname)\n+{\n+\treturn the_refs_backend->delete_reflog(refname);\n+}\n+\n+int reflog_expire(const char *refname, const unsigned char *sha1,\n+\t\t unsigned int flags,\n+\t\t reflog_expiry_prepare_fn prepare_fn,\n+\t\t reflog_expiry_should_prune_fn should_prune_fn,\n+\t\t reflog_expiry_cleanup_fn cleanup_fn,\n+\t\t void *policy_cb_data)\n+{\n+\treturn the_refs_backend->reflog_expire(refname, sha1, flags,\n+\t\t\t\t\t prepare_fn, should_prune_fn,\n+\t\t\t\t\t cleanup_fn, policy_cb_data);\n+}\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 31f38d0..a25fda4 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -2524,7 +2524,8 @@ static int log_ref_setup(const char *refname, struct strbuf *logfile, struct str\n }\n \n \n-int safe_create_reflog(const char *refname, int force_create, struct strbuf *err)\n+static int files_create_reflog(const char *refname, int force_create,\n+\t\t\t struct strbuf *err)\n {\n \tint ret;\n \tstruct strbuf sb = STRBUF_INIT;\n@@ -2780,7 +2781,7 @@ static int files_create_symref(const char *refname,\n \treturn ret;\n }\n \n-int reflog_exists(const char *refname)\n+static int files_reflog_exists(const char *refname)\n {\n \tstruct stat st;\n \n@@ -2788,7 +2789,7 @@ int reflog_exists(const char *refname)\n \t\tS_ISREG(st.st_mode);\n }\n \n-int delete_reflog(const char *refname)\n+static int files_delete_reflog(const char *refname)\n {\n \treturn remove_path(git_path(\"logs/%s\", refname));\n }\n@@ -2832,7 +2833,9 @@ static char *find_beginning_of_line(char *bob, char *scan)\n \treturn scan;\n }\n \n-int for_each_reflog_ent_reverse(const char *refname, each_reflog_ent_fn fn, void *cb_data)\n+static int files_for_each_reflog_ent_reverse(const char *refname,\n+\t\t\t\t\t each_reflog_ent_fn fn,\n+\t\t\t\t\t void *cb_data)\n {\n \tstruct strbuf sb = STRBUF_INIT;\n \tFILE *logfp;\n@@ -2934,7 +2937,8 @@ int for_each_reflog_ent_reverse(const char *refname, each_reflog_ent_fn fn, void\n \treturn ret;\n }\n \n-int for_each_reflog_ent(const char *refname, each_reflog_ent_fn fn, void *cb_data)\n+static int files_for_each_reflog_ent(const char *refname,\n+\t\t\t\t each_reflog_ent_fn fn, void *cb_data)\n {\n \tFILE *logfp;\n \tstruct strbuf sb = STRBUF_INIT;\n@@ -2996,7 +3000,7 @@ static int do_for_each_reflog(struct strbuf *name, each_ref_fn fn, void *cb_data\n \treturn retval;\n }\n \n-int for_each_reflog(each_ref_fn fn, void *cb_data)\n+static int files_for_each_reflog(each_ref_fn fn, void *cb_data)\n {\n \tint retval;\n \tstruct strbuf name;\n@@ -3306,12 +3310,12 @@ static int expire_reflog_ent(unsigned char *osha1, unsigned char *nsha1,\n \treturn 0;\n }\n \n-int reflog_expire(const char *refname, const unsigned char *sha1,\n-\t\t unsigned int flags,\n-\t\t reflog_expiry_prepare_fn prepare_fn,\n-\t\t reflog_expiry_should_prune_fn should_prune_fn,\n-\t\t reflog_expiry_cleanup_fn cleanup_fn,\n-\t\t void *policy_cb_data)\n+static int files_reflog_expire(const char *refname, const unsigned char *sha1,\n+\t\t\t unsigned int flags,\n+\t\t\t reflog_expiry_prepare_fn prepare_fn,\n+\t\t\t reflog_expiry_should_prune_fn should_prune_fn,\n+\t\t\t reflog_expiry_cleanup_fn cleanup_fn,\n+\t\t\t void *policy_cb_data)\n {\n \tstatic struct lock_file reflog_lock;\n \tstruct expire_reflog_cb cb;\n@@ -3416,6 +3420,14 @@ struct ref_storage_be refs_be_files = {\n \t\"files\",\n \tfiles_transaction_commit,\n \n+\tfiles_for_each_reflog_ent,\n+\tfiles_for_each_reflog_ent_reverse,\n+\tfiles_for_each_reflog,\n+\tfiles_reflog_exists,\n+\tfiles_create_reflog,\n+\tfiles_delete_reflog,\n+\tfiles_reflog_expire,\n+\n \tfiles_pack_refs,\n \tfiles_peel_ref,\n \tfiles_create_symref,\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex bb0d588..c92e0c3 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -214,6 +214,25 @@ int do_for_each_per_worktree_ref(const char *submodule, const char *base,\n typedef int ref_transaction_commit_fn(struct ref_transaction *transaction,\n \t\t\t\t struct strbuf *err);\n \n+/* reflog functions */\n+typedef int for_each_reflog_ent_fn(const char *refname,\n+\t\t\t\t each_reflog_ent_fn fn,\n+\t\t\t\t void *cb_data);\n+typedef int for_each_reflog_ent_reverse_fn(const char *refname,\n+\t\t\t\t\t each_reflog_ent_fn fn,\n+\t\t\t\t\t void *cb_data);\n+typedef int for_each_reflog_fn(each_ref_fn fn, void *cb_data);\n+typedef int reflog_exists_fn(const char *refname);\n+typedef int create_reflog_fn(const char *refname, int force_create,\n+\t\t\t struct strbuf *err);\n+typedef int delete_reflog_fn(const char *refname);\n+typedef int reflog_expire_fn(const char *refname, const unsigned char *sha1,\n+\t\t\t unsigned int flags,\n+\t\t\t reflog_expiry_prepare_fn prepare_fn,\n+\t\t\t reflog_expiry_should_prune_fn should_prune_fn,\n+\t\t\t reflog_expiry_cleanup_fn cleanup_fn,\n+\t\t\t void *policy_cb_data);\n+\n /* misc methods */\n typedef int pack_refs_fn(unsigned int flags);\n typedef int peel_ref_fn(const char *refname, unsigned char *sha1);\n@@ -237,6 +256,14 @@ struct ref_storage_be {\n \tconst char *name;\n \tref_transaction_commit_fn *transaction_commit;\n \n+\tfor_each_reflog_ent_fn *for_each_reflog_ent;\n+\tfor_each_reflog_ent_reverse_fn *for_each_reflog_ent_reverse;\n+\tfor_each_reflog_fn *for_each_reflog;\n+\treflog_exists_fn *reflog_exists;\n+\tcreate_reflog_fn *create_reflog;\n+\tdelete_reflog_fn *delete_reflog;\n+\treflog_expire_fn *reflog_expire;\n+\n \tpack_refs_fn *pack_refs;\n \tpeel_ref_fn *peel_ref;\n \tcreate_symref_fn *create_symref;\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279894","messageId":"1456793586-22082-13-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 12/33] refs: add method for initial ref transaction commit","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:45Z","receivedAt":"2016-03-01T00:52:45Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"Signed-off-by: Ronnie Sahlberg \nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n refs.c | 6 ++++++\n refs/files-backend.c | 5 +++--\n refs/refs-internal.h | 1 +\n 3 files changed, 10 insertions(+), 2 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 637df71..b5ef4fc 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1399,3 +1399,9 @@ int reflog_expire(const char *refname, const unsigned char *sha1,\n \t\t\t\t\t prepare_fn, should_prune_fn,\n \t\t\t\t\t cleanup_fn, policy_cb_data);\n }\n+\n+int initial_ref_transaction_commit(struct ref_transaction *transaction,\n+\t\t\t\t struct strbuf *err)\n+{\n+\treturn the_refs_backend->initial_transaction_commit(transaction, err);\n+}\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex a25fda4..1906875 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3194,8 +3194,8 @@ static int ref_present(const char *refname,\n \treturn string_list_has_string(affected_refnames, refname);\n }\n \n-int initial_ref_transaction_commit(struct ref_transaction *transaction,\n-\t\t\t\t struct strbuf *err)\n+static int files_initial_transaction_commit(struct ref_transaction *transaction,\n+\t\t\t\t\t struct strbuf *err)\n {\n \tint ret = 0, i;\n \tint n = transaction->nr;\n@@ -3419,6 +3419,7 @@ struct ref_storage_be refs_be_files = {\n \tNULL,\n \t\"files\",\n \tfiles_transaction_commit,\n+\tfiles_initial_transaction_commit,\n \n \tfiles_for_each_reflog_ent,\n \tfiles_for_each_reflog_ent_reverse,\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex c92e0c3..5aa3fa0 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -255,6 +255,7 @@ struct ref_storage_be {\n \tstruct ref_storage_be *next;\n \tconst char *name;\n \tref_transaction_commit_fn *transaction_commit;\n+\tref_transaction_commit_fn *initial_transaction_commit;\n \n \tfor_each_reflog_ent_fn *for_each_reflog_ent;\n \tfor_each_reflog_ent_reverse_fn *for_each_reflog_ent_reverse;\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279916","messageId":"1456793586-22082-14-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 13/33] refs: add method for delete_refs","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:46Z","receivedAt":"2016-03-01T00:52:46Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"In the file-based backend, delete_refs has some special optimization\nto deal with packed refs. In other backends, we might be able to make\nref deletion faster by putting all deletions into a single\ntransaction. So we need a special backend function for this.\n\nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n refs.c | 5 +++++\n refs/files-backend.c | 3 ++-\n refs/refs-internal.h | 2 ++\n 3 files changed, 9 insertions(+), 1 deletion(-)\n\ndiff --git a/refs.c b/refs.c\nindex b5ef4fc..d25eee2 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1405,3 +1405,8 @@ int initial_ref_transaction_commit(struct ref_transaction *transaction,\n {\n \treturn the_refs_backend->initial_transaction_commit(transaction, err);\n }\n+\n+int delete_refs(struct string_list *refnames)\n+{\n+\treturn the_refs_backend->delete_refs(refnames);\n+}\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 1906875..35328d2 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -2237,7 +2237,7 @@ static int delete_ref_loose(struct ref_lock *lock, int flag, struct strbuf *err)\n \treturn 0;\n }\n \n-int delete_refs(struct string_list *refnames)\n+static int files_delete_refs(struct string_list *refnames)\n {\n \tstruct strbuf err = STRBUF_INIT;\n \tint i, result = 0;\n@@ -3432,6 +3432,7 @@ struct ref_storage_be refs_be_files = {\n \tfiles_pack_refs,\n \tfiles_peel_ref,\n \tfiles_create_symref,\n+\tfiles_delete_refs,\n \n \tfiles_read_raw_ref,\n \tfiles_verify_refname_available,\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 5aa3fa0..beef457 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -239,6 +239,7 @@ typedef int peel_ref_fn(const char *refname, unsigned char *sha1);\n typedef int create_symref_fn(const char *ref_target,\n \t\t\t const char *refs_heads_master,\n \t\t\t const char *logmsg);\n+typedef int delete_refs_fn(struct string_list *refnames);\n \n /* resolution methods */\n typedef int read_raw_ref_fn(const char *refname, unsigned char *sha1,\n@@ -268,6 +269,7 @@ struct ref_storage_be {\n \tpack_refs_fn *pack_refs;\n \tpeel_ref_fn *peel_ref;\n \tcreate_symref_fn *create_symref;\n+\tdelete_refs_fn *delete_refs;\n \n \tread_raw_ref_fn *read_raw_ref;\n \tverify_refname_available_fn *verify_refname_available;\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279893","messageId":"1456793586-22082-15-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 14/33] refs: add methods to init refs db","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:47Z","receivedAt":"2016-03-01T00:52:47Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"Alternate refs backends might not need the refs/heads directory and so\non, so we make ref db initialization part of the backend.\n\nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n builtin/init-db.c | 20 ++++++++++----------\n refs.c | 5 +++++\n refs.h | 2 ++\n refs/files-backend.c | 16 ++++++++++++++++\n refs/refs-internal.h | 2 ++\n 5 files changed, 35 insertions(+), 10 deletions(-)\n\ndiff --git a/builtin/init-db.c b/builtin/init-db.c\nindex 6223b7d..e6d4e86 100644\n--- a/builtin/init-db.c\n+++ b/builtin/init-db.c\n@@ -177,13 +177,7 @@ static int create_default_files(const char *template_path)\n \tchar junk[2];\n \tint reinit;\n \tint filemode;\n-\n-\t/*\n-\t * Create .git/refs/{heads,tags}\n-\t */\n-\tsafe_create_dir(git_path_buf(&buf, \"refs\"), 1);\n-\tsafe_create_dir(git_path_buf(&buf, \"refs/heads\"), 1);\n-\tsafe_create_dir(git_path_buf(&buf, \"refs/tags\"), 1);\n+\tstruct strbuf err = STRBUF_INIT;\n \n \t/* Just look for `init.templatedir` */\n \tgit_config(git_init_db_config, NULL);\n@@ -207,12 +201,18 @@ static int create_default_files(const char *template_path)\n \t */\n \tif (shared_repository) {\n \t\tadjust_shared_perm(get_git_dir());\n-\t\tadjust_shared_perm(git_path_buf(&buf, \"refs\"));\n-\t\tadjust_shared_perm(git_path_buf(&buf, \"refs/heads\"));\n-\t\tadjust_shared_perm(git_path_buf(&buf, \"refs/tags\"));\n \t}\n \n \t/*\n+\t * We need to create a \"refs\" dir in any case so that older\n+\t * versions of git can tell that this is a repository.\n+\t */\n+\tsafe_create_dir(git_path(\"refs\"), 1);\n+\n+\tif (refs_init_db(shared_repository, &err))\n+\t\tdie(\"failed to set up refs db: %s\", err.buf);\n+\n+\t/*\n \t * Create the default symlink from \".git/HEAD\" to the \"master\"\n \t * branch, if it does not exist yet.\n \t */\ndiff --git a/refs.c b/refs.c\nindex d25eee2..b2697f6 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1319,6 +1319,11 @@ const char *resolve_ref_unsafe(const char *refname, int resolve_flags,\n }\n \n /* backend functions */\n+int refs_init_db(int shared, struct strbuf *err)\n+{\n+\treturn the_refs_backend->init_db(shared, err);\n+}\n+\n int ref_transaction_commit(struct ref_transaction *transaction,\n \t\t\t struct strbuf *err)\n {\ndiff --git a/refs.h b/refs.h\nindex 3405842..13ce2a0 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -66,6 +66,8 @@ extern int ref_exists(const char *refname);\n \n extern int is_branch(const char *refname);\n \n+extern int refs_init_db(int shared, struct strbuf *err);\n+\n /*\n * If refname is a non-symbolic reference that refers to a tag object,\n * and the tag can be (recursively) dereferenced to a non-tag object,\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 35328d2..acb4401 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3415,9 +3415,25 @@ static int files_reflog_expire(const char *refname, const unsigned char *sha1,\n \treturn -1;\n }\n \n+static int files_init_db(int shared, struct strbuf *err)\n+{\n+\t/*\n+\t * Create .git/refs/{heads,tags}\n+\t */\n+\tsafe_create_dir(git_path(\"refs/heads\"), 1);\n+\tsafe_create_dir(git_path(\"refs/tags\"), 1);\n+\tif (shared) {\n+\t\tadjust_shared_perm(git_path(\"refs\"));\n+\t\tadjust_shared_perm(git_path(\"refs/heads\"));\n+\t\tadjust_shared_perm(git_path(\"refs/tags\"));\n+\t}\n+\treturn 0;\n+}\n+\n struct ref_storage_be refs_be_files = {\n \tNULL,\n \t\"files\",\n+\tfiles_init_db,\n \tfiles_transaction_commit,\n \tfiles_initial_transaction_commit,\n \ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex beef457..dfd0326 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -211,6 +211,7 @@ int do_for_each_per_worktree_ref(const char *submodule, const char *base,\n \t\t\t\t void *cb_data);\n \n /* refs backends */\n+typedef int ref_init_db_fn(int shared, struct strbuf *err);\n typedef int ref_transaction_commit_fn(struct ref_transaction *transaction,\n \t\t\t\t struct strbuf *err);\n \n@@ -255,6 +256,7 @@ typedef int do_for_each_ref_fn(const char *submodule, const char *base,\n struct ref_storage_be {\n \tstruct ref_storage_be *next;\n \tconst char *name;\n+\tref_init_db_fn *init_db;\n \tref_transaction_commit_fn *transaction_commit;\n \tref_transaction_commit_fn *initial_transaction_commit;\n \n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279918","messageId":"1456793586-22082-16-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 15/33] refs: add method to rename refs","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:48Z","receivedAt":"2016-03-01T00:52:48Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"Signed-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n refs.c | 5 +++++\n refs/files-backend.c | 4 +++-\n refs/refs-internal.h | 8 ++++++++\n 3 files changed, 16 insertions(+), 1 deletion(-)\n\ndiff --git a/refs.c b/refs.c\nindex b2697f6..5364dc6 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1415,3 +1415,8 @@ int delete_refs(struct string_list *refnames)\n {\n \treturn the_refs_backend->delete_refs(refnames);\n }\n+\n+int rename_ref(const char *oldref, const char *newref, const char *logmsg)\n+{\n+\treturn the_refs_backend->rename_ref(oldref, newref, logmsg);\n+}\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex acb4401..534bbaf 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -2359,7 +2359,8 @@ static int commit_ref_update(struct ref_lock *lock,\n \t\t\t const unsigned char *sha1, const char *logmsg,\n \t\t\t int flags, struct strbuf *err);\n \n-int rename_ref(const char *oldrefname, const char *newrefname, const char *logmsg)\n+static int files_rename_ref(const char *oldrefname, const char *newrefname,\n+\t\t\t const char *logmsg)\n {\n \tunsigned char sha1[20], orig_sha1[20];\n \tint flag = 0, logmoved = 0;\n@@ -3449,6 +3450,7 @@ struct ref_storage_be refs_be_files = {\n \tfiles_peel_ref,\n \tfiles_create_symref,\n \tfiles_delete_refs,\n+\tfiles_rename_ref,\n \n \tfiles_read_raw_ref,\n \tfiles_verify_refname_available,\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex dfd0326..d6fe199 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -195,6 +195,11 @@ const char *find_descendant_ref(const char *dirname,\n \t\t\t\tconst struct string_list *extras,\n \t\t\t\tconst struct string_list *skip);\n \n+/*\n+ * Check if the new name does not conflict with any existing refs\n+ * (other than possibly the old ref). Return 0 if the ref can be\n+ * renamed to the new name.\n+ */\n int rename_ref_available(const char *oldname, const char *newname);\n \n /* We allow \"recursive\" symbolic refs. Only within reason, though */\n@@ -241,6 +246,8 @@ typedef int create_symref_fn(const char *ref_target,\n \t\t\t const char *refs_heads_master,\n \t\t\t const char *logmsg);\n typedef int delete_refs_fn(struct string_list *refnames);\n+typedef int rename_ref_fn(const char *oldref, const char *newref,\n+\t\t\t const char *logmsg);\n \n /* resolution methods */\n typedef int read_raw_ref_fn(const char *refname, unsigned char *sha1,\n@@ -272,6 +279,7 @@ struct ref_storage_be {\n \tpeel_ref_fn *peel_ref;\n \tcreate_symref_fn *create_symref;\n \tdelete_refs_fn *delete_refs;\n+\trename_ref_fn *rename_ref;\n \n \tread_raw_ref_fn *read_raw_ref;\n \tverify_refname_available_fn *verify_refname_available;\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279895","messageId":"1456793586-22082-17-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 16/33] refs: handle non-normal ref renames","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:49Z","receivedAt":"2016-03-01T00:52:49Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"Forbid cross-backend ref renames. This would be pretty weird, but\nsince it will break, we should prevent it.\n\nAlso make the files backend deal with all non-normal ref renames.\n\nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n refs.c | 12 +++++++++++-\n 1 file changed, 11 insertions(+), 1 deletion(-)\n\ndiff --git a/refs.c b/refs.c\nindex 5364dc6..4610a44 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1418,5 +1418,15 @@ int delete_refs(struct string_list *refnames)\n \n int rename_ref(const char *oldref, const char *newref, const char *logmsg)\n {\n-\treturn the_refs_backend->rename_ref(oldref, newref, logmsg);\n+\tif ((ref_type(oldref) == REF_TYPE_NORMAL) !=\n+\t (ref_type(newref) == REF_TYPE_NORMAL)) {\n+\t\treturn error(\n+\t\t\t_(\"Both ref arguments to rename_ref must be normal, \"\n+\t\t\t \"or both must be per-worktree/pseudorefs\"));\n+\t}\n+\tif (ref_type(oldref) == REF_TYPE_NORMAL)\n+\t\t/* The files backend always deals with non-normal refs */\n+\t\treturn the_refs_backend->rename_ref(oldref, newref, logmsg);\n+\telse\n+\t\treturn refs_be_files.rename_ref(oldref, newref, logmsg);\n }\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279892","messageId":"1456793586-22082-18-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 17/33] refs: make lock generic","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:50Z","receivedAt":"2016-03-01T00:52:50Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"Instead of using a files-backend-specific struct ref_lock, the generic\nref_transaction struct should provide a void pointer that backends can use\nfor their own lock data.\n\nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n refs/files-backend.c | 29 ++++++++++++++++-------------\n refs/refs-internal.h | 2 +-\n 2 files changed, 17 insertions(+), 14 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 534bbaf..01c85c0 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3065,11 +3065,12 @@ static int files_transaction_commit(struct ref_transaction *transaction,\n \t */\n \tfor (i = 0; i < n; i++) {\n \t\tstruct ref_update *update = updates[i];\n+\t\tstruct ref_lock *lock;\n \n \t\tif ((update->flags & REF_HAVE_NEW) &&\n \t\t is_null_sha1(update->new_sha1))\n \t\t\tupdate->flags |= REF_DELETING;\n-\t\tupdate->lock = lock_ref_sha1_basic(\n+\t\tlock = lock_ref_sha1_basic(\n \t\t\t\tupdate->refname,\n \t\t\t\t((update->flags & REF_HAVE_OLD) ?\n \t\t\t\t update->old_sha1 : NULL),\n@@ -3077,7 +3078,8 @@ static int files_transaction_commit(struct ref_transaction *transaction,\n \t\t\t\tupdate->flags,\n \t\t\t\t&update->type,\n \t\t\t\terr);\n-\t\tif (!update->lock) {\n+\t\tupdate->backend_data = lock;\n+\t\tif (!lock) {\n \t\t\tchar *reason;\n \n \t\t\tret = (errno == ENOTDIR)\n@@ -3095,12 +3097,12 @@ static int files_transaction_commit(struct ref_transaction *transaction,\n \t\t\t\t\t\t (update->flags & REF_NODEREF));\n \n \t\t\tif (!overwriting_symref &&\n-\t\t\t !hashcmp(update->lock->old_oid.hash, update->new_sha1)) {\n+\t\t\t !hashcmp(lock->old_oid.hash, update->new_sha1)) {\n \t\t\t\t/*\n \t\t\t\t * The reference already has the desired\n \t\t\t\t * value, so we don't need to write it.\n \t\t\t\t */\n-\t\t\t} else if (write_ref_to_lockfile(update->lock,\n+\t\t\t} else if (write_ref_to_lockfile(lock,\n \t\t\t\t\t\t\t update->new_sha1,\n \t\t\t\t\t\t\t err)) {\n \t\t\t\tchar *write_err = strbuf_detach(err, NULL);\n@@ -3109,7 +3111,7 @@ static int files_transaction_commit(struct ref_transaction *transaction,\n \t\t\t\t * The lock was freed upon failure of\n \t\t\t\t * write_ref_to_lockfile():\n \t\t\t\t */\n-\t\t\t\tupdate->lock = NULL;\n+\t\t\t\tupdate->backend_data = NULL;\n \t\t\t\tstrbuf_addf(err,\n \t\t\t\t\t \"cannot update the ref '%s': %s\",\n \t\t\t\t\t update->refname, write_err);\n@@ -3125,7 +3127,7 @@ static int files_transaction_commit(struct ref_transaction *transaction,\n \t\t\t * We didn't have to write anything to the lockfile.\n \t\t\t * Close it to free up the file descriptor:\n \t\t\t */\n-\t\t\tif (close_ref(update->lock)) {\n+\t\t\tif (close_ref(lock)) {\n \t\t\t\tstrbuf_addf(err, \"Couldn't close %s.lock\",\n \t\t\t\t\t update->refname);\n \t\t\t\tgoto cleanup;\n@@ -3138,16 +3140,16 @@ static int files_transaction_commit(struct ref_transaction *transaction,\n \t\tstruct ref_update *update = updates[i];\n \n \t\tif (update->flags & REF_NEEDS_COMMIT) {\n-\t\t\tif (commit_ref_update(update->lock,\n+\t\t\tif (commit_ref_update(update->backend_data,\n \t\t\t\t\t update->new_sha1, update->msg,\n \t\t\t\t\t update->flags, err)) {\n \t\t\t\t/* freed by commit_ref_update(): */\n-\t\t\t\tupdate->lock = NULL;\n+\t\t\t\tupdate->backend_data = NULL;\n \t\t\t\tret = TRANSACTION_GENERIC_ERROR;\n \t\t\t\tgoto cleanup;\n \t\t\t} else {\n \t\t\t\t/* freed by commit_ref_update(): */\n-\t\t\t\tupdate->lock = NULL;\n+\t\t\t\tupdate->backend_data = NULL;\n \t\t\t}\n \t\t}\n \t}\n@@ -3155,16 +3157,17 @@ static int files_transaction_commit(struct ref_transaction *transaction,\n \t/* Perform deletes now that updates are safely completed */\n \tfor (i = 0; i < n; i++) {\n \t\tstruct ref_update *update = updates[i];\n+\t\tstruct ref_lock *lock = update->backend_data;\n \n \t\tif (update->flags & REF_DELETING) {\n-\t\t\tif (delete_ref_loose(update->lock, update->type, err)) {\n+\t\t\tif (delete_ref_loose(lock, update->type, err)) {\n \t\t\t\tret = TRANSACTION_GENERIC_ERROR;\n \t\t\t\tgoto cleanup;\n \t\t\t}\n \n \t\t\tif (!(update->flags & REF_ISPRUNING))\n \t\t\t\tstring_list_append(&refs_to_delete,\n-\t\t\t\t\t\t update->lock->ref_name);\n+\t\t\t\t\t\t lock->ref_name);\n \t\t}\n \t}\n \n@@ -3180,8 +3183,8 @@ cleanup:\n \ttransaction->state = REF_TRANSACTION_CLOSED;\n \n \tfor (i = 0; i < n; i++)\n-\t\tif (updates[i]->lock)\n-\t\t\tunlock_ref(updates[i]->lock);\n+\t\tif (updates[i]->backend_data)\n+\t\t\tunlock_ref(updates[i]->backend_data);\n \tstring_list_clear(&refs_to_delete, 0);\n \tstring_list_clear(&affected_refnames, 0);\n \treturn ret;\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex d6fe199..8d091cb 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -144,7 +144,7 @@ struct ref_update {\n \t * REF_DELETING, and REF_ISPRUNING:\n \t */\n \tunsigned int flags;\n-\tstruct ref_lock *lock;\n+\tvoid *backend_data;\n \tint type;\n \tchar *msg;\n \tconst char refname[FLEX_ARRAY];\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279896","messageId":"1456793586-22082-19-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 18/33] refs: move duplicate check to common code","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:51Z","receivedAt":"2016-03-01T00:52:51Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"The check for duplicate refnames in a transaction is needed for\nall backends, so move it to the common code.\n\nref_transaction_commit_fn gains a new argument, the sorted\nstring_list of affected refnames.\n\nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n refs.c | 69 ++++++++++++++++++++++++++++++++++++++++++++++++++--\n refs/files-backend.c | 57 ++++---------------------------------------\n refs/refs-internal.h | 1 +\n 3 files changed, 73 insertions(+), 54 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 4610a44..9f695cd 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1143,6 +1143,36 @@ int head_ref(each_ref_fn fn, void *cb_data)\n }\n \n /*\n+ * Return 1 if there are any duplicate refnames in the updates in\n+ * `transaction`, and fill in err with an appropriate error message.\n+ * Fill in `refnames` with the refnames from the transaction.\n+ */\n+static int get_affected_refnames(struct ref_transaction *transaction,\n+\t\t\t\t struct string_list *refnames,\n+\t\t\t\t struct strbuf *err)\n+{\n+\tint i, n = transaction->nr;\n+\tstruct ref_update **updates;\n+\n+\tassert(err);\n+\n+\tupdates = transaction->updates;\n+\t/* Fail if a refname appears more than once in the transaction: */\n+\tfor (i = 0; i < n; i++)\n+\t\tstring_list_append(refnames, updates[i]->refname);\n+\tstring_list_sort(refnames);\n+\n+\tfor (i = 1; i < n; i++)\n+\t\tif (!strcmp(refnames->items[i - 1].string, refnames->items[i].string)) {\n+\t\t\tstrbuf_addf(err,\n+\t\t\t\t \"Multiple updates for ref '%s' not allowed.\",\n+\t\t\t\t refnames->items[i].string);\n+\t\t\treturn 1;\n+\t\t}\n+\treturn 0;\n+}\n+\n+/*\n * The common backend for the for_each_*ref* functions\n */\n static int do_for_each_ref(const char *submodule, const char *base,\n@@ -1327,7 +1357,29 @@ int refs_init_db(int shared, struct strbuf *err)\n int ref_transaction_commit(struct ref_transaction *transaction,\n \t\t\t struct strbuf *err)\n {\n-\treturn the_refs_backend->transaction_commit(transaction, err);\n+\tint ret = -1;\n+\tstruct string_list affected_refnames = STRING_LIST_INIT_NODUP;\n+\n+\tassert(err);\n+\n+\tif (transaction->state != REF_TRANSACTION_OPEN)\n+\t\tdie(\"BUG: commit called for transaction that is not open\");\n+\n+\tif (!transaction->nr) {\n+\t\ttransaction->state = REF_TRANSACTION_CLOSED;\n+\t\treturn 0;\n+\t}\n+\n+\tif (get_affected_refnames(transaction, &affected_refnames, err)) {\n+\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\tgoto done;\n+\t}\n+\n+\tret = the_refs_backend->transaction_commit(transaction,\n+\t\t\t\t\t\t &affected_refnames, err);\n+done:\n+\tstring_list_clear(&affected_refnames, 0);\n+\treturn ret;\n }\n \n int verify_refname_available(const char *refname, struct string_list *extra,\n@@ -1408,7 +1460,20 @@ int reflog_expire(const char *refname, const unsigned char *sha1,\n int initial_ref_transaction_commit(struct ref_transaction *transaction,\n \t\t\t\t struct strbuf *err)\n {\n-\treturn the_refs_backend->initial_transaction_commit(transaction, err);\n+\tstruct string_list affected_refnames = STRING_LIST_INIT_NODUP;\n+\tint ret;\n+\n+\tif (get_affected_refnames(transaction,\n+\t\t\t\t &affected_refnames, err)) {\n+\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\tgoto done;\n+\t}\n+\tret = the_refs_backend->initial_transaction_commit(transaction,\n+\t\t\t\t\t\t\t &affected_refnames,\n+\t\t\t\t\t\t\t err);\n+done:\n+\tstring_list_clear(&affected_refnames, 0);\n+\treturn ret;\n }\n \n int delete_refs(struct string_list *refnames)\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 01c85c0..1f565cb 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3011,24 +3011,8 @@ static int files_for_each_reflog(each_ref_fn fn, void *cb_data)\n \treturn retval;\n }\n \n-static int ref_update_reject_duplicates(struct string_list *refnames,\n-\t\t\t\t\tstruct strbuf *err)\n-{\n-\tint i, n = refnames->nr;\n-\n-\tassert(err);\n-\n-\tfor (i = 1; i < n; i++)\n-\t\tif (!strcmp(refnames->items[i - 1].string, refnames->items[i].string)) {\n-\t\t\tstrbuf_addf(err,\n-\t\t\t\t \"Multiple updates for ref '%s' not allowed.\",\n-\t\t\t\t refnames->items[i].string);\n-\t\t\treturn 1;\n-\t\t}\n-\treturn 0;\n-}\n-\n static int files_transaction_commit(struct ref_transaction *transaction,\n+\t\t\t\t struct string_list *affected_refnames,\n \t\t\t\t struct strbuf *err)\n {\n \tint ret = 0, i;\n@@ -3036,26 +3020,6 @@ static int files_transaction_commit(struct ref_transaction *transaction,\n \tstruct ref_update **updates = transaction->updates;\n \tstruct string_list refs_to_delete = STRING_LIST_INIT_NODUP;\n \tstruct string_list_item *ref_to_delete;\n-\tstruct string_list affected_refnames = STRING_LIST_INIT_NODUP;\n-\n-\tassert(err);\n-\n-\tif (transaction->state != REF_TRANSACTION_OPEN)\n-\t\tdie(\"BUG: commit called for transaction that is not open\");\n-\n-\tif (!n) {\n-\t\ttransaction->state = REF_TRANSACTION_CLOSED;\n-\t\treturn 0;\n-\t}\n-\n-\t/* Fail if a refname appears more than once in the transaction: */\n-\tfor (i = 0; i < n; i++)\n-\t\tstring_list_append(&affected_refnames, updates[i]->refname);\n-\tstring_list_sort(&affected_refnames);\n-\tif (ref_update_reject_duplicates(&affected_refnames, err)) {\n-\t\tret = TRANSACTION_GENERIC_ERROR;\n-\t\tgoto cleanup;\n-\t}\n \n \t/*\n \t * Acquire all locks, verify old values if provided, check\n@@ -3074,7 +3038,7 @@ static int files_transaction_commit(struct ref_transaction *transaction,\n \t\t\t\tupdate->refname,\n \t\t\t\t((update->flags & REF_HAVE_OLD) ?\n \t\t\t\t update->old_sha1 : NULL),\n-\t\t\t\t&affected_refnames, NULL,\n+\t\t\t\taffected_refnames, NULL,\n \t\t\t\tupdate->flags,\n \t\t\t\t&update->type,\n \t\t\t\terr);\n@@ -3186,7 +3150,6 @@ cleanup:\n \t\tif (updates[i]->backend_data)\n \t\t\tunlock_ref(updates[i]->backend_data);\n \tstring_list_clear(&refs_to_delete, 0);\n-\tstring_list_clear(&affected_refnames, 0);\n \treturn ret;\n }\n \n@@ -3199,27 +3162,18 @@ static int ref_present(const char *refname,\n }\n \n static int files_initial_transaction_commit(struct ref_transaction *transaction,\n+\t\t\t\t\t struct string_list *affected_refnames,\n \t\t\t\t\t struct strbuf *err)\n {\n \tint ret = 0, i;\n \tint n = transaction->nr;\n \tstruct ref_update **updates = transaction->updates;\n-\tstruct string_list affected_refnames = STRING_LIST_INIT_NODUP;\n \n \tassert(err);\n \n \tif (transaction->state != REF_TRANSACTION_OPEN)\n \t\tdie(\"BUG: commit called for transaction that is not open\");\n \n-\t/* Fail if a refname appears more than once in the transaction: */\n-\tfor (i = 0; i < n; i++)\n-\t\tstring_list_append(&affected_refnames, updates[i]->refname);\n-\tstring_list_sort(&affected_refnames);\n-\tif (ref_update_reject_duplicates(&affected_refnames, err)) {\n-\t\tret = TRANSACTION_GENERIC_ERROR;\n-\t\tgoto cleanup;\n-\t}\n-\n \t/*\n \t * It's really undefined to call this function in an active\n \t * repository or when there are existing references: we are\n@@ -3232,7 +3186,7 @@ static int files_initial_transaction_commit(struct ref_transaction *transaction,\n \t * so here we really only check that none of the references\n \t * that we are creating already exists.\n \t */\n-\tif (for_each_rawref(ref_present, &affected_refnames))\n+\tif (for_each_rawref(ref_present, affected_refnames))\n \t\tdie(\"BUG: initial ref transaction called with existing refs\");\n \n \tfor (i = 0; i < n; i++) {\n@@ -3242,7 +3196,7 @@ static int files_initial_transaction_commit(struct ref_transaction *transaction,\n \t\t !is_null_sha1(update->old_sha1))\n \t\t\tdie(\"BUG: initial ref transaction with old_sha1 set\");\n \t\tif (verify_refname_available(update->refname,\n-\t\t\t\t\t &affected_refnames, NULL,\n+\t\t\t\t\t affected_refnames, NULL,\n \t\t\t\t\t err)) {\n \t\t\tret = TRANSACTION_NAME_CONFLICT;\n \t\t\tgoto cleanup;\n@@ -3273,7 +3227,6 @@ static int files_initial_transaction_commit(struct ref_transaction *transaction,\n \n cleanup:\n \ttransaction->state = REF_TRANSACTION_CLOSED;\n-\tstring_list_clear(&affected_refnames, 0);\n \treturn ret;\n }\n \ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 8d091cb..94d162e 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -218,6 +218,7 @@ int do_for_each_per_worktree_ref(const char *submodule, const char *base,\n /* refs backends */\n typedef int ref_init_db_fn(int shared, struct strbuf *err);\n typedef int ref_transaction_commit_fn(struct ref_transaction *transaction,\n+\t\t\t\t struct string_list *affected_refnames,\n \t\t\t\t struct strbuf *err);\n \n /* reflog functions */\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279902","messageId":"1456793586-22082-20-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 19/33] refs: allow log-only updates","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:52Z","receivedAt":"2016-03-01T00:52:52Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"The refs infrastructure learns about log-only ref updates, which only\nupdate the reflog. Later, we will use this to separate symbolic\nreference resolution from ref updating.\n\nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n refs/files-backend.c | 15 ++++++++++-----\n refs/refs-internal.h | 7 +++++++\n 2 files changed, 17 insertions(+), 5 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 1f565cb..189b86e 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -2702,7 +2702,7 @@ static int commit_ref_update(struct ref_lock *lock,\n \t\t\t}\n \t\t}\n \t}\n-\tif (commit_ref(lock)) {\n+\tif (!(flags & REF_LOG_ONLY) && commit_ref(lock)) {\n \t\terror(\"Couldn't set %s\", lock->ref_name);\n \t\tunlock_ref(lock);\n \t\treturn -1;\n@@ -3056,7 +3056,8 @@ static int files_transaction_commit(struct ref_transaction *transaction,\n \t\t\tgoto cleanup;\n \t\t}\n \t\tif ((update->flags & REF_HAVE_NEW) &&\n-\t\t !(update->flags & REF_DELETING)) {\n+\t\t !(update->flags & REF_DELETING) &&\n+\t\t !(update->flags & REF_LOG_ONLY)) {\n \t\t\tint overwriting_symref = ((update->type & REF_ISSYMREF) &&\n \t\t\t\t\t\t (update->flags & REF_NODEREF));\n \n@@ -3086,7 +3087,9 @@ static int files_transaction_commit(struct ref_transaction *transaction,\n \t\t\t\tupdate->flags |= REF_NEEDS_COMMIT;\n \t\t\t}\n \t\t}\n-\t\tif (!(update->flags & REF_NEEDS_COMMIT)) {\n+\n+\t\tif (!(update->flags & REF_LOG_ONLY) &&\n+\t\t !(update->flags & REF_NEEDS_COMMIT)) {\n \t\t\t/*\n \t\t\t * We didn't have to write anything to the lockfile.\n \t\t\t * Close it to free up the file descriptor:\n@@ -3103,7 +3106,8 @@ static int files_transaction_commit(struct ref_transaction *transaction,\n \tfor (i = 0; i < n; i++) {\n \t\tstruct ref_update *update = updates[i];\n \n-\t\tif (update->flags & REF_NEEDS_COMMIT) {\n+\t\tif (update->flags & REF_NEEDS_COMMIT ||\n+\t\t update->flags & REF_LOG_ONLY) {\n \t\t\tif (commit_ref_update(update->backend_data,\n \t\t\t\t\t update->new_sha1, update->msg,\n \t\t\t\t\t update->flags, err)) {\n@@ -3123,7 +3127,8 @@ static int files_transaction_commit(struct ref_transaction *transaction,\n \t\tstruct ref_update *update = updates[i];\n \t\tstruct ref_lock *lock = update->backend_data;\n \n-\t\tif (update->flags & REF_DELETING) {\n+\t\tif (update->flags & REF_DELETING &&\n+\t\t !(update->flags & REF_LOG_ONLY)) {\n \t\t\tif (delete_ref_loose(lock, update->type, err)) {\n \t\t\t\tret = TRANSACTION_GENERIC_ERROR;\n \t\t\t\tgoto cleanup;\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 94d162e..dd76246 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -43,6 +43,13 @@\n */\n \n /*\n+ * Used as a flag in ref_update::flags when we want to log a ref\n+ * update but not actually perform it. This is used when a symbolic\n+ * ref update is split up.\n+ */\n+#define REF_LOG_ONLY 0x80\n+\n+/*\n * Return true iff refname is minimally safe. \"Safe\" here means that\n * deleting a loose reference by this name will not do any damage, for\n * example by causing a file that is not a reference to be deleted.\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279910","messageId":"1456793586-22082-21-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 20/33] refs: don't dereference on rename","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:53Z","receivedAt":"2016-03-01T00:52:53Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"When renaming refs, don't dereference either the origin or the destination\nbefore renaming.\n\nThe origin does not need to be dereferenced because it is presently\nforbidden to rename symbolic refs.\n\nNot dereferencing the destination fixes a bug where renaming on top of\na broken symref would use the pointed-to ref name for the moved\nreflog.\n\nAdd a test for the reflog bug.\n\nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n refs/files-backend.c | 15 +++++++++------\n t/t3200-branch.sh | 9 +++++++++\n 2 files changed, 18 insertions(+), 6 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 189b86e..7c557b2 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -2363,7 +2363,7 @@ static int files_rename_ref(const char *oldrefname, const char *newrefname,\n \t\t\t const char *logmsg)\n {\n \tunsigned char sha1[20], orig_sha1[20];\n-\tint flag = 0, logmoved = 0;\n+\tint flag = 0, logmoved = 0, resolve_flags;\n \tstruct ref_lock *lock;\n \tstruct stat loginfo;\n \tint log = !lstat(git_path(\"logs/%s\", oldrefname), &loginfo);\n@@ -2373,7 +2373,8 @@ static int files_rename_ref(const char *oldrefname, const char *newrefname,\n \tif (log && S_ISLNK(loginfo.st_mode))\n \t\treturn error(\"reflog for %s is a symlink\", oldrefname);\n \n-\tsymref = resolve_ref_unsafe(oldrefname, RESOLVE_REF_READING,\n+\tresolve_flags = RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE;\n+\tsymref = resolve_ref_unsafe(oldrefname, resolve_flags,\n \t\t\t\t orig_sha1, &flag);\n \tif (flag & REF_ISSYMREF)\n \t\treturn error(\"refname %s is a symbolic ref, renaming it is not supported\",\n@@ -2393,8 +2394,8 @@ static int files_rename_ref(const char *oldrefname, const char *newrefname,\n \t\tgoto rollback;\n \t}\n \n-\tif (!read_ref_full(newrefname, RESOLVE_REF_READING, sha1, NULL) &&\n-\t delete_ref(newrefname, sha1, REF_NODEREF)) {\n+\tif (!read_ref_full(newrefname, resolve_flags, sha1, NULL) &&\n+\t delete_ref(newrefname, NULL, REF_NODEREF)) {\n \t\tif (errno==EISDIR) {\n \t\t\tstruct strbuf path = STRBUF_INIT;\n \t\t\tint result;\n@@ -2418,7 +2419,8 @@ static int files_rename_ref(const char *oldrefname, const char *newrefname,\n \n \tlogmoved = log;\n \n-\tlock = lock_ref_sha1_basic(newrefname, NULL, NULL, NULL, 0, NULL, &err);\n+\tlock = lock_ref_sha1_basic(newrefname, NULL, NULL, NULL, REF_NODEREF,\n+\t\t\t\t NULL, &err);\n \tif (!lock) {\n \t\terror(\"unable to rename '%s' to '%s': %s\", oldrefname, newrefname, err.buf);\n \t\tstrbuf_release(&err);\n@@ -2436,7 +2438,8 @@ static int files_rename_ref(const char *oldrefname, const char *newrefname,\n \treturn 0;\n \n rollback:\n-\tlock = lock_ref_sha1_basic(oldrefname, NULL, NULL, NULL, 0, NULL, &err);\n+\tlock = lock_ref_sha1_basic(oldrefname, NULL, NULL, NULL, REF_NODEREF,\n+\t\t\t\t NULL, &err);\n \tif (!lock) {\n \t\terror(\"unable to lock %s for rollback: %s\", oldrefname, err.buf);\n \t\tstrbuf_release(&err);\ndiff --git a/t/t3200-branch.sh b/t/t3200-branch.sh\nindex a897248..144e9ce 100755\n--- a/t/t3200-branch.sh\n+++ b/t/t3200-branch.sh\n@@ -79,6 +79,15 @@ test_expect_success 'git branch -m dumps usage' '\n \ttest_i18ngrep \"branch name required\" err\n '\n \n+test_expect_success 'git branch -m m broken_symref should work' '\n+\ttest_when_finished \"git branch -D broken_symref\" &&\n+\tgit branch -l m &&\n+\tgit symbolic-ref refs/heads/broken_symref refs/heads/i_am_broken &&\n+\tgit branch -m m broken_symref &&\n+\tgit reflog exists refs/heads/broken_symref &&\n+\ttest_must_fail git reflog exists refs/heads/i_am_broken\n+'\n+\n test_expect_success 'git branch -m m m/m should work' '\n \tgit branch -l m &&\n \tgit branch -m m m/m &&\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279897","messageId":"1456793586-22082-22-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 21/33] refs: on symref reflog expire, lock symref not referrent","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:54Z","receivedAt":"2016-03-01T00:52:54Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"When locking a symbolic ref to expire a reflog, lock the symbolic\nref (using REF_NODEREF) instead of its referent.\n\nAdd a test for this.\n\nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n refs/files-backend.c | 3 ++-\n t/t1410-reflog.sh | 10 ++++++++++\n 2 files changed, 12 insertions(+), 1 deletion(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 7c557b2..1f36dde 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3300,7 +3300,8 @@ static int files_reflog_expire(const char *refname, const unsigned char *sha1,\n \t * reference itself, plus we might need to update the\n \t * reference if --updateref was specified:\n \t */\n-\tlock = lock_ref_sha1_basic(refname, sha1, NULL, NULL, 0, &type, &err);\n+\tlock = lock_ref_sha1_basic(refname, sha1, NULL, NULL, REF_NODEREF,\n+\t\t\t\t &type, &err);\n \tif (!lock) {\n \t\terror(\"cannot lock ref '%s': %s\", refname, err.buf);\n \t\tstrbuf_release(&err);\ndiff --git a/t/t1410-reflog.sh b/t/t1410-reflog.sh\nindex c623824..9cf91dc 100755\n--- a/t/t1410-reflog.sh\n+++ b/t/t1410-reflog.sh\n@@ -338,4 +338,14 @@ test_expect_failure 'reflog with non-commit entries displays all entries' '\n \ttest_line_count = 3 actual\n '\n \n+test_expect_success 'reflog expire operates on symref not referrent' '\n+\tgit branch -l the_symref &&\n+\tgit branch -l referrent &&\n+\tgit update-ref referrent HEAD &&\n+\tgit symbolic-ref refs/heads/the_symref refs/heads/referrent &&\n+\ttest_when_finished \"rm -f .git/refs/heads/referrent.lock\" &&\n+\ttouch .git/refs/heads/referrent.lock &&\n+\tgit reflog expire --expire=all the_symref\n+'\n+\n test_done\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279898","messageId":"1456793586-22082-23-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 22/33] refs: resolve symbolic refs first","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:55Z","receivedAt":"2016-03-01T00:52:55Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"Before committing ref updates, split symbolic ref updates into two\nparts: an update to the underlying ref, and a log-only update to the\nsymbolic ref. This ensures that both references are locked correctly\nwhile their reflogs are updated.\n\nIt is still possible to confuse git by concurrent updates, since the\nsplitting of symbolic refs does not happen under lock. So a symbolic ref\ncould be replaced by a plain ref in the middle of this operation, which\nwould lead to reflog discontinuities and missed old-ref checks.\n\nAll callers to lock_ref_sha1_basic outside of ref_transaction_commit\nnow set REF_NODEREF. And ref_transaction_commit only happens on\nalready-derefernced refs. So we can assume REF_NODEREF when resolving\ninside this function.\n\nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n refs.c | 69 ++++++++++++++++++++++++++\n refs/files-backend.c | 137 ++++++++++++++++++++++++++-------------------------\n refs/refs-internal.h | 8 +++\n 3 files changed, 148 insertions(+), 66 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 9f695cd..eee8c37 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1354,6 +1354,71 @@ int refs_init_db(int shared, struct strbuf *err)\n \treturn the_refs_backend->init_db(shared, err);\n }\n \n+/*\n+ * Special case for symbolic refs when REF_NODEREF is not turned on.\n+ * Dereference them here, mark them REF_LOG_ONLY, and add an update\n+ * for the underlying ref.\n+ */\n+static int dereference_symrefs(struct ref_transaction *transaction,\n+\t\t\t struct strbuf *err)\n+{\n+\tint i;\n+\tint nr = transaction->nr;\n+\n+\tfor (i = 0; i < nr; i++) {\n+\t\tstruct ref_update *update = transaction->updates[i];\n+\t\tconst char *resolved;\n+\t\tunsigned char sha1[20];\n+\t\tint resolve_flags = 0;\n+\t\tint mustexist = update->flags & REF_HAVE_OLD &&\n+\t\t\t!is_null_sha1(update->old_sha1);\n+\t\tint deleting = (update->flags & REF_HAVE_NEW) &&\n+\t\t\tis_null_sha1(update->new_sha1);\n+\n+\t\tif (mustexist)\n+\t\t\tresolve_flags |= RESOLVE_REF_READING;\n+\t\tif (deleting)\n+\t\t\tresolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME |\n+\t\t\t\tRESOLVE_REF_NO_RECURSE;\n+\n+\t\tif (strcmp(update->refname, \"HEAD\"))\n+\t\t\tupdate->flags |= REF_IS_NOT_HEAD;\n+\n+\t\tresolved = resolve_ref_unsafe(update->refname, resolve_flags,\n+\t\t\t\t\t sha1, &update->type);\n+\t\tif (!resolved) {\n+\t\t\t/*\n+\t\t\t * We may notice this breakage later and die\n+\t\t\t * with a sensible error message\n+\t\t\t */\n+\t\t\tupdate->type |= REF_ISBROKEN;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\thashcpy(update->read_sha1, sha1);\n+\n+\t\tif (update->flags & REF_NODEREF ||\n+\t\t !(update->type & REF_ISSYMREF))\n+\t\t\tcontinue;\n+\n+\t\t/* Create a new transaction for the underlying ref */\n+\t\tif (ref_transaction_update(transaction,\n+\t\t\t\t\t resolved,\n+\t\t\t\t\t update->new_sha1,\n+\t\t\t\t\t (update->flags & REF_HAVE_OLD) ?\n+\t\t\t\t\t update->old_sha1 : NULL,\n+\t\t\t\t\t update->flags & ~REF_IS_NOT_HEAD,\n+\t\t\t\t\t update->msg, err))\n+\t\t\treturn -1;\n+\n+\t\t/* Make the symbolic ref update non-recursive */\n+\t\tupdate->flags |= REF_LOG_ONLY | REF_NODEREF;\n+\t\tupdate->flags &= ~REF_HAVE_OLD;\n+\t}\n+\n+\treturn 0;\n+}\n+\n int ref_transaction_commit(struct ref_transaction *transaction,\n \t\t\t struct strbuf *err)\n {\n@@ -1370,6 +1435,10 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n \t\treturn 0;\n \t}\n \n+\tret = dereference_symrefs(transaction, err);\n+\tif (ret)\n+\t\tgoto done;\n+\n \tif (get_affected_refnames(transaction, &affected_refnames, err)) {\n \t\tret = TRANSACTION_GENERIC_ERROR;\n \t\tgoto done;\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 1f36dde..7ccaf88 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -7,7 +7,6 @@\n \n struct ref_lock {\n \tchar *ref_name;\n-\tchar *orig_ref_name;\n \tstruct lock_file *lk;\n \tstruct object_id old_oid;\n };\n@@ -1715,7 +1714,6 @@ static void unlock_ref(struct ref_lock *lock)\n \tif (lock->lk)\n \t\trollback_lock_file(lock->lk);\n \tfree(lock->ref_name);\n-\tfree(lock->orig_ref_name);\n \tfree(lock);\n }\n \n@@ -1771,6 +1769,7 @@ static int remove_empty_directories(struct strbuf *path)\n */\n static struct ref_lock *lock_ref_sha1_basic(const char *refname,\n \t\t\t\t\t const unsigned char *old_sha1,\n+\t\t\t\t\t const unsigned char *read_sha1,\n \t\t\t\t\t const struct string_list *extras,\n \t\t\t\t\t const struct string_list *skip,\n \t\t\t\t\t unsigned int flags, int *type_p,\n@@ -1778,14 +1777,14 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,\n {\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tstruct strbuf orig_ref_file = STRBUF_INIT;\n-\tconst char *orig_refname = refname;\n \tstruct ref_lock *lock;\n \tint last_errno = 0;\n \tint type;\n \tint lflags = 0;\n \tint mustexist = (old_sha1 && !is_null_sha1(old_sha1));\n-\tint resolve_flags = 0;\n+\tint resolve_flags = RESOLVE_REF_NO_RECURSE;\n \tint attempts_remaining = 3;\n+\tint resolved;\n \n \tassert(err);\n \n@@ -1795,65 +1794,65 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,\n \t\tresolve_flags |= RESOLVE_REF_READING;\n \tif (flags & REF_DELETING)\n \t\tresolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;\n-\tif (flags & REF_NODEREF) {\n-\t\tresolve_flags |= RESOLVE_REF_NO_RECURSE;\n+\tif (flags & REF_NODEREF)\n \t\tlflags |= LOCK_NO_DEREF;\n-\t}\n \n-\trefname = resolve_ref_unsafe(refname, resolve_flags,\n-\t\t\t\t lock->old_oid.hash, &type);\n-\tif (!refname && errno == EISDIR) {\n-\t\t/*\n-\t\t * we are trying to lock foo but we used to\n-\t\t * have foo/bar which now does not exist;\n-\t\t * it is normal for the empty directory 'foo'\n-\t\t * to remain.\n-\t\t */\n-\t\tstrbuf_git_path(&orig_ref_file, \"%s\", orig_refname);\n-\t\tif (remove_empty_directories(&orig_ref_file)) {\n+\tif (type_p && *type_p & REF_ISSYMREF) {\n+\t\thashcpy(lock->old_oid.hash, read_sha1);\n+\t} else {\n+\t\tresolved = !!resolve_ref_unsafe(refname, resolve_flags,\n+\t\t\t\t\t\tlock->old_oid.hash, &type);\n+\t\tif (!resolved && errno == EISDIR) {\n+\t\t\t/*\n+\t\t\t * we are trying to lock foo but we used to\n+\t\t\t * have foo/bar which now does not exist;\n+\t\t\t * it is normal for the empty directory 'foo'\n+\t\t\t * to remain.\n+\t\t\t */\n+\t\t\tstrbuf_git_path(&orig_ref_file, \"%s\", refname);\n+\t\t\tif (remove_empty_directories(&orig_ref_file)) {\n+\t\t\t\tstruct ref_dir *loose_refs;\n+\t\t\t\tloose_refs = get_loose_refs(&ref_cache);\n+\t\t\t\tlast_errno = errno;\n+\t\t\t\tif (!verify_refname_available_dir(refname, extras, skip,\n+\t\t\t\t\t\t\t\t loose_refs, err))\n+\t\t\t\t\tstrbuf_addf(err, \"there are still refs under '%s'\",\n+\t\t\t\t\t\t refname);\n+\t\t\t\tgoto error_return;\n+\t\t\t}\n+\t\t\tresolved = !!resolve_ref_unsafe(refname, resolve_flags,\n+\t\t\t\t\t\t\tlock->old_oid.hash, &type);\n+\t\t}\n+\n+\t\tif (type_p)\n+\t\t\t*type_p = type;\n+\t\tif (!resolved) {\n \t\t\tlast_errno = errno;\n-\t\t\tif (!verify_refname_available_dir(orig_refname, extras, skip,\n+\t\t\tif (last_errno != ENOTDIR ||\n+\t\t\t !verify_refname_available_dir(refname, extras, skip,\n \t\t\t\t\t\t\t get_loose_refs(&ref_cache), err))\n-\t\t\t\tstrbuf_addf(err, \"there are still refs under '%s'\",\n-\t\t\t\t\t orig_refname);\n+\t\t\t\tstrbuf_addf(err,\n+\t\t\t\t\t \"unable to resolve reference %s: %s\",\n+\t\t\t\t\t refname, strerror(last_errno));\n+\n+\t\t\tgoto error_return;\n+\t\t}\n+\t\t/*\n+\t\t * If the ref did not exist and we are creating it, make sure\n+\t\t * there is no existing packed ref whose name begins with our\n+\t\t * refname, nor a packed ref whose name is a proper prefix of\n+\t\t * our refname.\n+\t\t */\n+\t\tif (is_null_oid(&lock->old_oid) &&\n+\t\t verify_refname_available_dir(refname, extras, skip,\n+\t\t\t\t\t\t get_packed_refs(&ref_cache), err)) {\n+\t\t\tlast_errno = ENOTDIR;\n \t\t\tgoto error_return;\n \t\t}\n-\t\trefname = resolve_ref_unsafe(orig_refname, resolve_flags,\n-\t\t\t\t\t lock->old_oid.hash, &type);\n-\t}\n-\tif (type_p)\n-\t *type_p = type;\n-\tif (!refname) {\n-\t\tlast_errno = errno;\n-\t\tif (last_errno != ENOTDIR ||\n-\t\t !verify_refname_available_dir(orig_refname, extras, skip,\n-\t\t\t\t\t\t get_loose_refs(&ref_cache), err))\n-\t\t\tstrbuf_addf(err, \"unable to resolve reference %s: %s\",\n-\t\t\t\t orig_refname, strerror(last_errno));\n-\n-\t\tgoto error_return;\n-\t}\n-\n-\tif (flags & REF_NODEREF)\n-\t\trefname = orig_refname;\n-\n-\t/*\n-\t * If the ref did not exist and we are creating it, make sure\n-\t * there is no existing packed ref whose name begins with our\n-\t * refname, nor a packed ref whose name is a proper prefix of\n-\t * our refname.\n-\t */\n-\tif (is_null_oid(&lock->old_oid) &&\n-\t verify_refname_available_dir(refname, extras, skip,\n-\t\t\t\t\t get_packed_refs(&ref_cache), err)) {\n-\t\tlast_errno = ENOTDIR;\n-\t\tgoto error_return;\n \t}\n-\n \tlock->lk = xcalloc(1, sizeof(struct lock_file));\n \n \tlock->ref_name = xstrdup(refname);\n-\tlock->orig_ref_name = xstrdup(orig_refname);\n \tstrbuf_git_path(&ref_file, \"%s\", refname);\n \n retry:\n@@ -1885,7 +1884,13 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,\n \t\t\tgoto error_return;\n \t\t}\n \t}\n-\tif (verify_lock(lock, old_sha1, mustexist, err)) {\n+\n+\tif (type_p && *type_p & REF_ISSYMREF && !(*type_p & REF_ISBROKEN)) {\n+\t\t/*\n+\t\t * Old hash verification for symrefs happens on their\n+\t\t * base ref.\n+\t\t */\n+\t} else if (verify_lock(lock, old_sha1, mustexist, err)) {\n \t\tlast_errno = errno;\n \t\tgoto error_return;\n \t}\n@@ -2419,8 +2424,8 @@ static int files_rename_ref(const char *oldrefname, const char *newrefname,\n \n \tlogmoved = log;\n \n-\tlock = lock_ref_sha1_basic(newrefname, NULL, NULL, NULL, REF_NODEREF,\n-\t\t\t\t NULL, &err);\n+\tlock = lock_ref_sha1_basic(newrefname, NULL, NULL, NULL, NULL,\n+\t\t\t\t REF_NODEREF, NULL, &err);\n \tif (!lock) {\n \t\terror(\"unable to rename '%s' to '%s': %s\", oldrefname, newrefname, err.buf);\n \t\tstrbuf_release(&err);\n@@ -2438,8 +2443,9 @@ static int files_rename_ref(const char *oldrefname, const char *newrefname,\n \treturn 0;\n \n rollback:\n-\tlock = lock_ref_sha1_basic(oldrefname, NULL, NULL, NULL, REF_NODEREF,\n-\t\t\t\t NULL, &err);\n+\tlock = lock_ref_sha1_basic(oldrefname, NULL, NULL, NULL, NULL,\n+\t\t\t\t REF_NODEREF, NULL, &err);\n+\n \tif (!lock) {\n \t\terror(\"unable to lock %s for rollback: %s\", oldrefname, err.buf);\n \t\tstrbuf_release(&err);\n@@ -2667,9 +2673,7 @@ static int commit_ref_update(struct ref_lock *lock,\n \t\t\t int flags, struct strbuf *err)\n {\n \tclear_loose_ref_cache(&ref_cache);\n-\tif (log_ref_write(lock->ref_name, lock->old_oid.hash, sha1, logmsg, flags, err) < 0 ||\n-\t (strcmp(lock->ref_name, lock->orig_ref_name) &&\n-\t log_ref_write(lock->orig_ref_name, lock->old_oid.hash, sha1, logmsg, flags, err) < 0)) {\n+\tif (log_ref_write(lock->ref_name, lock->old_oid.hash, sha1, logmsg, flags, err) < 0) {\n \t\tchar *old_msg = strbuf_detach(err, NULL);\n \t\tstrbuf_addf(err, \"Cannot update the ref '%s': %s\",\n \t\t\t lock->ref_name, old_msg);\n@@ -2677,7 +2681,7 @@ static int commit_ref_update(struct ref_lock *lock,\n \t\tunlock_ref(lock);\n \t\treturn -1;\n \t}\n-\tif (strcmp(lock->orig_ref_name, \"HEAD\") != 0) {\n+\tif (flags & REF_IS_NOT_HEAD) {\n \t\t/*\n \t\t * Special hack: If a branch is updated directly and HEAD\n \t\t * points to it (may happen on the remote side of a push\n@@ -2772,8 +2776,8 @@ static int files_create_symref(const char *refname,\n \tstruct ref_lock *lock;\n \tint ret;\n \n-\tlock = lock_ref_sha1_basic(refname, NULL, NULL, NULL, REF_NODEREF, NULL,\n-\t\t\t\t &err);\n+\tlock = lock_ref_sha1_basic(refname, NULL, NULL, NULL, NULL, REF_NODEREF,\n+\t\t\t\t NULL, &err);\n \tif (!lock) {\n \t\terror(\"%s\", err.buf);\n \t\tstrbuf_release(&err);\n@@ -3041,6 +3045,7 @@ static int files_transaction_commit(struct ref_transaction *transaction,\n \t\t\t\tupdate->refname,\n \t\t\t\t((update->flags & REF_HAVE_OLD) ?\n \t\t\t\t update->old_sha1 : NULL),\n+\t\t\t\tupdate->read_sha1,\n \t\t\t\taffected_refnames, NULL,\n \t\t\t\tupdate->flags,\n \t\t\t\t&update->type,\n@@ -3287,7 +3292,7 @@ static int files_reflog_expire(const char *refname, const unsigned char *sha1,\n \tstruct ref_lock *lock;\n \tchar *log_file;\n \tint status = 0;\n-\tint type;\n+\tint type = 0;\n \tstruct strbuf err = STRBUF_INIT;\n \n \tmemset(&cb, 0, sizeof(cb));\n@@ -3300,7 +3305,7 @@ static int files_reflog_expire(const char *refname, const unsigned char *sha1,\n \t * reference itself, plus we might need to update the\n \t * reference if --updateref was specified:\n \t */\n-\tlock = lock_ref_sha1_basic(refname, sha1, NULL, NULL, REF_NODEREF,\n+\tlock = lock_ref_sha1_basic(refname, sha1, NULL, NULL, NULL, REF_NODEREF,\n \t\t\t\t &type, &err);\n \tif (!lock) {\n \t\terror(\"cannot lock ref '%s': %s\", refname, err.buf);\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex dd76246..0d3f9e7 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -49,6 +49,8 @@\n */\n #define REF_LOG_ONLY 0x80\n \n+#define REF_IS_NOT_HEAD 0x100\n+\n /*\n * Return true iff refname is minimally safe. \"Safe\" here means that\n * deleting a loose reference by this name will not do any damage, for\n@@ -147,6 +149,12 @@ struct ref_update {\n \t */\n \tunsigned char old_sha1[20];\n \t/*\n+\t * During the symbolic ref split stage, we resolve refs.\n+\t * We'll re-resolve non-symbolic refs once they are locked,\n+\t * but we store this to avoid re-resolving symbolic refs.\n+\t */\n+\tunsigned char read_sha1[20];\n+\t/*\n \t * One or more of REF_HAVE_NEW, REF_HAVE_OLD, REF_NODEREF,\n \t * REF_DELETING, and REF_ISPRUNING:\n \t */\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279901","messageId":"1456793586-22082-24-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 23/33] refs: always handle non-normal refs in files backend","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:56Z","receivedAt":"2016-03-01T00:52:56Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"Always handle non-normal (per-worktree or pseudo) refs in the files\nbackend instead of alternate backends.\n\nSometimes a ref transaction will update both a per-worktree ref and a\nnormal ref. For instance, an ordinary commit might update\nrefs/heads/master and HEAD (or at least HEAD's reflog).\n\nUpdates to normal refs continue to go through the chosen backend.\n\nUpdates to non-normal refs are moved to a separate files backend\ntransaction.\n\nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n refs.c | 110 +++++++++++++++++++++++++++++++++++++++++++++++++++++------------\n 1 file changed, 90 insertions(+), 20 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex eee8c37..6c05881 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -9,6 +9,11 @@\n #include \"object.h\"\n #include \"tag.h\"\n \n+static const char split_transaction_fail_warning[] = N_(\n+\t\"A ref transaction was split across two refs backends. Part of the \"\n+\t\"transaction succeeded, but then the update to the per-worktree refs \"\n+\t\"failed. Your repository may be in an inconsistent state.\");\n+\n /*\n * We always have a files backend and it is the default.\n */\n@@ -796,13 +801,19 @@ void ref_transaction_free(struct ref_transaction *transaction)\n \tfree(transaction);\n }\n \n+static void add_update_obj(struct ref_transaction *transaction,\n+\t\t\t struct ref_update *update)\n+{\n+\tALLOC_GROW(transaction->updates, transaction->nr + 1, transaction->alloc);\n+\ttransaction->updates[transaction->nr++] = update;\n+}\n+\n static struct ref_update *add_update(struct ref_transaction *transaction,\n \t\t\t\t const char *refname)\n {\n \tstruct ref_update *update;\n \tFLEX_ALLOC_STR(update, refname, refname);\n-\tALLOC_GROW(transaction->updates, transaction->nr + 1, transaction->alloc);\n-\ttransaction->updates[transaction->nr++] = update;\n+\tadd_update_obj(transaction, update);\n \treturn update;\n }\n \n@@ -1419,11 +1430,37 @@ static int dereference_symrefs(struct ref_transaction *transaction,\n \treturn 0;\n }\n \n-int ref_transaction_commit(struct ref_transaction *transaction,\n-\t\t\t struct strbuf *err)\n+/*\n+ * Move all non-normal ref updates into a specially-created\n+ * files-backend transaction\n+ */\n+static int move_abnormal_ref_updates(struct ref_transaction *transaction,\n+\t\t\t\t struct ref_transaction *files_transaction,\n+\t\t\t\t struct strbuf *err)\n+{\n+\tint i, normal_nr = 0;\n+\n+\tfor (i = 0; i < transaction->nr; i++) {\n+\t\tstruct ref_update *update = transaction->updates[i];\n+\n+\t\tif (ref_type(update->refname) == REF_TYPE_NORMAL)\n+\t\t\ttransaction->updates[normal_nr++] = update;\n+\t\telse\n+\t\t\tadd_update_obj(files_transaction, update);\n+\t}\n+\n+\ttransaction->nr = normal_nr;\n+\treturn 0;\n+}\n+\n+static int do_ref_transaction_commit(struct ref_transaction *transaction,\n+\t\t\t\t struct strbuf *err,\n+\t\t\t\t ref_transaction_commit_fn *commit_fn)\n {\n \tint ret = -1;\n \tstruct string_list affected_refnames = STRING_LIST_INIT_NODUP;\n+\tstruct string_list files_affected_refnames = STRING_LIST_INIT_NODUP;\n+\tstruct ref_transaction *files_transaction = NULL;\n \n \tassert(err);\n \n@@ -1439,18 +1476,60 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n \tif (ret)\n \t\tgoto done;\n \n+\tif (the_refs_backend != &refs_be_files) {\n+\t\tfiles_transaction = ref_transaction_begin(err);\n+\t\tif (!files_transaction)\n+\t\t\tgoto done;\n+\n+\t\tret = move_abnormal_ref_updates(transaction, files_transaction,\n+\t\t\t\t\t\terr);\n+\t\tif (ret)\n+\t\t\tgoto done;\n+\n+\t\t/* files backend commit */\n+\t\tif (get_affected_refnames(files_transaction,\n+\t\t\t\t\t\t &files_affected_refnames,\n+\t\t\t\t\t\t err)) {\n+\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\tgoto done;\n+\t\t}\n+\t}\n+\n+\t/* main backend commit */\n \tif (get_affected_refnames(transaction, &affected_refnames, err)) {\n \t\tret = TRANSACTION_GENERIC_ERROR;\n \t\tgoto done;\n \t}\n \n-\tret = the_refs_backend->transaction_commit(transaction,\n-\t\t\t\t\t\t &affected_refnames, err);\n+\tret = commit_fn(transaction, &affected_refnames, err);\n+\tif (ret)\n+\t\tgoto done;\n+\n+\tif (files_transaction) {\n+\t\tret = refs_be_files.transaction_commit(files_transaction,\n+\t\t\t\t\t\t &files_affected_refnames,\n+\t\t\t\t\t\t err);\n+\t\tif (ret) {\n+\t\t\twarning(split_transaction_fail_warning);\n+\t\t\tgoto done;\n+\t\t}\n+\t}\n+\n done:\n \tstring_list_clear(&affected_refnames, 0);\n+\tstring_list_clear(&files_affected_refnames, 0);\n+\tif (files_transaction)\n+\t\tref_transaction_free(files_transaction);\n \treturn ret;\n }\n \n+int ref_transaction_commit(struct ref_transaction *transaction,\n+\t\t\t struct strbuf *err)\n+{\n+\treturn do_ref_transaction_commit(transaction, err,\n+\t\t\t\t\t the_refs_backend->transaction_commit);\n+}\n+\n int verify_refname_available(const char *refname, struct string_list *extra,\n \t\t\t struct string_list *skip, struct strbuf *err)\n {\n@@ -1470,6 +1549,9 @@ int peel_ref(const char *refname, unsigned char *sha1)\n int create_symref(const char *ref_target, const char *refs_heads_master,\n \t\t const char *logmsg)\n {\n+\tif (ref_type(ref_target) != REF_TYPE_NORMAL)\n+\t\treturn refs_be_files.create_symref(ref_target, refs_heads_master,\n+\t\t\t\t\t\t logmsg);\n \treturn the_refs_backend->create_symref(ref_target, refs_heads_master,\n \t\t\t\t\t logmsg);\n }\n@@ -1529,20 +1611,8 @@ int reflog_expire(const char *refname, const unsigned char *sha1,\n int initial_ref_transaction_commit(struct ref_transaction *transaction,\n \t\t\t\t struct strbuf *err)\n {\n-\tstruct string_list affected_refnames = STRING_LIST_INIT_NODUP;\n-\tint ret;\n-\n-\tif (get_affected_refnames(transaction,\n-\t\t\t\t &affected_refnames, err)) {\n-\t\tret = TRANSACTION_GENERIC_ERROR;\n-\t\tgoto done;\n-\t}\n-\tret = the_refs_backend->initial_transaction_commit(transaction,\n-\t\t\t\t\t\t\t &affected_refnames,\n-\t\t\t\t\t\t\t err);\n-done:\n-\tstring_list_clear(&affected_refnames, 0);\n-\treturn ret;\n+\treturn do_ref_transaction_commit(transaction, err,\n+\t\t\t\t\t the_refs_backend->initial_transaction_commit);\n }\n \n int delete_refs(struct string_list *refnames)\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279900","messageId":"1456793586-22082-25-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 24/33] init: allow alternate ref strorage to be set for new repos","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:57Z","receivedAt":"2016-03-01T00:52:57Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"git init learns a new argument --ref-storage. Presently, only\n\"files\" is supported, but later we will add other storage backends.\n\nWhen this argument is used, the repository's extensions.refStorage\nconfiguration value is set (as well as core.repositoryformatversion),\nand the ref storage backend's initdb function is used to set up the\nref database.\n\nSigned-off-by: David Turner \nSigned-off-by: SZEDER Gábor \nSigned-off-by: Junio C Hamano \n---\n Documentation/git-init-db.txt | 2 +-\n Documentation/git-init.txt | 7 +++++-\n builtin/init-db.c | 44 +++++++++++++++++++++++++++-------\n cache.h | 2 ++\n contrib/completion/git-completion.bash | 3 ++-\n path.c | 29 ++++++++++++++++++++--\n refs.c | 2 ++\n refs.h | 5 ++++\n setup.c | 12 ++++++++++\n t/t0001-init.sh | 25 +++++++++++++++++++\n 10 files changed, 118 insertions(+), 13 deletions(-)\n\ndiff --git a/Documentation/git-init-db.txt b/Documentation/git-init-db.txt\nindex 648a6cd..d03fb69 100644\n--- a/Documentation/git-init-db.txt\n+++ b/Documentation/git-init-db.txt\n@@ -9,7 +9,7 @@ git-init-db - Creates an empty Git repository\n SYNOPSIS\n --------\n [verse]\n-'git init-db' [-q | --quiet] [--bare] [--template=] [--separate-git-dir ] [--shared[=]]\n+'git init-db' [-q | --quiet] [--bare] [--template=] [--separate-git-dir ] [--shared[=]] [--ref-storage=]\n \n \n DESCRIPTION\ndiff --git a/Documentation/git-init.txt b/Documentation/git-init.txt\nindex 8174d27..93f8d0c 100644\n--- a/Documentation/git-init.txt\n+++ b/Documentation/git-init.txt\n@@ -12,7 +12,7 @@ SYNOPSIS\n 'git init' [-q | --quiet] [--bare] [--template=]\n \t [--separate-git-dir ]\n \t [--shared[=]] [directory]\n-\n+\t [--ref-storage=]\n \n DESCRIPTION\n -----------\n@@ -113,6 +113,11 @@ does not exist, it will be created.\n \n --\n \n+--ref-storage=::\n+Type of refs storage backend. Default is to use the original \"files\"\n+storage, which stores ref data in files in `$GIT_DIR/refs` and\n+`$GIT_DIR/packed-refs`.\n+\n TEMPLATE DIRECTORY\n ------------------\n \ndiff --git a/builtin/init-db.c b/builtin/init-db.c\nindex e6d4e86..af7fe04 100644\n--- a/builtin/init-db.c\n+++ b/builtin/init-db.c\n@@ -23,6 +23,7 @@ static int init_is_bare_repository = 0;\n static int init_shared_repository = -1;\n static const char *init_db_template_dir;\n static const char *git_link;\n+static char *requested_ref_storage_backend;\n \n static void copy_templates_1(struct strbuf *path, struct strbuf *template,\n \t\t\t DIR *dir)\n@@ -178,6 +179,7 @@ static int create_default_files(const char *template_path)\n \tint reinit;\n \tint filemode;\n \tstruct strbuf err = STRBUF_INIT;\n+\tint repo_version = 0;\n \n \t/* Just look for `init.templatedir` */\n \tgit_config(git_init_db_config, NULL);\n@@ -204,6 +206,32 @@ static int create_default_files(const char *template_path)\n \t}\n \n \t/*\n+\t * Create the default symref from \".git/HEAD\" to the \"master\"\n+\t * branch, if it does not exist yet.\n+\t */\n+\tpath = git_path_buf(&buf, \"HEAD\");\n+\treinit = (!access(path, R_OK)\n+\t\t || readlink(path, junk, sizeof(junk)-1) != -1);\n+\tif (reinit) {\n+\t\tif (requested_ref_storage_backend &&\n+\t\t strcmp(ref_storage_backend, requested_ref_storage_backend))\n+\t\t\tdie(_(\"You can't change the refs storage type (was %s; you requested %s)\"),\n+\t\t\t ref_storage_backend,\n+\t\t\t requested_ref_storage_backend);\n+\t}\n+\n+\tif (requested_ref_storage_backend)\n+\t\tref_storage_backend = requested_ref_storage_backend;\n+\tif (strcmp(ref_storage_backend, \"files\")) {\n+\t\tgit_config_set(\"extensions.refStorage\", ref_storage_backend);\n+\t\tgit_config_set(\"core.repositoryformatversion\", ref_storage_backend);\n+\t\tif (set_ref_storage_backend(ref_storage_backend))\n+\t\t\tdie(_(\"Unknown ref storage backend %s\"),\n+\t\t\t ref_storage_backend);\n+\t\trepo_version = 1;\n+\t}\n+\n+\t/*\n \t * We need to create a \"refs\" dir in any case so that older\n \t * versions of git can tell that this is a repository.\n \t */\n@@ -212,13 +240,6 @@ static int create_default_files(const char *template_path)\n \tif (refs_init_db(shared_repository, &err))\n \t\tdie(\"failed to set up refs db: %s\", err.buf);\n \n-\t/*\n-\t * Create the default symlink from \".git/HEAD\" to the \"master\"\n-\t * branch, if it does not exist yet.\n-\t */\n-\tpath = git_path_buf(&buf, \"HEAD\");\n-\treinit = (!access(path, R_OK)\n-\t\t || readlink(path, junk, sizeof(junk)-1) != -1);\n \tif (!reinit) {\n \t\tif (create_symref(\"HEAD\", \"refs/heads/master\", NULL) < 0)\n \t\t\texit(1);\n@@ -226,7 +247,7 @@ static int create_default_files(const char *template_path)\n \n \t/* This forces creation of new config file */\n \txsnprintf(repo_version_string, sizeof(repo_version_string),\n-\t\t \"%d\", GIT_REPO_VERSION);\n+\t\t \"%d\", repo_version);\n \tgit_config_set(\"core.repositoryformatversion\", repo_version_string);\n \n \t/* Check filemode trustability */\n@@ -474,11 +495,18 @@ int cmd_init_db(int argc, const char **argv, const char *prefix)\n \t\tOPT_BIT('q', \"quiet\", &flags, N_(\"be quiet\"), INIT_DB_QUIET),\n \t\tOPT_STRING(0, \"separate-git-dir\", &real_git_dir, N_(\"gitdir\"),\n \t\t\t N_(\"separate git dir from working tree\")),\n+\t\tOPT_STRING(0, \"ref-storage\", &requested_ref_storage_backend,\n+\t\t\t N_(\"name\"), N_(\"name of backend type to use\")),\n \t\tOPT_END()\n \t};\n \n \targc = parse_options(argc, argv, prefix, init_db_options, init_db_usage, 0);\n \n+\tif (requested_ref_storage_backend &&\n+\t !ref_storage_backend_exists(requested_ref_storage_backend))\n+\t\tdie(_(\"Unknown ref storage backend %s\"),\n+\t\t requested_ref_storage_backend);\n+\n \tif (real_git_dir && !is_absolute_path(real_git_dir))\n \t\treal_git_dir = xstrdup(real_path(real_git_dir));\n \ndiff --git a/cache.h b/cache.h\nindex 7ceb760..3979a70 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -736,6 +736,8 @@ extern enum object_creation_mode object_creation_mode;\n \n extern char *notes_ref_name;\n \n+extern const char *ref_storage_backend;\n+\n extern int grafts_replace_parents;\n \n /*\ndiff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash\nindex e3918c8..850afd0 100644\n--- a/contrib/completion/git-completion.bash\n+++ b/contrib/completion/git-completion.bash\n@@ -1362,7 +1362,8 @@ _git_init ()\n \t\treturn\n \t\t;;\n \t--*)\n-\t\t__gitcomp \"--quiet --bare --template= --shared --shared=\"\n+\t\t__gitcomp \"--quiet --bare --template= --shared --shared=\n+\t\t\t--ref-storage=\"\n \t\treturn\n \t\t;;\n \tesac\ndiff --git a/path.c b/path.c\nindex 8b7e168..2e67a2b 100644\n--- a/path.c\n+++ b/path.c\n@@ -2,6 +2,7 @@\n * Utilities for paths and pathnames\n */\n #include \"cache.h\"\n+#include \"refs.h\"\n #include \"strbuf.h\"\n #include \"string-list.h\"\n #include \"dir.h\"\n@@ -511,8 +512,32 @@ int validate_headref(const char *path)\n \tint fd;\n \tssize_t len;\n \n-\tif (lstat(path, &st) < 0)\n-\t\treturn -1;\n+\tif (lstat(path, &st) < 0) {\n+\t\tint backend_type_set;\n+\t\tstruct strbuf config_path = STRBUF_INIT;\n+\t\tchar *pathdup = xstrdup(path);\n+\t\tchar *git_dir = dirname(pathdup);\n+\t\tchar *storage = NULL;\n+\n+\t\tstrbuf_addf(&config_path, \"%s/%s\", git_dir, \"config\");\n+\t\tfree(pathdup);\n+\n+\t\tif (git_config_from_file(ref_storage_backend_config,\n+\t\t\t\t\t config_path.buf, &storage)) {\n+\t\t\tstrbuf_release(&config_path);\n+\t\t\treturn -1;\n+\t\t}\n+\n+\t\tbackend_type_set = !!storage;\n+\t\tfree((void *)storage);\n+\t\tstrbuf_release(&config_path);\n+\t\t/*\n+\t\t * Alternate backends are assumed to keep HEAD\n+\t\t * in a valid state, so there's no need to do\n+\t\t * further validation.\n+\t\t */\n+\t\treturn backend_type_set ? 0 : -1;\n+\t}\n \n \t/* Make sure it is a \"refs/..\" symlink */\n \tif (S_ISLNK(st.st_mode)) {\ndiff --git a/refs.c b/refs.c\nindex 6c05881..a18cb4d 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -23,6 +23,8 @@ static struct ref_storage_be *the_refs_backend = &refs_be_files;\n */\n static struct ref_storage_be *refs_backends = &refs_be_files;\n \n+const char *ref_storage_backend = \"files\";\n+\n static struct ref_storage_be *find_ref_storage_backend(const char *name)\n {\n \tstruct ref_storage_be *be;\ndiff --git a/refs.h b/refs.h\nindex 13ce2a0..09486b2 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -512,6 +512,11 @@ extern int reflog_expire(const char *refname, const unsigned char *sha1,\n \t\t\t void *policy_cb_data);\n \n /*\n+ * Read the refdb storage backend name out of the config file\n+ */\n+int ref_storage_backend_config(const char *var, const char *value, void *ptr);\n+\n+/*\n * Switch to an alternate ref storage backend.\n */\n int set_ref_storage_backend(const char *name);\ndiff --git a/setup.c b/setup.c\nindex de1a2a7..bd3a2cf 100644\n--- a/setup.c\n+++ b/setup.c\n@@ -1,5 +1,6 @@\n #include \"cache.h\"\n #include \"dir.h\"\n+#include \"refs.h\"\n #include \"string-list.h\"\n \n static int inside_git_dir = -1;\n@@ -261,6 +262,15 @@ int get_common_dir_noenv(struct strbuf *sb, const char *gitdir)\n \treturn ret;\n }\n \n+int ref_storage_backend_config(const char *var, const char *value, void *ptr)\n+{\n+\tconst char **cdata = ptr;\n+\n+\tif (!strcmp(var, \"extensions.refstorage\"))\n+\t\treturn git_config_string(cdata, var, value);\n+\treturn 0;\n+}\n+\n /*\n * Test if it looks like we're at a git directory.\n * We want to see:\n@@ -388,6 +398,8 @@ static int check_repo_format(const char *var, const char *value, void *cb)\n \t\t\t;\n \t\telse if (!strcmp(ext, \"preciousobjects\"))\n \t\t\trepository_format_precious_objects = git_config_bool(var, value);\n+\t\telse if (!strcmp(ext, \"refstorage\"))\n+\t\t\tref_storage_backend = xstrdup(value);\n \t\telse\n \t\t\tstring_list_append(&unknown_extensions, ext);\n \t}\ndiff --git a/t/t0001-init.sh b/t/t0001-init.sh\nindex 295aa59..31c8279 100755\n--- a/t/t0001-init.sh\n+++ b/t/t0001-init.sh\n@@ -174,6 +174,31 @@ test_expect_success 'reinit' '\n \ttest_i18ncmp again/empty again/err2\n '\n \n+test_expect_success 'init with bogus storage backend fails' '\n+\n+\t(\n+\t\tmkdir again2 &&\n+\t\tcd again2 &&\n+\t\ttest_must_fail git init --ref-storage=test >out2 2>err2 &&\n+\t\ttest_i18ngrep \"Unknown ref storage backend test\" err2\n+\t)\n+'\n+\n+test_expect_failure 'reinit with changed storage backend fails' '\n+\n+\t(\n+\t\tmkdir again3 &&\n+\t\tcd again3 &&\n+\t\tgit init >out1 2>err1 &&\n+\t\tgit init --ref-storage=test >out2 2>err2\n+\t) &&\n+\ttest_i18ngrep \"Initialized empty\" again3/out1 &&\n+\ttest_i18ngrep \"Reinitialized existing\" again3/out2 &&\n+\t>again3/empty &&\n+\ttest_i18ncmp again3/empty again3/err1 &&\n+\ttest_i18ncmp again3/empty again3/err2\n+'\n+\n test_expect_success 'init with --template' '\n \tmkdir template-source &&\n \techo content >template-source/file &&\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279899","messageId":"1456793586-22082-26-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 25/33] refs: check submodules' ref storage config","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:58Z","receivedAt":"2016-03-01T00:52:58Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"All submodules must have the same ref storage (for now). Confirm that\nthis is so before attempting to do anything with submodule refs.\n\nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n refs.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++\n refs.h | 2 +-\n refs/files-backend.c | 8 ++------\n 3 files changed, 57 insertions(+), 7 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex a18cb4d..5d69179 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -301,6 +301,54 @@ int for_each_tag_ref(each_ref_fn fn, void *cb_data)\n \treturn for_each_ref_in(\"refs/tags/\", fn, cb_data);\n }\n \n+static int submodule_backend(const char *key, const char *value, void *data)\n+{\n+\tconst char **path = data;\n+\tchar **old_path = data;\n+\tif (!strcmp(key, \"extensions.refstorage\") &&\n+\t !git_config_string(path, key, \"extensions.refstorage\"))\n+\t\t\tfree(*old_path);\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Check that a submodule exists. If its ref storage backend differs\n+ * from the current backend, die. If the submodule exists, return\n+ * 0. Else return -1.\n+ */\n+static int check_submodule_backend(const char *submodule)\n+{\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tchar *submodule_storage_backend;\n+\tint result = -1;\n+\n+\tif (!submodule)\n+\t\treturn 0;\n+\n+\tsubmodule_storage_backend = xstrdup(\"files\");\n+\n+\tstrbuf_addstr(&sb, submodule);\n+\tif (!is_nonbare_repository_dir(&sb))\n+\t\tgoto done;\n+\n+\tstrbuf_reset(&sb);\n+\tstrbuf_git_path_submodule(&sb, submodule, \"config\");\n+\n+\tgit_config_from_file(submodule_backend, sb.buf,\n+\t\t\t &submodule_storage_backend);\n+\tif (strcmp(submodule_storage_backend, ref_storage_backend))\n+\t\tdie(_(\"Ref storage '%s' for submodule '%s' does not match our storage, '%s'\"),\n+\t\t submodule_storage_backend, submodule, ref_storage_backend);\n+\n+\tresult = 0;\n+done:\n+\tfree(submodule_storage_backend);\n+\tstrbuf_release(&sb);\n+\n+\treturn result;\n+}\n+\n int for_each_tag_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)\n {\n \treturn for_each_ref_in_submodule(submodule, \"refs/tags/\", fn, cb_data);\n@@ -313,6 +361,7 @@ int for_each_branch_ref(each_ref_fn fn, void *cb_data)\n \n int for_each_branch_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)\n {\n+\tcheck_submodule_backend(submodule);\n \treturn for_each_ref_in_submodule(submodule, \"refs/heads/\", fn, cb_data);\n }\n \n@@ -323,6 +372,7 @@ int for_each_remote_ref(each_ref_fn fn, void *cb_data)\n \n int for_each_remote_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)\n {\n+\tcheck_submodule_backend(submodule);\n \treturn for_each_ref_in_submodule(submodule, \"refs/remotes/\", fn, cb_data);\n }\n \n@@ -1138,6 +1188,7 @@ int head_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)\n \tint flag;\n \n \tif (submodule) {\n+\t\tcheck_submodule_backend(submodule);\n \t\tif (resolve_gitlink_ref(submodule, \"HEAD\", oid.hash) == 0)\n \t\t\treturn fn(\"HEAD\", &oid, 0, cb_data);\n \n@@ -1561,6 +1612,9 @@ int create_symref(const char *ref_target, const char *refs_heads_master,\n int resolve_gitlink_ref(const char *path, const char *refname,\n \t\t\tunsigned char *sha1)\n {\n+\tif (check_submodule_backend(path))\n+\t\treturn -1;\n+\n \treturn the_refs_backend->resolve_gitlink_ref(path, refname, sha1);\n }\n \ndiff --git a/refs.h b/refs.h\nindex 09486b2..1888ad8 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -512,7 +512,7 @@ extern int reflog_expire(const char *refname, const unsigned char *sha1,\n \t\t\t void *policy_cb_data);\n \n /*\n- * Read the refdb storage backend name out of the config file\n+ * Read the ref storage backend name out of the config file\n */\n int ref_storage_backend_config(const char *var, const char *value, void *ptr);\n \ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 7ccaf88..a13d419 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1347,13 +1347,9 @@ static int files_resolve_gitlink_ref(const char *path, const char *refname,\n \n \tstrbuf_add(&submodule, path, len);\n \trefs = lookup_ref_cache(submodule.buf);\n-\tif (!refs) {\n-\t\tif (!is_nonbare_repository_dir(&submodule)) {\n-\t\t\tstrbuf_release(&submodule);\n-\t\t\treturn -1;\n-\t\t}\n+\tif (!refs)\n \t\trefs = create_ref_cache(submodule.buf);\n-\t}\n+\n \tstrbuf_release(&submodule);\n \n \tretval = resolve_gitlink_ref_recursive(refs, refname, sha1, 0);\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279906","messageId":"1456793586-22082-27-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 26/33] clone: allow ref storage backend to be set for clone","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:52:59Z","receivedAt":"2016-03-01T00:52:59Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"Add a new option, --ref-storage, to allow the ref storage backend to\nbe set on new clones.\n\nSubmodules must use the same ref storage as the parent repository, so\nwe also pass the --ref-storage option option when cloning\nsubmodules.\n\nSigned-off-by: David Turner \nSigned-off-by: SZEDER Gábor \nSigned-off-by: Junio C Hamano \n---\n Documentation/git-clone.txt | 5 +++++\n builtin/clone.c | 5 +++++\n builtin/submodule--helper.c | 2 +-\n contrib/completion/git-completion.bash | 1 +\n git-submodule.sh | 13 +++++++++++++\n 5 files changed, 25 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt\nindex 9a95c6e..a5a48cb 100644\n--- a/Documentation/git-clone.txt\n+++ b/Documentation/git-clone.txt\n@@ -14,6 +14,7 @@ SYNOPSIS\n \t [-o ] [-b ] [-u ] [--reference ]\n \t [--dissociate] [--separate-git-dir ]\n \t [--depth ] [--[no-]single-branch]\n+\t [--ref-storage=]\n \t [--recursive | --recurse-submodules] [--jobs ] [--] \n \t []\n \n@@ -232,6 +233,10 @@ objects from the source repository into a pack in the cloned repository.\n \tThe number of submodules fetched at the same time.\n \tDefaults to the `submodule.fetchJobs` option.\n \n+--ref-storage=::\n+\tType of ref storage backend. Default is to use the original files\n+\tbased ref storage.\n+\n ::\n \tThe (possibly remote) repository to clone from. See the\n \t<> section below for more information on specifying\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex d1c9bcf..ed524e9 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -114,6 +114,8 @@ static struct option builtin_clone_options[] = {\n \t\t\tTRANSPORT_FAMILY_IPV4),\n \tOPT_SET_INT('6', \"ipv6\", &family, N_(\"use IPv6 addresses only\"),\n \t\t\tTRANSPORT_FAMILY_IPV6),\n+\tOPT_STRING(0, \"ref-storage\", &ref_storage_backend,\n+\t\t N_(\"name\"), N_(\"name of backend type to use\")),\n \tOPT_END()\n };\n \n@@ -752,6 +754,9 @@ static int checkout(void)\n \t\tif (max_jobs != -1)\n \t\t\targv_array_pushf(&args, \"--jobs=%d\", max_jobs);\n \n+\t\targv_array_pushl(&args, \"--ref-storage\",\n+\t\t\t\t ref_storage_backend, NULL);\n+\n \t\terr = run_command_v_opt(args.argv, RUN_GIT_CMD);\n \t\targv_array_clear(&args);\n \t}\ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex ed4f60b..f8cdce9 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -444,7 +444,7 @@ static int clone_submodule(const char *path, const char *gitdir, const char *url\n \t\targv_array_pushl(&cp.args, \"--reference\", reference, NULL);\n \tif (gitdir && *gitdir)\n \t\targv_array_pushl(&cp.args, \"--separate-git-dir\", gitdir, NULL);\n-\n+\targv_array_pushl(&cp.args, \"--ref-storage\", ref_storage_backend, NULL);\n \targv_array_push(&cp.args, url);\n \targv_array_push(&cp.args, path);\n \ndiff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash\nindex 850afd0..a0225f4 100644\n--- a/contrib/completion/git-completion.bash\n+++ b/contrib/completion/git-completion.bash\n@@ -1092,6 +1092,7 @@ _git_clone ()\n \t\t\t--depth\n \t\t\t--single-branch\n \t\t\t--branch\n+\t\t\t--ref-storage=\n \t\t\t\"\n \t\treturn\n \t\t;;\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 3924bff..5562a19 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -545,6 +545,14 @@ cmd_update()\n \t\t--checkout)\n \t\t\tupdate=\"checkout\"\n \t\t\t;;\n+\t\t--ref-storage=*)\n+\t\t\tref_storage=\"$1\"\n+\t\t\t;;\n+\t\t--ref-storage)\n+\t\t\tcase \"$2\" in '') usage ;; esac\n+\t\t\tref_storage=\"$2\"\n+\t\t\tshift\n+\t\t\t;;\n \t\t--depth)\n \t\t\tcase \"$2\" in '') usage ;; esac\n \t\t\tdepth=\"--depth=$2\"\n@@ -578,6 +586,11 @@ cmd_update()\n \tif test -n \"$init\"\n \tthen\n \t\tcmd_init \"--\" \"$@\" || return\n+\telse\n+\t\tif test -n \"$ref_storage\"\n+\t\tthen\n+\t\t\tusage\n+\t\tfi\n \tfi\n \n \t{\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279904","messageId":"1456793586-22082-28-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 27/33] svn: learn ref-storage argument","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:53:00Z","receivedAt":"2016-03-01T00:53:00Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"git svn learns to pass the ref-storage command-line argument (to init\nand clone) through to git init.\n\nSigned-off-by: David Turner \nSigned-off-by: SZEDER Gábor \nSigned-off-by: Eric Wong \nSigned-off-by: Junio C Hamano \n---\n contrib/completion/git-completion.bash | 2 +-\n git-svn.perl | 6 +++++-\n 2 files changed, 6 insertions(+), 2 deletions(-)\n\ndiff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash\nindex a0225f4..4b15631 100644\n--- a/contrib/completion/git-completion.bash\n+++ b/contrib/completion/git-completion.bash\n@@ -2477,7 +2477,7 @@ _git_svn ()\n \t\t\t--branches= --stdlayout --minimize-url\n \t\t\t--no-metadata --use-svm-props --use-svnsync-props\n \t\t\t--rewrite-root= --prefix= --use-log-author\n-\t\t\t--add-author-from $remote_opts\n+\t\t\t--add-author-from --ref-storage= $remote_opts\n \t\t\t\"\n \t\tlocal cmt_opts=\"\n \t\t\t--edit --rmdir --find-copies-harder --copy-similarity=\ndiff --git a/git-svn.perl b/git-svn.perl\nindex fa5f253..62464d1 100755\n--- a/git-svn.perl\n+++ b/git-svn.perl\n@@ -141,7 +141,7 @@ my %fc_opts = ( 'follow-parent|follow!' => \\$Git::SVN::_follow_parent,\n \t\t'localtime' => \\$Git::SVN::_localtime,\n \t\t%remote_opts );\n \n-my ($_trunk, @_tags, @_branches, $_stdlayout);\n+my ($_trunk, @_tags, @_branches, $_stdlayout, $_ref_storage);\n my %icv;\n my %init_opts = ( 'template=s' => \\$_template, 'shared:s' => \\$_shared,\n 'trunk|T=s' => \\$_trunk, 'tags|t=s@' => \\@_tags,\n@@ -153,6 +153,7 @@ my %init_opts = ( 'template=s' => \\$_template, 'shared:s' => \\$_shared,\n \t\t 'use-svnsync-props' => sub { $icv{useSvnsyncProps} = 1 },\n \t\t 'rewrite-root=s' => sub { $icv{rewriteRoot} = $_[1] },\n \t\t 'rewrite-uuid=s' => sub { $icv{rewriteUUID} = $_[1] },\n+\t\t 'ref-storage=s' => \\$_ref_storage,\n %remote_opts );\n my %cmt_opts = ( 'edit|e' => \\$_edit,\n \t\t'rmdir' => \\$Git::SVN::Editor::_rmdir,\n@@ -469,6 +470,9 @@ sub do_git_init_db {\n \t\t\t\tpush @init_db, \"--shared\";\n \t\t\t}\n \t\t}\n+\t\tif (defined $_ref_storage) {\n+\t\t\tpush @init_db, \"--ref-storage=\" . $_ref_storage;\n+\t\t}\n \t\tcommand_noisy(@init_db);\n \t\t$_repository = Git->repository(Repository => \".git\");\n \t}\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279903","messageId":"1456793586-22082-29-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 28/33] refs: register ref storage backends","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:53:01Z","receivedAt":"2016-03-01T00:53:01Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"Add new function register_ref_storage_backends(). This new function\nregisters all known ref storage backends... once there are any other\nthan the default. For now, it just registers the files backend.\n\nCall the function before calling set_ref_storage_backend.\n\nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n builtin/init-db.c | 3 +++\n refs.c | 22 ++++++++++++++++++----\n refs.h | 2 ++\n 3 files changed, 23 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/init-db.c b/builtin/init-db.c\nindex af7fe04..04cc522 100644\n--- a/builtin/init-db.c\n+++ b/builtin/init-db.c\n@@ -220,6 +220,7 @@ static int create_default_files(const char *template_path)\n \t\t\t requested_ref_storage_backend);\n \t}\n \n+\tregister_ref_storage_backends();\n \tif (requested_ref_storage_backend)\n \t\tref_storage_backend = requested_ref_storage_backend;\n \tif (strcmp(ref_storage_backend, \"files\")) {\n@@ -502,6 +503,8 @@ int cmd_init_db(int argc, const char **argv, const char *prefix)\n \n \targc = parse_options(argc, argv, prefix, init_db_options, init_db_usage, 0);\n \n+\tregister_ref_storage_backends();\n+\n \tif (requested_ref_storage_backend &&\n \t !ref_storage_backend_exists(requested_ref_storage_backend))\n \t\tdie(_(\"Unknown ref storage backend %s\"),\ndiff --git a/refs.c b/refs.c\nindex 5d69179..5fe0bac 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -14,14 +14,11 @@ static const char split_transaction_fail_warning[] = N_(\n \t\"transaction succeeded, but then the update to the per-worktree refs \"\n \t\"failed. Your repository may be in an inconsistent state.\");\n \n-/*\n- * We always have a files backend and it is the default.\n- */\n static struct ref_storage_be *the_refs_backend = &refs_be_files;\n /*\n * List of all available backends\n */\n-static struct ref_storage_be *refs_backends = &refs_be_files;\n+static struct ref_storage_be *refs_backends;\n \n const char *ref_storage_backend = \"files\";\n \n@@ -48,6 +45,23 @@ int ref_storage_backend_exists(const char *name)\n \treturn find_ref_storage_backend(name) != NULL;\n }\n \n+static void register_ref_storage_backend(struct ref_storage_be *be)\n+{\n+\tbe->next = refs_backends;\n+\trefs_backends = be;\n+}\n+\n+void register_ref_storage_backends(void)\n+{\n+\tif (refs_backends)\n+\t\treturn;\n+\t/*\n+\t * Add register_ref_storage_backend(ptr-to-backend)\n+\t * entries below when you add a new backend.\n+\t */\n+\tregister_ref_storage_backend(&refs_be_files);\n+}\n+\n /*\n * How to handle various characters in refnames:\n * 0: An acceptable character for refs\ndiff --git a/refs.h b/refs.h\nindex 1888ad8..c0964f5 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -523,4 +523,6 @@ int set_ref_storage_backend(const char *name);\n \n int ref_storage_backend_exists(const char *name);\n \n+void register_ref_storage_backends(void);\n+\n #endif /* REFS_H */\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279905","messageId":"1456793586-22082-30-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 29/33] setup: configure ref storage on setup","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:53:02Z","receivedAt":"2016-03-01T00:53:02Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"This sets up the existing backend early, so that other code which\nreads refs is reading from the right place.\n\nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n config.c | 1 +\n setup.c | 4 ++++\n 2 files changed, 5 insertions(+)\n\ndiff --git a/config.c b/config.c\nindex 9ba40bc..cca7e28 100644\n--- a/config.c\n+++ b/config.c\n@@ -11,6 +11,7 @@\n #include \"strbuf.h\"\n #include \"quote.h\"\n #include \"hashmap.h\"\n+#include \"refs.h\"\n #include \"string-list.h\"\n #include \"utf8.h\"\n \ndiff --git a/setup.c b/setup.c\nindex bd3a2cf..e2e1220 100644\n--- a/setup.c\n+++ b/setup.c\n@@ -457,6 +457,10 @@ static int check_repository_format_gently(const char *gitdir, int *nongit_ok)\n \t\tret = -1;\n \t}\n \n+\tregister_ref_storage_backends();\n+\tif (set_ref_storage_backend(ref_storage_backend))\n+\t\tdie(_(\"Unknown ref storage backend %s\"), ref_storage_backend);\n+\n \tstrbuf_release(&sb);\n \treturn ret;\n }\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279907","messageId":"1456793586-22082-31-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 30/33] refs: break out resolve_ref_unsafe_submodule","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:53:03Z","receivedAt":"2016-03-01T00:53:03Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"It will soon be useful for resolve_ref_unsafe to support submodules.\nBut since it is called from so many places, changing it would have\nbeen painful. Fortunately, it's just a thin wrapper around (the\nformer) resolve_ref_1. So now resolve_ref_1 becomes\nresolve_ref_unsafe_submodule, and it passes its submodule argument\nthrough to read_raw_ref.\n\nThe files backend doesn't need this functionality, but it doesn't\nhurt.\n\nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n refs.c | 41 +++++++++++++++++++++++++----------------\n refs/files-backend.c | 8 ++++++--\n refs/refs-internal.h | 19 ++++++++++++++++---\n 3 files changed, 47 insertions(+), 21 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 5fe0bac..d1cf707 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -60,6 +60,9 @@ void register_ref_storage_backends(void)\n \t * entries below when you add a new backend.\n \t */\n \tregister_ref_storage_backend(&refs_be_files);\n+#ifdef USE_LIBLMDB\n+\tregister_ref_storage_backend(&refs_be_lmdb);\n+#endif\n }\n \n /*\n@@ -320,8 +323,10 @@ static int submodule_backend(const char *key, const char *value, void *data)\n \tconst char **path = data;\n \tchar **old_path = data;\n \tif (!strcmp(key, \"extensions.refstorage\") &&\n-\t !git_config_string(path, key, \"extensions.refstorage\"))\n-\t\t\tfree(*old_path);\n+\t !git_config_string(path, key, \"extensions.refstorage\")) {\n+\t\tfree(*old_path);\n+\t\t*path = xstrdup(value);\n+\t}\n \n \treturn 0;\n }\n@@ -1313,21 +1318,22 @@ int for_each_rawref(each_ref_fn fn, void *cb_data)\n \t\t\t DO_FOR_EACH_INCLUDE_BROKEN, cb_data);\n }\n \n-static int read_raw_ref(const char *refname, unsigned char *sha1,\n-\t\t\tstruct strbuf *symref, struct strbuf *sb_path,\n-\t\t\tunsigned int *flags)\n+static int read_raw_ref(const char *submodule, const char *refname,\n+\t\t\tunsigned char *sha1, struct strbuf *symref,\n+\t\t\tstruct strbuf *sb_path, unsigned int *flags)\n {\n-\treturn the_refs_backend->read_raw_ref(refname, sha1, symref, sb_path,\n-\t\t\t\t\t flags);\n+\treturn the_refs_backend->read_raw_ref(submodule, refname, sha1,\n+\t\t\t\t\t symref, sb_path, flags);\n }\n \n /* This function needs to return a meaningful errno on failure */\n-static const char *resolve_ref_1(const char *refname,\n-\t\t\t\t int resolve_flags,\n-\t\t\t\t unsigned char *sha1,\n-\t\t\t\t int *flags,\n-\t\t\t\t struct strbuf *sb_refname,\n-\t\t\t\t struct strbuf *sb_path)\n+const char *resolve_ref_unsafe_submodule(const char *submodule,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t unsigned char *sha1,\n+\t\t\t\t\t int *flags,\n+\t\t\t\t\t struct strbuf *sb_refname,\n+\t\t\t\t\t struct strbuf *sb_path)\n {\n \tint bad_name = 0;\n \tint symref_count;\n@@ -1358,7 +1364,8 @@ static const char *resolve_ref_1(const char *refname,\n \tfor (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {\n \t\tunsigned int read_flags = 0;\n \n-\t\tif (read_raw_ref(refname, sha1, sb_refname, sb_path, &read_flags)) {\n+\t\tif (read_raw_ref(submodule, refname, sha1, sb_refname, sb_path,\n+\t\t\t\t &read_flags)) {\n \t\t\tint saved_errno = errno;\n \t\t\tif (flags)\n \t\t\t\t*flags |= read_flags;\n@@ -1419,8 +1426,8 @@ const char *resolve_ref_unsafe(const char *refname, int resolve_flags,\n \tstruct strbuf sb_path = STRBUF_INIT;\n \tconst char *ret;\n \n-\tret = resolve_ref_1(refname, resolve_flags, sha1, flags,\n-\t\t\t &sb_refname, &sb_path);\n+\tret = resolve_ref_unsafe_submodule(NULL, refname, resolve_flags, sha1,\n+\t\t\t\t\t flags, &sb_refname, &sb_path);\n \n \tstrbuf_release(&sb_path);\n \treturn ret;\n@@ -1663,6 +1670,8 @@ int safe_create_reflog(const char *refname, int force_create,\n \n int delete_reflog(const char *refname)\n {\n+\tif (ref_type(refname) != REF_TYPE_NORMAL)\n+\t\treturn refs_be_files.delete_reflog(refname);\n \treturn the_refs_backend->delete_reflog(refname);\n }\n \ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex a13d419..87c02f8 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1413,7 +1413,8 @@ static int resolve_missing_loose_ref(const char *refname,\n *\n * sb_path is workspace: the caller should allocate and free it.\n */\n-static int files_read_raw_ref(const char *refname, unsigned char *sha1,\n+static int files_read_raw_ref(const char *submodule,\n+\t\t\t const char *refname, unsigned char *sha1,\n \t\t\t struct strbuf *symref, struct strbuf *sb_path,\n \t\t\t unsigned int *flags)\n {\n@@ -1423,7 +1424,10 @@ static int files_read_raw_ref(const char *refname, unsigned char *sha1,\n \tconst char *buf;\n \n \tstrbuf_reset(sb_path);\n-\tstrbuf_git_path(sb_path, \"%s\", refname);\n+\tif (submodule)\n+\t\tstrbuf_git_path_submodule(sb_path, submodule, \"%s\", refname);\n+\telse\n+\t\tstrbuf_git_path(sb_path, \"%s\", refname);\n \tpath = sb_path->buf;\n \n \tfor (;;) {\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 0d3f9e7..9736959 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -230,6 +230,15 @@ int do_for_each_per_worktree_ref(const char *submodule, const char *base,\n \t\t\t\t each_ref_fn fn, int trim, int flags,\n \t\t\t\t void *cb_data);\n \n+\n+const char *resolve_ref_unsafe_submodule(const char *submodule,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t unsigned char *sha1,\n+\t\t\t\t\t int *flags,\n+\t\t\t\t\t struct strbuf *sb_refname,\n+\t\t\t\t\t struct strbuf *sb_path);\n+\n /* refs backends */\n typedef int ref_init_db_fn(int shared, struct strbuf *err);\n typedef int ref_transaction_commit_fn(struct ref_transaction *transaction,\n@@ -266,9 +275,9 @@ typedef int rename_ref_fn(const char *oldref, const char *newref,\n \t\t\t const char *logmsg);\n \n /* resolution methods */\n-typedef int read_raw_ref_fn(const char *refname, unsigned char *sha1,\n-\t\t\t struct strbuf *symref, struct strbuf *sb_path,\n-\t\t\t unsigned int *flags);\n+typedef int read_raw_ref_fn(const char *submodule, const char *refname,\n+\t\t\t unsigned char *sha1, struct strbuf *symref,\n+\t\t\t struct strbuf *sb_path, unsigned int *flags);\n typedef int verify_refname_available_fn(const char *refname, struct string_list *extra, struct string_list *skip, struct strbuf *err);\n typedef int resolve_gitlink_ref_fn(const char *path, const char *refname,\n \t\t\t\t unsigned char *sha1);\n@@ -305,4 +314,8 @@ struct ref_storage_be {\n };\n \n extern struct ref_storage_be refs_be_files;\n+\n+#ifdef USE_LIBLMDB\n+extern struct ref_storage_be refs_be_lmdb;\n+#endif\n #endif /* REFS_REFS_INTERNAL_H */\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279908","messageId":"1456793586-22082-32-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 31/33] refs: add LMDB refs storage backend","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:53:04Z","receivedAt":"2016-03-01T00:53:04Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"Add a database backend for refs using LMDB. This backend runs git\nfor-each-ref about 30% faster than the files backend with fully-packed\nrefs on a repo with ~120k refs. It's also about 4x faster than using\nfully-unpacked refs. In addition, and perhaps more importantly, it\navoids case-conflict issues on OS X.\n\nLMDB has a few features that make it suitable for usage in git:\n\n1. It is licensed under the OpenLDAP Public License, which is\nGPL-compatible [1].\n\n[1] http://www.gnu.org/licenses/license-list.en.html#newOpenLDAP\n\n2. It is relatively lightweight; it requires only one header file, and\nthe library code takes under 64k at runtime.\n\n3. It is well-tested: it's been used in OpenLDAP for years.\n\n4. It's very fast. LMDB's benchmarks show that it is among\nthe fastest key-value stores.\n\n5. It has a relatively simple concurrency story; readers don't\nblock writers and writers don't block readers.\n\nRonnie Sahlberg's original version of this patchset used tdb. The\nmajor disadvantage of tdb is that tdb is hard to build on OS X. It's\nalso not in homebrew. So lmdb seemed simpler.\n\nTo test this backend's correctness, I added support to the test suite\nto run under LMDB. Dozens of tests use manual ref/reflog\nreading/writing, or create submodules without passing --ref-storage to\ngit init. If those tests are changed to use the update-ref machinery\nor test-refs-lmdb-backend (or, in the case of packed-refs, corrupt\nrefs, and dumb fetch tests, are skipped), the only remaining failing\ntests are the git-new-workdir tests. Later, we'll add this support\nto the test suite.\n\nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n .gitignore | 1 +\n Documentation/config.txt | 9 +\n Documentation/git-clone.txt | 3 +-\n Documentation/git-init.txt | 3 +-\n Documentation/technical/refs-lmdb-backend.txt | 61 +\n Documentation/technical/repository-version.txt | 7 +\n Makefile | 12 +\n configure.ac | 33 +\n contrib/workdir/git-new-workdir | 3 +\n path.c | 1 +\n refs/lmdb-backend.c | 1886 ++++++++++++++++++++++++\n test-refs-lmdb-backend.c | 66 +\n 12 files changed, 2083 insertions(+), 2 deletions(-)\n create mode 100644 Documentation/technical/refs-lmdb-backend.txt\n create mode 100644 refs/lmdb-backend.c\n create mode 100644 test-refs-lmdb-backend.c\n\ndiff --git a/.gitignore b/.gitignore\nindex 5087ce1..b07bec1 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -200,6 +200,7 @@\n /test-path-utils\n /test-prio-queue\n /test-read-cache\n+/test-refs-lmdb-backend\n /test-regex\n /test-revision-walking\n /test-run-command\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 59d7046..3677b75 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -1153,6 +1153,15 @@ difftool..cmd::\n difftool.prompt::\n \tPrompt before each invocation of the diff tool.\n \n+extensions.refStorage::\n+\tType of ref storage backend. Default is to use the original\n+\tfiles based ref storage. When set to \"lmdb\", refs are stored\n+\tin an LMDB database. This setting reflects the refs storage\n+\tbackend that was chosen via the --ref-storage option when the\n+\trepository was originally created. It is currently not\n+\tpossible to change the refs storage backend of an existing\n+\trepository.\n+\n fetch.recurseSubmodules::\n \tThis option can be either set to a boolean value or to 'on-demand'.\n \tSetting it to a boolean changes the behavior of fetch and pull to\ndiff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt\nindex a5a48cb..cd6dc38 100644\n--- a/Documentation/git-clone.txt\n+++ b/Documentation/git-clone.txt\n@@ -235,7 +235,8 @@ objects from the source repository into a pack in the cloned repository.\n \n --ref-storage=::\n \tType of ref storage backend. Default is to use the original files\n-\tbased ref storage.\n+\tbased ref storage. Set to \"lmdb\" to store refs in the LMDB database\n+\tbackend.\n \n ::\n \tThe (possibly remote) repository to clone from. See the\ndiff --git a/Documentation/git-init.txt b/Documentation/git-init.txt\nindex 93f8d0c..129702b 100644\n--- a/Documentation/git-init.txt\n+++ b/Documentation/git-init.txt\n@@ -116,7 +116,8 @@ does not exist, it will be created.\n --ref-storage=::\n Type of refs storage backend. Default is to use the original \"files\"\n storage, which stores ref data in files in `$GIT_DIR/refs` and\n-`$GIT_DIR/packed-refs`.\n+`$GIT_DIR/packed-refs`. Set to \"lmdb\" to activate the LMDB storage\n+backend.\n \n TEMPLATE DIRECTORY\n ------------------\ndiff --git a/Documentation/technical/refs-lmdb-backend.txt b/Documentation/technical/refs-lmdb-backend.txt\nnew file mode 100644\nindex 0000000..b8b5a0a\n--- /dev/null\n+++ b/Documentation/technical/refs-lmdb-backend.txt\n@@ -0,0 +1,61 @@\n+Notes on the LMDB refs backend\n+==============================\n+\n+Design:\n+------\n+\n+Refs and reflogs are stored in a lmdb database in .git/refs.lmdb. All\n+keys and values are \\0-terminated.\n+\n+Keys for refs are the name of the ref (e.g. refs/heads/master).\n+Values are the value of the ref, in hex\n+(e.g. 61f23eb0f81357c19fa91e2b8c6f3906c3a8f9b0).\n+\n+All per-worktree refs (refs/bisect/* and HEAD) are stored using\n+the traditional files-based backend.\n+\n+Reflogs are stored as a series of database entries.\n+\n+For non-empty reflogs, there is one entry per logged ref update. The\n+key format is logs/[refname]\\0[timestamp]. The timestamp is a 64-bit\n+unsigned integer number of nanoseconds since 1/1/1970, stored in\n+network byte order. This means that reflog entries are\n+chronologically ordered. Because LMDB is a btree database, we can\n+efficiently iterate over these keys.\n+\n+For an empty reflog, there is a \"header\" entry to show that a reflog\n+exists. The header has the same format as an ordinary reflog entry,\n+but with a timestamp of all zeros and an empty value.\n+\n+Reflog values are in almost the same format as the original\n+files-based reflog, including the trailing LF. The exception is that\n+the SHAs are stored in binary instead of hex. The date in the reflog\n+value matches the date in the timestamp field.\n+\n+Weaknesses:\n+-----------\n+\n+The reflog format is somewhat inefficient: a binary format could store\n+reflog date/time information in somewhat less space.\n+\n+The rsync and file:// transports don't work yet, because they\n+don't use the refs API.\n+\n+git new-workdir is incompatible with the lmdb backend. Fortunately,\n+git new-workdir is deprecated, and worktrees work fine.\n+\n+LMDB locks the entire database for write. Any other writer waits\n+until the first writer is done before beginning. Readers do not wait\n+for writers, and writers do not wait for readers. The underlying\n+scheme is approximately MVCC; each reader's queries see the state of\n+the database as-of the time that the reader acquired its read lock.\n+This is not too far off from the files backend, which loads all refs\n+into memory when one is requested.\n+\n+LMDB requires write access to the filesystem for locking. In the case\n+where there are only readers (e.g. read-only repositories), this\n+requirement can be relaxed using the MDB_NOLOCK option. But we don't\n+yet have a config to do that.\n+\n+Don't use this over NFS or other remote filesystems. LMDB doesn't work\n+there.\ndiff --git a/Documentation/technical/repository-version.txt b/Documentation/technical/repository-version.txt\nindex 00ad379..b317ba9 100644\n--- a/Documentation/technical/repository-version.txt\n+++ b/Documentation/technical/repository-version.txt\n@@ -86,3 +86,10 @@ for testing format-1 compatibility.\n When the config key `extensions.preciousObjects` is set to `true`,\n objects in the repository MUST NOT be deleted (e.g., by `git-prune` or\n `git repack -d`).\n+\n+`refStorage`\n+~~~~~~~~~~~~\n+This extension allows the use of alternate ref storage backends. The\n+only defined values are `lmdb`, `files`, and the empty string (meaning\n+`files`). The latter two values should never be used, because the\n+files backend is the default and does not need repository version 1.\ndiff --git a/Makefile b/Makefile\nindex 10566d6..fd80e94 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1042,6 +1042,17 @@ ifdef USE_LIBPCRE\n \tEXTLIBS += -lpcre\n endif\n \n+ifdef USE_LIBLMDB\n+\tBASIC_CFLAGS += -DUSE_LIBLMDB\n+\tifdef LIBLMDBDIR\n+\t\tBASIC_CFLAGS += -I$(LIBLMDBDIR)/include\n+\t\tEXTLIBS += -L$(LIBLMDBDIR)/$(lib) $(CC_LD_DYNPATH)$(LIBLMDBDIR)/$(lib)\n+\tendif\n+\tEXTLIBS += -llmdb\n+\tLIB_OBJS += refs/lmdb-backend.o\n+\tTEST_PROGRAMS_NEED_X += test-refs-lmdb-backend\n+endif\n+\n ifdef HAVE_ALLOCA_H\n \tBASIC_CFLAGS += -DHAVE_ALLOCA_H\n endif\n@@ -2140,6 +2151,7 @@ GIT-BUILD-OPTIONS: FORCE\n \t@echo NO_CURL=\\''$(subst ','\\'',$(subst ','\\'',$(NO_CURL)))'\\' >>$@+\n \t@echo NO_EXPAT=\\''$(subst ','\\'',$(subst ','\\'',$(NO_EXPAT)))'\\' >>$@+\n \t@echo USE_LIBPCRE=\\''$(subst ','\\'',$(subst ','\\'',$(USE_LIBPCRE)))'\\' >>$@+\n+\t@echo USE_LIBLMDB=\\''$(subst ','\\'',$(subst ','\\'',$(USE_LIBLMDB)))'\\' >>$@+\n \t@echo NO_PERL=\\''$(subst ','\\'',$(subst ','\\'',$(NO_PERL)))'\\' >>$@+\n \t@echo NO_PYTHON=\\''$(subst ','\\'',$(subst ','\\'',$(NO_PYTHON)))'\\' >>$@+\n \t@echo NO_UNIX_SOCKETS=\\''$(subst ','\\'',$(subst ','\\'',$(NO_UNIX_SOCKETS)))'\\' >>$@+\ndiff --git a/configure.ac b/configure.ac\nindex 89e2590..3853bec 100644\n--- a/configure.ac\n+++ b/configure.ac\n@@ -271,6 +271,24 @@ AS_HELP_STRING([], [ARG can be also prefix for libpcre library and hea\n dnl it yet.\n \tGIT_CONF_SUBST([LIBPCREDIR])\n fi)\n+\n+USE_LIBLMDB=YesPlease\n+AC_ARG_WITH(liblmdb,\n+AS_HELP_STRING([--with-liblmdb],[support lmdb (default is YES])\n+AS_HELP_STRING([], [ARG can be also prefix for liblmdb library and headers]),\n+ if test \"$withval\" = \"no\"; then\n+\tUSE_LIBLMDB=\n+ elif test \"$withval\" = \"yes\"; then\n+\tUSE_LIBLMDB=YesPlease\n+ else\n+\tUSE_LIBLMDB=YesPlease\n+\tLIBLMDBDIR=$withval\n+\tAC_MSG_NOTICE([Setting LIBLMDBDIR to $LIBLMDBDIR])\n+ dnl USE_LIBLMDB can still be modified below, so don't substitute\n+ dnl it yet.\n+\tGIT_CONF_SUBST([LIBLMDBDIR])\n+ fi)\n+\n #\n # Define HAVE_ALLOCA_H if you have working alloca(3) defined in that header.\n AC_FUNC_ALLOCA\n@@ -510,6 +528,21 @@ GIT_CONF_SUBST([USE_LIBPCRE])\n \n fi\n \n+if test -n \"$USE_LIBLMDB\"; then\n+\n+GIT_STASH_FLAGS($LIBLMDBDIR)\n+\n+AC_CHECK_LIB([lmdb], [mdb_env_open],\n+[USE_LIBLMDB=YesPlease],\n+[USE_LIBLMDB=])\n+\n+GIT_UNSTASH_FLAGS($LIBLMDBDIR)\n+\n+GIT_CONF_SUBST([USE_LIBLMDB])\n+\n+fi\n+\n+\n #\n # Define NO_CURL if you do not have libcurl installed. git-http-pull and\n # git-http-push are not built, and you cannot use http:// and https://\ndiff --git a/contrib/workdir/git-new-workdir b/contrib/workdir/git-new-workdir\nindex 888c34a..d153ddf 100755\n--- a/contrib/workdir/git-new-workdir\n+++ b/contrib/workdir/git-new-workdir\n@@ -28,6 +28,9 @@ git_dir=$(cd \"$orig_git\" 2>/dev/null &&\n git rev-parse --git-dir 2>/dev/null) ||\n die \"Not a git repository: \\\"$orig_git\\\"\"\n \n+ref_storage=$(git config extensions.refstorage || echo \"files\")\n+test \"$ref_storage\" != \"files\" && die \"git-new-workdir is incompatible with ref storage other than 'files'\"\n+\n case \"$git_dir\" in\n .git)\n \tgit_dir=\"$orig_git/.git\"\ndiff --git a/path.c b/path.c\nindex 2e67a2b..347aba4 100644\n--- a/path.c\n+++ b/path.c\n@@ -112,6 +112,7 @@ static struct common_dir common_list[] = {\n \t{ 0, 1, 0, \"lost-found\" },\n \t{ 0, 1, 0, \"objects\" },\n \t{ 0, 1, 0, \"refs\" },\n+\t{ 0, 1, 0, \"refs.lmdb\" },\n \t{ 0, 1, 1, \"refs/bisect\" },\n \t{ 0, 1, 0, \"remotes\" },\n \t{ 0, 1, 0, \"worktrees\" },\ndiff --git a/refs/lmdb-backend.c b/refs/lmdb-backend.c\nnew file mode 100644\nindex 0000000..3e391e61\n--- /dev/null\n+++ b/refs/lmdb-backend.c\n@@ -0,0 +1,1886 @@\n+/*\n+ * This file implements a lmdb backend for refs.\n+ *\n+ * The design of this backend relies on lmdb's write lock -- that is, any\n+ * write transaction blocks all other writers. Thus, as soon as a ref\n+ * transaction is opened, we know that any values we read won't\n+ * change out from under us, and we have a fully-consistent view of the\n+ * database.\n+ *\n+ * We store the content of refs including the trailing \\0 so that\n+ * standard C string functions can handle them. Just like struct\n+ * strbuf.\n+ */\n+#include \"cache.h\"\n+#include \n+#include \"object.h\"\n+#include \"refs.h\"\n+#include \"refs-internal.h\"\n+#include \"tag.h\"\n+#include \"lockfile.h\"\n+\n+static MDB_env *main_env;\n+\n+static char *db_path;\n+\n+struct lmdb_transaction {\n+\tMDB_txn *txn;\n+\tMDB_dbi dbi;\n+\tMDB_cursor *cursor;\n+\tconst char *submodule;\n+\tint flags;\n+};\n+\n+static struct lmdb_transaction main_transaction;\n+\n+static int in_write_transaction(void)\n+{\n+\treturn main_transaction.txn && !(main_transaction.flags & MDB_RDONLY);\n+}\n+\n+static int cleanup_registered;\n+\n+static void cleanup_txn(void)\n+{\n+\tif (main_transaction.txn)\n+\t\tmdb_txn_abort(main_transaction.txn);\n+}\n+\n+static void init_env(MDB_env **env, const char *path)\n+{\n+\tint ret;\n+\tif (*env)\n+\t\treturn;\n+\n+\tassert(path);\n+\n+\tret = mdb_env_create(env);\n+\tif (ret)\n+\t\tdie(\"BUG: mdb_env_create failed: %s\", mdb_strerror(ret));\n+\tret = mdb_env_set_maxreaders(*env, 1000);\n+\tif (ret)\n+\t\tdie(\"BUG: mdb_env_set_maxreaders failed: %s\", mdb_strerror(ret));\n+\tret = mdb_env_set_mapsize(*env, (1<<30));\n+\tif (ret)\n+\t\tdie(\"BUG: mdb_set_mapsize failed: %s\", mdb_strerror(ret));\n+\tfor (;;) {\n+\t\t/*\n+\t\t * The LMDB docs say, \"Opening a database can fail if\n+\t\t * another process is opening or closing it at exactly\n+\t\t * the same time.\" They don't specify what will\n+\t\t * happen in this case. From reading LDMB's code, it\n+\t\t * appears that init_env will return an error code\n+\t\t * from fcntl: EAGAIN or EACCES. So we'll retry in\n+\t\t * those cases.\n+\t\t */\n+\t\tret = mdb_env_open(*env, path, MDB_NOSUBDIR, 0664);\n+\t\tif (ret == 0)\n+\t\t\tbreak;\n+\t\tif (ret != EAGAIN && ret != EACCES)\n+\t\t\tdie(\"BUG: mdb_env_open (%s) failed: %s\", path,\n+\t\t\t mdb_strerror(ret));\n+\t}\n+\n+\tif (!cleanup_registered) {\n+\t\tcleanup_registered = 1;\n+\t\tatexit(cleanup_txn);\n+\t}\n+}\n+\n+static int lmdb_init_db(int shared, struct strbuf *err)\n+{\n+\t/*\n+\t * To create a db, all we need to do is set up the db path\n+\t * variable.\n+\t */\n+\n+\tif (!db_path)\n+\t\tdb_path = xstrdup(real_path(git_path(\"refs.lmdb\")));\n+\treturn 0;\n+}\n+\n+static void mdb_cursor_open_or_die(struct lmdb_transaction *transaction,\n+\t\t\t\t MDB_cursor **cursor)\n+{\n+\tint ret = mdb_cursor_open(transaction->txn, transaction->dbi, cursor);\n+\tif (ret)\n+\t\tdie(\"BUG: mdb_cursor_open failed: %s\", mdb_strerror(ret));\n+}\n+\n+static void submodule_path(struct strbuf *sb, const char *submodule,\n+\t\t\t const char *refname)\n+{\n+\tif (submodule)\n+\t\tstrbuf_git_path_submodule(sb, submodule, \"%s\", refname);\n+\telse\n+\t\tstrbuf_git_path(sb, \"%s\", refname);\n+}\n+\n+static int read_per_worktree_ref(const char *submodule, const char *refname,\n+\t\t\t\t struct MDB_val *val, int *needs_free)\n+{\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tstruct strbuf path = STRBUF_INIT;\n+\tstruct stat st;\n+\tint ret = -1;\n+\tint symlink_head;\n+\n+\tsubmodule_path(&path, submodule, refname);\n+\n+#ifdef NO_SYMLINK_HEAD\n+\tsymlink_head = 0;\n+#else\n+\tsymlink_head = 1;\n+#endif\n+\n+\tif (symlink_head) {\n+\t\tif (lstat(path.buf, &st)) {\n+\t\t\tif (errno == ENOENT)\n+\t\t\t\tret = MDB_NOTFOUND;\n+\t\t\tgoto done;\n+\t\t}\n+\t\tif (S_ISLNK(st.st_mode)) {\n+\t\t\tstrbuf_readlink(&sb, path.buf, 0);\n+\t\t\tif (starts_with(sb.buf, \"refs/\") &&\n+\t\t\t !check_refname_format(sb.buf, 0)) {\n+\t\t\t\tval->mv_data = xstrfmt(\"ref: %s\", sb.buf);\n+\t\t\t\tval->mv_size = strlen(val->mv_data) + 1;\n+\t\t\t\tret = 0;\n+\t\t\t} else {\n+\t\t\t\tret = MDB_NOTFOUND;\n+\t\t\t}\n+\t\t\tstrbuf_release(&sb);\n+\t\t\tgoto done;\n+\t\t}\n+\t}\n+\n+\tif (strbuf_read_file(&sb, path.buf, 200) < 0) {\n+\t\tstrbuf_release(&sb);\n+\t\tif (errno == ENOENT)\n+\t\t\tret = MDB_NOTFOUND;\n+\t\tgoto done;\n+\t}\n+\tstrbuf_rtrim(&sb);\n+\n+\tval->mv_data = strbuf_detach(&sb, &val->mv_size);\n+\tval->mv_size++;\n+\n+\tret = 0;\n+done:\n+\tstrbuf_release(&path);\n+\t*needs_free = !ret;\n+\treturn ret;\n+}\n+\n+static void write_per_worktree_ref(const char *submodule, const char *refname,\n+\t\t\t\t MDB_val *val)\n+{\n+\tstatic struct lock_file lock;\n+\tint fd;\n+\tint len = val->mv_size - 1;\n+\tstruct strbuf path = STRBUF_INIT;\n+\n+\tsubmodule_path(&path, submodule, refname);\n+\tsafe_create_leading_directories(path.buf);\n+\n+\tfd = hold_lock_file_for_update(&lock, path.buf, LOCK_DIE_ON_ERROR);\n+\tstrbuf_release(&path);\n+\n+\tif (write_in_full(fd, val->mv_data, len) != len ||\n+\t write_in_full(fd, \"\\n\", 1) != 1)\n+\t\tdie_errno(_(\"failed to write new HEAD\"));\n+\n+\tif (commit_lock_file(&lock))\n+\t\tdie_errno(_(\"failed to write new HEAD\"));\n+}\n+\n+static int del_per_worktree_ref(const char *submodule, const char *refname,\n+\t\t\t\tMDB_val *val)\n+{\n+\tstruct strbuf path = STRBUF_INIT;\n+\tint result;\n+\n+\t/*\n+\t * Returning deleted ref data is not yet implemented, but no\n+\t * callers need it.\n+\t */\n+\tassert(val == NULL);\n+\n+\tsubmodule_path(&path, submodule, refname);\n+\n+\tresult = unlink(path.buf);\n+\tstrbuf_release(&path);\n+\tif (result && errno != ENOENT)\n+\t\treturn 1;\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Read a ref. If the ref is a per-worktree ref, read it from disk.\n+ * Otherwise, read it from LMDB. LMDB manages its own memory, so the\n+ * data returned in *val will ordinarily not need to be freed. But\n+ * when a per-worktree ref is (successfully) read, non-LMDB memory is\n+ * allocated. In this case, *needs_free is set so that the caller can\n+ * free the memory when it is done with it.\n+ */\n+static int mdb_get_or_die(struct lmdb_transaction *transaction, MDB_val *key,\n+\t\t\t MDB_val *val, int *needs_free)\n+{\n+\tint ret;\n+\n+\tif (ref_type(key->mv_data) != REF_TYPE_NORMAL)\n+\t\treturn read_per_worktree_ref(transaction->submodule,\n+\t\t\t\t\t key->mv_data, val, needs_free);\n+\n+\t*needs_free = 0;\n+\tret = mdb_get(transaction->txn, transaction->dbi, key, val);\n+\tif (ret) {\n+\t\tif (ret != MDB_NOTFOUND)\n+\t\t\tdie(\"BUG: mdb_get failed: %s\", mdb_strerror(ret));\n+\t\treturn ret;\n+\t}\n+\treturn 0;\n+}\n+\n+static int mdb_del_or_die(struct lmdb_transaction *transaction, MDB_val *key,\n+\t\t\t MDB_val *val)\n+{\n+\tint ret;\n+\n+\tif (ref_type(key->mv_data) != REF_TYPE_NORMAL)\n+\t\tdie(\"BUG: this backend should only try to delete normal refs\");\n+\n+\tret = mdb_del(transaction->txn, transaction->dbi, key, val);\n+\tif (ret) {\n+\t\tif (ret != MDB_NOTFOUND)\n+\t\t\tdie(\"BUG: mdb_del failed: %s\", mdb_strerror(ret));\n+\t\treturn ret;\n+\t}\n+\treturn 0;\n+}\n+\n+static void mdb_put_or_die(struct lmdb_transaction *transaction, MDB_val *key,\n+\t\t\t MDB_val *val, int mode)\n+{\n+\tint ret;\n+\n+\tif (ref_type(key->mv_data) != REF_TYPE_NORMAL)\n+\t\tdie(\"BUG: this backend should only try to write normal refs\");\n+\n+\tret = mdb_put(transaction->txn, transaction->dbi, key, val, mode);\n+\tif (ret) {\n+\t\tif (ret == MDB_BAD_VALSIZE) {\n+\t\t\tMDB_env *env = mdb_txn_env(transaction->txn);\n+\t\t\tdie(_(\"Ref name %s too long (max size is %d)\"),\n+\t\t\t (const char *)key->mv_data,\n+\t\t\t mdb_env_get_maxkeysize(env));\n+\t\t} else {\n+\t\t\tdie(\"BUG: mdb_put failed: (%s -> %s) %s\",\n+\t\t\t (const char *)key->mv_data,\n+\t\t\t (const char *)val->mv_data, mdb_strerror(ret));\n+\t\t}\n+\t}\n+}\n+\n+static int mdb_cursor_get_or_die(MDB_cursor *cursor, MDB_val *key, MDB_val *val, int mode)\n+{\n+\tint ret;\n+\n+\tret = mdb_cursor_get(cursor, key, val, mode);\n+\tif (ret) {\n+\t\tif (ret != MDB_NOTFOUND)\n+\t\t\tdie(\"BUG: mdb_cursor_get failed: %s\", mdb_strerror(ret));\n+\t\treturn ret;\n+\t}\n+\tassert(((char *)val->mv_data)[val->mv_size - 1] == 0);\n+\treturn 0;\n+}\n+\n+static int mdb_cursor_del_or_die(MDB_cursor *cursor, int flags)\n+{\n+\tint ret = mdb_cursor_del(cursor, flags);\n+\tif (ret) {\n+\t\tif (ret != MDB_NOTFOUND)\n+\t\t\tdie(\"BUG: mdb_cursor_del failed: %s\", mdb_strerror(ret));\n+\t\treturn ret;\n+\t}\n+\treturn 0;\n+}\n+\n+/*\n+ * Begin a transaction. Because only one transaction per thread is\n+ * permitted, we use a global transaction object. If a read-write\n+ * transaction is presently already in-progress, and a read-only\n+ * transaction is requested, the read-write transaction will be\n+ * returned instead. If a read-write transaction is requested and a\n+ * read-only transaction is open, the read-only transaction will be\n+ * closed.\n+ *\n+ * It is a bug to request a read-write transaction during another\n+ * read-write transaction.\n+ *\n+ * As a result, it is unsafe to retain read-only transactions past the\n+ * point where a read-write transaction might be needed. For\n+ * instance, any call that has callbacks outside this module must\n+ * conclude all of its reads from the database before calling those\n+ * callbacks, or must reacquire the transaction after its callbacks\n+ * are completed.\n+ */\n+static int lmdb_transaction_begin_flags(struct strbuf *err, unsigned int flags)\n+{\n+\tint ret;\n+\tMDB_txn *txn;\n+\tstatic size_t last_txnid;\n+\tint force_restart = 0;\n+\tMDB_envinfo stat;\n+\n+\tif (!db_path)\n+\t\tdb_path = xstrdup(real_path(git_path(\"refs.lmdb\")));\n+\n+\tinit_env(&main_env, db_path);\n+\n+\t/*\n+\t * Since each transaction sees a consistent view of the db,\n+\t * downstream processes that write the db won't be seen in\n+\t * this transaction. We can check if the last transaction id\n+\t * has changed since this read transaction was started, and if\n+\t * so, we want to reopen the transaction.\n+\t */\n+\n+\tmdb_env_info(main_env, &stat);\n+\tif (stat.me_last_txnid != last_txnid) {\n+\t\tforce_restart = 1;\n+\t\tlast_txnid = stat.me_last_txnid;\n+\t}\n+\n+\tif (!main_transaction.txn) {\n+\t\tret = mdb_txn_begin(main_env, NULL, flags, &txn);\n+\t\tif (ret) {\n+\t\t\tstrbuf_addf(err, \"BUG: mdb_txn_begin failed: %s\",\n+\t\t\t\t mdb_strerror(ret));\n+\t\t\treturn -1;\n+\t\t}\n+\t\tret = mdb_dbi_open(txn, NULL, 0, &main_transaction.dbi);\n+\t\tif (ret) {\n+\t\t\tstrbuf_addf(err, \"BUG: mdb_txn_open failed: %s\",\n+\t\t\t\t mdb_strerror(ret));\n+\t\t\treturn -1;\n+\t\t}\n+\t\tmain_transaction.txn = txn;\n+\t\tmain_transaction.flags = flags;\n+\t\treturn 0;\n+\t}\n+\n+\tif (main_transaction.flags == flags && !(flags & MDB_RDONLY))\n+\t\tdie(\"BUG: rw transaction started during another rw txn\");\n+\n+\tif (force_restart || (main_transaction.flags != flags && main_transaction.flags & MDB_RDONLY)) {\n+\t\t/*\n+\t\t * RO -> RW, or forced restart due to possible changes\n+\t\t * from downstream processes.\n+\t\t */\n+\t\tmdb_txn_abort(main_transaction.txn);\n+\t\tmain_transaction.txn = NULL;\n+\t\tret = mdb_txn_begin(main_env, NULL, flags, &txn);\n+\t\tif (ret) {\n+\t\t\tstrbuf_addf(err, \"BUG: restarting txn: mdb_txn_begin failed: %s\",\n+\t\t\t\t mdb_strerror(ret));\n+\t\t\treturn -1;\n+\t\t}\n+\t\tret = mdb_dbi_open(txn, NULL, 0, &main_transaction.dbi);\n+\t\tif (ret) {\n+\t\t\tstrbuf_addf(err, \"BUG: mdb_txn_open failed: %s\",\n+\t\t\t\t mdb_strerror(ret));\n+\t\t\treturn -1;\n+\t\t}\n+\t\tmain_transaction.txn = txn;\n+\t\tmain_transaction.flags = flags;\n+\t}\n+\t/* RW -> RO just keeps the RW txn */\n+\treturn 0;\n+}\n+\n+static struct lmdb_transaction *lmdb_transaction_begin_flags_or_die(int flags)\n+{\n+\tstruct strbuf err = STRBUF_INIT;\n+\tif (lmdb_transaction_begin_flags(&err, flags))\n+\t\tdie(\"%s\", err.buf);\n+\treturn &main_transaction;\n+}\n+\n+static int verify_refname_available_txn(struct lmdb_transaction *transaction,\n+\t\t\t\t\tconst char *refname,\n+\t\t\t\t\tstruct string_list *extras,\n+\t\t\t\t\tstruct string_list *skip,\n+\t\t\t\t\tstruct strbuf *err)\n+{\n+\tMDB_cursor *cursor;\n+\tMDB_val key;\n+\tMDB_val val;\n+\tint mdb_ret;\n+\tsize_t refname_len;\n+\tchar *search_key;\n+\tconst char *extra_refname;\n+\tint ret = 1;\n+\tsize_t i;\n+\n+\tmdb_cursor_open_or_die(transaction, &cursor);\n+\n+\trefname_len = strlen(refname) + 2;\n+\tkey.mv_size = refname_len;\n+\tsearch_key = xmalloc(refname_len);\n+\tmemcpy(search_key, refname, refname_len - 2);\n+\tsearch_key[refname_len - 2] = '/';\n+\tsearch_key[refname_len - 1] = 0;\n+\tkey.mv_data = search_key;\n+\n+\t/* Check for subdirs of refname: we start at refname/ */\n+\tmdb_ret = mdb_cursor_get_or_die(cursor, &key, &val, MDB_SET_RANGE);\n+\n+\twhile (!mdb_ret) {\n+\t\tif (starts_with(key.mv_data, refname) &&\n+\t\t ((char *)key.mv_data)[refname_len - 2] == '/') {\n+\t\t\tif (skip && string_list_has_string(skip, key.mv_data))\n+\t\t\t\tgoto next;\n+\n+\t\t\tstrbuf_addf(err, _(\"'%s' exists; cannot create '%s'\"), (char *)key.mv_data, refname);\n+\t\t\tgoto done;\n+\t\t}\n+\t\tbreak;\n+next:\n+\t\tmdb_ret = mdb_cursor_get_or_die(cursor, &key, &val, MDB_NEXT);\n+\t}\n+\n+\t/* Check for parent dirs of refname. */\n+\tfor (i = 0; i < refname_len - 2; i++) {\n+\t\tif (search_key[i] == '/') {\n+\t\t\tsearch_key[i] = 0;\n+\t\t\tif (skip && string_list_has_string(skip, search_key)) {\n+\t\t\t\tsearch_key[i] = '/';\n+\t\t\t\tcontinue;\n+\t\t\t}\n+\n+\t\t\tif (extras && string_list_has_string(extras, search_key)) {\n+\t\t\t\tstrbuf_addf(err, _(\"cannot process '%s' and '%s' at the same time\"),\n+\t\t\t\t\t refname, search_key);\n+\t\t\t\tgoto done;\n+\t\t\t}\n+\n+\t\t\tkey.mv_data = search_key;\n+\t\t\tkey.mv_size = i + 1;\n+\t\t\tif (!mdb_cursor_get_or_die(cursor, &key, &val, MDB_SET)) {\n+\t\t\t\tstrbuf_addf(err, _(\"'%s' exists; cannot create '%s'\"), (char *)key.mv_data, refname);\n+\t\t\t\tgoto done;\n+\t\t\t}\n+\t\t\tsearch_key[i] = '/';\n+\t\t}\n+\t}\n+\n+\textra_refname = find_descendant_ref(refname, extras, skip);\n+\tif (extra_refname) {\n+\t\tstrbuf_addf(err,\n+\t\t\t _(\"cannot process '%s' and '%s' at the same time\"),\n+\t\t\t refname, extra_refname);\n+\t\tret = 1;\n+\t} else {\n+\t\tret = 0;\n+\t}\n+done:\n+\tmdb_cursor_close(cursor);\n+\tfree(search_key);\n+\treturn ret;\n+}\n+\n+static MDB_env *submodule_txn_begin(struct lmdb_transaction *transaction)\n+{\n+\tint ret;\n+\tMDB_env *submodule_env = NULL;\n+\tstruct strbuf path = STRBUF_INIT;\n+\n+\tstrbuf_git_path_submodule(&path, transaction->submodule, \"refs.lmdb\");\n+\n+\tinit_env(&submodule_env, path.buf);\n+\n+\tret = mdb_txn_begin(submodule_env, NULL, MDB_RDONLY, &transaction->txn);\n+\tif (ret)\n+\t\tdie(\"BUG: mdb_txn_begin failed: %s\", mdb_strerror(ret));\n+\n+\tret = mdb_dbi_open(transaction->txn, NULL, 0, &transaction->dbi);\n+\tif (ret)\n+\t\tdie(\"BUG: mdb_txn_open failed: %s\", mdb_strerror(ret));\n+\n+\tstrbuf_release(&path);\n+\treturn submodule_env;\n+}\n+\n+static struct lmdb_transaction *get_submodule_transaction(const char *submodule)\n+{\n+\tstatic char *last_submodule;\n+\tstatic struct lmdb_transaction transaction = {NULL};\n+\tstatic MDB_env *env;\n+\n+\tif (submodule == NULL) {\n+\t\tif (last_submodule != NULL) {\n+\t\t\tif (env)\n+\t\t\t\tmdb_env_close(env);\n+\t\t\tenv = NULL;\n+\t\t\tfree((char *)transaction.submodule);\n+\t\t\ttransaction.submodule = NULL;\n+\t\t\tfree(last_submodule);\n+\t\t\tlast_submodule = NULL;\n+\t\t}\n+\t\treturn lmdb_transaction_begin_flags_or_die(MDB_RDONLY);\n+\t}\n+\n+\tif (submodule != NULL && last_submodule != NULL &&\n+\t !strcmp(submodule, last_submodule) && transaction.txn != NULL)\n+\t\treturn &transaction;\n+\n+\tfree((char *)transaction.submodule);\n+\ttransaction.submodule = xstrdup(submodule);\n+\ttransaction.txn = NULL;\n+\n+\tlast_submodule = xstrdup(submodule);\n+\tenv = submodule_txn_begin(&transaction);\n+\n+\treturn &transaction;\n+}\n+\n+static int lmdb_read_raw_ref(const char *submodule, const char *refname,\n+\t\t\t unsigned char *sha1, struct strbuf *symref,\n+\t\t\t struct strbuf *sb_path, unsigned int *flags)\n+{\n+\tchar *buf;\n+\tstruct MDB_val key, val;\n+\tint needs_free = 0;\n+\tint ret = -1;\n+\tstruct lmdb_transaction *transaction;\n+\n+\ttransaction = get_submodule_transaction(submodule);\n+\n+\tval.mv_size = 0;\n+\tval.mv_data = NULL;\n+\n+\tif (flags)\n+\t\t*flags = 0;\n+\n+\tkey.mv_data = (void *)refname;\n+\tkey.mv_size = strlen(refname) + 1;\n+\tif (mdb_get_or_die(transaction, &key, &val, &needs_free)) {\n+\t\terrno = ENOENT;\n+\t\treturn -1;\n+\t}\n+\n+\tbuf = val.mv_data;\n+\tassert(buf[val.mv_size - 1] == 0);\n+\n+\tif (starts_with(buf, \"ref:\")) {\n+\t\tbuf += 4;\n+\t\twhile (isspace(*buf))\n+\t\t\tbuf++;\n+\n+\t\tstrbuf_reset(symref);\n+\t\tstrbuf_addstr(symref, buf);\n+\t\tif (flags)\n+\t\t\t*flags |= REF_ISSYMREF;\n+\t\tret = 0;\n+\t\tgoto done;\n+\t}\n+\n+\t/*\n+\t * Please note that FETCH_HEAD has additional\n+\t * data after the sha.\n+\t */\n+\tif (get_sha1_hex(buf, sha1) ||\n+\t (buf[40] != '\\0' && !isspace(buf[40]))) {\n+\t\tif (flags)\n+\t\t\t*flags |= REF_ISBROKEN;\n+\t\terrno = EINVAL;\n+\t\tgoto done;\n+\t}\n+\tret = 0;\n+\tgoto done;\n+\n+done:\n+\tif (needs_free)\n+\t\tfree(val.mv_data);\n+\treturn ret;\n+\n+}\n+\n+static void write_u64(char *buf, uint64_t number)\n+{\n+\tint i;\n+\n+\tfor (i = 0; i < 8; i++)\n+\t\tbuf[i] = (number >> (i * 8)) & 0xff;\n+}\n+\n+static int show_one_reflog_ent(struct strbuf *sb, each_reflog_ent_fn fn, void *cb_data)\n+{\n+\tunsigned char osha1[20], nsha1[20];\n+\tchar *email_end, *message;\n+\tunsigned long timestamp;\n+\tint tz;\n+\n+\t/* old (raw sha) new (raw sha) name SP time TAB msg LF */\n+\tif (sb->len < 41 || sb->buf[sb->len - 1] != '\\n' ||\n+\t !(email_end = strchr(sb->buf + 40, '>')) ||\n+\t email_end[1] != ' ')\n+\t\treturn 0; /* corrupt? */\n+\n+\ttimestamp = strtoul(email_end + 2, &message, 10);\n+\n+\tif (!timestamp ||\n+\t !message || message[0] != ' ' ||\n+\t (message[1] != '+' && message[1] != '-') ||\n+\t !isdigit(message[2]) || !isdigit(message[3]) ||\n+\t !isdigit(message[4]) || !isdigit(message[5]))\n+\t\treturn 0; /* corrupt? */\n+\n+\thashcpy(osha1, (const unsigned char *)sb->buf);\n+\thashcpy(nsha1, (const unsigned char *)sb->buf + 20);\n+\n+\temail_end[1] = '\\0';\n+\ttz = strtol(message + 1, NULL, 10);\n+\tif (message[6] != '\\t')\n+\t\tmessage += 6;\n+\telse\n+\t\tmessage += 7;\n+\treturn fn(osha1, nsha1, sb->buf + 40, timestamp, tz, message, cb_data);\n+}\n+\n+static void format_reflog_entry(struct strbuf *buf,\n+\t\t\t\tconst unsigned char *old_sha1,\n+\t\t\t\tconst unsigned char *new_sha1,\n+\t\t\t\tconst char *committer, const char *msg)\n+{\n+\tint len;\n+\tint msglen;\n+\n+\tassert(buf->len == 0);\n+\tstrbuf_add(buf, old_sha1, 20);\n+\tstrbuf_add(buf, new_sha1, 20);\n+\tstrbuf_addstr(buf, committer);\n+\tstrbuf_addch(buf, '\\n');\n+\n+\tlen = buf->len;\n+\tmsglen = msg ? strlen(msg) : 0;\n+\tif (msglen) {\n+\t\tint copied;\n+\t\tstrbuf_grow(buf, msglen + 1);\n+\t\tcopied = copy_reflog_msg(buf->buf + 40 + strlen(committer), msg) - 1;\n+\t\tbuf->len = len + copied;\n+\t\tbuf->buf[buf->len] = 0;\n+\t}\n+}\n+\n+static int log_ref_write(const char *refname,\n+\t\t\t const unsigned char *old_sha1,\n+\t\t\t const unsigned char *new_sha1,\n+\t\t\t const char *msg,\n+\t\t\t int flags,\n+\t\t\t struct strbuf *err)\n+{\n+\tMDB_val key, val;\n+\tuint64_t now = getnanotime();\n+\tint result;\n+\tstruct strbuf log_key = STRBUF_INIT;\n+\tint refname_len;\n+\tMDB_cursor *cursor;\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tconst char *timestamp;\n+\tuint64_t zero = 0;\n+\n+\tif (log_all_ref_updates < 0)\n+\t\tlog_all_ref_updates = !is_bare_repository();\n+\n+\t/* it is assumed that we are in a ref transaction here */\n+\tassert(main_transaction.txn);\n+\n+\tresult = safe_create_reflog(refname, flags & REF_FORCE_CREATE_REFLOG, err);\n+\tif (result)\n+\t\treturn result;\n+\n+\t/* \"logs/\" + refname + \\0 + 8-byte timestamp for sorting and expiry. */\n+\trefname_len = strlen(refname);\n+\n+\tstrbuf_addf(&log_key, \"logs/%s\", refname);\n+\tstrbuf_add(&log_key, &zero, 8);\n+\tkey.mv_data = log_key.buf;\n+\tkey.mv_size = log_key.len + 1;\n+\n+\tmdb_cursor_open_or_die(&main_transaction, &cursor);\n+\n+\t/* if no reflog exists, we're done */\n+\tif (mdb_cursor_get_or_die(cursor, &key, &val, MDB_SET_RANGE) ||\n+\t strcmp(key.mv_data, log_key.buf))\n+\t\tgoto done;\n+\n+\t/* Is this a header? We only need the header for empty reflogs */\n+\ttimestamp = (const char *)key.mv_data + refname_len + 6;\n+\tif (ntohll(*(uint64_t *)timestamp) == 0)\n+\t\tmdb_cursor_del_or_die(cursor, 0);\n+\n+\tkey.mv_data = log_key.buf;\n+\n+\twrite_u64((char *)key.mv_data + refname_len + 6, htonll(now));\n+\n+\tformat_reflog_entry(&buf, old_sha1, new_sha1,\n+\t\t\t git_committer_info(0), msg);\n+\tassert(buf.len >= 42);\n+\tval.mv_data = buf.buf;\n+\tval.mv_size = buf.len + 1;\n+\n+\tmdb_put_or_die(&main_transaction, &key, &val, 0);\n+\tstrbuf_release(&buf);\n+\n+done:\n+\tstrbuf_release(&log_key);\n+\tmdb_cursor_close(cursor);\n+\treturn 0;\n+}\n+\n+static int lmdb_verify_refname_available(const char *refname,\n+\t\t\t\t\t struct string_list *extras,\n+\t\t\t\t\t struct string_list *skip,\n+\t\t\t\t\t struct strbuf *err)\n+{\n+\tlmdb_transaction_begin_flags_or_die(MDB_RDONLY);\n+\treturn verify_refname_available_txn(&main_transaction, refname, extras, skip, err);\n+}\n+\n+/*\n+ * Attempt to resolve `refname` to `old_sha1` (if old_sha1 is\n+ * non-null). The return value is a pointer to a newly-allocated\n+ * string containing the next ref name that this resolves to. So if\n+ * HEAD is a symbolic ref to refs/heads/example, which is itself a\n+ * symbolic ref to refs/heads/foo, return refs/heads/example,\n+ * and fill in resolved_sha1 with the sha of refs/heads/foo.\n+ */\n+static char *check_ref(MDB_txn *txn, const char *refname,\n+\t\t const unsigned char *old_sha1,\n+\t\t unsigned char *resolved_sha1, int flags,\n+\t\t int *type_p)\n+{\n+\tint mustexist = (old_sha1 && !is_null_sha1(old_sha1));\n+\tint resolve_flags = 0;\n+\tint type;\n+\tchar *resolved_refname;\n+\n+\tif (mustexist)\n+\t\tresolve_flags |= RESOLVE_REF_READING;\n+\tif (flags & REF_DELETING) {\n+\t\tresolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;\n+\t\tif (flags & REF_NODEREF)\n+\t\t\tresolve_flags |= RESOLVE_REF_NO_RECURSE;\n+\t}\n+\n+\t/*\n+\t * The first time we resolve the refname, we're just trying to\n+\t * see if there is any ref at all by this name, even if it is\n+\t * a broken symref.\n+\t */\n+\trefname = resolve_ref_unsafe(refname, resolve_flags,\n+\t\t\t\t resolved_sha1, &type);\n+\tif (type_p)\n+\t\t*type_p = type;\n+\n+\tif (!refname)\n+\t\treturn NULL;\n+\n+\t/*\n+\t * Need to copy refname here because the resolve_ref_unsafe\n+\t * returns a pointer to a static buffer that could get mangled\n+\t * by the second call.\n+\t */\n+\tresolved_refname = xstrdup(refname);\n+\n+\tif (old_sha1) {\n+\t\tif (flags & REF_NODEREF) {\n+\t\t\tresolve_flags &= ~RESOLVE_REF_NO_RECURSE;\n+\n+\t\t\tresolve_ref_unsafe(refname, resolve_flags,\n+\t\t\t\t\t resolved_sha1, &type);\n+\t\t}\n+\t\tif (hashcmp(old_sha1, resolved_sha1)) {\n+\t\t\terror(_(\"ref %s is at %s but expected %s\"), refname,\n+\t\t\t sha1_to_hex(resolved_sha1), sha1_to_hex(old_sha1));\n+\n+\t\t\treturn NULL;\n+\t\t}\n+\t}\n+\treturn resolved_refname;\n+}\n+\n+static int mdb_transaction_commit(struct lmdb_transaction *transaction,\n+\t\t\t\t struct strbuf *err)\n+{\n+\tint result;\n+\n+\tresult = mdb_txn_commit(transaction->txn);\n+\tif (result && err)\n+\t\tstrbuf_addstr(err, mdb_strerror(result));\n+\n+\ttransaction->txn = NULL;\n+\treturn result;\n+}\n+\n+static void mdb_transaction_abort(struct lmdb_transaction *transaction)\n+{\n+\tmdb_txn_abort(transaction->txn);\n+\ttransaction->txn = NULL;\n+}\n+\n+static int lmdb_delete_reflog(const char *refname)\n+{\n+\tMDB_val key, val;\n+\tchar *log_path;\n+\tint len;\n+\tMDB_cursor *cursor;\n+\tint ret = 0;\n+\tint mdb_ret;\n+\tstruct strbuf err = STRBUF_INIT;\n+\tint in_transaction;\n+\n+\tin_transaction = in_write_transaction();\n+\n+\tlog_path = xstrfmt(\"logs/%s\", refname);\n+\tlen = strlen(log_path) + 1;\n+\n+\tkey.mv_data = log_path;\n+\tkey.mv_size = len;\n+\n+\tif (!in_transaction)\n+\t\tlmdb_transaction_begin_flags_or_die(0);\n+\n+\tmdb_cursor_open_or_die(&main_transaction, &cursor);\n+\n+\tmdb_ret = mdb_cursor_get_or_die(cursor, &key, &val, MDB_SET_RANGE);\n+\n+\twhile (!mdb_ret) {\n+\t\tif (key.mv_size < len)\n+\t\t\tbreak;\n+\n+\t\tif (!starts_with(key.mv_data, log_path) ||\n+\t\t ((char *)key.mv_data)[len - 1] != 0)\n+\t\t\tbreak;\n+\n+\t\tmdb_cursor_del_or_die(cursor, 0);\n+\t\tmdb_ret = mdb_cursor_get_or_die(cursor, &key, &val, MDB_NEXT);\n+\t}\n+\n+\tfree(log_path);\n+\tmdb_cursor_close(cursor);\n+\tmain_transaction.cursor = NULL;\n+\n+\tif (!in_transaction && mdb_transaction_commit(&main_transaction, &err)) {\n+\t\twarning(\"%s\", err.buf);\n+\t\tret = 01;\n+\t}\n+\tstrbuf_release(&err);\n+\treturn ret;\n+}\n+\n+#define REF_NO_REFLOG 0x8000\n+\n+static int lmdb_transaction_update(const char *refname,\n+\t\t\t\t const unsigned char *new_sha1,\n+\t\t\t\t const unsigned char *old_sha1,\n+\t\t\t\t unsigned int flags, const char *msg,\n+\t\t\t\t struct strbuf *err)\n+{\n+\tconst char *orig_refname = refname;\n+\tMDB_val key, val;\n+\tunsigned char resolved_sha1[20];\n+\tint type;\n+\tint ret = -1;\n+\n+\tif ((flags & REF_HAVE_NEW) && is_null_sha1(new_sha1))\n+\t\tflags |= REF_DELETING;\n+\n+\tif (new_sha1 && !is_null_sha1(new_sha1) &&\n+\t check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n+\t\tstrbuf_addf(err, _(\"refusing to update ref with bad name %s\"),\n+\t\t\t refname);\n+\t\treturn TRANSACTION_GENERIC_ERROR;\n+\t}\n+\n+\trefname = check_ref(main_transaction.txn, orig_refname, old_sha1,\n+\t\t\t resolved_sha1, flags, &type);\n+\tif (refname == NULL) {\n+\t\tstrbuf_addf(err, _(\"cannot lock the ref '%s'\"), orig_refname);\n+\t\treturn TRANSACTION_GENERIC_ERROR;\n+\t}\n+\n+\tif (!(flags & REF_DELETING) && is_null_sha1(resolved_sha1) &&\n+\t verify_refname_available_txn(&main_transaction, refname, NULL, NULL, err))\n+\t\treturn TRANSACTION_NAME_CONFLICT;\n+\n+\tif (flags & REF_NODEREF) {\n+\t\tfree((void *)refname);\n+\t\trefname = orig_refname;\n+\t}\n+\n+\tkey.mv_size = strlen(refname) + 1;\n+\tkey.mv_data = (void *)refname;\n+\n+\tif ((flags & REF_HAVE_NEW) && !is_null_sha1(new_sha1)) {\n+\t\tint overwriting_symref = ((type & REF_ISSYMREF) &&\n+\t\t\t\t\t (flags & REF_NODEREF));\n+\n+\t\tstruct object *o = parse_object(new_sha1);\n+\t\tif (!o) {\n+\t\t\tstrbuf_addf(err,\n+\t\t\t\t _(\"Trying to write ref %s with nonexistent object %s\"),\n+\t\t\t\t refname, sha1_to_hex(new_sha1));\n+\t\t\tgoto done;\n+\t\t}\n+\t\tif (o->type != OBJ_COMMIT && is_branch(refname)) {\n+\t\t\tstrbuf_addf(err,\n+\t\t\t\t _(\"Trying to write non-commit object %s to branch %s\"),\n+\t\t\t\t sha1_to_hex(new_sha1), refname);\n+\t\t\tgoto done;\n+\t\t}\n+\n+\t\tif (!overwriting_symref\n+\t\t && !hashcmp(resolved_sha1, new_sha1)) {\n+\t\t\t/*\n+\t\t\t * The reference already has the desired\n+\t\t\t * value, so we don't need to write it.\n+\t\t\t */\n+\t\t\tflags |= REF_NO_REFLOG;\n+\t\t} else {\n+\t\t\tval.mv_size = 41;\n+\t\t\tif (new_sha1)\n+\t\t\t\tval.mv_data = sha1_to_hex(new_sha1);\n+\t\t\telse\n+\t\t\t\tval.mv_data = sha1_to_hex(null_sha1);\n+\t\t\tmdb_put_or_die(&main_transaction, &key, &val, 0);\n+\t\t}\n+\t}\n+\n+\tif (flags & REF_DELETING) {\n+\t\tif (mdb_del_or_die(&main_transaction, &key, NULL)) {\n+\t\t\tif (old_sha1 && !is_null_sha1(old_sha1)) {\n+\t\t\t\tstrbuf_addf(err, _(\"No such ref %s\"), refname);\n+\t\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\t\tgoto done;\n+\t\t\t}\n+\t\t}\n+\t\tlmdb_delete_reflog(orig_refname);\n+\t} else if (!(flags & REF_NO_REFLOG)) {\n+\t\tif (!new_sha1)\n+\t\t\tnew_sha1 = null_sha1;\n+\t\tif (log_ref_write(orig_refname, resolved_sha1,\n+\t\t\t\t new_sha1, msg, flags, err) < 0)\n+\t\t\tgoto done;\n+\t\tif (strcmp(refname, orig_refname) &&\n+\t\t log_ref_write(refname, resolved_sha1,\n+\t\t\t\t new_sha1, msg, flags, err) < 0)\n+\t\t\tgoto done;\n+\t}\n+\n+\tret = 0;\n+done:\n+\tif (refname != orig_refname)\n+\t\tfree((void *) refname);\n+\treturn ret;\n+}\n+\n+static int lmdb_transaction_commit(struct ref_transaction *ref_transaction,\n+\t\t\t\t struct string_list *affected_refnames,\n+\t\t\t\t struct strbuf *err)\n+{\n+\tint ret = 0, i;\n+\tint n = ref_transaction->nr;\n+\tstruct ref_update **updates = ref_transaction->updates;\n+\n+\t/*\n+\t * We might already be in a write transaction, because some\n+\t * lmdb backend functionality is implemented in terms of\n+\t * (other stuff) + ref_transaction_commit\n+\t */\n+\tif (!in_write_transaction())\n+\t\tlmdb_transaction_begin_flags_or_die(0);\n+\n+\tfor (i = 0; i < n; i++) {\n+\t\tstruct ref_update *update = updates[i];\n+\n+\t\tif (lmdb_transaction_update(update->refname,\n+\t\t\t\t\t update->new_sha1,\n+\t\t\t\t\t (update->flags & REF_HAVE_OLD) ?\n+\t\t\t\t\t update->old_sha1 : NULL,\n+\t\t\t\t\t update->flags,\n+\t\t\t\t\t update->msg,\n+\t\t\t\t\t err)) {\n+\t\t\tmdb_transaction_abort(&main_transaction);\n+\t\t\tret = -1;\n+\t\t\tgoto cleanup;\n+\t\t}\n+\n+\t}\n+\tret = mdb_transaction_commit(&main_transaction, err);\n+\n+cleanup:\n+\tref_transaction->state = REF_TRANSACTION_CLOSED;\n+\treturn ret;\n+}\n+\n+static int rename_reflog_ent(unsigned char *osha1, unsigned char *nsha1,\n+\t\t\t const char *email, unsigned long timestamp, int tz,\n+\t\t\t const char *message, void *cb_data)\n+{\n+\n+\tconst char *newrefname = cb_data;\n+\tMDB_val key, new_key, val;\n+\tstruct strbuf new_key_buf = STRBUF_INIT;\n+\n+\tassert(main_transaction.cursor);\n+\n+\tif (mdb_cursor_get_or_die(main_transaction.cursor, &key, &val, MDB_GET_CURRENT))\n+\t\tdie(\"BUG: renaming ref: mdb_cursor_get failed to get current\");\n+\n+\t/* This must really be a reflog entry */\n+\tassert(val.mv_size > 42);\n+\n+\tstrbuf_addf(&new_key_buf, \"logs/%s\", newrefname);\n+\tstrbuf_add(&new_key_buf, (char *)key.mv_data + key.mv_size - 8, 8);\n+\n+\tnew_key.mv_size = new_key_buf.len + 1;\n+\tnew_key.mv_data = new_key_buf.buf;\n+\tmdb_put_or_die(&main_transaction, &new_key, &val, 0);\n+\tmdb_cursor_del_or_die(main_transaction.cursor, 0);\n+\tstrbuf_release(&new_key_buf);\n+\treturn 0;\n+}\n+\n+static int lmdb_rename_ref(const char *oldref, const char *newref, const char *logmsg)\n+{\n+\tunsigned char orig_sha1[20];\n+\tint flag = 0, resolve_flags;\n+\tint log = reflog_exists(oldref);\n+\tconst char *symref = NULL;\n+\tstruct strbuf err = STRBUF_INIT;\n+\tstruct ref_transaction *ref_transaction;\n+\n+\tif (!strcmp(oldref, newref))\n+\t\treturn 0;\n+\n+\tlmdb_transaction_begin_flags_or_die(0);\n+\n+\tref_transaction = ref_transaction_begin(&err);\n+\tif (!ref_transaction)\n+\t\tdie(\"%s\", err.buf);\n+\n+\tresolve_flags = RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE;\n+\tsymref = resolve_ref_unsafe(oldref, resolve_flags,\n+\t\t\t\t orig_sha1, &flag);\n+\tif (flag & REF_ISSYMREF) {\n+\t\terror(_(\"refname %s is a symbolic ref, renaming it is not supported\"),\n+\t\t oldref);\n+\t\tgoto fail;\n+\t}\n+\tif (!symref) {\n+\t\tmdb_transaction_abort(&main_transaction);\n+\t\terror(_(\"refname %s not found\"), oldref);\n+\t\tgoto fail;\n+\t}\n+\tif (!rename_ref_available(oldref, newref))\n+\t\tgoto fail;\n+\n+\t/* Copy the reflog from the old to the new */\n+\tif (log) {\n+\t\tstruct strbuf old_log_sentinel = STRBUF_INIT;\n+\t\tMDB_val key;\n+\t\tint log_all;\n+\n+\t\tlog_all = log_all_ref_updates;\n+\t\tlog_all_ref_updates = 1;\n+\t\tif (safe_create_reflog(newref, 0, &err)) {\n+\t\t\terror(_(\"can't create reflog for %s: %s\"), newref, err.buf);\n+\t\t\tstrbuf_release(&err);\n+\t\t\tgoto fail;\n+\t\t}\n+\t\tlog_all_ref_updates = log_all;\n+\n+\t\tfor_each_reflog_ent(oldref, rename_reflog_ent, (void *)newref);\n+\t\tstrbuf_addf(&old_log_sentinel, \"logs/%sxxxxxxxx\", oldref);\n+\t\tmemset(old_log_sentinel.buf + old_log_sentinel.len - 8, 0, 8);\n+\n+\t\tkey.mv_size = old_log_sentinel.len;\n+\t\tkey.mv_data = old_log_sentinel.buf;\n+\n+\t\t/* It's OK if the old reflog is missing */\n+\t\tmdb_del_or_die(&main_transaction, &key, NULL);\n+\t\tstrbuf_release(&old_log_sentinel);\n+\t}\n+\n+\tif (ref_transaction_delete(ref_transaction, oldref,\n+\t\t\t\t orig_sha1, REF_NODEREF, NULL, &err)) {\n+\t\terror(_(\"unable to delete old %s\"), oldref);\n+\t\tgoto fail;\n+\t}\n+\n+\tif (ref_transaction_update(ref_transaction, newref, orig_sha1, NULL,\n+\t\t\t\t REF_NODEREF, logmsg, &err)) {\n+\t\terror(\"%s\", err.buf);\n+\t\tgoto fail;\n+\t}\n+\n+\tif (ref_transaction_commit(ref_transaction, &err)) {\n+\t\terror(\"%s\", err.buf);\n+\t\tgoto fail;\n+\t}\n+\n+\treturn 0;\n+\n+fail:\n+\tref_transaction_free(ref_transaction);\n+\tstrbuf_release(&err);\n+\tmdb_transaction_abort(&main_transaction);\n+\tmain_transaction.txn = NULL;\n+\treturn 1;\n+}\n+\n+static int lmdb_delete_refs(struct string_list *refnames)\n+{\n+\tint i;\n+\tstruct strbuf err = STRBUF_INIT;\n+\tint result = 0;\n+\n+\tif (!refnames->nr)\n+\t\treturn 0;\n+\n+\tlmdb_transaction_begin_flags_or_die(0);\n+\n+\tfor (i = 0; i < refnames->nr; i++) {\n+\t\tconst char *refname = refnames->items[i].string;\n+\n+\t\tif (lmdb_transaction_update(refname, null_sha1, NULL,\n+\t\t\t\t\t REF_DELETING, NULL, &err))\n+\t\t\tresult |= error(_(\"could not remove reference %s: %s\"),\n+\t\t\t\t\trefname, err.buf);\n+\t}\n+\n+\tresult |= mdb_transaction_commit(&main_transaction, &err);\n+\tstrbuf_release(&err);\n+\treturn 0;\n+}\n+\n+static int lmdb_for_each_reflog_ent_order(const char *refname,\n+\t\t\t\t\t each_reflog_ent_fn fn,\n+\t\t\t\t\t void *cb_data, int reverse)\n+{\n+\tMDB_val key, val;\n+\tchar *search_key;\n+\tchar *log_path;\n+\tint len;\n+\tMDB_cursor *cursor;\n+\tint ret = 0;\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tenum MDB_cursor_op direction = reverse ? MDB_PREV : MDB_NEXT;\n+\tuint64_t zero = 0ULL;\n+\n+\tlog_path = xstrfmt(\"logs/%s\", refname);\n+\tlen = strlen(log_path) + 1;\n+\n+\tif (reverse) {\n+\t\t/*\n+\t\t * For a reverse search, start at the key\n+\t\t * lexicographically after the searched-for key.\n+\t\t * That's the one with \\1 appended to the key.\n+\t\t */\n+\t\tsearch_key = xstrfmt(\"%s\\1\", log_path);\n+\t\tkey.mv_size = len + 1;\n+\t} else {\n+\t\tsearch_key = xstrdup(log_path);\n+\t\tkey.mv_size = len;\n+\t}\n+\n+\tkey.mv_data = search_key;\n+\n+\tlmdb_transaction_begin_flags_or_die(MDB_RDONLY);\n+\n+\tmdb_cursor_open_or_die(&main_transaction, &cursor);\n+\n+\tmain_transaction.cursor = cursor;\n+\n+\t/*\n+\t * MDB's cursor API requires that the first mdb_cursor_get be\n+\t * called with MDB_SET_RANGE. For reverse searches, this will\n+\t * give us the entry one-past the entry we're looking for, so\n+\t * we should jump back using MDB_PREV.\n+\t */\n+\tmdb_cursor_get_or_die(cursor, &key, &val, MDB_SET_RANGE);\n+\tif (direction == MDB_PREV)\n+\t\tmdb_cursor_get_or_die(cursor, &key, &val, direction);\n+\n+\tdo {\n+\t\tif (key.mv_size < len)\n+\t\t\tbreak;\n+\n+\t\tif (!starts_with(key.mv_data, log_path) || ((char *)key.mv_data)[len - 1] != 0)\n+\t\t\tbreak;\n+\n+\t\tif (!memcmp(&zero, ((char *)key.mv_data) + key.mv_size - 8, 8))\n+\t\t\tcontinue;\n+\n+\t\tassert(val.mv_size != 0);\n+\n+\t\tstrbuf_add(&sb, val.mv_data, val.mv_size - 1);\n+\t\tret = show_one_reflog_ent(&sb, fn, cb_data);\n+\t\tif (ret)\n+\t\t\tbreak;\n+\n+\t\tstrbuf_reset(&sb);\n+\t} while (!mdb_cursor_get_or_die(cursor, &key, &val, direction));\n+\n+\tstrbuf_release(&sb);\n+\tfree(log_path);\n+\tfree(search_key);\n+\tmdb_cursor_close(cursor);\n+\treturn ret;\n+}\n+\n+static int lmdb_for_each_reflog_ent(const char *refname,\n+\t\t\t\t each_reflog_ent_fn fn,\n+\t\t\t\t void *cb_data)\n+{\n+\tif (ref_type(refname) != REF_TYPE_NORMAL)\n+\t\treturn refs_be_files.for_each_reflog_ent(refname, fn, cb_data);\n+\treturn lmdb_for_each_reflog_ent_order(refname, fn, cb_data, 0);\n+}\n+\n+static int lmdb_for_each_reflog_ent_reverse(const char *refname,\n+\t\t\t\t\t each_reflog_ent_fn fn,\n+\t\t\t\t\t void *cb_data)\n+{\n+\tif (ref_type(refname) != REF_TYPE_NORMAL)\n+\t\treturn refs_be_files.for_each_reflog_ent_reverse(refname, fn, cb_data);\n+\treturn lmdb_for_each_reflog_ent_order(refname, fn, cb_data, 1);\n+}\n+\n+static int lmdb_reflog_exists(const char *refname)\n+{\n+\tMDB_val key, val;\n+\tchar *log_path;\n+\tint len;\n+\tMDB_cursor *cursor;\n+\tint ret = 1;\n+\n+\tif (ref_type(refname) != REF_TYPE_NORMAL)\n+\t\treturn refs_be_files.reflog_exists(refname);\n+\n+\tlog_path = xstrfmt(\"logs/%s\", refname);\n+\tlen = strlen(log_path) + 1;\n+\n+\tkey.mv_data = log_path;\n+\tkey.mv_size = len;\n+\n+\tlmdb_transaction_begin_flags_or_die(MDB_RDONLY);\n+\tmdb_cursor_open_or_die(&main_transaction, &cursor);\n+\n+\tif (mdb_cursor_get_or_die(cursor, &key, &val, MDB_SET_RANGE) ||\n+\t !starts_with(key.mv_data, log_path))\n+\t\tret = 0;\n+\n+\tfree(log_path);\n+\tmdb_cursor_close(cursor);\n+\n+\treturn ret;\n+}\n+\n+struct wrapped_each_ref_fn {\n+\teach_ref_fn *fn;\n+\tvoid *cb_data;\n+};\n+\n+static int check_reflog(const char *refname,\n+\t\t\tconst struct object_id *oid, int flags, void *cb_data)\n+{\n+\tstruct wrapped_each_ref_fn *wrapped = cb_data;\n+\n+\tif (reflog_exists(refname))\n+\t\treturn wrapped->fn(refname, oid, 0, wrapped->cb_data);\n+\n+\treturn 0;\n+}\n+\n+static int lmdb_for_each_reflog(each_ref_fn fn, void *cb_data)\n+{\n+\tstruct wrapped_each_ref_fn wrapped = {fn, cb_data};\n+\tint result = head_ref(fn, cb_data);\n+\tif (result)\n+\t\treturn result;\n+\treturn for_each_ref(check_reflog, &wrapped);\n+}\n+\n+static void strbuf_reflog_header(struct strbuf *sb, const char *refname)\n+{\n+\tuint64_t zero = 0;\n+\n+\tstrbuf_addf(sb, \"logs/%s\", refname);\n+\tstrbuf_add(sb, &zero, 8);\n+}\n+\n+static int lmdb_create_reflog(const char *refname, int force_create, struct strbuf *err)\n+{\n+\t/*\n+\t * We mark that there is a reflog by creating a key of the\n+\t * form logs/$refname followed by nine \\0 (one for\n+\t * string-termination, 8 in lieu of a timestamp), with an empty\n+\t * value.\n+\t */\n+\n+\tint in_transaction = in_write_transaction();\n+\tMDB_val key, val;\n+\tstruct strbuf key_buf = STRBUF_INIT;\n+\n+\tif (!force_create && !should_autocreate_reflog(refname))\n+\t\treturn 0;\n+\n+\tif (!in_transaction)\n+\t\tlmdb_transaction_begin_flags_or_die(0);\n+\n+\tstrbuf_reflog_header(&key_buf, refname);\n+\tkey.mv_size = key_buf.len + 1;\n+\tkey.mv_data = key_buf.buf;\n+\n+\tval.mv_size = 0;\n+\tval.mv_data = NULL;\n+\tmdb_put_or_die(&main_transaction, &key, &val, 0);\n+\n+\tstrbuf_release(&key_buf);\n+\tif (!in_transaction)\n+\t\treturn mdb_transaction_commit(&main_transaction, err);\n+\treturn 0;\n+}\n+\n+struct expire_reflog_cb {\n+\tunsigned int flags;\n+\treflog_expiry_should_prune_fn *should_prune_fn;\n+\tvoid *policy_cb;\n+\tunsigned char last_kept_sha1[20];\n+};\n+\n+static int expire_reflog_ent(unsigned char *osha1, unsigned char *nsha1,\n+\t\t\t const char *email, unsigned long timestamp, int tz,\n+\t\t\t const char *message, void *cb_data)\n+{\n+\tstruct expire_reflog_cb *cb = cb_data;\n+\tstruct expire_reflog_policy_cb *policy_cb = cb->policy_cb;\n+\n+\tif (cb->flags & EXPIRE_REFLOGS_REWRITE)\n+\t\tosha1 = cb->last_kept_sha1;\n+\n+\tif ((*cb->should_prune_fn)(osha1, nsha1, email, timestamp, tz,\n+\t\t\t\t message, policy_cb)) {\n+\t\tif (cb->flags & EXPIRE_REFLOGS_DRY_RUN)\n+\t\t\tprintf(\"would prune %s\", message);\n+\t\telse {\n+\t\t\tif (cb->flags & EXPIRE_REFLOGS_VERBOSE)\n+\t\t\t\tprintf(\"prune %s\", message);\n+\n+\t\t\tmdb_cursor_del_or_die(main_transaction.cursor, 0);\n+\t\t}\n+\t} else {\n+\t\thashcpy(cb->last_kept_sha1, nsha1);\n+\t\tif (cb->flags & EXPIRE_REFLOGS_VERBOSE)\n+\t\t\tprintf(\"keep %s\", message);\n+\t}\n+\treturn 0;\n+}\n+\n+static int write_ref(const char *refname, const unsigned char *sha1)\n+{\n+\tstruct strbuf err = STRBUF_INIT;\n+\tstruct ref_transaction *transaction;\n+\n+\ttransaction = ref_transaction_begin(&err);\n+\tif (!transaction) {\n+\t\terror(\"%s\", err.buf);\n+\t\tstrbuf_release(&err);\n+\t\treturn -1;\n+\t}\n+\n+\tif (ref_transaction_update(transaction, refname, sha1, NULL,\n+\t\t\t\t REF_NO_REFLOG, NULL, &err)) {\n+\t\terror(\"%s\", err.buf);\n+\t\tstrbuf_release(&err);\n+\t\treturn -1;\n+\t}\n+\n+\tif (ref_transaction_commit(transaction, &err)) {\n+\t\terror(\"%s\", err.buf);\n+\t\tstrbuf_release(&err);\n+\t\treturn -1;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static int lmdb_reflog_expire(const char *refname, const unsigned char *sha1,\n+\t\t\t unsigned int flags,\n+\t\t\t reflog_expiry_prepare_fn prepare_fn,\n+\t\t\t reflog_expiry_should_prune_fn should_prune_fn,\n+\t\t\t reflog_expiry_cleanup_fn cleanup_fn,\n+\t\t\t void *policy_cb_data)\n+{\n+\tstruct expire_reflog_cb cb;\n+\tint dry_run = flags & EXPIRE_REFLOGS_DRY_RUN;\n+\tint status = 0;\n+\tstruct strbuf err = STRBUF_INIT;\n+\tunsigned char resolved_sha1[20];\n+\tint type;\n+\tchar *resolved;\n+\n+\tif (ref_type(refname) != REF_TYPE_NORMAL)\n+\t\treturn refs_be_files.reflog_expire(refname, sha1, flags, prepare_fn,\n+\t\t\t\t\t should_prune_fn, cleanup_fn,\n+\t\t\t\t\t policy_cb_data);\n+\n+\tmemset(&cb, 0, sizeof(cb));\n+\tcb.flags = flags;\n+\tcb.policy_cb = policy_cb_data;\n+\tcb.should_prune_fn = should_prune_fn;\n+\n+\tlmdb_transaction_begin_flags_or_die(dry_run ? MDB_RDONLY : 0);\n+\n+\tresolved = check_ref(main_transaction.txn, refname, sha1,\n+\t\t\t resolved_sha1, 0, &type);\n+\tif (!resolved)\n+\t\tdie(_(\"Failed to resolve %s\"), refname);\n+\tfree(resolved);\n+\n+\t(*prepare_fn)(refname, sha1, cb.policy_cb);\n+\tlmdb_for_each_reflog_ent(refname, expire_reflog_ent, &cb);\n+\t(*cleanup_fn)(cb.policy_cb);\n+\n+\tif (!dry_run) {\n+\t\t/*\n+\t\t * It doesn't make sense to adjust a reference pointed\n+\t\t * to by a symbolic ref based on expiring entries in\n+\t\t * the symbolic reference's reflog. Nor can we update\n+\t\t * a reference if there are no remaining reflog\n+\t\t * entries.\n+\t\t */\n+\t\tint update = (flags & EXPIRE_REFLOGS_UPDATE_REF) &&\n+\t\t\t!(type & REF_ISSYMREF) &&\n+\t\t\t!is_null_sha1(cb.last_kept_sha1);\n+\n+\t\tif (mdb_transaction_commit(&main_transaction, &err)) {\n+\t\t\tstatus |= error(_(\"couldn't write logs/%s: %s\"),\n+\t\t\t\t\trefname, err.buf);\n+\t\t\tstrbuf_release(&err);\n+\t\t} else if (update && write_ref(refname, cb.last_kept_sha1)) {\n+\t\t\tstatus |= error(_(\"couldn't set %s\"),\n+\t\t\t\t\trefname);\n+\t\t}\n+\t}\n+\treturn status;\n+}\n+\n+static int lmdb_pack_refs(unsigned int flags)\n+{\n+\t/* This concept does not exist in this backend. */\n+\treturn 0;\n+}\n+\n+static int lmdb_peel_ref(const char *refname, unsigned char *sha1)\n+{\n+\tint flag;\n+\tunsigned char base[20];\n+\n+\tif (read_ref_full(refname, RESOLVE_REF_READING, base, &flag))\n+\t\treturn -1;\n+\n+\treturn peel_object(base, sha1);\n+}\n+\n+static int lmdb_create_symref(const char *ref_target,\n+\t\t\t const char *refs_heads_master,\n+\t\t\t const char *logmsg)\n+{\n+\n+\tstruct strbuf err = STRBUF_INIT;\n+\tunsigned char old_sha1[20], new_sha1[20];\n+\tMDB_val key, val;\n+\tchar *valdata;\n+\tint ret = 0;\n+\tint in_transaction;\n+\n+\tin_transaction = in_write_transaction();\n+\n+\tif (logmsg && read_ref(ref_target, old_sha1))\n+\t\thashclr(old_sha1);\n+\n+\tkey.mv_size = strlen(ref_target) + 1;\n+\tkey.mv_data = xstrdup(ref_target);\n+\n+\tvaldata = xstrfmt(\"ref: %s\", refs_heads_master);\n+\tval.mv_data = valdata;\n+\tval.mv_size = strlen(valdata) + 1;\n+\n+\tif (!in_transaction)\n+\t\tlmdb_transaction_begin_flags_or_die(0);\n+\n+\tmdb_put_or_die(&main_transaction, &key, &val, 0);\n+\n+\t/* TODO: Don't create ref d/f conflicts */\n+\n+\tif (logmsg && !read_ref(refs_heads_master, new_sha1) &&\n+\t log_ref_write(ref_target, old_sha1, new_sha1, logmsg, 0, &err)) {\n+\t\terror(_(\"create_symref: log_ref_write failed: %s\"), err.buf);\n+\t\tret = -1;\n+\t\tgoto done;\n+\t}\n+\n+\tif (!in_transaction && mdb_transaction_commit(&main_transaction, &err)) {\n+\t\terror(_(\"create_symref: commit failed: %s\"), err.buf);\n+\t\tret = -1;\n+\t}\n+\n+done:\n+\tstrbuf_release(&err);\n+\tfree(key.mv_data);\n+\tfree(valdata);\n+\n+\treturn ret;\n+}\n+\n+static int lmdb_resolve_gitlink_ref(const char *submodule, const char *refname,\n+\t\t\t\t unsigned char *sha1)\n+{\n+\tstatic struct strbuf sb_refname = STRBUF_INIT;\n+\tstruct strbuf sb_path = STRBUF_INIT;\n+\tconst char *ret;\n+\n+\tret = resolve_ref_unsafe_submodule(submodule, refname,\n+\t\t\t\t\t RESOLVE_REF_READING, sha1,\n+\t\t\t\t\t NULL, &sb_refname, &sb_path);\n+\tstrbuf_release(&sb_path);\n+\tstrbuf_release(&sb_refname);\n+\treturn ret ? 0 : -1;\n+}\n+\n+/*\n+ * Call fn for each reference for which the refname begins with base.\n+ * If trim is non-zero, then trim that many characters off the\n+ * beginning of each refname before passing the refname to fn. flags\n+ * can be DO_FOR_EACH_INCLUDE_BROKEN to include broken references in\n+ * the iteration. If fn ever returns a non-zero value, stop the\n+ * iteration and return that value; otherwise, return 0.\n+ */\n+static int lmdb_do_for_each_ref(const char *submodule, const char *base,\n+\t\t\t\teach_ref_fn fn, int trim, int flags,\n+\t\t\t\tvoid *cb_data)\n+{\n+\n+\tMDB_val key, val;\n+\tMDB_cursor *cursor;\n+\tint baselen;\n+\tchar *search_key;\n+\tint retval;\n+\tint mdb_ret;\n+\tstruct lmdb_transaction *transaction;\n+\n+\tretval = do_for_each_per_worktree_ref(submodule, base, fn,\n+\t\t\t\t\t trim, flags, cb_data);\n+\tif (retval)\n+\t\treturn retval;\n+\n+\tif (ref_paranoia < 0)\n+\t\tref_paranoia = git_env_bool(\"GIT_REF_PARANOIA\", 0);\n+\tif (ref_paranoia)\n+\t\tflags |= DO_FOR_EACH_INCLUDE_BROKEN;\n+\n+\tif (!base || !*base) {\n+\t\tbase = \"refs/\";\n+\t\ttrim = 0;\n+\t}\n+\n+\ttransaction = get_submodule_transaction(submodule);\n+\n+\tsearch_key = xstrdup(base);\n+\tbaselen = strlen(base);\n+\tkey.mv_size = baselen + 1;\n+\tkey.mv_data = search_key;\n+\n+\tmdb_cursor_open_or_die(transaction, &cursor);\n+\n+\tmdb_ret = mdb_cursor_get_or_die(cursor, &key, &val, MDB_SET_RANGE);\n+\n+\twhile (!mdb_ret) {\n+\t\tstruct object_id oid;\n+\t\tint parsed_flags = 0;\n+\t\tconst char *refname;\n+\t\tstruct strbuf sb_refname = STRBUF_INIT;\n+\t\tstruct strbuf sb_path = STRBUF_INIT;\n+\n+\t\tif (memcmp(key.mv_data, base, baselen))\n+\t\t\tbreak;\n+\n+\t\trefname = (const char *)key.mv_data;\n+\t\tresolve_ref_unsafe_submodule(submodule, refname, 0, oid.hash,\n+\t\t\t\t\t &parsed_flags, &sb_refname,\n+\t\t\t\t\t &sb_path);\n+\t\tif (!(parsed_flags & REF_ISBROKEN) && is_null_oid(&oid))\n+\t\t\tparsed_flags |= REF_ISBROKEN;\n+\n+\t\tstrbuf_release(&sb_path);\n+\t\tstrbuf_release(&sb_refname);\n+\n+\t\tif (flags & DO_FOR_EACH_INCLUDE_BROKEN ||\n+\t\t (!(parsed_flags & REF_ISBROKEN) &&\n+\t\t has_sha1_file(oid.hash))) {\n+\t\t\tretval = fn(refname + (trim ? baselen : 0), &oid, parsed_flags, cb_data);\n+\t\t\tif (retval)\n+\t\t\t\tbreak;\n+\t\t}\n+\n+\t\tmdb_ret = mdb_cursor_get_or_die(cursor, &key, &val, MDB_NEXT);\n+\t}\n+\n+\tmdb_cursor_close(cursor);\n+\tfree(search_key);\n+\n+\tif (submodule) {\n+\t\tMDB_env *env = mdb_txn_env(transaction->txn);\n+\t\tmdb_transaction_abort(transaction);\n+\t\tmdb_env_close(env);\n+\t}\n+\treturn retval;\n+}\n+\n+/* For testing only! */\n+int test_refdb_raw_read(const char *key)\n+{\n+\tMDB_val key_val, val;\n+\tchar *keydup;\n+\tint ret;\n+\tint needs_free = 0;\n+\n+\tlmdb_transaction_begin_flags_or_die(MDB_RDONLY);\n+\tkeydup = xstrdup(key);\n+\tkey_val.mv_data = keydup;\n+\tkey_val.mv_size = strlen(key) + 1;\n+\n+\tret = mdb_get_or_die(&main_transaction, &key_val, &val, &needs_free);\n+\tfree(keydup);\n+\tswitch (ret) {\n+\tcase 0:\n+\t\tprintf(\"%s\\n\", (char *)val.mv_data);\n+\t\treturn 0;\n+\tcase MDB_NOTFOUND:\n+\t\tfprintf(stderr, \"%s not found\\n\", key);\n+\t\treturn 1;\n+\tdefault:\n+\t\treturn 2;\n+\t}\n+\tif (needs_free)\n+\t\tfree(val.mv_data);\n+}\n+\n+/* For testing only! */\n+void test_refdb_raw_write(const char *key, const char *value)\n+{\n+\tMDB_val key_val, val;\n+\tchar *keydup, *valdup;\n+\n+\tif (ref_type(key) != REF_TYPE_NORMAL) {\n+\t\tval.mv_data = (void *)value;\n+\t\tval.mv_size = strlen(value) + 1;\n+\t\twrite_per_worktree_ref(NULL, key, &val);\n+\t\treturn;\n+\t}\n+\n+\tlmdb_transaction_begin_flags_or_die(0);\n+\n+\tkeydup = xstrdup(key);\n+\tkey_val.mv_data = keydup;\n+\tkey_val.mv_size = strlen(key) + 1;\n+\n+\tvaldup = xstrdup(value);\n+\tval.mv_data = valdup;\n+\tval.mv_size = strlen(value) + 1;\n+\n+\tmdb_put_or_die(&main_transaction, &key_val, &val, 0);\n+\tassert(mdb_transaction_commit(&main_transaction, NULL) == 0);\n+\n+\tfree(keydup);\n+\tfree(valdup);\n+}\n+\n+/* For testing only! */\n+int test_refdb_raw_delete(const char *key)\n+{\n+\tMDB_val key_val;\n+\tchar *keydup;\n+\tint ret;\n+\n+\tif (ref_type(key) != REF_TYPE_NORMAL)\n+\t\treturn del_per_worktree_ref(NULL, key, NULL);\n+\n+\tlmdb_transaction_begin_flags_or_die(0);\n+\tkeydup = xstrdup(key);\n+\tkey_val.mv_data = keydup;\n+\tkey_val.mv_size = strlen(key) + 1;\n+\n+\tret = mdb_del_or_die(&main_transaction, &key_val, NULL);\n+\n+\tassert(mdb_transaction_commit(&main_transaction, NULL) == 0);\n+\n+\tfree(keydup);\n+\treturn ret;\n+}\n+\n+static int print_raw_reflog_ent(unsigned char *osha1, unsigned char *nsha1,\n+\t\t\t\tconst char *email, unsigned long timestamp,\n+\t\t\t\tint tz, const char *message, void *cb_data)\n+{\n+\tint *any = cb_data;\n+\t*any = 1;\n+\n+\tif (*message != '\\n')\n+\t\tprintf(\"%s %s %s %lu %+05d\\t%s\", sha1_to_hex(osha1),\n+\t\t sha1_to_hex(nsha1),\n+\t\t email, timestamp, tz, message);\n+\telse\n+\t\tprintf(\"%s %s %s %lu %+05d\\n\", sha1_to_hex(osha1),\n+\t\t sha1_to_hex(nsha1),\n+\t\t email, timestamp, tz);\n+\treturn 0;\n+}\n+\n+/* For testing only! */\n+int test_refdb_raw_reflog(const char *refname)\n+{\n+\tint any = 0;\n+\n+\tfor_each_reflog_ent(refname, print_raw_reflog_ent, &any);\n+\n+\treturn !any;\n+}\n+\n+/* For testing only! */\n+void test_refdb_raw_delete_reflog(char *refname)\n+{\n+\tMDB_val key, val;\n+\tint mdb_ret;\n+\tchar *search_key;\n+\tMDB_cursor *cursor;\n+\tint len;\n+\n+\tsearch_key = xstrfmt(\"logs/%s\", refname ? refname : \"\");\n+\tlen = strlen(search_key) + 1;\n+\n+\tkey.mv_data = search_key;\n+\tkey.mv_size = len;\n+\n+\tlmdb_transaction_begin_flags_or_die(0);\n+\n+\tmdb_cursor_open_or_die(&main_transaction, &cursor);\n+\n+\tmdb_ret = mdb_cursor_get_or_die(cursor, &key, &val, MDB_SET_RANGE);\n+\twhile (!mdb_ret) {\n+\t\tif (!starts_with(key.mv_data, search_key))\n+\t\t\tbreak;\n+\t\tif (refname && ((char *)val.mv_data)[len - 1] == 0)\n+\t\t\tbreak;\n+\n+\t\tmdb_cursor_del_or_die(cursor, 0);\n+\t\tmdb_ret = mdb_cursor_get_or_die(cursor, &key, &val, MDB_NEXT);\n+\t}\n+\n+\tfree(search_key);\n+\tmdb_cursor_close(cursor);\n+\n+\tassert(mdb_transaction_commit(&main_transaction, NULL) == 0);\n+\treturn;\n+}\n+\n+static void format_lmdb_reflog_ent(struct strbuf *dst, struct strbuf *src)\n+{\n+\tunsigned char osha1[20], nsha1[20];\n+\tconst char *msg;\n+\n+\tget_sha1_hex(src->buf, osha1);\n+\tget_sha1_hex(src->buf + 41, nsha1);\n+\n+\tmsg = strchr(src->buf + 82, '\\t');\n+\tif (msg)\n+\t\tmsg += 1;\n+\n+\tformat_reflog_entry(dst, osha1, nsha1, src->buf + 82, msg);\n+}\n+\n+/* For testing only! */\n+void test_refdb_raw_append_reflog(const char *refname)\n+{\n+\tstruct strbuf input = STRBUF_INIT;\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tuint64_t now = getnanotime();\n+\tMDB_val key, val;\n+\tstruct strbuf key_buf = STRBUF_INIT;\n+\n+\tstrbuf_reflog_header(&key_buf, refname);\n+\tkey.mv_size = key_buf.len + 1;\n+\tkey.mv_data = key_buf.buf;\n+\n+\tlmdb_transaction_begin_flags_or_die(0);\n+\n+\t/* We do not remove the header entry here, because this is\n+\t * just for tests, so it's OK to be a bit inefficient */\n+\n+\twhile (strbuf_getline(&input, stdin) != EOF) {\n+\t\t/* \"logs/\" + \\0 + 8-byte timestamp for sorting and expiry */\n+\t\twrite_u64((char *)key.mv_data + key.mv_size - 8, htonll(now++));\n+\n+\t\t/*\n+\t\t * Convert the input from files-reflog format to\n+\t\t * lmdb-reflog-format\n+\t\t */\n+\n+\t\tformat_lmdb_reflog_ent(&sb, &input);\n+\t\tval.mv_data = sb.buf;\n+\t\tval.mv_size = sb.len + 1;\n+\t\tmdb_put_or_die(&main_transaction, &key, &val, 0);\n+\t\tstrbuf_reset(&sb);\n+\t\tinput.len = 0;\n+\t}\n+\n+\tstrbuf_release(&input);\n+\tstrbuf_release(&sb);\n+\tassert(mdb_transaction_commit(&main_transaction, NULL) == 0);\n+\tstrbuf_release(&key_buf);\n+}\n+\n+struct ref_storage_be refs_be_lmdb = {\n+\tNULL,\n+\t\"lmdb\",\n+\tlmdb_init_db,\n+\tlmdb_transaction_commit,\n+\tlmdb_transaction_commit, /* initial commit */\n+\n+\tlmdb_for_each_reflog_ent,\n+\tlmdb_for_each_reflog_ent_reverse,\n+\tlmdb_for_each_reflog,\n+\tlmdb_reflog_exists,\n+\tlmdb_create_reflog,\n+\tlmdb_delete_reflog,\n+\tlmdb_reflog_expire,\n+\n+\tlmdb_pack_refs,\n+\tlmdb_peel_ref,\n+\tlmdb_create_symref,\n+\tlmdb_delete_refs,\n+\tlmdb_rename_ref,\n+\n+\tlmdb_read_raw_ref,\n+\tlmdb_verify_refname_available,\n+\tlmdb_resolve_gitlink_ref,\n+\n+\tlmdb_do_for_each_ref,\n+};\ndiff --git a/test-refs-lmdb-backend.c b/test-refs-lmdb-backend.c\nnew file mode 100644\nindex 0000000..f1b18c3\n--- /dev/null\n+++ b/test-refs-lmdb-backend.c\n@@ -0,0 +1,66 @@\n+#include \"cache.h\"\n+#include \"string-list.h\"\n+#include \"parse-options.h\"\n+#include \"refs.h\"\n+#include \"refs/refs-internal.h\"\n+\n+static const char * const test_refs_be_lmdb_usage[] = {\n+\t\"git test-refs-lmdb-backend \",\n+\t\"git test-refs-lmdb-backend \",\n+\tNULL,\n+};\n+\n+int test_refdb_raw_read(const char *key);\n+void test_refdb_raw_write(const char *key, const char *value);\n+int test_refdb_raw_reflog(const char *refname);\n+int test_refdb_raw_delete(const char *key);\n+void test_refdb_raw_delete_reflog(const char *refname);\n+void test_refdb_raw_append_reflog(const char *refname);\n+\n+int main(int argc, const char **argv)\n+{\n+\tconst char *delete = NULL;\n+\tconst char *reflog = NULL;\n+\tconst char *append_reflog = NULL;\n+\tint delete_missing_error = 0;\n+\tint clear_reflog = 0;\n+\n+\tstruct option options[] = {\n+\t\tOPT_STRING('d', NULL, &delete, \"branch\", \"delete refdb entry\"),\n+\t\tOPT_STRING('l', NULL, &reflog, \"branch\", \"show reflog\"),\n+\t\tOPT_STRING('a', NULL, &append_reflog, \"branch\", \"append to reflog\"),\n+\t\tOPT_BOOL('c', NULL, &clear_reflog, \"delete reflog. If a branch is provided, the reflog for that branch will be deleted; else all reflogs will be deleted.\"),\n+\t\tOPT_BOOL('x', NULL, &delete_missing_error,\n+\t\t\t \"deleting a missing key is an error\"),\n+\t\tOPT_END(),\n+\t};\n+\n+\targc = parse_options(argc, argv, \"\", options, test_refs_be_lmdb_usage,\n+\t\t\t 0);\n+\n+\tif (!append_reflog && !clear_reflog && !delete && !reflog && argc != 1 && argc != 2)\n+\t\tusage_with_options(test_refs_be_lmdb_usage,\n+\t\t\t\t options);\n+\n+\tsetup_git_directory();\n+\tgit_config(git_default_config, NULL);\n+\n+\tif (set_ref_storage_backend(\"lmdb\"))\n+\t\tdie(\"could not set lmdb reference backend\");\n+\n+\tif (clear_reflog) {\n+\t\ttest_refdb_raw_delete_reflog(argv[0]);\n+\t} else if (append_reflog) {\n+\t\ttest_refdb_raw_append_reflog(append_reflog);\n+\t} else if (reflog) {\n+\t\treturn test_refdb_raw_reflog(reflog);\n+\t} else if (delete) {\n+\t\tif (test_refdb_raw_delete(delete) && delete_missing_error)\n+\t\t\treturn 1;\n+\t} else if (argc == 1) {\n+\t\treturn test_refdb_raw_read(argv[0]);\n+\t} else {\n+\t\ttest_refdb_raw_write(argv[0], argv[1]);\n+\t}\n+\treturn 0;\n+}\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279909","messageId":"1456793586-22082-33-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 32/33] refs: tests for lmdb backend","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:53:05Z","receivedAt":"2016-03-01T00:53:05Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"Add tests for the database backend.\n\nSigned-off-by: David Turner \nHelped-by: Dennis Kaarsemaker \nSigned-off-by: Junio C Hamano \n---\n t/t1460-refs-lmdb-backend.sh | 1109 +++++++++++++++++++++++++++++++++++\n t/t1470-refs-lmdb-backend-reflog.sh | 359 ++++++++++++\n t/t1480-refs-lmdb-submodule.sh | 85 +++\n t/test-lib.sh | 1 +\n 4 files changed, 1554 insertions(+)\n create mode 100755 t/t1460-refs-lmdb-backend.sh\n create mode 100755 t/t1470-refs-lmdb-backend-reflog.sh\n create mode 100755 t/t1480-refs-lmdb-submodule.sh\n\ndiff --git a/t/t1460-refs-lmdb-backend.sh b/t/t1460-refs-lmdb-backend.sh\nnew file mode 100755\nindex 0000000..31442e7\n--- /dev/null\n+++ b/t/t1460-refs-lmdb-backend.sh\n@@ -0,0 +1,1109 @@\n+#!/bin/sh\n+#\n+# Copyright (c) 2015 Twitter, Inc\n+# Copyright (c) 2006 Shawn Pearce\n+# This test is based on t1400-update-ref.sh\n+#\n+\n+test_description='Test lmdb refs backend'\n+TEST_NO_CREATE_REPO=1\n+. ./test-lib.sh\n+\n+if ! test_have_prereq LMDB\n+then\n+\tskip_all=\"Skipping lmdb refs backend tests, lmdb backend not built\"\n+\ttest_done\n+fi\n+\n+raw_ref() {\n+\ttest-refs-lmdb-backend \"$1\"\n+}\n+\n+delete_ref() {\n+\ttest-refs-lmdb-backend -d \"$1\"\n+}\n+\n+write_ref() {\n+\ttest-refs-lmdb-backend \"$1\" \"$2\"\n+}\n+\n+raw_reflog() {\n+\ttest-refs-lmdb-backend -l \"$1\"\n+}\n+\n+delete_all_reflogs() {\n+\ttest-refs-lmdb-backend -c\n+}\n+\n+append_reflog() {\n+\ttest-refs-lmdb-backend -a \"$1\"\n+}\n+\n+Z=$_z40\n+\n+test_expect_success setup '\n+\tgit init --ref-storage=lmdb &&\n+\tfor name in A B C D E F\n+\tdo\n+\t\ttest_tick &&\n+\t\tT=$(git write-tree) &&\n+\t\tsha1=$(echo $name | git commit-tree $T) &&\n+\t\teval $name=$sha1\n+\tdone\n+'\n+\n+m=refs/heads/master\n+n_dir=refs/heads/gu\n+n=$n_dir/fixes\n+\n+test_expect_success \\\n+\t\"create $m\" \\\n+\t\"git update-ref $m $A &&\n+\t test $A\"' = $(raw_ref '\"$m\"')'\n+test_expect_success \\\n+\t\"create $m\" \\\n+\t\"git update-ref $m $B $A &&\n+\t test $B\"' = $(raw_ref '\"$m\"')'\n+test_expect_success \"fail to delete $m with stale ref\" '\n+\ttest_must_fail git update-ref -d $m $A &&\n+\ttest $B = \"$(raw_ref $m)\"\n+'\n+test_expect_success \"delete $m\" '\n+\tgit update-ref -d $m $B &&\n+\t! raw_ref $m\n+'\n+delete_ref $m\n+\n+test_expect_success \"delete $m without oldvalue verification\" \"\n+\tgit update-ref $m $A &&\n+\ttest $A = \\$(raw_ref $m) &&\n+\tgit update-ref -d $m &&\n+\t! raw_ref $m\n+\"\n+delete_ref $m\n+\n+test_expect_success \\\n+\t\"fail to create $n\" \\\n+\t\"git update-ref $n_dir $A &&\n+\t test_must_fail git update-ref $n $A >out 2>err\"\n+\n+delete_ref $n_dir\n+rm -f out err\n+\n+test_expect_success \\\n+\t\"create $m (by HEAD)\" \\\n+\t\"git update-ref HEAD $A &&\n+\t test $A\"' = $(raw_ref '\"$m\"')'\n+test_expect_success \\\n+\t\"create $m (by HEAD)\" \\\n+\t\"git update-ref HEAD $B $A &&\n+\t test $B\"' = $(raw_ref '\"$m\"')'\n+test_expect_success \"fail to delete $m (by HEAD) with stale ref\" '\n+\ttest_must_fail git update-ref -d HEAD $A &&\n+\ttest $B = $(raw_ref '\"$m\"')\n+'\n+test_expect_success \"delete $m (by HEAD)\" '\n+\tgit update-ref -d HEAD $B &&\n+\t! raw_ref $m\n+'\n+delete_ref $m\n+\n+test_expect_success \\\n+\t\"create $m (by HEAD)\" \\\n+\t\"git update-ref HEAD $A &&\n+\t test $A\"' = $(raw_ref '\"$m\"')'\n+test_expect_success \\\n+\t\"pack refs\" \\\n+\t\"git pack-refs --all\"\n+test_expect_success \\\n+\t\"move $m (by HEAD)\" \\\n+\t\"git update-ref HEAD $B $A &&\n+\t test $B\"' = $(raw_ref '\"$m\"')'\n+test_expect_success \"delete $m (by HEAD) should remove both packed and loose $m\" '\n+\tgit update-ref -d HEAD $B &&\n+\t! raw_ref $m\n+'\n+delete_ref $m\n+\n+OLD_HEAD=$(raw_ref HEAD)\n+test_expect_success \"delete symref without dereference\" '\n+\tgit update-ref --no-deref -d HEAD &&\n+\t! raw_ref HEAD\n+'\n+write_ref HEAD \"$OLD_HEAD\"\n+\n+test_expect_success \"delete symref without dereference when the referred ref is packed\" '\n+\techo foo >foo.c &&\n+\tgit add foo.c &&\n+\tgit commit -m foo &&\n+\tgit pack-refs --all &&\n+\tgit update-ref --no-deref -d HEAD &&\n+\t! raw_ref HEAD\n+'\n+write_ref HEAD \"$OLD_HEAD\"\n+delete_ref $m\n+\n+test_expect_success 'update-ref -d is not confused by self-reference' '\n+\tgit symbolic-ref refs/heads/self refs/heads/self &&\n+\ttest_when_finished \"delete_ref refs/heads/self\" &&\n+\ttest_must_fail git update-ref -d refs/heads/self\n+'\n+\n+test_expect_success 'update-ref --no-deref -d can delete self-reference' '\n+\tgit symbolic-ref refs/heads/self refs/heads/self &&\n+\ttest_when_finished \"delete_ref refs/heads/self\" &&\n+\tgit update-ref --no-deref -d refs/heads/self\n+'\n+\n+test_expect_success 'update-ref --no-deref -d can delete reference to bad ref' '\n+\ttest-refs-lmdb-backend refs/heads/bad \"\" &&\n+\ttest_when_finished \"delete_ref refs/heads/bad\" &&\n+\tgit symbolic-ref refs/heads/ref-to-bad refs/heads/bad &&\n+\ttest_when_finished \"delete_ref refs/heads/ref-to-bad\" &&\n+\traw_ref refs/heads/ref-to-bad &&\n+\tgit update-ref --no-deref -d refs/heads/ref-to-bad &&\n+\t! raw_ref refs/heads/ref-to-bad\n+'\n+\n+test_expect_success '(not) create HEAD with old sha1' \"\n+\ttest_must_fail git update-ref HEAD $A $B\n+\"\n+test_expect_success \"(not) prior created .git/$m\" \"\n+\t! raw_ref $m\n+\"\n+delete_ref $m\n+\n+test_expect_success \\\n+\t\"create HEAD\" \\\n+\t\"git update-ref HEAD $A\"\n+test_expect_success '(not) change HEAD with wrong SHA1' \"\n+\ttest_must_fail git update-ref HEAD $B $Z\n+\"\n+test_expect_success \"(not) changed .git/$m\" \"\n+\t! test $B\"' = $(raw_ref '\"$m\"')\n+'\n+\n+: a repository with working tree always has reflog these days...\n+delete_all_reflogs\n+: | append_reflog $m\n+delete_ref $m\n+\n+test_expect_success \\\n+\t\"create $m (logged by touch)\" \\\n+\t'GIT_COMMITTER_DATE=\"2005-05-26 23:30\" \\\n+\t git update-ref HEAD '\"$A\"' -m \"Initial Creation\" &&\n+\t test '\"$A\"' = $(raw_ref '\"$m\"')'\n+test_expect_success \\\n+\t\"update $m (logged by touch)\" \\\n+\t'GIT_COMMITTER_DATE=\"2005-05-26 23:31\" \\\n+\t git update-ref HEAD'\" $B $A \"'-m \"Switch\" &&\n+\t test '\"$B\"' = $(raw_ref '\"$m\"')'\n+test_expect_success \\\n+\t\"set $m (logged by touch)\" \\\n+\t'GIT_COMMITTER_DATE=\"2005-05-26 23:41\" \\\n+\t git update-ref HEAD'\" $A &&\n+\t test $A\"' = $(raw_ref '\"$m\"')'\n+\n+cat >expect < 1117150200 +0000\tInitial Creation\n+$A $B $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150260 +0000\tSwitch\n+$B $A $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150860 +0000\n+EOF\n+test_expect_success \\\n+\t\"verifying $m's log\" \\\n+\t\"raw_reflog $m >actual &&\n+\t test_cmp expect actual\"\n+delete_ref $m\n+delete_all_reflogs\n+: | append_reflog $m\n+rm -f actual expect\n+\n+test_expect_success \\\n+\t'enable core.logAllRefUpdates' \\\n+\t'git config core.logAllRefUpdates true &&\n+\t test true = $(git config --bool --get core.logAllRefUpdates)'\n+\n+test_expect_success \\\n+\t\"create $m (logged by config)\" \\\n+\t'GIT_COMMITTER_DATE=\"2005-05-26 23:32\" \\\n+\t git update-ref HEAD'\" $A \"'-m \"Initial Creation\" &&\n+\t test '\"$A\"' = $(raw_ref '\"$m\"')'\n+test_expect_success \\\n+\t\"update $m (logged by config)\" \\\n+\t'GIT_COMMITTER_DATE=\"2005-05-26 23:33\" \\\n+\t git update-ref HEAD'\" $B $A \"'-m \"Switch\" &&\n+\t test '\"$B\"' = $(raw_ref '\"$m\"')'\n+test_expect_success \\\n+\t\"set $m (logged by config)\" \\\n+\t'GIT_COMMITTER_DATE=\"2005-05-26 23:43\" \\\n+\t git update-ref HEAD '\"$A &&\n+\t test $A\"' = $(raw_ref '\"$m\"')'\n+\n+cat >expect < 1117150320 +0000\tInitial Creation\n+$A $B $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150380 +0000\tSwitch\n+$B $A $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150980 +0000\n+EOF\n+test_expect_success \\\n+\t\"verifying $m's log\" \\\n+\t'raw_reflog $m >actual &&\n+\ttest_cmp expect actual'\n+delete_ref $m\n+rm -f expect\n+\n+git update-ref $m $D\n+git reflog expire --expire=all $m\n+\n+append_reflog $m < 1117150320 -0500\n+$C $A $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150350 -0500\n+$A $B $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150380 -0500\n+$F $Z $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150680 -0500\n+$Z $E $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150980 -0500\n+EOF\n+\n+ed=\"Thu, 26 May 2005 18:32:00 -0500\"\n+gd=\"Thu, 26 May 2005 18:33:00 -0500\"\n+ld=\"Thu, 26 May 2005 18:43:00 -0500\"\n+test_expect_success \\\n+\t'Query \"master@{May 25 2005}\" (before history)' \\\n+\t'rm -f o e &&\n+\t git rev-parse --verify \"master@{May 25 2005}\" >o 2>e &&\n+\t test '\"$C\"' = $(cat o) &&\n+\t test \"warning: Log for '\\'master\\'' only goes back to $ed.\" = \"$(cat e)\"'\n+test_expect_success \\\n+\t\"Query master@{2005-05-25} (before history)\" \\\n+\t'rm -f o e &&\n+\t git rev-parse --verify master@{2005-05-25} >o 2>e &&\n+\t test '\"$C\"' = $(cat o) &&\n+\t echo test \"warning: Log for '\\'master\\'' only goes back to $ed.\" = \"$(cat e)\"'\n+test_expect_success \\\n+\t'Query \"master@{May 26 2005 23:31:59}\" (1 second before history)' \\\n+\t'rm -f o e &&\n+\t git rev-parse --verify \"master@{May 26 2005 23:31:59}\" >o 2>e &&\n+\t test '\"$C\"' = $(cat o) &&\n+\t test \"warning: Log for '\\''master'\\'' only goes back to $ed.\" = \"$(cat e)\"'\n+test_expect_success \\\n+\t'Query \"master@{May 26 2005 23:32:00}\" (exactly history start)' \\\n+\t'rm -f o e &&\n+\t git rev-parse --verify \"master@{May 26 2005 23:32:00}\" >o 2>e &&\n+\t test '\"$C\"' = $(cat o) &&\n+\t test \"\" = \"$(cat e)\"'\n+test_expect_success \\\n+\t'Query \"master@{May 26 2005 23:32:30}\" (first non-creation change)' \\\n+\t'rm -f o e &&\n+\t git rev-parse --verify \"master@{May 26 2005 23:32:30}\" >o 2>e &&\n+\t test '\"$A\"' = $(cat o) &&\n+\t test \"\" = \"$(cat e)\"'\n+test_expect_success \\\n+\t'Query \"master@{2005-05-26 23:33:01}\" (middle of history with gap)' \\\n+\t'rm -f o e &&\n+\t git rev-parse --verify \"master@{2005-05-26 23:33:01}\" >o 2>e &&\n+\t test '\"$B\"' = $(cat o) &&\n+\t test \"warning: Log for ref '\"$m has gap after $gd\"'.\" = \"$(cat e)\"'\n+test_expect_success \\\n+\t'Query \"master@{2005-05-26 23:38:00}\" (middle of history)' \\\n+\t'rm -f o e &&\n+\t git rev-parse --verify \"master@{2005-05-26 23:38:00}\" >o 2>e &&\n+\t test '\"$Z\"' = $(cat o) &&\n+\t test \"\" = \"$(cat e)\"'\n+test_expect_success \\\n+\t'Query \"master@{2005-05-26 23:43:00}\" (exact end of history)' \\\n+\t'rm -f o e &&\n+\t git rev-parse --verify \"master@{2005-05-26 23:43:00}\" >o 2>e &&\n+\t test '\"$E\"' = $(cat o) &&\n+\t test \"\" = \"$(cat e)\"'\n+test_expect_success \\\n+\t'Query \"master@{2005-05-28}\" (past end of history)' \\\n+\t'rm -f o e &&\n+\t git rev-parse --verify \"master@{2005-05-28}\" >o 2>e &&\n+\t test '\"$D\"' = $(cat o) &&\n+\t test \"warning: Log for ref '\"$m unexpectedly ended on $ld\"'.\" = \"$(cat e)\"'\n+\n+\n+git reflog expire --expire=all $m\n+delete_ref $m\n+\n+test_expect_success \\\n+ 'creating initial files' \\\n+ 'echo TEST >F &&\n+ git add F &&\n+\t GIT_AUTHOR_DATE=\"2005-05-26 23:30\" \\\n+\t GIT_COMMITTER_DATE=\"2005-05-26 23:30\" git commit -m add -a &&\n+\t h_TEST=$(git rev-parse --verify HEAD) &&\n+\t echo The other day this did not work. >M &&\n+\t echo And then Bob told me how to fix it. >>M &&\n+\t echo OTHER >F &&\n+\t GIT_AUTHOR_DATE=\"2005-05-26 23:41\" \\\n+\t GIT_COMMITTER_DATE=\"2005-05-26 23:41\" git commit -F M -a &&\n+\t h_OTHER=$(git rev-parse --verify HEAD) &&\n+\t GIT_AUTHOR_DATE=\"2005-05-26 23:44\" \\\n+\t GIT_COMMITTER_DATE=\"2005-05-26 23:44\" git commit --amend &&\n+\t h_FIXED=$(git rev-parse --verify HEAD) &&\n+\t echo Merged initial commit and a later commit. >M &&\n+\t echo $h_TEST >.git/MERGE_HEAD &&\n+\t GIT_AUTHOR_DATE=\"2005-05-26 23:45\" \\\n+\t GIT_COMMITTER_DATE=\"2005-05-26 23:45\" git commit -F M &&\n+\t h_MERGED=$(git rev-parse --verify HEAD) &&\n+\t rm -f M'\n+\n+cat >expect < 1117150200 +0000\tcommit (initial): add\n+$h_TEST $h_OTHER $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150860 +0000\tcommit: The other day this did not work.\n+$h_OTHER $h_FIXED $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117151040 +0000\tcommit (amend): The other day this did not work.\n+$h_FIXED $h_MERGED $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117151100 +0000\tcommit (merge): Merged initial commit and a later commit.\n+EOF\n+test_expect_success \\\n+\t'git commit logged updates' \\\n+\t\"raw_reflog $m >actual &&\n+\ttest_cmp expect actual\"\n+unset h_TEST h_OTHER h_FIXED h_MERGED\n+\n+test_expect_success \\\n+\t'git cat-file blob master:F (expect OTHER)' \\\n+\t'test OTHER = $(git cat-file blob master:F)'\n+test_expect_success \\\n+\t'git cat-file blob master@{2005-05-26 23:30}:F (expect TEST)' \\\n+\t'test TEST = $(git cat-file blob \"master@{2005-05-26 23:30}:F\")'\n+test_expect_success \\\n+\t'git cat-file blob master@{2005-05-26 23:42}:F (expect OTHER)' \\\n+\t'test OTHER = $(git cat-file blob \"master@{2005-05-26 23:42}:F\")'\n+\n+a=refs/heads/a\n+b=refs/heads/b\n+c=refs/heads/c\n+E='\"\"'\n+F='%s\\0'\n+pws='path with space'\n+\n+test_expect_success 'stdin test setup' '\n+\techo \"$pws\" >\"$pws\" &&\n+\tgit add -- \"$pws\" &&\n+\tgit commit -m \"$pws\"\n+'\n+\n+test_expect_success '-z fails without --stdin' '\n+\ttest_must_fail git update-ref -z $m $m $m 2>err &&\n+\tgrep \"usage: git update-ref\" err\n+'\n+\n+test_expect_success 'stdin works with no input' '\n+\t>stdin &&\n+\tgit update-ref --stdin stdin &&\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: empty command in input\" err\n+'\n+\n+test_expect_success 'stdin fails on only whitespace' '\n+\techo \" \" >stdin &&\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: whitespace before command: \" err\n+'\n+\n+test_expect_success 'stdin fails on leading whitespace' '\n+\techo \" create $a $m\" >stdin &&\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: whitespace before command: create $a $m\" err\n+'\n+\n+test_expect_success 'stdin fails on unknown command' '\n+\techo \"unknown $a\" >stdin &&\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: unknown command: unknown $a\" err\n+'\n+\n+test_expect_success 'stdin fails on unbalanced quotes' '\n+\techo \"create $a \\\"master\" >stdin &&\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: badly quoted argument: \\\\\\\"master\" err\n+'\n+\n+test_expect_success 'stdin fails on invalid escape' '\n+\techo \"create $a \\\"ma\\zter\\\"\" >stdin &&\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: badly quoted argument: \\\\\\\"ma\\\\\\\\zter\\\\\\\"\" err\n+'\n+\n+test_expect_success 'stdin fails on junk after quoted argument' '\n+\techo \"create \\\"$a\\\"master\" >stdin &&\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: unexpected character after quoted argument: \\\\\\\"$a\\\\\\\"master\" err\n+'\n+\n+test_expect_success 'stdin fails create with no ref' '\n+\techo \"create \" >stdin &&\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: create: missing \" err\n+'\n+\n+test_expect_success 'stdin fails create with no new value' '\n+\techo \"create $a\" >stdin &&\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: create $a: missing \" err\n+'\n+\n+test_expect_success 'stdin fails create with too many arguments' '\n+\techo \"create $a $m $m\" >stdin &&\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: create $a: extra input: $m\" err\n+'\n+\n+test_expect_success 'stdin fails update with no ref' '\n+\techo \"update \" >stdin &&\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: update: missing \" err\n+'\n+\n+test_expect_success 'stdin fails update with no new value' '\n+\techo \"update $a\" >stdin &&\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: update $a: missing \" err\n+'\n+\n+test_expect_success 'stdin fails update with too many arguments' '\n+\techo \"update $a $m $m $m\" >stdin &&\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: update $a: extra input: $m\" err\n+'\n+\n+test_expect_success 'stdin fails delete with no ref' '\n+\techo \"delete \" >stdin &&\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: delete: missing \" err\n+'\n+\n+test_expect_success 'stdin fails delete with too many arguments' '\n+\techo \"delete $a $m $m\" >stdin &&\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: delete $a: extra input: $m\" err\n+'\n+\n+test_expect_success 'stdin fails verify with too many arguments' '\n+\techo \"verify $a $m $m\" >stdin &&\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: verify $a: extra input: $m\" err\n+'\n+\n+test_expect_success 'stdin fails option with unknown name' '\n+\techo \"option unknown\" >stdin &&\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: option unknown: unknown\" err\n+'\n+\n+test_expect_success 'stdin fails with duplicate refs' '\n+\tcat >stdin <<-EOF &&\n+\tcreate $a $m\n+\tcreate $b $m\n+\tcreate $a $m\n+\tEOF\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: Multiple updates for ref '\"'\"'$a'\"'\"' not allowed.\" err\n+'\n+\n+test_expect_success 'stdin create ref works' '\n+\techo \"create $a $m\" >stdin &&\n+\tgit update-ref --stdin expect &&\n+\tgit rev-parse $a >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin succeeds with quoted argument' '\n+\tgit update-ref -d $a &&\n+\techo \"create $a \\\"$m\\\"\" >stdin &&\n+\tgit update-ref --stdin expect &&\n+\tgit rev-parse $a >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin succeeds with escaped character' '\n+\tgit update-ref -d $a &&\n+\techo \"create $a \\\"ma\\\\163ter\\\"\" >stdin &&\n+\tgit update-ref --stdin expect &&\n+\tgit rev-parse $a >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin update ref creates with zero old value' '\n+\techo \"update $b $m $Z\" >stdin &&\n+\tgit update-ref --stdin expect &&\n+\tgit rev-parse $b >actual &&\n+\ttest_cmp expect actual &&\n+\tgit update-ref -d $b\n+'\n+\n+test_expect_success 'stdin update ref creates with empty old value' '\n+\techo \"update $b $m $E\" >stdin &&\n+\tgit update-ref --stdin expect &&\n+\tgit rev-parse $b >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin create ref works with path with space to blob' '\n+\techo \"create refs/blobs/pws \\\"$m:$pws\\\"\" >stdin &&\n+\tgit update-ref --stdin expect &&\n+\tgit rev-parse refs/blobs/pws >actual &&\n+\ttest_cmp expect actual &&\n+\tgit update-ref -d refs/blobs/pws\n+'\n+\n+test_expect_success 'stdin update ref fails with wrong old value' '\n+\techo \"update $c $m $m~1\" >stdin &&\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: cannot lock the ref '\"'\"'$c'\"'\"'\" err &&\n+\ttest_must_fail git rev-parse --verify -q $c\n+'\n+\n+test_expect_success 'stdin update ref fails with bad old value' '\n+\techo \"update $c $m does-not-exist\" >stdin &&\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: update $c: invalid : does-not-exist\" err &&\n+\ttest_must_fail git rev-parse --verify -q $c\n+'\n+\n+test_expect_success 'stdin create ref fails with bad new value' '\n+\techo \"create $c does-not-exist\" >stdin &&\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: create $c: invalid : does-not-exist\" err &&\n+\ttest_must_fail git rev-parse --verify -q $c\n+'\n+\n+test_expect_success 'stdin create ref fails with zero new value' '\n+\techo \"create $c \" >stdin &&\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: create $c: zero \" err &&\n+\ttest_must_fail git rev-parse --verify -q $c\n+'\n+\n+test_expect_success 'stdin update ref works with right old value' '\n+\techo \"update $b $m~1 $m\" >stdin &&\n+\tgit update-ref --stdin expect &&\n+\tgit rev-parse $b >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin delete ref fails with wrong old value' '\n+\techo \"delete $a $m~1\" >stdin &&\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: cannot lock the ref '\"'\"'$a'\"'\"'\" err &&\n+\tgit rev-parse $m >expect &&\n+\tgit rev-parse $a >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin delete ref fails with zero old value' '\n+\techo \"delete $a \" >stdin &&\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: delete $a: zero \" err &&\n+\tgit rev-parse $m >expect &&\n+\tgit rev-parse $a >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin update symref works option no-deref' '\n+\tgit symbolic-ref refs/TESTSYMREF $b &&\n+\tcat >stdin <<-EOF &&\n+\toption no-deref\n+\tupdate refs/TESTSYMREF $a $b\n+\tEOF\n+\tgit update-ref --stdin expect &&\n+\tgit rev-parse $a >actual &&\n+\ttest_cmp expect actual &&\n+\tgit rev-parse $m~1 >expect &&\n+\tgit rev-parse $b >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin delete symref works option no-deref' '\n+\tgit symbolic-ref refs/TESTSYMREF $b &&\n+\tcat >stdin <<-EOF &&\n+\toption no-deref\n+\tdelete refs/TESTSYMREF $b\n+\tEOF\n+\tgit update-ref --stdin expect &&\n+\tgit rev-parse $b >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin delete ref works with right old value' '\n+\techo \"delete $b $m~1\" >stdin &&\n+\tgit update-ref --stdin stdin <<-EOF &&\n+\tupdate $a $m\n+\tcreate $b $m\n+\tverify $c\n+\tEOF\n+\tgit update-ref --stdin expect &&\n+\tgit rev-parse $a >actual &&\n+\ttest_cmp expect actual &&\n+\tgit rev-parse $b >actual &&\n+\ttest_cmp expect actual &&\n+\ttest_must_fail git rev-parse --verify -q $c\n+'\n+\n+test_expect_success 'stdin verify succeeds for correct value' '\n+\tgit rev-parse $m >expect &&\n+\techo \"verify $m $m\" >stdin &&\n+\tgit update-ref --stdin actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin verify succeeds for missing reference' '\n+\techo \"verify refs/heads/missing $Z\" >stdin &&\n+\tgit update-ref --stdin stdin &&\n+\tgit update-ref --stdin expect &&\n+\techo \"verify $m $m~1\" >stdin &&\n+\ttest_must_fail git update-ref --stdin actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin verify fails for mistaken null value' '\n+\tgit rev-parse $m >expect &&\n+\techo \"verify $m $Z\" >stdin &&\n+\ttest_must_fail git update-ref --stdin actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin verify fails for mistaken empty value' '\n+\tM=$(git rev-parse $m) &&\n+\ttest_when_finished \"git update-ref $m $M\" &&\n+\tgit rev-parse $m >expect &&\n+\techo \"verify $m\" >stdin &&\n+\ttest_must_fail git update-ref --stdin actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin update refs works with identity updates' '\n+\tcat >stdin <<-EOF &&\n+\tupdate $a $m $m\n+\tupdate $b $m $m\n+\tupdate $c $Z $E\n+\tEOF\n+\tgit update-ref --stdin expect &&\n+\tgit rev-parse $a >actual &&\n+\ttest_cmp expect actual &&\n+\tgit rev-parse $b >actual &&\n+\ttest_cmp expect actual &&\n+\ttest_must_fail git rev-parse --verify -q $c\n+'\n+\n+test_expect_success 'stdin update refs fails with wrong old value' '\n+\tgit update-ref $c $m &&\n+\tcat >stdin <<-EOF &&\n+\tupdate $a $m $m\n+\tupdate $b $m $m\n+\tupdate $c ''\n+\tEOF\n+\ttest_must_fail git update-ref --stdin err &&\n+\tgrep \"fatal: cannot lock the ref '\"'\"'$c'\"'\"'\" err &&\n+\tgit rev-parse $m >expect &&\n+\tgit rev-parse $a >actual &&\n+\ttest_cmp expect actual &&\n+\tgit rev-parse $b >actual &&\n+\ttest_cmp expect actual &&\n+\tgit rev-parse $c >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin delete refs works with packed and loose refs' '\n+\tgit pack-refs --all &&\n+\tgit update-ref $c $m~1 &&\n+\tcat >stdin <<-EOF &&\n+\tdelete $a $m\n+\tupdate $b $Z $m\n+\tupdate $c $E $m~1\n+\tEOF\n+\tgit update-ref --stdin stdin &&\n+\tgit update-ref -z --stdin stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: whitespace before command: \" err\n+'\n+\n+test_expect_success 'stdin -z fails on empty command' '\n+\tprintf $F \"\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: empty command in input\" err\n+'\n+\n+test_expect_success 'stdin -z fails on only whitespace' '\n+\tprintf $F \" \" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: whitespace before command: \" err\n+'\n+\n+test_expect_success 'stdin -z fails on leading whitespace' '\n+\tprintf $F \" create $a\" \"$m\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: whitespace before command: create $a\" err\n+'\n+\n+test_expect_success 'stdin -z fails on unknown command' '\n+\tprintf $F \"unknown $a\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: unknown command: unknown $a\" err\n+'\n+\n+test_expect_success 'stdin -z fails create with no ref' '\n+\tprintf $F \"create \" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: create: missing \" err\n+'\n+\n+test_expect_success 'stdin -z fails create with no new value' '\n+\tprintf $F \"create $a\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: create $a: unexpected end of input when reading \" err\n+'\n+\n+test_expect_success 'stdin -z fails create with too many arguments' '\n+\tprintf $F \"create $a\" \"$m\" \"$m\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: unknown command: $m\" err\n+'\n+\n+test_expect_success 'stdin -z fails update with no ref' '\n+\tprintf $F \"update \" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: update: missing \" err\n+'\n+\n+test_expect_success 'stdin -z fails update with too few args' '\n+\tprintf $F \"update $a\" \"$m\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: update $a: unexpected end of input when reading \" err\n+'\n+\n+test_expect_success 'stdin -z emits warning with empty new value' '\n+\tgit update-ref $a $m &&\n+\tprintf $F \"update $a\" \"\" \"\" >stdin &&\n+\tgit update-ref -z --stdin err &&\n+\tgrep \"warning: update $a: missing , treating as zero\" err &&\n+\ttest_must_fail git rev-parse --verify -q $a\n+'\n+\n+test_expect_success 'stdin -z fails update with no new value' '\n+\tprintf $F \"update $a\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: update $a: unexpected end of input when reading \" err\n+'\n+\n+test_expect_success 'stdin -z fails update with no old value' '\n+\tprintf $F \"update $a\" \"$m\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: update $a: unexpected end of input when reading \" err\n+'\n+\n+test_expect_success 'stdin -z fails update with too many arguments' '\n+\tprintf $F \"update $m\" \"$m\" \"$m\" \"$m\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: unknown command: $m\" err\n+'\n+\n+test_expect_success 'stdin -z fails delete with no ref' '\n+\tprintf $F \"delete \" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: delete: missing \" err\n+'\n+\n+test_expect_success 'stdin -z fails delete with no old value' '\n+\tprintf $F \"delete $a\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: delete $a: unexpected end of input when reading \" err\n+'\n+\n+test_expect_success 'stdin -z fails delete with too many arguments' '\n+\tprintf $F \"delete $m\" \"$m\" \"$m\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: unknown command: $m\" err\n+'\n+\n+test_expect_success 'stdin -z fails verify with too many arguments' '\n+\tprintf $F \"verify $m\" \"$m\" \"$m\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: unknown command: $m\" err\n+'\n+\n+test_expect_success 'stdin -z fails verify with no old value' '\n+\tprintf $F \"verify $a\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: verify $a: unexpected end of input when reading \" err\n+'\n+\n+test_expect_success 'stdin -z fails option with unknown name' '\n+\tprintf $F \"option unknown\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: option unknown: unknown\" err\n+'\n+\n+test_expect_success 'stdin -z fails with duplicate refs' '\n+\tprintf $F \"create $a\" \"$m\" \"create $b\" \"$m\" \"create $a\" \"$m\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: Multiple updates for ref '\"'\"'$a'\"'\"' not allowed.\" err\n+'\n+\n+test_expect_success 'stdin -z create ref works' '\n+\tprintf $F \"create $a\" \"$m\" >stdin &&\n+\tgit update-ref -z --stdin expect &&\n+\tgit rev-parse $a >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin -z update ref creates with zero old value' '\n+\tprintf $F \"update $b\" \"$m\" \"$Z\" >stdin &&\n+\tgit update-ref -z --stdin expect &&\n+\tgit rev-parse $b >actual &&\n+\ttest_cmp expect actual &&\n+\tgit update-ref -d $b\n+'\n+\n+test_expect_success 'stdin -z update ref creates with empty old value' '\n+\tprintf $F \"update $b\" \"$m\" \"\" >stdin &&\n+\tgit update-ref -z --stdin expect &&\n+\tgit rev-parse $b >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin -z create ref works with path with space to blob' '\n+\tprintf $F \"create refs/blobs/pws\" \"$m:$pws\" >stdin &&\n+\tgit update-ref -z --stdin expect &&\n+\tgit rev-parse refs/blobs/pws >actual &&\n+\ttest_cmp expect actual &&\n+\tgit update-ref -d refs/blobs/pws\n+'\n+\n+test_expect_success 'stdin -z update ref fails with wrong old value' '\n+\tprintf $F \"update $c\" \"$m\" \"$m~1\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: cannot lock the ref '\"'\"'$c'\"'\"'\" err &&\n+\ttest_must_fail git rev-parse --verify -q $c\n+'\n+\n+test_expect_success 'stdin -z update ref fails with bad old value' '\n+\tprintf $F \"update $c\" \"$m\" \"does-not-exist\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: update $c: invalid : does-not-exist\" err &&\n+\ttest_must_fail git rev-parse --verify -q $c\n+'\n+\n+test_expect_success 'stdin -z create ref fails when ref exists' '\n+\tgit update-ref $c $m &&\n+\tgit rev-parse \"$c\" >expect &&\n+\tprintf $F \"create $c\" \"$m~1\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: cannot lock the ref '\"'\"'$c'\"'\"'\" err &&\n+\tgit rev-parse \"$c\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin -z create ref fails with bad new value' '\n+\tgit update-ref -d \"$c\" &&\n+\tprintf $F \"create $c\" \"does-not-exist\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: create $c: invalid : does-not-exist\" err &&\n+\ttest_must_fail git rev-parse --verify -q $c\n+'\n+\n+test_expect_success 'stdin -z create ref fails with empty new value' '\n+\tprintf $F \"create $c\" \"\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: create $c: missing \" err &&\n+\ttest_must_fail git rev-parse --verify -q $c\n+'\n+\n+test_expect_success 'stdin -z update ref works with right old value' '\n+\tprintf $F \"update $b\" \"$m~1\" \"$m\" >stdin &&\n+\tgit update-ref -z --stdin expect &&\n+\tgit rev-parse $b >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin -z delete ref fails with wrong old value' '\n+\tprintf $F \"delete $a\" \"$m~1\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: cannot lock the ref '\"'\"'$a'\"'\"'\" err &&\n+\tgit rev-parse $m >expect &&\n+\tgit rev-parse $a >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin -z delete ref fails with zero old value' '\n+\tprintf $F \"delete $a\" \"$Z\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: delete $a: zero \" err &&\n+\tgit rev-parse $m >expect &&\n+\tgit rev-parse $a >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin -z update symref works option no-deref' '\n+\tgit symbolic-ref refs/TESTSYMREF $b &&\n+\tprintf $F \"option no-deref\" \"update refs/TESTSYMREF\" \"$a\" \"$b\" >stdin &&\n+\tgit update-ref -z --stdin expect &&\n+\tgit rev-parse $a >actual &&\n+\ttest_cmp expect actual &&\n+\tgit rev-parse $m~1 >expect &&\n+\tgit rev-parse $b >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin -z delete symref works option no-deref' '\n+\tgit symbolic-ref refs/TESTSYMREF $b &&\n+\tprintf $F \"option no-deref\" \"delete refs/TESTSYMREF\" \"$b\" >stdin &&\n+\tgit update-ref -z --stdin expect &&\n+\tgit rev-parse $b >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin -z delete ref works with right old value' '\n+\tprintf $F \"delete $b\" \"$m~1\" >stdin &&\n+\tgit update-ref -z --stdin stdin &&\n+\tgit update-ref -z --stdin expect &&\n+\tgit rev-parse $a >actual &&\n+\ttest_cmp expect actual &&\n+\tgit rev-parse $b >actual &&\n+\ttest_cmp expect actual &&\n+\ttest_must_fail git rev-parse --verify -q $c\n+'\n+\n+test_expect_success 'stdin -z verify succeeds for correct value' '\n+\tgit rev-parse $m >expect &&\n+\tprintf $F \"verify $m\" \"$m\" >stdin &&\n+\tgit update-ref -z --stdin actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin -z verify succeeds for missing reference' '\n+\tprintf $F \"verify refs/heads/missing\" \"$Z\" >stdin &&\n+\tgit update-ref -z --stdin stdin &&\n+\tgit update-ref -z --stdin expect &&\n+\tprintf $F \"verify $m\" \"$m~1\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin -z verify fails for mistaken null value' '\n+\tgit rev-parse $m >expect &&\n+\tprintf $F \"verify $m\" \"$Z\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin -z verify fails for mistaken empty value' '\n+\tM=$(git rev-parse $m) &&\n+\ttest_when_finished \"git update-ref $m $M\" &&\n+\tgit rev-parse $m >expect &&\n+\tprintf $F \"verify $m\" \"\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin -z update refs works with identity updates' '\n+\tprintf $F \"update $a\" \"$m\" \"$m\" \"update $b\" \"$m\" \"$m\" \"update $c\" \"$Z\" \"\" >stdin &&\n+\tgit update-ref -z --stdin expect &&\n+\tgit rev-parse $a >actual &&\n+\ttest_cmp expect actual &&\n+\tgit rev-parse $b >actual &&\n+\ttest_cmp expect actual &&\n+\ttest_must_fail git rev-parse --verify -q $c\n+'\n+\n+test_expect_success 'stdin -z update refs fails with wrong old value' '\n+\tgit update-ref $c $m &&\n+\tprintf $F \"update $a\" \"$m\" \"$m\" \"update $b\" \"$m\" \"$m\" \"update $c\" \"$m\" \"$Z\" >stdin &&\n+\ttest_must_fail git update-ref -z --stdin err &&\n+\tgrep \"fatal: cannot lock the ref '\"'\"'$c'\"'\"'\" err &&\n+\tgit rev-parse $m >expect &&\n+\tgit rev-parse $a >actual &&\n+\ttest_cmp expect actual &&\n+\tgit rev-parse $b >actual &&\n+\ttest_cmp expect actual &&\n+\tgit rev-parse $c >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stdin -z delete refs works with packed and loose refs' '\n+\tgit pack-refs --all &&\n+\tgit update-ref $c $m~1 &&\n+\tprintf $F \"delete $a\" \"$m\" \"update $b\" \"$Z\" \"$m\" \"update $c\" \"\" \"$m~1\" >stdin &&\n+\tgit update-ref -z --stdin /dev/null || test-refs-lmdb-backend -l \"$1\"\n+}\n+\n+append_reflog() {\n+\ttest-refs-lmdb-backend -a \"$1\"\n+}\n+\n+check_have () {\n+\tgaah= &&\n+\tfor N in \"$@\"\n+\tdo\n+\t\teval \"o=\\$$N\" && git cat-file -t $o || {\n+\t\t\techo Gaah $N\n+\t\t\tgaah=$N\n+\t\t\tbreak\n+\t\t}\n+\tdone &&\n+\ttest -z \"$gaah\"\n+}\n+\n+check_fsck () {\n+\toutput=$(git fsck --full)\n+\tcase \"$1\" in\n+\t'')\n+\t\ttest -z \"$output\" ;;\n+\t*)\n+\t\techo \"$output\" | grep \"$1\" ;;\n+\tesac\n+}\n+\n+corrupt () {\n+\taa=${1%??????????????????????????????????????} zz=${1#??}\n+\tmv .git/objects/$aa/$zz .git/$aa$zz\n+}\n+\n+recover () {\n+\taa=${1%??????????????????????????????????????} zz=${1#??}\n+\tmkdir -p .git/objects/$aa\n+\tmv .git/$aa$zz .git/objects/$aa/$zz\n+}\n+\n+check_dont_have () {\n+\tgaah= &&\n+\tfor N in \"$@\"\n+\tdo\n+\t\teval \"o=\\$$N\"\n+\t\tgit cat-file -t $o && {\n+\t\t\techo Gaah $N\n+\t\t\tgaah=$N\n+\t\t\tbreak\n+\t\t}\n+\tdone\n+\ttest -z \"$gaah\"\n+}\n+\n+test_expect_success setup '\n+\tgit init --ref-storage=lmdb &&\n+\tmkdir -p A/B &&\n+\techo rat >C &&\n+\techo ox >A/D &&\n+\techo tiger >A/B/E &&\n+\tgit add . &&\n+\n+\ttest_tick && git commit -m rabbit &&\n+\tH=`git rev-parse --verify HEAD` &&\n+\tA=`git rev-parse --verify HEAD:A` &&\n+\tB=`git rev-parse --verify HEAD:A/B` &&\n+\tC=`git rev-parse --verify HEAD:C` &&\n+\tD=`git rev-parse --verify HEAD:A/D` &&\n+\tE=`git rev-parse --verify HEAD:A/B/E` &&\n+\tcheck_fsck &&\n+\n+\ttest_chmod +x C &&\n+\tgit add C &&\n+\ttest_tick && git commit -m dragon &&\n+\tL=`git rev-parse --verify HEAD` &&\n+\tcheck_fsck &&\n+\n+\trm -f C A/B/E &&\n+\techo snake >F &&\n+\techo horse >A/G &&\n+\tgit add F A/G &&\n+\ttest_tick && git commit -a -m sheep &&\n+\tF=`git rev-parse --verify HEAD:F` &&\n+\tG=`git rev-parse --verify HEAD:A/G` &&\n+\tI=`git rev-parse --verify HEAD:A` &&\n+\tJ=`git rev-parse --verify HEAD` &&\n+\tcheck_fsck &&\n+\n+\trm -f A/G &&\n+\ttest_tick && git commit -a -m monkey &&\n+\tK=`git rev-parse --verify HEAD` &&\n+\tcheck_fsck &&\n+\n+\tcheck_have A B C D E F G H I J K L &&\n+\n+\tgit prune &&\n+\n+\tcheck_have A B C D E F G H I J K L &&\n+\n+\tcheck_fsck &&\n+\n+\traw_reflog refs/heads/master >reflog &&\n+\ttest_when_finished rm -f reflog &&\n+\ttest_line_count = 4 reflog\n+'\n+\n+test_expect_success rewind '\n+\ttest_tick && git reset --hard HEAD~2 &&\n+\ttest -f C &&\n+\ttest -f A/B/E &&\n+\t! test -f F &&\n+\t! test -f A/G &&\n+\n+\tcheck_have A B C D E F G H I J K L &&\n+\n+\tgit prune &&\n+\n+\tcheck_have A B C D E F G H I J K L &&\n+\n+\traw_reflog refs/heads/master >reflog &&\n+\ttest_when_finished rm -f reflog &&\n+\ttest_line_count = 5 reflog\n+'\n+\n+test_expect_success 'corrupt and check' '\n+\n+\tcorrupt $F &&\n+\tcheck_fsck \"missing blob $F\"\n+\n+'\n+\n+test_expect_success 'reflog expire --dry-run should not touch reflog' '\n+\n+\tgit reflog expire --dry-run \\\n+\t\t--expire=$(($test_tick - 10000)) \\\n+\t\t--expire-unreachable=$(($test_tick - 10000)) \\\n+\t\t--stale-fix \\\n+\t\t--all &&\n+\n+\traw_reflog refs/heads/master >reflog &&\n+\ttest_when_finished rm -f reflog &&\n+\ttest_line_count = 5 reflog &&\n+\n+\tcheck_fsck \"missing blob $F\"\n+'\n+\n+test_expect_success 'reflog expire' '\n+\n+\tgit reflog expire --verbose \\\n+\t\t--expire=$(($test_tick - 10000)) \\\n+\t\t--expire-unreachable=$(($test_tick - 10000)) \\\n+\t\t--stale-fix \\\n+\t\t--all &&\n+\n+\techo git reflog expire --verbose \\\n+\t\t--expire=$(($test_tick - 10000)) \\\n+\t\t--expire-unreachable=$(($test_tick - 10000)) \\\n+\t\t--stale-fix \\\n+\t\t--all &&\n+\n+\traw_reflog refs/heads/master >reflog &&\n+\ttest_when_finished rm -f reflog &&\n+\ttest_line_count = 2 reflog &&\n+\n+\tcheck_fsck \"dangling commit $K\"\n+'\n+\n+test_expect_success 'prune and fsck' '\n+\n+\tgit prune &&\n+\tcheck_fsck &&\n+\n+\tcheck_have A B C D E H L &&\n+\tcheck_dont_have F G I J K\n+\n+'\n+\n+test_expect_success 'recover and check' '\n+\n+\trecover $F &&\n+\tcheck_fsck \"dangling blob $F\"\n+\n+'\n+\n+test_expect_success 'delete' '\n+\techo 1 > C &&\n+\ttest_tick &&\n+\tgit commit -m rat C &&\n+\n+\techo 2 > C &&\n+\ttest_tick &&\n+\tgit commit -m ox C &&\n+\n+\techo 3 > C &&\n+\ttest_tick &&\n+\tgit commit -m tiger C &&\n+\n+\tHEAD_entry_count=$(git reflog | wc -l) &&\n+\tmaster_entry_count=$(git reflog show master | wc -l) &&\n+\n+\ttest $HEAD_entry_count = 5 &&\n+\ttest $master_entry_count = 5 &&\n+\n+\n+\tgit reflog delete master@{1} &&\n+\tgit reflog show master > output &&\n+\ttest $(($master_entry_count - 1)) = $(wc -l < output) &&\n+\ttest $HEAD_entry_count = $(git reflog | wc -l) &&\n+\t! grep ox < output &&\n+\n+\tmaster_entry_count=$(wc -l < output) &&\n+\n+\tgit reflog delete HEAD@{1} &&\n+\ttest $(($HEAD_entry_count -1)) = $(git reflog | wc -l) &&\n+\ttest $master_entry_count = $(git reflog show master | wc -l) &&\n+\n+\tHEAD_entry_count=$(git reflog | wc -l) &&\n+\n+\tgit reflog delete master@{07.04.2005.15:15:00.-0700} &&\n+\tgit reflog show master > output &&\n+\ttest $(($master_entry_count - 1)) = $(wc -l < output) &&\n+\t! grep dragon < output\n+\n+'\n+\n+test_expect_success 'rewind2' '\n+\n+\ttest_tick && git reset --hard HEAD~2 &&\n+\traw_reflog refs/heads/master >reflog &&\n+\ttest_when_finished rm -f reflog &&\n+\ttest_line_count = 4 reflog\n+'\n+\n+test_expect_success '--expire=never' '\n+\n+\tgit reflog expire --verbose \\\n+\t\t--expire=never \\\n+\t\t--expire-unreachable=never \\\n+\t\t--all &&\n+\traw_reflog refs/heads/master >reflog &&\n+\ttest_when_finished rm -f reflog &&\n+\ttest_line_count = 4 reflog\n+'\n+\n+test_expect_success 'gc.reflogexpire=never' '\n+\n+\tgit config gc.reflogexpire never &&\n+\tgit config gc.reflogexpireunreachable never &&\n+\tgit reflog expire --verbose --all &&\n+\traw_reflog refs/heads/master >reflog &&\n+\ttest_when_finished rm -f reflog &&\n+\ttest_line_count = 4 reflog\n+'\n+\n+test_expect_success 'gc.reflogexpire=false' '\n+\n+\tgit config gc.reflogexpire false &&\n+\tgit config gc.reflogexpireunreachable false &&\n+\tgit reflog expire --verbose --all &&\n+\traw_reflog refs/heads/master >reflog &&\n+\ttest_when_finished rm -f reflog &&\n+\ttest_line_count = 4 reflog &&\n+\n+\tgit config --unset gc.reflogexpire &&\n+\tgit config --unset gc.reflogexpireunreachable\n+\n+'\n+\n+test_expect_success 'checkout should not delete log for packed ref' '\n+\ttest $(git reflog master | wc -l) = 4 &&\n+\tgit branch foo &&\n+\tgit pack-refs --all &&\n+\tgit checkout foo &&\n+\ttest $(git reflog master | wc -l) = 4\n+'\n+\n+test_expect_success 'stale dirs do not cause d/f conflicts (reflogs on)' '\n+\ttest_when_finished \"git branch -d one || git branch -d one/two\" &&\n+\n+\tgit branch one/two master &&\n+\techo \"one/two@{0} branch: Created from master\" >expect &&\n+\tgit log -g --format=\"%gd %gs\" one/two >actual &&\n+\ttest_cmp expect actual &&\n+\tgit branch -d one/two &&\n+\n+\t# now logs/refs/heads/one is a stale directory, but\n+\t# we should move it out of the way to create \"one\" reflog\n+\tgit branch one master &&\n+\techo \"one@{0} branch: Created from master\" >expect &&\n+\tgit log -g --format=\"%gd %gs\" one >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'stale dirs do not cause d/f conflicts (reflogs off)' '\n+\ttest_when_finished \"git branch -d one || git branch -d one/two\" &&\n+\n+\tgit branch one/two master &&\n+\techo \"one/two@{0} branch: Created from master\" >expect &&\n+\tgit log -g --format=\"%gd %gs\" one/two >actual &&\n+\ttest_cmp expect actual &&\n+\tgit branch -d one/two &&\n+\n+\t# same as before, but we only create a reflog for \"one\" if\n+\t# it already exists, which it does not\n+\tgit -c core.logallrefupdates=false branch one master &&\n+\t: >expect &&\n+\tgit log -g --format=\"%gd %gs\" one >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+# Triggering the bug detected by this test requires a newline to fall\n+# exactly BUFSIZ-1 bytes from the end of the file. We don't know\n+# what that value is, since it's platform dependent. However, if\n+# we choose some value N, we also catch any D which divides N evenly\n+# (since we will read backwards in chunks of D). So we choose 8K,\n+# which catches glibc (with an 8K BUFSIZ) and *BSD (1K).\n+#\n+# Each line is 114 characters, so we need 75 to still have a few before the\n+# last 8K. The 89-character padding on the final entry lines up our\n+# newline exactly.\n+test_expect_success 'parsing reverse reflogs at BUFSIZ boundaries' '\n+\tgit checkout -b reflogskip &&\n+\tz38=00000000000000000000000000000000000000 &&\n+\tident=\"abc 0000000001 +0000\" &&\n+\tfor i in $(test_seq 1 75); do\n+\t\tprintf \"$z38%02d $z38%02d %s\\t\" $i $(($i+1)) \"$ident\" &&\n+\t\tif test $i = 75; then\n+\t\t\tfor j in $(test_seq 1 89); do\n+\t\t\t\tprintf X\n+\t\t\tdone\n+\t\telse\n+\t\t\tprintf X\n+\t\tfi &&\n+\t\tprintf \"\\n\"\n+\tdone | append_reflog refs/heads/reflogskip &&\n+\tgit rev-parse reflogskip@{73} >actual &&\n+\techo ${z38}03 >expect &&\n+\ttest_cmp expect actual\n+'\n+\n+test_done\ndiff --git a/t/t1480-refs-lmdb-submodule.sh b/t/t1480-refs-lmdb-submodule.sh\nnew file mode 100755\nindex 0000000..57a5fd2\n--- /dev/null\n+++ b/t/t1480-refs-lmdb-submodule.sh\n@@ -0,0 +1,85 @@\n+#!/bin/sh\n+#\n+# Copyright (c) 2016 Twitter, Inc\n+# Based on t5531-deep-submodule-push.sh\n+\n+test_description='Test lmdb refs backend'\n+TEST_NO_CREATE_REPO=1\n+. ./test-lib.sh\n+\n+if ! test_have_prereq LMDB\n+then\n+\tskip_all=\"Skipping lmdb refs backend tests, lmdb backend not built\"\n+\ttest_done\n+fi\n+\n+test_expect_success setup '\n+\tmkdir pub.git &&\n+\tGIT_DIR=pub.git git init --bare &&\n+\tGIT_DIR=pub.git git config receive.fsckobjects true &&\n+\tmkdir work &&\n+\t(\n+\t\tcd work &&\n+\t\tgit init --ref-storage=lmdb &&\n+\t\tgit config push.default matching &&\n+\t\tmkdir -p gar/bage &&\n+\t\t(\n+\t\t\tcd gar/bage &&\n+\t\t\tgit init --ref-storage=lmdb &&\n+\t\t\tgit config push.default matching &&\n+\t\t\t>junk &&\n+\t\t\tgit add junk &&\n+\t\t\tgit commit -m \"Initial junk\"\n+\t\t) &&\n+\t\tgit add gar/bage &&\n+\t\tgit commit -m \"Initial superproject\"\n+\t)\n+'\n+\n+test_expect_success 'submodules have same ref storage' '\n+\tgit init --ref-storage=lmdb test &&\n+\t(\n+\t\tcd test &&\n+\t\tgit submodule add ../work/gar/bage w\n+\t) &&\n+\t(\n+\t\tcd test/w &&\n+\t\tgit config extensions.refstorage >cfg &&\n+\t\techo lmdb >expect &&\n+\t\ttest_cmp cfg expect\n+\t)\n+'\n+\n+test_expect_success 'push with correct backend' '\n+\t(\n+\t\tcd work/gar/bage &&\n+\t\t>junk2 &&\n+\t\tgit add junk2 &&\n+\t\tgit commit -m \"Second junk\"\n+\t) &&\n+\t(\n+\t\tcd work &&\n+\t\tgit add gar/bage &&\n+\t\tgit commit -m \"Second commit for gar/bage\" &&\n+\t\tgit push --recurse-submodules=check ../pub.git master\n+\t)\n+'\n+\n+test_expect_success 'commit with different backend fails' '\n+\t(\n+\t\tcd work/gar/bage &&\n+\t\ttest_commit junk3 &&\n+\t\t# manually convert to files-backend\n+\t\tgitdir=\"$(git rev-parse --git-dir)\" &&\n+\t\tmkdir -p \"$gitdir/refs/heads\" &&\n+\t\tgit rev-parse HEAD >\"$gitdir/refs/heads/master\" &&\n+\t\tgit config --local --unset extensions.refStorage &&\n+\t\trm -r \"$gitdir/refs.lmdb\"\n+\t) &&\n+\t(\n+\t\tcd work &&\n+\t\ttest_must_fail git add gar/bage\n+\t)\n+'\n+\n+test_done\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 0b47eb6..ce40770 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -960,6 +960,7 @@ test -z \"$NO_PERL\" && test_set_prereq PERL\n test -z \"$NO_PYTHON\" && test_set_prereq PYTHON\n test -n \"$USE_LIBPCRE\" && test_set_prereq LIBPCRE\n test -z \"$NO_GETTEXT\" && test_set_prereq GETTEXT\n+test -n \"$USE_LIBLMDB\" && test_set_prereq LMDB\n \n # Can we rely on git's output in the C locale?\n if test -n \"$GETTEXT_POISON\"\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279914","messageId":"1456793586-22082-34-git-send-email-dturner@twopensource.com","threadId":"41574","inReplyTo":"1456793586-22082-1-git-send-email-dturner@twopensource.com","subject":"[PATCH v7 33/33] tests: add ref-storage argument","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T00:53:06Z","receivedAt":"2016-03-01T00:53:06Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"Add a --ref-storage argument to make it easy to test alternate ref\nstorage backends. This means that every test that calls git init or\ngit clone must use the new ref storage argument.\n\nModify many tests to work under alternate ref storage backends.\n\nIntroduce abstractions for raw ref/reflog reading/writing in tests\ninstead of directly frobbing the filesystem.\n\nConditionally skip tests that are not expected to succeed under this\ncondition. Most of this is straightforward. Of particular note are the\nfollowing test changes:\n\n* The rearrangement of commands in t1401 is because without HEAD in\nthe right place, git doesn't recognized the trash dir as a git repo,\nso no git commands work.\n\n* t1430-bad-ref-name specifically blocks lmdb because other alternate\nbackends might want to keep this test.\n\nSigned-off-by: David Turner \nSigned-off-by: Junio C Hamano \n---\n t/README | 6 +++\n t/lib-submodule-update.sh | 15 +++++--\n t/lib-t6000.sh | 7 ++-\n t/t0008-ignores.sh | 2 +-\n t/t0062-revision-walking.sh | 6 +++\n t/t1021-rerere-in-workdir.sh | 6 +++\n t/t1200-tutorial.sh | 8 +++-\n t/t1302-repo-version.sh | 6 +++\n t/t1305-config-include.sh | 17 ++++++--\n t/t1400-update-ref.sh | 6 +++\n t/t1401-symbolic-ref.sh | 17 +++++---\n t/t1404-update-ref-df-conflicts.sh | 8 +++-\n t/t1410-reflog.sh | 6 +++\n t/t1430-bad-ref-name.sh | 6 +++\n t/t1450-fsck.sh | 12 ++---\n t/t1506-rev-parse-diagnosis.sh | 4 +-\n t/t2013-checkout-submodule.sh | 2 +-\n t/t2105-update-index-gitfile.sh | 4 +-\n t/t2107-update-index-basic.sh | 6 +--\n t/t2201-add-update-typechange.sh | 4 +-\n t/t3001-ls-files-others-exclude.sh | 2 +-\n t/t3010-ls-files-killed-modified.sh | 4 +-\n t/t3040-subprojects-basic.sh | 4 +-\n t/t3050-subprojects-fetch.sh | 2 +-\n t/t3200-branch.sh | 75 ++++++++++++++++++--------------\n t/t3210-pack-refs.sh | 7 +++\n t/t3211-peel-ref.sh | 6 +++\n t/t3308-notes-merge.sh | 2 +-\n t/t3404-rebase-interactive.sh | 2 +-\n t/t3600-rm.sh | 2 +-\n t/t3800-mktag.sh | 4 +-\n t/t3903-stash.sh | 2 +-\n t/t4010-diff-pathspec.sh | 2 +-\n t/t4020-diff-external.sh | 2 +-\n t/t4027-diff-submodule.sh | 2 +-\n t/t4035-diff-quiet.sh | 2 +-\n t/t4255-am-submodule.sh | 2 +-\n t/t5000-tar-tree.sh | 3 +-\n t/t5304-prune.sh | 2 +-\n t/t5312-prune-corruption.sh | 11 ++++-\n t/t5500-fetch-pack.sh | 10 ++---\n t/t5510-fetch.sh | 30 ++++++-------\n t/t5526-fetch-submodules.sh | 4 +-\n t/t5527-fetch-odd-refs.sh | 7 +++\n t/t5537-fetch-shallow.sh | 7 +++\n t/t5700-clone-reference.sh | 42 +++++++++---------\n t/t6001-rev-list-graft.sh | 3 +-\n t/t6010-merge-base.sh | 2 +-\n t/t6050-replace.sh | 4 +-\n t/t6120-describe.sh | 6 ++-\n t/t6301-for-each-ref-errors.sh | 12 ++---\n t/t7201-co.sh | 2 +-\n t/t7300-clean.sh | 25 ++++++-----\n t/t7400-submodule-basic.sh | 22 +++++-----\n t/t7402-submodule-rebase.sh | 2 +-\n t/t7405-submodule-merge.sh | 10 ++---\n t/t9104-git-svn-follow-parent.sh | 3 +-\n t/t9115-git-svn-dcommit-funky-renames.sh | 2 +-\n t/t9350-fast-export.sh | 6 +--\n t/t9902-completion.sh | 4 +-\n t/t9903-bash-prompt.sh | 2 +-\n t/test-lib-functions.sh | 53 +++++++++++++++++++++-\n t/test-lib.sh | 11 +++++\n 63 files changed, 370 insertions(+), 185 deletions(-)\n\ndiff --git a/t/README b/t/README\nindex 1dc908e..8e047cb 100644\n--- a/t/README\n+++ b/t/README\n@@ -178,6 +178,12 @@ appropriately before running \"make\".\n \tthis feature by setting the GIT_TEST_CHAIN_LINT environment\n \tvariable to \"1\" or \"0\", respectively.\n \n+--ref-storage=::\n+\tIf --ref-storage is set, run tests under an alternate ref\n+\tbackend. Tests that rely on the original files backend will\n+\tbe skipped. You may also enable or disable this feature by\n+\tsetting the GIT_TEST_REF_STORAGE environment variable\n+\n You can also set the GIT_TEST_INSTALLED environment variable to\n the bindir of an existing git installation to test that installation.\n You still need to have built this git sandbox, from which various\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 79cdd34..b037eb3 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -32,7 +32,7 @@\n # invalid_sub1\n #\n create_lib_submodule_repo () {\n-\tgit init submodule_update_repo &&\n+\tgit init $ref_storage_arg submodule_update_repo &&\n \t(\n \t\tcd submodule_update_repo &&\n \t\techo \"expect\" >>.gitignore &&\n@@ -125,7 +125,16 @@ test_git_directory_is_unchanged () {\n \t\t# \"$1/.git/config\" lacks it...\n \t\tgit config --unset core.worktree\n \t) &&\n-\tdiff -r \".git/modules/$1\" \"$1/.git\" &&\n+\tif test \"$ref_storage\" = \"lmdb\"\n+\tthen\n+\t diff -x refs.lmdb -r \".git/modules/$1\" \"$1/.git\" &&\n+\t mdb_dump -p \".git/modules/$1/refs.lmdb\" >one &&\n+\t mdb_dump -p \"$1/.git/refs.lmdb\" >two &&\n+\t diff one two &&\n+\t rm one two\n+\telse\n+\t diff -r \".git/modules/$1\" \"$1/.git\"\n+\tfi\n \t(\n \t\t# ... and then restore.\n \t\tcd \".git/modules/$1\" &&\n@@ -147,7 +156,7 @@ prolog () {\n # should be updated to an existing commit.\n reset_work_tree_to () {\n \trm -rf submodule_update &&\n-\tgit clone submodule_update_repo submodule_update &&\n+\tgit clone $ref_storage_arg submodule_update_repo submodule_update &&\n \t(\n \t\tcd submodule_update &&\n \t\trm -rf sub1 &&\ndiff --git a/t/lib-t6000.sh b/t/lib-t6000.sh\nindex 3f2d873..5fb4345 100644\n--- a/t/lib-t6000.sh\n+++ b/t/lib-t6000.sh\n@@ -4,11 +4,10 @@ mkdir -p .git/refs/tags\n \n >sed.script\n \n-# Answer the sha1 has associated with the tag. The tag must exist in .git/refs/tags\n+# Answer the sha1 has associated with the tag. The tag must exist under refs/tags\n tag () {\n \t_tag=$1\n-\ttest -f \".git/refs/tags/$_tag\" || error \"tag: \\\"$_tag\\\" does not exist\"\n-\tcat \".git/refs/tags/$_tag\"\n+\tgit rev-parse --verify \"refs/tags/$_tag\" || error \"tag: \\\"$_tag\\\" does not exist\"\n }\n \n # Generate a commit using the text specified to make it unique and the tree\n@@ -26,7 +25,7 @@ save_tag () {\n \t_tag=$1\n \ttest -n \"$_tag\" || error \"usage: save_tag tag commit-args ...\"\n \tshift 1\n-\t\"$@\" >\".git/refs/tags/$_tag\"\n+\twrite_ref \"refs/tags/$_tag\" $(\"$@\")\n \n \techo \"s/$(tag $_tag)/$_tag/g\" >sed.script.tmp\n \tcat sed.script >>sed.script.tmp\ndiff --git a/t/t0008-ignores.sh b/t/t0008-ignores.sh\nindex 89544dd..63fe6d5 100755\n--- a/t/t0008-ignores.sh\n+++ b/t/t0008-ignores.sh\n@@ -184,7 +184,7 @@ test_expect_success 'setup' '\n \tfi &&\n \t(\n \t\tcd a/submodule &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\techo a >a &&\n \t\tgit add a &&\n \t\tgit commit -m\"commit in submodule\"\ndiff --git a/t/t0062-revision-walking.sh b/t/t0062-revision-walking.sh\nindex 113c728..0bd3981 100755\n--- a/t/t0062-revision-walking.sh\n+++ b/t/t0062-revision-walking.sh\n@@ -7,6 +7,12 @@ test_description='Test revision walking api'\n \n . ./test-lib.sh\n \n+if test \"$ref_storage\" != \"files\"\n+then\n+\tskip_all=\"Alternate storage doesn't do test-revision-walking\"\n+\ttest_done\n+fi\n+\n cat >run_twice_expected <<-EOF\n 1st\n > add b\ndiff --git a/t/t1021-rerere-in-workdir.sh b/t/t1021-rerere-in-workdir.sh\nindex 301e071..8fe5de7 100755\n--- a/t/t1021-rerere-in-workdir.sh\n+++ b/t/t1021-rerere-in-workdir.sh\n@@ -3,6 +3,12 @@\n test_description='rerere run in a workdir'\n . ./test-lib.sh\n \n+if test \"$ref_storage\" != \"files\"\n+then\n+\tskip_all=\"Workdirs don't support alternate ref backends\"\n+\ttest_done\n+fi\n+\n test_expect_success SYMLINKS setup '\n \tgit config rerere.enabled true &&\n \t>world &&\ndiff --git a/t/t1200-tutorial.sh b/t/t1200-tutorial.sh\nindex 397ccb6..c67c7da 100755\n--- a/t/t1200-tutorial.sh\n+++ b/t/t1200-tutorial.sh\n@@ -93,12 +93,16 @@ test_expect_success 'git whatchanged -p --root' '\n \n test_expect_success 'git tag my-first-tag' '\n \tgit tag my-first-tag &&\n-\ttest_cmp .git/refs/heads/master .git/refs/tags/my-first-tag\n+\tgit rev-parse --verify refs/heads/master >expect &&\n+\tgit rev-parse --verify refs/tags/my-first-tag >actual &&\n+\ttest_cmp expect actual\n '\n \n test_expect_success 'git checkout -b mybranch' '\n \tgit checkout -b mybranch &&\n-\ttest_cmp .git/refs/heads/master .git/refs/heads/mybranch\n+\tgit rev-parse --verify refs/heads/master >expect &&\n+\tgit rev-parse --verify refs/heads/mybranch >actual &&\n+\ttest_cmp expect actual\n '\n \n cat > branch.expect <test.patch <<-\\EOF &&\n \tdiff --git a/test.txt b/test.txt\ndiff --git a/t/t1305-config-include.sh b/t/t1305-config-include.sh\nindex 9ba2ba1..220cd63 100755\n--- a/t/t1305-config-include.sh\n+++ b/t/t1305-config-include.sh\n@@ -45,14 +45,23 @@ test_expect_success 'include options can still be examined' '\n \ttest_cmp expect actual\n '\n \n-test_expect_success 'listing includes option and expansion' '\n-\techo \"[test]one = 1\" >one &&\n-\techo \"[include]path = one\" >.gitconfig &&\n-\tcat >expect <<-\\EOF &&\n+write_expected_config()\n+{\n+\tcat <<-\\EOF &&\n \tinclude.path=one\n \ttest.one=1\n \tEOF\n+\tif test \"$ref_storage\" != \"files\"\n+\tthen\n+\t\techo \"extensions.refstorage=$ref_storage\"\n+\tfi\n+}\n+\n+test_expect_success 'listing includes option and expansion' '\n+\techo \"[test]one = 1\" >one &&\n+\techo \"[include]path = one\" >.gitconfig &&\n \tgit config --list >actual.full &&\n+\twrite_expected_config >expect &&\n \tgrep -v ^core actual.full >actual &&\n \ttest_cmp expect actual\n '\ndiff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh\nindex af1b20d..878dc53 100755\n--- a/t/t1400-update-ref.sh\n+++ b/t/t1400-update-ref.sh\n@@ -6,6 +6,12 @@\n test_description='Test git update-ref and basic ref logging'\n . ./test-lib.sh\n \n+if test \"$ref_storage\" != \"files\"\n+then\n+\tskip_all=\"This test is ref storage backend-specific\"\n+\ttest_done\n+fi\n+\n Z=$_z40\n \n test_expect_success setup '\ndiff --git a/t/t1401-symbolic-ref.sh b/t/t1401-symbolic-ref.sh\nindex 417eecc..5bfe7d2 100755\n--- a/t/t1401-symbolic-ref.sh\n+++ b/t/t1401-symbolic-ref.sh\n@@ -35,15 +35,16 @@ reset_to_sane\n \n test_expect_success 'symbolic-ref deletes HEAD' '\n \tgit symbolic-ref -d HEAD &&\n-\ttest_path_is_file .git/refs/heads/foo &&\n-\ttest_path_is_missing .git/HEAD\n+\ttest_path_is_missing .git/HEAD &&\n+\treset_to_sane &&\n+\tgit rev-parse --verify refs/heads/foo\n '\n reset_to_sane\n \n test_expect_success 'symbolic-ref deletes dangling HEAD' '\n \tgit symbolic-ref HEAD refs/heads/missing &&\n \tgit symbolic-ref -d HEAD &&\n-\ttest_path_is_missing .git/refs/heads/missing &&\n+\ttest_must_fail git rev-parse --verify refs/heads/missing &&\n \ttest_path_is_missing .git/HEAD\n '\n reset_to_sane\n@@ -58,7 +59,7 @@ reset_to_sane\n test_expect_success 'symbolic-ref fails to delete real ref' '\n \techo \"fatal: Cannot delete refs/heads/foo, not a symbolic ref\" >expect &&\n \ttest_must_fail git symbolic-ref -d refs/heads/foo >actual 2>&1 &&\n-\ttest_path_is_file .git/refs/heads/foo &&\n+\tgit rev-parse --verify refs/heads/foo &&\n \ttest_cmp expect actual\n '\n reset_to_sane\n@@ -114,7 +115,13 @@ test_expect_success 'symbolic-ref writes reflog entry' '\n \ttest_cmp expect actual\n '\n \n-test_expect_success 'symbolic-ref does not create ref d/f conflicts' '\n+if test \"$ref_storage\" = \"files\"\n+then\n+ test_cmd=test_expect_success\n+else\n+ test_cmd=test_expect_failure\n+fi\n+$test_cmd 'symbolic-ref does not create ref d/f conflicts' '\n \tgit checkout -b df &&\n \ttest_commit df &&\n \ttest_must_fail git symbolic-ref refs/heads/df/conflict refs/heads/df &&\ndiff --git a/t/t1404-update-ref-df-conflicts.sh b/t/t1404-update-ref-df-conflicts.sh\nindex 66bafb5..e918abc 100755\n--- a/t/t1404-update-ref-df-conflicts.sh\n+++ b/t/t1404-update-ref-df-conflicts.sh\n@@ -96,7 +96,13 @@ test_expect_success 'new ref is a deeper prefix of existing packed' '\n \n '\n \n-test_expect_success 'one new ref is a simple prefix of another' '\n+expect=test_expect_success\n+if test \"$ref_storage\" != \"files\"\n+then\n+\texpect=test_expect_failure\n+fi\n+\n+$expect 'one new ref is a simple prefix of another' '\n \n \tprefix=refs/5 &&\n \ttest_update_rejected $prefix \"a e\" false \"b c c/x d\" \\\ndiff --git a/t/t1410-reflog.sh b/t/t1410-reflog.sh\nindex 9cf91dc..41ff1a2 100755\n--- a/t/t1410-reflog.sh\n+++ b/t/t1410-reflog.sh\n@@ -6,6 +6,12 @@\n test_description='Test prune and reflog expiration'\n . ./test-lib.sh\n \n+if test \"$ref_storage\" != \"files\"\n+then\n+\tskip_all=\"This test is ref storage backend-specific\"\n+\ttest_done\n+fi\n+\n check_have () {\n \tgaah= &&\n \tfor N in \"$@\"\ndiff --git a/t/t1430-bad-ref-name.sh b/t/t1430-bad-ref-name.sh\nindex c465abe..bfcec3c 100755\n--- a/t/t1430-bad-ref-name.sh\n+++ b/t/t1430-bad-ref-name.sh\n@@ -3,6 +3,12 @@\n test_description='Test handling of ref names that check-ref-format rejects'\n . ./test-lib.sh\n \n+if test \"$ref_storage\" = \"lmdb\"\n+then\n+\tskip_all=\"The lmdb backend refuses to save refs with bad names\"\n+\ttest_done\n+fi\n+\n test_expect_success setup '\n \ttest_commit one &&\n \ttest_commit two\ndiff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh\nindex e66b7cb..5fd3921 100755\n--- a/t/t1450-fsck.sh\n+++ b/t/t1450-fsck.sh\n@@ -75,7 +75,7 @@ test_expect_success 'object with bad sha1' '\n '\n \n test_expect_success 'branch pointing to non-commit' '\n-\tgit rev-parse HEAD^{tree} >.git/refs/heads/invalid &&\n+\twrite_ref refs/heads/invalid $(git rev-parse HEAD^{tree}) &&\n \ttest_when_finished \"git update-ref -d refs/heads/invalid\" &&\n \ttest_must_fail git fsck 2>out &&\n \tcat out &&\n@@ -220,7 +220,7 @@ test_expect_success 'tag pointing to nonexistent' '\n \n \ttag=$(git hash-object -t tag -w --stdin .git/refs/tags/invalid &&\n+\twrite_ref refs/tags/invalid $tag &&\n \ttest_when_finished \"git update-ref -d refs/tags/invalid\" &&\n \ttest_must_fail git fsck --tags >out &&\n \tcat out &&\n@@ -241,7 +241,7 @@ test_expect_success 'tag pointing to something else than its type' '\n \n \ttag=$(git hash-object -t tag -w --stdin .git/refs/tags/wrong &&\n+\twrite_ref refs/tags/wrong $tag &&\n \ttest_when_finished \"git update-ref -d refs/tags/wrong\" &&\n \ttest_must_fail git fsck --tags\n '\n@@ -258,7 +258,7 @@ test_expect_success 'tag with incorrect tag name & missing tagger' '\n \n \ttag=$(git hash-object -t tag -w --stdin .git/refs/tags/wrong &&\n+\twrite_ref refs/tags/wrong $tag &&\n \ttest_when_finished \"git update-ref -d refs/tags/wrong\" &&\n \tgit fsck --tags 2>out &&\n \n@@ -282,7 +282,7 @@ test_expect_success 'tag with bad tagger' '\n \n \ttag=$(git hash-object --literally -t tag -w --stdin .git/refs/tags/wrong &&\n+\twrite_ref refs/tags/wrong $tag &&\n \ttest_when_finished \"git update-ref -d refs/tags/wrong\" &&\n \ttest_must_fail git fsck --tags 2>out &&\n \tgrep \"error in tag .*: invalid author/committer\" out\n@@ -301,7 +301,7 @@ test_expect_success 'tag with NUL in header' '\n \n \ttag=$(git hash-object --literally -t tag -w --stdin .git/refs/tags/wrong &&\n+\twrite_ref refs/tags/wrong $tag &&\n \ttest_when_finished \"git update-ref -d refs/tags/wrong\" &&\n \ttest_must_fail git fsck --tags 2>out &&\n \tcat out &&\ndiff --git a/t/t1506-rev-parse-diagnosis.sh b/t/t1506-rev-parse-diagnosis.sh\nindex 613d9bf..a9a15ca 100755\n--- a/t/t1506-rev-parse-diagnosis.sh\n+++ b/t/t1506-rev-parse-diagnosis.sh\n@@ -167,7 +167,7 @@ test_expect_success 'relative path when cwd is outside worktree' '\n '\n \n test_expect_success 'relative path when startup_info is NULL' '\n-\ttest_must_fail test-match-trees HEAD:./file.txt HEAD:./file.txt 2>error &&\n+\ttest_must_fail test-match-trees $(git rev-parse HEAD):./file.txt $(git rev-parse HEAD):./file.txt 2>error &&\n \tgrep \"BUG: startup_info struct is not initialized.\" error\n '\n \n@@ -213,7 +213,7 @@ test_expect_success 'arg before dashdash must be a revision (ambiguous)' '\n \t{\n \t\t# we do not want to use rev-parse here, because\n \t\t# we are testing it\n-\t\tcat .git/refs/heads/foobar &&\n+\t\traw_ref refs/heads/foobar &&\n \t\tprintf \"%s\\n\" --\n \t} >expect &&\n \tgit rev-parse foobar -- >actual &&\ndiff --git a/t/t2013-checkout-submodule.sh b/t/t2013-checkout-submodule.sh\nindex 6847f75..94e8ec8 100755\n--- a/t/t2013-checkout-submodule.sh\n+++ b/t/t2013-checkout-submodule.sh\n@@ -8,7 +8,7 @@ test_description='checkout can handle submodules'\n test_expect_success 'setup' '\n \tmkdir submodule &&\n \t(cd submodule &&\n-\t git init &&\n+\t git init $ref_storage_arg &&\n \t test_commit first) &&\n \tgit add submodule &&\n \ttest_tick &&\ndiff --git a/t/t2105-update-index-gitfile.sh b/t/t2105-update-index-gitfile.sh\nindex a7f3d47..ce151a4 100755\n--- a/t/t2105-update-index-gitfile.sh\n+++ b/t/t2105-update-index-gitfile.sh\n@@ -11,7 +11,7 @@ test_description='git update-index for gitlink to .git file.\n test_expect_success 'submodule with absolute .git file' '\n \tmkdir sub1 &&\n \t(cd sub1 &&\n-\t git init &&\n+\t git init $ref_storage_arg &&\n \t REAL=\"$(pwd)/.real\" &&\n \t mv .git \"$REAL\" &&\n \t echo \"gitdir: $REAL\" >.git &&\n@@ -25,7 +25,7 @@ test_expect_success 'add gitlink to absolute .git file' '\n test_expect_success 'submodule with relative .git file' '\n \tmkdir sub2 &&\n \t(cd sub2 &&\n-\t git init &&\n+\t git init $ref_storage_arg &&\n \t mv .git .real &&\n \t echo \"gitdir: .real\" >.git &&\n \t test_commit first)\ndiff --git a/t/t2107-update-index-basic.sh b/t/t2107-update-index-basic.sh\nindex dfe02f4..b00adef 100755\n--- a/t/t2107-update-index-basic.sh\n+++ b/t/t2107-update-index-basic.sh\n@@ -22,7 +22,7 @@ test_expect_success 'update-index -h with corrupt index' '\n \tmkdir broken &&\n \t(\n \t\tcd broken &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\t>.git/index &&\n \t\ttest_expect_code 129 git update-index -h >usage 2>&1\n \t) &&\n@@ -43,7 +43,7 @@ test_expect_success '--cacheinfo does not accept blob null sha1' '\n '\n \n test_expect_success '--cacheinfo does not accept gitlink null sha1' '\n-\tgit init submodule &&\n+\tgit init submodule $ref_storage_arg &&\n \t(cd submodule && test_commit foo) &&\n \tgit add submodule &&\n \tgit rev-parse :submodule >expect &&\n@@ -70,7 +70,7 @@ test_expect_success '.lock files cleaned up' '\n \t(\n \tcd cleanup &&\n \tmkdir worktree &&\n-\tgit init repo &&\n+\tgit init $ref_storage_arg repo &&\n \tcd repo &&\n \tgit config core.worktree ../../worktree &&\n \t# --refresh triggers late setup_work_tree,\ndiff --git a/t/t2201-add-update-typechange.sh b/t/t2201-add-update-typechange.sh\nindex 954fc51..882a5b7 100755\n--- a/t/t2201-add-update-typechange.sh\n+++ b/t/t2201-add-update-typechange.sh\n@@ -46,7 +46,7 @@ test_expect_success modify '\n \t>yomin/yomin &&\n \t(\n \t\tcd yomin &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\tgit add yomin &&\n \t\tgit commit -m \"sub initial\"\n \t) &&\n@@ -60,7 +60,7 @@ test_expect_success modify '\n \t>yonk/yonk &&\n \t(\n \t\tcd yonk &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\tgit add yonk &&\n \t\tgit commit -m \"sub initial\"\n \t) &&\ndiff --git a/t/t3001-ls-files-others-exclude.sh b/t/t3001-ls-files-others-exclude.sh\nindex d043078..ee42c1e 100755\n--- a/t/t3001-ls-files-others-exclude.sh\n+++ b/t/t3001-ls-files-others-exclude.sh\n@@ -194,7 +194,7 @@ test_expect_success 'subdirectory ignore (setup)' '\n \tmkdir -p top/l1/l2 &&\n \t(\n \t\tcd top &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\techo /.gitignore >.gitignore &&\n \t\techo l1 >>.gitignore &&\n \t\techo l2 >l1/.gitignore &&\ndiff --git a/t/t3010-ls-files-killed-modified.sh b/t/t3010-ls-files-killed-modified.sh\nindex 580e158..3cf2a74 100755\n--- a/t/t3010-ls-files-killed-modified.sh\n+++ b/t/t3010-ls-files-killed-modified.sh\n@@ -55,9 +55,9 @@ test_expect_success 'git update-index --add to add various paths.' '\n \t: >path9 &&\n \tdate >path10 &&\n \tgit update-index --add -- path0 path?/file? pathx/ju path7 path8 path9 path10 &&\n-\tgit init submod1 &&\n+\tgit init $ref_storage_arg submod1 &&\n \tgit -C submod1 commit --allow-empty -m \"empty 1\" &&\n-\tgit init submod2 &&\n+\tgit init $ref_storage_arg submod2 &&\n \tgit -C submod2 commit --allow-empty -m \"empty 2\" &&\n \tgit update-index --add submod[12] &&\n \t(\ndiff --git a/t/t3040-subprojects-basic.sh b/t/t3040-subprojects-basic.sh\nindex 0a4ff6d..368c03e 100755\n--- a/t/t3040-subprojects-basic.sh\n+++ b/t/t3040-subprojects-basic.sh\n@@ -11,10 +11,10 @@ test_expect_success 'setup: create superproject' '\n \n test_expect_success 'setup: create subprojects' '\n \tmkdir sub1 &&\n-\t( cd sub1 && git init && : >Makefile && git add * &&\n+\t( cd sub1 && git init $ref_storage_arg && : >Makefile && git add * &&\n \tgit commit -q -m \"subproject 1\" ) &&\n \tmkdir sub2 &&\n-\t( cd sub2 && git init && : >Makefile && git add * &&\n+\t( cd sub2 && git init $ref_storage_arg && : >Makefile && git add * &&\n \tgit commit -q -m \"subproject 2\" ) &&\n \tgit update-index --add sub1 &&\n \tgit add sub2 &&\ndiff --git a/t/t3050-subprojects-fetch.sh b/t/t3050-subprojects-fetch.sh\nindex 2f5f41a..8d26bb9 100755\n--- a/t/t3050-subprojects-fetch.sh\n+++ b/t/t3050-subprojects-fetch.sh\n@@ -8,7 +8,7 @@ test_expect_success setup '\n \ttest_tick &&\n \tmkdir -p sub && (\n \t\tcd sub &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\t>subfile &&\n \t\tgit add subfile &&\n \t\tgit commit -m \"subproject commit #1\"\ndiff --git a/t/t3200-branch.sh b/t/t3200-branch.sh\nindex 144e9ce..aa31444 100755\n--- a/t/t3200-branch.sh\n+++ b/t/t3200-branch.sh\n@@ -19,26 +19,26 @@ test_expect_success 'prepare a trivial repository' '\n \n test_expect_success 'git branch --help should not have created a bogus branch' '\n \ttest_might_fail git branch --man --help /dev/null 2>&1 &&\n-\ttest_path_is_missing .git/refs/heads/--help\n+\ttest_must_fail git rev-parse --verify refs/heads/--help\n '\n \n test_expect_success 'branch -h in broken repository' '\n \tmkdir broken &&\n \t(\n \t\tcd broken &&\n-\t\tgit init &&\n-\t\t>.git/refs/heads/master &&\n+\t\tgit init $ref_storage_arg &&\n+\t\twrite_ref refs/heads/master \"\" &&\n \t\ttest_expect_code 129 git branch -h >usage 2>&1\n \t) &&\n \ttest_i18ngrep \"[Uu]sage\" broken/usage\n '\n \n test_expect_success 'git branch abc should create a branch' '\n-\tgit branch abc && test_path_is_file .git/refs/heads/abc\n+\tgit branch abc && git rev-parse --verify refs/heads/abc\n '\n \n test_expect_success 'git branch a/b/c should create a branch' '\n-\tgit branch a/b/c && test_path_is_file .git/refs/heads/a/b/c\n+\tgit branch a/b/c && git rev-parse --verify refs/heads/a/b/c\n '\n \n test_expect_success 'git branch HEAD should fail' '\n@@ -51,14 +51,14 @@ EOF\n test_expect_success 'git branch -l d/e/f should create a branch and a log' '\n \tGIT_COMMITTER_DATE=\"2005-05-26 23:30\" \\\n \tgit branch -l d/e/f &&\n-\ttest_path_is_file .git/refs/heads/d/e/f &&\n-\ttest_path_is_file .git/logs/refs/heads/d/e/f &&\n-\ttest_cmp expect .git/logs/refs/heads/d/e/f\n+\tgit rev-parse --verify refs/heads/d/e/f &&\n+\traw_reflog refs/heads/d/e/f > reflog &&\n+\ttest_cmp expect reflog\n '\n \n test_expect_success 'git branch -d d/e/f should delete a branch and a log' '\n \tgit branch -d d/e/f &&\n-\ttest_path_is_missing .git/refs/heads/d/e/f &&\n+\ttest_must_fail git rev-parse --verify refs/heads/d/e/f &&\n \ttest_must_fail git reflog exists refs/heads/d/e/f\n '\n \n@@ -156,34 +156,34 @@ test_expect_success 'git branch -M master2 master2 should work when master is ch\n \n test_expect_success 'git branch -v -d t should work' '\n \tgit branch t &&\n-\ttest_path_is_file .git/refs/heads/t &&\n+\tgit rev-parse --verify refs/heads/t &&\n \tgit branch -v -d t &&\n-\ttest_path_is_missing .git/refs/heads/t\n+\ttest_must_fail git rev-parse --verify refs/heads/t\n '\n \n test_expect_success 'git branch -v -m t s should work' '\n \tgit branch t &&\n-\ttest_path_is_file .git/refs/heads/t &&\n+\tgit rev-parse --verify refs/heads/t &&\n \tgit branch -v -m t s &&\n-\ttest_path_is_missing .git/refs/heads/t &&\n-\ttest_path_is_file .git/refs/heads/s &&\n+\ttest_must_fail git rev-parse --verify refs/heads/t &&\n+\tgit rev-parse --verify refs/heads/s &&\n \tgit branch -d s\n '\n \n test_expect_success 'git branch -m -d t s should fail' '\n \tgit branch t &&\n-\ttest_path_is_file .git/refs/heads/t &&\n+\tgit rev-parse refs/heads/t &&\n \ttest_must_fail git branch -m -d t s &&\n \tgit branch -d t &&\n-\ttest_path_is_missing .git/refs/heads/t\n+\ttest_must_fail git rev-parse refs/heads/t\n '\n \n test_expect_success 'git branch --list -d t should fail' '\n \tgit branch t &&\n-\ttest_path_is_file .git/refs/heads/t &&\n+\tgit rev-parse refs/heads/t &&\n \ttest_must_fail git branch --list -d t &&\n \tgit branch -d t &&\n-\ttest_path_is_missing .git/refs/heads/t\n+\ttest_must_fail git rev-parse refs/heads/t\n '\n \n test_expect_success 'git branch --column' '\n@@ -263,6 +263,10 @@ EOF\n \ttest_cmp expected actual\n '\n \n+# All alternate ref storage backends require config, so we just skip\n+# the \"no-config\" test if we're using one.\n+if test $ref_storage = \"files\"\n+then\n mv .git/config .git/config-saved\n \n test_expect_success 'git branch -m q q2 without config should succeed' '\n@@ -271,6 +275,7 @@ test_expect_success 'git branch -m q q2 without config should succeed' '\n '\n \n mv .git/config-saved .git/config\n+fi\n \n git config branch.s/s.dummy Hello\n \n@@ -294,26 +299,26 @@ test_expect_success 'deleting a symref' '\n \tgit symbolic-ref refs/heads/symref refs/heads/target &&\n \techo \"Deleted branch symref (was refs/heads/target).\" >expect &&\n \tgit branch -d symref >actual &&\n-\ttest_path_is_file .git/refs/heads/target &&\n-\ttest_path_is_missing .git/refs/heads/symref &&\n+\tgit rev-parse refs/heads/target &&\n+\ttest_must_fail git rev-parse refs/heads/symref &&\n \ttest_i18ncmp expect actual\n '\n \n test_expect_success 'deleting a dangling symref' '\n \tgit symbolic-ref refs/heads/dangling-symref nowhere &&\n-\ttest_path_is_file .git/refs/heads/dangling-symref &&\n+\traw_ref refs/heads/dangling-symref &&\n \techo \"Deleted branch dangling-symref (was nowhere).\" >expect &&\n \tgit branch -d dangling-symref >actual &&\n-\ttest_path_is_missing .git/refs/heads/dangling-symref &&\n+\ttest_must_fail git rev-parse refs/heads/dangling-symref &&\n \ttest_i18ncmp expect actual\n '\n \n test_expect_success 'deleting a self-referential symref' '\n \tgit symbolic-ref refs/heads/self-reference refs/heads/self-reference &&\n-\ttest_path_is_file .git/refs/heads/self-reference &&\n+\traw_ref refs/heads/self-reference &&\n \techo \"Deleted branch self-reference (was refs/heads/self-reference).\" >expect &&\n \tgit branch -d self-reference >actual &&\n-\ttest_path_is_missing .git/refs/heads/self-reference &&\n+\ttest_must_fail git rev-parse refs/heads/self-reference &&\n \ttest_i18ncmp expect actual\n '\n \n@@ -321,16 +326,20 @@ test_expect_success 'renaming a symref is not allowed' '\n \tgit symbolic-ref refs/heads/master2 refs/heads/master &&\n \ttest_must_fail git branch -m master2 master3 &&\n \tgit symbolic-ref refs/heads/master2 &&\n-\ttest_path_is_file .git/refs/heads/master &&\n-\ttest_path_is_missing .git/refs/heads/master3\n+\tgit rev-parse refs/heads/master &&\n+\ttest_must_fail git rev-parse refs/heads/master3\n '\n \n+# lmdb doesn't support store reflogs in the filesystem\n+if test $ref_storage != \"lmdb\"\n+then\n test_expect_success SYMLINKS 'git branch -m u v should fail when the reflog for u is a symlink' '\n \tgit branch -l u &&\n \tmv .git/logs/refs/heads/u real-u &&\n \tln -s real-u .git/logs/refs/heads/u &&\n \ttest_must_fail git branch -m u v\n '\n+fi\n \n test_expect_success 'test tracking setup via --track' '\n \tgit config remote.local.url . &&\n@@ -570,9 +579,9 @@ EOF\n test_expect_success 'git checkout -b g/h/i -l should create a branch and a log' '\n \tGIT_COMMITTER_DATE=\"2005-05-26 23:30\" \\\n \tgit checkout -b g/h/i -l master &&\n-\ttest_path_is_file .git/refs/heads/g/h/i &&\n-\ttest_path_is_file .git/logs/refs/heads/g/h/i &&\n-\ttest_cmp expect .git/logs/refs/heads/g/h/i\n+\tgit rev-parse refs/heads/g/h/i &&\n+\traw_reflog refs/heads/g/h/i > reflog &&\n+\ttest_cmp expect reflog\n '\n \n test_expect_success 'checkout -b makes reflog by default' '\n@@ -922,17 +931,17 @@ test_expect_success '--merged catches invalid object names' '\n \n test_expect_success 'tracking with unexpected .fetch refspec' '\n \trm -rf a b c d &&\n-\tgit init a &&\n+\tgit init $ref_storage_arg a &&\n \t(\n \t\tcd a &&\n \t\ttest_commit a\n \t) &&\n-\tgit init b &&\n+\tgit init $ref_storage_arg b &&\n \t(\n \t\tcd b &&\n \t\ttest_commit b\n \t) &&\n-\tgit init c &&\n+\tgit init $ref_storage_arg c &&\n \t(\n \t\tcd c &&\n \t\ttest_commit c &&\n@@ -940,7 +949,7 @@ test_expect_success 'tracking with unexpected .fetch refspec' '\n \t\tgit remote add b ../b &&\n \t\tgit fetch --all\n \t) &&\n-\tgit init d &&\n+\tgit init $ref_storage_arg d &&\n \t(\n \t\tcd d &&\n \t\tgit remote add c ../c &&\ndiff --git a/t/t3210-pack-refs.sh b/t/t3210-pack-refs.sh\nindex 9b182a0..5d55be3 100755\n--- a/t/t3210-pack-refs.sh\n+++ b/t/t3210-pack-refs.sh\n@@ -11,6 +11,13 @@ semantic is still the same.\n '\n . ./test-lib.sh\n \n+backend=$ref_storage\n+if test \"$backend\" = \"lmdb\"\n+then\n+\tskip_all=\"The lmdb ref storage doesn't pack refs\"\n+\ttest_done\n+fi\n+\n test_expect_success 'enable reflogs' '\n \tgit config core.logallrefupdates true\n '\ndiff --git a/t/t3211-peel-ref.sh b/t/t3211-peel-ref.sh\nindex 3b7caca..84d1e8e 100755\n--- a/t/t3211-peel-ref.sh\n+++ b/t/t3211-peel-ref.sh\n@@ -3,6 +3,12 @@\n test_description='tests for the peel_ref optimization of packed-refs'\n . ./test-lib.sh\n \n+if test \"$ref_storage\" = \"lmdb\"\n+then\n+\tskip_all=\"The lmdb ref storage doesn't pack refs\"\n+\ttest_done\n+fi\n+\n test_expect_success 'create annotated tag in refs/tags' '\n \ttest_commit base &&\n \tgit tag -m annotated foo\ndiff --git a/t/t3308-notes-merge.sh b/t/t3308-notes-merge.sh\nindex 19aed7e..49148a2 100755\n--- a/t/t3308-notes-merge.sh\n+++ b/t/t3308-notes-merge.sh\n@@ -79,7 +79,7 @@ test_expect_success 'fail to merge empty notes ref into empty notes ref (z => y)\n test_expect_success 'fail to merge into various non-notes refs' '\n \ttest_must_fail git -c \"core.notesRef=refs/notes\" notes merge x &&\n \ttest_must_fail git -c \"core.notesRef=refs/notes/\" notes merge x &&\n-\tmkdir -p .git/refs/notes/dir &&\n+\tgit update-ref refs/notes/dir/test HEAD &&\n \ttest_must_fail git -c \"core.notesRef=refs/notes/dir\" notes merge x &&\n \ttest_must_fail git -c \"core.notesRef=refs/notes/dir/\" notes merge x &&\n \ttest_must_fail git -c \"core.notesRef=refs/heads/master\" notes merge x &&\ndiff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh\nindex 544f9ad..5508eb0 100755\n--- a/t/t3404-rebase-interactive.sh\n+++ b/t/t3404-rebase-interactive.sh\n@@ -608,7 +608,7 @@ test_expect_success 'submodule rebase setup' '\n \tgit checkout A &&\n \tmkdir sub &&\n \t(\n-\t\tcd sub && git init && >elif &&\n+\t\tcd sub && git init $ref_storage_arg && >elif &&\n \t\tgit add elif && git commit -m \"submodule initial\"\n \t) &&\n \techo 1 >file1 &&\ndiff --git a/t/t3600-rm.sh b/t/t3600-rm.sh\nindex d046d98..01d207e 100755\n--- a/t/t3600-rm.sh\n+++ b/t/t3600-rm.sh\n@@ -220,7 +220,7 @@ test_expect_success 'Remove nonexistent file returns nonzero exit status' '\n test_expect_success 'Call \"rm\" from outside the work tree' '\n \tmkdir repo &&\n \t(cd repo &&\n-\t git init &&\n+\t git init $ref_storage_arg &&\n \t echo something > somefile &&\n \t git add somefile &&\n \t git commit -m \"add a file\" &&\ndiff --git a/t/t3800-mktag.sh b/t/t3800-mktag.sh\nindex 8eb4794..e7ac52e 100755\n--- a/t/t3800-mktag.sh\n+++ b/t/t3800-mktag.sh\n@@ -224,7 +224,7 @@ EOF\n \n test_expect_success \\\n 'allow empty tag email' \\\n- 'git mktag .git/refs/tags/mytag 2>message'\n+ 'write_ref refs/tags/mytag $(git mktag message)'\n \n ############################################################\n # 16. disallow spaces in tag email\n@@ -352,7 +352,7 @@ EOF\n \n test_expect_success \\\n 'create valid tag' \\\n- 'git mktag .git/refs/tags/mytag 2>message'\n+ 'write_ref refs/tags/mytag $(git mktag message)'\n \n ############################################################\n # 25. check mytag\ndiff --git a/t/t3903-stash.sh b/t/t3903-stash.sh\nindex 2142c1f..1e3b50e 100755\n--- a/t/t3903-stash.sh\n+++ b/t/t3903-stash.sh\n@@ -671,7 +671,7 @@ test_expect_success 'store updates stash ref and reflog' '\n \tgit reset --hard &&\n \t! grep quux bazzy &&\n \tgit stash store -m quuxery $STASH_ID &&\n-\ttest $(cat .git/refs/stash) = $STASH_ID &&\n+\ttest $(git rev-parse stash) = $STASH_ID &&\n \tgit reflog --format=%H stash| grep $STASH_ID &&\n \tgit stash pop &&\n \tgrep quux bazzy\ndiff --git a/t/t4010-diff-pathspec.sh b/t/t4010-diff-pathspec.sh\nindex 43c488b..76b3b64 100755\n--- a/t/t4010-diff-pathspec.sh\n+++ b/t/t4010-diff-pathspec.sh\n@@ -112,7 +112,7 @@ test_expect_success 'diff-tree -r with wildcard' '\n \n test_expect_success 'setup submodules' '\n \ttest_tick &&\n-\tgit init submod &&\n+\tgit init $ref_storage_arg submod &&\n \t( cd submod && test_commit first; ) &&\n \tgit add submod &&\n \tgit commit -m first &&\ndiff --git a/t/t4020-diff-external.sh b/t/t4020-diff-external.sh\nindex 0446201..b03b4f6 100755\n--- a/t/t4020-diff-external.sh\n+++ b/t/t4020-diff-external.sh\n@@ -247,7 +247,7 @@ test_expect_success 'clean up crlf leftovers' '\n '\n \n test_expect_success 'submodule diff' '\n-\tgit init sub &&\n+\tgit init $ref_storage_arg sub &&\n \t( cd sub && test_commit sub1 ) &&\n \tgit add sub &&\n \ttest_tick &&\ndiff --git a/t/t4027-diff-submodule.sh b/t/t4027-diff-submodule.sh\nindex 518bf95..147351d 100755\n--- a/t/t4027-diff-submodule.sh\n+++ b/t/t4027-diff-submodule.sh\n@@ -344,7 +344,7 @@ test_expect_success 'combined (empty submodule)' '\n \n test_expect_success 'combined (with submodule)' '\n \trm -fr sub &&\n-\tgit clone --no-checkout . sub &&\n+\tgit clone $ref_storage_arg --no-checkout . sub &&\n \tgit diff >actual &&\n \ttest_cmp expect.withsub actual\n '\ndiff --git a/t/t4035-diff-quiet.sh b/t/t4035-diff-quiet.sh\nindex 461f4bb..553cabe 100755\n--- a/t/t4035-diff-quiet.sh\n+++ b/t/t4035-diff-quiet.sh\n@@ -13,7 +13,7 @@ test_expect_success 'setup' '\n \tgit commit -a -m second &&\n \tmkdir -p test-outside/repo && (\n \t\tcd test-outside/repo &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\techo \"1 1\" >a &&\n \t\tgit add . &&\n \t\tgit commit -m 1\ndiff --git a/t/t4255-am-submodule.sh b/t/t4255-am-submodule.sh\nindex 0ba8194..427b601 100755\n--- a/t/t4255-am-submodule.sh\n+++ b/t/t4255-am-submodule.sh\n@@ -22,7 +22,7 @@ test_expect_success 'setup diff.submodule' '\n \ttest_commit one &&\n \tINITIAL=$(git rev-parse HEAD) &&\n \n-\tgit init submodule &&\n+\tgit init $ref_storage_arg submodule &&\n \t(\n \t\tcd submodule &&\n \t\ttest_commit two &&\ndiff --git a/t/t5000-tar-tree.sh b/t/t5000-tar-tree.sh\nindex 4b68bba..79249ef 100755\n--- a/t/t5000-tar-tree.sh\n+++ b/t/t5000-tar-tree.sh\n@@ -190,7 +190,8 @@ test_expect_success \\\n test_expect_success \\\n 'git get-tar-commit-id' \\\n 'git get-tar-commit-id b.commitid &&\n- test_cmp .git/$(git symbolic-ref HEAD) b.commitid'\n+ raw_ref $(git symbolic-ref HEAD) > expect &&\n+ test_cmp expect b.commitid'\n \n test_expect_success 'git archive with --output, override inferred format' '\n \tgit archive --format=tar --output=d4.zip HEAD &&\ndiff --git a/t/t5304-prune.sh b/t/t5304-prune.sh\nindex ad7ad2f..cfe8038 100755\n--- a/t/t5304-prune.sh\n+++ b/t/t5304-prune.sh\n@@ -93,7 +93,7 @@ test_expect_success 'prune: prune nonsense parameters' '\n test_expect_success 'prune: prune unreachable heads' '\n \n \tgit config core.logAllRefUpdates false &&\n-\tmv .git/logs .git/logs.old &&\n+\tdelete_all_reflogs &&\n \t: > file2 &&\n \tgit add file2 &&\n \tgit commit -m temporary &&\ndiff --git a/t/t5312-prune-corruption.sh b/t/t5312-prune-corruption.sh\nindex da9d599..f118735 100755\n--- a/t/t5312-prune-corruption.sh\n+++ b/t/t5312-prune-corruption.sh\n@@ -21,7 +21,7 @@ test_expect_success 'create history reachable only from a bogus-named ref' '\n \ttest_tick && git commit --allow-empty -m bogus &&\n \tbogus=$(git rev-parse HEAD) &&\n \tgit cat-file commit $bogus >saved &&\n-\techo $bogus >.git/refs/heads/bogus..name &&\n+\twrite_ref refs/heads/bogus..name $bogus &&\n \tgit reset --hard HEAD^\n '\n \n@@ -47,7 +47,7 @@ test_expect_success 'destructive repack keeps packed object' '\n \n # subsequent tests will have different corruptions\n test_expect_success 'clean up bogus ref' '\n-\trm .git/refs/heads/bogus..name\n+\tdelete_ref refs/heads/bogus..name\n '\n \n # We create two new objects here, \"one\" and \"two\". Our\n@@ -85,6 +85,13 @@ test_expect_success 'pack-refs does not silently delete broken loose ref' '\n \ttest_cmp expect actual\n '\n \n+backend=$ref_storage\n+if test \"$backend\" = \"lmdb\"\n+then\n+ skip=\"The lmdb backend doesn't write a packed-refs file\"\n+ test_done\n+fi\n+\n # we do not want to count on running pack-refs to\n # actually pack it, as it is perfectly reasonable to\n # skip processing a broken ref\ndiff --git a/t/t5500-fetch-pack.sh b/t/t5500-fetch-pack.sh\nindex f512098..0ab58f5 100755\n--- a/t/t5500-fetch-pack.sh\n+++ b/t/t5500-fetch-pack.sh\n@@ -30,7 +30,7 @@ add () {\n \ttest_tick &&\n \tcommit=$(echo \"$text\" | git commit-tree $tree $parents) &&\n \teval \"$name=$commit; export $name\" &&\n-\techo $commit > .git/refs/heads/$branch &&\n+\tgit update-ref refs/heads/$branch $commit &&\n \teval ${branch}TIP=$commit\n }\n \n@@ -45,10 +45,10 @@ pull_to_client () {\n \n \t\t\tcase \"$heads\" in\n \t\t\t *A*)\n-\t\t\t\t echo $ATIP > .git/refs/heads/A;;\n+\t\t\t\t git update-ref refs/heads/A $ATIP;;\n \t\t\tesac &&\n \t\t\tcase \"$heads\" in *B*)\n-\t\t\t echo $BTIP > .git/refs/heads/B;;\n+\t\t\t git update-ref refs/heads/B $BTIP;;\n \t\t\tesac &&\n \t\t\tgit symbolic-ref HEAD refs/heads/$(echo $heads \\\n \t\t\t\t| sed -e \"s/^\\(.\\).*$/\\1/\") &&\n@@ -92,8 +92,8 @@ test_expect_success 'setup' '\n \t\tcur=$(($cur+1))\n \tdone &&\n \tadd B1 $A1 &&\n-\techo $ATIP > .git/refs/heads/A &&\n-\techo $BTIP > .git/refs/heads/B &&\n+\tgit update-ref refs/heads/A $ATIP &&\n+\tgit update-ref refs/heads/B $BTIP &&\n \tgit symbolic-ref HEAD refs/heads/B\n '\n \ndiff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh\nindex 0c10c85..9591271 100755\n--- a/t/t5510-fetch.sh\n+++ b/t/t5510-fetch.sh\n@@ -28,20 +28,20 @@ test_expect_success setup '\n \tgit commit -a -m original'\n \n test_expect_success \"clone and setup child repos\" '\n-\tgit clone . one &&\n+\tgit clone $ref_storage_arg . one &&\n \t(\n \t\tcd one &&\n \t\techo >file updated by one &&\n \t\tgit commit -a -m \"updated by one\"\n \t) &&\n-\tgit clone . two &&\n+\tgit clone $ref_storage_arg . two &&\n \t(\n \t\tcd two &&\n \t\tgit config branch.master.remote one &&\n \t\tgit config remote.one.url ../one/.git/ &&\n \t\tgit config remote.one.fetch refs/heads/master:refs/heads/one\n \t) &&\n-\tgit clone . three &&\n+\tgit clone $ref_storage_arg . three &&\n \t(\n \t\tcd three &&\n \t\tgit config branch.master.remote two &&\n@@ -53,8 +53,8 @@ test_expect_success \"clone and setup child repos\" '\n \t\t\techo \"Pull: refs/heads/one:refs/heads/one\"\n \t\t} >.git/remotes/two\n \t) &&\n-\tgit clone . bundle &&\n-\tgit clone . seven\n+\tgit clone $ref_storage_arg . bundle &&\n+\tgit clone $ref_storage_arg . seven\n '\n \n test_expect_success \"fetch test\" '\n@@ -63,7 +63,7 @@ test_expect_success \"fetch test\" '\n \tgit commit -a -m \"updated by origin\" &&\n \tcd two &&\n \tgit fetch &&\n-\ttest -f .git/refs/heads/one &&\n+\tgit rev-parse --verify refs/heads/one &&\n \tmine=$(git rev-parse refs/heads/one) &&\n \this=$(cd ../one && git rev-parse refs/heads/master) &&\n \ttest \"z$mine\" = \"z$his\"\n@@ -73,8 +73,8 @@ test_expect_success \"fetch test for-merge\" '\n \tcd \"$D\" &&\n \tcd three &&\n \tgit fetch &&\n-\ttest -f .git/refs/heads/two &&\n-\ttest -f .git/refs/heads/one &&\n+\tgit rev-parse --verify refs/heads/two &&\n+\tgit rev-parse --verify refs/heads/one &&\n \tmaster_in_two=$(cd ../two && git rev-parse master) &&\n \tone_in_two=$(cd ../two && git rev-parse one) &&\n \t{\n@@ -180,7 +180,7 @@ test_expect_success 'fetch tags when there is no tags' '\n \n mkdir notags &&\n cd notags &&\n- git init &&\n+ git init $ref_storage_arg &&\n \n git fetch -t ..\n \n@@ -194,7 +194,7 @@ test_expect_success 'fetch following tags' '\n \n \tmkdir four &&\n \tcd four &&\n-\tgit init &&\n+\tgit init $ref_storage_arg &&\n \n \tgit fetch .. :track &&\n \tgit show-ref --verify refs/tags/anno &&\n@@ -204,7 +204,7 @@ test_expect_success 'fetch following tags' '\n \n test_expect_success 'fetch uses remote ref names to describe new refs' '\n \tcd \"$D\" &&\n-\tgit init descriptive &&\n+\tgit init $ref_storage_arg descriptive &&\n \t(\n \t\tcd descriptive &&\n \t\tgit config remote.o.url .. &&\n@@ -238,7 +238,7 @@ test_expect_success 'fetch must not resolve short tag name' '\n \n \tmkdir five &&\n \tcd five &&\n-\tgit init &&\n+\tgit init $ref_storage_arg &&\n \n \ttest_must_fail git fetch .. anno:five\n \n@@ -251,7 +251,7 @@ test_expect_success 'fetch can now resolve short remote name' '\n \n \tmkdir six &&\n \tcd six &&\n-\tgit init &&\n+\tgit init $ref_storage_arg &&\n \n \tgit fetch .. six:six\n '\n@@ -529,7 +529,7 @@ test_expect_success \"should be able to fetch with duplicate refspecs\" '\n \tmkdir dups &&\n \t(\n \t\tcd dups &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\tgit config branch.master.remote three &&\n \t\tgit config remote.three.url ../three/.git &&\n \t\tgit config remote.three.fetch +refs/heads/*:refs/remotes/origin/* &&\n@@ -665,7 +665,7 @@ test_expect_success 'fetching a one-level ref works' '\n \ttest_commit extra &&\n \tgit reset --hard HEAD^ &&\n \tgit update-ref refs/foo extra &&\n-\tgit init one-level &&\n+\tgit init $ref_storage_arg one-level &&\n \t(\n \t\tcd one-level &&\n \t\tgit fetch .. HEAD refs/foo\ndiff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh\nindex 954d0e4..0fef8ac 100755\n--- a/t/t5526-fetch-submodules.sh\n+++ b/t/t5526-fetch-submodules.sh\n@@ -38,7 +38,7 @@ test_expect_success setup '\n \tmkdir deepsubmodule &&\n \t(\n \t\tcd deepsubmodule &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\techo deepsubcontent > deepsubfile &&\n \t\tgit add deepsubfile &&\n \t\tgit commit -m new deepsubfile\n@@ -46,7 +46,7 @@ test_expect_success setup '\n \tmkdir submodule &&\n \t(\n \t\tcd submodule &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\techo subcontent > subfile &&\n \t\tgit add subfile &&\n \t\tgit submodule add \"$pwd/deepsubmodule\" subdir/deepsubmodule &&\ndiff --git a/t/t5527-fetch-odd-refs.sh b/t/t5527-fetch-odd-refs.sh\nindex 207899a..b9ecfa0 100755\n--- a/t/t5527-fetch-odd-refs.sh\n+++ b/t/t5527-fetch-odd-refs.sh\n@@ -26,6 +26,13 @@ test_expect_success 'suffix ref is ignored during fetch' '\n \ttest_cmp expect actual\n '\n \n+backend=$ref_storage\n+if test \"$backend\" = \"lmdb\"\n+then\n+ skip=\"The lmdb backend doesn't do crazy-long refs\"\n+ test_done\n+fi\n+\n test_expect_success 'try to create repo with absurdly long refname' '\n \tref240=$_z40/$_z40/$_z40/$_z40/$_z40/$_z40 &&\n \tref1440=$ref240/$ref240/$ref240/$ref240/$ref240/$ref240 &&\ndiff --git a/t/t5537-fetch-shallow.sh b/t/t5537-fetch-shallow.sh\nindex df8d2f0..114d25a 100755\n--- a/t/t5537-fetch-shallow.sh\n+++ b/t/t5537-fetch-shallow.sh\n@@ -173,6 +173,13 @@ EOF\n \t)\n '\n \n+backend=$ref_storage\n+if test \"$backend\" = \"lmdb\"\n+then\n+\tskip=\"The lmdb backend needs write access for its locks\"\n+\ttest_done\n+fi\n+\n test_expect_success POSIXPERM,SANITY 'shallow fetch from a read-only repo' '\n \tcp -R .git read-only.git &&\n \tfind read-only.git -print | xargs chmod -w &&\ndiff --git a/t/t5700-clone-reference.sh b/t/t5700-clone-reference.sh\nindex 4320082..49e7868 100755\n--- a/t/t5700-clone-reference.sh\n+++ b/t/t5700-clone-reference.sh\n@@ -34,14 +34,14 @@ test_expect_success 'preparing first repository' '\n '\n \n test_expect_success 'preparing second repository' '\n-\tgit clone A B &&\n+\tgit clone $ref_storage_arg A B &&\n \tcommit_in B file2 &&\n \tgit -C B repack -ad &&\n \tgit -C B prune\n '\n \n test_expect_success 'cloning with reference (-l -s)' '\n-\tgit clone -l -s --reference B A C\n+\tgit clone $ref_storage_arg -l -s --reference B A C\n '\n \n test_expect_success 'existence of info/alternates' '\n@@ -57,7 +57,7 @@ test_expect_success 'that reference gets used' '\n '\n \n test_expect_success 'cloning with reference (no -l -s)' '\n-\tGIT_TRACE_PACKET=$U.D git clone --reference B \"file://$(pwd)/A\" D\n+\tGIT_TRACE_PACKET=$U.D git clone $ref_storage_arg --reference B \"file://$(pwd)/A\" D\n '\n \n test_expect_success 'fetched no objects' '\n@@ -108,20 +108,20 @@ test_expect_success 'preparing alternate repository #1' '\n '\n \n test_expect_success 'cloning alternate repo #2 and adding changes to repo #1' '\n-\tgit clone F G &&\n+\tgit clone $ref_storage_arg F G &&\n \tcommit_in F file2\n '\n \n test_expect_success 'cloning alternate repo #1, using #2 as reference' '\n-\tgit clone --reference G F H\n+\tgit clone $ref_storage_arg --reference G F H\n '\n \n test_expect_success 'cloning with reference being subset of source (-l -s)' '\n-\tgit clone -l -s --reference A B E\n+\tgit clone $ref_storage_arg -l -s --reference A B E\n '\n \n test_expect_success 'cloning with multiple references drops duplicates' '\n-\tgit clone -s --reference B --reference A --reference B A dups &&\n+\tgit clone $ref_storage_arg -s --reference B --reference A --reference B A dups &&\n \ttest_line_count = 2 dups/.git/objects/info/alternates\n '\n \n@@ -129,11 +129,11 @@ test_expect_success 'clone with reference from a tagged repository' '\n \t(\n \t\tcd A && git tag -a -m tagged HEAD\n \t) &&\n-\tgit clone --reference=A A I\n+\tgit clone $ref_storage_arg --reference=A A I\n '\n \n test_expect_success 'prepare branched repository' '\n-\tgit clone A J &&\n+\tgit clone $ref_storage_arg A J &&\n \t(\n \t\tcd J &&\n \t\tgit checkout -b other master^ &&\n@@ -145,7 +145,7 @@ test_expect_success 'prepare branched repository' '\n '\n \n test_expect_success 'fetch with incomplete alternates' '\n-\tgit init K &&\n+\tgit init $ref_storage_arg K &&\n \techo \"$base_dir/A/.git/objects\" >K/.git/objects/info/alternates &&\n \t(\n \t\tcd K &&\n@@ -160,29 +160,29 @@ test_expect_success 'fetch with incomplete alternates' '\n '\n \n test_expect_success 'clone using repo with gitfile as a reference' '\n-\tgit clone --separate-git-dir=L A M &&\n-\tgit clone --reference=M A N &&\n+\tgit clone $ref_storage_arg --separate-git-dir=L A M &&\n+\tgit clone $ref_storage_arg --reference=M A N &&\n \techo \"$base_dir/L/objects\" >expected &&\n \ttest_cmp expected \"$base_dir/N/.git/objects/info/alternates\"\n '\n \n test_expect_success 'clone using repo pointed at by gitfile as reference' '\n-\tgit clone --reference=M/.git A O &&\n+\tgit clone $ref_storage_arg --reference=M/.git A O &&\n \techo \"$base_dir/L/objects\" >expected &&\n \ttest_cmp expected \"$base_dir/O/.git/objects/info/alternates\"\n '\n \n test_expect_success 'clone and dissociate from reference' '\n-\tgit init P &&\n+\tgit init $ref_storage_arg P &&\n \t(\n \t\tcd P &&\ttest_commit one\n \t) &&\n-\tgit clone P Q &&\n+\tgit clone $ref_storage_arg P Q &&\n \t(\n \t\tcd Q && test_commit two\n \t) &&\n-\tgit clone --no-local --reference=P Q R &&\n-\tgit clone --no-local --reference=P --dissociate Q S &&\n+\tgit clone $ref_storage_arg --no-local --reference=P Q R &&\n+\tgit clone $ref_storage_arg --no-local --reference=P --dissociate Q S &&\n \t# removing the reference P would corrupt R but not S\n \trm -fr P &&\n \ttest_must_fail git -C R fsck &&\n@@ -198,14 +198,14 @@ test_expect_success 'clone, dissociate from partial reference and repack' '\n \t\ttest_commit two &&\n \t\tgit repack\n \t) &&\n-\tgit clone --bare P Q &&\n+\tgit clone $ref_storage_arg --bare P Q &&\n \t(\n \t\tcd P &&\n \t\tgit checkout -b second &&\n \t\ttest_commit three &&\n \t\tgit repack\n \t) &&\n-\tgit clone --bare --dissociate --reference=P Q R &&\n+\tgit clone $ref_storage_arg --bare --dissociate --reference=P Q R &&\n \tls R/objects/pack/*.pack >packs.txt &&\n \ttest_line_count = 1 packs.txt\n '\n@@ -214,9 +214,9 @@ test_expect_success 'clone, dissociate from alternates' '\n \trm -fr A B C &&\n \ttest_create_repo A &&\n \tcommit_in A file1 &&\n-\tgit clone --reference=A A B &&\n+\tgit clone $ref_storage_arg --reference=A A B &&\n \ttest_line_count = 1 B/.git/objects/info/alternates &&\n-\tgit clone --local --dissociate B C &&\n+\tgit clone $ref_storage_arg --local --dissociate B C &&\n \t! test -f C/.git/objects/info/alternates &&\n \t( cd C && git fsck )\n '\ndiff --git a/t/t6001-rev-list-graft.sh b/t/t6001-rev-list-graft.sh\nindex 05ddc69..0c009ab 100755\n--- a/t/t6001-rev-list-graft.sh\n+++ b/t/t6001-rev-list-graft.sh\n@@ -20,7 +20,8 @@ test_expect_success setup '\n \tgit commit -a -m \"Third in one history.\" &&\n \tA2=$(git rev-parse --verify HEAD) &&\n \n-\trm -f .git/refs/heads/master .git/index &&\n+\tdelete_ref refs/heads/master &&\n+\trm -f .git/index &&\n \n \techo >fileA fileA again &&\n \techo >subdir/fileB fileB again &&\ndiff --git a/t/t6010-merge-base.sh b/t/t6010-merge-base.sh\nindex 39b3238..b2003c5 100755\n--- a/t/t6010-merge-base.sh\n+++ b/t/t6010-merge-base.sh\n@@ -34,7 +34,7 @@ doit () {\n \n \tcommit=$(echo $NAME | git commit-tree $T $PARENTS) &&\n \n-\techo $commit >.git/refs/tags/$NAME &&\n+\tgit update-ref refs/tags/$NAME $commit &&\n \techo $commit\n }\n \ndiff --git a/t/t6050-replace.sh b/t/t6050-replace.sh\nindex c630aba..60f0011 100755\n--- a/t/t6050-replace.sh\n+++ b/t/t6050-replace.sh\n@@ -124,13 +124,13 @@ tagger T A Gger <> 0 +0000\n EOF\n \n test_expect_success 'tag replaced commit' '\n- git mktag .git/refs/tags/mytag 2>message\n+ write_ref refs/tags/mytag $(git mktag message)\n '\n \n test_expect_success '\"git fsck\" works' '\n git fsck master >fsck_master.out &&\n grep \"dangling commit $R\" fsck_master.out &&\n- grep \"dangling tag $(cat .git/refs/tags/mytag)\" fsck_master.out &&\n+ grep \"dangling tag $(raw_ref refs/tags/mytag)\" fsck_master.out &&\n test -z \"$(git fsck)\"\n '\n \ndiff --git a/t/t6120-describe.sh b/t/t6120-describe.sh\nindex 85f2694..ca30c18 100755\n--- a/t/t6120-describe.sh\n+++ b/t/t6120-describe.sh\n@@ -128,7 +128,8 @@ test_expect_success 'no warning was displayed for A' '\n '\n \n test_expect_success 'rename tag A to Q locally' '\n-\tmv .git/refs/tags/A .git/refs/tags/Q\n+\twrite_ref refs/tags/Q $(raw_ref refs/tags/A) &&\n+\tdelete_ref refs/tags/A\n '\n cat - >err.expect <.git/$r &&\n-\ttest_when_finished \"rm -f .git/$r\" &&\n+\twrite_ref $r '' &&\n+\ttest_when_finished \"delete_ref $r\" &&\n \techo \"warning: ignoring broken ref $r\" >broken-err &&\n \tgit for-each-ref >out 2>err &&\n \ttest_cmp full-list out &&\n@@ -26,8 +26,8 @@ test_expect_success 'Broken refs are reported correctly' '\n \n test_expect_success 'NULL_SHA1 refs are reported correctly' '\n \tr=refs/heads/zeros &&\n-\techo $ZEROS >.git/$r &&\n-\ttest_when_finished \"rm -f .git/$r\" &&\n+\twrite_ref $r $ZEROS &&\n+\ttest_when_finished \"delete_ref $r\" &&\n \techo \"warning: ignoring broken ref $r\" >zeros-err &&\n \tgit for-each-ref >out 2>err &&\n \ttest_cmp full-list out &&\n@@ -39,8 +39,8 @@ test_expect_success 'NULL_SHA1 refs are reported correctly' '\n \n test_expect_success 'Missing objects are reported correctly' '\n \tr=refs/heads/missing &&\n-\techo $MISSING >.git/$r &&\n-\ttest_when_finished \"rm -f .git/$r\" &&\n+\twrite_ref $r $MISSING &&\n+\ttest_when_finished \"delete_ref $r\" &&\n \techo \"fatal: missing object $MISSING for $r\" >missing-err &&\n \ttest_must_fail git for-each-ref 2>err &&\n \ttest_cmp missing-err err &&\ndiff --git a/t/t7201-co.sh b/t/t7201-co.sh\nindex 8859236..ab1fb99 100755\n--- a/t/t7201-co.sh\n+++ b/t/t7201-co.sh\n@@ -65,7 +65,7 @@ test_expect_success setup '\n test_expect_success \"checkout from non-existing branch\" '\n \n \tgit checkout -b delete-me master &&\n-\trm .git/refs/heads/delete-me &&\n+\tgit update-ref -d --no-deref refs/heads/delete-me &&\n \ttest refs/heads/delete-me = \"$(git symbolic-ref HEAD)\" &&\n \tgit checkout master &&\n \ttest refs/heads/master = \"$(git symbolic-ref HEAD)\"\ndiff --git a/t/t7300-clean.sh b/t/t7300-clean.sh\nindex 86ceb38..36832ec 100755\n--- a/t/t7300-clean.sh\n+++ b/t/t7300-clean.sh\n@@ -431,7 +431,7 @@ test_expect_success 'nested git work tree' '\n \tmkdir -p foo bar baz/boo &&\n \t(\n \t\tcd foo &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\ttest_commit nested hello.world\n \t) &&\n \t(\n@@ -440,7 +440,7 @@ test_expect_success 'nested git work tree' '\n \t) &&\n \t(\n \t\tcd baz/boo &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\ttest_commit deeply.nested deeper.world\n \t) &&\n \tgit clean -f -d &&\n@@ -455,6 +455,7 @@ test_expect_success 'should clean things that almost look like git but are not'\n \trm -fr almost_git almost_bare_git almost_submodule &&\n \tmkdir -p almost_git/.git/objects &&\n \tmkdir -p almost_git/.git/refs &&\n+\tmkdir -p almost_git/.git/refs.lmdb &&\n \tcat >almost_git/.git/HEAD <<-\\EOF &&\n \tgarbage\n \tEOF\n@@ -475,7 +476,7 @@ test_expect_success 'should not clean submodules' '\n \tmkdir repo to_clean &&\n \t(\n \t\tcd repo &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\ttest_commit msg hello.world\n \t) &&\n \tgit submodule add ./repo/.git sub1 &&\n@@ -511,7 +512,7 @@ test_expect_success POSIXPERM 'should avoid cleaning possible submodules' '\n \n test_expect_success 'nested (empty) git should be kept' '\n \trm -fr empty_repo to_clean &&\n-\tgit init empty_repo &&\n+\tgit init $ref_storage_arg empty_repo &&\n \tmkdir to_clean &&\n \t>to_clean/should_clean.this &&\n \tgit clean -f -d &&\n@@ -521,7 +522,7 @@ test_expect_success 'nested (empty) git should be kept' '\n \n test_expect_success 'nested bare repositories should be cleaned' '\n \trm -fr bare1 bare2 subdir &&\n-\tgit init --bare bare1 &&\n+\tgit init $ref_storage_arg --bare bare1 &&\n \tgit clone --local --bare . bare2 &&\n \tmkdir subdir &&\n \tcp -r bare2 subdir/bare3 &&\n@@ -534,7 +535,7 @@ test_expect_success 'nested bare repositories should be cleaned' '\n test_expect_failure 'nested (empty) bare repositories should be cleaned even when in .git' '\n \trm -fr strange_bare &&\n \tmkdir strange_bare &&\n-\tgit init --bare strange_bare/.git &&\n+\tgit init $ref_storage_arg --bare strange_bare/.git &&\n \tgit clean -f -d &&\n \ttest_path_is_missing strange_bare\n '\n@@ -552,7 +553,7 @@ test_expect_success 'giving path in nested git work tree will remove it' '\n \tmkdir repo &&\n \t(\n \t\tcd repo &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\tmkdir -p bar/baz &&\n \t\ttest_commit msg bar/baz/hello.world\n \t) &&\n@@ -567,7 +568,7 @@ test_expect_success 'giving path to nested .git will not remove it' '\n \tmkdir repo untracked &&\n \t(\n \t\tcd repo &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\ttest_commit msg hello.world\n \t) &&\n \tgit clean -f -d repo/.git &&\n@@ -582,7 +583,7 @@ test_expect_success 'giving path to nested .git/ will remove contents' '\n \tmkdir repo untracked &&\n \t(\n \t\tcd repo &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\ttest_commit msg hello.world\n \t) &&\n \tgit clean -f -d repo/.git/ &&\n@@ -596,7 +597,7 @@ test_expect_success 'force removal of nested git work tree' '\n \tmkdir -p foo bar baz/boo &&\n \t(\n \t\tcd foo &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\ttest_commit nested hello.world\n \t) &&\n \t(\n@@ -605,7 +606,7 @@ test_expect_success 'force removal of nested git work tree' '\n \t) &&\n \t(\n \t\tcd baz/boo &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\ttest_commit deeply.nested deeper.world\n \t) &&\n \tgit clean -f -f -d &&\n@@ -619,7 +620,7 @@ test_expect_success 'git clean -e' '\n \tmkdir repo &&\n \t(\n \t\tcd repo &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\ttouch known 1 2 3 &&\n \t\tgit add known &&\n \t\tgit clean -f -e 1 -e 2 &&\ndiff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh\nindex 5572327..5892e48 100755\n--- a/t/t7400-submodule-basic.sh\n+++ b/t/t7400-submodule-basic.sh\n@@ -32,7 +32,7 @@ test_expect_success 'setup - repository in init subdirectory' '\n \tmkdir init &&\n \t(\n \t\tcd init &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\techo a >a &&\n \t\tgit add a &&\n \t\tgit commit -m \"submodule commit 1\" &&\n@@ -52,8 +52,8 @@ test_expect_success 'setup - hide init subdirectory' '\n '\n \n test_expect_success 'setup - repository to add submodules to' '\n-\tgit init addtest &&\n-\tgit init addtest-ignore\n+\tgit init $ref_storage_arg addtest &&\n+\tgit init $ref_storage_arg addtest-ignore\n '\n \n # The 'submodule add' tests need some repository to add as a submodule.\n@@ -540,7 +540,7 @@ test_expect_success 'add submodules without specifying an explicit path' '\n \tmkdir repo &&\n \t(\n \t\tcd repo &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\techo r >r &&\n \t\tgit add r &&\n \t\tgit commit -m \"repo commit 1\"\n@@ -585,11 +585,11 @@ test_expect_success 'set up for relative path tests' '\n \tmkdir reltest &&\n \t(\n \t\tcd reltest &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\tmkdir sub &&\n \t\t(\n \t\t\tcd sub &&\n-\t\t\tgit init &&\n+\t\t\tgit init $ref_storage_arg &&\n \t\t\ttest_commit foo\n \t\t) &&\n \t\tgit add sub &&\n@@ -754,7 +754,7 @@ test_expect_success '../bar/a/b/c works with relative local path - ../foo/bar.gi\n \t\tcp pristine-.git-config .git/config &&\n \t\tcp pristine-.gitmodules .gitmodules &&\n \t\tmkdir -p a/b/c &&\n-\t\t(cd a/b/c; git init) &&\n+\t\t(cd a/b/c; git init $ref_storage_arg) &&\n \t\tgit config remote.origin.url ../foo/bar.git &&\n \t\tgit submodule add ../bar/a/b/c ./a/b/c &&\n \t\tgit submodule init &&\n@@ -975,7 +975,7 @@ test_expect_success 'submodule with UTF-8 name' '\n \tmkdir \"$svname\" &&\n \t(\n \t\tcd \"$svname\" &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\t>sub &&\n \t\tgit add sub &&\n \t\tgit commit -m \"init sub\"\n@@ -990,7 +990,7 @@ test_expect_success 'submodule add clone shallow submodule' '\n \tpwd=$(pwd) &&\n \t(\n \t\tcd super &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\tgit submodule add --depth=1 file://\"$pwd\"/example2 submodule &&\n \t\t(\n \t\t\tcd submodule &&\n@@ -1003,7 +1003,7 @@ test_expect_success 'submodule helper list is not confused by common prefixes' '\n \tmkdir -p dir1/b &&\n \t(\n \t\tcd dir1/b &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\techo hi >testfile2 &&\n \t\tgit add . &&\n \t\tgit commit -m \"test1\"\n@@ -1011,7 +1011,7 @@ test_expect_success 'submodule helper list is not confused by common prefixes' '\n \tmkdir -p dir2/b &&\n \t(\n \t\tcd dir2/b &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\techo hello >testfile1 &&\n \t\tgit add . &&\n \t\tgit commit -m \"test2\"\ndiff --git a/t/t7402-submodule-rebase.sh b/t/t7402-submodule-rebase.sh\nindex 8e32f19..85166da 100755\n--- a/t/t7402-submodule-rebase.sh\n+++ b/t/t7402-submodule-rebase.sh\n@@ -13,7 +13,7 @@ test_expect_success setup '\n \tgit add file &&\n \ttest_tick &&\n \tgit commit -m initial &&\n-\tgit clone . submodule &&\n+\tgit clone $ref_storage_arg . submodule &&\n \tgit add submodule &&\n \ttest_tick &&\n \tgit commit -m submodule &&\ndiff --git a/t/t7405-submodule-merge.sh b/t/t7405-submodule-merge.sh\nindex 0d5b42a..9d147df 100755\n--- a/t/t7405-submodule-merge.sh\n+++ b/t/t7405-submodule-merge.sh\n@@ -18,7 +18,7 @@ test_expect_success setup '\n \n \tmkdir sub &&\n \t(cd sub &&\n-\t git init &&\n+\t git init $ref_storage_arg &&\n \t echo original > file &&\n \t git add file &&\n \t test_tick &&\n@@ -68,10 +68,10 @@ test_expect_success setup '\n test_expect_success 'setup for merge search' '\n \tmkdir merge-search &&\n \t(cd merge-search &&\n-\tgit init &&\n+\tgit init $ref_storage_arg &&\n \tmkdir sub &&\n \t(cd sub &&\n-\t git init &&\n+\t git init $ref_storage_arg &&\n \t echo \"file-a\" > file-a &&\n \t git add file-a &&\n \t git commit -m \"sub-a\" &&\n@@ -232,10 +232,10 @@ test_expect_success 'merging with a modify/modify conflict between merge bases'\n test_expect_success 'setup for recursive merge with submodule' '\n \tmkdir merge-recursive &&\n \t(cd merge-recursive &&\n-\t git init &&\n+\t git init $ref_storage_arg &&\n \t mkdir sub &&\n \t (cd sub &&\n-\t git init &&\n+\t git init $ref_storage_arg &&\n \t test_commit a &&\n \t git checkout -b sub-b master &&\n \t test_commit b &&\ndiff --git a/t/t9104-git-svn-follow-parent.sh b/t/t9104-git-svn-follow-parent.sh\nindex cd480ed..0c9c8f9 100755\n--- a/t/t9104-git-svn-follow-parent.sh\n+++ b/t/t9104-git-svn-follow-parent.sh\n@@ -213,7 +213,8 @@ test_expect_success \"multi-fetch continues to work\" \"\n \t\"\n \n test_expect_success \"multi-fetch works off a 'clean' repository\" '\n-\trm -r \"$GIT_DIR/svn\" \"$GIT_DIR/refs/remotes\" \"$GIT_DIR/logs\" &&\n+\trm -rf \"$GIT_DIR/svn\" \"$GIT_DIR/refs/remotes\" &&\n+\tgit reflog expire --all --expire=all &&\n \tmkdir \"$GIT_DIR/svn\" &&\n \tgit svn multi-fetch\n \t'\ndiff --git a/t/t9115-git-svn-dcommit-funky-renames.sh b/t/t9115-git-svn-dcommit-funky-renames.sh\nindex 0990f8d..7b9140a 100755\n--- a/t/t9115-git-svn-dcommit-funky-renames.sh\n+++ b/t/t9115-git-svn-dcommit-funky-renames.sh\n@@ -60,7 +60,7 @@ test_expect_success 'add a file with plus signs' '\n \t'\n \n test_expect_success 'clone the repository to test rebase' '\n-\tgit svn clone \"$svnrepo\" test-rebase &&\n+\tgit svn clone $ref_storage_arg \"$svnrepo\" test-rebase &&\n \t(\n \t\tcd test-rebase &&\n \t\techo test-rebase >test-rebase &&\ndiff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh\nindex b5149fd..b36008c 100755\n--- a/t/t9350-fast-export.sh\n+++ b/t/t9350-fast-export.sh\n@@ -42,7 +42,7 @@ test_expect_success 'fast-export | fast-import' '\n \tWER=$(git rev-parse --verify wer) &&\n \tMUSS=$(git rev-parse --verify muss) &&\n \tmkdir new &&\n-\tgit --git-dir=new/.git init &&\n+\tgit --git-dir=new/.git init $ref_storage_arg &&\n \tgit fast-export --all |\n \t(cd new &&\n \t git fast-import &&\n@@ -158,7 +158,7 @@ test_expect_success 'setup submodule' '\n \tmkdir sub &&\n \t(\n \t\tcd sub &&\n-\t\tgit init &&\n+\t\tgit init $ref_storage_arg &&\n \t\techo test file > file &&\n \t\tgit add file &&\n \t\tgit commit -m sub_initial\n@@ -183,7 +183,7 @@ test_expect_success 'submodule fast-export | fast-import' '\n \tSUBENT2=$(git ls-tree master sub) &&\n \trm -rf new &&\n \tmkdir new &&\n-\tgit --git-dir=new/.git init &&\n+\tgit --git-dir=new/.git init $ref_storage_arg &&\n \tgit fast-export --signed-tags=strip --all |\n \t(cd new &&\n \t git fast-import &&\ndiff --git a/t/t9902-completion.sh b/t/t9902-completion.sh\nindex 2ba62fb..605816a 100755\n--- a/t/t9902-completion.sh\n+++ b/t/t9902-completion.sh\n@@ -126,7 +126,7 @@ actual=\"$TRASH_DIRECTORY/actual\"\n \n test_expect_success 'setup for __gitdir tests' '\n \tmkdir -p subdir/subsubdir &&\n-\tgit init otherrepo\n+\tgit init $ref_storage_arg otherrepo\n '\n \n test_expect_success '__gitdir - from command line (through $__git_dir)' '\n@@ -177,7 +177,7 @@ test_expect_success '__gitdir - cwd is a .git directory' '\n test_expect_success '__gitdir - parent is a .git directory' '\n \techo \"$(pwd -P)/.git\" >expected &&\n \t(\n-\t\tcd .git/refs/heads &&\n+\t\tcd .git/objects &&\n \t\t__gitdir >\"$actual\"\n \t) &&\n \ttest_cmp expected \"$actual\"\ndiff --git a/t/t9903-bash-prompt.sh b/t/t9903-bash-prompt.sh\nindex ffbfa0e..ae18e14 100755\n--- a/t/t9903-bash-prompt.sh\n+++ b/t/t9903-bash-prompt.sh\n@@ -148,7 +148,7 @@ test_expect_success 'prompt - inside .git directory' '\n test_expect_success 'prompt - deep inside .git directory' '\n \tprintf \" (GIT_DIR!)\" >expected &&\n \t(\n-\t\tcd .git/refs/heads &&\n+\t\tcd .git/objects &&\n \t\t__git_ps1 >\"$actual\"\n \t) &&\n \ttest_cmp expected \"$actual\"\ndiff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh\nindex 8d99eb3..9907201 100644\n--- a/t/test-lib-functions.sh\n+++ b/t/test-lib-functions.sh\n@@ -783,8 +783,8 @@ test_create_repo () {\n \trepo=\"$1\"\n \tmkdir -p \"$repo\"\n \t(\n-\t\tcd \"$repo\" || error \"Cannot setup test environment\"\n-\t\t\"$GIT_EXEC_PATH/git-init\" \"--template=$GIT_BUILD_DIR/templates/blt/\" >&3 2>&4 ||\n+\t cd \"$repo\" || error \"Cannot setup test environment\"\n+\t\t\"$GIT_EXEC_PATH/git-init\" $ref_storage_arg \"--template=$GIT_BUILD_DIR/templates/blt/\" >&3 2>&4 ||\n \t\terror \"cannot run git init -- have you built things yet?\"\n \t\tmv .git/hooks .git/hooks-disabled\n \t) || exit\n@@ -941,3 +941,52 @@ mingw_read_file_strip_cr_ () {\n \t\teval \"$1=\\$$1\\$line\"\n \tdone\n }\n+\n+raw_ref() {\n+ if test $ref_storage = \"lmdb\"\n+ then\n+\ttest-refs-lmdb-backend \"$1\" 2>/dev/null\n+ else\n+\tcat \".git/$1\"\n+ fi\n+}\n+\n+delete_ref() {\n+ if test $ref_storage = \"lmdb\"\n+ then\n+\ttest-refs-lmdb-backend -d \"$1\" 2>/dev/null\n+ else\n+\trm \".git/$1\"\n+ fi\n+}\n+\n+write_ref() {\n+ if test $ref_storage = \"lmdb\"\n+ then\n+\ttest-refs-lmdb-backend \"$1\" \"$2\" 2>/dev/null\n+ else\n+\techo \"$2\" > \".git/$1\"\n+ fi\n+}\n+\n+raw_reflog() {\n+ if test $ref_storage = \"lmdb\"\n+ then\n+\ttest-refs-lmdb-backend -l \"$1\" 2>/dev/null\n+ else\n+\tcat \".git/logs/$1\"\n+ fi\n+}\n+\n+delete_all_reflogs() {\n+ if test $ref_storage = \"lmdb\"\n+ then\n+\ttest-refs-lmdb-backend -c\n+ fi\n+ # We have to do this in any case to handle logs for per-worktree refs\n+ rm -rf .git/logs\n+}\n+\n+append_reflog() {\n+\ttest-refs-lmdb-backend -a \"$1\"\n+}\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex ce40770..e736554 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -186,6 +186,8 @@ test \"x$TERM\" != \"xdumb\" && (\n \t) &&\n \tcolor=t\n \n+ref_storage=\"files\"\n+ref_storage_arg=\n while test \"$#\" -ne 0\n do\n \tcase \"$1\" in\n@@ -201,6 +203,9 @@ do\n \t\t\texit 1;\n \t\t}\n \t\trun_list=$1; shift ;;\n+\t--ref-storage=*)\n+\t GIT_TEST_REF_STORAGE=$(expr \"z$1\" : 'z[^=]*=\\(.*\\)'); shift\n+\t ;;\n \t--run=*)\n \t\trun_list=$(expr \"z$1\" : 'z[^=]*=\\(.*\\)'); shift ;;\n \t-h|--h|--he|--hel|--help)\n@@ -247,6 +252,12 @@ do\n \tesac\n done\n \n+if test -n \"$GIT_TEST_REF_STORAGE\"\n+then\n+\tref_storage=$GIT_TEST_REF_STORAGE\n+\tref_storage_arg=\"--ref-storage=$ref_storage\"\n+fi\n+\n if test -n \"$valgrind_only\"\n then\n \ttest -z \"$valgrind\" && valgrind=memcheck\n-- \n2.4.2.767.g62658d5-twtrsrc\n"},{"id":"279919","messageId":"CACsJy8AUqB_1XZweTmAV4atDeUWZJhzEJoY2t9nJqp3CNcViXA@mail.gmail.com","threadId":"41574","inReplyTo":"1456793586-22082-32-git-send-email-dturner@twopensource.com","subject":"Re: [PATCH v7 31/33] refs: add LMDB refs storage backend","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2016-03-01T01:31:05Z","receivedAt":"2016-03-01T01:31:05Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Mar 1, 2016 at 7:53 AM, David Turner wrote:\n> +Weaknesses:\n> +-----------\n> +\n> +The reflog format is somewhat inefficient: a binary format could store\n> +reflog date/time information in somewhat less space.\n\nThis raises an interesting question. What if we want to change lmdb\nformat in future (e.g. to address this weakness)? Do we need some sort\nof versioning in lmdb database? I suppose you can always add \"lmdb2\"\nbackend that shares most of the code with current lmdb backend. Not\nsure if that's what you had in mind though.\n--\nDuy\n"},{"id":"279920","messageId":"1456796138.18017.77.camel@twopensource.com","threadId":"41574","inReplyTo":"CACsJy8AUqB_1XZweTmAV4atDeUWZJhzEJoY2t9nJqp3CNcViXA@mail.gmail.com","subject":"Re: [PATCH v7 31/33] refs: add LMDB refs storage backend","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T01:35:38Z","receivedAt":"2016-03-01T01:35:38Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"On Tue, 2016-03-01 at 08:31 +0700, Duy Nguyen wrote:\n> On Tue, Mar 1, 2016 at 7:53 AM, David Turner <\n> dturner@twopensource.com> wrote:\n> > +Weaknesses:\n> > +-----------\n> > +\n> > +The reflog format is somewhat inefficient: a binary format could\n> > store\n> > +reflog date/time information in somewhat less space.\n> \n> This raises an interesting question. What if we want to change lmdb\n> format in future (e.g. to address this weakness)? Do we need some\n> sort\n> of versioning in lmdb database? I suppose you can always add \"lmdb2\"\n> backend that shares most of the code with current lmdb backend. Not\n> sure if that's what you had in mind though.\n\nThe weakness of versioning inside the LMDB database is that in order to\ncheck the version, you need to already have LMDB. That's the argument\nfor \"lmdb2\". \n\nI had sort of hoped that this version would be \"good enough\" that we\ncould avoid modifying it. So maybe that means we ought to address the\nreflog format. But I'm not sure that it's that big a deal. What do\nyou think?\n"},{"id":"279922","messageId":"CACsJy8A8rwcKFb_fH2BZN=nuqMdhA2u8xCP4fN0KZmDAczQPog@mail.gmail.com","threadId":"41574","inReplyTo":"1456796138.18017.77.camel@twopensource.com","subject":"Re: [PATCH v7 31/33] refs: add LMDB refs storage backend","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2016-03-01T01:45:50Z","receivedAt":"2016-03-01T01:45:50Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Mar 1, 2016 at 8:35 AM, David Turner wrote:\n> On Tue, 2016-03-01 at 08:31 +0700, Duy Nguyen wrote:\n>> On Tue, Mar 1, 2016 at 7:53 AM, David Turner <\n>> dturner@twopensource.com> wrote:\n>> > +Weaknesses:\n>> > +-----------\n>> > +\n>> > +The reflog format is somewhat inefficient: a binary format could\n>> > store\n>> > +reflog date/time information in somewhat less space.\n>>\n>> This raises an interesting question. What if we want to change lmdb\n>> format in future (e.g. to address this weakness)? Do we need some\n>> sort\n>> of versioning in lmdb database? I suppose you can always add \"lmdb2\"\n>> backend that shares most of the code with current lmdb backend. Not\n>> sure if that's what you had in mind though.\n>\n> The weakness of versioning inside the LMDB database is that in order to\n> check the version, you need to already have LMDB. That's the argument\n> for \"lmdb2\".\n>\n> I had sort of hoped that this version would be \"good enough\" that we\n> could avoid modifying it. So maybe that means we ought to address the\n> reflog format. But I'm not sure that it's that big a deal. What do\n> you think?\n\nI'm not using lmdb yet (can't because if I convert my git.git to use\nit, all topics I'm working on must be rebased on lmdb :-( ) so I don't\nhave any strong opinion about this. As long as we have an escape hatch\n(that won't lead to a too messy future) I think we're good.\n\nNote that storing in binary also has disadvantage. If we move from\nSHA-1 to SHA-XXX in future, we can easily detect incorrect reflog line\nin text form, it's a bit harder to do in binary form when you pack\neverything (including some variable stuff like email) in one value.\nBut I think we don't have to care about that right now, there will be\nlots of problems migrating to SHA-XXX anyway.\n-- \nDuy\n"},{"id":"279941","messageId":"20160301083535.GA4952@sigill.intra.peff.net","threadId":"41574","inReplyTo":"1456793586-22082-2-git-send-email-dturner@twopensource.com","subject":"Re: [PATCH v7 01/33] setup: call setup_git_directory_gently before accessing refs","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-03-01T08:35:35Z","receivedAt":"2016-03-01T08:35:35Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Feb 29, 2016 at 07:52:34PM -0500, David Turner wrote:\n\n> Usually, git calls some form of setup_git_directory at startup. But\n> sometimes, it doesn't. Usually, that's OK because it's not really\n> using the repository. But in some cases, it is using the repo. In\n> those cases, either setup_git_directory_gently must be called, or the\n> repository (e.g. the refs) must not be accessed.\n\nIt's actually not just setup_git_directory(). We can also use\ncheck_repository_format(), which is used by enter_repo() (and hence by\nthings like upload-pack). I think the rule really ought to be: if we\ndidn't have check_repository_format_gently() tell us we have a valid\nrepo, we should not access any repo elements (refs, objects, etc).\n\nI started earlier today on a patch series to identify and fix these\ncases independent of your series. The basic strategy was to adapt the\nexisting \"struct startup_info\" to be available everywhere, and have\nrelevant bits of code assert() on it, or even behave differently (e.g.,\nif some library code should do different things in a repo versus not).\n\nBut I think we can probably just scrap the assert() part of that. The\nassertions I put in were unsurprisingly at the entry points to the ref\ncode. And your series supersedes that; we can't do anything with the\nrefs until the ref backend is setup, and if we only do so in\ncheck_repository_format_gently(), then it amounts to the same thing.\n\nFor the \"behave differently\" part, I needed it for the .mailmap case,\nbut you fixed it below without having to add that.\n\nI think it's worth going through the changes here and comparing notes\nwith what my series would have done.\n\n> diff --git a/builtin/grep.c b/builtin/grep.c\n> index 9e3f1cf..1e36b52 100644\n> --- a/builtin/grep.c\n> +++ b/builtin/grep.c\n> @@ -531,6 +531,7 @@ static int grep_directory(struct grep_opt *opt, const struct pathspec *pathspec,\n> \tif (exc_std)\n> \t\tsetup_standard_excludes(&dir);\n> \n> +\tdir.flags |= DIR_NO_GITLINKS;\n> \tfill_directory(&dir, pathspec);\n> \tfor (i = 0; i < dir.nr; i++) {\n> \t\tif (!dir_path_match(dir.entries[i], pathspec, 0, NULL))\n\nThis one is interesting, because the ref access in fill_directory() is\nonly for hitting submodule refs. In theory, I guess a command operating\nin a non-repo could want to know about and do something with embedded\ngit repos.\n\nAnd indeed, it does produce a behavior change here. With a repo like:\n\n mkdir non-repo && cd non-repo &&\n git init sub &&\n (cd sub && echo foo >file && git add . && git commit -m foo)\n\nrunning:\n\n git grep --no-index foo\n\ndoes not currently find sub/file (because it does not descend into what\nit think is a sub-repository), but it _does_ with your patch. I'm\ninclined to say that's actually a behavior improvement. \"grep\n--no-index\" on a directory is about behaving as a recursive grep, and\nshould probably descend into sub-repos (it probably should also avoid\nlooking inside .git directories, though, and I think it still does, even\nwith your patch).\n\nThe fill_directory() also touches the_index, which it should not in a\nnon-repository. But I think that's probably OK, because we simply don't\nread the index in the first place (so it behaves naturally as if the\nindex is empty).\n\n> diff --git a/builtin/log.c b/builtin/log.c\n> index 0d738d6..1d0e43e 100644\n> --- a/builtin/log.c\n> +++ b/builtin/log.c\n> @@ -975,7 +975,7 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,\n> \n> \tstrbuf_release(&sb);\n> \n> -\tshortlog_init(&log);\n> +\tshortlog_init(&log, 0);\n> \tlog.wrap_lines = 1;\n> \tlog.wrap = 72;\n> \tlog.in1 = 2;\n\nThis looks right. If we are making a cover letter for format-patch, we\nknow we have a repo, and thus nongit is always 0. Though I admit the\ndouble-negating confused me for a minute. I don't know if there's a way\naround it, though, because \"nongit\" is what comes out of\nsetup_git_directory().\n\n> diff --git a/builtin/shortlog.c b/builtin/shortlog.c\n> index bfc082e..ab4305b 100644\n> --- a/builtin/shortlog.c\n> +++ b/builtin/shortlog.c\n> @@ -219,11 +219,12 @@ static int parse_wrap_args(const struct option *opt, const char *arg, int unset)\n> \treturn 0;\n> }\n> \n> -void shortlog_init(struct shortlog *log)\n> +void shortlog_init(struct shortlog *log, int nongit)\n> {\n> \tmemset(log, 0, sizeof(*log));\n> \n> -\tread_mailmap(&log->mailmap, &log->common_repo_prefix);\n> +\tif (!nongit)\n> +\t\tread_mailmap(&log->mailmap, &log->common_repo_prefix);\n\nMy fix for this was to teach read_mailmap to avoid looking for\nHEAD:.mailmap if we are not in a repository, but to continue with the\nothers (.mailmap in the cwd, and the mailmap.file config variable).\n\nYours disables the .mailmap entirely. That makes some sense for looking\nat \".mailmap\" in the working tree; if we do not have a repository, we\nshould not look at a mailmap (though I guess you could argue the\nopposite, that a .mailmap in the current directory of a non-repo is\nworth looking at). But I'd think the mailmap.file config would apply\neven to shortlog invoked outside a repository.\n\nTo be perfectly honest, I cannot imagine that shortlog is invoked with\ndata on stdin much at all these days, let alone outside of a repository.\nBut I do think your patch is a potential regression there, if anybody\ndoes do that.\n\n> diff --git a/git.c b/git.c\n> index 6cc0c07..51e0508 100644\n> --- a/git.c\n> +++ b/git.c\n> @@ -376,7 +376,7 @@ static struct cmd_struct commands[] = {\n> \t{ \"am\", cmd_am, RUN_SETUP | NEED_WORK_TREE },\n> \t{ \"annotate\", cmd_annotate, RUN_SETUP },\n> \t{ \"apply\", cmd_apply, RUN_SETUP_GENTLY },\n> -\t{ \"archive\", cmd_archive },\n> +\t{ \"archive\", cmd_archive, RUN_SETUP_GENTLY },\n> \t{ \"bisect--helper\", cmd_bisect__helper, RUN_SETUP },\n> \t{ \"blame\", cmd_blame, RUN_SETUP },\n> \t{ \"branch\", cmd_branch, RUN_SETUP },\n\nI didn't have to touch this case in my experimenting. I wonder if it's\nbecause I resolved the \"grep\" case a little differently.\n\nI taught get_ref_cache() to only assert() that we have a repository when\nwe are looking at the main ref-cache, not a submodule. In theory, we can\nlook at a submodule from inside an outer non-repo (it's not really a\nsubmodule then, but just a plain git dir). I don't think there's\nanything in git right now that says you can't do so, though I think your\nrefs-backend work does introduce that restriction (because it actually\nrequires the submodules to use the same backend).\n\nSo with that requirement, I think we do need to require a repo even to\naccess submodule refs. Is that what triggered this change?\n\nI'd think you would need a matching line inside cmd_archive, too. It\nshould allow \"--remote\" without a repo, but generating a local archive\ndoes need one. And indeed, I see in write_archive() that we run\nsetup_git_repository ourselves, and die if we're not in a git repo. So\nI'm puzzled about which code path accesses the refs.\n\n> diff --git a/test-match-trees.c b/test-match-trees.c\n> index 109f03e..4dad709 100644\n> --- a/test-match-trees.c\n> +++ b/test-match-trees.c\n> @@ -6,6 +6,8 @@ int main(int ac, char **av)\n> \tunsigned char hash1[20], hash2[20], shifted[20];\n> \tstruct tree *one, *two;\n> \n> +\tsetup_git_directory();\n> +\n> \tif (get_sha1(av[1], hash1))\n> \t\tdie(\"cannot parse %s as an object name\", av[1]);\n> \tif (get_sha1(av[2], hash2))\n\nThis one is weird. The test-match-trees program is only used one time in\nour test suite, and then it is only as a hack because it is an external\nthat does not have startup_info setup. I think that test is somewhat\nbogus (and is obsoleted by my approach), and we could probably get rid\nof this program entirely.\n\nBut your patch here is certainly the right thing to do if we are keeping\nit.\n\n> diff --git a/test-revision-walking.c b/test-revision-walking.c\n> index 285f06b..3d03133 100644\n> --- a/test-revision-walking.c\n> +++ b/test-revision-walking.c\n> @@ -50,6 +50,8 @@ int main(int argc, char **argv)\n> \tif (argc < 2)\n> \t\treturn 1;\n> \n> +\tsetup_git_directory();\n> +\n> \tif (!strcmp(argv[1], \"run-twice\")) {\n> \t\tprintf(\"1st\\n\");\n> \t\tif (!run_revision_walk())\n\nThis one I solved in the same way. Yay, we agreed on one! :)\n\n-Peff\n"},{"id":"279942","messageId":"20160301084830.GB4952@sigill.intra.peff.net","threadId":"41574","inReplyTo":"1456793586-22082-30-git-send-email-dturner@twopensource.com","subject":"Re: [PATCH v7 29/33] setup: configure ref storage on setup","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-03-01T08:48:30Z","receivedAt":"2016-03-01T08:48:30Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Feb 29, 2016 at 07:53:02PM -0500, David Turner wrote:\n\n> diff --git a/setup.c b/setup.c\n> index bd3a2cf..e2e1220 100644\n> --- a/setup.c\n> +++ b/setup.c\n> @@ -457,6 +457,10 @@ static int check_repository_format_gently(const char *gitdir, int *nongit_ok)\n> \t\tret = -1;\n> \t}\n> \n> +\tregister_ref_storage_backends();\n> +\tif (set_ref_storage_backend(ref_storage_backend))\n> +\t\tdie(_(\"Unknown ref storage backend %s\"), ref_storage_backend);\n> +\n> \tstrbuf_release(&sb);\n> \treturn ret;\n> }\n\nMuch nicer than the one it replaces, I think.\n\nThis whole block should probably go inside\n\n if (ret == 0) {\n ...\n }\n\nIf we are doing setup_git_repository_gently() and we do _not_ find a\nvalid repository, we would not want to enable the ref storage.\n\nI also wondered what happens to ref_storage_backend in a case where we\ncall check_repository_format_gently() multiple times. I think we don't\nactually call it at all for submodules, so we at least we know we are\nalways dealing with the main repository.\n\nBut what if we call check_repository_format_gently(), find an\nextensions.refstorage config key, set the global, but then _don't_\nactually accept the repo (e.g., because its version is too high). Then\nwe keep looking and find another repo, which does not have\nextensions.refstorage set. But we use the stale value from the first\ndirectory.\n\nI admit this is a pretty unlikely scenario. But I think it does point to\na mis-design in the way we read extensions in the config callback. They\nshould not go into globals until we're sure we're accepting this config\nas the actual repository. So the existing \"preciousobjects\" extension\nhas this problem, too.\n\n-Peff\n"},{"id":"279969","messageId":"20160301145048.GK12887@sigill.intra.peff.net","threadId":"41574","inReplyTo":"20160301084830.GB4952@sigill.intra.peff.net","subject":"Re: [PATCH v7 29/33] setup: configure ref storage on setup","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-03-01T14:50:48Z","receivedAt":"2016-03-01T14:50:48Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Mar 01, 2016 at 03:48:30AM -0500, Jeff King wrote:\n\n> On Mon, Feb 29, 2016 at 07:53:02PM -0500, David Turner wrote:\n> \n> > diff --git a/setup.c b/setup.c\n> > index bd3a2cf..e2e1220 100644\n> > --- a/setup.c\n> > +++ b/setup.c\n> > @@ -457,6 +457,10 @@ static int check_repository_format_gently(const char *gitdir, int *nongit_ok)\n> > \t\tret = -1;\n> > \t}\n> > \n> > +\tregister_ref_storage_backends();\n> > +\tif (set_ref_storage_backend(ref_storage_backend))\n> > +\t\tdie(_(\"Unknown ref storage backend %s\"), ref_storage_backend);\n> > +\n> > \tstrbuf_release(&sb);\n> > \treturn ret;\n> > }\n> \n> Much nicer than the one it replaces, I think.\n> \n> This whole block should probably go inside\n> \n> if (ret == 0) {\n> ...\n> }\n> \n> If we are doing setup_git_repository_gently() and we do _not_ find a\n> valid repository, we would not want to enable the ref storage.\n\nSo in the new world order of the patch series I just posted, this would\nprobably look like:\n\n 1. Add a ref_backend string to \"struct repository_format\", and parse it\n in the callback.\n\n 2. The bottom of check_repository_format_gently() is only reached when\n we have a workable repo. So from there, you can:\n\n set_ref_storage_backend(candidate.ref_backend);\n\n I think you'd probably want to check nongit_ok before dying on\n failure (even without my patches).\n\n 3. Elsewhere, you can use read_repository_format() to get the backend\n speculatively (e.g., for submodules), rather than doing a custom\n git_config_from_file invocation.\n\nNone of which is to say that building on my series is a foregone\nconclusion; I just wanted to point you in the right direction if you do\nwant to.\n\n-Peff\n"},{"id":"279980","messageId":"56D5CECD.6020301@ramsayjones.plus.com","threadId":"41574","inReplyTo":"1456793586-22082-30-git-send-email-dturner@twopensource.com","subject":"Re: [PATCH v7 29/33] setup: configure ref storage on setup","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2016-03-01T17:18:05Z","receivedAt":"2016-03-01T17:18:05Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"\n\nOn 01/03/16 00:53, David Turner wrote:\n> This sets up the existing backend early, so that other code which\n> reads refs is reading from the right place.\n> \n> Signed-off-by: David Turner \n> Signed-off-by: Junio C Hamano \n> ---\n> config.c | 1 +\n> setup.c | 4 ++++\n> 2 files changed, 5 insertions(+)\n> \n> diff --git a/config.c b/config.c\n> index 9ba40bc..cca7e28 100644\n> --- a/config.c\n> +++ b/config.c\n> @@ -11,6 +11,7 @@\n> #include \"strbuf.h\"\n> #include \"quote.h\"\n> #include \"hashmap.h\"\n> +#include \"refs.h\"\n> #include \"string-list.h\"\n> #include \"utf8.h\"\n> \n\nI was just skimming these patches as they passed by, and this\ncaught my eye. If this include is required (eg. to fix a compiler\nwarning), then it should probably be in an earlier patch.\nOtherwise, it should be in a later patch, no?\n\nATB,\nRamsay Jones\n\n> diff --git a/setup.c b/setup.c\n> index bd3a2cf..e2e1220 100644\n> --- a/setup.c\n> +++ b/setup.c\n> @@ -457,6 +457,10 @@ static int check_repository_format_gently(const char *gitdir, int *nongit_ok)\n> \t\tret = -1;\n> \t}\n> \n> +\tregister_ref_storage_backends();\n> +\tif (set_ref_storage_backend(ref_storage_backend))\n> +\t\tdie(_(\"Unknown ref storage backend %s\"), ref_storage_backend);\n> +\n> \tstrbuf_release(&sb);\n> \treturn ret;\n> }\n> \n"},{"id":"279981","messageId":"56D5CF9C.4010808@ramsayjones.plus.com","threadId":"41574","inReplyTo":"1456793586-22082-31-git-send-email-dturner@twopensource.com","subject":"Re: [PATCH v7 30/33] refs: break out resolve_ref_unsafe_submodule","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2016-03-01T17:21:32Z","receivedAt":"2016-03-01T17:21:32Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"\n\nOn 01/03/16 00:53, David Turner wrote:\n> It will soon be useful for resolve_ref_unsafe to support submodules.\n> But since it is called from so many places, changing it would have\n> been painful. Fortunately, it's just a thin wrapper around (the\n> former) resolve_ref_1. So now resolve_ref_1 becomes\n> resolve_ref_unsafe_submodule, and it passes its submodule argument\n> through to read_raw_ref.\n> \n> The files backend doesn't need this functionality, but it doesn't\n> hurt.\n> \n> Signed-off-by: David Turner \n> Signed-off-by: Junio C Hamano \n> ---\n> refs.c | 41 +++++++++++++++++++++++++----------------\n> refs/files-backend.c | 8 ++++++--\n> refs/refs-internal.h | 19 ++++++++++++++++---\n> 3 files changed, 47 insertions(+), 21 deletions(-)\n> \n> diff --git a/refs.c b/refs.c\n> index 5fe0bac..d1cf707 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -60,6 +60,9 @@ void register_ref_storage_backends(void)\n> \t * entries below when you add a new backend.\n> \t */\n> \tregister_ref_storage_backend(&refs_be_files);\n> +#ifdef USE_LIBLMDB\n> +\tregister_ref_storage_backend(&refs_be_lmdb);\n> +#endif\n\nAgain, just skimming patches, ...\n\nThe lmdb refs backend (hence refs_be_lmdb) is not introduced until\nthe next patch [31/33], right?\n\nATB,\nRamsay Jones\n"},{"id":"279995","messageId":"1456859791.18017.78.camel@twopensource.com","threadId":"41574","inReplyTo":"56D5CECD.6020301@ramsayjones.plus.com","subject":"Re: [PATCH v7 29/33] setup: configure ref storage on setup","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T19:16:31Z","receivedAt":"2016-03-01T19:16:31Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"On Tue, 2016-03-01 at 17:18 +0000, Ramsay Jones wrote:\n> \n> On 01/03/16 00:53, David Turner wrote:\n> > This sets up the existing backend early, so that other code which\n> > reads refs is reading from the right place.\n> > \n> > Signed-off-by: David Turner \n> > Signed-off-by: Junio C Hamano \n> > ---\n> > config.c | 1 +\n> > setup.c | 4 ++++\n> > 2 files changed, 5 insertions(+)\n> > \n> > diff --git a/config.c b/config.c\n> > index 9ba40bc..cca7e28 100644\n> > --- a/config.c\n> > +++ b/config.c\n> > @@ -11,6 +11,7 @@\n> > #include \"strbuf.h\"\n> > #include \"quote.h\"\n> > #include \"hashmap.h\"\n> > +#include \"refs.h\"\n> > #include \"string-list.h\"\n> > #include \"utf8.h\"\n> > \n> \n> I was just skimming these patches as they passed by, and this\n> caught my eye. If this include is required (eg. to fix a compiler\n> warning), then it should probably be in an earlier patch.\n> Otherwise, it should be in a later patch, no?\n\nActually, it's cruft from the previous version of this series :(. I\nlooked at the patch and didn't notice that it was in config.c instead\nof setup.c. Oops. Will remove.\n"},{"id":"279996","messageId":"1456859859.18017.79.camel@twopensource.com","threadId":"41574","inReplyTo":"56D5CF9C.4010808@ramsayjones.plus.com","subject":"Re: [PATCH v7 30/33] refs: break out resolve_ref_unsafe_submodule","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T19:17:39Z","receivedAt":"2016-03-01T19:17:39Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"On Tue, 2016-03-01 at 17:21 +0000, Ramsay Jones wrote:\n> \n> On 01/03/16 00:53, David Turner wrote:\n> > It will soon be useful for resolve_ref_unsafe to support\n> > submodules.\n> > But since it is called from so many places, changing it would have\n> > been painful. Fortunately, it's just a thin wrapper around (the\n> > former) resolve_ref_1. So now resolve_ref_1 becomes\n> > resolve_ref_unsafe_submodule, and it passes its submodule argument\n> > through to read_raw_ref.\n> > \n> > The files backend doesn't need this functionality, but it doesn't\n> > hurt.\n> > \n> > Signed-off-by: David Turner \n> > Signed-off-by: Junio C Hamano \n> > ---\n> > refs.c | 41 +++++++++++++++++++++++++---------------\n> > -\n> > refs/files-backend.c | 8 ++++++--\n> > refs/refs-internal.h | 19 ++++++++++++++++---\n> > 3 files changed, 47 insertions(+), 21 deletions(-)\n> > \n> > diff --git a/refs.c b/refs.c\n> > index 5fe0bac..d1cf707 100644\n> > --- a/refs.c\n> > +++ b/refs.c\n> > @@ -60,6 +60,9 @@ void register_ref_storage_backends(void)\n> > \t * entries below when you add a new backend.\n> > \t */\n> > \tregister_ref_storage_backend(&refs_be_files);\n> > +#ifdef USE_LIBLMDB\n> > +\tregister_ref_storage_backend(&refs_be_lmdb);\n> > +#endif\n> \n> Again, just skimming patches, ...\n> \n> The lmdb refs backend (hence refs_be_lmdb) is not introduced until\n> the next patch [31/33], right?\n\nYep.\n"},{"id":"280022","messageId":"1456876072.5981.5.camel@twopensource.com","threadId":"41574","inReplyTo":"20160301083535.GA4952@sigill.intra.peff.net","subject":"Re: [PATCH v7 01/33] setup: call setup_git_directory_gently before accessing refs","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-01T23:47:52Z","receivedAt":"2016-03-01T23:47:52Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"On Tue, 2016-03-01 at 03:35 -0500, Jeff King wrote:\n> On Mon, Feb 29, 2016 at 07:52:34PM -0500, David Turner wrote:\n> \n> > Usually, git calls some form of setup_git_directory at startup. \n> > But\n> > sometimes, it doesn't. Usually, that's OK because it's not really\n> > using the repository. But in some cases, it is using the repo. In\n> > those cases, either setup_git_directory_gently must be called, or\n> > the\n> > repository (e.g. the refs) must not be accessed.\n> \n> It's actually not just setup_git_directory(). We can also use\n> check_repository_format(), which is used by enter_repo() (and hence\n> by\n> things like upload-pack). I think the rule really ought to be: if we\n> didn't have check_repository_format_gently() tell us we have a valid\n> repo, we should not access any repo elements (refs, objects, etc).\n\nI'll change that commit message to say\n\"check_repository_format_gently\".\n\n> > diff --git a/builtin/grep.c b/builtin/grep.\n> [snip: this is a probably-good behavior change]\n\nAgreed.\n\n> My fix for this was to teach read_mailmap to avoid looking for\n> HEAD:.mailmap if we are not in a repository, but to continue with the\n> others (.mailmap in the cwd, and the mailmap.file config variable).\n> ...\n> But I do think your patch is a potential regression there, if anybody\n> does do that.\n\nYour version sounds better. But I don't see it in the patch set you\nsent earlier? \n\n> > diff --git a/git.c b/git.c\n> > index 6cc0c07..51e0508 100644\n> > --- a/git.c\n> > +++ b/git.c\n> > @@ -376,7 +376,7 @@ static struct cmd_struct commands[] = {\n> > \t{ \"am\", cmd_am, RUN_SETUP | NEED_WORK_TREE },\n> > \t{ \"annotate\", cmd_annotate, RUN_SETUP },\n> > \t{ \"apply\", cmd_apply, RUN_SETUP_GENTLY },\n> > -\t{ \"archive\", cmd_archive },\n> > +\t{ \"archive\", cmd_archive, RUN_SETUP_GENTLY },\n> > \t{ \"bisect--helper\", cmd_bisect__helper, RUN_SETUP },\n> > \t{ \"blame\", cmd_blame, RUN_SETUP },\n> > \t{ \"branch\", cmd_branch, RUN_SETUP },\n> \n> I didn't have to touch this case in my experimenting. I wonder if\n> it's\n> because I resolved the \"grep\" case a little differently.\n>\n> I taught get_ref_cache() to only assert() that we have a repository\n> when\n> we are looking at the main ref-cache, not a submodule. In theory, we\n> can\n> look at a submodule from inside an outer non-repo (it's not really a\n> submodule then, but just a plain git dir). I don't think there's\n> anything in git right now that says you can't do so, though I think\n> your\n> refs-backend work does introduce that restriction (because it\n> actually\n> requires the submodules to use the same backend).\n> \n> So with that requirement, I think we do need to require a repo even\n> to\n> access submodule refs. Is that what triggered this change?\n\nNo. What triggered this change was a test failure with your earlier\npatch on master -- none of my stuff at all. The failing command was:\n\ngit archive --remote=. HEAD\n\nWhen writing my patch, I had assumed that the issue was the resolve_ref\non the HEAD that's an argument -- but it's not. The actual traceback\nis:\n\n#0 die (\n err=err@entry=0x57ddb0 \"BUG: resolve_ref called without\ninitializing repo\") at usage.c:99\n#1 0x00000000004f7ed9 in resolve_ref_1 (sb_refname=0x7c4a50\n, \n sb_contents=0x7fffffffcfc0, sb_path=0x7fffffffcfe0,\nflags=0x7fffffffdaaa, \n sha1=0x7fffffffd100 \"\\b\\326\\377\\377\\377\\177\",\nresolve_flags=5572384, \n refname=0x2 )\n at refs/files-backend.c:1429\n#2 resolve_ref_unsafe (refname=refname@entry=0x550b3b \"HEAD\", \n resolve_flags=resolve_flags@entry=0, \n sha1=sha1@entry=0x7fffffffd100 \"\\b\\326\\377\\377\\377\\177\", \n flags=flags@entry=0x7fffffffd0fc) at refs/files-backend.c:1600\n#3 0x00000000004ffe69 in read_config () at remote.c:471\n#4 0x0000000000500235 in read_config () at remote.c:705\n#5 remote_get_1 (name=0x7fffffffdaaa \".\", \n get_default=get_default@entry=0x4fe230 )\n at remote.c:688\n#6 0x00000000005004ca in remote_get (name=) at\nremote.c:713\n#7 0x00000000004159d8 in run_remote_archiver (name_hint=0x0, \n exec=0x550720 \"git-upload-archive\", remote=, \n argv=0x7fffffffd608, argc=2) at builtin/archive.c:35\n#8 cmd_archive (argc=2, argv=0x7fffffffd608, prefix=0x0)\n at builtin/archive.c:104\n#9 0x0000000000406051 in run_builtin (argv=0x7fffffffd608, argc=3, \n p=0x7bd7a0 ) at git.c:357\n#10 handle_builtin (argc=3, argv=0x7fffffffd608) at git.c:540\n#11 0x000000000040519a in main (argc=3, av=) at\ngit.c:671\n\n> I'd think you would need a matching line inside cmd_archive, too. It\n> should allow \"--remote\" without a repo, but generating a local\n> archive\n> does need one. And indeed, I see in write_archive() that we run\n> setup_git_repository ourselves, and die if we're not in a git repo.\n> So\n> I'm puzzled about which code path accesses the refs.\n\nI agree that --remote should work without a repo, It seems that we do\nn't test this and we probably should. \n\nI'm not sure what the right way to fix this is -- in read_config, we're\nabout to access some stuff in a repo (config, HEAD). It's OK to skip\nthat stuff if we're not in a repo, but we don't want to run\nsetup_git_directory twice (that breaks some stuff), and some of the\nother callers have already called it. On top of your earlier\nrepo_initialized patch, we could add the following to read_config:\n\n+ if (!repo_initialized) {\n+ int nongit = 0;\n+ setup_git_directory_gently(&nongit);\n+ if (nongit)\n+ return;\n+ }\n\nBut that patch I think was not intended to be permanent. Still, it\ndoes seem odd that there's no straightforward way to know if the repo\nis initialized. Am I missing something? \n\n> > diff --git a/test-match-trees.c b/test-match-trees.c\n> But your patch here is certainly the right thing to do if we are \n> keeping it.\n\nLet's keep it for now; we could always remove it later.\n"},{"id":"280025","messageId":"1456878830.5981.7.camel@twopensource.com","threadId":"41574","inReplyTo":"1456876072.5981.5.camel@twopensource.com","subject":"Re: [PATCH v7 01/33] setup: call setup_git_directory_gently before accessing refs","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-02T00:33:50Z","receivedAt":"2016-03-02T00:33:50Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"On Tue, 2016-03-01 at 18:47 -0500, David Turner wrote:\n> On Tue, 2016-03-01 at 03:35 -0500, Jeff King wrote:\n> > On Mon, Feb 29, 2016 at 07:52:34PM -0500, David Turner wrote:\n> > \n> > > Usually, git calls some form of setup_git_directory at startup. \n> > > But\n> > > sometimes, it doesn't. Usually, that's OK because it's not\n> > > really\n> > > using the repository. But in some cases, it is using the repo. \n> > > In\n> > > those cases, either setup_git_directory_gently must be called, or\n> > > the\n> > > repository (e.g. the refs) must not be accessed.\n> > \n> > It's actually not just setup_git_directory(). We can also use\n> > check_repository_format(), which is used by enter_repo() (and hence\n> > by\n> > things like upload-pack). I think the rule really ought to be: if\n> > we\n> > didn't have check_repository_format_gently() tell us we have a\n> > valid\n> > repo, we should not access any repo elements (refs, objects, etc).\n> \n> I'll change that commit message to say\n> \"check_repository_format_gently\".\n> \n> > > diff --git a/builtin/grep.c b/builtin/grep.\n> > [snip: this is a probably-good behavior change]\n> \n> Agreed.\n> \n> > My fix for this was to teach read_mailmap to avoid looking for\n> > HEAD:.mailmap if we are not in a repository, but to continue with\n> > the\n> > others (.mailmap in the cwd, and the mailmap.file config variable).\n> > ...\n> > But I do think your patch is a potential regression there, if\n> > anybody\n> > does do that.\n> \n> Your version sounds better. But I don't see it in the patch set you\n> sent earlier? \n> \n> > > diff --git a/git.c b/git.c\n> > > index 6cc0c07..51e0508 100644\n> > > --- a/git.c\n> > > +++ b/git.c\n> > > @@ -376,7 +376,7 @@ static struct cmd_struct commands[] = {\n> > > \t{ \"am\", cmd_am, RUN_SETUP | NEED_WORK_TREE },\n> > > \t{ \"annotate\", cmd_annotate, RUN_SETUP },\n> > > \t{ \"apply\", cmd_apply, RUN_SETUP_GENTLY },\n> > > -\t{ \"archive\", cmd_archive },\n> > > +\t{ \"archive\", cmd_archive, RUN_SETUP_GENTLY },\n> > > \t{ \"bisect--helper\", cmd_bisect__helper, RUN_SETUP },\n> > > \t{ \"blame\", cmd_blame, RUN_SETUP },\n> > > \t{ \"branch\", cmd_branch, RUN_SETUP },\n> > \n> > I didn't have to touch this case in my experimenting. I wonder if\n> > it's\n> > because I resolved the \"grep\" case a little differently.\n> > \n> > I taught get_ref_cache() to only assert() that we have a repository\n> > when\n> > we are looking at the main ref-cache, not a submodule. In theory,\n> > we\n> > can\n> > look at a submodule from inside an outer non-repo (it's not really\n> > a\n> > submodule then, but just a plain git dir). I don't think there's\n> > anything in git right now that says you can't do so, though I think\n> > your\n> > refs-backend work does introduce that restriction (because it\n> > actually\n> > requires the submodules to use the same backend).\n> > \n> > So with that requirement, I think we do need to require a repo even\n> > to\n> > access submodule refs. Is that what triggered this change?\n> \n> No. What triggered this change was a test failure with your earlier\n> patch on master -- none of my stuff at all. The failing command was:\n> \n> git archive --remote=. HEAD\n> \n> When writing my patch, I had assumed that the issue was the\n> resolve_ref\n> on the HEAD that's an argument -- but it's not. The actual traceback\n> is:\n> \n> #0 die (\n> err=err@entry=0x57ddb0 \"BUG: resolve_ref called without\n> initializing repo\") at usage.c:99\n> #1 0x00000000004f7ed9 in resolve_ref_1 (sb_refname=0x7c4a50\n> , \n> sb_contents=0x7fffffffcfc0, sb_path=0x7fffffffcfe0,\n> flags=0x7fffffffdaaa, \n> sha1=0x7fffffffd100 \"\\b\\326\\377\\377\\377\\177\",\n> resolve_flags=5572384, \n> refname=0x2 )\n> at refs/files-backend.c:1429\n> #2 resolve_ref_unsafe (refname=refname@entry=0x550b3b \"HEAD\", \n> resolve_flags=resolve_flags@entry=0, \n> sha1=sha1@entry=0x7fffffffd100 \"\\b\\326\\377\\377\\377\\177\", \n> flags=flags@entry=0x7fffffffd0fc) at refs/files-backend.c:1600\n> #3 0x00000000004ffe69 in read_config () at remote.c:471\n> #4 0x0000000000500235 in read_config () at remote.c:705\n> #5 remote_get_1 (name=0x7fffffffdaaa \".\", \n> get_default=get_default@entry=0x4fe230 )\n> at remote.c:688\n> #6 0x00000000005004ca in remote_get (name=) at\n> remote.c:713\n> #7 0x00000000004159d8 in run_remote_archiver (name_hint=0x0, \n> exec=0x550720 \"git-upload-archive\", remote=, \n> argv=0x7fffffffd608, argc=2) at builtin/archive.c:35\n> #8 cmd_archive (argc=2, argv=0x7fffffffd608, prefix=0x0)\n> at builtin/archive.c:104\n> #9 0x0000000000406051 in run_builtin (argv=0x7fffffffd608, argc=3, \n> p=0x7bd7a0 ) at git.c:357\n> #10 handle_builtin (argc=3, argv=0x7fffffffd608) at git.c:540\n> #11 0x000000000040519a in main (argc=3, av=) at\n> git.c:671\n> \n> > I'd think you would need a matching line inside cmd_archive, too.\n> > It\n> > should allow \"--remote\" without a repo, but generating a local\n> > archive\n> > does need one. And indeed, I see in write_archive() that we run\n> > setup_git_repository ourselves, and die if we're not in a git repo.\n> > So\n> > I'm puzzled about which code path accesses the refs.\n> \n> I agree that --remote should work without a repo, It seems that we\n> do\n> n't test this and we probably should. \n> \n> I'm not sure what the right way to fix this is -- in read_config,\n> we're\n> about to access some stuff in a repo (config, HEAD). It's OK to skip\n> that stuff if we're not in a repo, but we don't want to run\n> setup_git_directory twice (that breaks some stuff), and some of the\n> other callers have already called it. On top of your earlier\n> repo_initialized patch, we could add the following to read_config:\n> \n> + if (!repo_initialized) {\n> + int nongit = 0;\n> + setup_git_directory_gently(&nongit);\n> + if (nongit)\n> + return;\n> + }\n> \n> But that patch I think was not intended to be permanent. Still, it\n> does seem odd that there's no straightforward way to know if the repo\n> is initialized. Am I missing something? \n\nI guess we could add a bit in startup_info. Was that what you were\ntalking about there?\n"},{"id":"280031","messageId":"20160302024509.GA20625@sigill.intra.peff.net","threadId":"41574","inReplyTo":"1456876072.5981.5.camel@twopensource.com","subject":"Re: [PATCH v7 01/33] setup: call setup_git_directory_gently before accessing refs","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-03-02T02:45:09Z","receivedAt":"2016-03-02T02:45:09Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Mar 01, 2016 at 06:47:52PM -0500, David Turner wrote:\n\n> > My fix for this was to teach read_mailmap to avoid looking for\n> > HEAD:.mailmap if we are not in a repository, but to continue with the\n> > others (.mailmap in the cwd, and the mailmap.file config variable).\n> > ...\n> > But I do think your patch is a potential regression there, if anybody\n> > does do that.\n> \n> Your version sounds better. But I don't see it in the patch set you\n> sent earlier? \n\nIt's not. Sorry to be unclear. There were _two_ cleanups I was talking\nabout (cases where we don't check whether we're in a repo, and fact that\nthe repo startup code is unreliable), and I got sucked into the second\none. I'll try to work up and share my startup_info one today.\n\n> When writing my patch, I had assumed that the issue was the resolve_ref\n> on the HEAD that's an argument -- but it's not. The actual traceback\n> is:\n> [...]\n> #2 resolve_ref_unsafe (refname=refname@entry=0x550b3b \"HEAD\", \n> resolve_flags=resolve_flags@entry=0, \n> sha1=sha1@entry=0x7fffffffd100 \"\\b\\326\\377\\377\\377\\177\", \n> flags=flags@entry=0x7fffffffd0fc) at refs/files-backend.c:1600\n> #3 0x00000000004ffe69 in read_config () at remote.c:471\n\nOh, right. I did see problems here but missed them when comparing my\npatch to yours. I ended up in remote.c:read_config, having it check\nwhether startup_info->have_repository is set; if it isn't, there is no\npoint in looking at HEAD.\n\nThat covers this case, and several others I happened across. Thanks for\nclarifying.\n\n> I'm not sure what the right way to fix this is -- in read_config, we're\n> about to access some stuff in a repo (config, HEAD). It's OK to skip\n> that stuff if we're not in a repo, but we don't want to run\n> setup_git_directory twice (that breaks some stuff), and some of the\n> other callers have already called it. On top of your earlier\n> repo_initialized patch, we could add the following to read_config:\n> \n> + if (!repo_initialized) {\n> + int nongit = 0;\n> + setup_git_directory_gently(&nongit);\n> + if (nongit)\n> + return;\n> + }\n> \n> But that patch I think was not intended to be permanent. Still, it\n> does seem odd that there's no straightforward way to know if the repo\n> is initialized. Am I missing something? \n\nNo, there isn't a straightforward way; I think we'll have to add one.\nI'll polish up my series which does this.\n\n-Peff\n"},{"id":"281258","messageId":"56EE2F0D.5030709@alum.mit.edu","threadId":"41574","inReplyTo":"1456793586-22082-5-git-send-email-dturner@twopensource.com","subject":"Re: [PATCH v7 04/33] files-backend: break out ref reading","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2016-03-20T05:03:09Z","receivedAt":"2016-03-20T05:03:09Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 03/01/2016 01:52 AM, David Turner wrote:\n> Refactor resolve_ref_1 in terms of a new function read_raw_ref, which\n> is responsible for reading ref data from the ref storage.\n> \n> Later, we will make read_raw_ref a pluggable backend function, and make\n> resolve_ref_unsafe common.\n> \n> Testing done: Hacked in code to run both old and new version of\n> resolve_ref_1 and compare all outputs, failing dramatically if outputs\n> differed. Ran test suite.\n\nI like that you are splitting up resolve_ref_1(), which was too\ncomplicated and convoluted before.\n\nThis is a textually large change and I'm still auditing it, but\nmeanwhile I have a question...\n\n> [...]\n> -\t\tif (--depth < 0) {\n> -\t\t\terrno = ELOOP;\n> -\t\t\treturn NULL;\n> -\t\t}\n\nThe old version set errno to ELOOP if there were too many layers of\nsymrefs. The new version doesn't seem to set errno at all in that case.\nI think that is a regression, though I might be misunderstanding something.\n\nMichael\n"},{"id":"281420","messageId":"56F10344.6000408@alum.mit.edu","threadId":"41574","inReplyTo":"56EE2F0D.5030709@alum.mit.edu","subject":"Re: [PATCH v7 04/33] files-backend: break out ref reading","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2016-03-22T08:33:08Z","receivedAt":"2016-03-22T08:33:08Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 03/20/2016 06:03 AM, Michael Haggerty wrote:\n> On 03/01/2016 01:52 AM, David Turner wrote:\n>> Refactor resolve_ref_1 in terms of a new function read_raw_ref, which\n>> is responsible for reading ref data from the ref storage.\n>>\n>> Later, we will make read_raw_ref a pluggable backend function, and make\n>> resolve_ref_unsafe common.\n>>\n>> Testing done: Hacked in code to run both old and new version of\n>> resolve_ref_1 and compare all outputs, failing dramatically if outputs\n>> differed. Ran test suite.\n\nI found some other subtle differences in the behavior after this patch.\nI'm still working out the details.\n\nMichael\n"},{"id":"281561","messageId":"56F26DB5.70708@alum.mit.edu","threadId":"41574","inReplyTo":"1456793586-22082-5-git-send-email-dturner@twopensource.com","subject":"Re: [PATCH v7 04/33] files-backend: break out ref reading","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2016-03-23T10:19:33Z","receivedAt":"2016-03-23T10:19:33Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 03/01/2016 01:52 AM, David Turner wrote:\n> Refactor resolve_ref_1 in terms of a new function read_raw_ref, which\n> is responsible for reading ref data from the ref storage.\n> \n> Later, we will make read_raw_ref a pluggable backend function, and make\n> resolve_ref_unsafe common.\n> \n> Testing done: Hacked in code to run both old and new version of\n> resolve_ref_1 and compare all outputs, failing dramatically if outputs\n> differed. Ran test suite.\n\nI spent an inordinate amount of time trying to review this patch. It is\nmuch too big and does too much at the same time. And, in fact, it makes\nsome mistakes, which were impossible to see until I picked the patch\napart into smaller steps.\n\nThe reference-reading code before this patch wasn't structured very\nwell. Part of the reason that the code is a mess is that it is pretty\nintricate and has to get things just right to avoid race conditions. But\nthe rest of the reason is that it was overdue for a refactoring, and\nthis patch shows a great way forward.\n\nBecause of the intricacy of this code, it is really important to do a\ncareful job changing it. To me that means refactoring in the smallest\npossible steps, ideally so that each step is obviously correct.\n\nSo to review your patch, I picked it apart into tiny preparatory\nrefactorings, followed by the main patch (the extraction of the function\nread_raw_ref()), followed by some more cleanups. When I did so I found\nthat there were some differences between my end product and yours. Some\nof these introduce minor bugs, so it is worth fixing them.\n\nI've annotated your patch below, but in my opinion a better way forward\nwould be to commit not this single giant patch, but rather the\npicked-apart version, which also addresses my comments below. I just\nsubmitted that patch series [1]. It's twenty-one patches(!), though a\nbit over half of them do things that go beyond this patch. In the\nfuture, it would help the review process if you would submit smaller\npatches that do a single thing at a time.\n\nI've reviewed the patches that precede this one and they look fine to\nme. I haven't reviewed this version of patches 05 through 33 yet.\n\nMichael\n\n[1] http://mid.gmane.org/cover.1458723959.git.mhagger@alum.mit.edu\n\n> Signed-off-by: David Turner \n> Helped-by: Duy Nguyen \n> Signed-off-by: Junio C Hamano \n> ---\n> refs/files-backend.c | 265 ++++++++++++++++++++++++++++++---------------------\n> 1 file changed, 159 insertions(+), 106 deletions(-)\n> \n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index 9676ec2..8c6a58e 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -1369,12 +1369,11 @@ static struct ref_entry *get_packed_ref(const char *refname)\n> \n> /*\n> * A loose ref file doesn't exist; check for a packed ref. The\n> - * options are forwarded from resolve_safe_unsafe().\n> + * options are forwarded from resolve_ref_unsafe().\n> */\n> static int resolve_missing_loose_ref(const char *refname,\n> -\t\t\t\t int resolve_flags,\n> \t\t\t\t unsigned char *sha1,\n> -\t\t\t\t int *flags)\n> +\t\t\t\t unsigned int *flags)\n\nThis patch changes some flags variables from \"int\" to \"unsigned int\"\nwithout discussion. I think the change is OK, but why not do it in a\nseparate patch?\n\n> {\n> \tstruct ref_entry *entry;\n> \n> @@ -1390,64 +1389,48 @@ static int resolve_missing_loose_ref(const char *refname,\n> \t\treturn 0;\n> \t}\n> \t/* The reference is not a packed reference, either. */\n> -\tif (resolve_flags & RESOLVE_REF_READING) {\n> -\t\terrno = ENOENT;\n> -\t\treturn -1;\n> -\t} else {\n> -\t\thashclr(sha1);\n> -\t\treturn 0;\n> -\t}\n> +\terrno = ENOENT;\n> +\treturn -1;\n> }\n> \n> -/* This function needs to return a meaningful errno on failure */\n> -static const char *resolve_ref_1(const char *refname,\n> -\t\t\t\t int resolve_flags,\n> -\t\t\t\t unsigned char *sha1,\n> -\t\t\t\t int *flags,\n> -\t\t\t\t struct strbuf *sb_refname,\n> -\t\t\t\t struct strbuf *sb_path,\n> -\t\t\t\t struct strbuf *sb_contents)\n> +/*\n> + * Read a raw ref from the filesystem or packed refs file.\n> + *\n> + * If the ref is a sha1, fill in sha1 and return 0.\n> + *\n> + * If the ref is symbolic, fill in *symref with the referrent\n> + * (e.g. \"refs/heads/master\") and return 0. The caller is responsible\n> + * for validating the referrent. Set REF_ISSYMREF in flags.\n> + *\n> + * If the ref is neither a symbolic ref nor a sha1, it is broken. Set\n> + * REF_ISBROKEN in flags, set errno to EINVAL, and return -1.\n> + *\n> + * If the ref doesn't exist, set errno to ENOENT and return -1.\n> + *\n> + * If there is another error reading the ref, set errno appropriately and\n> + * return -1.\n> + *\n> + * Backend-specific flags might be set in flags as well, regardless of\n> + * outcome.\n> + *\n> + * sb_path is workspace: the caller should allocate and free it.\n> + */\n> +static int read_raw_ref(const char *refname, unsigned char *sha1,\n> +\t\t\tstruct strbuf *symref, struct strbuf *sb_path,\n> +\t\t\tunsigned int *flags)\n> {\n> -\tint depth = MAXDEPTH;\n> -\tint bad_name = 0;\n> -\n> -\tif (flags)\n> -\t\t*flags = 0;\n> +\tstruct strbuf sb_contents = STRBUF_INIT;\n\nI see you've pushed responsibility for managing the scratch space\nsb_contents from resolve_ref_unsafe() down to read_raw_ref(). This seems\nlike a good idea to decouple the code better (albeit at the expense of\nsome extra memory allocations). But why don't you push responsibility\nfor managing sb_path down at the same time? It seems that the same\nconsiderations would apply.\n\nBTW if you do so, then resolve_ref_unsafe() becomes trivial and\nresolve_ref_1() could easily be inlined into it.\n\n> +\tint ret = -1;\n> +\tconst char *path;\n> +\tconst char *buf;\n> \n> -\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n> -\t\tif (flags)\n> -\t\t\t*flags |= REF_BAD_NAME;\n> +\tstrbuf_reset(sb_path);\n> +\tstrbuf_git_path(sb_path, \"%s\", refname);\n> +\tpath = sb_path->buf;\n> \n> -\t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n> -\t\t !refname_is_safe(refname)) {\n> -\t\t\terrno = EINVAL;\n> -\t\t\treturn NULL;\n> -\t\t}\n> -\t\t/*\n> -\t\t * dwim_ref() uses REF_ISBROKEN to distinguish between\n> -\t\t * missing refs and refs that were present but invalid,\n> -\t\t * to complain about the latter to stderr.\n> -\t\t *\n> -\t\t * We don't know whether the ref exists, so don't set\n> -\t\t * REF_ISBROKEN yet.\n> -\t\t */\n> -\t\tbad_name = 1;\n> -\t}\n> \tfor (;;) {\n> -\t\tconst char *path;\n> \t\tstruct stat st;\n> -\t\tchar *buf;\n> \t\tint fd;\n> -\n> -\t\tif (--depth < 0) {\n> -\t\t\terrno = ELOOP;\n> -\t\t\treturn NULL;\n> -\t\t}\n> -\n> -\t\tstrbuf_reset(sb_path);\n> -\t\tstrbuf_git_path(sb_path, \"%s\", refname);\n> -\t\tpath = sb_path->buf;\n> -\n> \t\t/*\n> \t\t * We might have to loop back here to avoid a race\n> \t\t * condition: first we lstat() the file, then we try\n> @@ -1457,49 +1440,45 @@ static const char *resolve_ref_1(const char *refname,\n> \t\t * we don't want to report that as an error but rather\n> \t\t * try again starting with the lstat().\n> \t\t */\n> -\tstat_ref:\n> +\n> \t\tif (lstat(path, &st) < 0) {\n> \t\t\tif (errno != ENOENT)\n> -\t\t\t\treturn NULL;\n> -\t\t\tif (resolve_missing_loose_ref(refname, resolve_flags,\n> -\t\t\t\t\t\t sha1, flags))\n> -\t\t\t\treturn NULL;\n> -\t\t\tif (bad_name) {\n> -\t\t\t\thashclr(sha1);\n> -\t\t\t\tif (flags)\n> -\t\t\t\t\t*flags |= REF_ISBROKEN;\n> -\t\t\t}\n> -\t\t\treturn refname;\n> +\t\t\t\tbreak;\n> +\t\t\tif (resolve_missing_loose_ref(refname, sha1, flags))\n> +\t\t\t\tbreak;\n> +\t\t\tret = 0;\n> +\t\t\tbreak;\n> \t\t}\n> \n> \t\t/* Follow \"normalized\" - ie \"refs/..\" symlinks by hand */\n> \t\tif (S_ISLNK(st.st_mode)) {\n> -\t\t\tstrbuf_reset(sb_contents);\n> -\t\t\tif (strbuf_readlink(sb_contents, path, 0) < 0) {\n> +\t\t\tstrbuf_reset(&sb_contents);\n> +\t\t\tif (strbuf_readlink(&sb_contents, path, 0) < 0) {\n> \t\t\t\tif (errno == ENOENT || errno == EINVAL)\n> \t\t\t\t\t/* inconsistent with lstat; retry */\n> -\t\t\t\t\tgoto stat_ref;\n> +\t\t\t\t\tcontinue;\n> \t\t\t\telse\n> -\t\t\t\t\treturn NULL;\n> +\t\t\t\t\tbreak;\n> \t\t\t}\n> -\t\t\tif (starts_with(sb_contents->buf, \"refs/\") &&\n> -\t\t\t !check_refname_format(sb_contents->buf, 0)) {\n> -\t\t\t\tstrbuf_swap(sb_refname, sb_contents);\n> -\t\t\t\trefname = sb_refname->buf;\n> +\t\t\tif (starts_with(sb_contents.buf, \"refs/\") &&\n> +\t\t\t !check_refname_format(sb_contents.buf, 0)) {\n> +\t\t\t\tstrbuf_swap(&sb_contents, symref);\n> \t\t\t\tif (flags)\n> \t\t\t\t\t*flags |= REF_ISSYMREF;\n> -\t\t\t\tif (resolve_flags & RESOLVE_REF_NO_RECURSE) {\n> -\t\t\t\t\thashclr(sha1);\n> -\t\t\t\t\treturn refname;\n> -\t\t\t\t}\n> -\t\t\t\tcontinue;\n> +\t\t\t\tret = 0;\n> +\t\t\t\tbreak;\n> +\t\t\t} else {\n> +\t\t\t\t/* bogus symlink ref */\n> +\t\t\t\tif (flags)\n> +\t\t\t\t\t*flags |= REF_ISBROKEN;\n> +\t\t\t\tbreak;\n\nI think this is a functional change.\n\nSome background: We still support writing symrefs as symlinks if\ncore.prefersymlinkrefs is set (the code is in create_ref_symlink()).\nSuch symlinks contain the full ref name (e.g., \"refs/heads/foo\") and are\nread via the\n\n if starts_with(sb_contents.buf, \"refs/\")\n\nblock above.\n\nI suppose that this code was used in the old days when \"HEAD\" was the\nonly symbolic reference, in which case the symlink would really point at\nthe file representing the referred-to loose reference. Nowadays symbolic\nreferences can live elsewhere, and references can be packed, so the\nsymlink representing the symbolic reference might not even point at a\nfile at the filesystem level. But that's not really a problem; the OS\ndoesn't care what bytes you stick in a symlink.\n\nAnyway, at this point in the code we're not talking about that kind of\nsymlink. We're talking about a symlink that might point legitimately at\nanother file at the filesystem level, even though its contents might be\nsomething like \"../foo/bar\" that doesn't even look like a reference name.\n\nThe old resolve_ref_1() did NOT consider such a symlink to be bogus *if*\nit pointed to a file. Instead the code fell through to a questionable\nS_ISDIR() test followed by an attempt to read the loose reference file\ndirectly through the symlink. Thus, for example, the old code would\nresolve a symlink \"refs/heads/foo\" with contents \"bar\" to\n\"refs/heads/bar\", albeit without marking it REF_ISSYMREF.\n\nWe might not want to continue supporting this kind of\nsymlink-as-reference. It seems pretty silly and broken to me (e.g., if\nthe referred-to reference is packed, it would break). But maybe the\nold-timers can think of a reason to preserve this behavior. Let's have\nthat discussion.\n\nEither way, it is NOT OK to change behavior like this in a big patch\nwithout even mentioning it in the commit message. Ideally, the behavior\nchange should be done in a separate patch with an explicit justification.\n\n> \t\t\t}\n> \t\t}\n> \n> \t\t/* Is it a directory? */\n> \t\tif (S_ISDIR(st.st_mode)) {\n> \t\t\terrno = EISDIR;\n> -\t\t\treturn NULL;\n> +\t\t\tbreak;\n> \t\t}\n> \n> \t\t/*\n> @@ -1510,35 +1489,110 @@ static const char *resolve_ref_1(const char *refname,\n> \t\tif (fd < 0) {\n> \t\t\tif (errno == ENOENT)\n> \t\t\t\t/* inconsistent with lstat; retry */\n> -\t\t\t\tgoto stat_ref;\n> +\t\t\t\tcontinue;\n> \t\t\telse\n> -\t\t\t\treturn NULL;\n> +\t\t\t\tbreak;\n> \t\t}\n> -\t\tstrbuf_reset(sb_contents);\n> -\t\tif (strbuf_read(sb_contents, fd, 256) < 0) {\n> +\t\tstrbuf_reset(&sb_contents);\n> +\t\tif (strbuf_read(&sb_contents, fd, 256) < 0) {\n> \t\t\tint save_errno = errno;\n> \t\t\tclose(fd);\n> \t\t\terrno = save_errno;\n> -\t\t\treturn NULL;\n> +\t\t\tbreak;\n> \t\t}\n> \t\tclose(fd);\n> -\t\tstrbuf_rtrim(sb_contents);\n> +\t\tstrbuf_rtrim(&sb_contents);\n> +\t\tbuf = sb_contents.buf;\n> +\t\tif (starts_with(buf, \"ref:\")) {\n> +\t\t\tbuf += 4;\n> +\t\t\twhile (isspace(*buf))\n> +\t\t\t\tbuf++;\n> +\n> +\t\t\tstrbuf_reset(symref);\n> +\t\t\tstrbuf_addstr(symref, buf);\n> +\t\t\tif (flags)\n\n\nread_raw_ref() is only called with a valid flags pointer, so you can get\nrid of these \"if (flags)\" guards (here and elsewhere).\n\nIn fact, the code in resolve_ref_1() could be simplified the same way if\nresolve_ref_unsafe() (the only public entry point) would contain\n\nint unused_flags;\n\nif (!flags)\nflags = &unused_flags;\n\nbut feel free to consider that outside of the scope of your patch series.\n\n\n> +\t\t\t\t*flags |= REF_ISSYMREF;\n> +\t\t\tret = 0;\n> +\t\t\tbreak;\n> +\t\t}\n> \n> \t\t/*\n> -\t\t * Is it a symbolic ref?\n> +\t\t * Please note that FETCH_HEAD has additional\n> +\t\t * data after the sha.\n> \t\t */\n> -\t\tif (!starts_with(sb_contents->buf, \"ref:\")) {\n> -\t\t\t/*\n> -\t\t\t * Please note that FETCH_HEAD has a second\n> -\t\t\t * line containing other data.\n> -\t\t\t */\n> -\t\t\tif (get_sha1_hex(sb_contents->buf, sha1) ||\n> -\t\t\t (sb_contents->buf[40] != '\\0' && !isspace(sb_contents->buf[40]))) {\n> +\t\tif (get_sha1_hex(buf, sha1) ||\n> +\t\t (buf[40] != '\\0' && !isspace(buf[40]))) {\n> +\t\t\tif (flags)\n> +\t\t\t\t*flags |= REF_ISBROKEN;\n> +\t\t\terrno = EINVAL;\n> +\t\t\tbreak;\n> +\t\t}\n> +\t\tret = 0;\n> +\t\tbreak;\n> +\t}\n> +\n> +\tstrbuf_release(&sb_contents);\n> +\treturn ret;\n> +}\n> +\n> +/* This function needs to return a meaningful errno on failure */\n> +static const char *resolve_ref_1(const char *refname,\n> +\t\t\t\t int resolve_flags,\n> +\t\t\t\t unsigned char *sha1,\n> +\t\t\t\t int *flags,\n> +\t\t\t\t struct strbuf *sb_refname,\n> +\t\t\t\t struct strbuf *sb_path)\n> +{\n> +\tint bad_name = 0;\n> +\tint symref_count;\n> +\n> +\tif (flags)\n> +\t\t*flags = 0;\n> +\n> +\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n> +\t\tif (flags)\n> +\t\t\t*flags |= REF_BAD_NAME;\n> +\n> +\t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n> +\t\t !refname_is_safe(refname)) {\n> +\t\t\terrno = EINVAL;\n> +\t\t\treturn NULL;\n> +\t\t}\n> +\t\t/*\n> +\t\t * dwim_ref() uses REF_ISBROKEN to distinguish between\n> +\t\t * missing refs and refs that were present but invalid,\n> +\t\t * to complain about the latter to stderr.\n> +\t\t *\n> +\t\t * We don't know whether the ref exists, so don't set\n> +\t\t * REF_ISBROKEN yet.\n> +\t\t */\n> +\t\tbad_name = 1;\n> +\t}\n> +\n> +\tfor (symref_count = 0; symref_count < MAXDEPTH; symref_count++) {\n> +\t\tunsigned int read_flags = 0;\n> +\n> +\t\tif (read_raw_ref(refname, sha1, sb_refname, sb_path, &read_flags)) {\n> +\t\t\tint saved_errno = errno;\n> +\t\t\tif (flags)\n> +\t\t\t\t*flags |= read_flags;\n> +\t\t\terrno = saved_errno;\n> +\t\t\tif (bad_name) {\n> +\t\t\t\thashclr(sha1);\n> \t\t\t\tif (flags)\n> \t\t\t\t\t*flags |= REF_ISBROKEN;\n> -\t\t\t\terrno = EINVAL;\n> +\t\t\t}\n> +\t\t\tif (resolve_flags & RESOLVE_REF_READING || errno != ENOENT) {\n> \t\t\t\treturn NULL;\n> +\t\t\t} else {\n> +\t\t\t\thashclr(sha1);\n> +\t\t\t\treturn refname;\n> \t\t\t}\n> +\t\t}\n> +\t\tif (flags)\n> +\t\t\t*flags |= read_flags;\n> +\n> +\t\tif (!(read_flags & REF_ISSYMREF)) {\n> \t\t\tif (bad_name) {\n> \t\t\t\thashclr(sha1);\n> \t\t\t\tif (flags)\n> @@ -1546,44 +1600,43 @@ static const char *resolve_ref_1(const char *refname,\n> \t\t\t}\n> \t\t\treturn refname;\n> \t\t}\n> -\t\tif (flags)\n> -\t\t\t*flags |= REF_ISSYMREF;\n> -\t\tbuf = sb_contents->buf + 4;\n> -\t\twhile (isspace(*buf))\n> -\t\t\tbuf++;\n> -\t\tstrbuf_reset(sb_refname);\n> -\t\tstrbuf_addstr(sb_refname, buf);\n> +\n> \t\trefname = sb_refname->buf;\n> \t\tif (resolve_flags & RESOLVE_REF_NO_RECURSE) {\n> \t\t\thashclr(sha1);\n> +\t\t\tif (bad_name && flags)\n> +\t\t\t\t*flags |= REF_ISBROKEN;\n> \t\t\treturn refname;\n> \t\t}\n> -\t\tif (check_refname_format(buf, REFNAME_ALLOW_ONELEVEL)) {\n> +\n> +\t\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n> \t\t\tif (flags)\n> \t\t\t\t*flags |= REF_ISBROKEN;\n> -\n> \t\t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n> -\t\t\t !refname_is_safe(buf)) {\n> +\t\t\t !refname_is_safe(refname)) {\n> \t\t\t\terrno = EINVAL;\n> \t\t\t\treturn NULL;\n> \t\t\t}\n> \t\t\tbad_name = 1;\n> \t\t}\n> \t}\n> +\n> +\tif (flags)\n> +\t\t*flags |= REF_ISBROKEN;\n> +\treturn NULL;\n> }\n> \n> const char *resolve_ref_unsafe(const char *refname, int resolve_flags,\n> \t\t\t unsigned char *sha1, int *flags)\n> {\n> \tstatic struct strbuf sb_refname = STRBUF_INIT;\n> -\tstruct strbuf sb_contents = STRBUF_INIT;\n> \tstruct strbuf sb_path = STRBUF_INIT;\n> \tconst char *ret;\n> \n> \tret = resolve_ref_1(refname, resolve_flags, sha1, flags,\n> -\t\t\t &sb_refname, &sb_path, &sb_contents);\n> +\t\t\t &sb_refname, &sb_path);\n> +\n> \tstrbuf_release(&sb_path);\n> -\tstrbuf_release(&sb_contents);\n> \treturn ret;\n> }\n> \n> \n"},{"id":"281640","messageId":"56F3924C.3040408@alum.mit.edu","threadId":"41574","inReplyTo":"1456793586-22082-10-git-send-email-dturner@twopensource.com","subject":"Re: [PATCH v7 09/33] refs: reduce the visibility of do_for_each_ref()","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2016-03-24T07:07:56Z","receivedAt":"2016-03-24T07:07:56Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 03/01/2016 01:52 AM, David Turner wrote:\n> From: Ramsay Jones \n> \n> Now that we have moved do_for_each_ref into refs.c, it no longer needs\n> to be public.\n> \n> Signed-off-by: Ramsay Jones \n> Signed-off-by: Junio C Hamano \n> ---\n> refs.c | 19 +++++++++++--------\n> refs/refs-internal.h | 6 ------\n> 2 files changed, 11 insertions(+), 14 deletions(-)\n> \n> diff --git a/refs.c b/refs.c\n> index dc5682a..cea5997 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -1142,6 +1142,17 @@ int head_ref(each_ref_fn fn, void *cb_data)\n> \treturn head_ref_submodule(NULL, fn, cb_data);\n> }\n> \n> +/*\n> + * The common backend for the for_each_*ref* functions\n> + */\n> +static int do_for_each_ref(const char *submodule, const char *base,\n> +\t\t each_ref_fn fn, int trim, int flags,\n> +\t\t void *cb_data)\n\nThe two lines above are indented incorrectly.\n\n> +{\n> +\treturn the_refs_backend->do_for_each_ref(submodule, base, fn, trim,\n> +\t\t\t\t\t\t flags, cb_data);\n> +}\n> +\n> int for_each_ref(each_ref_fn fn, void *cb_data)\n> {\n> \treturn do_for_each_ref(NULL, \"\", fn, 0, 0, cb_data);\n> @@ -1342,11 +1353,3 @@ int resolve_gitlink_ref(const char *path, const char *refname,\n> {\n> \treturn the_refs_backend->resolve_gitlink_ref(path, refname, sha1);\n> }\n> -\n> -int do_for_each_ref(const char *submodule, const char *base,\n> -\t\t each_ref_fn fn, int trim, int flags,\n> -\t\t void *cb_data)\n> -{\n> -\treturn the_refs_backend->do_for_each_ref(submodule, base, fn, trim,\n> -\t\t\t\t\t\t flags, cb_data);\n> -}\n\nNit: in the previous patch, please put the function where you want it so\nthat you don't have to move it in this patch.\n\n> [...]\n\nMichael\n"},{"id":"281643","messageId":"56F3972C.7020901@alum.mit.edu","threadId":"41574","inReplyTo":"1456793586-22082-15-git-send-email-dturner@twopensource.com","subject":"Re: [PATCH v7 14/33] refs: add methods to init refs db","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2016-03-24T07:28:44Z","receivedAt":"2016-03-24T07:28:44Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 03/01/2016 01:52 AM, David Turner wrote:\n> Alternate refs backends might not need the refs/heads directory and so\n> on, so we make ref db initialization part of the backend.\n> \n> Signed-off-by: David Turner \n> Signed-off-by: Junio C Hamano \n> ---\n> builtin/init-db.c | 20 ++++++++++----------\n> refs.c | 5 +++++\n> refs.h | 2 ++\n> refs/files-backend.c | 16 ++++++++++++++++\n> refs/refs-internal.h | 2 ++\n> 5 files changed, 35 insertions(+), 10 deletions(-)\n> \n> diff --git a/builtin/init-db.c b/builtin/init-db.c\n> index 6223b7d..e6d4e86 100644\n> --- a/builtin/init-db.c\n> +++ b/builtin/init-db.c\n> @@ -177,13 +177,7 @@ static int create_default_files(const char *template_path)\n> \tchar junk[2];\n> \tint reinit;\n> \tint filemode;\n> -\n> -\t/*\n> -\t * Create .git/refs/{heads,tags}\n> -\t */\n> -\tsafe_create_dir(git_path_buf(&buf, \"refs\"), 1);\n> -\tsafe_create_dir(git_path_buf(&buf, \"refs/heads\"), 1);\n> -\tsafe_create_dir(git_path_buf(&buf, \"refs/tags\"), 1);\n> +\tstruct strbuf err = STRBUF_INIT;\n> \n> \t/* Just look for `init.templatedir` */\n> \tgit_config(git_init_db_config, NULL);\n> @@ -207,12 +201,18 @@ static int create_default_files(const char *template_path)\n> \t */\n> \tif (shared_repository) {\n> \t\tadjust_shared_perm(get_git_dir());\n> -\t\tadjust_shared_perm(git_path_buf(&buf, \"refs\"));\n\nGiven that this function is creating the \"refs\" directory, it seems like\nadjust_shared_perm() should be called for it here, too (rather than in\nthe backend-specific code).\n\n> -\t\tadjust_shared_perm(git_path_buf(&buf, \"refs/heads\"));\n> -\t\tadjust_shared_perm(git_path_buf(&buf, \"refs/tags\"));\n> \t}\n> \n> \t/*\n> +\t * We need to create a \"refs\" dir in any case so that older\n> +\t * versions of git can tell that this is a repository.\n> +\t */\n> +\tsafe_create_dir(git_path(\"refs\"), 1);\n> +\n> +\tif (refs_init_db(shared_repository, &err))\n> +\t\tdie(\"failed to set up refs db: %s\", err.buf);\n> +\n> +\t/*\n> \t * Create the default symlink from \".git/HEAD\" to the \"master\"\n> \t * branch, if it does not exist yet.\n> \t */\n> diff --git a/refs.c b/refs.c\n> index d25eee2..b2697f6 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -1319,6 +1319,11 @@ const char *resolve_ref_unsafe(const char *refname, int resolve_flags,\n> }\n> \n> /* backend functions */\n> +int refs_init_db(int shared, struct strbuf *err)\n> +{\n> +\treturn the_refs_backend->init_db(shared, err);\n> +}\n> +\n> int ref_transaction_commit(struct ref_transaction *transaction,\n> \t\t\t struct strbuf *err)\n> {\n> diff --git a/refs.h b/refs.h\n> index 3405842..13ce2a0 100644\n> --- a/refs.h\n> +++ b/refs.h\n> @@ -66,6 +66,8 @@ extern int ref_exists(const char *refname);\n> \n> extern int is_branch(const char *refname);\n> \n> +extern int refs_init_db(int shared, struct strbuf *err);\n> +\n> /*\n> * If refname is a non-symbolic reference that refers to a tag object,\n> * and the tag can be (recursively) dereferenced to a non-tag object,\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index 35328d2..acb4401 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -3415,9 +3415,25 @@ static int files_reflog_expire(const char *refname, const unsigned char *sha1,\n> \treturn -1;\n> }\n> \n> +static int files_init_db(int shared, struct strbuf *err)\n> +{\n> +\t/*\n> +\t * Create .git/refs/{heads,tags}\n> +\t */\n> +\tsafe_create_dir(git_path(\"refs/heads\"), 1);\n> +\tsafe_create_dir(git_path(\"refs/tags\"), 1);\n> +\tif (shared) {\n> +\t\tadjust_shared_perm(git_path(\"refs\"));\n> +\t\tadjust_shared_perm(git_path(\"refs/heads\"));\n> +\t\tadjust_shared_perm(git_path(\"refs/tags\"));\n> +\t}\n> +\treturn 0;\n> +}\n> +\n> struct ref_storage_be refs_be_files = {\n> \tNULL,\n> \t\"files\",\n> +\tfiles_init_db,\n> \tfiles_transaction_commit,\n> \tfiles_initial_transaction_commit,\n> \n> diff --git a/refs/refs-internal.h b/refs/refs-internal.h\n> index beef457..dfd0326 100644\n> --- a/refs/refs-internal.h\n> +++ b/refs/refs-internal.h\n> @@ -211,6 +211,7 @@ int do_for_each_per_worktree_ref(const char *submodule, const char *base,\n> \t\t\t\t void *cb_data);\n> \n> /* refs backends */\n> +typedef int ref_init_db_fn(int shared, struct strbuf *err);\n> typedef int ref_transaction_commit_fn(struct ref_transaction *transaction,\n> \t\t\t\t struct strbuf *err);\n> \n> @@ -255,6 +256,7 @@ typedef int do_for_each_ref_fn(const char *submodule, const char *base,\n> struct ref_storage_be {\n> \tstruct ref_storage_be *next;\n> \tconst char *name;\n> +\tref_init_db_fn *init_db;\n> \tref_transaction_commit_fn *transaction_commit;\n> \tref_transaction_commit_fn *initial_transaction_commit;\n> \n> \n\nMichael\n"},{"id":"281702","messageId":"1458842695.28595.11.camel@twopensource.com","threadId":"41574","inReplyTo":"56F3972C.7020901@alum.mit.edu","subject":"Re: [PATCH v7 14/33] refs: add methods to init refs db","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-24T18:04:55Z","receivedAt":"2016-03-24T18:04:55Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"On Thu, 2016-03-24 at 08:28 +0100, Michael Haggerty wrote:\n> > \tif (shared_repository) {\n> > \t\tadjust_shared_perm(get_git_dir());\n> > -\t\tadjust_shared_perm(git_path_buf(&buf, \"refs\"));\n> \n> Given that this function is creating the \"refs\" directory, it seems\n> like\n> adjust_shared_perm() should be called for it here, too (rather than\n> in\n> the backend-specific code).\n\n\nGood point.\n"},{"id":"281708","messageId":"1458845775.2976.1.camel@twopensource.com","threadId":"41574","inReplyTo":"56F3924C.3040408@alum.mit.edu","subject":"Re: [PATCH v7 09/33] refs: reduce the visibility of do_for_each_ref()","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-03-24T18:56:15Z","receivedAt":"2016-03-24T18:56:15Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"On Thu, 2016-03-24 at 08:07 +0100, Michael Haggerty wrote:\n> > +/*\n> > + * The common backend for the for_each_*ref* functions\n> > + */\n> > +static int do_for_each_ref(const char *submodule, const char\n> > *base,\n> > +\t\t each_ref_fn fn, int trim, int flags,\n> > +\t\t void *cb_data)\n> \n> The two lines above are indented incorrectly.\n\nFixed, thanks.\n\n> > -\n> > -int do_for_each_ref(const char *submodule, const char *base,\n> > -\t\t each_ref_fn fn, int trim, int flags,\n> > -\t\t void *cb_data)\n> > -{\n> > -\treturn the_refs_backend->do_for_each_ref(submodule, base,\n> > fn, trim,\n> > -\t\t\t\t\t\t flags, cb_data);\n> > -}\n> \n> Nit: in the previous patch, please put the function where you want it\n> so\n> that you don't have to move it in this patch.\n> \n> > [...]\n> \n> Michael\n\nOk.\n"},{"id":"281712","messageId":"56F443BD.1030100@alum.mit.edu","threadId":"41574","inReplyTo":"1456793586-22082-18-git-send-email-dturner@twopensource.com","subject":"Re: [PATCH v7 17/33] refs: make lock generic","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2016-03-24T19:45:01Z","receivedAt":"2016-03-24T19:45:01Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 03/01/2016 01:52 AM, David Turner wrote:\n> Instead of using a files-backend-specific struct ref_lock, the generic\n> ref_transaction struct should provide a void pointer that backends can use\n> for their own lock data.\n\nAside from the comments I've already mentioned, I've checked the patches\nup to and including this one and they look fine to me.\n\nMichael\n"},{"id":"284034","messageId":"5718E0FC.7090501@alum.mit.edu","threadId":"41574","inReplyTo":"1456793586-22082-20-git-send-email-dturner@twopensource.com","subject":"Re: [PATCH v7 19/33] refs: allow log-only updates","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2016-04-21T14:17:32Z","receivedAt":"2016-04-21T14:17:32Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 03/01/2016 01:52 AM, David Turner wrote:\n> The refs infrastructure learns about log-only ref updates, which only\n> update the reflog. Later, we will use this to separate symbolic\n> reference resolution from ref updating.\n> \n> Signed-off-by: David Turner \n> Signed-off-by: Junio C Hamano \n> ---\n> refs/files-backend.c | 15 ++++++++++-----\n> refs/refs-internal.h | 7 +++++++\n> 2 files changed, 17 insertions(+), 5 deletions(-)\n> \n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index 1f565cb..189b86e 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -2702,7 +2702,7 @@ static int commit_ref_update(struct ref_lock *lock,\n> \t\t\t}\n> \t\t}\n> \t}\n> -\tif (commit_ref(lock)) {\n> +\tif (!(flags & REF_LOG_ONLY) && commit_ref(lock)) {\n> \t\terror(\"Couldn't set %s\", lock->ref_name);\n> \t\tunlock_ref(lock);\n> \t\treturn -1;\n> @@ -3056,7 +3056,8 @@ static int files_transaction_commit(struct ref_transaction *transaction,\n> \t\t\tgoto cleanup;\n> \t\t}\n> \t\tif ((update->flags & REF_HAVE_NEW) &&\n> -\t\t !(update->flags & REF_DELETING)) {\n> +\t\t !(update->flags & REF_DELETING) &&\n> +\t\t !(update->flags & REF_LOG_ONLY)) {\n> \t\t\tint overwriting_symref = ((update->type & REF_ISSYMREF) &&\n> \t\t\t\t\t\t (update->flags & REF_NODEREF));\n> \n> @@ -3086,7 +3087,9 @@ static int files_transaction_commit(struct ref_transaction *transaction,\n> \t\t\t\tupdate->flags |= REF_NEEDS_COMMIT;\n> \t\t\t}\n> \t\t}\n> -\t\tif (!(update->flags & REF_NEEDS_COMMIT)) {\n> +\n> +\t\tif (!(update->flags & REF_LOG_ONLY) &&\n> +\t\t !(update->flags & REF_NEEDS_COMMIT)) {\n\nI was just going over this series again, and I think this hunk is\nincorrect. If REF_LOG_ONLY, we created and opened the lockfile. And we\ndidn't call write_ref_to_logfile(), so the lockfile is still open. That\nmeans that we want to call close_ref() here to free up the file\ndescriptor. (Note that close_ref() closes the lockfile but doesn't\nrelease the lock. That is done further down by unlock_ref().)\n\nSo I think this hunk should be omitted.\n\nI realize that this patch series is obsolete, so there is no need to\nre-submit. I just wanted to get a sanity check as I implement a new\nversion of this patch that I'm not misunderstanding something.\n\n> [...]\n\nMichael\n"},{"id":"284435","messageId":"1461602814.25914.4.camel@twopensource.com","threadId":"41574","inReplyTo":"5718E0FC.7090501@alum.mit.edu","subject":"Re: [PATCH v7 19/33] refs: allow log-only updates","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2016-04-25T16:46:54Z","receivedAt":"2016-04-25T16:46:54Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"On Thu, 2016-04-21 at 16:17 +0200, Michael Haggerty wrote:\n> On 03/01/2016 01:52 AM, David Turner wrote:\n> > The refs infrastructure learns about log-only ref updates, which\n> > only\n> > update the reflog. Later, we will use this to separate symbolic\n> > reference resolution from ref updating.\n> > \n> > Signed-off-by: David Turner \n> > Signed-off-by: Junio C Hamano \n> > ---\n> > refs/files-backend.c | 15 ++++++++++-----\n> > refs/refs-internal.h | 7 +++++++\n> > 2 files changed, 17 insertions(+), 5 deletions(-)\n> > \n> > diff --git a/refs/files-backend.c b/refs/files-backend.c\n> > index 1f565cb..189b86e 100644\n> > --- a/refs/files-backend.c\n> > +++ b/refs/files-backend.c\n> > @@ -2702,7 +2702,7 @@ static int commit_ref_update(struct ref_lock\n> > *lock,\n> > \t\t\t}\n> > \t\t}\n> > \t}\n> > -\tif (commit_ref(lock)) {\n> > +\tif (!(flags & REF_LOG_ONLY) && commit_ref(lock)) {\n> > \t\terror(\"Couldn't set %s\", lock->ref_name);\n> > \t\tunlock_ref(lock);\n> > \t\treturn -1;\n> > @@ -3056,7 +3056,8 @@ static int files_transaction_commit(struct\n> > ref_transaction *transaction,\n> > \t\t\tgoto cleanup;\n> > \t\t}\n> > \t\tif ((update->flags & REF_HAVE_NEW) &&\n> > -\t\t !(update->flags & REF_DELETING)) {\n> > +\t\t !(update->flags & REF_DELETING) &&\n> > +\t\t !(update->flags & REF_LOG_ONLY)) {\n> > \t\t\tint overwriting_symref = ((update->type &\n> > REF_ISSYMREF) &&\n> > \t\t\t\t\t\t (update->flags &\n> > REF_NODEREF));\n> > \n> > @@ -3086,7 +3087,9 @@ static int files_transaction_commit(struct\n> > ref_transaction *transaction,\n> > \t\t\t\tupdate->flags |= REF_NEEDS_COMMIT;\n> > \t\t\t}\n> > \t\t}\n> > -\t\tif (!(update->flags & REF_NEEDS_COMMIT)) {\n> > +\n> > +\t\tif (!(update->flags & REF_LOG_ONLY) &&\n> > +\t\t !(update->flags & REF_NEEDS_COMMIT)) {\n> \n> I was just going over this series again, and I think this hunk is\n> incorrect. If REF_LOG_ONLY, we created and opened the lockfile. And\n> we\n> didn't call write_ref_to_logfile(), so the lockfile is still open.\n> That\n> means that we want to call close_ref() here to free up the file\n> descriptor. (Note that close_ref() closes the lockfile but doesn't\n> release the lock. That is done further down by unlock_ref().)\n> \n> So I think this hunk should be omitted.\n> \n> I realize that this patch series is obsolete, so there is no need to\n> re-submit. I just wanted to get a sanity check as I implement a new\n> version of this patch that I'm not misunderstanding something.\n\nI think your logic seems sound, but if you're going to change this,\nplease make sure tests still pass -- as you know, this area is\nsomething of a minefield.\n"}]}