{"thread":{"id":"46189","subject":"[PATCH 00/28] Create a reference backend for packed refs","startedAt":"2017-06-15T14:47:53Z","lastAt":"2017-07-26T23:49:40Z","messageCount":52,"participants":["Michael Haggerty","Stefan Beller","Junio C Hamano","Jeff King","Jonathan Nieder"],"isPatch":true,"patchVersion":1,"patchTotal":28},"messages":[{"id":"322327","messageId":"cover.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":null,"subject":"[PATCH 00/28] Create a reference backend for packed refs","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:05Z","receivedAt":"2017-06-15T14:47:53Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"This patch series continues the saga of picking apart the code for\nhandling packed references from the code for handling loose\nreferences, all in preparation for making big changes to how the\npacked-ref reading and writing works as described in [1]. As a\nreminder, the final goal is to read the \"packed-refs\" file using mmap,\nparsing it on the fly instead of storing it into an in-memory\n`ref_cache`, and to read and parse only the parts of the file that are\nactually needed, giving a big speedup for many operations in\nrepositories that have lots of refs.\n\nIn this episode, we create a `packed_ref_store` class, implementing\npart of the `ref_store` API, that represents the packed references\nwithin a repository. The `files_ref_store` now contains an instance of\n`packed_ref_store` and delegates to it for the operations that have to\ntouch the packed refs.\n\nAfter this patch series, `packed_ref_store` supports:\n\n* Iteration\n* `peel_ref`\n* `pack_refs` (they're already packed, so it's a NOOP)\n* `read_raw_ref`\n\nA future patch series will add support for:\n\n* Reference transactions (`transaction_prepare`, `transaction_finish`,\n `transaction_abort`, `initial_transaction_commit`)\n* `delete_refs`\n\nOperations that `packed_ref_store` will probably never support:\n\n* `create_symref`\n* `rename_ref` (could be supported, but is probably not useful)\n* Reflog-related operations\n\nIn addition, all of the packed-refs related code has been moved to a\nnew module, \"refs/packed-backend.{c,h}\". This includes some functions\nthat are still called by `files_ref_store` directly to update the\npacked refs.\n\nThe patch series is long, but I think relatively straightforward. In\nparticular, patches 2-14 are quite mechanical. Its main point is to\nseparate concerns, but it does bring one end-user advantage: if there\nis a problem parsing the \"packed-refs\" file, we now report an error\nand die. The old code just ignored lines that it didn't understand.\n\nI've developed these patches on top of master plus the following\npatches, which are followups to mh/packed-refs-store-prep:\n\n* lock_packed_refs(): fix cache validity check\n* for_each_bisect_ref(): don't trim refnames\n\nThe patches can also be obtained from my GitHub fork [2] as branch\n\"packed-ref-store\".\n\nMichael\n\n[1] http://public-inbox.org/git/cover.1490026594.git.mhagger@alum.mit.edu/\n[2] https://github.com/mhagger/git\n\nMichael Haggerty (28):\n add_packed_ref(): teach function to overwrite existing refs\n packed_ref_store: new struct\n packed_ref_store: move `packed_refs_path` here\n packed_ref_store: move `packed_refs_lock` member here\n clear_packed_ref_cache(): take a `packed_ref_store *` parameter\n validate_packed_ref_cache(): take a `packed_ref_store *` parameter\n get_packed_ref_cache(): take a `packed_ref_store *` parameter\n get_packed_refs(): take a `packed_ref_store *` parameter\n add_packed_ref(): take a `packed_ref_store *` parameter\n lock_packed_refs(): take a `packed_ref_store *` parameter\n commit_packed_refs(): take a `packed_ref_store *` parameter\n rollback_packed_refs(): take a `packed_ref_store *` parameter\n get_packed_ref(): take a `packed_ref_store *` parameter\n repack_without_refs(): take a `packed_ref_store *` parameter\n packed_peel_ref(): new function, extracted from `files_peel_ref()`\n packed_ref_store: support iteration\n packed_read_raw_ref(): new function, replacing `resolve_packed_ref()`\n packed-backend: new module for handling packed references\n packed_ref_store: make class into a subclass of `ref_store`\n commit_packed_refs(): report errors rather than dying\n commit_packed_refs(): use a staging file separate from the lockfile\n packed_refs_lock(): function renamed from lock_packed_refs()\n packed_refs_lock(): report errors via a `struct strbuf *err`\n packed_refs_unlock(), packed_refs_is_locked(): new functions\n clear_packed_ref_cache(): don't protest if the lock is held\n commit_packed_refs(): remove call to `packed_refs_unlock()`\n repack_without_refs(): don't lock or unlock the packed refs\n read_packed_refs(): die if `packed-refs` contains bogus data\n\n Makefile | 1 +\n refs.c | 18 ++\n refs/files-backend.c | 619 ++++-------------------------------\n refs/packed-backend.c | 868 ++++++++++++++++++++++++++++++++++++++++++++++++++\n refs/packed-backend.h | 25 ++\n refs/refs-internal.h | 10 +\n 6 files changed, 981 insertions(+), 560 deletions(-)\n create mode 100644 refs/packed-backend.c\n create mode 100644 refs/packed-backend.h\n\n-- \n2.11.0\n\n"},{"id":"322328","messageId":"e904644e228affb2bcb99166cd5ab893db75247e.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 01/28] add_packed_ref(): teach function to overwrite existing refs","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:06Z","receivedAt":"2017-06-15T14:47:55Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"Teach `add_packed_ref()` to overwrite an existing entry if one already\nexists for the specified `refname`. This means that we can call it\nfrom `files_pack_refs()`, thereby reducing the amount that the latter\nfunction needs to know about the internals of packed-reference\nhandling.\n\nSigned-off-by: Michael Haggerty \n---\n refs/files-backend.c | 40 ++++++++++++++++++----------------------\n 1 file changed, 18 insertions(+), 22 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex b040bb3b0a..87cecde231 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -413,15 +413,16 @@ static struct ref_dir *get_packed_refs(struct files_ref_store *refs)\n }\n \n /*\n- * Add a reference to the in-memory packed reference cache. This may\n- * only be called while the packed-refs file is locked (see\n- * lock_packed_refs()). To actually write the packed-refs file, call\n- * commit_packed_refs().\n+ * Add or overwrite a reference in the in-memory packed reference\n+ * cache. This may only be called while the packed-refs file is locked\n+ * (see lock_packed_refs()). To actually write the packed-refs file,\n+ * call commit_packed_refs().\n */\n static void add_packed_ref(struct files_ref_store *refs,\n \t\t\t const char *refname, const struct object_id *oid)\n {\n-\tstruct packed_ref_cache *packed_ref_cache = get_packed_ref_cache(refs);\n+\tstruct ref_dir *packed_refs;\n+\tstruct ref_entry *packed_entry;\n \n \tif (!is_lock_file_locked(&refs->packed_refs_lock))\n \t\tdie(\"BUG: packed refs not locked\");\n@@ -429,8 +430,17 @@ static void add_packed_ref(struct files_ref_store *refs,\n \tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL))\n \t\tdie(\"Reference has invalid format: '%s'\", refname);\n \n-\tadd_ref_entry(get_packed_ref_dir(packed_ref_cache),\n-\t\t create_ref_entry(refname, oid, REF_ISPACKED));\n+\tpacked_refs = get_packed_refs(refs);\n+\tpacked_entry = find_ref_entry(packed_refs, refname);\n+\tif (packed_entry) {\n+\t\t/* Overwrite the existing entry: */\n+\t\toidcpy(&packed_entry->u.value.oid, oid);\n+\t\tpacked_entry->flag = REF_ISPACKED;\n+\t\toidclr(&packed_entry->u.value.peeled);\n+\t} else {\n+\t\tpacked_entry = create_ref_entry(refname, oid, REF_ISPACKED);\n+\t\tadd_ref_entry(packed_refs, packed_entry);\n+\t}\n }\n \n /*\n@@ -1526,12 +1536,10 @@ static int files_pack_refs(struct ref_store *ref_store, unsigned int flags)\n \t\tfiles_downcast(ref_store, REF_STORE_WRITE | REF_STORE_ODB,\n \t\t\t \"pack_refs\");\n \tstruct ref_iterator *iter;\n-\tstruct ref_dir *packed_refs;\n \tint ok;\n \tstruct ref_to_prune *refs_to_prune = NULL;\n \n \tlock_packed_refs(refs, LOCK_DIE_ON_ERROR);\n-\tpacked_refs = get_packed_refs(refs);\n \n \titer = cache_ref_iterator_begin(get_loose_ref_cache(refs), NULL, 0);\n \twhile ((ok = ref_iterator_advance(iter)) == ITER_OK) {\n@@ -1540,8 +1548,6 @@ static int files_pack_refs(struct ref_store *ref_store, unsigned int flags)\n \t\t * in the packed ref cache. If the reference should be\n \t\t * pruned, also add it to refs_to_prune.\n \t\t */\n-\t\tstruct ref_entry *packed_entry;\n-\n \t\tif (!should_pack_ref(iter->refname, iter->oid, iter->flags,\n \t\t\t\t flags))\n \t\t\tcontinue;\n@@ -1552,17 +1558,7 @@ static int files_pack_refs(struct ref_store *ref_store, unsigned int flags)\n \t\t * we don't copy the peeled status, because we want it\n \t\t * to be re-peeled.\n \t\t */\n-\t\tpacked_entry = find_ref_entry(packed_refs, iter->refname);\n-\t\tif (packed_entry) {\n-\t\t\t/* Overwrite existing packed entry with info from loose entry */\n-\t\t\tpacked_entry->flag = REF_ISPACKED;\n-\t\t\toidcpy(&packed_entry->u.value.oid, iter->oid);\n-\t\t} else {\n-\t\t\tpacked_entry = create_ref_entry(iter->refname, iter->oid,\n-\t\t\t\t\t\t\tREF_ISPACKED);\n-\t\t\tadd_ref_entry(packed_refs, packed_entry);\n-\t\t}\n-\t\toidclr(&packed_entry->u.value.peeled);\n+\t\tadd_packed_ref(refs, iter->refname, iter->oid);\n \n \t\t/* Schedule the loose reference for pruning if requested. */\n \t\tif ((flags & PACK_REFS_PRUNE)) {\n-- \n2.11.0\n\n"},{"id":"322329","messageId":"2aa91c07ef7f0f9c679aab8898341520eda76016.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 03/28] packed_ref_store: move `packed_refs_path` here","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:08Z","receivedAt":"2017-06-15T14:47:57Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"Move `packed_refs_path` from `files_ref_store` to `packed_ref_store`,\nand rename it to `path` since its meaning is clear from its new\ncontext.\n\nInline `files_packed_refs_path()`.\n\nSigned-off-by: Michael Haggerty \n---\n refs/files-backend.c | 25 ++++++++++++-------------\n 1 file changed, 12 insertions(+), 13 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 2efb71cee9..c4b8e2f63b 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -54,6 +54,9 @@ struct packed_ref_cache {\n struct packed_ref_store {\n \tunsigned int store_flags;\n \n+\t/* The path of the \"packed-refs\" file: */\n+\tchar *path;\n+\n \t/*\n \t * A cache of the values read from the `packed-refs` file, if\n \t * it might still be current; otherwise, NULL.\n@@ -61,11 +64,13 @@ struct packed_ref_store {\n \tstruct packed_ref_cache *cache;\n };\n \n-static struct packed_ref_store *packed_ref_store_create(unsigned int store_flags)\n+static struct packed_ref_store *packed_ref_store_create(\n+\t\tconst char *path, unsigned int store_flags)\n {\n \tstruct packed_ref_store *refs = xcalloc(1, sizeof(*refs));\n \n \trefs->store_flags = store_flags;\n+\trefs->path = xstrdup(path);\n \treturn refs;\n }\n \n@@ -79,7 +84,6 @@ struct files_ref_store {\n \n \tchar *gitdir;\n \tchar *gitcommondir;\n-\tchar *packed_refs_path;\n \n \tstruct ref_cache *loose;\n \n@@ -154,8 +158,8 @@ static struct ref_store *files_ref_store_create(const char *gitdir,\n \tget_common_dir_noenv(&sb, gitdir);\n \trefs->gitcommondir = strbuf_detach(&sb, NULL);\n \tstrbuf_addf(&sb, \"%s/packed-refs\", refs->gitcommondir);\n-\trefs->packed_refs_path = strbuf_detach(&sb, NULL);\n-\trefs->packed_ref_store = packed_ref_store_create(flags);\n+\trefs->packed_ref_store = packed_ref_store_create(sb.buf, flags);\n+\tstrbuf_release(&sb);\n \n \treturn ref_store;\n }\n@@ -343,11 +347,6 @@ static struct packed_ref_cache *read_packed_refs(const char *packed_refs_file)\n \treturn packed_refs;\n }\n \n-static const char *files_packed_refs_path(struct files_ref_store *refs)\n-{\n-\treturn refs->packed_refs_path;\n-}\n-\n static void files_reflog_path(struct files_ref_store *refs,\n \t\t\t struct strbuf *sb,\n \t\t\t const char *refname)\n@@ -401,7 +400,7 @@ static void validate_packed_ref_cache(struct files_ref_store *refs)\n {\n \tif (refs->packed_ref_store->cache &&\n \t !stat_validity_check(&refs->packed_ref_store->cache->validity,\n-\t\t\t\t files_packed_refs_path(refs)))\n+\t\t\t\t refs->packed_ref_store->path))\n \t\tclear_packed_ref_cache(refs);\n }\n \n@@ -415,7 +414,7 @@ static void validate_packed_ref_cache(struct files_ref_store *refs)\n */\n static struct packed_ref_cache *get_packed_ref_cache(struct files_ref_store *refs)\n {\n-\tconst char *packed_refs_file = files_packed_refs_path(refs);\n+\tconst char *packed_refs_file = refs->packed_ref_store->path;\n \n \tif (!is_lock_file_locked(&refs->packed_refs_lock))\n \t\tvalidate_packed_ref_cache(refs);\n@@ -1352,7 +1351,7 @@ static int lock_packed_refs(struct files_ref_store *refs, int flags)\n \t}\n \n \tif (hold_lock_file_for_update_timeout(\n-\t\t\t &refs->packed_refs_lock, files_packed_refs_path(refs),\n+\t\t\t &refs->packed_refs_lock, refs->packed_ref_store->path,\n \t\t\t flags, timeout_value) < 0)\n \t\treturn -1;\n \n@@ -1633,7 +1632,7 @@ static int repack_without_refs(struct files_ref_store *refs,\n \t\treturn 0; /* no refname exists in packed refs */\n \n \tif (lock_packed_refs(refs, 0)) {\n-\t\tunable_to_lock_message(files_packed_refs_path(refs), errno, err);\n+\t\tunable_to_lock_message(refs->packed_ref_store->path, errno, err);\n \t\treturn -1;\n \t}\n \tpacked = get_packed_refs(refs);\n-- \n2.11.0\n\n"},{"id":"322330","messageId":"eea8f3d25e9d3d3a0b63604338f10cfa658bbb43.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 04/28] packed_ref_store: move `packed_refs_lock` member here","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:09Z","receivedAt":"2017-06-15T14:48:02Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"Move the `packed_refs_lock` member from `files_ref_store` to\n`packed_ref_store`, and rename it to `lock` since it's now more\nobvious what it is locking.\n\nSigned-off-by: Michael Haggerty \n---\n refs/files-backend.c | 31 ++++++++++++++++---------------\n 1 file changed, 16 insertions(+), 15 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex c4b8e2f63b..de8293493f 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -62,6 +62,12 @@ struct packed_ref_store {\n \t * it might still be current; otherwise, NULL.\n \t */\n \tstruct packed_ref_cache *cache;\n+\n+\t/*\n+\t * Lock used for the \"packed-refs\" file. Note that this (and\n+\t * thus the enclosing `packed_ref_store`) must not be freed.\n+\t */\n+\tstruct lock_file lock;\n };\n \n static struct packed_ref_store *packed_ref_store_create(\n@@ -87,12 +93,6 @@ struct files_ref_store {\n \n \tstruct ref_cache *loose;\n \n-\t/*\n-\t * Lock used for the \"packed-refs\" file. Note that this (and\n-\t * thus the enclosing `files_ref_store`) must not be freed.\n-\t */\n-\tstruct lock_file packed_refs_lock;\n-\n \tstruct packed_ref_store *packed_ref_store;\n };\n \n@@ -125,7 +125,7 @@ static void clear_packed_ref_cache(struct files_ref_store *refs)\n \tif (refs->packed_ref_store->cache) {\n \t\tstruct packed_ref_cache *packed_refs = refs->packed_ref_store->cache;\n \n-\t\tif (is_lock_file_locked(&refs->packed_refs_lock))\n+\t\tif (is_lock_file_locked(&refs->packed_ref_store->lock))\n \t\t\tdie(\"BUG: packed-ref cache cleared while locked\");\n \t\trefs->packed_ref_store->cache = NULL;\n \t\trelease_packed_ref_cache(packed_refs);\n@@ -416,7 +416,7 @@ static struct packed_ref_cache *get_packed_ref_cache(struct files_ref_store *ref\n {\n \tconst char *packed_refs_file = refs->packed_ref_store->path;\n \n-\tif (!is_lock_file_locked(&refs->packed_refs_lock))\n+\tif (!is_lock_file_locked(&refs->packed_ref_store->lock))\n \t\tvalidate_packed_ref_cache(refs);\n \n \tif (!refs->packed_ref_store->cache)\n@@ -447,7 +447,7 @@ static void add_packed_ref(struct files_ref_store *refs,\n \tstruct ref_dir *packed_refs;\n \tstruct ref_entry *packed_entry;\n \n-\tif (!is_lock_file_locked(&refs->packed_refs_lock))\n+\tif (!is_lock_file_locked(&refs->packed_ref_store->lock))\n \t\tdie(\"BUG: packed refs not locked\");\n \n \tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL))\n@@ -1351,7 +1351,8 @@ static int lock_packed_refs(struct files_ref_store *refs, int flags)\n \t}\n \n \tif (hold_lock_file_for_update_timeout(\n-\t\t\t &refs->packed_refs_lock, refs->packed_ref_store->path,\n+\t\t\t &refs->packed_ref_store->lock,\n+\t\t\t refs->packed_ref_store->path,\n \t\t\t flags, timeout_value) < 0)\n \t\treturn -1;\n \n@@ -1388,10 +1389,10 @@ static int commit_packed_refs(struct files_ref_store *refs)\n \n \tfiles_assert_main_repository(refs, \"commit_packed_refs\");\n \n-\tif (!is_lock_file_locked(&refs->packed_refs_lock))\n+\tif (!is_lock_file_locked(&refs->packed_ref_store->lock))\n \t\tdie(\"BUG: packed-refs not locked\");\n \n-\tout = fdopen_lock_file(&refs->packed_refs_lock, \"w\");\n+\tout = fdopen_lock_file(&refs->packed_ref_store->lock, \"w\");\n \tif (!out)\n \t\tdie_errno(\"unable to fdopen packed-refs descriptor\");\n \n@@ -1409,7 +1410,7 @@ static int commit_packed_refs(struct files_ref_store *refs)\n \tif (ok != ITER_DONE)\n \t\tdie(\"error while iterating over references\");\n \n-\tif (commit_lock_file(&refs->packed_refs_lock)) {\n+\tif (commit_lock_file(&refs->packed_ref_store->lock)) {\n \t\tsave_errno = errno;\n \t\terror = -1;\n \t}\n@@ -1430,9 +1431,9 @@ static void rollback_packed_refs(struct files_ref_store *refs)\n \n \tfiles_assert_main_repository(refs, \"rollback_packed_refs\");\n \n-\tif (!is_lock_file_locked(&refs->packed_refs_lock))\n+\tif (!is_lock_file_locked(&refs->packed_ref_store->lock))\n \t\tdie(\"BUG: packed-refs not locked\");\n-\trollback_lock_file(&refs->packed_refs_lock);\n+\trollback_lock_file(&refs->packed_ref_store->lock);\n \trelease_packed_ref_cache(packed_ref_cache);\n \tclear_packed_ref_cache(refs);\n }\n-- \n2.11.0\n\n"},{"id":"322331","messageId":"e8404678ec31e1b1f1096e3db7f6a4d562e30473.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 07/28] get_packed_ref_cache(): take a `packed_ref_store *` parameter","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:12Z","receivedAt":"2017-06-15T14:48:03Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"It only cares about the packed-refs part of the reference store.\n\nSigned-off-by: Michael Haggerty \n---\n refs/files-backend.c | 26 ++++++++++++--------------\n 1 file changed, 12 insertions(+), 14 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex f061506bf0..b2ef7b3bb9 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -404,24 +404,22 @@ static void validate_packed_ref_cache(struct packed_ref_store *refs)\n }\n \n /*\n- * Get the packed_ref_cache for the specified files_ref_store,\n+ * Get the packed_ref_cache for the specified packed_ref_store,\n * creating and populating it if it hasn't been read before or if the\n * file has been changed (according to its `validity` field) since it\n * was last read. On the other hand, if we hold the lock, then assume\n * that the file hasn't been changed out from under us, so skip the\n * extra `stat()` call in `stat_validity_check()`.\n */\n-static struct packed_ref_cache *get_packed_ref_cache(struct files_ref_store *refs)\n+static struct packed_ref_cache *get_packed_ref_cache(struct packed_ref_store *refs)\n {\n-\tconst char *packed_refs_file = refs->packed_ref_store->path;\n+\tif (!is_lock_file_locked(&refs->lock))\n+\t\tvalidate_packed_ref_cache(refs);\n \n-\tif (!is_lock_file_locked(&refs->packed_ref_store->lock))\n-\t\tvalidate_packed_ref_cache(refs->packed_ref_store);\n-\n-\tif (!refs->packed_ref_store->cache)\n-\t\trefs->packed_ref_store->cache = read_packed_refs(packed_refs_file);\n+\tif (!refs->cache)\n+\t\trefs->cache = read_packed_refs(refs->path);\n \n-\treturn refs->packed_ref_store->cache;\n+\treturn refs->cache;\n }\n \n static struct ref_dir *get_packed_ref_dir(struct packed_ref_cache *packed_ref_cache)\n@@ -431,7 +429,7 @@ static struct ref_dir *get_packed_ref_dir(struct packed_ref_cache *packed_ref_ca\n \n static struct ref_dir *get_packed_refs(struct files_ref_store *refs)\n {\n-\treturn get_packed_ref_dir(get_packed_ref_cache(refs));\n+\treturn get_packed_ref_dir(get_packed_ref_cache(refs->packed_ref_store));\n }\n \n /*\n@@ -1151,7 +1149,7 @@ static struct ref_iterator *files_ref_iterator_begin(\n \tloose_iter = cache_ref_iterator_begin(get_loose_ref_cache(refs),\n \t\t\t\t\t prefix, 1);\n \n-\titer->packed_ref_cache = get_packed_ref_cache(refs);\n+\titer->packed_ref_cache = get_packed_ref_cache(refs->packed_ref_store);\n \tacquire_packed_ref_cache(iter->packed_ref_cache);\n \tpacked_iter = cache_ref_iterator_begin(iter->packed_ref_cache->cache,\n \t\t\t\t\t prefix, 0);\n@@ -1365,7 +1363,7 @@ static int lock_packed_refs(struct files_ref_store *refs, int flags)\n \t */\n \tvalidate_packed_ref_cache(refs->packed_ref_store);\n \n-\tpacked_ref_cache = get_packed_ref_cache(refs);\n+\tpacked_ref_cache = get_packed_ref_cache(refs->packed_ref_store);\n \t/* Increment the reference count to prevent it from being freed: */\n \tacquire_packed_ref_cache(packed_ref_cache);\n \treturn 0;\n@@ -1380,7 +1378,7 @@ static int lock_packed_refs(struct files_ref_store *refs, int flags)\n static int commit_packed_refs(struct files_ref_store *refs)\n {\n \tstruct packed_ref_cache *packed_ref_cache =\n-\t\tget_packed_ref_cache(refs);\n+\t\tget_packed_ref_cache(refs->packed_ref_store);\n \tint ok, error = 0;\n \tint save_errno = 0;\n \tFILE *out;\n@@ -1426,7 +1424,7 @@ static int commit_packed_refs(struct files_ref_store *refs)\n static void rollback_packed_refs(struct files_ref_store *refs)\n {\n \tstruct packed_ref_cache *packed_ref_cache =\n-\t\tget_packed_ref_cache(refs);\n+\t\tget_packed_ref_cache(refs->packed_ref_store);\n \n \tfiles_assert_main_repository(refs, \"rollback_packed_refs\");\n \n-- \n2.11.0\n\n"},{"id":"322332","messageId":"7581705d9afade88b663aaaa9451fd128c749b2a.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 08/28] get_packed_refs(): take a `packed_ref_store *` parameter","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:13Z","receivedAt":"2017-06-15T14:48:06Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"It only cares about the packed-refs part of the reference store.\n\nSigned-off-by: Michael Haggerty \n---\n refs/files-backend.c | 10 +++++-----\n 1 file changed, 5 insertions(+), 5 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex b2ef7b3bb9..bc5c0de84e 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -427,9 +427,9 @@ static struct ref_dir *get_packed_ref_dir(struct packed_ref_cache *packed_ref_ca\n \treturn get_ref_dir(packed_ref_cache->cache->root);\n }\n \n-static struct ref_dir *get_packed_refs(struct files_ref_store *refs)\n+static struct ref_dir *get_packed_refs(struct packed_ref_store *refs)\n {\n-\treturn get_packed_ref_dir(get_packed_ref_cache(refs->packed_ref_store));\n+\treturn get_packed_ref_dir(get_packed_ref_cache(refs));\n }\n \n /*\n@@ -450,7 +450,7 @@ static void add_packed_ref(struct files_ref_store *refs,\n \tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL))\n \t\tdie(\"Reference has invalid format: '%s'\", refname);\n \n-\tpacked_refs = get_packed_refs(refs);\n+\tpacked_refs = get_packed_refs(refs->packed_ref_store);\n \tpacked_entry = find_ref_entry(packed_refs, refname);\n \tif (packed_entry) {\n \t\t/* Overwrite the existing entry: */\n@@ -592,7 +592,7 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)\n static struct ref_entry *get_packed_ref(struct files_ref_store *refs,\n \t\t\t\t\tconst char *refname)\n {\n-\treturn find_ref_entry(get_packed_refs(refs), refname);\n+\treturn find_ref_entry(get_packed_refs(refs->packed_ref_store), refname);\n }\n \n /*\n@@ -1633,7 +1633,7 @@ static int repack_without_refs(struct files_ref_store *refs,\n \t\tunable_to_lock_message(refs->packed_ref_store->path, errno, err);\n \t\treturn -1;\n \t}\n-\tpacked = get_packed_refs(refs);\n+\tpacked = get_packed_refs(refs->packed_ref_store);\n \n \t/* Remove refnames from the cache */\n \tfor_each_string_list_item(refname, refnames)\n-- \n2.11.0\n\n"},{"id":"322333","messageId":"c3d68e8b44bae235af802ebed7cdaaf5eedc3508.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 09/28] add_packed_ref(): take a `packed_ref_store *` parameter","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:14Z","receivedAt":"2017-06-15T14:48:07Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"It only cares about the packed-refs part of the reference store.\n\nSigned-off-by: Michael Haggerty \n---\n refs/files-backend.c | 10 +++++-----\n 1 file changed, 5 insertions(+), 5 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex bc5c0de84e..4943207098 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -438,19 +438,19 @@ static struct ref_dir *get_packed_refs(struct packed_ref_store *refs)\n * (see lock_packed_refs()). To actually write the packed-refs file,\n * call commit_packed_refs().\n */\n-static void add_packed_ref(struct files_ref_store *refs,\n+static void add_packed_ref(struct packed_ref_store *refs,\n \t\t\t const char *refname, const struct object_id *oid)\n {\n \tstruct ref_dir *packed_refs;\n \tstruct ref_entry *packed_entry;\n \n-\tif (!is_lock_file_locked(&refs->packed_ref_store->lock))\n+\tif (!is_lock_file_locked(&refs->lock))\n \t\tdie(\"BUG: packed refs not locked\");\n \n \tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL))\n \t\tdie(\"Reference has invalid format: '%s'\", refname);\n \n-\tpacked_refs = get_packed_refs(refs->packed_ref_store);\n+\tpacked_refs = get_packed_refs(refs);\n \tpacked_entry = find_ref_entry(packed_refs, refname);\n \tif (packed_entry) {\n \t\t/* Overwrite the existing entry: */\n@@ -1579,7 +1579,7 @@ static int files_pack_refs(struct ref_store *ref_store, unsigned int flags)\n \t\t * we don't copy the peeled status, because we want it\n \t\t * to be re-peeled.\n \t\t */\n-\t\tadd_packed_ref(refs, iter->refname, iter->oid);\n+\t\tadd_packed_ref(refs->packed_ref_store, iter->refname, iter->oid);\n \n \t\t/* Schedule the loose reference for pruning if requested. */\n \t\tif ((flags & PACK_REFS_PRUNE)) {\n@@ -3210,7 +3210,7 @@ static int files_initial_transaction_commit(struct ref_store *ref_store,\n \n \t\tif ((update->flags & REF_HAVE_NEW) &&\n \t\t !is_null_oid(&update->new_oid))\n-\t\t\tadd_packed_ref(refs, update->refname,\n+\t\t\tadd_packed_ref(refs->packed_ref_store, update->refname,\n \t\t\t\t &update->new_oid);\n \t}\n \n-- \n2.11.0\n\n"},{"id":"322334","messageId":"91360272dd266d2635721b5afac9fa2fbda5494f.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 10/28] lock_packed_refs(): take a `packed_ref_store *` parameter","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:15Z","receivedAt":"2017-06-15T14:48:10Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"It only cares about the packed-refs part of the reference store.\n\nSigned-off-by: Michael Haggerty \n---\n refs/files-backend.c | 31 ++++++++++++++++++++++---------\n 1 file changed, 22 insertions(+), 9 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 4943207098..0d8f39089f 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -80,6 +80,19 @@ static struct packed_ref_store *packed_ref_store_create(\n \treturn refs;\n }\n \n+/*\n+ * Die if refs is not the main ref store. caller is used in any\n+ * necessary error messages.\n+ */\n+static void packed_assert_main_repository(struct packed_ref_store *refs,\n+\t\t\t\t\t const char *caller)\n+{\n+\tif (refs->store_flags & REF_STORE_MAIN)\n+\t\treturn;\n+\n+\tdie(\"BUG: operation %s only allowed for main ref store\", caller);\n+}\n+\n /*\n * Future: need to be in \"struct repository\"\n * when doing a full libification.\n@@ -1334,13 +1347,13 @@ static void write_packed_entry(FILE *fh, const char *refname,\n * hold_lock_file_for_update(). Return 0 on success. On errors, set\n * errno appropriately and return a nonzero value.\n */\n-static int lock_packed_refs(struct files_ref_store *refs, int flags)\n+static int lock_packed_refs(struct packed_ref_store *refs, int flags)\n {\n \tstatic int timeout_configured = 0;\n \tstatic int timeout_value = 1000;\n \tstruct packed_ref_cache *packed_ref_cache;\n \n-\tfiles_assert_main_repository(refs, \"lock_packed_refs\");\n+\tpacked_assert_main_repository(refs, \"lock_packed_refs\");\n \n \tif (!timeout_configured) {\n \t\tgit_config_get_int(\"core.packedrefstimeout\", &timeout_value);\n@@ -1348,8 +1361,8 @@ static int lock_packed_refs(struct files_ref_store *refs, int flags)\n \t}\n \n \tif (hold_lock_file_for_update_timeout(\n-\t\t\t &refs->packed_ref_store->lock,\n-\t\t\t refs->packed_ref_store->path,\n+\t\t\t &refs->lock,\n+\t\t\t refs->path,\n \t\t\t flags, timeout_value) < 0)\n \t\treturn -1;\n \n@@ -1361,9 +1374,9 @@ static int lock_packed_refs(struct files_ref_store *refs, int flags)\n \t * cache is still valid. We've just locked the file, but it\n \t * might have changed the moment *before* we locked it.\n \t */\n-\tvalidate_packed_ref_cache(refs->packed_ref_store);\n+\tvalidate_packed_ref_cache(refs);\n \n-\tpacked_ref_cache = get_packed_ref_cache(refs->packed_ref_store);\n+\tpacked_ref_cache = get_packed_ref_cache(refs);\n \t/* Increment the reference count to prevent it from being freed: */\n \tacquire_packed_ref_cache(packed_ref_cache);\n \treturn 0;\n@@ -1560,7 +1573,7 @@ static int files_pack_refs(struct ref_store *ref_store, unsigned int flags)\n \tint ok;\n \tstruct ref_to_prune *refs_to_prune = NULL;\n \n-\tlock_packed_refs(refs, LOCK_DIE_ON_ERROR);\n+\tlock_packed_refs(refs->packed_ref_store, LOCK_DIE_ON_ERROR);\n \n \titer = cache_ref_iterator_begin(get_loose_ref_cache(refs), NULL, 0);\n \twhile ((ok = ref_iterator_advance(iter)) == ITER_OK) {\n@@ -1629,7 +1642,7 @@ static int repack_without_refs(struct files_ref_store *refs,\n \tif (!needs_repacking)\n \t\treturn 0; /* no refname exists in packed refs */\n \n-\tif (lock_packed_refs(refs, 0)) {\n+\tif (lock_packed_refs(refs->packed_ref_store, 0)) {\n \t\tunable_to_lock_message(refs->packed_ref_store->path, errno, err);\n \t\treturn -1;\n \t}\n@@ -3198,7 +3211,7 @@ static int files_initial_transaction_commit(struct ref_store *ref_store,\n \t\t}\n \t}\n \n-\tif (lock_packed_refs(refs, 0)) {\n+\tif (lock_packed_refs(refs->packed_ref_store, 0)) {\n \t\tstrbuf_addf(err, \"unable to lock packed-refs file: %s\",\n \t\t\t strerror(errno));\n \t\tret = TRANSACTION_GENERIC_ERROR;\n-- \n2.11.0\n\n"},{"id":"322335","messageId":"1af134b756a56acd0c2f651708c1d8e67d6b6f52.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 11/28] commit_packed_refs(): take a `packed_ref_store *` parameter","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:16Z","receivedAt":"2017-06-15T14:48:12Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"It only cares about the packed-refs part of the reference store.\n\nSigned-off-by: Michael Haggerty \n---\n refs/files-backend.c | 18 +++++++++---------\n 1 file changed, 9 insertions(+), 9 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 0d8f39089f..5d159620f0 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1388,21 +1388,21 @@ static int lock_packed_refs(struct packed_ref_store *refs, int flags)\n * lock_packed_refs()). Return zero on success. On errors, set errno\n * and return a nonzero value\n */\n-static int commit_packed_refs(struct files_ref_store *refs)\n+static int commit_packed_refs(struct packed_ref_store *refs)\n {\n \tstruct packed_ref_cache *packed_ref_cache =\n-\t\tget_packed_ref_cache(refs->packed_ref_store);\n+\t\tget_packed_ref_cache(refs);\n \tint ok, error = 0;\n \tint save_errno = 0;\n \tFILE *out;\n \tstruct ref_iterator *iter;\n \n-\tfiles_assert_main_repository(refs, \"commit_packed_refs\");\n+\tpacked_assert_main_repository(refs, \"commit_packed_refs\");\n \n-\tif (!is_lock_file_locked(&refs->packed_ref_store->lock))\n+\tif (!is_lock_file_locked(&refs->lock))\n \t\tdie(\"BUG: packed-refs not locked\");\n \n-\tout = fdopen_lock_file(&refs->packed_ref_store->lock, \"w\");\n+\tout = fdopen_lock_file(&refs->lock, \"w\");\n \tif (!out)\n \t\tdie_errno(\"unable to fdopen packed-refs descriptor\");\n \n@@ -1420,7 +1420,7 @@ static int commit_packed_refs(struct files_ref_store *refs)\n \tif (ok != ITER_DONE)\n \t\tdie(\"error while iterating over references\");\n \n-\tif (commit_lock_file(&refs->packed_ref_store->lock)) {\n+\tif (commit_lock_file(&refs->lock)) {\n \t\tsave_errno = errno;\n \t\terror = -1;\n \t}\n@@ -1606,7 +1606,7 @@ static int files_pack_refs(struct ref_store *ref_store, unsigned int flags)\n \tif (ok != ITER_DONE)\n \t\tdie(\"error while iterating over references\");\n \n-\tif (commit_packed_refs(refs))\n+\tif (commit_packed_refs(refs->packed_ref_store))\n \t\tdie_errno(\"unable to overwrite old ref-pack file\");\n \n \tprune_refs(refs, refs_to_prune);\n@@ -1662,7 +1662,7 @@ static int repack_without_refs(struct files_ref_store *refs,\n \t}\n \n \t/* Write what remains */\n-\tret = commit_packed_refs(refs);\n+\tret = commit_packed_refs(refs->packed_ref_store);\n \tif (ret)\n \t\tstrbuf_addf(err, \"unable to overwrite old ref-pack file: %s\",\n \t\t\t strerror(errno));\n@@ -3227,7 +3227,7 @@ static int files_initial_transaction_commit(struct ref_store *ref_store,\n \t\t\t\t &update->new_oid);\n \t}\n \n-\tif (commit_packed_refs(refs)) {\n+\tif (commit_packed_refs(refs->packed_ref_store)) {\n \t\tstrbuf_addf(err, \"unable to commit packed-refs file: %s\",\n \t\t\t strerror(errno));\n \t\tret = TRANSACTION_GENERIC_ERROR;\n-- \n2.11.0\n\n"},{"id":"322336","messageId":"ff3d5b9fa4bbced88b91a19f93a39c56dd8f2989.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 13/28] get_packed_ref(): take a `packed_ref_store *` parameter","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:18Z","receivedAt":"2017-06-15T14:48:16Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"It only cares about the packed-refs part of the reference store.\n\nSigned-off-by: Michael Haggerty \n---\n refs/files-backend.c | 12 +++++++-----\n 1 file changed, 7 insertions(+), 5 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex a08d3fbadf..2b9d93d3b6 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -602,10 +602,10 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)\n * Return the ref_entry for the given refname from the packed\n * references. If it does not exist, return NULL.\n */\n-static struct ref_entry *get_packed_ref(struct files_ref_store *refs,\n+static struct ref_entry *get_packed_ref(struct packed_ref_store *refs,\n \t\t\t\t\tconst char *refname)\n {\n-\treturn find_ref_entry(get_packed_refs(refs->packed_ref_store), refname);\n+\treturn find_ref_entry(get_packed_refs(refs), refname);\n }\n \n /*\n@@ -621,7 +621,7 @@ static int resolve_packed_ref(struct files_ref_store *refs,\n \t * The loose reference file does not exist; check for a packed\n \t * reference.\n \t */\n-\tentry = get_packed_ref(refs, refname);\n+\tentry = get_packed_ref(refs->packed_ref_store, refname);\n \tif (entry) {\n \t\thashcpy(sha1, entry->u.value.oid.hash);\n \t\t*flags |= REF_ISPACKED;\n@@ -1044,7 +1044,9 @@ static int files_peel_ref(struct ref_store *ref_store,\n \t * have REF_KNOWS_PEELED.\n \t */\n \tif (flag & REF_ISPACKED) {\n-\t\tstruct ref_entry *r = get_packed_ref(refs, refname);\n+\t\tstruct ref_entry *r =\n+\t\t\tget_packed_ref(refs->packed_ref_store, refname);\n+\n \t\tif (r) {\n \t\t\tif (peel_entry(r, 0))\n \t\t\t\treturn -1;\n@@ -1631,7 +1633,7 @@ static int repack_without_refs(struct files_ref_store *refs,\n \n \t/* Look for a packed ref */\n \tfor_each_string_list_item(refname, refnames) {\n-\t\tif (get_packed_ref(refs, refname->string)) {\n+\t\tif (get_packed_ref(refs->packed_ref_store, refname->string)) {\n \t\t\tneeds_repacking = 1;\n \t\t\tbreak;\n \t\t}\n-- \n2.11.0\n\n"},{"id":"322337","messageId":"ce5f2c68da927408a49a76c51b18d3378958ea76.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 14/28] repack_without_refs(): take a `packed_ref_store *` parameter","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:19Z","receivedAt":"2017-06-15T14:48:18Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"It only cares about the packed-refs part of the reference store.\n\nSigned-off-by: Michael Haggerty \n---\n refs/files-backend.c | 20 ++++++++++----------\n 1 file changed, 10 insertions(+), 10 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 2b9d93d3b6..c206791b91 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1621,19 +1621,19 @@ static int files_pack_refs(struct ref_store *ref_store, unsigned int flags)\n *\n * The refs in 'refnames' needn't be sorted. `err` must not be NULL.\n */\n-static int repack_without_refs(struct files_ref_store *refs,\n+static int repack_without_refs(struct packed_ref_store *refs,\n \t\t\t struct string_list *refnames, struct strbuf *err)\n {\n \tstruct ref_dir *packed;\n \tstruct string_list_item *refname;\n \tint ret, needs_repacking = 0, removed = 0;\n \n-\tfiles_assert_main_repository(refs, \"repack_without_refs\");\n+\tpacked_assert_main_repository(refs, \"repack_without_refs\");\n \tassert(err);\n \n \t/* Look for a packed ref */\n \tfor_each_string_list_item(refname, refnames) {\n-\t\tif (get_packed_ref(refs->packed_ref_store, refname->string)) {\n+\t\tif (get_packed_ref(refs, refname->string)) {\n \t\t\tneeds_repacking = 1;\n \t\t\tbreak;\n \t\t}\n@@ -1643,11 +1643,11 @@ static int repack_without_refs(struct files_ref_store *refs,\n \tif (!needs_repacking)\n \t\treturn 0; /* no refname exists in packed refs */\n \n-\tif (lock_packed_refs(refs->packed_ref_store, 0)) {\n-\t\tunable_to_lock_message(refs->packed_ref_store->path, errno, err);\n+\tif (lock_packed_refs(refs, 0)) {\n+\t\tunable_to_lock_message(refs->path, errno, err);\n \t\treturn -1;\n \t}\n-\tpacked = get_packed_refs(refs->packed_ref_store);\n+\tpacked = get_packed_refs(refs);\n \n \t/* Remove refnames from the cache */\n \tfor_each_string_list_item(refname, refnames)\n@@ -1658,12 +1658,12 @@ static int repack_without_refs(struct files_ref_store *refs,\n \t\t * All packed entries disappeared while we were\n \t\t * acquiring the lock.\n \t\t */\n-\t\trollback_packed_refs(refs->packed_ref_store);\n+\t\trollback_packed_refs(refs);\n \t\treturn 0;\n \t}\n \n \t/* Write what remains */\n-\tret = commit_packed_refs(refs->packed_ref_store);\n+\tret = commit_packed_refs(refs);\n \tif (ret)\n \t\tstrbuf_addf(err, \"unable to overwrite old ref-pack file: %s\",\n \t\t\t strerror(errno));\n@@ -1681,7 +1681,7 @@ static int files_delete_refs(struct ref_store *ref_store, const char *msg,\n \tif (!refnames->nr)\n \t\treturn 0;\n \n-\tresult = repack_without_refs(refs, refnames, &err);\n+\tresult = repack_without_refs(refs->packed_ref_store, refnames, &err);\n \tif (result) {\n \t\t/*\n \t\t * If we failed to rewrite the packed-refs file, then\n@@ -3101,7 +3101,7 @@ static int files_transaction_finish(struct ref_store *ref_store,\n \t\t}\n \t}\n \n-\tif (repack_without_refs(refs, &refs_to_delete, err)) {\n+\tif (repack_without_refs(refs->packed_ref_store, &refs_to_delete, err)) {\n \t\tret = TRANSACTION_GENERIC_ERROR;\n \t\tgoto cleanup;\n \t}\n-- \n2.11.0\n\n"},{"id":"322338","messageId":"f9c4c47e52c076c42896d22d89591a93163afda3.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 15/28] packed_peel_ref(): new function, extracted from `files_peel_ref()`","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:20Z","receivedAt":"2017-06-15T14:48:20Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"This will later become a method of `packed_ref_store`.\n\nSigned-off-by: Michael Haggerty \n---\n refs/files-backend.c | 26 +++++++++++++++-----------\n 1 file changed, 15 insertions(+), 11 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex c206791b91..185d05e1d6 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1013,6 +1013,18 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \treturn ret;\n }\n \n+static int packed_peel_ref(struct packed_ref_store *refs,\n+\t\t\t const char *refname, unsigned char *sha1)\n+{\n+\tstruct ref_entry *r = get_packed_ref(refs, refname);\n+\n+\tif (!r || peel_entry(r, 0))\n+\t\treturn -1;\n+\n+\thashcpy(sha1, r->u.value.peeled.hash);\n+\treturn 0;\n+}\n+\n static int files_peel_ref(struct ref_store *ref_store,\n \t\t\t const char *refname, unsigned char *sha1)\n {\n@@ -1043,17 +1055,9 @@ static int files_peel_ref(struct ref_store *ref_store,\n \t * be expensive and (b) loose references anyway usually do not\n \t * have REF_KNOWS_PEELED.\n \t */\n-\tif (flag & REF_ISPACKED) {\n-\t\tstruct ref_entry *r =\n-\t\t\tget_packed_ref(refs->packed_ref_store, refname);\n-\n-\t\tif (r) {\n-\t\t\tif (peel_entry(r, 0))\n-\t\t\t\treturn -1;\n-\t\t\thashcpy(sha1, r->u.value.peeled.hash);\n-\t\t\treturn 0;\n-\t\t}\n-\t}\n+\tif (flag & REF_ISPACKED &&\n+\t !packed_peel_ref(refs->packed_ref_store, refname, sha1))\n+\t\treturn 0;\n \n \treturn peel_object(base, sha1);\n }\n-- \n2.11.0\n\n"},{"id":"322339","messageId":"ca6b06eb7071305e5ddbd3803263285ca89785b3.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 16/28] packed_ref_store: support iteration","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:21Z","receivedAt":"2017-06-15T14:48:22Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"Add the infrastructure to iterate over a `packed_ref_store`. It's a\nlot of boilerplate, but it's all part of a campaign to make\n`packed_ref_store` implement `ref_store`. In the future, this iterator\nwill work much differently.\n\nSigned-off-by: Michael Haggerty \n---\n refs/files-backend.c | 107 ++++++++++++++++++++++++++++++++++++++++++++++-----\n 1 file changed, 98 insertions(+), 9 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 185d05e1d6..e57cdeba36 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1062,10 +1062,102 @@ static int files_peel_ref(struct ref_store *ref_store,\n \treturn peel_object(base, sha1);\n }\n \n+struct packed_ref_iterator {\n+\tstruct ref_iterator base;\n+\n+\tstruct packed_ref_cache *cache;\n+\tstruct ref_iterator *iter0;\n+\tunsigned int flags;\n+};\n+\n+static int packed_ref_iterator_advance(struct ref_iterator *ref_iterator)\n+{\n+\tstruct packed_ref_iterator *iter =\n+\t\t(struct packed_ref_iterator *)ref_iterator;\n+\tint ok;\n+\n+\twhile ((ok = ref_iterator_advance(iter->iter0)) == ITER_OK) {\n+\t\tif (iter->flags & DO_FOR_EACH_PER_WORKTREE_ONLY &&\n+\t\t ref_type(iter->iter0->refname) != REF_TYPE_PER_WORKTREE)\n+\t\t\tcontinue;\n+\n+\t\tif (!(iter->flags & DO_FOR_EACH_INCLUDE_BROKEN) &&\n+\t\t !ref_resolves_to_object(iter->iter0->refname,\n+\t\t\t\t\t iter->iter0->oid,\n+\t\t\t\t\t iter->iter0->flags))\n+\t\t\tcontinue;\n+\n+\t\titer->base.refname = iter->iter0->refname;\n+\t\titer->base.oid = iter->iter0->oid;\n+\t\titer->base.flags = iter->iter0->flags;\n+\t\treturn ITER_OK;\n+\t}\n+\n+\titer->iter0 = NULL;\n+\tif (ref_iterator_abort(ref_iterator) != ITER_DONE)\n+\t\tok = ITER_ERROR;\n+\n+\treturn ok;\n+}\n+\n+static int packed_ref_iterator_peel(struct ref_iterator *ref_iterator,\n+\t\t\t\t struct object_id *peeled)\n+{\n+\tstruct packed_ref_iterator *iter =\n+\t\t(struct packed_ref_iterator *)ref_iterator;\n+\n+\treturn ref_iterator_peel(iter->iter0, peeled);\n+}\n+\n+static int packed_ref_iterator_abort(struct ref_iterator *ref_iterator)\n+{\n+\tstruct packed_ref_iterator *iter =\n+\t\t(struct packed_ref_iterator *)ref_iterator;\n+\tint ok = ITER_DONE;\n+\n+\tif (iter->iter0)\n+\t\tok = ref_iterator_abort(iter->iter0);\n+\n+\trelease_packed_ref_cache(iter->cache);\n+\tbase_ref_iterator_free(ref_iterator);\n+\treturn ok;\n+}\n+\n+static struct ref_iterator_vtable packed_ref_iterator_vtable = {\n+\tpacked_ref_iterator_advance,\n+\tpacked_ref_iterator_peel,\n+\tpacked_ref_iterator_abort\n+};\n+\n+static struct ref_iterator *packed_ref_iterator_begin(\n+\t\tstruct packed_ref_store *refs,\n+\t\tconst char *prefix, unsigned int flags)\n+{\n+\tstruct packed_ref_iterator *iter;\n+\tstruct ref_iterator *ref_iterator;\n+\n+\titer = xcalloc(1, sizeof(*iter));\n+\tref_iterator = &iter->base;\n+\tbase_ref_iterator_init(ref_iterator, &packed_ref_iterator_vtable);\n+\n+\t/*\n+\t * Note that get_packed_ref_cache() internally checks whether\n+\t * the packed-ref cache is up to date with what is on disk,\n+\t * and re-reads it if not.\n+\t */\n+\n+\titer->cache = get_packed_ref_cache(refs);\n+\tacquire_packed_ref_cache(iter->cache);\n+\titer->iter0 = cache_ref_iterator_begin(iter->cache->cache, prefix, 0);\n+\n+\titer->flags = flags;\n+\n+\treturn ref_iterator;\n+}\n+\n struct files_ref_iterator {\n \tstruct ref_iterator base;\n \n-\tstruct packed_ref_cache *packed_ref_cache;\n \tstruct ref_iterator *iter0;\n \tunsigned int flags;\n };\n@@ -1118,7 +1210,6 @@ static int files_ref_iterator_abort(struct ref_iterator *ref_iterator)\n \tif (iter->iter0)\n \t\tok = ref_iterator_abort(iter->iter0);\n \n-\trelease_packed_ref_cache(iter->packed_ref_cache);\n \tbase_ref_iterator_free(ref_iterator);\n \treturn ok;\n }\n@@ -1160,18 +1251,16 @@ static struct ref_iterator *files_ref_iterator_begin(\n \t * (If they've already been read, that's OK; we only need to\n \t * guarantee that they're read before the packed refs, not\n \t * *how much* before.) After that, we call\n-\t * get_packed_ref_cache(), which internally checks whether the\n-\t * packed-ref cache is up to date with what is on disk, and\n-\t * re-reads it if not.\n+\t * packed_ref_iterator_begin(), which internally checks\n+\t * whether the packed-ref cache is up to date with what is on\n+\t * disk, and re-reads it if not.\n \t */\n \n \tloose_iter = cache_ref_iterator_begin(get_loose_ref_cache(refs),\n \t\t\t\t\t prefix, 1);\n \n-\titer->packed_ref_cache = get_packed_ref_cache(refs->packed_ref_store);\n-\tacquire_packed_ref_cache(iter->packed_ref_cache);\n-\tpacked_iter = cache_ref_iterator_begin(iter->packed_ref_cache->cache,\n-\t\t\t\t\t prefix, 0);\n+\tpacked_iter = packed_ref_iterator_begin(refs->packed_ref_store,\n+\t\t\t\t\t\tprefix, 0);\n \n \titer->iter0 = overlay_ref_iterator_begin(loose_iter, packed_iter);\n \titer->flags = flags;\n-- \n2.11.0\n\n"},{"id":"322340","messageId":"5b565b2eced3b91819090707779e63908970833d.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 17/28] packed_read_raw_ref(): new function, replacing `resolve_packed_ref()`","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:22Z","receivedAt":"2017-06-15T14:48:27Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"Add a new function, `packed_read_raw_ref()`, which is nearly a\n`read_raw_ref_fn`. Use it in place of `resolve_packed_ref()`.\n\nSigned-off-by: Michael Haggerty \n---\n refs/files-backend.c | 36 +++++++++++++++++-------------------\n 1 file changed, 17 insertions(+), 19 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex e57cdeba36..ac4764f6f7 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -608,27 +608,23 @@ static struct ref_entry *get_packed_ref(struct packed_ref_store *refs,\n \treturn find_ref_entry(get_packed_refs(refs), refname);\n }\n \n-/*\n- * A loose ref file doesn't exist; check for a packed ref.\n- */\n-static int resolve_packed_ref(struct files_ref_store *refs,\n-\t\t\t const char *refname,\n-\t\t\t unsigned char *sha1, unsigned int *flags)\n+static int packed_read_raw_ref(struct packed_ref_store *refs,\n+\t\t\t const char *refname, unsigned char *sha1,\n+\t\t\t struct strbuf *referent, unsigned int *type)\n {\n \tstruct ref_entry *entry;\n \n-\t/*\n-\t * The loose reference file does not exist; check for a packed\n-\t * reference.\n-\t */\n-\tentry = get_packed_ref(refs->packed_ref_store, refname);\n-\tif (entry) {\n-\t\thashcpy(sha1, entry->u.value.oid.hash);\n-\t\t*flags |= REF_ISPACKED;\n-\t\treturn 0;\n+\t*type = 0;\n+\n+\tentry = get_packed_ref(refs, refname);\n+\tif (!entry) {\n+\t\terrno = ENOENT;\n+\t\treturn -1;\n \t}\n-\t/* refname is not a packed reference. */\n-\treturn -1;\n+\n+\thashcpy(sha1, entry->u.value.oid.hash);\n+\t*type = REF_ISPACKED;\n+\treturn 0;\n }\n \n static int files_read_raw_ref(struct ref_store *ref_store,\n@@ -674,7 +670,8 @@ static int files_read_raw_ref(struct ref_store *ref_store,\n \tif (lstat(path, &st) < 0) {\n \t\tif (errno != ENOENT)\n \t\t\tgoto out;\n-\t\tif (resolve_packed_ref(refs, refname, sha1, type)) {\n+\t\tif (packed_read_raw_ref(refs->packed_ref_store, refname,\n+\t\t\t\t\tsha1, referent, type)) {\n \t\t\terrno = ENOENT;\n \t\t\tgoto out;\n \t\t}\n@@ -713,7 +710,8 @@ static int files_read_raw_ref(struct ref_store *ref_store,\n \t\t * ref is supposed to be, there could still be a\n \t\t * packed ref:\n \t\t */\n-\t\tif (resolve_packed_ref(refs, refname, sha1, type)) {\n+\t\tif (packed_read_raw_ref(refs->packed_ref_store, refname,\n+\t\t\t\t\tsha1, referent, type)) {\n \t\t\terrno = EISDIR;\n \t\t\tgoto out;\n \t\t}\n-- \n2.11.0\n\n"},{"id":"322341","messageId":"9d0f6cb6de5e51c77cb5306be1743087eb799c66.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 19/28] packed_ref_store: make class into a subclass of `ref_store`","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:24Z","receivedAt":"2017-06-15T14:48:30Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"Add the infrastructure to make `packed_ref_store` implement\n`ref_store`, at least formally (few of the methods are actually\nimplemented yet). Change the functions in its interface to take\n`ref_store *` arguments. Change `files_ref_store` to store a pointer\nto `ref_store *` and to call functions via the virtual `ref_store`\ninterface where possible. This also means that a few\n`packed_ref_store` functions can become static.\n\nThis is a work in progress. Some more `ref_store` methods will soon be\nimplemented (e.g., those having to do with reference transactions).\nBut some of them will never be implemented (e.g., those having to do\nwith symrefs or reflogs).\n\nSigned-off-by: Michael Haggerty \n---\n refs/files-backend.c | 16 ++--\n refs/packed-backend.c | 231 +++++++++++++++++++++++++++++++++++++++++++++-----\n refs/packed-backend.h | 23 ++---\n refs/refs-internal.h | 1 +\n 4 files changed, 226 insertions(+), 45 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 83ea773728..039d0343cb 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -28,7 +28,7 @@ struct files_ref_store {\n \n \tstruct ref_cache *loose;\n \n-\tstruct packed_ref_store *packed_ref_store;\n+\tstruct ref_store *packed_ref_store;\n };\n \n static void clear_loose_ref_cache(struct files_ref_store *refs)\n@@ -311,8 +311,8 @@ static int files_read_raw_ref(struct ref_store *ref_store,\n \tif (lstat(path, &st) < 0) {\n \t\tif (errno != ENOENT)\n \t\t\tgoto out;\n-\t\tif (packed_read_raw_ref(refs->packed_ref_store, refname,\n-\t\t\t\t\tsha1, referent, type)) {\n+\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n+\t\t\t\t sha1, referent, type)) {\n \t\t\terrno = ENOENT;\n \t\t\tgoto out;\n \t\t}\n@@ -351,8 +351,8 @@ static int files_read_raw_ref(struct ref_store *ref_store,\n \t\t * ref is supposed to be, there could still be a\n \t\t * packed ref:\n \t\t */\n-\t\tif (packed_read_raw_ref(refs->packed_ref_store, refname,\n-\t\t\t\t\tsha1, referent, type)) {\n+\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n+\t\t\t\t sha1, referent, type)) {\n \t\t\terrno = EISDIR;\n \t\t\tgoto out;\n \t\t}\n@@ -683,7 +683,7 @@ static int files_peel_ref(struct ref_store *ref_store,\n \t * have REF_KNOWS_PEELED.\n \t */\n \tif (flag & REF_ISPACKED &&\n-\t !packed_peel_ref(refs->packed_ref_store, refname, sha1))\n+\t !refs_peel_ref(refs->packed_ref_store, refname, sha1))\n \t\treturn 0;\n \n \treturn peel_object(base, sha1);\n@@ -793,8 +793,8 @@ static struct ref_iterator *files_ref_iterator_begin(\n \tloose_iter = cache_ref_iterator_begin(get_loose_ref_cache(refs),\n \t\t\t\t\t prefix, 1);\n \n-\tpacked_iter = packed_ref_iterator_begin(refs->packed_ref_store,\n-\t\t\t\t\t\tprefix, 0);\n+\tpacked_iter = refs_ref_iterator_begin(refs->packed_ref_store,\n+\t\t\t\t\t prefix, 0, 0);\n \n \titer->iter0 = overlay_ref_iterator_begin(loose_iter, packed_iter);\n \titer->flags = flags;\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 6fa988c953..5dbe4e4e59 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -50,6 +50,8 @@ static int release_packed_ref_cache(struct packed_ref_cache *packed_refs)\n * `ref_store`.\n */\n struct packed_ref_store {\n+\tstruct ref_store base;\n+\n \tunsigned int store_flags;\n \n \t/* The path of the \"packed-refs\" file: */\n@@ -68,14 +70,17 @@ struct packed_ref_store {\n \tstruct lock_file lock;\n };\n \n-struct packed_ref_store *packed_ref_store_create(\n-\t\tconst char *path, unsigned int store_flags)\n+struct ref_store *packed_ref_store_create(const char *path,\n+\t\t\t\t\t unsigned int store_flags)\n {\n \tstruct packed_ref_store *refs = xcalloc(1, sizeof(*refs));\n+\tstruct ref_store *ref_store = (struct ref_store *)refs;\n \n+\tbase_ref_store_init(ref_store, &refs_be_packed);\n \trefs->store_flags = store_flags;\n+\n \trefs->path = xstrdup(path);\n-\treturn refs;\n+\treturn ref_store;\n }\n \n /*\n@@ -91,6 +96,31 @@ static void packed_assert_main_repository(struct packed_ref_store *refs,\n \tdie(\"BUG: operation %s only allowed for main ref store\", caller);\n }\n \n+/*\n+ * Downcast `ref_store` to `packed_ref_store`. Die if `ref_store` is\n+ * not a `packed_ref_store`. Also die if `packed_ref_store` doesn't\n+ * support at least the flags specified in `required_flags`. `caller`\n+ * is used in any necessary error messages.\n+ */\n+static struct packed_ref_store *packed_downcast(struct ref_store *ref_store,\n+\t\t\t\t\t\tunsigned int required_flags,\n+\t\t\t\t\t\tconst char *caller)\n+{\n+\tstruct packed_ref_store *refs;\n+\n+\tif (ref_store->be != &refs_be_packed)\n+\t\tdie(\"BUG: ref_store is type \\\"%s\\\" not \\\"packed\\\" in %s\",\n+\t\t ref_store->be->name, caller);\n+\n+\trefs = (struct packed_ref_store *)ref_store;\n+\n+\tif ((refs->store_flags & required_flags) != required_flags)\n+\t\tdie(\"BUG: unallowed operation (%s), requires %x, has %x\\n\",\n+\t\t caller, required_flags, refs->store_flags);\n+\n+\treturn refs;\n+}\n+\n static void clear_packed_ref_cache(struct packed_ref_store *refs)\n {\n \tif (refs->cache) {\n@@ -287,9 +317,12 @@ static struct ref_dir *get_packed_refs(struct packed_ref_store *refs)\n * (see lock_packed_refs()). To actually write the packed-refs file,\n * call commit_packed_refs().\n */\n-void add_packed_ref(struct packed_ref_store *refs,\n+void add_packed_ref(struct ref_store *ref_store,\n \t\t const char *refname, const struct object_id *oid)\n {\n+\tstruct packed_ref_store *refs =\n+\t\tpacked_downcast(ref_store, REF_STORE_WRITE,\n+\t\t\t\t\"add_packed_ref\");\n \tstruct ref_dir *packed_refs;\n \tstruct ref_entry *packed_entry;\n \n@@ -322,10 +355,13 @@ static struct ref_entry *get_packed_ref(struct packed_ref_store *refs,\n \treturn find_ref_entry(get_packed_refs(refs), refname);\n }\n \n-int packed_read_raw_ref(struct packed_ref_store *refs,\n-\t\t\tconst char *refname, unsigned char *sha1,\n-\t\t\tstruct strbuf *referent, unsigned int *type)\n+static int packed_read_raw_ref(struct ref_store *ref_store,\n+\t\t\t const char *refname, unsigned char *sha1,\n+\t\t\t struct strbuf *referent, unsigned int *type)\n {\n+\tstruct packed_ref_store *refs =\n+\t packed_downcast(ref_store, REF_STORE_READ, \"read_raw_ref\");\n+\n \tstruct ref_entry *entry;\n \n \t*type = 0;\n@@ -341,9 +377,12 @@ int packed_read_raw_ref(struct packed_ref_store *refs,\n \treturn 0;\n }\n \n-int packed_peel_ref(struct packed_ref_store *refs,\n-\t\t const char *refname, unsigned char *sha1)\n+static int packed_peel_ref(struct ref_store *ref_store,\n+\t\t\t const char *refname, unsigned char *sha1)\n {\n+\tstruct packed_ref_store *refs =\n+\t packed_downcast(ref_store, REF_STORE_READ | REF_STORE_ODB,\n+\t\t\t\t\"peel_ref\");\n \tstruct ref_entry *r = get_packed_ref(refs, refname);\n \n \tif (!r || peel_entry(r, 0))\n@@ -420,12 +459,18 @@ static struct ref_iterator_vtable packed_ref_iterator_vtable = {\n \tpacked_ref_iterator_abort\n };\n \n-struct ref_iterator *packed_ref_iterator_begin(\n-\t\tstruct packed_ref_store *refs,\n+static struct ref_iterator *packed_ref_iterator_begin(\n+\t\tstruct ref_store *ref_store,\n \t\tconst char *prefix, unsigned int flags)\n {\n+\tstruct packed_ref_store *refs;\n \tstruct packed_ref_iterator *iter;\n \tstruct ref_iterator *ref_iterator;\n+\tunsigned int required_flags = REF_STORE_READ;\n+\n+\tif (!(flags & DO_FOR_EACH_INCLUDE_BROKEN))\n+\t\trequired_flags |= REF_STORE_ODB;\n+\trefs = packed_downcast(ref_store, required_flags, \"ref_iterator_begin\");\n \n \titer = xcalloc(1, sizeof(*iter));\n \tref_iterator = &iter->base;\n@@ -459,14 +504,15 @@ static void write_packed_entry(FILE *fh, const char *refname,\n \t\tfprintf_or_die(fh, \"^%s\\n\", sha1_to_hex(peeled));\n }\n \n-int lock_packed_refs(struct packed_ref_store *refs, int flags)\n+int lock_packed_refs(struct ref_store *ref_store, int flags)\n {\n+\tstruct packed_ref_store *refs =\n+\t packed_downcast(ref_store, REF_STORE_WRITE | REF_STORE_MAIN,\n+\t\t\t\t\"lock_packed_refs\");\n \tstatic int timeout_configured = 0;\n \tstatic int timeout_value = 1000;\n \tstruct packed_ref_cache *packed_ref_cache;\n \n-\tpacked_assert_main_repository(refs, \"lock_packed_refs\");\n-\n \tif (!timeout_configured) {\n \t\tgit_config_get_int(\"core.packedrefstimeout\", &timeout_value);\n \t\ttimeout_configured = 1;\n@@ -507,8 +553,11 @@ static const char PACKED_REFS_HEADER[] =\n * lock_packed_refs()). Return zero on success. On errors, set errno\n * and return a nonzero value.\n */\n-int commit_packed_refs(struct packed_ref_store *refs)\n+int commit_packed_refs(struct ref_store *ref_store)\n {\n+\tstruct packed_ref_store *refs =\n+\t packed_downcast(ref_store, REF_STORE_WRITE | REF_STORE_MAIN,\n+\t\t\t\t\"commit_packed_refs\");\n \tstruct packed_ref_cache *packed_ref_cache =\n \t\tget_packed_ref_cache(refs);\n \tint ok, error = 0;\n@@ -516,8 +565,6 @@ int commit_packed_refs(struct packed_ref_store *refs)\n \tFILE *out;\n \tstruct ref_iterator *iter;\n \n-\tpacked_assert_main_repository(refs, \"commit_packed_refs\");\n-\n \tif (!is_lock_file_locked(&refs->lock))\n \t\tdie(\"BUG: packed-refs not locked\");\n \n@@ -573,9 +620,12 @@ static void rollback_packed_refs(struct packed_ref_store *refs)\n *\n * The refs in 'refnames' needn't be sorted. `err` must not be NULL.\n */\n-int repack_without_refs(struct packed_ref_store *refs,\n+int repack_without_refs(struct ref_store *ref_store,\n \t\t\tstruct string_list *refnames, struct strbuf *err)\n {\n+\tstruct packed_ref_store *refs =\n+\t packed_downcast(ref_store, REF_STORE_WRITE | REF_STORE_MAIN,\n+\t\t\t\t\"repack_without_refs\");\n \tstruct ref_dir *packed;\n \tstruct string_list_item *refname;\n \tint ret, needs_repacking = 0, removed = 0;\n@@ -595,7 +645,7 @@ int repack_without_refs(struct packed_ref_store *refs,\n \tif (!needs_repacking)\n \t\treturn 0; /* no refname exists in packed refs */\n \n-\tif (lock_packed_refs(refs, 0)) {\n+\tif (lock_packed_refs(&refs->base, 0)) {\n \t\tunable_to_lock_message(refs->path, errno, err);\n \t\treturn -1;\n \t}\n@@ -615,10 +665,151 @@ int repack_without_refs(struct packed_ref_store *refs,\n \t}\n \n \t/* Write what remains */\n-\tret = commit_packed_refs(refs);\n+\tret = commit_packed_refs(&refs->base);\n \tif (ret)\n \t\tstrbuf_addf(err, \"unable to overwrite old ref-pack file: %s\",\n \t\t\t strerror(errno));\n \treturn ret;\n }\n \n+static int packed_init_db(struct ref_store *ref_store, struct strbuf *err)\n+{\n+\t/* Nothing to do. */\n+\treturn 0;\n+}\n+\n+static int packed_transaction_prepare(struct ref_store *ref_store,\n+\t\t\t\t struct ref_transaction *transaction,\n+\t\t\t\t struct strbuf *err)\n+{\n+\tdie(\"BUG: not implemented yet\");\n+}\n+\n+static int packed_transaction_abort(struct ref_store *ref_store,\n+\t\t\t\t struct ref_transaction *transaction,\n+\t\t\t\t struct strbuf *err)\n+{\n+\tdie(\"BUG: not implemented yet\");\n+}\n+\n+static int packed_transaction_finish(struct ref_store *ref_store,\n+\t\t\t\t struct ref_transaction *transaction,\n+\t\t\t\t struct strbuf *err)\n+{\n+\tdie(\"BUG: not implemented yet\");\n+}\n+\n+static int packed_initial_transaction_commit(struct ref_store *ref_store,\n+\t\t\t\t\t struct ref_transaction *transaction,\n+\t\t\t\t\t struct strbuf *err)\n+{\n+\treturn ref_transaction_commit(transaction, err);\n+}\n+\n+static int packed_delete_refs(struct ref_store *ref_store, const char *msg,\n+\t\t\t struct string_list *refnames, unsigned int flags)\n+{\n+\tdie(\"BUG: not implemented yet\");\n+}\n+\n+static int packed_pack_refs(struct ref_store *ref_store, unsigned int flags)\n+{\n+\t/*\n+\t * Packed refs are already packed. It might be that loose refs\n+\t * are packed *into* a packed refs store, but that is done by\n+\t * updating the packed references via a transaction.\n+\t */\n+\treturn 0;\n+}\n+\n+static int packed_create_symref(struct ref_store *ref_store,\n+\t\t\t const char *refname, const char *target,\n+\t\t\t const char *logmsg)\n+{\n+\tdie(\"BUG: packed reference store does not support symrefs\");\n+}\n+\n+static int packed_rename_ref(struct ref_store *ref_store,\n+\t\t\t const char *oldrefname, const char *newrefname,\n+\t\t\t const char *logmsg)\n+{\n+\tdie(\"BUG: packed reference store does not support renaming references\");\n+}\n+\n+static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_store)\n+{\n+\treturn empty_ref_iterator_begin();\n+}\n+\n+static int packed_for_each_reflog_ent(struct ref_store *ref_store,\n+\t\t\t\t const char *refname,\n+\t\t\t\t each_reflog_ent_fn fn, void *cb_data)\n+{\n+\treturn 0;\n+}\n+\n+static int packed_for_each_reflog_ent_reverse(struct ref_store *ref_store,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t each_reflog_ent_fn fn,\n+\t\t\t\t\t void *cb_data)\n+{\n+\treturn 0;\n+}\n+\n+static int packed_reflog_exists(struct ref_store *ref_store,\n+\t\t\t const char *refname)\n+{\n+\treturn 0;\n+}\n+\n+static int packed_create_reflog(struct ref_store *ref_store,\n+\t\t\t const char *refname, int force_create,\n+\t\t\t struct strbuf *err)\n+{\n+\tdie(\"BUG: packed reference store does not support reflogs\");\n+}\n+\n+static int packed_delete_reflog(struct ref_store *ref_store,\n+\t\t\t const char *refname)\n+{\n+\treturn 0;\n+}\n+\n+static int packed_reflog_expire(struct ref_store *ref_store,\n+\t\t\t\tconst char *refname, const unsigned char *sha1,\n+\t\t\t\tunsigned int flags,\n+\t\t\t\treflog_expiry_prepare_fn prepare_fn,\n+\t\t\t\treflog_expiry_should_prune_fn should_prune_fn,\n+\t\t\t\treflog_expiry_cleanup_fn cleanup_fn,\n+\t\t\t\tvoid *policy_cb_data)\n+{\n+\treturn 0;\n+}\n+\n+struct ref_storage_be refs_be_packed = {\n+\tNULL,\n+\t\"packed\",\n+\tpacked_ref_store_create,\n+\tpacked_init_db,\n+\tpacked_transaction_prepare,\n+\tpacked_transaction_finish,\n+\tpacked_transaction_abort,\n+\tpacked_initial_transaction_commit,\n+\n+\tpacked_pack_refs,\n+\tpacked_peel_ref,\n+\tpacked_create_symref,\n+\tpacked_delete_refs,\n+\tpacked_rename_ref,\n+\n+\tpacked_ref_iterator_begin,\n+\tpacked_read_raw_ref,\n+\n+\tpacked_reflog_iterator_begin,\n+\tpacked_for_each_reflog_ent,\n+\tpacked_for_each_reflog_ent_reverse,\n+\tpacked_reflog_exists,\n+\tpacked_create_reflog,\n+\tpacked_delete_reflog,\n+\tpacked_reflog_expire\n+};\ndiff --git a/refs/packed-backend.h b/refs/packed-backend.h\nindex 22e8817ac4..beea9c14b5 100644\n--- a/refs/packed-backend.h\n+++ b/refs/packed-backend.h\n@@ -1,33 +1,22 @@\n #ifndef REFS_PACKED_BACKEND_H\n #define REFS_PACKED_BACKEND_H\n \n-struct packed_ref_store *packed_ref_store_create(\n-\t\tconst char *path, unsigned int store_flags);\n-\n-int packed_read_raw_ref(struct packed_ref_store *refs,\n-\t\t\tconst char *refname, unsigned char *sha1,\n-\t\t\tstruct strbuf *referent, unsigned int *type);\n-\n-int packed_peel_ref(struct packed_ref_store *refs,\n-\t\t const char *refname, unsigned char *sha1);\n-\n-struct ref_iterator *packed_ref_iterator_begin(\n-\t\tstruct packed_ref_store *refs,\n-\t\tconst char *prefix, unsigned int flags);\n+struct ref_store *packed_ref_store_create(const char *path,\n+\t\t\t\t\t unsigned int store_flags);\n \n /*\n * Lock the packed-refs file for writing. Flags is passed to\n * hold_lock_file_for_update(). Return 0 on success. On errors, set\n * errno appropriately and return a nonzero value.\n */\n-int lock_packed_refs(struct packed_ref_store *refs, int flags);\n+int lock_packed_refs(struct ref_store *ref_store, int flags);\n \n-void add_packed_ref(struct packed_ref_store *refs,\n+void add_packed_ref(struct ref_store *ref_store,\n \t\t const char *refname, const struct object_id *oid);\n \n-int commit_packed_refs(struct packed_ref_store *refs);\n+int commit_packed_refs(struct ref_store *ref_store);\n \n-int repack_without_refs(struct packed_ref_store *refs,\n+int repack_without_refs(struct ref_store *ref_store,\n \t\t\tstruct string_list *refnames, struct strbuf *err);\n \n #endif /* REFS_PACKED_BACKEND_H */\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 6f8f9f5619..4789106fc0 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -664,6 +664,7 @@ struct ref_storage_be {\n };\n \n extern struct ref_storage_be refs_be_files;\n+extern struct ref_storage_be refs_be_packed;\n \n /*\n * A representation of the reference store for the main repository or\n-- \n2.11.0\n\n"},{"id":"322342","messageId":"7766f7b4eeccf408dd6b03fd8656f59c84c1f2b1.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 20/28] commit_packed_refs(): report errors rather than dying","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:25Z","receivedAt":"2017-06-15T14:48:32Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"Report errors via a `struct strbuf *err` rather than by calling\n`die()`. To enable this goal, change `write_packed_entry()` to report\nerrors via a return value and `errno` rather than dying.\n\nSigned-off-by: Michael Haggerty \n---\n refs/files-backend.c | 10 +++---\n refs/packed-backend.c | 85 +++++++++++++++++++++++++++++++++------------------\n refs/packed-backend.h | 2 +-\n 3 files changed, 61 insertions(+), 36 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 039d0343cb..4323394a52 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1082,6 +1082,7 @@ static int files_pack_refs(struct ref_store *ref_store, unsigned int flags)\n \tstruct ref_iterator *iter;\n \tint ok;\n \tstruct ref_to_prune *refs_to_prune = NULL;\n+\tstruct strbuf err = STRBUF_INIT;\n \n \tlock_packed_refs(refs->packed_ref_store, LOCK_DIE_ON_ERROR);\n \n@@ -1116,10 +1117,11 @@ static int files_pack_refs(struct ref_store *ref_store, unsigned int flags)\n \tif (ok != ITER_DONE)\n \t\tdie(\"error while iterating over references\");\n \n-\tif (commit_packed_refs(refs->packed_ref_store))\n-\t\tdie_errno(\"unable to overwrite old ref-pack file\");\n+\tif (commit_packed_refs(refs->packed_ref_store, &err))\n+\t\tdie(\"unable to overwrite old ref-pack file: %s\", err.buf);\n \n \tprune_refs(refs, refs_to_prune);\n+\tstrbuf_release(&err);\n \treturn 0;\n }\n \n@@ -2681,9 +2683,7 @@ static int files_initial_transaction_commit(struct ref_store *ref_store,\n \t\t\t\t &update->new_oid);\n \t}\n \n-\tif (commit_packed_refs(refs->packed_ref_store)) {\n-\t\tstrbuf_addf(err, \"unable to commit packed-refs file: %s\",\n-\t\t\t strerror(errno));\n+\tif (commit_packed_refs(refs->packed_ref_store, err)) {\n \t\tret = TRANSACTION_GENERIC_ERROR;\n \t\tgoto cleanup;\n \t}\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 5dbe4e4e59..5bee49d497 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -493,15 +493,19 @@ static struct ref_iterator *packed_ref_iterator_begin(\n \n /*\n * Write an entry to the packed-refs file for the specified refname.\n- * If peeled is non-NULL, write it as the entry's peeled value.\n+ * If peeled is non-NULL, write it as the entry's peeled value. On\n+ * error, return a nonzero value and leave errno set at the value left\n+ * by the failing call to `fprintf()`.\n */\n-static void write_packed_entry(FILE *fh, const char *refname,\n-\t\t\t const unsigned char *sha1,\n-\t\t\t const unsigned char *peeled)\n+static int write_packed_entry(FILE *fh, const char *refname,\n+\t\t\t const unsigned char *sha1,\n+\t\t\t const unsigned char *peeled)\n {\n-\tfprintf_or_die(fh, \"%s %s\\n\", sha1_to_hex(sha1), refname);\n-\tif (peeled)\n-\t\tfprintf_or_die(fh, \"^%s\\n\", sha1_to_hex(peeled));\n+\tif (fprintf(fh, \"%s %s\\n\", sha1_to_hex(sha1), refname) < 0 ||\n+\t (peeled && fprintf(fh, \"^%s\\n\", sha1_to_hex(peeled)) < 0))\n+\t\treturn -1;\n+\n+\treturn 0;\n }\n \n int lock_packed_refs(struct ref_store *ref_store, int flags)\n@@ -550,49 +554,74 @@ static const char PACKED_REFS_HEADER[] =\n /*\n * Write the current version of the packed refs cache from memory to\n * disk. The packed-refs file must already be locked for writing (see\n- * lock_packed_refs()). Return zero on success. On errors, set errno\n- * and return a nonzero value.\n+ * lock_packed_refs()). Return zero on success. On errors, rollback\n+ * the lockfile, write an error message to `err`, and return a nonzero\n+ * value.\n */\n-int commit_packed_refs(struct ref_store *ref_store)\n+int commit_packed_refs(struct ref_store *ref_store, struct strbuf *err)\n {\n \tstruct packed_ref_store *refs =\n \t packed_downcast(ref_store, REF_STORE_WRITE | REF_STORE_MAIN,\n \t\t\t\t\"commit_packed_refs\");\n \tstruct packed_ref_cache *packed_ref_cache =\n \t\tget_packed_ref_cache(refs);\n-\tint ok, error = 0;\n-\tint save_errno = 0;\n+\tint ok;\n+\tint ret = -1;\n \tFILE *out;\n \tstruct ref_iterator *iter;\n \n \tif (!is_lock_file_locked(&refs->lock))\n-\t\tdie(\"BUG: packed-refs not locked\");\n+\t\tdie(\"BUG: commit_packed_refs() called when unlocked\");\n \n \tout = fdopen_lock_file(&refs->lock, \"w\");\n-\tif (!out)\n-\t\tdie_errno(\"unable to fdopen packed-refs descriptor\");\n+\tif (!out) {\n+\t\tstrbuf_addf(err, \"unable to fdopen packed-refs tempfile: %s\",\n+\t\t\t strerror(errno));\n+\t\tgoto error;\n+\t}\n \n-\tfprintf_or_die(out, \"%s\", PACKED_REFS_HEADER);\n+\tif (fprintf(out, \"%s\", PACKED_REFS_HEADER) < 0) {\n+\t\tstrbuf_addf(err, \"error writing to %s: %s\",\n+\t\t\t get_lock_file_path(&refs->lock), strerror(errno));\n+\t\tgoto error;\n+\t}\n \n \titer = cache_ref_iterator_begin(packed_ref_cache->cache, NULL, 0);\n \twhile ((ok = ref_iterator_advance(iter)) == ITER_OK) {\n \t\tstruct object_id peeled;\n \t\tint peel_error = ref_iterator_peel(iter, &peeled);\n \n-\t\twrite_packed_entry(out, iter->refname, iter->oid->hash,\n-\t\t\t\t peel_error ? NULL : peeled.hash);\n+\t\tif (write_packed_entry(out, iter->refname, iter->oid->hash,\n+\t\t\t\t peel_error ? NULL : peeled.hash)) {\n+\t\t\tstrbuf_addf(err, \"error writing to %s: %s\",\n+\t\t\t\t get_lock_file_path(&refs->lock),\n+\t\t\t\t strerror(errno));\n+\t\t\tref_iterator_abort(iter);\n+\t\t\tgoto error;\n+\t\t}\n \t}\n \n-\tif (ok != ITER_DONE)\n-\t\tdie(\"error while iterating over references\");\n+\tif (ok != ITER_DONE) {\n+\t\tstrbuf_addf(err, \"unable to write packed-refs file: \"\n+\t\t\t \"error iterating over old contents\");\n+\t\tgoto error;\n+\t}\n \n \tif (commit_lock_file(&refs->lock)) {\n-\t\tsave_errno = errno;\n-\t\terror = -1;\n+\t\tstrbuf_addf(err, \"error overwriting %s: %s\",\n+\t\t\t refs->path, strerror(errno));\n+\t\tgoto out;\n \t}\n+\n+\tret = 0;\n+\tgoto out;\n+\n+error:\n+\trollback_lock_file(&refs->lock);\n+\n+out:\n \trelease_packed_ref_cache(packed_ref_cache);\n-\terrno = save_errno;\n-\treturn error;\n+\treturn ret;\n }\n \n /*\n@@ -628,7 +657,7 @@ int repack_without_refs(struct ref_store *ref_store,\n \t\t\t\t\"repack_without_refs\");\n \tstruct ref_dir *packed;\n \tstruct string_list_item *refname;\n-\tint ret, needs_repacking = 0, removed = 0;\n+\tint needs_repacking = 0, removed = 0;\n \n \tpacked_assert_main_repository(refs, \"repack_without_refs\");\n \tassert(err);\n@@ -665,11 +694,7 @@ int repack_without_refs(struct ref_store *ref_store,\n \t}\n \n \t/* Write what remains */\n-\tret = commit_packed_refs(&refs->base);\n-\tif (ret)\n-\t\tstrbuf_addf(err, \"unable to overwrite old ref-pack file: %s\",\n-\t\t\t strerror(errno));\n-\treturn ret;\n+\treturn commit_packed_refs(&refs->base, err);\n }\n \n static int packed_init_db(struct ref_store *ref_store, struct strbuf *err)\ndiff --git a/refs/packed-backend.h b/refs/packed-backend.h\nindex beea9c14b5..3d4057b65b 100644\n--- a/refs/packed-backend.h\n+++ b/refs/packed-backend.h\n@@ -14,7 +14,7 @@ int lock_packed_refs(struct ref_store *ref_store, int flags);\n void add_packed_ref(struct ref_store *ref_store,\n \t\t const char *refname, const struct object_id *oid);\n \n-int commit_packed_refs(struct ref_store *ref_store);\n+int commit_packed_refs(struct ref_store *ref_store, struct strbuf *err);\n \n int repack_without_refs(struct ref_store *ref_store,\n \t\t\tstruct string_list *refnames, struct strbuf *err);\n-- \n2.11.0\n\n"},{"id":"322343","messageId":"2779ee9a75e75ff80898e51e46d561b8074e0dc2.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 21/28] commit_packed_refs(): use a staging file separate from the lockfile","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:26Z","receivedAt":"2017-06-15T14:48:34Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"We will want to be able to hold the lockfile for `packed-refs` even\nafter we have activated the new values. So use a separate tempfile,\n`packed-refs.new`, as a place to stage the new contents of the\n`packed-refs` file. For now this is all done within\n`commit_packed_refs()`, but that will change shortly.\n\nSigned-off-by: Michael Haggerty \n---\n refs/packed-backend.c | 40 ++++++++++++++++++++++++++++++++--------\n 1 file changed, 32 insertions(+), 8 deletions(-)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 5bee49d497..6619052e96 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -68,6 +68,13 @@ struct packed_ref_store {\n \t * thus the enclosing `packed_ref_store`) must not be freed.\n \t */\n \tstruct lock_file lock;\n+\n+\t/*\n+\t * Temporary file used when rewriting new contents to the\n+\t * \"packed-refs\" file. Note that this (and thus the enclosing\n+\t * `packed_ref_store`) must not be freed.\n+\t */\n+\tstruct tempfile tempfile;\n };\n \n struct ref_store *packed_ref_store_create(const char *path,\n@@ -522,10 +529,16 @@ int lock_packed_refs(struct ref_store *ref_store, int flags)\n \t\ttimeout_configured = 1;\n \t}\n \n+\t/*\n+\t * Note that we close the lockfile immediately because we\n+\t * don't write new content to it, but rather to a separate\n+\t * tempfile.\n+\t */\n \tif (hold_lock_file_for_update_timeout(\n \t\t\t &refs->lock,\n \t\t\t refs->path,\n-\t\t\t flags, timeout_value) < 0)\n+\t\t\t flags, timeout_value) < 0 ||\n+\t close_lock_file(&refs->lock))\n \t\treturn -1;\n \n \t/*\n@@ -567,13 +580,23 @@ int commit_packed_refs(struct ref_store *ref_store, struct strbuf *err)\n \t\tget_packed_ref_cache(refs);\n \tint ok;\n \tint ret = -1;\n+\tstruct strbuf sb = STRBUF_INIT;\n \tFILE *out;\n \tstruct ref_iterator *iter;\n \n \tif (!is_lock_file_locked(&refs->lock))\n \t\tdie(\"BUG: commit_packed_refs() called when unlocked\");\n \n-\tout = fdopen_lock_file(&refs->lock, \"w\");\n+\tstrbuf_addf(&sb, \"%s.new\", refs->path);\n+\tif (create_tempfile(&refs->tempfile, sb.buf) < 0) {\n+\t\tstrbuf_addf(err, \"unable to create file %s: %s\",\n+\t\t\t sb.buf, strerror(errno));\n+\t\tstrbuf_release(&sb);\n+\t\tgoto out;\n+\t}\n+\tstrbuf_release(&sb);\n+\n+\tout = fdopen_tempfile(&refs->tempfile, \"w\");\n \tif (!out) {\n \t\tstrbuf_addf(err, \"unable to fdopen packed-refs tempfile: %s\",\n \t\t\t strerror(errno));\n@@ -582,7 +605,7 @@ int commit_packed_refs(struct ref_store *ref_store, struct strbuf *err)\n \n \tif (fprintf(out, \"%s\", PACKED_REFS_HEADER) < 0) {\n \t\tstrbuf_addf(err, \"error writing to %s: %s\",\n-\t\t\t get_lock_file_path(&refs->lock), strerror(errno));\n+\t\t\t get_tempfile_path(&refs->tempfile), strerror(errno));\n \t\tgoto error;\n \t}\n \n@@ -594,7 +617,7 @@ int commit_packed_refs(struct ref_store *ref_store, struct strbuf *err)\n \t\tif (write_packed_entry(out, iter->refname, iter->oid->hash,\n \t\t\t\t peel_error ? NULL : peeled.hash)) {\n \t\t\tstrbuf_addf(err, \"error writing to %s: %s\",\n-\t\t\t\t get_lock_file_path(&refs->lock),\n+\t\t\t\t get_tempfile_path(&refs->tempfile),\n \t\t\t\t strerror(errno));\n \t\t\tref_iterator_abort(iter);\n \t\t\tgoto error;\n@@ -602,13 +625,13 @@ int commit_packed_refs(struct ref_store *ref_store, struct strbuf *err)\n \t}\n \n \tif (ok != ITER_DONE) {\n-\t\tstrbuf_addf(err, \"unable to write packed-refs file: \"\n+\t\tstrbuf_addf(err, \"unable to rewrite packed-refs file: \"\n \t\t\t \"error iterating over old contents\");\n \t\tgoto error;\n \t}\n \n-\tif (commit_lock_file(&refs->lock)) {\n-\t\tstrbuf_addf(err, \"error overwriting %s: %s\",\n+\tif (rename_tempfile(&refs->tempfile, refs->path)) {\n+\t\tstrbuf_addf(err, \"error replacing %s: %s\",\n \t\t\t refs->path, strerror(errno));\n \t\tgoto out;\n \t}\n@@ -617,9 +640,10 @@ int commit_packed_refs(struct ref_store *ref_store, struct strbuf *err)\n \tgoto out;\n \n error:\n-\trollback_lock_file(&refs->lock);\n+\tdelete_tempfile(&refs->tempfile);\n \n out:\n+\trollback_lock_file(&refs->lock);\n \trelease_packed_ref_cache(packed_ref_cache);\n \treturn ret;\n }\n-- \n2.11.0\n\n"},{"id":"322344","messageId":"fdc991cfb05588fc93778a6ce3df197ac6c95146.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 22/28] packed_refs_lock(): function renamed from lock_packed_refs()","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:27Z","receivedAt":"2017-06-15T14:48:37Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"Rename `lock_packed_refs()` to `packed_refs_lock()` for consistency\nwith how other methods are named. Also, it's about to get some\ncompanions.\n\nSigned-off-by: Michael Haggerty \n---\n refs/files-backend.c | 4 ++--\n refs/packed-backend.c | 10 +++++-----\n refs/packed-backend.h | 2 +-\n 3 files changed, 8 insertions(+), 8 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 4323394a52..3fc2254e33 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1084,7 +1084,7 @@ static int files_pack_refs(struct ref_store *ref_store, unsigned int flags)\n \tstruct ref_to_prune *refs_to_prune = NULL;\n \tstruct strbuf err = STRBUF_INIT;\n \n-\tlock_packed_refs(refs->packed_ref_store, LOCK_DIE_ON_ERROR);\n+\tpacked_refs_lock(refs->packed_ref_store, LOCK_DIE_ON_ERROR);\n \n \titer = cache_ref_iterator_begin(get_loose_ref_cache(refs), NULL, 0);\n \twhile ((ok = ref_iterator_advance(iter)) == ITER_OK) {\n@@ -2667,7 +2667,7 @@ static int files_initial_transaction_commit(struct ref_store *ref_store,\n \t\t}\n \t}\n \n-\tif (lock_packed_refs(refs->packed_ref_store, 0)) {\n+\tif (packed_refs_lock(refs->packed_ref_store, 0)) {\n \t\tstrbuf_addf(err, \"unable to lock packed-refs file: %s\",\n \t\t\t strerror(errno));\n \t\tret = TRANSACTION_GENERIC_ERROR;\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 6619052e96..eb9da04576 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -321,7 +321,7 @@ static struct ref_dir *get_packed_refs(struct packed_ref_store *refs)\n /*\n * Add or overwrite a reference in the in-memory packed reference\n * cache. This may only be called while the packed-refs file is locked\n- * (see lock_packed_refs()). To actually write the packed-refs file,\n+ * (see packed_refs_lock()). To actually write the packed-refs file,\n * call commit_packed_refs().\n */\n void add_packed_ref(struct ref_store *ref_store,\n@@ -515,11 +515,11 @@ static int write_packed_entry(FILE *fh, const char *refname,\n \treturn 0;\n }\n \n-int lock_packed_refs(struct ref_store *ref_store, int flags)\n+int packed_refs_lock(struct ref_store *ref_store, int flags)\n {\n \tstruct packed_ref_store *refs =\n \t packed_downcast(ref_store, REF_STORE_WRITE | REF_STORE_MAIN,\n-\t\t\t\t\"lock_packed_refs\");\n+\t\t\t\t\"packed_refs_lock\");\n \tstatic int timeout_configured = 0;\n \tstatic int timeout_value = 1000;\n \tstruct packed_ref_cache *packed_ref_cache;\n@@ -567,7 +567,7 @@ static const char PACKED_REFS_HEADER[] =\n /*\n * Write the current version of the packed refs cache from memory to\n * disk. The packed-refs file must already be locked for writing (see\n- * lock_packed_refs()). Return zero on success. On errors, rollback\n+ * packed_refs_lock()). Return zero on success. On errors, rollback\n * the lockfile, write an error message to `err`, and return a nonzero\n * value.\n */\n@@ -698,7 +698,7 @@ int repack_without_refs(struct ref_store *ref_store,\n \tif (!needs_repacking)\n \t\treturn 0; /* no refname exists in packed refs */\n \n-\tif (lock_packed_refs(&refs->base, 0)) {\n+\tif (packed_refs_lock(&refs->base, 0)) {\n \t\tunable_to_lock_message(refs->path, errno, err);\n \t\treturn -1;\n \t}\ndiff --git a/refs/packed-backend.h b/refs/packed-backend.h\nindex 3d4057b65b..dbc00d3396 100644\n--- a/refs/packed-backend.h\n+++ b/refs/packed-backend.h\n@@ -9,7 +9,7 @@ struct ref_store *packed_ref_store_create(const char *path,\n * hold_lock_file_for_update(). Return 0 on success. On errors, set\n * errno appropriately and return a nonzero value.\n */\n-int lock_packed_refs(struct ref_store *ref_store, int flags);\n+int packed_refs_lock(struct ref_store *ref_store, int flags);\n \n void add_packed_ref(struct ref_store *ref_store,\n \t\t const char *refname, const struct object_id *oid);\n-- \n2.11.0\n\n"},{"id":"322345","messageId":"831f8a3f633374f173f630d9c65d870fffd0896e.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 18/28] packed-backend: new module for handling packed references","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:23Z","receivedAt":"2017-06-15T14:48:41Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"Now that the interface between `files_ref_store` and\n`packed_ref_store` is relatively narrow, move the latter into a new\nmodule, \"refs/packed-backend.h\" and \"refs/packed-backend.c\". It still\ndoesn't quite implement the `ref_store` interface, but it will soon.\n\nThis commit moves code around and adjusts its visibility, but doesn't\nchange anything.\n\nSigned-off-by: Michael Haggerty \n---\n Makefile | 1 +\n refs.c | 18 ++\n refs/files-backend.c | 640 +-------------------------------------------------\n refs/packed-backend.c | 624 ++++++++++++++++++++++++++++++++++++++++++++++++\n refs/packed-backend.h | 33 +++\n refs/refs-internal.h | 9 +\n 6 files changed, 686 insertions(+), 639 deletions(-)\n create mode 100644 refs/packed-backend.c\n create mode 100644 refs/packed-backend.h\n\ndiff --git a/Makefile b/Makefile\nindex 33b888730f..478a8eac5d 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -821,6 +821,7 @@ LIB_OBJS += reflog-walk.o\n LIB_OBJS += refs.o\n LIB_OBJS += refs/files-backend.o\n LIB_OBJS += refs/iterator.o\n+LIB_OBJS += refs/packed-backend.o\n LIB_OBJS += refs/ref-cache.o\n LIB_OBJS += ref-filter.o\n LIB_OBJS += remote.o\ndiff --git a/refs.c b/refs.c\nindex 32177969f0..5880c12372 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -173,6 +173,24 @@ int refname_is_safe(const char *refname)\n \treturn 1;\n }\n \n+/*\n+ * Return true if refname, which has the specified oid and flags, can\n+ * be resolved to an object in the database. If the referred-to object\n+ * does not exist, emit a warning and return false.\n+ */\n+int ref_resolves_to_object(const char *refname,\n+\t\t\t const struct object_id *oid,\n+\t\t\t unsigned int flags)\n+{\n+\tif (flags & REF_ISBROKEN)\n+\t\treturn 0;\n+\tif (!has_sha1_file(oid->hash)) {\n+\t\terror(\"%s does not point to a valid object!\", refname);\n+\t\treturn 0;\n+\t}\n+\treturn 1;\n+}\n+\n char *refs_resolve_refdup(struct ref_store *refs,\n \t\t\t const char *refname, int resolve_flags,\n \t\t\t unsigned char *sha1, int *flags)\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex ac4764f6f7..83ea773728 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -2,6 +2,7 @@\n #include \"../refs.h\"\n #include \"refs-internal.h\"\n #include \"ref-cache.h\"\n+#include \"packed-backend.h\"\n #include \"../iterator.h\"\n #include \"../dir-iterator.h\"\n #include \"../lockfile.h\"\n@@ -14,85 +15,6 @@ struct ref_lock {\n \tstruct object_id old_oid;\n };\n \n-/*\n- * Return true if refname, which has the specified oid and flags, can\n- * be resolved to an object in the database. If the referred-to object\n- * does not exist, emit a warning and return false.\n- */\n-static int ref_resolves_to_object(const char *refname,\n-\t\t\t\t const struct object_id *oid,\n-\t\t\t\t unsigned int flags)\n-{\n-\tif (flags & REF_ISBROKEN)\n-\t\treturn 0;\n-\tif (!has_sha1_file(oid->hash)) {\n-\t\terror(\"%s does not point to a valid object!\", refname);\n-\t\treturn 0;\n-\t}\n-\treturn 1;\n-}\n-\n-struct packed_ref_cache {\n-\tstruct ref_cache *cache;\n-\n-\t/*\n-\t * Count of references to the data structure in this instance,\n-\t * including the pointer from files_ref_store::packed if any.\n-\t * The data will not be freed as long as the reference count\n-\t * is nonzero.\n-\t */\n-\tunsigned int referrers;\n-\n-\t/* The metadata from when this packed-refs cache was read */\n-\tstruct stat_validity validity;\n-};\n-\n-/*\n- * A container for `packed-refs`-related data. It is not (yet) a\n- * `ref_store`.\n- */\n-struct packed_ref_store {\n-\tunsigned int store_flags;\n-\n-\t/* The path of the \"packed-refs\" file: */\n-\tchar *path;\n-\n-\t/*\n-\t * A cache of the values read from the `packed-refs` file, if\n-\t * it might still be current; otherwise, NULL.\n-\t */\n-\tstruct packed_ref_cache *cache;\n-\n-\t/*\n-\t * Lock used for the \"packed-refs\" file. Note that this (and\n-\t * thus the enclosing `packed_ref_store`) must not be freed.\n-\t */\n-\tstruct lock_file lock;\n-};\n-\n-static struct packed_ref_store *packed_ref_store_create(\n-\t\tconst char *path, unsigned int store_flags)\n-{\n-\tstruct packed_ref_store *refs = xcalloc(1, sizeof(*refs));\n-\n-\trefs->store_flags = store_flags;\n-\trefs->path = xstrdup(path);\n-\treturn refs;\n-}\n-\n-/*\n- * Die if refs is not the main ref store. caller is used in any\n- * necessary error messages.\n- */\n-static void packed_assert_main_repository(struct packed_ref_store *refs,\n-\t\t\t\t\t const char *caller)\n-{\n-\tif (refs->store_flags & REF_STORE_MAIN)\n-\t\treturn;\n-\n-\tdie(\"BUG: operation %s only allowed for main ref store\", caller);\n-}\n-\n /*\n * Future: need to be in \"struct repository\"\n * when doing a full libification.\n@@ -109,42 +31,6 @@ struct files_ref_store {\n \tstruct packed_ref_store *packed_ref_store;\n };\n \n-/*\n- * Increment the reference count of *packed_refs.\n- */\n-static void acquire_packed_ref_cache(struct packed_ref_cache *packed_refs)\n-{\n-\tpacked_refs->referrers++;\n-}\n-\n-/*\n- * Decrease the reference count of *packed_refs. If it goes to zero,\n- * free *packed_refs and return true; otherwise return false.\n- */\n-static int release_packed_ref_cache(struct packed_ref_cache *packed_refs)\n-{\n-\tif (!--packed_refs->referrers) {\n-\t\tfree_ref_cache(packed_refs->cache);\n-\t\tstat_validity_clear(&packed_refs->validity);\n-\t\tfree(packed_refs);\n-\t\treturn 1;\n-\t} else {\n-\t\treturn 0;\n-\t}\n-}\n-\n-static void clear_packed_ref_cache(struct packed_ref_store *refs)\n-{\n-\tif (refs->cache) {\n-\t\tstruct packed_ref_cache *cache = refs->cache;\n-\n-\t\tif (is_lock_file_locked(&refs->lock))\n-\t\t\tdie(\"BUG: packed-ref cache cleared while locked\");\n-\t\trefs->cache = NULL;\n-\t\trelease_packed_ref_cache(cache);\n-\t}\n-}\n-\n static void clear_loose_ref_cache(struct files_ref_store *refs)\n {\n \tif (refs->loose) {\n@@ -215,151 +101,6 @@ static struct files_ref_store *files_downcast(struct ref_store *ref_store,\n \treturn refs;\n }\n \n-/* The length of a peeled reference line in packed-refs, including EOL: */\n-#define PEELED_LINE_LENGTH 42\n-\n-/*\n- * The packed-refs header line that we write out. Perhaps other\n- * traits will be added later. The trailing space is required.\n- */\n-static const char PACKED_REFS_HEADER[] =\n-\t\"# pack-refs with: peeled fully-peeled \\n\";\n-\n-/*\n- * Parse one line from a packed-refs file. Write the SHA1 to sha1.\n- * Return a pointer to the refname within the line (null-terminated),\n- * or NULL if there was a problem.\n- */\n-static const char *parse_ref_line(struct strbuf *line, struct object_id *oid)\n-{\n-\tconst char *ref;\n-\n-\tif (parse_oid_hex(line->buf, oid, &ref) < 0)\n-\t\treturn NULL;\n-\tif (!isspace(*ref++))\n-\t\treturn NULL;\n-\n-\tif (isspace(*ref))\n-\t\treturn NULL;\n-\n-\tif (line->buf[line->len - 1] != '\\n')\n-\t\treturn NULL;\n-\tline->buf[--line->len] = 0;\n-\n-\treturn ref;\n-}\n-\n-/*\n- * Read from `packed_refs_file` into a newly-allocated\n- * `packed_ref_cache` and return it. The return value will already\n- * have its reference count incremented.\n- *\n- * A comment line of the form \"# pack-refs with: \" may contain zero or\n- * more traits. We interpret the traits as follows:\n- *\n- * No traits:\n- *\n- * Probably no references are peeled. But if the file contains a\n- * peeled value for a reference, we will use it.\n- *\n- * peeled:\n- *\n- * References under \"refs/tags/\", if they *can* be peeled, *are*\n- * peeled in this file. References outside of \"refs/tags/\" are\n- * probably not peeled even if they could have been, but if we find\n- * a peeled value for such a reference we will use it.\n- *\n- * fully-peeled:\n- *\n- * All references in the file that can be peeled are peeled.\n- * Inversely (and this is more important), any references in the\n- * file for which no peeled value is recorded is not peelable. This\n- * trait should typically be written alongside \"peeled\" for\n- * compatibility with older clients, but we do not require it\n- * (i.e., \"peeled\" is a no-op if \"fully-peeled\" is set).\n- */\n-static struct packed_ref_cache *read_packed_refs(const char *packed_refs_file)\n-{\n-\tFILE *f;\n-\tstruct packed_ref_cache *packed_refs = xcalloc(1, sizeof(*packed_refs));\n-\tstruct ref_entry *last = NULL;\n-\tstruct strbuf line = STRBUF_INIT;\n-\tenum { PEELED_NONE, PEELED_TAGS, PEELED_FULLY } peeled = PEELED_NONE;\n-\tstruct ref_dir *dir;\n-\n-\tacquire_packed_ref_cache(packed_refs);\n-\tpacked_refs->cache = create_ref_cache(NULL, NULL);\n-\tpacked_refs->cache->root->flag &= ~REF_INCOMPLETE;\n-\n-\tf = fopen(packed_refs_file, \"r\");\n-\tif (!f) {\n-\t\tif (errno == ENOENT) {\n-\t\t\t/*\n-\t\t\t * This is OK; it just means that no\n-\t\t\t * \"packed-refs\" file has been written yet,\n-\t\t\t * which is equivalent to it being empty.\n-\t\t\t */\n-\t\t\treturn packed_refs;\n-\t\t} else {\n-\t\t\tdie_errno(\"couldn't read %s\", packed_refs_file);\n-\t\t}\n-\t}\n-\n-\tstat_validity_update(&packed_refs->validity, fileno(f));\n-\n-\tdir = get_ref_dir(packed_refs->cache->root);\n-\twhile (strbuf_getwholeline(&line, f, '\\n') != EOF) {\n-\t\tstruct object_id oid;\n-\t\tconst char *refname;\n-\t\tconst char *traits;\n-\n-\t\tif (skip_prefix(line.buf, \"# pack-refs with:\", &traits)) {\n-\t\t\tif (strstr(traits, \" fully-peeled \"))\n-\t\t\t\tpeeled = PEELED_FULLY;\n-\t\t\telse if (strstr(traits, \" peeled \"))\n-\t\t\t\tpeeled = PEELED_TAGS;\n-\t\t\t/* perhaps other traits later as well */\n-\t\t\tcontinue;\n-\t\t}\n-\n-\t\trefname = parse_ref_line(&line, &oid);\n-\t\tif (refname) {\n-\t\t\tint flag = REF_ISPACKED;\n-\n-\t\t\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n-\t\t\t\tif (!refname_is_safe(refname))\n-\t\t\t\t\tdie(\"packed refname is dangerous: %s\", refname);\n-\t\t\t\toidclr(&oid);\n-\t\t\t\tflag |= REF_BAD_NAME | REF_ISBROKEN;\n-\t\t\t}\n-\t\t\tlast = create_ref_entry(refname, &oid, flag);\n-\t\t\tif (peeled == PEELED_FULLY ||\n-\t\t\t (peeled == PEELED_TAGS && starts_with(refname, \"refs/tags/\")))\n-\t\t\t\tlast->flag |= REF_KNOWS_PEELED;\n-\t\t\tadd_ref_entry(dir, last);\n-\t\t\tcontinue;\n-\t\t}\n-\t\tif (last &&\n-\t\t line.buf[0] == '^' &&\n-\t\t line.len == PEELED_LINE_LENGTH &&\n-\t\t line.buf[PEELED_LINE_LENGTH - 1] == '\\n' &&\n-\t\t !get_oid_hex(line.buf + 1, &oid)) {\n-\t\t\toidcpy(&last->u.value.peeled, &oid);\n-\t\t\t/*\n-\t\t\t * Regardless of what the file header said,\n-\t\t\t * we definitely know the value of *this*\n-\t\t\t * reference:\n-\t\t\t */\n-\t\t\tlast->flag |= REF_KNOWS_PEELED;\n-\t\t}\n-\t}\n-\n-\tfclose(f);\n-\tstrbuf_release(&line);\n-\n-\treturn packed_refs;\n-}\n-\n static void files_reflog_path(struct files_ref_store *refs,\n \t\t\t struct strbuf *sb,\n \t\t\t const char *refname)\n@@ -405,77 +146,6 @@ static void files_ref_path(struct files_ref_store *refs,\n \t}\n }\n \n-/*\n- * Check that the packed refs cache (if any) still reflects the\n- * contents of the file. If not, clear the cache.\n- */\n-static void validate_packed_ref_cache(struct packed_ref_store *refs)\n-{\n-\tif (refs->cache &&\n-\t !stat_validity_check(&refs->cache->validity, refs->path))\n-\t\tclear_packed_ref_cache(refs);\n-}\n-\n-/*\n- * Get the packed_ref_cache for the specified packed_ref_store,\n- * creating and populating it if it hasn't been read before or if the\n- * file has been changed (according to its `validity` field) since it\n- * was last read. On the other hand, if we hold the lock, then assume\n- * that the file hasn't been changed out from under us, so skip the\n- * extra `stat()` call in `stat_validity_check()`.\n- */\n-static struct packed_ref_cache *get_packed_ref_cache(struct packed_ref_store *refs)\n-{\n-\tif (!is_lock_file_locked(&refs->lock))\n-\t\tvalidate_packed_ref_cache(refs);\n-\n-\tif (!refs->cache)\n-\t\trefs->cache = read_packed_refs(refs->path);\n-\n-\treturn refs->cache;\n-}\n-\n-static struct ref_dir *get_packed_ref_dir(struct packed_ref_cache *packed_ref_cache)\n-{\n-\treturn get_ref_dir(packed_ref_cache->cache->root);\n-}\n-\n-static struct ref_dir *get_packed_refs(struct packed_ref_store *refs)\n-{\n-\treturn get_packed_ref_dir(get_packed_ref_cache(refs));\n-}\n-\n-/*\n- * Add or overwrite a reference in the in-memory packed reference\n- * cache. This may only be called while the packed-refs file is locked\n- * (see lock_packed_refs()). To actually write the packed-refs file,\n- * call commit_packed_refs().\n- */\n-static void add_packed_ref(struct packed_ref_store *refs,\n-\t\t\t const char *refname, const struct object_id *oid)\n-{\n-\tstruct ref_dir *packed_refs;\n-\tstruct ref_entry *packed_entry;\n-\n-\tif (!is_lock_file_locked(&refs->lock))\n-\t\tdie(\"BUG: packed refs not locked\");\n-\n-\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL))\n-\t\tdie(\"Reference has invalid format: '%s'\", refname);\n-\n-\tpacked_refs = get_packed_refs(refs);\n-\tpacked_entry = find_ref_entry(packed_refs, refname);\n-\tif (packed_entry) {\n-\t\t/* Overwrite the existing entry: */\n-\t\toidcpy(&packed_entry->u.value.oid, oid);\n-\t\tpacked_entry->flag = REF_ISPACKED;\n-\t\toidclr(&packed_entry->u.value.peeled);\n-\t} else {\n-\t\tpacked_entry = create_ref_entry(refname, oid, REF_ISPACKED);\n-\t\tadd_ref_entry(packed_refs, packed_entry);\n-\t}\n-}\n-\n /*\n * Read the loose references from the namespace dirname into dir\n * (without recursing). dirname must end with '/'. dir must be the\n@@ -598,35 +268,6 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)\n \treturn refs->loose;\n }\n \n-/*\n- * Return the ref_entry for the given refname from the packed\n- * references. If it does not exist, return NULL.\n- */\n-static struct ref_entry *get_packed_ref(struct packed_ref_store *refs,\n-\t\t\t\t\tconst char *refname)\n-{\n-\treturn find_ref_entry(get_packed_refs(refs), refname);\n-}\n-\n-static int packed_read_raw_ref(struct packed_ref_store *refs,\n-\t\t\t const char *refname, unsigned char *sha1,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n-{\n-\tstruct ref_entry *entry;\n-\n-\t*type = 0;\n-\n-\tentry = get_packed_ref(refs, refname);\n-\tif (!entry) {\n-\t\terrno = ENOENT;\n-\t\treturn -1;\n-\t}\n-\n-\thashcpy(sha1, entry->u.value.oid.hash);\n-\t*type = REF_ISPACKED;\n-\treturn 0;\n-}\n-\n static int files_read_raw_ref(struct ref_store *ref_store,\n \t\t\t const char *refname, unsigned char *sha1,\n \t\t\t struct strbuf *referent, unsigned int *type)\n@@ -1011,18 +652,6 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \treturn ret;\n }\n \n-static int packed_peel_ref(struct packed_ref_store *refs,\n-\t\t\t const char *refname, unsigned char *sha1)\n-{\n-\tstruct ref_entry *r = get_packed_ref(refs, refname);\n-\n-\tif (!r || peel_entry(r, 0))\n-\t\treturn -1;\n-\n-\thashcpy(sha1, r->u.value.peeled.hash);\n-\treturn 0;\n-}\n-\n static int files_peel_ref(struct ref_store *ref_store,\n \t\t\t const char *refname, unsigned char *sha1)\n {\n@@ -1060,99 +689,6 @@ static int files_peel_ref(struct ref_store *ref_store,\n \treturn peel_object(base, sha1);\n }\n \n-struct packed_ref_iterator {\n-\tstruct ref_iterator base;\n-\n-\tstruct packed_ref_cache *cache;\n-\tstruct ref_iterator *iter0;\n-\tunsigned int flags;\n-};\n-\n-static int packed_ref_iterator_advance(struct ref_iterator *ref_iterator)\n-{\n-\tstruct packed_ref_iterator *iter =\n-\t\t(struct packed_ref_iterator *)ref_iterator;\n-\tint ok;\n-\n-\twhile ((ok = ref_iterator_advance(iter->iter0)) == ITER_OK) {\n-\t\tif (iter->flags & DO_FOR_EACH_PER_WORKTREE_ONLY &&\n-\t\t ref_type(iter->iter0->refname) != REF_TYPE_PER_WORKTREE)\n-\t\t\tcontinue;\n-\n-\t\tif (!(iter->flags & DO_FOR_EACH_INCLUDE_BROKEN) &&\n-\t\t !ref_resolves_to_object(iter->iter0->refname,\n-\t\t\t\t\t iter->iter0->oid,\n-\t\t\t\t\t iter->iter0->flags))\n-\t\t\tcontinue;\n-\n-\t\titer->base.refname = iter->iter0->refname;\n-\t\titer->base.oid = iter->iter0->oid;\n-\t\titer->base.flags = iter->iter0->flags;\n-\t\treturn ITER_OK;\n-\t}\n-\n-\titer->iter0 = NULL;\n-\tif (ref_iterator_abort(ref_iterator) != ITER_DONE)\n-\t\tok = ITER_ERROR;\n-\n-\treturn ok;\n-}\n-\n-static int packed_ref_iterator_peel(struct ref_iterator *ref_iterator,\n-\t\t\t\t struct object_id *peeled)\n-{\n-\tstruct packed_ref_iterator *iter =\n-\t\t(struct packed_ref_iterator *)ref_iterator;\n-\n-\treturn ref_iterator_peel(iter->iter0, peeled);\n-}\n-\n-static int packed_ref_iterator_abort(struct ref_iterator *ref_iterator)\n-{\n-\tstruct packed_ref_iterator *iter =\n-\t\t(struct packed_ref_iterator *)ref_iterator;\n-\tint ok = ITER_DONE;\n-\n-\tif (iter->iter0)\n-\t\tok = ref_iterator_abort(iter->iter0);\n-\n-\trelease_packed_ref_cache(iter->cache);\n-\tbase_ref_iterator_free(ref_iterator);\n-\treturn ok;\n-}\n-\n-static struct ref_iterator_vtable packed_ref_iterator_vtable = {\n-\tpacked_ref_iterator_advance,\n-\tpacked_ref_iterator_peel,\n-\tpacked_ref_iterator_abort\n-};\n-\n-static struct ref_iterator *packed_ref_iterator_begin(\n-\t\tstruct packed_ref_store *refs,\n-\t\tconst char *prefix, unsigned int flags)\n-{\n-\tstruct packed_ref_iterator *iter;\n-\tstruct ref_iterator *ref_iterator;\n-\n-\titer = xcalloc(1, sizeof(*iter));\n-\tref_iterator = &iter->base;\n-\tbase_ref_iterator_init(ref_iterator, &packed_ref_iterator_vtable);\n-\n-\t/*\n-\t * Note that get_packed_ref_cache() internally checks whether\n-\t * the packed-ref cache is up to date with what is on disk,\n-\t * and re-reads it if not.\n-\t */\n-\n-\titer->cache = get_packed_ref_cache(refs);\n-\tacquire_packed_ref_cache(iter->cache);\n-\titer->iter0 = cache_ref_iterator_begin(iter->cache->cache, prefix, 0);\n-\n-\titer->flags = flags;\n-\n-\treturn ref_iterator;\n-}\n-\n struct files_ref_iterator {\n \tstruct ref_iterator base;\n \n@@ -1422,124 +958,6 @@ static struct ref_lock *lock_ref_sha1_basic(struct files_ref_store *refs,\n \treturn lock;\n }\n \n-/*\n- * Write an entry to the packed-refs file for the specified refname.\n- * If peeled is non-NULL, write it as the entry's peeled value.\n- */\n-static void write_packed_entry(FILE *fh, const char *refname,\n-\t\t\t const unsigned char *sha1,\n-\t\t\t const unsigned char *peeled)\n-{\n-\tfprintf_or_die(fh, \"%s %s\\n\", sha1_to_hex(sha1), refname);\n-\tif (peeled)\n-\t\tfprintf_or_die(fh, \"^%s\\n\", sha1_to_hex(peeled));\n-}\n-\n-/*\n- * Lock the packed-refs file for writing. Flags is passed to\n- * hold_lock_file_for_update(). Return 0 on success. On errors, set\n- * errno appropriately and return a nonzero value.\n- */\n-static int lock_packed_refs(struct packed_ref_store *refs, int flags)\n-{\n-\tstatic int timeout_configured = 0;\n-\tstatic int timeout_value = 1000;\n-\tstruct packed_ref_cache *packed_ref_cache;\n-\n-\tpacked_assert_main_repository(refs, \"lock_packed_refs\");\n-\n-\tif (!timeout_configured) {\n-\t\tgit_config_get_int(\"core.packedrefstimeout\", &timeout_value);\n-\t\ttimeout_configured = 1;\n-\t}\n-\n-\tif (hold_lock_file_for_update_timeout(\n-\t\t\t &refs->lock,\n-\t\t\t refs->path,\n-\t\t\t flags, timeout_value) < 0)\n-\t\treturn -1;\n-\n-\t/*\n-\t * Now that we hold the `packed-refs` lock, make sure that our\n-\t * cache matches the current version of the file. Normally\n-\t * `get_packed_ref_cache()` does that for us, but that\n-\t * function assumes that when the file is locked, any existing\n-\t * cache is still valid. We've just locked the file, but it\n-\t * might have changed the moment *before* we locked it.\n-\t */\n-\tvalidate_packed_ref_cache(refs);\n-\n-\tpacked_ref_cache = get_packed_ref_cache(refs);\n-\t/* Increment the reference count to prevent it from being freed: */\n-\tacquire_packed_ref_cache(packed_ref_cache);\n-\treturn 0;\n-}\n-\n-/*\n- * Write the current version of the packed refs cache from memory to\n- * disk. The packed-refs file must already be locked for writing (see\n- * lock_packed_refs()). Return zero on success. On errors, set errno\n- * and return a nonzero value\n- */\n-static int commit_packed_refs(struct packed_ref_store *refs)\n-{\n-\tstruct packed_ref_cache *packed_ref_cache =\n-\t\tget_packed_ref_cache(refs);\n-\tint ok, error = 0;\n-\tint save_errno = 0;\n-\tFILE *out;\n-\tstruct ref_iterator *iter;\n-\n-\tpacked_assert_main_repository(refs, \"commit_packed_refs\");\n-\n-\tif (!is_lock_file_locked(&refs->lock))\n-\t\tdie(\"BUG: packed-refs not locked\");\n-\n-\tout = fdopen_lock_file(&refs->lock, \"w\");\n-\tif (!out)\n-\t\tdie_errno(\"unable to fdopen packed-refs descriptor\");\n-\n-\tfprintf_or_die(out, \"%s\", PACKED_REFS_HEADER);\n-\n-\titer = cache_ref_iterator_begin(packed_ref_cache->cache, NULL, 0);\n-\twhile ((ok = ref_iterator_advance(iter)) == ITER_OK) {\n-\t\tstruct object_id peeled;\n-\t\tint peel_error = ref_iterator_peel(iter, &peeled);\n-\n-\t\twrite_packed_entry(out, iter->refname, iter->oid->hash,\n-\t\t\t\t peel_error ? NULL : peeled.hash);\n-\t}\n-\n-\tif (ok != ITER_DONE)\n-\t\tdie(\"error while iterating over references\");\n-\n-\tif (commit_lock_file(&refs->lock)) {\n-\t\tsave_errno = errno;\n-\t\terror = -1;\n-\t}\n-\trelease_packed_ref_cache(packed_ref_cache);\n-\terrno = save_errno;\n-\treturn error;\n-}\n-\n-/*\n- * Rollback the lockfile for the packed-refs file, and discard the\n- * in-memory packed reference cache. (The packed-refs file will be\n- * read anew if it is needed again after this function is called.)\n- */\n-static void rollback_packed_refs(struct packed_ref_store *refs)\n-{\n-\tstruct packed_ref_cache *packed_ref_cache = get_packed_ref_cache(refs);\n-\n-\tpacked_assert_main_repository(refs, \"rollback_packed_refs\");\n-\n-\tif (!is_lock_file_locked(&refs->lock))\n-\t\tdie(\"BUG: packed-refs not locked\");\n-\trollback_lock_file(&refs->lock);\n-\trelease_packed_ref_cache(packed_ref_cache);\n-\tclear_packed_ref_cache(refs);\n-}\n-\n struct ref_to_prune {\n \tstruct ref_to_prune *next;\n \tunsigned char sha1[20];\n@@ -1705,62 +1123,6 @@ static int files_pack_refs(struct ref_store *ref_store, unsigned int flags)\n \treturn 0;\n }\n \n-/*\n- * Rewrite the packed-refs file, omitting any refs listed in\n- * 'refnames'. On error, leave packed-refs unchanged, write an error\n- * message to 'err', and return a nonzero value.\n- *\n- * The refs in 'refnames' needn't be sorted. `err` must not be NULL.\n- */\n-static int repack_without_refs(struct packed_ref_store *refs,\n-\t\t\t struct string_list *refnames, struct strbuf *err)\n-{\n-\tstruct ref_dir *packed;\n-\tstruct string_list_item *refname;\n-\tint ret, needs_repacking = 0, removed = 0;\n-\n-\tpacked_assert_main_repository(refs, \"repack_without_refs\");\n-\tassert(err);\n-\n-\t/* Look for a packed ref */\n-\tfor_each_string_list_item(refname, refnames) {\n-\t\tif (get_packed_ref(refs, refname->string)) {\n-\t\t\tneeds_repacking = 1;\n-\t\t\tbreak;\n-\t\t}\n-\t}\n-\n-\t/* Avoid locking if we have nothing to do */\n-\tif (!needs_repacking)\n-\t\treturn 0; /* no refname exists in packed refs */\n-\n-\tif (lock_packed_refs(refs, 0)) {\n-\t\tunable_to_lock_message(refs->path, errno, err);\n-\t\treturn -1;\n-\t}\n-\tpacked = get_packed_refs(refs);\n-\n-\t/* Remove refnames from the cache */\n-\tfor_each_string_list_item(refname, refnames)\n-\t\tif (remove_entry_from_dir(packed, refname->string) != -1)\n-\t\t\tremoved = 1;\n-\tif (!removed) {\n-\t\t/*\n-\t\t * All packed entries disappeared while we were\n-\t\t * acquiring the lock.\n-\t\t */\n-\t\trollback_packed_refs(refs);\n-\t\treturn 0;\n-\t}\n-\n-\t/* Write what remains */\n-\tret = commit_packed_refs(refs);\n-\tif (ret)\n-\t\tstrbuf_addf(err, \"unable to overwrite old ref-pack file: %s\",\n-\t\t\t strerror(errno));\n-\treturn ret;\n-}\n-\n static int files_delete_refs(struct ref_store *ref_store, const char *msg,\n \t\t\t struct string_list *refnames, unsigned int flags)\n {\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nnew file mode 100644\nindex 0000000000..6fa988c953\n--- /dev/null\n+++ b/refs/packed-backend.c\n@@ -0,0 +1,624 @@\n+#include \"../cache.h\"\n+#include \"../refs.h\"\n+#include \"refs-internal.h\"\n+#include \"ref-cache.h\"\n+#include \"packed-backend.h\"\n+#include \"../iterator.h\"\n+#include \"../lockfile.h\"\n+\n+struct packed_ref_cache {\n+\tstruct ref_cache *cache;\n+\n+\t/*\n+\t * Count of references to the data structure in this instance,\n+\t * including the pointer from files_ref_store::packed if any.\n+\t * The data will not be freed as long as the reference count\n+\t * is nonzero.\n+\t */\n+\tunsigned int referrers;\n+\n+\t/* The metadata from when this packed-refs cache was read */\n+\tstruct stat_validity validity;\n+};\n+\n+/*\n+ * Increment the reference count of *packed_refs.\n+ */\n+static void acquire_packed_ref_cache(struct packed_ref_cache *packed_refs)\n+{\n+\tpacked_refs->referrers++;\n+}\n+\n+/*\n+ * Decrease the reference count of *packed_refs. If it goes to zero,\n+ * free *packed_refs and return true; otherwise return false.\n+ */\n+static int release_packed_ref_cache(struct packed_ref_cache *packed_refs)\n+{\n+\tif (!--packed_refs->referrers) {\n+\t\tfree_ref_cache(packed_refs->cache);\n+\t\tstat_validity_clear(&packed_refs->validity);\n+\t\tfree(packed_refs);\n+\t\treturn 1;\n+\t} else {\n+\t\treturn 0;\n+\t}\n+}\n+\n+/*\n+ * A container for `packed-refs`-related data. It is not (yet) a\n+ * `ref_store`.\n+ */\n+struct packed_ref_store {\n+\tunsigned int store_flags;\n+\n+\t/* The path of the \"packed-refs\" file: */\n+\tchar *path;\n+\n+\t/*\n+\t * A cache of the values read from the `packed-refs` file, if\n+\t * it might still be current; otherwise, NULL.\n+\t */\n+\tstruct packed_ref_cache *cache;\n+\n+\t/*\n+\t * Lock used for the \"packed-refs\" file. Note that this (and\n+\t * thus the enclosing `packed_ref_store`) must not be freed.\n+\t */\n+\tstruct lock_file lock;\n+};\n+\n+struct packed_ref_store *packed_ref_store_create(\n+\t\tconst char *path, unsigned int store_flags)\n+{\n+\tstruct packed_ref_store *refs = xcalloc(1, sizeof(*refs));\n+\n+\trefs->store_flags = store_flags;\n+\trefs->path = xstrdup(path);\n+\treturn refs;\n+}\n+\n+/*\n+ * Die if refs is not the main ref store. caller is used in any\n+ * necessary error messages.\n+ */\n+static void packed_assert_main_repository(struct packed_ref_store *refs,\n+\t\t\t\t\t const char *caller)\n+{\n+\tif (refs->store_flags & REF_STORE_MAIN)\n+\t\treturn;\n+\n+\tdie(\"BUG: operation %s only allowed for main ref store\", caller);\n+}\n+\n+static void clear_packed_ref_cache(struct packed_ref_store *refs)\n+{\n+\tif (refs->cache) {\n+\t\tstruct packed_ref_cache *cache = refs->cache;\n+\n+\t\tif (is_lock_file_locked(&refs->lock))\n+\t\t\tdie(\"BUG: packed-ref cache cleared while locked\");\n+\t\trefs->cache = NULL;\n+\t\trelease_packed_ref_cache(cache);\n+\t}\n+}\n+\n+/* The length of a peeled reference line in packed-refs, including EOL: */\n+#define PEELED_LINE_LENGTH 42\n+\n+/*\n+ * Parse one line from a packed-refs file. Write the SHA1 to sha1.\n+ * Return a pointer to the refname within the line (null-terminated),\n+ * or NULL if there was a problem.\n+ */\n+static const char *parse_ref_line(struct strbuf *line, struct object_id *oid)\n+{\n+\tconst char *ref;\n+\n+\tif (parse_oid_hex(line->buf, oid, &ref) < 0)\n+\t\treturn NULL;\n+\tif (!isspace(*ref++))\n+\t\treturn NULL;\n+\n+\tif (isspace(*ref))\n+\t\treturn NULL;\n+\n+\tif (line->buf[line->len - 1] != '\\n')\n+\t\treturn NULL;\n+\tline->buf[--line->len] = 0;\n+\n+\treturn ref;\n+}\n+\n+/*\n+ * Read from `packed_refs_file` into a newly-allocated\n+ * `packed_ref_cache` and return it. The return value will already\n+ * have its reference count incremented.\n+ *\n+ * A comment line of the form \"# pack-refs with: \" may contain zero or\n+ * more traits. We interpret the traits as follows:\n+ *\n+ * No traits:\n+ *\n+ * Probably no references are peeled. But if the file contains a\n+ * peeled value for a reference, we will use it.\n+ *\n+ * peeled:\n+ *\n+ * References under \"refs/tags/\", if they *can* be peeled, *are*\n+ * peeled in this file. References outside of \"refs/tags/\" are\n+ * probably not peeled even if they could have been, but if we find\n+ * a peeled value for such a reference we will use it.\n+ *\n+ * fully-peeled:\n+ *\n+ * All references in the file that can be peeled are peeled.\n+ * Inversely (and this is more important), any references in the\n+ * file for which no peeled value is recorded is not peelable. This\n+ * trait should typically be written alongside \"peeled\" for\n+ * compatibility with older clients, but we do not require it\n+ * (i.e., \"peeled\" is a no-op if \"fully-peeled\" is set).\n+ */\n+static struct packed_ref_cache *read_packed_refs(const char *packed_refs_file)\n+{\n+\tFILE *f;\n+\tstruct packed_ref_cache *packed_refs = xcalloc(1, sizeof(*packed_refs));\n+\tstruct ref_entry *last = NULL;\n+\tstruct strbuf line = STRBUF_INIT;\n+\tenum { PEELED_NONE, PEELED_TAGS, PEELED_FULLY } peeled = PEELED_NONE;\n+\tstruct ref_dir *dir;\n+\n+\tacquire_packed_ref_cache(packed_refs);\n+\tpacked_refs->cache = create_ref_cache(NULL, NULL);\n+\tpacked_refs->cache->root->flag &= ~REF_INCOMPLETE;\n+\n+\tf = fopen(packed_refs_file, \"r\");\n+\tif (!f) {\n+\t\tif (errno == ENOENT) {\n+\t\t\t/*\n+\t\t\t * This is OK; it just means that no\n+\t\t\t * \"packed-refs\" file has been written yet,\n+\t\t\t * which is equivalent to it being empty.\n+\t\t\t */\n+\t\t\treturn packed_refs;\n+\t\t} else {\n+\t\t\tdie_errno(\"couldn't read %s\", packed_refs_file);\n+\t\t}\n+\t}\n+\n+\tstat_validity_update(&packed_refs->validity, fileno(f));\n+\n+\tdir = get_ref_dir(packed_refs->cache->root);\n+\twhile (strbuf_getwholeline(&line, f, '\\n') != EOF) {\n+\t\tstruct object_id oid;\n+\t\tconst char *refname;\n+\t\tconst char *traits;\n+\n+\t\tif (skip_prefix(line.buf, \"# pack-refs with:\", &traits)) {\n+\t\t\tif (strstr(traits, \" fully-peeled \"))\n+\t\t\t\tpeeled = PEELED_FULLY;\n+\t\t\telse if (strstr(traits, \" peeled \"))\n+\t\t\t\tpeeled = PEELED_TAGS;\n+\t\t\t/* perhaps other traits later as well */\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\trefname = parse_ref_line(&line, &oid);\n+\t\tif (refname) {\n+\t\t\tint flag = REF_ISPACKED;\n+\n+\t\t\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n+\t\t\t\tif (!refname_is_safe(refname))\n+\t\t\t\t\tdie(\"packed refname is dangerous: %s\", refname);\n+\t\t\t\toidclr(&oid);\n+\t\t\t\tflag |= REF_BAD_NAME | REF_ISBROKEN;\n+\t\t\t}\n+\t\t\tlast = create_ref_entry(refname, &oid, flag);\n+\t\t\tif (peeled == PEELED_FULLY ||\n+\t\t\t (peeled == PEELED_TAGS && starts_with(refname, \"refs/tags/\")))\n+\t\t\t\tlast->flag |= REF_KNOWS_PEELED;\n+\t\t\tadd_ref_entry(dir, last);\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (last &&\n+\t\t line.buf[0] == '^' &&\n+\t\t line.len == PEELED_LINE_LENGTH &&\n+\t\t line.buf[PEELED_LINE_LENGTH - 1] == '\\n' &&\n+\t\t !get_oid_hex(line.buf + 1, &oid)) {\n+\t\t\toidcpy(&last->u.value.peeled, &oid);\n+\t\t\t/*\n+\t\t\t * Regardless of what the file header said,\n+\t\t\t * we definitely know the value of *this*\n+\t\t\t * reference:\n+\t\t\t */\n+\t\t\tlast->flag |= REF_KNOWS_PEELED;\n+\t\t}\n+\t}\n+\n+\tfclose(f);\n+\tstrbuf_release(&line);\n+\n+\treturn packed_refs;\n+}\n+\n+/*\n+ * Check that the packed refs cache (if any) still reflects the\n+ * contents of the file. If not, clear the cache.\n+ */\n+static void validate_packed_ref_cache(struct packed_ref_store *refs)\n+{\n+\tif (refs->cache &&\n+\t !stat_validity_check(&refs->cache->validity, refs->path))\n+\t\tclear_packed_ref_cache(refs);\n+}\n+\n+/*\n+ * Get the packed_ref_cache for the specified packed_ref_store,\n+ * creating and populating it if it hasn't been read before or if the\n+ * file has been changed (according to its `validity` field) since it\n+ * was last read. On the other hand, if we hold the lock, then assume\n+ * that the file hasn't been changed out from under us, so skip the\n+ * extra `stat()` call in `stat_validity_check()`.\n+ */\n+static struct packed_ref_cache *get_packed_ref_cache(struct packed_ref_store *refs)\n+{\n+\tif (!is_lock_file_locked(&refs->lock))\n+\t\tvalidate_packed_ref_cache(refs);\n+\n+\tif (!refs->cache)\n+\t\trefs->cache = read_packed_refs(refs->path);\n+\n+\treturn refs->cache;\n+}\n+\n+static struct ref_dir *get_packed_ref_dir(struct packed_ref_cache *packed_ref_cache)\n+{\n+\treturn get_ref_dir(packed_ref_cache->cache->root);\n+}\n+\n+static struct ref_dir *get_packed_refs(struct packed_ref_store *refs)\n+{\n+\treturn get_packed_ref_dir(get_packed_ref_cache(refs));\n+}\n+\n+/*\n+ * Add or overwrite a reference in the in-memory packed reference\n+ * cache. This may only be called while the packed-refs file is locked\n+ * (see lock_packed_refs()). To actually write the packed-refs file,\n+ * call commit_packed_refs().\n+ */\n+void add_packed_ref(struct packed_ref_store *refs,\n+\t\t const char *refname, const struct object_id *oid)\n+{\n+\tstruct ref_dir *packed_refs;\n+\tstruct ref_entry *packed_entry;\n+\n+\tif (!is_lock_file_locked(&refs->lock))\n+\t\tdie(\"BUG: packed refs not locked\");\n+\n+\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL))\n+\t\tdie(\"Reference has invalid format: '%s'\", refname);\n+\n+\tpacked_refs = get_packed_refs(refs);\n+\tpacked_entry = find_ref_entry(packed_refs, refname);\n+\tif (packed_entry) {\n+\t\t/* Overwrite the existing entry: */\n+\t\toidcpy(&packed_entry->u.value.oid, oid);\n+\t\tpacked_entry->flag = REF_ISPACKED;\n+\t\toidclr(&packed_entry->u.value.peeled);\n+\t} else {\n+\t\tpacked_entry = create_ref_entry(refname, oid, REF_ISPACKED);\n+\t\tadd_ref_entry(packed_refs, packed_entry);\n+\t}\n+}\n+\n+/*\n+ * Return the ref_entry for the given refname from the packed\n+ * references. If it does not exist, return NULL.\n+ */\n+static struct ref_entry *get_packed_ref(struct packed_ref_store *refs,\n+\t\t\t\t\tconst char *refname)\n+{\n+\treturn find_ref_entry(get_packed_refs(refs), refname);\n+}\n+\n+int packed_read_raw_ref(struct packed_ref_store *refs,\n+\t\t\tconst char *refname, unsigned char *sha1,\n+\t\t\tstruct strbuf *referent, unsigned int *type)\n+{\n+\tstruct ref_entry *entry;\n+\n+\t*type = 0;\n+\n+\tentry = get_packed_ref(refs, refname);\n+\tif (!entry) {\n+\t\terrno = ENOENT;\n+\t\treturn -1;\n+\t}\n+\n+\thashcpy(sha1, entry->u.value.oid.hash);\n+\t*type = REF_ISPACKED;\n+\treturn 0;\n+}\n+\n+int packed_peel_ref(struct packed_ref_store *refs,\n+\t\t const char *refname, unsigned char *sha1)\n+{\n+\tstruct ref_entry *r = get_packed_ref(refs, refname);\n+\n+\tif (!r || peel_entry(r, 0))\n+\t\treturn -1;\n+\n+\thashcpy(sha1, r->u.value.peeled.hash);\n+\treturn 0;\n+}\n+\n+struct packed_ref_iterator {\n+\tstruct ref_iterator base;\n+\n+\tstruct packed_ref_cache *cache;\n+\tstruct ref_iterator *iter0;\n+\tunsigned int flags;\n+};\n+\n+static int packed_ref_iterator_advance(struct ref_iterator *ref_iterator)\n+{\n+\tstruct packed_ref_iterator *iter =\n+\t\t(struct packed_ref_iterator *)ref_iterator;\n+\tint ok;\n+\n+\twhile ((ok = ref_iterator_advance(iter->iter0)) == ITER_OK) {\n+\t\tif (iter->flags & DO_FOR_EACH_PER_WORKTREE_ONLY &&\n+\t\t ref_type(iter->iter0->refname) != REF_TYPE_PER_WORKTREE)\n+\t\t\tcontinue;\n+\n+\t\tif (!(iter->flags & DO_FOR_EACH_INCLUDE_BROKEN) &&\n+\t\t !ref_resolves_to_object(iter->iter0->refname,\n+\t\t\t\t\t iter->iter0->oid,\n+\t\t\t\t\t iter->iter0->flags))\n+\t\t\tcontinue;\n+\n+\t\titer->base.refname = iter->iter0->refname;\n+\t\titer->base.oid = iter->iter0->oid;\n+\t\titer->base.flags = iter->iter0->flags;\n+\t\treturn ITER_OK;\n+\t}\n+\n+\titer->iter0 = NULL;\n+\tif (ref_iterator_abort(ref_iterator) != ITER_DONE)\n+\t\tok = ITER_ERROR;\n+\n+\treturn ok;\n+}\n+\n+static int packed_ref_iterator_peel(struct ref_iterator *ref_iterator,\n+\t\t\t\t struct object_id *peeled)\n+{\n+\tstruct packed_ref_iterator *iter =\n+\t\t(struct packed_ref_iterator *)ref_iterator;\n+\n+\treturn ref_iterator_peel(iter->iter0, peeled);\n+}\n+\n+static int packed_ref_iterator_abort(struct ref_iterator *ref_iterator)\n+{\n+\tstruct packed_ref_iterator *iter =\n+\t\t(struct packed_ref_iterator *)ref_iterator;\n+\tint ok = ITER_DONE;\n+\n+\tif (iter->iter0)\n+\t\tok = ref_iterator_abort(iter->iter0);\n+\n+\trelease_packed_ref_cache(iter->cache);\n+\tbase_ref_iterator_free(ref_iterator);\n+\treturn ok;\n+}\n+\n+static struct ref_iterator_vtable packed_ref_iterator_vtable = {\n+\tpacked_ref_iterator_advance,\n+\tpacked_ref_iterator_peel,\n+\tpacked_ref_iterator_abort\n+};\n+\n+struct ref_iterator *packed_ref_iterator_begin(\n+\t\tstruct packed_ref_store *refs,\n+\t\tconst char *prefix, unsigned int flags)\n+{\n+\tstruct packed_ref_iterator *iter;\n+\tstruct ref_iterator *ref_iterator;\n+\n+\titer = xcalloc(1, sizeof(*iter));\n+\tref_iterator = &iter->base;\n+\tbase_ref_iterator_init(ref_iterator, &packed_ref_iterator_vtable);\n+\n+\t/*\n+\t * Note that get_packed_ref_cache() internally checks whether\n+\t * the packed-ref cache is up to date with what is on disk,\n+\t * and re-reads it if not.\n+\t */\n+\n+\titer->cache = get_packed_ref_cache(refs);\n+\tacquire_packed_ref_cache(iter->cache);\n+\titer->iter0 = cache_ref_iterator_begin(iter->cache->cache, prefix, 0);\n+\n+\titer->flags = flags;\n+\n+\treturn ref_iterator;\n+}\n+\n+/*\n+ * Write an entry to the packed-refs file for the specified refname.\n+ * If peeled is non-NULL, write it as the entry's peeled value.\n+ */\n+static void write_packed_entry(FILE *fh, const char *refname,\n+\t\t\t const unsigned char *sha1,\n+\t\t\t const unsigned char *peeled)\n+{\n+\tfprintf_or_die(fh, \"%s %s\\n\", sha1_to_hex(sha1), refname);\n+\tif (peeled)\n+\t\tfprintf_or_die(fh, \"^%s\\n\", sha1_to_hex(peeled));\n+}\n+\n+int lock_packed_refs(struct packed_ref_store *refs, int flags)\n+{\n+\tstatic int timeout_configured = 0;\n+\tstatic int timeout_value = 1000;\n+\tstruct packed_ref_cache *packed_ref_cache;\n+\n+\tpacked_assert_main_repository(refs, \"lock_packed_refs\");\n+\n+\tif (!timeout_configured) {\n+\t\tgit_config_get_int(\"core.packedrefstimeout\", &timeout_value);\n+\t\ttimeout_configured = 1;\n+\t}\n+\n+\tif (hold_lock_file_for_update_timeout(\n+\t\t\t &refs->lock,\n+\t\t\t refs->path,\n+\t\t\t flags, timeout_value) < 0)\n+\t\treturn -1;\n+\n+\t/*\n+\t * Now that we hold the `packed-refs` lock, make sure that our\n+\t * cache matches the current version of the file. Normally\n+\t * `get_packed_ref_cache()` does that for us, but that\n+\t * function assumes that when the file is locked, any existing\n+\t * cache is still valid. We've just locked the file, but it\n+\t * might have changed the moment *before* we locked it.\n+\t */\n+\tvalidate_packed_ref_cache(refs);\n+\n+\tpacked_ref_cache = get_packed_ref_cache(refs);\n+\t/* Increment the reference count to prevent it from being freed: */\n+\tacquire_packed_ref_cache(packed_ref_cache);\n+\treturn 0;\n+}\n+\n+/*\n+ * The packed-refs header line that we write out. Perhaps other\n+ * traits will be added later. The trailing space is required.\n+ */\n+static const char PACKED_REFS_HEADER[] =\n+\t\"# pack-refs with: peeled fully-peeled \\n\";\n+\n+/*\n+ * Write the current version of the packed refs cache from memory to\n+ * disk. The packed-refs file must already be locked for writing (see\n+ * lock_packed_refs()). Return zero on success. On errors, set errno\n+ * and return a nonzero value.\n+ */\n+int commit_packed_refs(struct packed_ref_store *refs)\n+{\n+\tstruct packed_ref_cache *packed_ref_cache =\n+\t\tget_packed_ref_cache(refs);\n+\tint ok, error = 0;\n+\tint save_errno = 0;\n+\tFILE *out;\n+\tstruct ref_iterator *iter;\n+\n+\tpacked_assert_main_repository(refs, \"commit_packed_refs\");\n+\n+\tif (!is_lock_file_locked(&refs->lock))\n+\t\tdie(\"BUG: packed-refs not locked\");\n+\n+\tout = fdopen_lock_file(&refs->lock, \"w\");\n+\tif (!out)\n+\t\tdie_errno(\"unable to fdopen packed-refs descriptor\");\n+\n+\tfprintf_or_die(out, \"%s\", PACKED_REFS_HEADER);\n+\n+\titer = cache_ref_iterator_begin(packed_ref_cache->cache, NULL, 0);\n+\twhile ((ok = ref_iterator_advance(iter)) == ITER_OK) {\n+\t\tstruct object_id peeled;\n+\t\tint peel_error = ref_iterator_peel(iter, &peeled);\n+\n+\t\twrite_packed_entry(out, iter->refname, iter->oid->hash,\n+\t\t\t\t peel_error ? NULL : peeled.hash);\n+\t}\n+\n+\tif (ok != ITER_DONE)\n+\t\tdie(\"error while iterating over references\");\n+\n+\tif (commit_lock_file(&refs->lock)) {\n+\t\tsave_errno = errno;\n+\t\terror = -1;\n+\t}\n+\trelease_packed_ref_cache(packed_ref_cache);\n+\terrno = save_errno;\n+\treturn error;\n+}\n+\n+/*\n+ * Rollback the lockfile for the packed-refs file, and discard the\n+ * in-memory packed reference cache. (The packed-refs file will be\n+ * read anew if it is needed again after this function is called.)\n+ */\n+static void rollback_packed_refs(struct packed_ref_store *refs)\n+{\n+\tstruct packed_ref_cache *packed_ref_cache = get_packed_ref_cache(refs);\n+\n+\tpacked_assert_main_repository(refs, \"rollback_packed_refs\");\n+\n+\tif (!is_lock_file_locked(&refs->lock))\n+\t\tdie(\"BUG: packed-refs not locked\");\n+\trollback_lock_file(&refs->lock);\n+\trelease_packed_ref_cache(packed_ref_cache);\n+\tclear_packed_ref_cache(refs);\n+}\n+\n+/*\n+ * Rewrite the packed-refs file, omitting any refs listed in\n+ * 'refnames'. On error, leave packed-refs unchanged, write an error\n+ * message to 'err', and return a nonzero value.\n+ *\n+ * The refs in 'refnames' needn't be sorted. `err` must not be NULL.\n+ */\n+int repack_without_refs(struct packed_ref_store *refs,\n+\t\t\tstruct string_list *refnames, struct strbuf *err)\n+{\n+\tstruct ref_dir *packed;\n+\tstruct string_list_item *refname;\n+\tint ret, needs_repacking = 0, removed = 0;\n+\n+\tpacked_assert_main_repository(refs, \"repack_without_refs\");\n+\tassert(err);\n+\n+\t/* Look for a packed ref */\n+\tfor_each_string_list_item(refname, refnames) {\n+\t\tif (get_packed_ref(refs, refname->string)) {\n+\t\t\tneeds_repacking = 1;\n+\t\t\tbreak;\n+\t\t}\n+\t}\n+\n+\t/* Avoid locking if we have nothing to do */\n+\tif (!needs_repacking)\n+\t\treturn 0; /* no refname exists in packed refs */\n+\n+\tif (lock_packed_refs(refs, 0)) {\n+\t\tunable_to_lock_message(refs->path, errno, err);\n+\t\treturn -1;\n+\t}\n+\tpacked = get_packed_refs(refs);\n+\n+\t/* Remove refnames from the cache */\n+\tfor_each_string_list_item(refname, refnames)\n+\t\tif (remove_entry_from_dir(packed, refname->string) != -1)\n+\t\t\tremoved = 1;\n+\tif (!removed) {\n+\t\t/*\n+\t\t * All packed entries disappeared while we were\n+\t\t * acquiring the lock.\n+\t\t */\n+\t\trollback_packed_refs(refs);\n+\t\treturn 0;\n+\t}\n+\n+\t/* Write what remains */\n+\tret = commit_packed_refs(refs);\n+\tif (ret)\n+\t\tstrbuf_addf(err, \"unable to overwrite old ref-pack file: %s\",\n+\t\t\t strerror(errno));\n+\treturn ret;\n+}\n+\ndiff --git a/refs/packed-backend.h b/refs/packed-backend.h\nnew file mode 100644\nindex 0000000000..22e8817ac4\n--- /dev/null\n+++ b/refs/packed-backend.h\n@@ -0,0 +1,33 @@\n+#ifndef REFS_PACKED_BACKEND_H\n+#define REFS_PACKED_BACKEND_H\n+\n+struct packed_ref_store *packed_ref_store_create(\n+\t\tconst char *path, unsigned int store_flags);\n+\n+int packed_read_raw_ref(struct packed_ref_store *refs,\n+\t\t\tconst char *refname, unsigned char *sha1,\n+\t\t\tstruct strbuf *referent, unsigned int *type);\n+\n+int packed_peel_ref(struct packed_ref_store *refs,\n+\t\t const char *refname, unsigned char *sha1);\n+\n+struct ref_iterator *packed_ref_iterator_begin(\n+\t\tstruct packed_ref_store *refs,\n+\t\tconst char *prefix, unsigned int flags);\n+\n+/*\n+ * Lock the packed-refs file for writing. Flags is passed to\n+ * hold_lock_file_for_update(). Return 0 on success. On errors, set\n+ * errno appropriately and return a nonzero value.\n+ */\n+int lock_packed_refs(struct packed_ref_store *refs, int flags);\n+\n+void add_packed_ref(struct packed_ref_store *refs,\n+\t\t const char *refname, const struct object_id *oid);\n+\n+int commit_packed_refs(struct packed_ref_store *refs);\n+\n+int repack_without_refs(struct packed_ref_store *refs,\n+\t\t\tstruct string_list *refnames, struct strbuf *err);\n+\n+#endif /* REFS_PACKED_BACKEND_H */\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 192f9f85c9..6f8f9f5619 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -77,6 +77,15 @@\n */\n int refname_is_safe(const char *refname);\n \n+/*\n+ * Helper function: return true if refname, which has the specified\n+ * oid and flags, can be resolved to an object in the database. If the\n+ * referred-to object does not exist, emit a warning and return false.\n+ */\n+int ref_resolves_to_object(const char *refname,\n+\t\t\t const struct object_id *oid,\n+\t\t\t unsigned int flags);\n+\n enum peel_status {\n \t/* object was peeled successfully: */\n \tPEEL_PEELED = 0,\n-- \n2.11.0\n\n"},{"id":"322346","messageId":"6ec1897ceccdc0cb8cf49b8c7da77ad3e978f52e.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 24/28] packed_refs_unlock(), packed_refs_is_locked(): new functions","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:29Z","receivedAt":"2017-06-15T14:48:42Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"Add two new public functions, `packed_refs_unlock()` and\n`packed_refs_is_locked()`, with which callers can manage and query the\n`packed-refs` lock externally.\n\nCall `packed_refs_unlock()` from `commit_packed_refs()` and\n`rollback_packed_refs()`.\n\nSigned-off-by: Michael Haggerty \n---\n refs/packed-backend.c | 31 +++++++++++++++++++++++++------\n refs/packed-backend.h | 3 +++\n 2 files changed, 28 insertions(+), 6 deletions(-)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 24451343b8..ff6326ddb8 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -563,6 +563,29 @@ int packed_refs_lock(struct ref_store *ref_store, int flags, struct strbuf *err)\n \treturn 0;\n }\n \n+void packed_refs_unlock(struct ref_store *ref_store)\n+{\n+\tstruct packed_ref_store *refs = packed_downcast(\n+\t\t\tref_store,\n+\t\t\tREF_STORE_READ | REF_STORE_WRITE,\n+\t\t\t\"packed_refs_unlock\");\n+\n+\tif (!is_lock_file_locked(&refs->lock))\n+\t\tdie(\"BUG: packed_refs_unlock() called when not locked\");\n+\trollback_lock_file(&refs->lock);\n+\trelease_packed_ref_cache(refs->cache);\n+}\n+\n+int packed_refs_is_locked(struct ref_store *ref_store)\n+{\n+\tstruct packed_ref_store *refs = packed_downcast(\n+\t\t\tref_store,\n+\t\t\tREF_STORE_READ | REF_STORE_WRITE,\n+\t\t\t\"packed_refs_is_locked\");\n+\n+\treturn is_lock_file_locked(&refs->lock);\n+}\n+\n /*\n * The packed-refs header line that we write out. Perhaps other\n * traits will be added later. The trailing space is required.\n@@ -649,8 +672,7 @@ int commit_packed_refs(struct ref_store *ref_store, struct strbuf *err)\n \tdelete_tempfile(&refs->tempfile);\n \n out:\n-\trollback_lock_file(&refs->lock);\n-\trelease_packed_ref_cache(packed_ref_cache);\n+\tpacked_refs_unlock(ref_store);\n \treturn ret;\n }\n \n@@ -661,14 +683,11 @@ int commit_packed_refs(struct ref_store *ref_store, struct strbuf *err)\n */\n static void rollback_packed_refs(struct packed_ref_store *refs)\n {\n-\tstruct packed_ref_cache *packed_ref_cache = get_packed_ref_cache(refs);\n-\n \tpacked_assert_main_repository(refs, \"rollback_packed_refs\");\n \n \tif (!is_lock_file_locked(&refs->lock))\n \t\tdie(\"BUG: packed-refs not locked\");\n-\trollback_lock_file(&refs->lock);\n-\trelease_packed_ref_cache(packed_ref_cache);\n+\tpacked_refs_unlock(&refs->base);\n \tclear_packed_ref_cache(refs);\n }\n \ndiff --git a/refs/packed-backend.h b/refs/packed-backend.h\nindex 210e3f35ce..03b7c1de95 100644\n--- a/refs/packed-backend.h\n+++ b/refs/packed-backend.h\n@@ -11,6 +11,9 @@ struct ref_store *packed_ref_store_create(const char *path,\n */\n int packed_refs_lock(struct ref_store *ref_store, int flags, struct strbuf *err);\n \n+void packed_refs_unlock(struct ref_store *ref_store);\n+int packed_refs_is_locked(struct ref_store *ref_store);\n+\n void add_packed_ref(struct ref_store *ref_store,\n \t\t const char *refname, const struct object_id *oid);\n \n-- \n2.11.0\n\n"},{"id":"322347","messageId":"7d23641b9e9d3843fe9ddb36a372c79af79ef465.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 26/28] commit_packed_refs(): remove call to `packed_refs_unlock()`","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:31Z","receivedAt":"2017-06-15T14:48:46Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"Instead, change the callers of `commit_packed_refs()` to call\n`packed_refs_unlock()`.\n\nSigned-off-by: Michael Haggerty \n---\n refs/files-backend.c | 2 ++\n refs/packed-backend.c | 18 ++++++++----------\n 2 files changed, 10 insertions(+), 10 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 802ed9e2e9..09dad2806e 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1119,6 +1119,7 @@ static int files_pack_refs(struct ref_store *ref_store, unsigned int flags)\n \n \tif (commit_packed_refs(refs->packed_ref_store, &err))\n \t\tdie(\"unable to overwrite old ref-pack file: %s\", err.buf);\n+\tpacked_refs_unlock(refs->packed_ref_store);\n \n \tprune_refs(refs, refs_to_prune);\n \tstrbuf_release(&err);\n@@ -2687,6 +2688,7 @@ static int files_initial_transaction_commit(struct ref_store *ref_store,\n \t}\n \n cleanup:\n+\tpacked_refs_unlock(refs->packed_ref_store);\n \ttransaction->state = REF_TRANSACTION_CLOSED;\n \tstring_list_clear(&affected_refnames, 0);\n \treturn ret;\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 1732e3aad4..54b48d1f02 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -606,7 +606,6 @@ int commit_packed_refs(struct ref_store *ref_store, struct strbuf *err)\n \tstruct packed_ref_cache *packed_ref_cache =\n \t\tget_packed_ref_cache(refs);\n \tint ok;\n-\tint ret = -1;\n \tstruct strbuf sb = STRBUF_INIT;\n \tFILE *out;\n \tstruct ref_iterator *iter;\n@@ -619,7 +618,7 @@ int commit_packed_refs(struct ref_store *ref_store, struct strbuf *err)\n \t\tstrbuf_addf(err, \"unable to create file %s: %s\",\n \t\t\t sb.buf, strerror(errno));\n \t\tstrbuf_release(&sb);\n-\t\tgoto out;\n+\t\treturn -1;\n \t}\n \tstrbuf_release(&sb);\n \n@@ -660,18 +659,14 @@ int commit_packed_refs(struct ref_store *ref_store, struct strbuf *err)\n \tif (rename_tempfile(&refs->tempfile, refs->path)) {\n \t\tstrbuf_addf(err, \"error replacing %s: %s\",\n \t\t\t refs->path, strerror(errno));\n-\t\tgoto out;\n+\t\treturn -1;\n \t}\n \n-\tret = 0;\n-\tgoto out;\n+\treturn 0;\n \n error:\n \tdelete_tempfile(&refs->tempfile);\n-\n-out:\n-\tpacked_refs_unlock(ref_store);\n-\treturn ret;\n+\treturn -1;\n }\n \n /*\n@@ -705,6 +700,7 @@ int repack_without_refs(struct ref_store *ref_store,\n \tstruct ref_dir *packed;\n \tstruct string_list_item *refname;\n \tint needs_repacking = 0, removed = 0;\n+\tint ret;\n \n \tpacked_assert_main_repository(refs, \"repack_without_refs\");\n \tassert(err);\n@@ -740,7 +736,9 @@ int repack_without_refs(struct ref_store *ref_store,\n \t}\n \n \t/* Write what remains */\n-\treturn commit_packed_refs(&refs->base, err);\n+\tret = commit_packed_refs(&refs->base, err);\n+\tpacked_refs_unlock(ref_store);\n+\treturn ret;\n }\n \n static int packed_init_db(struct ref_store *ref_store, struct strbuf *err)\n-- \n2.11.0\n\n"},{"id":"322348","messageId":"c13b2fad1fe0bc521b58fdf3115ee47a11db770b.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 28/28] read_packed_refs(): die if `packed-refs` contains bogus data","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:33Z","receivedAt":"2017-06-15T14:48:49Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"The old code ignored any lines that it didn't understand. This is\ndangerous. Instead, `die()` if the `packed-refs` file contains any\nlines that we don't know how to handle.\n\nSigned-off-by: Michael Haggerty \n---\n refs/packed-backend.c | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 721afd066a..311fd014ce 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -253,9 +253,7 @@ static struct packed_ref_cache *read_packed_refs(const char *packed_refs_file)\n \t\t\t (peeled == PEELED_TAGS && starts_with(refname, \"refs/tags/\")))\n \t\t\t\tlast->flag |= REF_KNOWS_PEELED;\n \t\t\tadd_ref_entry(dir, last);\n-\t\t\tcontinue;\n-\t\t}\n-\t\tif (last &&\n+\t\t} else if (last &&\n \t\t line.buf[0] == '^' &&\n \t\t line.len == PEELED_LINE_LENGTH &&\n \t\t line.buf[PEELED_LINE_LENGTH - 1] == '\\n' &&\n@@ -267,6 +265,8 @@ static struct packed_ref_cache *read_packed_refs(const char *packed_refs_file)\n \t\t\t * reference:\n \t\t\t */\n \t\t\tlast->flag |= REF_KNOWS_PEELED;\n+\t\t} else {\n+\t\t\tdie(\"unexpected line in %s: %s\", packed_refs_file, line.buf);\n \t\t}\n \t}\n \n-- \n2.11.0\n\n"},{"id":"322349","messageId":"64020d0dc582f44e5f83c36db0afdfdfdf7ea1b6.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 23/28] packed_refs_lock(): report errors via a `struct strbuf *err`","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:28Z","receivedAt":"2017-06-15T14:48:50Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"That way the callers don't have to come up with error messages\nthemselves.\n\nSigned-off-by: Michael Haggerty \n---\n refs/files-backend.c | 6 ++----\n refs/packed-backend.c | 17 +++++++++++------\n refs/packed-backend.h | 6 +++---\n 3 files changed, 16 insertions(+), 13 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 3fc2254e33..802ed9e2e9 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1084,7 +1084,7 @@ static int files_pack_refs(struct ref_store *ref_store, unsigned int flags)\n \tstruct ref_to_prune *refs_to_prune = NULL;\n \tstruct strbuf err = STRBUF_INIT;\n \n-\tpacked_refs_lock(refs->packed_ref_store, LOCK_DIE_ON_ERROR);\n+\tpacked_refs_lock(refs->packed_ref_store, LOCK_DIE_ON_ERROR, &err);\n \n \titer = cache_ref_iterator_begin(get_loose_ref_cache(refs), NULL, 0);\n \twhile ((ok = ref_iterator_advance(iter)) == ITER_OK) {\n@@ -2667,9 +2667,7 @@ static int files_initial_transaction_commit(struct ref_store *ref_store,\n \t\t}\n \t}\n \n-\tif (packed_refs_lock(refs->packed_ref_store, 0)) {\n-\t\tstrbuf_addf(err, \"unable to lock packed-refs file: %s\",\n-\t\t\t strerror(errno));\n+\tif (packed_refs_lock(refs->packed_ref_store, 0, err)) {\n \t\tret = TRANSACTION_GENERIC_ERROR;\n \t\tgoto cleanup;\n \t}\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex eb9da04576..24451343b8 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -515,7 +515,7 @@ static int write_packed_entry(FILE *fh, const char *refname,\n \treturn 0;\n }\n \n-int packed_refs_lock(struct ref_store *ref_store, int flags)\n+int packed_refs_lock(struct ref_store *ref_store, int flags, struct strbuf *err)\n {\n \tstruct packed_ref_store *refs =\n \t packed_downcast(ref_store, REF_STORE_WRITE | REF_STORE_MAIN,\n@@ -537,9 +537,15 @@ int packed_refs_lock(struct ref_store *ref_store, int flags)\n \tif (hold_lock_file_for_update_timeout(\n \t\t\t &refs->lock,\n \t\t\t refs->path,\n-\t\t\t flags, timeout_value) < 0 ||\n-\t close_lock_file(&refs->lock))\n+\t\t\t flags, timeout_value) < 0) {\n+\t\tunable_to_lock_message(refs->path, errno, err);\n+\t\treturn -1;\n+\t}\n+\n+\tif (close_lock_file(&refs->lock)) {\n+\t\tstrbuf_addf(err, \"unable to close %s: %s\", refs->path, strerror(errno));\n \t\treturn -1;\n+\t}\n \n \t/*\n \t * Now that we hold the `packed-refs` lock, make sure that our\n@@ -698,10 +704,9 @@ int repack_without_refs(struct ref_store *ref_store,\n \tif (!needs_repacking)\n \t\treturn 0; /* no refname exists in packed refs */\n \n-\tif (packed_refs_lock(&refs->base, 0)) {\n-\t\tunable_to_lock_message(refs->path, errno, err);\n+\tif (packed_refs_lock(&refs->base, 0, err))\n \t\treturn -1;\n-\t}\n+\n \tpacked = get_packed_refs(refs);\n \n \t/* Remove refnames from the cache */\ndiff --git a/refs/packed-backend.h b/refs/packed-backend.h\nindex dbc00d3396..210e3f35ce 100644\n--- a/refs/packed-backend.h\n+++ b/refs/packed-backend.h\n@@ -6,10 +6,10 @@ struct ref_store *packed_ref_store_create(const char *path,\n \n /*\n * Lock the packed-refs file for writing. Flags is passed to\n- * hold_lock_file_for_update(). Return 0 on success. On errors, set\n- * errno appropriately and return a nonzero value.\n+ * hold_lock_file_for_update(). Return 0 on success. On errors, write\n+ * an error message to `err` and return a nonzero value.\n */\n-int packed_refs_lock(struct ref_store *ref_store, int flags);\n+int packed_refs_lock(struct ref_store *ref_store, int flags, struct strbuf *err);\n \n void add_packed_ref(struct ref_store *ref_store,\n \t\t const char *refname, const struct object_id *oid);\n-- \n2.11.0\n\n"},{"id":"322350","messageId":"44c3fe546430eead4af08479c2bc30cef37d973e.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 27/28] repack_without_refs(): don't lock or unlock the packed refs","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:32Z","receivedAt":"2017-06-15T14:48:52Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"Change `repack_without_refs()` to expect the packed-refs lock to be\nheld already, and not to release the lock before returning. Change the\ncallers to deal with lock management.\n\nThis change makes it possible for callers to hold the packed-refs lock\nfor a longer span of time, a possibility that will eventually make it\npossible to fix some longstanding races.\n\nThe only semantic change here is that `repack_without_refs()` used to\nforgot to release the lock in the `if (!removed)` exit path. That\nomission is now fixed.\n\nSigned-off-by: Michael Haggerty \n---\n refs/files-backend.c | 47 +++++++++++++++++++++++++++++++----------------\n refs/packed-backend.c | 32 ++++++++------------------------\n 2 files changed, 39 insertions(+), 40 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 09dad2806e..972dd5c0c9 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1137,24 +1137,16 @@ static int files_delete_refs(struct ref_store *ref_store, const char *msg,\n \tif (!refnames->nr)\n \t\treturn 0;\n \n-\tresult = repack_without_refs(refs->packed_ref_store, refnames, &err);\n-\tif (result) {\n-\t\t/*\n-\t\t * If we failed to rewrite the packed-refs file, then\n-\t\t * it is unsafe to try to remove loose refs, because\n-\t\t * doing so might expose an obsolete packed value for\n-\t\t * a reference that might even point at an object that\n-\t\t * has been garbage collected.\n-\t\t */\n-\t\tif (refnames->nr == 1)\n-\t\t\terror(_(\"could not delete reference %s: %s\"),\n-\t\t\t refnames->items[0].string, err.buf);\n-\t\telse\n-\t\t\terror(_(\"could not delete references: %s\"), err.buf);\n+\tif (packed_refs_lock(refs->packed_ref_store, 0, &err))\n+\t\tgoto error;\n \n-\t\tgoto out;\n+\tif (repack_without_refs(refs->packed_ref_store, refnames, &err)) {\n+\t\tpacked_refs_unlock(refs->packed_ref_store);\n+\t\tgoto error;\n \t}\n \n+\tpacked_refs_unlock(refs->packed_ref_store);\n+\n \tfor (i = 0; i < refnames->nr; i++) {\n \t\tconst char *refname = refnames->items[i].string;\n \n@@ -1162,9 +1154,24 @@ static int files_delete_refs(struct ref_store *ref_store, const char *msg,\n \t\t\tresult |= error(_(\"could not remove reference %s\"), refname);\n \t}\n \n-out:\n \tstrbuf_release(&err);\n \treturn result;\n+\n+error:\n+\t/*\n+\t * If we failed to rewrite the packed-refs file, then it is\n+\t * unsafe to try to remove loose refs, because doing so might\n+\t * expose an obsolete packed value for a reference that might\n+\t * even point at an object that has been garbage collected.\n+\t */\n+\tif (refnames->nr == 1)\n+\t\terror(_(\"could not delete reference %s: %s\"),\n+\t\t refnames->items[0].string, err.buf);\n+\telse\n+\t\terror(_(\"could not delete references: %s\"), err.buf);\n+\n+\tstrbuf_release(&err);\n+\treturn -1;\n }\n \n /*\n@@ -2557,11 +2564,19 @@ static int files_transaction_finish(struct ref_store *ref_store,\n \t\t}\n \t}\n \n+\tif (packed_refs_lock(refs->packed_ref_store, 0, err)) {\n+\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\tgoto cleanup;\n+\t}\n+\n \tif (repack_without_refs(refs->packed_ref_store, &refs_to_delete, err)) {\n \t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\tpacked_refs_unlock(refs->packed_ref_store);\n \t\tgoto cleanup;\n \t}\n \n+\tpacked_refs_unlock(refs->packed_ref_store);\n+\n \t/* Delete the reflogs of any references that were deleted: */\n \tfor_each_string_list_item(ref_to_delete, &refs_to_delete) {\n \t\tstrbuf_reset(&sb);\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 54b48d1f02..721afd066a 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -669,25 +669,12 @@ int commit_packed_refs(struct ref_store *ref_store, struct strbuf *err)\n \treturn -1;\n }\n \n-/*\n- * Rollback the lockfile for the packed-refs file, and discard the\n- * in-memory packed reference cache. (The packed-refs file will be\n- * read anew if it is needed again after this function is called.)\n- */\n-static void rollback_packed_refs(struct packed_ref_store *refs)\n-{\n-\tpacked_assert_main_repository(refs, \"rollback_packed_refs\");\n-\n-\tif (!is_lock_file_locked(&refs->lock))\n-\t\tdie(\"BUG: packed-refs not locked\");\n-\tpacked_refs_unlock(&refs->base);\n-\tclear_packed_ref_cache(refs);\n-}\n-\n /*\n * Rewrite the packed-refs file, omitting any refs listed in\n * 'refnames'. On error, leave packed-refs unchanged, write an error\n- * message to 'err', and return a nonzero value.\n+ * message to 'err', and return a nonzero value. The packed refs lock\n+ * must be held when calling this function; it will still be held when\n+ * the function returns.\n *\n * The refs in 'refnames' needn't be sorted. `err` must not be NULL.\n */\n@@ -700,11 +687,13 @@ int repack_without_refs(struct ref_store *ref_store,\n \tstruct ref_dir *packed;\n \tstruct string_list_item *refname;\n \tint needs_repacking = 0, removed = 0;\n-\tint ret;\n \n \tpacked_assert_main_repository(refs, \"repack_without_refs\");\n \tassert(err);\n \n+\tif (!is_lock_file_locked(&refs->lock))\n+\t\tdie(\"BUG: repack_without_refs called without holding lock\");\n+\n \t/* Look for a packed ref */\n \tfor_each_string_list_item(refname, refnames) {\n \t\tif (get_packed_ref(refs, refname->string)) {\n@@ -717,9 +706,6 @@ int repack_without_refs(struct ref_store *ref_store,\n \tif (!needs_repacking)\n \t\treturn 0; /* no refname exists in packed refs */\n \n-\tif (packed_refs_lock(&refs->base, 0, err))\n-\t\treturn -1;\n-\n \tpacked = get_packed_refs(refs);\n \n \t/* Remove refnames from the cache */\n@@ -731,14 +717,12 @@ int repack_without_refs(struct ref_store *ref_store,\n \t\t * All packed entries disappeared while we were\n \t\t * acquiring the lock.\n \t\t */\n-\t\trollback_packed_refs(refs);\n+\t\tclear_packed_ref_cache(refs);\n \t\treturn 0;\n \t}\n \n \t/* Write what remains */\n-\tret = commit_packed_refs(&refs->base, err);\n-\tpacked_refs_unlock(ref_store);\n-\treturn ret;\n+\treturn commit_packed_refs(&refs->base, err);\n }\n \n static int packed_init_db(struct ref_store *ref_store, struct strbuf *err)\n-- \n2.11.0\n\n"},{"id":"322351","messageId":"3be564c3f49454cc0ea48bf5ebb70a34e17bd41d.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 25/28] clear_packed_ref_cache(): don't protest if the lock is held","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:30Z","receivedAt":"2017-06-15T14:48:54Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"The existing callers already check that the lock isn't held just\nbefore calling `clear_packed_ref_cache()`, and in the near future we\nwant to be able to call this function when the lock is held.\n\nSigned-off-by: Michael Haggerty \n---\n refs/packed-backend.c | 2 --\n 1 file changed, 2 deletions(-)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex ff6326ddb8..1732e3aad4 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -133,8 +133,6 @@ static void clear_packed_ref_cache(struct packed_ref_store *refs)\n \tif (refs->cache) {\n \t\tstruct packed_ref_cache *cache = refs->cache;\n \n-\t\tif (is_lock_file_locked(&refs->lock))\n-\t\t\tdie(\"BUG: packed-ref cache cleared while locked\");\n \t\trefs->cache = NULL;\n \t\trelease_packed_ref_cache(cache);\n \t}\n-- \n2.11.0\n\n"},{"id":"322352","messageId":"ba97a6b127e31d76557464e6c88d8c33831ae173.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 12/28] rollback_packed_refs(): take a `packed_ref_store *` parameter","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:17Z","receivedAt":"2017-06-15T14:49:00Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"It only cares about the packed-refs part of the reference store.\n\nSigned-off-by: Michael Haggerty \n---\n refs/files-backend.c | 15 +++++++--------\n 1 file changed, 7 insertions(+), 8 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 5d159620f0..a08d3fbadf 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1434,18 +1434,17 @@ static int commit_packed_refs(struct packed_ref_store *refs)\n * in-memory packed reference cache. (The packed-refs file will be\n * read anew if it is needed again after this function is called.)\n */\n-static void rollback_packed_refs(struct files_ref_store *refs)\n+static void rollback_packed_refs(struct packed_ref_store *refs)\n {\n-\tstruct packed_ref_cache *packed_ref_cache =\n-\t\tget_packed_ref_cache(refs->packed_ref_store);\n+\tstruct packed_ref_cache *packed_ref_cache = get_packed_ref_cache(refs);\n \n-\tfiles_assert_main_repository(refs, \"rollback_packed_refs\");\n+\tpacked_assert_main_repository(refs, \"rollback_packed_refs\");\n \n-\tif (!is_lock_file_locked(&refs->packed_ref_store->lock))\n+\tif (!is_lock_file_locked(&refs->lock))\n \t\tdie(\"BUG: packed-refs not locked\");\n-\trollback_lock_file(&refs->packed_ref_store->lock);\n+\trollback_lock_file(&refs->lock);\n \trelease_packed_ref_cache(packed_ref_cache);\n-\tclear_packed_ref_cache(refs->packed_ref_store);\n+\tclear_packed_ref_cache(refs);\n }\n \n struct ref_to_prune {\n@@ -1657,7 +1656,7 @@ static int repack_without_refs(struct files_ref_store *refs,\n \t\t * All packed entries disappeared while we were\n \t\t * acquiring the lock.\n \t\t */\n-\t\trollback_packed_refs(refs);\n+\t\trollback_packed_refs(refs->packed_ref_store);\n \t\treturn 0;\n \t}\n \n-- \n2.11.0\n\n"},{"id":"322353","messageId":"f15a626f80ccddbdeceb01438e02d61dd8483275.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 02/28] packed_ref_store: new struct","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:07Z","receivedAt":"2017-06-15T14:49:03Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"Start extracting the packed-refs-related data structures into a new\nclass, `packed_ref_store`. It doesn't yet implement `ref_store`, but\nit will.\n\nSigned-off-by: Michael Haggerty \n---\n refs/files-backend.c | 42 +++++++++++++++++++++++++++++++++---------\n 1 file changed, 33 insertions(+), 9 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 87cecde231..2efb71cee9 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -47,6 +47,28 @@ struct packed_ref_cache {\n \tstruct stat_validity validity;\n };\n \n+/*\n+ * A container for `packed-refs`-related data. It is not (yet) a\n+ * `ref_store`.\n+ */\n+struct packed_ref_store {\n+\tunsigned int store_flags;\n+\n+\t/*\n+\t * A cache of the values read from the `packed-refs` file, if\n+\t * it might still be current; otherwise, NULL.\n+\t */\n+\tstruct packed_ref_cache *cache;\n+};\n+\n+static struct packed_ref_store *packed_ref_store_create(unsigned int store_flags)\n+{\n+\tstruct packed_ref_store *refs = xcalloc(1, sizeof(*refs));\n+\n+\trefs->store_flags = store_flags;\n+\treturn refs;\n+}\n+\n /*\n * Future: need to be in \"struct repository\"\n * when doing a full libification.\n@@ -60,13 +82,14 @@ struct files_ref_store {\n \tchar *packed_refs_path;\n \n \tstruct ref_cache *loose;\n-\tstruct packed_ref_cache *packed;\n \n \t/*\n \t * Lock used for the \"packed-refs\" file. Note that this (and\n \t * thus the enclosing `files_ref_store`) must not be freed.\n \t */\n \tstruct lock_file packed_refs_lock;\n+\n+\tstruct packed_ref_store *packed_ref_store;\n };\n \n /*\n@@ -95,12 +118,12 @@ static int release_packed_ref_cache(struct packed_ref_cache *packed_refs)\n \n static void clear_packed_ref_cache(struct files_ref_store *refs)\n {\n-\tif (refs->packed) {\n-\t\tstruct packed_ref_cache *packed_refs = refs->packed;\n+\tif (refs->packed_ref_store->cache) {\n+\t\tstruct packed_ref_cache *packed_refs = refs->packed_ref_store->cache;\n \n \t\tif (is_lock_file_locked(&refs->packed_refs_lock))\n \t\t\tdie(\"BUG: packed-ref cache cleared while locked\");\n-\t\trefs->packed = NULL;\n+\t\trefs->packed_ref_store->cache = NULL;\n \t\trelease_packed_ref_cache(packed_refs);\n \t}\n }\n@@ -132,6 +155,7 @@ static struct ref_store *files_ref_store_create(const char *gitdir,\n \trefs->gitcommondir = strbuf_detach(&sb, NULL);\n \tstrbuf_addf(&sb, \"%s/packed-refs\", refs->gitcommondir);\n \trefs->packed_refs_path = strbuf_detach(&sb, NULL);\n+\trefs->packed_ref_store = packed_ref_store_create(flags);\n \n \treturn ref_store;\n }\n@@ -375,8 +399,8 @@ static void files_ref_path(struct files_ref_store *refs,\n */\n static void validate_packed_ref_cache(struct files_ref_store *refs)\n {\n-\tif (refs->packed &&\n-\t !stat_validity_check(&refs->packed->validity,\n+\tif (refs->packed_ref_store->cache &&\n+\t !stat_validity_check(&refs->packed_ref_store->cache->validity,\n \t\t\t\t files_packed_refs_path(refs)))\n \t\tclear_packed_ref_cache(refs);\n }\n@@ -396,10 +420,10 @@ static struct packed_ref_cache *get_packed_ref_cache(struct files_ref_store *ref\n \tif (!is_lock_file_locked(&refs->packed_refs_lock))\n \t\tvalidate_packed_ref_cache(refs);\n \n-\tif (!refs->packed)\n-\t\trefs->packed = read_packed_refs(packed_refs_file);\n+\tif (!refs->packed_ref_store->cache)\n+\t\trefs->packed_ref_store->cache = read_packed_refs(packed_refs_file);\n \n-\treturn refs->packed;\n+\treturn refs->packed_ref_store->cache;\n }\n \n static struct ref_dir *get_packed_ref_dir(struct packed_ref_cache *packed_ref_cache)\n-- \n2.11.0\n\n"},{"id":"322354","messageId":"ee2a855dd49a88a8545fcb1654782a407ee4af8a.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 06/28] validate_packed_ref_cache(): take a `packed_ref_store *` parameter","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:11Z","receivedAt":"2017-06-15T14:49:04Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"It only cares about the packed-refs part of the reference store.\n\nSigned-off-by: Michael Haggerty \n---\n refs/files-backend.c | 13 ++++++-------\n 1 file changed, 6 insertions(+), 7 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 2b0db60b2b..f061506bf0 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -396,12 +396,11 @@ static void files_ref_path(struct files_ref_store *refs,\n * Check that the packed refs cache (if any) still reflects the\n * contents of the file. If not, clear the cache.\n */\n-static void validate_packed_ref_cache(struct files_ref_store *refs)\n+static void validate_packed_ref_cache(struct packed_ref_store *refs)\n {\n-\tif (refs->packed_ref_store->cache &&\n-\t !stat_validity_check(&refs->packed_ref_store->cache->validity,\n-\t\t\t\t refs->packed_ref_store->path))\n-\t\tclear_packed_ref_cache(refs->packed_ref_store);\n+\tif (refs->cache &&\n+\t !stat_validity_check(&refs->cache->validity, refs->path))\n+\t\tclear_packed_ref_cache(refs);\n }\n \n /*\n@@ -417,7 +416,7 @@ static struct packed_ref_cache *get_packed_ref_cache(struct files_ref_store *ref\n \tconst char *packed_refs_file = refs->packed_ref_store->path;\n \n \tif (!is_lock_file_locked(&refs->packed_ref_store->lock))\n-\t\tvalidate_packed_ref_cache(refs);\n+\t\tvalidate_packed_ref_cache(refs->packed_ref_store);\n \n \tif (!refs->packed_ref_store->cache)\n \t\trefs->packed_ref_store->cache = read_packed_refs(packed_refs_file);\n@@ -1364,7 +1363,7 @@ static int lock_packed_refs(struct files_ref_store *refs, int flags)\n \t * cache is still valid. We've just locked the file, but it\n \t * might have changed the moment *before* we locked it.\n \t */\n-\tvalidate_packed_ref_cache(refs);\n+\tvalidate_packed_ref_cache(refs->packed_ref_store);\n \n \tpacked_ref_cache = get_packed_ref_cache(refs);\n \t/* Increment the reference count to prevent it from being freed: */\n-- \n2.11.0\n\n"},{"id":"322355","messageId":"3d7e45d0f1b725f20d15835304a576f9a7629c0a.1497534157.git.mhagger@alum.mit.edu","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"[PATCH 05/28] clear_packed_ref_cache(): take a `packed_ref_store *` parameter","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-15T14:47:10Z","receivedAt":"2017-06-15T14:49:08Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"It only cares about the packed-refs part of the reference store.\n\nSigned-off-by: Michael Haggerty \n---\n refs/files-backend.c | 16 ++++++++--------\n 1 file changed, 8 insertions(+), 8 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex de8293493f..2b0db60b2b 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -120,15 +120,15 @@ static int release_packed_ref_cache(struct packed_ref_cache *packed_refs)\n \t}\n }\n \n-static void clear_packed_ref_cache(struct files_ref_store *refs)\n+static void clear_packed_ref_cache(struct packed_ref_store *refs)\n {\n-\tif (refs->packed_ref_store->cache) {\n-\t\tstruct packed_ref_cache *packed_refs = refs->packed_ref_store->cache;\n+\tif (refs->cache) {\n+\t\tstruct packed_ref_cache *cache = refs->cache;\n \n-\t\tif (is_lock_file_locked(&refs->packed_ref_store->lock))\n+\t\tif (is_lock_file_locked(&refs->lock))\n \t\t\tdie(\"BUG: packed-ref cache cleared while locked\");\n-\t\trefs->packed_ref_store->cache = NULL;\n-\t\trelease_packed_ref_cache(packed_refs);\n+\t\trefs->cache = NULL;\n+\t\trelease_packed_ref_cache(cache);\n \t}\n }\n \n@@ -401,7 +401,7 @@ static void validate_packed_ref_cache(struct files_ref_store *refs)\n \tif (refs->packed_ref_store->cache &&\n \t !stat_validity_check(&refs->packed_ref_store->cache->validity,\n \t\t\t\t refs->packed_ref_store->path))\n-\t\tclear_packed_ref_cache(refs);\n+\t\tclear_packed_ref_cache(refs->packed_ref_store);\n }\n \n /*\n@@ -1435,7 +1435,7 @@ static void rollback_packed_refs(struct files_ref_store *refs)\n \t\tdie(\"BUG: packed-refs not locked\");\n \trollback_lock_file(&refs->packed_ref_store->lock);\n \trelease_packed_ref_cache(packed_ref_cache);\n-\tclear_packed_ref_cache(refs);\n+\tclear_packed_ref_cache(refs->packed_ref_store);\n }\n \n struct ref_to_prune {\n-- \n2.11.0\n\n"},{"id":"322438","messageId":"CAGZ79kYOtGc5p2bU5-oWCq60N87v4CbcdYjAurzirjRsyVdEwQ@mail.gmail.com","threadId":"46189","inReplyTo":"2aa91c07ef7f0f9c679aab8898341520eda76016.1497534157.git.mhagger@alum.mit.edu","subject":"Re: [PATCH 03/28] packed_ref_store: move `packed_refs_path` here","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-16T05:30:00Z","receivedAt":"2017-06-16T05:30:06Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Jun 15, 2017 at 7:47 AM, Michael Haggerty wrote:\n> Move `packed_refs_path` from `files_ref_store` to `packed_ref_store`,\n> and rename it to `path` since its meaning is clear from its new\n> context.\n>\n> Inline `files_packed_refs_path()`.\n>\n> Signed-off-by: Michael Haggerty \n> ---\n> refs/files-backend.c | 25 ++++++++++++-------------\n> 1 file changed, 12 insertions(+), 13 deletions(-)\n>\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index 2efb71cee9..c4b8e2f63b 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -54,6 +54,9 @@ struct packed_ref_cache {\n> struct packed_ref_store {\n> unsigned int store_flags;\n>\n> + /* The path of the \"packed-refs\" file: */\n> + char *path;\n> +\n> /*\n> * A cache of the values read from the `packed-refs` file, if\n> * it might still be current; otherwise, NULL.\n> @@ -61,11 +64,13 @@ struct packed_ref_store {\n> struct packed_ref_cache *cache;\n> };\n>\n> -static struct packed_ref_store *packed_ref_store_create(unsigned int store_flags)\n> +static struct packed_ref_store *packed_ref_store_create(\n> + const char *path, unsigned int store_flags)\n> {\n> struct packed_ref_store *refs = xcalloc(1, sizeof(*refs));\n>\n> refs->store_flags = store_flags;\n> + refs->path = xstrdup(path);\n\nmental note: the destructor (if introduced later) has to\nfree the path.\n"},{"id":"322439","messageId":"CAGZ79ka3u_otP6M+oRBb4dQdMQx6kFHaXihLsBpM23gqzwqOGw@mail.gmail.com","threadId":"46189","inReplyTo":"eea8f3d25e9d3d3a0b63604338f10cfa658bbb43.1497534157.git.mhagger@alum.mit.edu","subject":"Re: [PATCH 04/28] packed_ref_store: move `packed_refs_lock` member here","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-16T05:39:08Z","receivedAt":"2017-06-16T05:39:28Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Jun 15, 2017 at 7:47 AM, Michael Haggerty wrote:\n> Move the `packed_refs_lock` member from `files_ref_store` to\n> `packed_ref_store`, and rename it to `lock` since it's now more\n> obvious what it is locking.\n>\n> Signed-off-by: Michael Haggerty \n> ---\n> refs/files-backend.c | 31 ++++++++++++++++---------------\n> 1 file changed, 16 insertions(+), 15 deletions(-)\n>\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index c4b8e2f63b..de8293493f 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -62,6 +62,12 @@ struct packed_ref_store {\n> * it might still be current; otherwise, NULL.\n> */\n> struct packed_ref_cache *cache;\n> +\n> + /*\n> + * Lock used for the \"packed-refs\" file. Note that this (and\n> + * thus the enclosing `packed_ref_store`) must not be freed.\n> + */\n> + struct lock_file lock;\n> };\n>\n> static struct packed_ref_store *packed_ref_store_create(\n> @@ -87,12 +93,6 @@ struct files_ref_store {\n>\n> struct ref_cache *loose;\n>\n> - /*\n> - * Lock used for the \"packed-refs\" file. Note that this (and\n> - * thus the enclosing `files_ref_store`) must not be freed.\n> - */\n> - struct lock_file packed_refs_lock;\n> -\n> struct packed_ref_store *packed_ref_store;\n> };\n>\n> @@ -125,7 +125,7 @@ static void clear_packed_ref_cache(struct files_ref_store *refs)\n> if (refs->packed_ref_store->cache) {\n> struct packed_ref_cache *packed_refs = refs->packed_ref_store->cache;\n>\n> - if (is_lock_file_locked(&refs->packed_refs_lock))\n> + if (is_lock_file_locked(&refs->packed_ref_store->lock))\n\nI sort of stumble over the name due to singular/plural/genetive issues:\nThe store contains multiple of \"packed ref\"(s) or the store is specialized in\npacked ref(s). On the other hand in English you seem to reference the\nspecialisation of a store in singular: It's a \"gun store\" (not \"guns store\")\nor \"furniture store\" or \"fresh fish store\" (though that could be\nplural as well).\n\nIt just sounded odd to me. (though think of it as a minor nit)\n"},{"id":"322440","messageId":"CAGZ79kbeJk5PmJVdLva-VauLPa4MbQdNRWfQmyHtVqVa3JDL+A@mail.gmail.com","threadId":"46189","inReplyTo":"f9c4c47e52c076c42896d22d89591a93163afda3.1497534157.git.mhagger@alum.mit.edu","subject":"Re: [PATCH 15/28] packed_peel_ref(): new function, extracted from `files_peel_ref()`","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-16T05:42:32Z","receivedAt":"2017-06-16T05:42:37Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Jun 15, 2017 at 7:47 AM, Michael Haggerty wrote:\n> This will later become a method of `packed_ref_store`.\n\nAlso while touching it, maybe rename sha1 to object_hash\n(not saying object_id as that would be confusing with the actual\noid struct), maybe?\n"},{"id":"322441","messageId":"CAGZ79kYFm8vZ9-qzM8sNsF_xgywvdcdZBAAJw3QJfQqBRr+OZQ@mail.gmail.com","threadId":"46189","inReplyTo":"831f8a3f633374f173f630d9c65d870fffd0896e.1497534157.git.mhagger@alum.mit.edu","subject":"Re: [PATCH 18/28] packed-backend: new module for handling packed references","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-16T05:48:24Z","receivedAt":"2017-06-16T05:48:29Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Jun 15, 2017 at 7:47 AM, Michael Haggerty wrote:\n> Now that the interface between `files_ref_store` and\n> `packed_ref_store` is relatively narrow, move the latter into a new\n> module, \"refs/packed-backend.h\" and \"refs/packed-backend.c\". It still\n> doesn't quite implement the `ref_store` interface, but it will soon.\n>\n> This commit moves code around and adjusts its visibility, but doesn't\n> change anything.\n\nI did not look into it in detail (as the origin/sb/diff-color-move\nseries ought to help here, but I am in the mid of rewriting that again\nto have a better design; also on my laptop, such that I'd have install\nthat series).\n\nI think this commit represents another common case of\ncode movement, where you would not just move around code, but\nalso add/remove some keywords (e.g. the static word in C, or a class\nname changes in Java/C++)\n\nMaybe we would want to not just color up moved blocks but show\nthe adjacant lines of moved blocks in a word diff (such that the\nclass name change would be highlighted) in a comment or such.\n"},{"id":"322445","messageId":"bf1fcee3-2f2f-b153-5774-dee5b2a2790d@alum.mit.edu","threadId":"46189","inReplyTo":"CAGZ79ka3u_otP6M+oRBb4dQdMQx6kFHaXihLsBpM23gqzwqOGw@mail.gmail.com","subject":"Re: [PATCH 04/28] packed_ref_store: move `packed_refs_lock` member here","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-16T06:43:09Z","receivedAt":"2017-06-16T06:43:33Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 06/16/2017 07:39 AM, Stefan Beller wrote:\n> On Thu, Jun 15, 2017 at 7:47 AM, Michael Haggerty wrote:\n>> [...]\n>> @@ -125,7 +125,7 @@ static void clear_packed_ref_cache(struct files_ref_store *refs)\n>> if (refs->packed_ref_store->cache) {\n>> struct packed_ref_cache *packed_refs = refs->packed_ref_store->cache;\n>>\n>> - if (is_lock_file_locked(&refs->packed_refs_lock))\n>> + if (is_lock_file_locked(&refs->packed_ref_store->lock))\n> \n> I sort of stumble over the name due to singular/plural/genetive issues:\n> The store contains multiple of \"packed ref\"(s) or the store is specialized in\n> packed ref(s). On the other hand in English you seem to reference the\n> specialisation of a store in singular: It's a \"gun store\" (not \"guns store\")\n> or \"furniture store\" or \"fresh fish store\" (though that could be\n> plural as well).\n> \n> It just sounded odd to me. (though think of it as a minor nit)\n\nI chose that name because it is a `ref_store`, with `packed_` being a\nshort prefix that tells what kind of `ref_store`.\n\nThe next question is, why `ref_store` as opposed to `refs_store`? To me\nit sounds more natural in English for the reasons that you mentioned.\n\nMichael\n\n"},{"id":"322446","messageId":"db616bc6-4d31-6f7c-4862-9161f4bb36ce@alum.mit.edu","threadId":"46189","inReplyTo":"CAGZ79kbeJk5PmJVdLva-VauLPa4MbQdNRWfQmyHtVqVa3JDL+A@mail.gmail.com","subject":"Re: [PATCH 15/28] packed_peel_ref(): new function, extracted from `files_peel_ref()`","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-16T06:46:34Z","receivedAt":"2017-06-16T06:46:44Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 06/16/2017 07:42 AM, Stefan Beller wrote:\n> On Thu, Jun 15, 2017 at 7:47 AM, Michael Haggerty wrote:\n>> This will later become a method of `packed_ref_store`.\n> \n> Also while touching it, maybe rename sha1 to object_hash\n> (not saying object_id as that would be confusing with the actual\n> oid struct), maybe?\n\nHmmm, my impression was that most of the `unsigned char *` hashes are\nstill called sha1, and they are renamed to `oid` at the moment that they\nare converted to `struct object_id *`. I only see two instances of the\nstring \"object_hash\" in the code:\n\n$ git grep object_hash\nobject.c:static void grow_object_hash(void)\nobject.c: grow_object_hash();\n\nMichael\n\n"},{"id":"322463","messageId":"CAGZ79kbavn0QdqaoJJ+DXh28OLb1jDgdqA01soawQZ86fp0XkA@mail.gmail.com","threadId":"46189","inReplyTo":"bf1fcee3-2f2f-b153-5774-dee5b2a2790d@alum.mit.edu","subject":"Re: [PATCH 04/28] packed_ref_store: move `packed_refs_lock` member here","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-16T15:44:56Z","receivedAt":"2017-06-16T15:45:02Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Jun 15, 2017 at 11:43 PM, Michael Haggerty wrote:\n> I chose that name because it is a `ref_store`, with `packed_` being a\n> short prefix that tells what kind of `ref_store`.\n>\n> The next question is, why `ref_store` as opposed to `refs_store`? To me\n> it sounds more natural in English for the reasons that you mentioned.\n\nYeah after some thought on language this sounds good to me, too.\nI still have to work on natural. ;)\n"},{"id":"322464","messageId":"CAGZ79kb=eq88VR5nQggnLQJX-gTZ37uhm3PhuhC4oX8dw8vG5w@mail.gmail.com","threadId":"46189","inReplyTo":"db616bc6-4d31-6f7c-4862-9161f4bb36ce@alum.mit.edu","subject":"Re: [PATCH 15/28] packed_peel_ref(): new function, extracted from `files_peel_ref()`","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-16T15:47:47Z","receivedAt":"2017-06-16T15:47:52Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Jun 15, 2017 at 11:46 PM, Michael Haggerty wrote:\n> On 06/16/2017 07:42 AM, Stefan Beller wrote:\n>> On Thu, Jun 15, 2017 at 7:47 AM, Michael Haggerty wrote:\n>>> This will later become a method of `packed_ref_store`.\n>>\n>> Also while touching it, maybe rename sha1 to object_hash\n>> (not saying object_id as that would be confusing with the actual\n>> oid struct), maybe?\n>\n> Hmmm, my impression was that most of the `unsigned char *` hashes are\n> still called sha1, and they are renamed to `oid` at the moment that they\n> are converted to `struct object_id *`. I only see two instances of the\n> string \"object_hash\" in the code:\n>\n> $ git grep object_hash\n> object.c:static void grow_object_hash(void)\n> object.c: grow_object_hash();\n>\n> Michael\n\nYeah maybe we can just defer it to the proper conversion series.\nI just jumped the gun here.\n"},{"id":"322596","messageId":"xmqqzid3stdw.fsf@gitster.mtv.corp.google.com","threadId":"46189","inReplyTo":"cover.1497534157.git.mhagger@alum.mit.edu","subject":"Re: [PATCH 00/28] Create a reference backend for packed refs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-06-19T18:51:39Z","receivedAt":"2017-06-19T18:51:57Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Michael Haggerty writes:\n\n> I've developed these patches on top of master plus the following\n> patches, which are followups to mh/packed-refs-store-prep:\n>\n> * lock_packed_refs(): fix cache validity check\n> * for_each_bisect_ref(): don't trim refnames\n>\n> The patches can also be obtained from my GitHub fork [2] as branch\n> \"packed-ref-store\".\n>\n> Michael\n>\n> [1] http://public-inbox.org/git/cover.1490026594.git.mhagger@alum.mit.edu/\n> [2] https://github.com/mhagger/git\n\nThanks.\n\nBoth in the version I queued and a fresh fetching of c13b2fad\n(\"read_packed_refs(): die if `packed-refs` contains bogus data\",\n2017-06-12) from your https://github.com/mhagger/git repository\nseems to exhibit an annoying error message in my local repository I\nuse for the primary work:\n\n $ git fetch https://github.com/mhagger/git packed-ref-store\n $ git checkout FETCH_HEAD\n $ make\n $ ./git describe next\n error: refs/notes/amlog does not point to a valid object!\n v2.13.1-611-g7e3b11ae1b\n $ grep refs/notes/amlog .git/packed-refs\n ed07e83cff8e407464fb2f5e84bd311da9c87565 refs/notes/amlog\n $ git rev-parse refs/notes/amlog\n b3079212325398e406078585c785c892d6e572f0\n $ git cat-file -t ed07e83cff8e407464fb2f5e84bd311da9c87565\n fatal: git cat-file: could not get object info\n $ git cat-file -t b3079212325398e406078585c785c892d6e572f0\n commit\n\nIs the iterator over packed-refs correctly skipping over what are\ncovered by loose refs? The entries in the packed-refs file that are\nsuperseded by loose refs should be allowed to point at an already\nexpired object.\n"},{"id":"322599","messageId":"xmqqvanrsru4.fsf@gitster.mtv.corp.google.com","threadId":"46189","inReplyTo":"xmqqzid3stdw.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 00/28] Create a reference backend for packed refs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-06-19T19:25:07Z","receivedAt":"2017-06-19T19:25:16Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> Is the iterator over packed-refs correctly skipping over what are\n> covered by loose refs? The entries in the packed-refs file that are\n> superseded by loose refs should be allowed to point at an already\n> expired object.\n\nHere it is in a test form for easier diagnosis.\n\n t/t1408-packed-refs.sh | 46 ++++++++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 46 insertions(+)\n\ndiff --git a/t/t1408-packed-refs.sh b/t/t1408-packed-refs.sh\nnew file mode 100755\nindex 0000000000..35533c8593\n--- /dev/null\n+++ b/t/t1408-packed-refs.sh\n@@ -0,0 +1,46 @@\n+#!/bin/sh\n+\n+test_description='packed-refs entries are covered by loose refs'\n+\n+. ./test-lib.sh\n+\n+test_expect_success setup '\n+\ttest_tick &&\n+\tgit commit --allow-empty -m one &&\n+\tone=$(git rev-parse HEAD) && \n+\tgit for-each-ref >actual &&\n+\techo \"$one commit\trefs/heads/master\" >expect &&\n+\ttest_cmp expect actual &&\n+\n+\tgit pack-refs --all &&\n+\tgit for-each-ref >actual &&\n+\techo \"$one commit\trefs/heads/master\" >expect &&\n+\ttest_cmp expect actual &&\n+\n+\tcat .git/packed-refs &&\n+\n+\tgit checkout --orphan another &&\n+\ttest_tick &&\n+\tgit commit --allow-empty -m two &&\n+\ttwo=$(git rev-parse HEAD) && \n+\tgit checkout -B master &&\n+\tgit branch -D another &&\n+\n+\tcat .git/packed-refs &&\n+\n+\tgit for-each-ref >actual &&\n+\techo \"$two commit\trefs/heads/master\" >expect &&\n+\ttest_cmp expect actual &&\n+\n+\tgit reflog expire --expire=now --all &&\n+\tgit prune &&\n+\tgit tag -m v1.0 v1.0 master\n+'\n+\n+test_expect_success 'no error from stale entry in packed-refs' '\n+\tgit describe master >actual 2>&1 &&\n+\techo \"v1.0\" >expect &&\n+\ttest_cmp expect actual\n+'\n+\n+test_done\n"},{"id":"322601","messageId":"20170619194315.tzdmo7xel2dztoju@sigill.intra.peff.net","threadId":"46189","inReplyTo":"xmqqvanrsru4.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 00/28] Create a reference backend for packed refs","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2017-06-19T19:43:15Z","receivedAt":"2017-06-19T19:43:29Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jun 19, 2017 at 12:25:07PM -0700, Junio C Hamano wrote:\n\n> Junio C Hamano writes:\n> \n> > Is the iterator over packed-refs correctly skipping over what are\n> > covered by loose refs? The entries in the packed-refs file that are\n> > superseded by loose refs should be allowed to point at an already\n> > expired object.\n> \n> Here it is in a test form for easier diagnosis.\n\nThanks, I was just starting to do that myself. The problem is in\nca6b06eb7 (packed_ref_store: support iteration, 2017-05-15) and is\npretty obvious: the packed_ref iterator checks whether the entry\nresolves.\n\nI think that _neither_ of the loose and packed iterators should be\nchecking this. It's only the merged result (where loose trumps packed)\nthat should bother checking.\n\n-Peff\n"},{"id":"322604","messageId":"20170619195330.fhjlfiqmcwhgttaa@sigill.intra.peff.net","threadId":"46189","inReplyTo":"20170619194315.tzdmo7xel2dztoju@sigill.intra.peff.net","subject":"Re: [PATCH 00/28] Create a reference backend for packed refs","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2017-06-19T19:53:30Z","receivedAt":"2017-06-19T19:53:37Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jun 19, 2017 at 03:43:15PM -0400, Jeff King wrote:\n\n> > > Is the iterator over packed-refs correctly skipping over what are\n> > > covered by loose refs? The entries in the packed-refs file that are\n> > > superseded by loose refs should be allowed to point at an already\n> > > expired object.\n> > \n> > Here it is in a test form for easier diagnosis.\n> \n> Thanks, I was just starting to do that myself. The problem is in\n> ca6b06eb7 (packed_ref_store: support iteration, 2017-05-15) and is\n> pretty obvious: the packed_ref iterator checks whether the entry\n> resolves.\n> \n> I think that _neither_ of the loose and packed iterators should be\n> checking this. It's only the merged result (where loose trumps packed)\n> that should bother checking.\n\nIt looks like this is mostly already the case. files_ref_iterator\ncombines the two and does check. The loose iteration is done by\ncache_ref_iterator[1], and does not. So it's just this new packed_refs\niterator that is wrong, and we just need:\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 311fd014c..ad1143e64 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -416,12 +416,6 @@ static int packed_ref_iterator_advance(struct ref_iterator *ref_iterator)\n \t\t ref_type(iter->iter0->refname) != REF_TYPE_PER_WORKTREE)\n \t\t\tcontinue;\n \n-\t\tif (!(iter->flags & DO_FOR_EACH_INCLUDE_BROKEN) &&\n-\t\t !ref_resolves_to_object(iter->iter0->refname,\n-\t\t\t\t\t iter->iter0->oid,\n-\t\t\t\t\t iter->iter0->flags))\n-\t\t\tcontinue;\n-\n \t\titer->base.refname = iter->iter0->refname;\n \t\titer->base.oid = iter->iter0->oid;\n \t\titer->base.flags = iter->iter0->flags;\n@@ -473,8 +467,6 @@ static struct ref_iterator *packed_ref_iterator_begin(\n \tstruct ref_iterator *ref_iterator;\n \tunsigned int required_flags = REF_STORE_READ;\n \n-\tif (!(flags & DO_FOR_EACH_INCLUDE_BROKEN))\n-\t\trequired_flags |= REF_STORE_ODB;\n \trefs = packed_downcast(ref_store, required_flags, \"ref_iterator_begin\");\n \n \titer = xcalloc(1, sizeof(*iter));\n\nAt least that makes sense to me and passes the tests (including the new\none). I haven't actually reviewed the patches yet.\n\n-Peff\n"},{"id":"322916","messageId":"05ea33a0-adde-54df-07e8-bdfba9c7d503@alum.mit.edu","threadId":"46189","inReplyTo":"20170619195330.fhjlfiqmcwhgttaa@sigill.intra.peff.net","subject":"Re: [PATCH 00/28] Create a reference backend for packed refs","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-06-22T07:36:20Z","receivedAt":"2017-06-22T07:36:34Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 06/19/2017 09:53 PM, Jeff King wrote:\n> On Mon, Jun 19, 2017 at 03:43:15PM -0400, Jeff King wrote:\n> \n>>>> Is the iterator over packed-refs correctly skipping over what are\n>>>> covered by loose refs? The entries in the packed-refs file that are\n>>>> superseded by loose refs should be allowed to point at an already\n>>>> expired object.\n>>>\n>>> Here it is in a test form for easier diagnosis.\n>>\n>> Thanks, I was just starting to do that myself. The problem is in\n>> ca6b06eb7 (packed_ref_store: support iteration, 2017-05-15) and is\n>> pretty obvious: the packed_ref iterator checks whether the entry\n>> resolves.\n>>\n>> I think that _neither_ of the loose and packed iterators should be\n>> checking this. It's only the merged result (where loose trumps packed)\n>> that should bother checking.\n\nThanks for the bug report and the analysis, which is exactly right.\n\nBut I'd like to fix the problem a *little* differently than Peff\nsuggested. To keep `packed_ref_store` from deviating more than necessary\nfrom the `ref_store` interface, I propose that we leave the code for\nrejecting broken refs where it is, and instead invoke\n`packed_ref_iterator_begin()` with the `DO_FOR_EACH_INCLUDE_BROKEN` flag.\n\nI have prepared a re-roll of the patch series, but I can't submit it\nuntil I have Junio's signoff on the test that he suggested [1]. Junio?\n\nThanks,\nMichael\n\n[1]\nhttp://public-inbox.org/git/xmqqvanrsru4.fsf@gitster.mtv.corp.google.com/\n\n"},{"id":"322933","messageId":"xmqqwp84kpfo.fsf@gitster.mtv.corp.google.com","threadId":"46189","inReplyTo":"05ea33a0-adde-54df-07e8-bdfba9c7d503@alum.mit.edu","subject":"Re: [PATCH 00/28] Create a reference backend for packed refs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-06-22T15:33:15Z","receivedAt":"2017-06-22T15:33:23Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Michael Haggerty writes:\n\n> I have prepared a re-roll of the patch series, but I can't submit it\n> until I have Junio's signoff on the test that he suggested [1]. Junio?\n\nSure, and thanks.\n"},{"id":"324849","messageId":"xmqqy3riiu0f.fsf@gitster.mtv.corp.google.com","threadId":"46189","inReplyTo":"2779ee9a75e75ff80898e51e46d561b8074e0dc2.1497534157.git.mhagger@alum.mit.edu","subject":"Re: [PATCH 21/28] commit_packed_refs(): use a staging file separate from the lockfile","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-07-20T23:11:12Z","receivedAt":"2017-07-20T23:11:20Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Michael Haggerty writes:\n\n> We will want to be able to hold the lockfile for `packed-refs` even\n> after we have activated the new values. So use a separate tempfile,\n> `packed-refs.new`, as a place to stage the new contents of the\n> `packed-refs` file. For now this is all done within\n> `commit_packed_refs()`, but that will change shortly.\n>\n> Signed-off-by: Michael Haggerty \n> ---\n\nThe layout created by \"contrib/workdir/git-new-workdir\" will be\nbroken by this line of change. \"git worktree\" is supposed to know\nthat refs/packed-refs is a shared thing and lives in common-dir,\nso it shouldn't be affected.\n\nDo we care about the ancient layout that used symlinks to emulate\nthe more modern worktree one?\n"},{"id":"324851","messageId":"20170720233051.GB159617@aiede.mtv.corp.google.com","threadId":"46189","inReplyTo":"xmqqy3riiu0f.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 21/28] commit_packed_refs(): use a staging file separate from the lockfile","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2017-07-20T23:30:51Z","receivedAt":"2017-07-20T23:31:00Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Junio C Hamano wrote:\n> Michael Haggerty writes:\n\n>> We will want to be able to hold the lockfile for `packed-refs` even\n>> after we have activated the new values. So use a separate tempfile,\n>> `packed-refs.new`, as a place to stage the new contents of the\n>> `packed-refs` file. For now this is all done within\n>> `commit_packed_refs()`, but that will change shortly.\n>>\n>> Signed-off-by: Michael Haggerty \n>> ---\n>\n> The layout created by \"contrib/workdir/git-new-workdir\" will be\n> broken by this line of change. \"git worktree\" is supposed to know\n> that refs/packed-refs is a shared thing and lives in common-dir,\n> so it shouldn't be affected.\n>\n> Do we care about the ancient layout that used symlinks to emulate\n> the more modern worktree one?\n\nI think we do care. In the context of people's changing workflows,\n\"git worktree\" is a relatively new tool. Breaking the older\ngit-new-workdir (and tools that emulate it) would affect a large\nnumber of users that don't necessarily know how to clean up the\nresult.\n\nThanks,\nJonathan\n"},{"id":"324854","messageId":"xmqqpocuib06.fsf@gitster.mtv.corp.google.com","threadId":"46189","inReplyTo":"20170720233051.GB159617@aiede.mtv.corp.google.com","subject":"Re: [PATCH 21/28] commit_packed_refs(): use a staging file separate from the lockfile","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-07-21T06:01:45Z","receivedAt":"2017-07-21T06:01:55Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Nieder writes:\n\n>> Do we care about the ancient layout that used symlinks to emulate\n>> the more modern worktree one?\n>\n> I think we do care. In the context of people's changing workflows,\n> \"git worktree\" is a relatively new tool. Breaking the older\n> git-new-workdir (and tools that emulate it) would affect a large\n> number of users that don't necessarily know how to clean up the\n> result.\n\nFair enough. Even though the git-new-workdir is on its way out, and\nwe really do want to do certain updates under lock, we cannot just\nallow the topic to graduate to 'master' in its current form without\nsome transition plan or mitigation strategy.\n\nThe general strategy we take for these atomic update of a file\nentity at $path is to:\n\n (1) try to create $path.lock exclusively; if we cannot, somebody\n else holds the lock so fail (or backoff and retry, which\n amounts to the same thing in the larger picture).\n\n (2) update $path.lock and close the fd.\n\n (3) rename $path.lock to $path atomically to unlock.\n\nWould it be sufficent to tweak the above procedure with a new,\nzeroth step?\n\n (0) see $path is a symlink; if so, readlink it and assign the\n result to $path. Then go on to step (1) above.\n\nI do not think such an enhancement would break atomicity guarantee\nwe want from the locking. When updating .git/packed-refs in an\nancient new-workdir'ed directory, we would end up locking the\ncorresponding file in the real repository, which is exactly what we\nwant anyway. As the basic assumption of having a symbolic link in\nthe new-workdir'ed directory is that a symlink can stay the same\nforever while the linked target will be updated, I think this would\nbe a reasonable short-term \"fix\".\n\nIt would be ideal if we can do this at somewhat a low level, i.e. in\nthe lockfile subsystem.\n\nThoughts? An even easier way out may be to just make pack-refs a\nno-op when .git/refs/ is a symbolic link, but as a solution that is\nfar from satisfactory, while \"locking the right file\" smells right,\nat least to me.\n\n"},{"id":"324855","messageId":"xmqqlgnii7mj.fsf@gitster.mtv.corp.google.com","threadId":"46189","inReplyTo":"xmqqpocuib06.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 21/28] commit_packed_refs(): use a staging file separate from the lockfile","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-07-21T07:14:44Z","receivedAt":"2017-07-21T07:14:52Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> The general strategy we take for these atomic update of a file\n> entity at $path is to:\n>\n> (1) try to create $path.lock exclusively; if we cannot, somebody\n> else holds the lock so fail (or backoff and retry, which\n> amounts to the same thing in the larger picture).\n>\n> (2) update $path.lock and close the fd.\n>\n> (3) rename $path.lock to $path atomically to unlock.\n>\n> Would it be sufficent to tweak the above procedure with a new,\n> zeroth step?\n>\n> (0) see $path is a symlink; if so, readlink it and assign the\n> result to $path. Then go on to step (1) above.\n>\n> I do not think such an enhancement would break atomicity guarantee\n> we want from the locking. When updating .git/packed-refs in an\n> ancient new-workdir'ed directory, we would end up locking the\n> corresponding file in the real repository, which is exactly what we\n> want anyway. As the basic assumption of having a symbolic link in\n> the new-workdir'ed directory is that a symlink can stay the same\n> forever while the linked target will be updated, I think this would\n> be a reasonable short-term \"fix\".\n>\n> It would be ideal if we can do this at somewhat a low level, i.e. in\n> the lockfile subsystem.\n\nOne thing I forgot to mention. For the above to work safely, we\nshould think through the possible interaction this may have with the\ncore.preferSymlinkRefs configuration. If the above is implemented\nnaively, an update of a symref (e.g. \"HEAD\") that points at\nsomething to point at something else would end up taking a lock on\nthe underlying ref and updating it, which is not what we want at\nall.\n\nPerhaps it is about time we stopped supporting the configuration.\nWe may be able to come up with a solution while keeping it alive,\nbut dropping it would mean we would have one less thing we need to\nworry about.\n\n\n\n"},{"id":"324954","messageId":"20170724165907.53zx657qmizgl4lb@sigill.intra.peff.net","threadId":"46189","inReplyTo":"xmqqpocuib06.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 21/28] commit_packed_refs(): use a staging file separate from the lockfile","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2017-07-24T16:59:07Z","receivedAt":"2017-07-24T16:59:13Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Jul 20, 2017 at 11:01:45PM -0700, Junio C Hamano wrote:\n\n> The general strategy we take for these atomic update of a file\n> entity at $path is to:\n> \n> (1) try to create $path.lock exclusively; if we cannot, somebody\n> else holds the lock so fail (or backoff and retry, which\n> amounts to the same thing in the larger picture).\n> \n> (2) update $path.lock and close the fd.\n> \n> (3) rename $path.lock to $path atomically to unlock.\n> \n> Would it be sufficent to tweak the above procedure with a new,\n> zeroth step?\n> \n> (0) see $path is a symlink; if so, readlink it and assign the\n> result to $path. Then go on to step (1) above.\n> \n> I do not think such an enhancement would break atomicity guarantee\n> we want from the locking. When updating .git/packed-refs in an\n> ancient new-workdir'ed directory, we would end up locking the\n> corresponding file in the real repository, which is exactly what we\n> want anyway. As the basic assumption of having a symbolic link in\n> the new-workdir'ed directory is that a symlink can stay the same\n> forever while the linked target will be updated, I think this would\n> be a reasonable short-term \"fix\".\n\nThis seems like the correct path to me. If the existing behavior is to\nlock the referring symref, that seems like a violation of the lock\nprocedure in the first place. Because if \"A\" points to \"B\", we take\n\"A.lock\" and then modify \"B\". But \"B\" may have any number of \"A\" links\npointing to it, eliminating the purpose of the lock.\n\nI thought we already did this, though. And that modifying HEAD (which\nmight be a symlink) required LOCK_NO_DEREF.\n\n-Peff\n"},{"id":"324957","messageId":"20170724170915.GD13924@aiede.mtv.corp.google.com","threadId":"46189","inReplyTo":"20170724165907.53zx657qmizgl4lb@sigill.intra.peff.net","subject":"Re: [PATCH 21/28] commit_packed_refs(): use a staging file separate from the lockfile","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2017-07-24T17:09:15Z","receivedAt":"2017-07-24T17:09:23Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nJeff King wrote:\n\n> This seems like the correct path to me. If the existing behavior is to\n> lock the referring symref, that seems like a violation of the lock\n> procedure in the first place. Because if \"A\" points to \"B\", we take\n> \"A.lock\" and then modify \"B\". But \"B\" may have any number of \"A\" links\n> pointing to it, eliminating the purpose of the lock.\n>\n> I thought we already did this, though. And that modifying HEAD (which\n> might be a symlink) required LOCK_NO_DEREF.\n\nYes, I believe the lockfile API already does so. Since this patch\ncreates a \".new\" file, not using the lockfile API, it doesn't benefit\nfrom that, though.\n\nThanks,\nJonathan\n"},{"id":"324958","messageId":"20170724171046.rcq2vqqzikto4lbi@sigill.intra.peff.net","threadId":"46189","inReplyTo":"20170724170915.GD13924@aiede.mtv.corp.google.com","subject":"Re: [PATCH 21/28] commit_packed_refs(): use a staging file separate from the lockfile","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2017-07-24T17:10:47Z","receivedAt":"2017-07-24T17:10:53Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jul 24, 2017 at 10:09:15AM -0700, Jonathan Nieder wrote:\n\n> Jeff King wrote:\n> \n> > This seems like the correct path to me. If the existing behavior is to\n> > lock the referring symref, that seems like a violation of the lock\n> > procedure in the first place. Because if \"A\" points to \"B\", we take\n> > \"A.lock\" and then modify \"B\". But \"B\" may have any number of \"A\" links\n> > pointing to it, eliminating the purpose of the lock.\n> >\n> > I thought we already did this, though. And that modifying HEAD (which\n> > might be a symlink) required LOCK_NO_DEREF.\n> \n> Yes, I believe the lockfile API already does so. Since this patch\n> creates a \".new\" file, not using the lockfile API, it doesn't benefit\n> from that, though.\n\nAh, I see. This bug makes much more sense, then. And I agree doubly that\nmatching the lock-code's behavior is the right thing to do.\n\n-Peff\n"},{"id":"325006","messageId":"xmqqzibtbje5.fsf@gitster.mtv.corp.google.com","threadId":"46189","inReplyTo":"20170724171046.rcq2vqqzikto4lbi@sigill.intra.peff.net","subject":"Re: [PATCH 21/28] commit_packed_refs(): use a staging file separate from the lockfile","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-07-24T21:43:46Z","receivedAt":"2017-07-24T21:43:59Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Mon, Jul 24, 2017 at 10:09:15AM -0700, Jonathan Nieder wrote:\n>\n>> Jeff King wrote:\n>> \n>> > This seems like the correct path to me. If the existing behavior is to\n>> > lock the referring symref, that seems like a violation of the lock\n>> > procedure in the first place. Because if \"A\" points to \"B\", we take\n>> > \"A.lock\" and then modify \"B\". But \"B\" may have any number of \"A\" links\n>> > pointing to it, eliminating the purpose of the lock.\n>> >\n>> > I thought we already did this, though. And that modifying HEAD (which\n>> > might be a symlink) required LOCK_NO_DEREF.\n>> \n>> Yes, I believe the lockfile API already does so. Since this patch\n>> creates a \".new\" file, not using the lockfile API, it doesn't benefit\n>> from that, though.\n>\n> Ah, I see. This bug makes much more sense, then. And I agree doubly that\n> matching the lock-code's behavior is the right thing to do.\n\nOK. Anybody wants to take a crack at it (it is of lower priority to\nme during the -rc freeze to deal with problems in topics on 'next'\nor higher)?\n\nThanks.\n"},{"id":"325170","messageId":"CAMy9T_Hj5sYwwoA4mBZFbUmriAzSaW4VWYwB3F-3DQ-6UpBU3A@mail.gmail.com","threadId":"46189","inReplyTo":"xmqqzibtbje5.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 21/28] commit_packed_refs(): use a staging file separate from the lockfile","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2017-07-26T23:49:33Z","receivedAt":"2017-07-26T23:49:40Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"I just submitted a patch [1] that fixes the `packed-refs`-is-a-symlink\nproblem in combination with git-new-workdir.\n\nI haven't considered any possible problems related to\n`core.preferSymlinkRefs`. That behavior should be unaffected by the\npacked-ref-store patch series and I'm not very interested in working\non it.\n\nMichael\n\n[1] https://public-inbox.org/git/d0da02a8b6f0272fa70ae3b1dc80fee6c6ee8d18.1501111803.git.mhagger@alum.mit.edu/\n\nOn Mon, Jul 24, 2017 at 2:43 PM, Junio C Hamano wrote:\n> Jeff King writes:\n>\n>> On Mon, Jul 24, 2017 at 10:09:15AM -0700, Jonathan Nieder wrote:\n>>\n>>> Jeff King wrote:\n>>>\n>>> > This seems like the correct path to me. If the existing behavior is to\n>>> > lock the referring symref, that seems like a violation of the lock\n>>> > procedure in the first place. Because if \"A\" points to \"B\", we take\n>>> > \"A.lock\" and then modify \"B\". But \"B\" may have any number of \"A\" links\n>>> > pointing to it, eliminating the purpose of the lock.\n>>> >\n>>> > I thought we already did this, though. And that modifying HEAD (which\n>>> > might be a symlink) required LOCK_NO_DEREF.\n>>>\n>>> Yes, I believe the lockfile API already does so. Since this patch\n>>> creates a \".new\" file, not using the lockfile API, it doesn't benefit\n>>> from that, though.\n>>\n>> Ah, I see. This bug makes much more sense, then. And I agree doubly that\n>> matching the lock-code's behavior is the right thing to do.\n>\n> OK. Anybody wants to take a crack at it (it is of lower priority to\n> me during the -rc freeze to deal with problems in topics on 'next'\n> or higher)?\n>\n> Thanks.\n"}]}