{"thread":{"id":"36972","subject":"[PATCH v20 00/48] Use ref transactions","startedAt":"2014-06-20T14:42:41Z","lastAt":"2014-08-22T12:41:05Z","messageCount":99,"participants":["Ronnie Sahlberg","Karsten Blees","Michael Haggerty","Junio C Hamano","Jeff King","Jonathan Nieder"],"isPatch":true,"patchVersion":20,"patchTotal":48},"messages":[{"id":"244755","messageId":"1403275409-28173-1-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":null,"subject":"[PATCH v20 00/48] Use ref transactions","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:42:41Z","receivedAt":"2014-06-20T14:42:41Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"This patch series can also be found at\nhttps://github.com/rsahlberg/git/tree/ref-transactions\n\nThis patch series is based on current master and expands on the transaction\nAPI. It converts all ref updates, inside refs.c as well as external, to use the\ntransaction API for updates. This makes most of the ref updates to become\natomic when there are failures locking or writing to a ref.\n\nThis version completes the work to convert all ref updates to use transactions.\nNow that all updates are through transactions I will start working on\ncleaning up the reading of refs and to create an api for managing reflogs but\nall that will go in a different patch series.\n\nVersion 20:\n - Whitespace and style changes suggested by Jun.\n\n\nRonnie Sahlberg (48):\n refs.c: remove ref_transaction_rollback\n refs.c: ref_transaction_commit should not free the transaction\n refs.c: constify the sha arguments for\n ref_transaction_create|delete|update\n refs.c: allow passing NULL to ref_transaction_free\n refs.c: add a strbuf argument to ref_transaction_commit for error\n logging\n lockfile.c: add a new public function unable_to_lock_message\n lockfile.c: make lock_file return a meaningful errno on failurei\n refs.c: add an err argument to repack_without_refs\n refs.c: make sure log_ref_setup returns a meaningful errno\n refs.c: verify_lock should set errno to something meaningful\n refs.c: make remove_empty_directories always set errno to something\n sane\n refs.c: commit_packed_refs to return a meaningful errno on failure\n refs.c: make resolve_ref_unsafe set errno to something meaningful on\n error\n refs.c: log_ref_write should try to return meaningful errno\n refs.c: make ref_update_reject_duplicates take a strbuf argument for\n errors\n refs.c: make update_ref_write update a strbuf on failure\n update-ref: use err argument to get error from ref_transaction_commit\n refs.c: remove the onerr argument to ref_transaction_commit\n refs.c: change ref_transaction_update() to do error checking and\n return status\n refs.c: change ref_transaction_create to do error checking and return\n status\n refs.c: update ref_transaction_delete to check for error and return\n status\n refs.c: make ref_transaction_begin take an err argument\n refs.c: add transaction.status and track OPEN/CLOSED/ERROR\n tag.c: use ref transactions when doing updates\n replace.c: use the ref transaction functions for updates\n commit.c: use ref transactions for updates\n sequencer.c: use ref transactions for all ref updates\n fast-import.c: change update_branch to use ref transactions\n branch.c: use ref transaction for all ref updates\n refs.c: change update_ref to use a transaction\n receive-pack.c: use a reference transaction for updating the refs\n fast-import.c: use a ref transaction when dumping tags\n walker.c: use ref transaction for ref updates\n refs.c: make lock_ref_sha1 static\n refs.c: remove the update_ref_lock function\n refs.c: remove the update_ref_write function\n refs.c: remove lock_ref_sha1\n refs.c: make prune_ref use a transaction to delete the ref\n refs.c: make delete_ref use a transaction\n refs.c: add an err argument to delete_ref_loose\n refs.c: pass the ref log message to _create/delete/update instead of\n _commit\n refs.c: pass NULL as *flags to read_ref_full\n refs.c: move the check for valid refname to lock_ref_sha1_basic\n refs.c: call lock_ref_sha1_basic directly from commit\n refs.c: pass a skip list to name_conflict_fn\n refs.c: propagate any errno==ENOTDIR from _commit back to the callers\n fetch.c: change s_update_ref to use a ref transaction\n refs.c: make write_ref_sha1 static\n\n branch.c | 30 +--\n builtin/commit.c | 24 ++-\n builtin/fetch.c | 36 ++--\n builtin/receive-pack.c | 97 ++++++----\n builtin/remote.c | 5 +-\n builtin/replace.c | 15 +-\n builtin/tag.c | 15 +-\n builtin/update-ref.c | 34 ++--\n cache.h | 4 +-\n fast-import.c | 53 ++++--\n lockfile.c | 39 ++--\n refs.c | 506 ++++++++++++++++++++++++++++++++-----------------\n refs.h | 131 +++++++++----\n sequencer.c | 24 ++-\n walker.c | 58 +++---\n wrapper.c | 14 +-\n 16 files changed, 710 insertions(+), 375 deletions(-)\n\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244765","messageId":"1403275409-28173-2-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 01/48] refs.c: remove ref_transaction_rollback","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:42:42Z","receivedAt":"2014-06-20T14:42:42Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"We do not yet need both a rollback and a free function for transactions.\nRemove ref_transaction_rollback and use ref_transaction_free instead.\n\nAt a later stage we may reintroduce a rollback function if we want to start\nadding reusable transactions and similar.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 7 +------\n refs.h | 16 +++++++---------\n 2 files changed, 8 insertions(+), 15 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex dc45774..6d841a0 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -3334,7 +3334,7 @@ struct ref_transaction *ref_transaction_begin(void)\n \treturn xcalloc(1, sizeof(struct ref_transaction));\n }\n \n-static void ref_transaction_free(struct ref_transaction *transaction)\n+void ref_transaction_free(struct ref_transaction *transaction)\n {\n \tint i;\n \n@@ -3345,11 +3345,6 @@ static void ref_transaction_free(struct ref_transaction *transaction)\n \tfree(transaction);\n }\n \n-void ref_transaction_rollback(struct ref_transaction *transaction)\n-{\n-\tref_transaction_free(transaction);\n-}\n-\n static struct ref_update *add_update(struct ref_transaction *transaction,\n \t\t\t\t const char *refname)\n {\ndiff --git a/refs.h b/refs.h\nindex 4e3050d..cfd1832 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -219,18 +219,12 @@ enum action_on_err {\n \n /*\n * Begin a reference transaction. The reference transaction must\n- * eventually be commited using ref_transaction_commit() or rolled\n- * back using ref_transaction_rollback().\n+ * eventually be commited using ref_transaction_commit() or freed by\n+ * calling ref_transaction_free().\n */\n struct ref_transaction *ref_transaction_begin(void);\n \n /*\n- * Roll back a ref_transaction and free all associated data.\n- */\n-void ref_transaction_rollback(struct ref_transaction *transaction);\n-\n-\n-/*\n * The following functions add a reference check or update to a\n * ref_transaction. In all of them, refname is the name of the\n * reference to be affected. The functions make internal copies of\n@@ -238,7 +232,6 @@ void ref_transaction_rollback(struct ref_transaction *transaction);\n * can be REF_NODEREF; it is passed to update_ref_lock().\n */\n \n-\n /*\n * Add a reference update to transaction. new_sha1 is the value that\n * the reference should have after the update, or zeros if it should\n@@ -280,6 +273,11 @@ void ref_transaction_delete(struct ref_transaction *transaction,\n int ref_transaction_commit(struct ref_transaction *transaction,\n \t\t\t const char *msg, enum action_on_err onerr);\n \n+/*\n+ * Free an existing transaction and all associated data.\n+ */\n+void ref_transaction_free(struct ref_transaction *transaction);\n+\n /** Lock a ref and then write its file */\n int update_ref(const char *action, const char *refname,\n \t\tconst unsigned char *sha1, const unsigned char *oldval,\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244756","messageId":"1403275409-28173-3-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 02/48] refs.c: ref_transaction_commit should not free the transaction","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:42:43Z","receivedAt":"2014-06-20T14:42:43Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Reviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n builtin/update-ref.c | 1 +\n refs.c | 1 -\n refs.h | 5 ++---\n 3 files changed, 3 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/update-ref.c b/builtin/update-ref.c\nindex 405267f..1fd7a89 100644\n--- a/builtin/update-ref.c\n+++ b/builtin/update-ref.c\n@@ -369,6 +369,7 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)\n \t\tupdate_refs_stdin();\n \t\tret = ref_transaction_commit(transaction, msg,\n \t\t\t\t\t UPDATE_REFS_DIE_ON_ERR);\n+\t\tref_transaction_free(transaction);\n \t\treturn ret;\n \t}\n \ndiff --git a/refs.c b/refs.c\nindex 6d841a0..d9cac6d 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -3509,7 +3509,6 @@ cleanup:\n \t\tif (updates[i]->lock)\n \t\t\tunlock_ref(updates[i]->lock);\n \tfree(delnames);\n-\tref_transaction_free(transaction);\n \treturn ret;\n }\n \ndiff --git a/refs.h b/refs.h\nindex cfd1832..7d946f6 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -219,8 +219,7 @@ enum action_on_err {\n \n /*\n * Begin a reference transaction. The reference transaction must\n- * eventually be commited using ref_transaction_commit() or freed by\n- * calling ref_transaction_free().\n+ * be freed by calling ref_transaction_free().\n */\n struct ref_transaction *ref_transaction_begin(void);\n \n@@ -268,7 +267,7 @@ void ref_transaction_delete(struct ref_transaction *transaction,\n /*\n * Commit all of the changes that have been queued in transaction, as\n * atomically as possible. Return a nonzero value if there is a\n- * problem. The ref_transaction is freed by this function.\n+ * problem.\n */\n int ref_transaction_commit(struct ref_transaction *transaction,\n \t\t\t const char *msg, enum action_on_err onerr);\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244764","messageId":"1403275409-28173-4-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 03/48] refs.c: constify the sha arguments for ref_transaction_create|delete|update","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:42:44Z","receivedAt":"2014-06-20T14:42:44Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"ref_transaction_create|delete|update has no need to modify the sha1\narguments passed to it so it should use const unsigned char* instead\nof unsigned char*.\n\nSome functions, such as fast_forward_to(), already have its old/new\nsha1 arguments as consts. This function will at some point need to\nuse ref_transaction_update() in which case this change is required.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 7 ++++---\n refs.h | 7 ++++---\n 2 files changed, 8 insertions(+), 6 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex d9cac6d..21ed465 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -3359,7 +3359,8 @@ static struct ref_update *add_update(struct ref_transaction *transaction,\n \n void ref_transaction_update(struct ref_transaction *transaction,\n \t\t\t const char *refname,\n-\t\t\t unsigned char *new_sha1, unsigned char *old_sha1,\n+\t\t\t const unsigned char *new_sha1,\n+\t\t\t const unsigned char *old_sha1,\n \t\t\t int flags, int have_old)\n {\n \tstruct ref_update *update = add_update(transaction, refname);\n@@ -3373,7 +3374,7 @@ void ref_transaction_update(struct ref_transaction *transaction,\n \n void ref_transaction_create(struct ref_transaction *transaction,\n \t\t\t const char *refname,\n-\t\t\t unsigned char *new_sha1,\n+\t\t\t const unsigned char *new_sha1,\n \t\t\t int flags)\n {\n \tstruct ref_update *update = add_update(transaction, refname);\n@@ -3387,7 +3388,7 @@ void ref_transaction_create(struct ref_transaction *transaction,\n \n void ref_transaction_delete(struct ref_transaction *transaction,\n \t\t\t const char *refname,\n-\t\t\t unsigned char *old_sha1,\n+\t\t\t const unsigned char *old_sha1,\n \t\t\t int flags, int have_old)\n {\n \tstruct ref_update *update = add_update(transaction, refname);\ndiff --git a/refs.h b/refs.h\nindex 7d946f6..8c7f9c4 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -240,7 +240,8 @@ struct ref_transaction *ref_transaction_begin(void);\n */\n void ref_transaction_update(struct ref_transaction *transaction,\n \t\t\t const char *refname,\n-\t\t\t unsigned char *new_sha1, unsigned char *old_sha1,\n+\t\t\t const unsigned char *new_sha1,\n+\t\t\t const unsigned char *old_sha1,\n \t\t\t int flags, int have_old);\n \n /*\n@@ -251,7 +252,7 @@ void ref_transaction_update(struct ref_transaction *transaction,\n */\n void ref_transaction_create(struct ref_transaction *transaction,\n \t\t\t const char *refname,\n-\t\t\t unsigned char *new_sha1,\n+\t\t\t const unsigned char *new_sha1,\n \t\t\t int flags);\n \n /*\n@@ -261,7 +262,7 @@ void ref_transaction_create(struct ref_transaction *transaction,\n */\n void ref_transaction_delete(struct ref_transaction *transaction,\n \t\t\t const char *refname,\n-\t\t\t unsigned char *old_sha1,\n+\t\t\t const unsigned char *old_sha1,\n \t\t\t int flags, int have_old);\n \n /*\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244758","messageId":"1403275409-28173-5-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 04/48] refs.c: allow passing NULL to ref_transaction_free","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:42:45Z","receivedAt":"2014-06-20T14:42:45Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Allow ref_transaction_free(NULL) as a no-op. This makes ref_transaction_free\neasier to use and more similar to plain 'free'.\n\nIn particular, it lets us rollback unconditionally as part of cleanup code\nafter setting 'transaction = NULL' if a transaction has been committed or\nrolled back already.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 3 +++\n 1 file changed, 3 insertions(+)\n\ndiff --git a/refs.c b/refs.c\nindex 21ed465..1d6dece 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -3338,6 +3338,9 @@ void ref_transaction_free(struct ref_transaction *transaction)\n {\n \tint i;\n \n+\tif (!transaction)\n+\t\treturn;\n+\n \tfor (i = 0; i < transaction->nr; i++)\n \t\tfree(transaction->updates[i]);\n \n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244759","messageId":"1403275409-28173-6-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 05/48] refs.c: add a strbuf argument to ref_transaction_commit for error logging","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:42:46Z","receivedAt":"2014-06-20T14:42:46Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Add a strbuf argument to _commit so that we can pass an error string back to\nthe caller. So that we can do error logging from the caller instead of from\n_commit.\n\nLonger term plan is to first convert all callers to use onerr==QUIET_ON_ERR\nand craft any log messages from the callers themselves and finally remove the\nonerr argument completely.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n builtin/update-ref.c | 2 +-\n refs.c | 6 +++++-\n refs.h | 5 ++++-\n 3 files changed, 10 insertions(+), 3 deletions(-)\n\ndiff --git a/builtin/update-ref.c b/builtin/update-ref.c\nindex 1fd7a89..22617af 100644\n--- a/builtin/update-ref.c\n+++ b/builtin/update-ref.c\n@@ -367,7 +367,7 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)\n \t\tif (end_null)\n \t\t\tline_termination = '\\0';\n \t\tupdate_refs_stdin();\n-\t\tret = ref_transaction_commit(transaction, msg,\n+\t\tret = ref_transaction_commit(transaction, msg, NULL,\n \t\t\t\t\t UPDATE_REFS_DIE_ON_ERR);\n \t\tref_transaction_free(transaction);\n \t\treturn ret;\ndiff --git a/refs.c b/refs.c\nindex 1d6dece..db05602 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -3444,7 +3444,8 @@ static int ref_update_reject_duplicates(struct ref_update **updates, int n,\n }\n \n int ref_transaction_commit(struct ref_transaction *transaction,\n-\t\t\t const char *msg, enum action_on_err onerr)\n+\t\t\t const char *msg, struct strbuf *err,\n+\t\t\t enum action_on_err onerr)\n {\n \tint ret = 0, delnum = 0, i;\n \tconst char **delnames;\n@@ -3473,6 +3474,9 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n \t\t\t\t\t update->flags,\n \t\t\t\t\t &update->type, onerr);\n \t\tif (!update->lock) {\n+\t\t\tif (err)\n+\t\t\t\tstrbuf_addf(err, \"Cannot lock the ref '%s'.\",\n+\t\t\t\t\t update->refname);\n \t\t\tret = 1;\n \t\t\tgoto cleanup;\n \t\t}\ndiff --git a/refs.h b/refs.h\nindex 8c7f9c4..09d3564 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -269,9 +269,12 @@ void ref_transaction_delete(struct ref_transaction *transaction,\n * Commit all of the changes that have been queued in transaction, as\n * atomically as possible. Return a nonzero value if there is a\n * problem.\n+ * If err is non-NULL we will add an error string to it to explain why\n+ * the transaction failed. The string does not end in newline.\n */\n int ref_transaction_commit(struct ref_transaction *transaction,\n-\t\t\t const char *msg, enum action_on_err onerr);\n+\t\t\t const char *msg, struct strbuf *err,\n+\t\t\t enum action_on_err onerr);\n \n /*\n * Free an existing transaction and all associated data.\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244760","messageId":"1403275409-28173-7-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 06/48] lockfile.c: add a new public function unable_to_lock_message","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:42:47Z","receivedAt":"2014-06-20T14:42:47Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Introducing a new unable_to_lock_message helper, which has nicer\nsemantics than unable_to_lock_error and cleans up lockfile.c a little.\n\nSigned-off-by: Ronnie Sahlberg \n---\n cache.h | 2 ++\n lockfile.c | 22 ++++++++++++----------\n 2 files changed, 14 insertions(+), 10 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex cbe1935..8b12aa8 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -559,6 +559,8 @@ struct lock_file {\n #define LOCK_DIE_ON_ERROR 1\n #define LOCK_NODEREF 2\n extern int unable_to_lock_error(const char *path, int err);\n+extern void unable_to_lock_message(const char *path, int err,\n+\t\t\t\t struct strbuf *buf);\n extern NORETURN void unable_to_lock_index_die(const char *path, int err);\n extern int hold_lock_file_for_update(struct lock_file *, const char *path, int);\n extern int hold_lock_file_for_append(struct lock_file *, const char *path, int);\ndiff --git a/lockfile.c b/lockfile.c\nindex 8fbcb6a..464031b 100644\n--- a/lockfile.c\n+++ b/lockfile.c\n@@ -157,33 +157,35 @@ static int lock_file(struct lock_file *lk, const char *path, int flags)\n \treturn lk->fd;\n }\n \n-static char *unable_to_lock_message(const char *path, int err)\n+void unable_to_lock_message(const char *path, int err, struct strbuf *buf)\n {\n-\tstruct strbuf buf = STRBUF_INIT;\n-\n \tif (err == EEXIST) {\n-\t\tstrbuf_addf(&buf, \"Unable to create '%s.lock': %s.\\n\\n\"\n+\t\tstrbuf_addf(buf, \"Unable to create '%s.lock': %s.\\n\\n\"\n \t\t \"If no other git process is currently running, this probably means a\\n\"\n \t\t \"git process crashed in this repository earlier. Make sure no other git\\n\"\n \t\t \"process is running and remove the file manually to continue.\",\n \t\t\t absolute_path(path), strerror(err));\n \t} else\n-\t\tstrbuf_addf(&buf, \"Unable to create '%s.lock': %s\",\n+\t\tstrbuf_addf(buf, \"Unable to create '%s.lock': %s\",\n \t\t\t absolute_path(path), strerror(err));\n-\treturn strbuf_detach(&buf, NULL);\n }\n \n int unable_to_lock_error(const char *path, int err)\n {\n-\tchar *msg = unable_to_lock_message(path, err);\n-\terror(\"%s\", msg);\n-\tfree(msg);\n+\tstruct strbuf buf = STRBUF_INIT;\n+\n+\tunable_to_lock_message(path, err, &buf);\n+\terror(\"%s\", buf.buf);\n+\tstrbuf_release(&buf);\n \treturn -1;\n }\n \n NORETURN void unable_to_lock_index_die(const char *path, int err)\n {\n-\tdie(\"%s\", unable_to_lock_message(path, err));\n+\tstruct strbuf buf = STRBUF_INIT;\n+\n+\tunable_to_lock_message(path, err, &buf);\n+\tdie(\"%s\", buf.buf);\n }\n \n int hold_lock_file_for_update(struct lock_file *lk, const char *path, int flags)\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244757","messageId":"1403275409-28173-8-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 07/48] lockfile.c: make lock_file return a meaningful errno on failurei","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:42:48Z","receivedAt":"2014-06-20T14:42:48Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Making errno when returning from lock_file() meaningful, which should\nfix\n\n * an existing almost-bug in lock_ref_sha1_basic where it assumes\n errno==ENOENT is meaningful and could waste some work on retries\n\n * an existing bug in repack_without_refs where it prints\n strerror(errno) and picks advice based on errno, despite errno\n potentially being zero and potentially having been clobbered by\n that point\n\nSigned-off-by: Ronnie Sahlberg \n---\n lockfile.c | 17 ++++++++++++-----\n refs.c | 1 +\n refs.h | 1 +\n 3 files changed, 14 insertions(+), 5 deletions(-)\n\ndiff --git a/lockfile.c b/lockfile.c\nindex 464031b..a921d77 100644\n--- a/lockfile.c\n+++ b/lockfile.c\n@@ -121,7 +121,7 @@ static char *resolve_symlink(char *p, size_t s)\n \treturn p;\n }\n \n-\n+/* Make sure errno contains a meaningful value on error */\n static int lock_file(struct lock_file *lk, const char *path, int flags)\n {\n \t/*\n@@ -130,8 +130,10 @@ static int lock_file(struct lock_file *lk, const char *path, int flags)\n \t */\n \tstatic const size_t max_path_len = sizeof(lk->filename) - 5;\n \n-\tif (strlen(path) >= max_path_len)\n+\tif (strlen(path) >= max_path_len) {\n+\t\terrno = ENAMETOOLONG;\n \t\treturn -1;\n+\t}\n \tstrcpy(lk->filename, path);\n \tif (!(flags & LOCK_NODEREF))\n \t\tresolve_symlink(lk->filename, max_path_len);\n@@ -148,9 +150,13 @@ static int lock_file(struct lock_file *lk, const char *path, int flags)\n \t\t\tlock_file_list = lk;\n \t\t\tlk->on_list = 1;\n \t\t}\n-\t\tif (adjust_shared_perm(lk->filename))\n-\t\t\treturn error(\"cannot fix permission bits on %s\",\n-\t\t\t\t lk->filename);\n+\t\tif (adjust_shared_perm(lk->filename)) {\n+\t\t\tint save_errno = errno;\n+\t\t\terror(\"cannot fix permission bits on %s\",\n+\t\t\t lk->filename);\n+\t\t\terrno = save_errno;\n+\t\t\treturn -1;\n+\t\t}\n \t}\n \telse\n \t\tlk->filename[0] = 0;\n@@ -188,6 +194,7 @@ NORETURN void unable_to_lock_index_die(const char *path, int err)\n \tdie(\"%s\", buf.buf);\n }\n \n+/* This should return a meaningful errno on failure */\n int hold_lock_file_for_update(struct lock_file *lk, const char *path, int flags)\n {\n \tint fd = lock_file(lk, path, flags);\ndiff --git a/refs.c b/refs.c\nindex db05602..e9d53e4 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -2212,6 +2212,7 @@ static int write_packed_entry_fn(struct ref_entry *entry, void *cb_data)\n \treturn 0;\n }\n \n+/* This should return a meaningful errno on failure */\n int lock_packed_refs(int flags)\n {\n \tstruct packed_ref_cache *packed_ref_cache;\ndiff --git a/refs.h b/refs.h\nindex 09d3564..64f25d9 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -82,6 +82,7 @@ extern void warn_dangling_symrefs(FILE *fp, const char *msg_fmt, const struct st\n /*\n * Lock the packed-refs file for writing. Flags is passed to\n * hold_lock_file_for_update(). Return 0 on success.\n+ * Errno is set to something meaningful on error.\n */\n extern int lock_packed_refs(int flags);\n \n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244762","messageId":"1403275409-28173-9-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 08/48] refs.c: add an err argument to repack_without_refs","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:42:49Z","receivedAt":"2014-06-20T14:42:49Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Update repack_without_refs to take an err argument and update it if there\nis a failure. Pass the err variable from ref_transaction_commit to this\nfunction so that callers can print a meaningful error message if _commit\nfails due to this function.\n\nSigned-off-by: Ronnie Sahlberg \n---\n builtin/remote.c | 5 +++--\n refs.c | 19 ++++++++++++++-----\n refs.h | 3 ++-\n 3 files changed, 19 insertions(+), 8 deletions(-)\n\ndiff --git a/builtin/remote.c b/builtin/remote.c\nindex c9102e8..401feb3 100644\n--- a/builtin/remote.c\n+++ b/builtin/remote.c\n@@ -755,7 +755,7 @@ static int remove_branches(struct string_list *branches)\n \tbranch_names = xmalloc(branches->nr * sizeof(*branch_names));\n \tfor (i = 0; i < branches->nr; i++)\n \t\tbranch_names[i] = branches->items[i].string;\n-\tresult |= repack_without_refs(branch_names, branches->nr);\n+\tresult |= repack_without_refs(branch_names, branches->nr, NULL);\n \tfree(branch_names);\n \n \tfor (i = 0; i < branches->nr; i++) {\n@@ -1333,7 +1333,8 @@ static int prune_remote(const char *remote, int dry_run)\n \t\tfor (i = 0; i < states.stale.nr; i++)\n \t\t\tdelete_refs[i] = states.stale.items[i].util;\n \t\tif (!dry_run)\n-\t\t\tresult |= repack_without_refs(delete_refs, states.stale.nr);\n+\t\t\tresult |= repack_without_refs(delete_refs,\n+\t\t\t\t\t\t states.stale.nr, NULL);\n \t\tfree(delete_refs);\n \t}\n \ndiff --git a/refs.c b/refs.c\nindex e9d53e4..67a0217 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -2456,12 +2456,12 @@ static int curate_packed_ref_fn(struct ref_entry *entry, void *cb_data)\n \treturn 0;\n }\n \n-int repack_without_refs(const char **refnames, int n)\n+int repack_without_refs(const char **refnames, int n, struct strbuf *err)\n {\n \tstruct ref_dir *packed;\n \tstruct string_list refs_to_delete = STRING_LIST_INIT_DUP;\n \tstruct string_list_item *ref_to_delete;\n-\tint i, removed = 0;\n+\tint i, ret, removed = 0;\n \n \t/* Look for a packed ref */\n \tfor (i = 0; i < n; i++)\n@@ -2473,6 +2473,11 @@ int repack_without_refs(const char **refnames, int n)\n \t\treturn 0; /* no refname exists in packed refs */\n \n \tif (lock_packed_refs(0)) {\n+\t\tif (err) {\n+\t\t\tunable_to_lock_message(git_path(\"packed-refs\"), errno,\n+\t\t\t\t\t err);\n+\t\t\treturn -1;\n+\t\t}\n \t\tunable_to_lock_error(git_path(\"packed-refs\"), errno);\n \t\treturn error(\"cannot delete '%s' from packed refs\", refnames[i]);\n \t}\n@@ -2499,12 +2504,16 @@ int repack_without_refs(const char **refnames, int n)\n \t}\n \n \t/* Write what remains */\n-\treturn commit_packed_refs();\n+\tret = commit_packed_refs();\n+\tif (ret && err)\n+\t\tstrbuf_addf(err, \"unable to overwrite old ref-pack file: %s\",\n+\t\t\t strerror(errno));\n+\treturn ret;\n }\n \n static int repack_without_ref(const char *refname)\n {\n-\treturn repack_without_refs(&refname, 1);\n+\treturn repack_without_refs(&refname, 1, NULL);\n }\n \n static int delete_ref_loose(struct ref_lock *lock, int flag)\n@@ -3508,7 +3517,7 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n \t\t}\n \t}\n \n-\tret |= repack_without_refs(delnames, delnum);\n+\tret |= repack_without_refs(delnames, delnum, err);\n \tfor (i = 0; i < delnum; i++)\n \t\tunlink_or_warn(git_path(\"logs/%s\", delnames[i]));\n \tclear_loose_ref_cache(&ref_cache);\ndiff --git a/refs.h b/refs.h\nindex 64f25d9..65f7637 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -122,7 +122,8 @@ extern void rollback_packed_refs(void);\n */\n int pack_refs(unsigned int flags);\n \n-extern int repack_without_refs(const char **refnames, int n);\n+extern int repack_without_refs(const char **refnames, int n,\n+\t\t\t struct strbuf *err);\n \n extern int ref_exists(const char *);\n \n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244799","messageId":"1403275409-28173-10-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 09/48] refs.c: make sure log_ref_setup returns a meaningful errno","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:42:50Z","receivedAt":"2014-06-20T14:42:50Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Making errno when returning from log_ref_setup() meaningful,\n\nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 27 +++++++++++++++++++--------\n refs.h | 4 +++-\n 2 files changed, 22 insertions(+), 9 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 67a0217..9ea519c 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -2751,6 +2751,7 @@ static int copy_msg(char *buf, const char *msg)\n \treturn cp - buf;\n }\n \n+/* This function must set a meaningful errno on failure */\n int log_ref_setup(const char *refname, char *logfile, int bufsize)\n {\n \tint logfd, oflags = O_APPEND | O_WRONLY;\n@@ -2761,9 +2762,12 @@ int log_ref_setup(const char *refname, char *logfile, int bufsize)\n \t starts_with(refname, \"refs/remotes/\") ||\n \t starts_with(refname, \"refs/notes/\") ||\n \t !strcmp(refname, \"HEAD\"))) {\n-\t\tif (safe_create_leading_directories(logfile) < 0)\n-\t\t\treturn error(\"unable to create directory for %s\",\n-\t\t\t\t logfile);\n+\t\tif (safe_create_leading_directories(logfile) < 0) {\n+\t\t\tint save_errno = errno;\n+\t\t\terror(\"unable to create directory for %s\", logfile);\n+\t\t\terrno = save_errno;\n+\t\t\treturn -1;\n+\t\t}\n \t\toflags |= O_CREAT;\n \t}\n \n@@ -2774,15 +2778,22 @@ int log_ref_setup(const char *refname, char *logfile, int bufsize)\n \n \t\tif ((oflags & O_CREAT) && errno == EISDIR) {\n \t\t\tif (remove_empty_directories(logfile)) {\n-\t\t\t\treturn error(\"There are still logs under '%s'\",\n-\t\t\t\t\t logfile);\n+\t\t\t\tint save_errno = errno;\n+\t\t\t\terror(\"There are still logs under '%s'\",\n+\t\t\t\t logfile);\n+\t\t\t\terrno = save_errno;\n+\t\t\t\treturn -1;\n \t\t\t}\n \t\t\tlogfd = open(logfile, oflags, 0666);\n \t\t}\n \n-\t\tif (logfd < 0)\n-\t\t\treturn error(\"Unable to append to %s: %s\",\n-\t\t\t\t logfile, strerror(errno));\n+\t\tif (logfd < 0) {\n+\t\t\tint save_errno = errno;\n+\t\t\terror(\"Unable to append to %s: %s\", logfile,\n+\t\t\t strerror(errno));\n+\t\t\terrno = save_errno;\n+\t\t\treturn -1;\n+\t\t}\n \t}\n \n \tadjust_shared_perm(logfile);\ndiff --git a/refs.h b/refs.h\nindex 65f7637..82cc5cb 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -158,7 +158,9 @@ extern void unlock_ref(struct ref_lock *lock);\n /** Writes sha1 into the ref specified by the lock. **/\n extern int write_ref_sha1(struct ref_lock *lock, const unsigned char *sha1, const char *msg);\n \n-/** Setup reflog before using. **/\n+/*\n+ * Setup reflog before using. Set errno to something meaningful on failure.\n+ */\n int log_ref_setup(const char *refname, char *logfile, int bufsize);\n \n /** Reads log for the value of ref during at_time. **/\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244797","messageId":"1403275409-28173-11-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 10/48] refs.c: verify_lock should set errno to something meaningful","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:42:51Z","receivedAt":"2014-06-20T14:42:51Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Making errno when returning from verify_lock() meaningful, which\nshould almost but not completely fix\n\n * a bug in \"git fetch\"'s s_update_ref, which trusts the result of an\n errno == ENOTDIR check to detect D/F conflicts\n\nENOTDIR makes sense as a sign that a file was in the way of a\ndirectory we wanted to create. Should \"git fetch\" also look for\nENOTEMPTY or EEXIST to catch cases where a directory was in the way\nof a file to be created?\n\nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 4 ++++\n refs.h | 6 +++++-\n 2 files changed, 9 insertions(+), 1 deletion(-)\n\ndiff --git a/refs.c b/refs.c\nindex 9ea519c..a48f805 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1932,18 +1932,22 @@ int refname_match(const char *abbrev_name, const char *full_name)\n \treturn 0;\n }\n \n+/* This function should make sure errno is meaningful on error */\n static struct ref_lock *verify_lock(struct ref_lock *lock,\n \tconst unsigned char *old_sha1, int mustexist)\n {\n \tif (read_ref_full(lock->ref_name, lock->old_sha1, mustexist, NULL)) {\n+\t\tint save_errno = errno;\n \t\terror(\"Can't verify ref %s\", lock->ref_name);\n \t\tunlock_ref(lock);\n+\t\terrno = save_errno;\n \t\treturn NULL;\n \t}\n \tif (hashcmp(lock->old_sha1, old_sha1)) {\n \t\terror(\"Ref %s is at %s but expected %s\", lock->ref_name,\n \t\t\tsha1_to_hex(lock->old_sha1), sha1_to_hex(old_sha1));\n \t\tunlock_ref(lock);\n+\t\terrno = EBUSY;\n \t\treturn NULL;\n \t}\n \treturn lock;\ndiff --git a/refs.h b/refs.h\nindex 82cc5cb..8d6cac7 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -137,11 +137,15 @@ extern int ref_exists(const char *);\n */\n extern int peel_ref(const char *refname, unsigned char *sha1);\n \n-/** Locks a \"refs/\" ref returning the lock on success and NULL on failure. **/\n+/*\n+ * Locks a \"refs/\" ref returning the lock on success and NULL on failure.\n+ * On failure errno is set to something meaningful.\n+ */\n extern struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char *old_sha1);\n \n /** Locks any ref (for 'HEAD' type refs). */\n #define REF_NODEREF\t0x01\n+/* errno is set to something meaningful on failure */\n extern struct ref_lock *lock_any_ref_for_update(const char *refname,\n \t\t\t\t\t\tconst unsigned char *old_sha1,\n \t\t\t\t\t\tint flags, int *type_p);\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244790","messageId":"1403275409-28173-12-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 11/48] refs.c: make remove_empty_directories always set errno to something sane","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:42:52Z","receivedAt":"2014-06-20T14:42:52Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Making errno when returning from remove_empty_directories() more\nobviously meaningful, which should provide some peace of mind for\npeople auditing lock_ref_sha1_basic.\n\nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 5 ++++-\n 1 file changed, 4 insertions(+), 1 deletion(-)\n\ndiff --git a/refs.c b/refs.c\nindex a48f805..cc69581 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1960,14 +1960,16 @@ static int remove_empty_directories(const char *file)\n \t * only empty directories), remove them.\n \t */\n \tstruct strbuf path;\n-\tint result;\n+\tint result, save_errno;\n \n \tstrbuf_init(&path, 20);\n \tstrbuf_addstr(&path, file);\n \n \tresult = remove_dir_recursively(&path, REMOVE_DIR_EMPTY_ONLY);\n+\tsave_errno = errno;\n \n \tstrbuf_release(&path);\n+\terrno = save_errno;\n \n \treturn result;\n }\n@@ -2056,6 +2058,7 @@ int dwim_log(const char *str, int len, unsigned char *sha1, char **log)\n \treturn logs_found;\n }\n \n+/* This function should make sure errno is meaningful on error */\n static struct ref_lock *lock_ref_sha1_basic(const char *refname,\n \t\t\t\t\t const unsigned char *old_sha1,\n \t\t\t\t\t int flags, int *type_p)\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244761","messageId":"1403275409-28173-13-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 12/48] refs.c: commit_packed_refs to return a meaningful errno on failure","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:42:53Z","receivedAt":"2014-06-20T14:42:53Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Making errno when returning from commit_packed_refs() meaningful,\nwhich should fix\n\n * a bug in \"git clone\" where it prints strerror(errno) based on\n errno, despite errno possibly being zero and potentially having\n been clobbered by that point\n * the same kind of bug in \"git pack-refs\"\n\nand prepares for repack_without_refs() to get a meaningful\nerror message when commit_packed_refs() fails without falling into\nthe same bug.\n\nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 10 +++++++++-\n refs.h | 1 +\n 2 files changed, 10 insertions(+), 1 deletion(-)\n\ndiff --git a/refs.c b/refs.c\nindex cc69581..7a815be 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -2239,11 +2239,16 @@ int lock_packed_refs(int flags)\n \treturn 0;\n }\n \n+/*\n+ * Commit the packed refs changes.\n+ * On error we must make sure that errno contains a meaningful value.\n+ */\n int commit_packed_refs(void)\n {\n \tstruct packed_ref_cache *packed_ref_cache =\n \t\tget_packed_ref_cache(&ref_cache);\n \tint error = 0;\n+\tint save_errno = 0;\n \n \tif (!packed_ref_cache->lock)\n \t\tdie(\"internal error: packed-refs not locked\");\n@@ -2253,10 +2258,13 @@ int commit_packed_refs(void)\n \tdo_for_each_entry_in_dir(get_packed_ref_dir(packed_ref_cache),\n \t\t\t\t 0, write_packed_entry_fn,\n \t\t\t\t &packed_ref_cache->lock->fd);\n-\tif (commit_lock_file(packed_ref_cache->lock))\n+\tif (commit_lock_file(packed_ref_cache->lock)) {\n+\t\tsave_errno = errno;\n \t\terror = -1;\n+\t}\n \tpacked_ref_cache->lock = NULL;\n \trelease_packed_ref_cache(packed_ref_cache);\n+\terrno = save_errno;\n \treturn error;\n }\n \ndiff --git a/refs.h b/refs.h\nindex 8d6cac7..e588ff8 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -98,6 +98,7 @@ extern void add_packed_ref(const char *refname, const unsigned char *sha1);\n * Write the current version of the packed refs cache from memory to\n * disk. The packed-refs file must already be locked for writing (see\n * lock_packed_refs()). Return zero on success.\n+ * Sets errno to something meaningful on error.\n */\n extern int commit_packed_refs(void);\n \n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244795","messageId":"1403275409-28173-14-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 13/48] refs.c: make resolve_ref_unsafe set errno to something meaningful on error","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:42:54Z","receivedAt":"2014-06-20T14:42:54Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Making errno when returning from resolve_ref_unsafe() meaningful,\nwhich should fix\n\n * a bug in lock_ref_sha1_basic, where it assumes EISDIR\n means it failed due to a directory being in the way\n\nSigned-off-by: Ronnie Sahlberg \n---\n cache.h | 2 +-\n refs.c | 19 +++++++++++++++----\n 2 files changed, 16 insertions(+), 5 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 8b12aa8..e7ec626 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -979,7 +979,7 @@ extern int read_ref(const char *refname, unsigned char *sha1);\n * NULL. If more than MAXDEPTH recursive symbolic lookups are needed,\n * give up and return NULL.\n *\n- * errno is sometimes set on errors, but not always.\n+ * errno is set to something meaningful on error.\n */\n extern const char *resolve_ref_unsafe(const char *ref, unsigned char *sha1, int reading, int *flag);\n extern char *resolve_refdup(const char *ref, unsigned char *sha1, int reading, int *flag);\ndiff --git a/refs.c b/refs.c\nindex 7a815be..211429d 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1334,6 +1334,7 @@ static const char *handle_missing_loose_ref(const char *refname,\n \t}\n }\n \n+/* This function needs to return a meaningful errno on failure */\n const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int reading, int *flag)\n {\n \tint depth = MAXDEPTH;\n@@ -1344,8 +1345,10 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int rea\n \tif (flag)\n \t\t*flag = 0;\n \n-\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL))\n+\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n+\t\terrno = EINVAL;\n \t\treturn NULL;\n+\t}\n \n \tfor (;;) {\n \t\tchar path[PATH_MAX];\n@@ -1353,8 +1356,10 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int rea\n \t\tchar *buf;\n \t\tint fd;\n \n-\t\tif (--depth < 0)\n+\t\tif (--depth < 0) {\n+\t\t\terrno = ELOOP;\n \t\t\treturn NULL;\n+\t\t}\n \n \t\tgit_snpath(path, sizeof(path), \"%s\", refname);\n \n@@ -1416,9 +1421,13 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int rea\n \t\t\t\treturn NULL;\n \t\t}\n \t\tlen = read_in_full(fd, buffer, sizeof(buffer)-1);\n-\t\tclose(fd);\n-\t\tif (len < 0)\n+\t\tif (len < 0) {\n+\t\t\tint save_errno = errno;\n+\t\t\tclose(fd);\n+\t\t\terrno = save_errno;\n \t\t\treturn NULL;\n+\t\t}\n+\t\tclose(fd);\n \t\twhile (len && isspace(buffer[len-1]))\n \t\t\tlen--;\n \t\tbuffer[len] = '\\0';\n@@ -1435,6 +1444,7 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int rea\n \t\t\t (buffer[40] != '\\0' && !isspace(buffer[40]))) {\n \t\t\t\tif (flag)\n \t\t\t\t\t*flag |= REF_ISBROKEN;\n+\t\t\t\terrno = EINVAL;\n \t\t\t\treturn NULL;\n \t\t\t}\n \t\t\treturn refname;\n@@ -1447,6 +1457,7 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int rea\n \t\tif (check_refname_format(buf, REFNAME_ALLOW_ONELEVEL)) {\n \t\t\tif (flag)\n \t\t\t\t*flag |= REF_ISBROKEN;\n+\t\t\terrno = EINVAL;\n \t\t\treturn NULL;\n \t\t}\n \t\trefname = strcpy(refname_buffer, buf);\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244793","messageId":"1403275409-28173-15-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 14/48] refs.c: log_ref_write should try to return meaningful errno","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:42:55Z","receivedAt":"2014-06-20T14:42:55Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Making errno from write_ref_sha1() meaningful, which should fix\n\n* a bug in \"git checkout -b\" where it prints strerror(errno)\n  despite errno possibly being zero or clobbered\n\n* a bug in \"git fetch\"'s s_update_ref, which trusts the result of an\n  errno == ENOTDIR check to detect D/F conflicts\n\nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 28 +++++++++++++++++++++++-----\n 1 file changed, 23 insertions(+), 5 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 211429d..61570c9 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -2859,8 +2859,19 @@ static int log_ref_write(const char *refname, const unsigned char *old_sha1,\n \t\tlen += copy_msg(logrec + len - 1, msg) - 1;\n \twritten = len <= maxlen ? write_in_full(logfd, logrec, len) : -1;\n \tfree(logrec);\n-\tif (close(logfd) != 0 || written != len)\n-\t\treturn error(\"Unable to append to %s\", log_file);\n+\tif (written != len) {\n+\t\tint save_errno = errno;\n+\t\tclose(logfd);\n+\t\terror(\"Unable to append to %s\", log_file);\n+\t\terrno = save_errno;\n+\t\treturn -1;\n+\t}\n+\tif (close(logfd)) {\n+\t\tint save_errno = errno;\n+\t\terror(\"Unable to append to %s\", log_file);\n+\t\terrno = save_errno;\n+\t\treturn -1;\n+\t}\n \treturn 0;\n }\n \n@@ -2869,14 +2880,17 @@ static int is_branch(const char *refname)\n \treturn !strcmp(refname, \"HEAD\") || starts_with(refname, \"refs/heads/\");\n }\n \n+/* This function must return a meaningful errno */\n int write_ref_sha1(struct ref_lock *lock,\n \tconst unsigned char *sha1, const char *logmsg)\n {\n \tstatic char term = '\\n';\n \tstruct object *o;\n \n-\tif (!lock)\n+\tif (!lock) {\n+\t\terrno = EINVAL;\n \t\treturn -1;\n+\t}\n \tif (!lock->force_write && !hashcmp(lock->old_sha1, sha1)) {\n \t\tunlock_ref(lock);\n \t\treturn 0;\n@@ -2886,19 +2900,23 @@ int write_ref_sha1(struct ref_lock *lock,\n \t\terror(\"Trying to write ref %s with nonexistent object %s\",\n \t\t\tlock->ref_name, sha1_to_hex(sha1));\n \t\tunlock_ref(lock);\n+\t\terrno = EINVAL;\n \t\treturn -1;\n \t}\n \tif (o->type != OBJ_COMMIT && is_branch(lock->ref_name)) {\n \t\terror(\"Trying to write non-commit object %s to branch %s\",\n \t\t\tsha1_to_hex(sha1), lock->ref_name);\n \t\tunlock_ref(lock);\n+\t\terrno = EINVAL;\n \t\treturn -1;\n \t}\n \tif (write_in_full(lock->lock_fd, sha1_to_hex(sha1), 40) != 40 ||\n-\t write_in_full(lock->lock_fd, &term, 1) != 1\n-\t\t|| close_ref(lock) < 0) {\n+\t write_in_full(lock->lock_fd, &term, 1) != 1 ||\n+\t close_ref(lock) < 0) {\n+\t\tint save_errno = errno;\n \t\terror(\"Couldn't write %s\", lock->lk->filename);\n \t\tunlock_ref(lock);\n+\t\terrno = save_errno;\n \t\treturn -1;\n \t}\n \tclear_loose_ref_cache(&ref_cache);\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244763","messageId":"1403275409-28173-16-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 15/48] refs.c: make ref_update_reject_duplicates take a strbuf argument for errors","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:42:56Z","receivedAt":"2014-06-20T14:42:56Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Make ref_update_reject_duplicates return any error that occurs through a\nnew strbuf argument. This means that when a transaction commit fails in\nthis function we will now be able to pass a helpful error message back to the\ncaller.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 6 +++++-\n 1 file changed, 5 insertions(+), 1 deletion(-)\n\ndiff --git a/refs.c b/refs.c\nindex 61570c9..115f143 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -3488,6 +3488,7 @@ static int ref_update_compare(const void *r1, const void *r2)\n }\n \n static int ref_update_reject_duplicates(struct ref_update **updates, int n,\n+\t\t\t\t\tstruct strbuf *err,\n \t\t\t\t\tenum action_on_err onerr)\n {\n \tint i;\n@@ -3495,6 +3496,9 @@ static int ref_update_reject_duplicates(struct ref_update **updates, int n,\n \t\tif (!strcmp(updates[i - 1]->refname, updates[i]->refname)) {\n \t\t\tconst char *str =\n \t\t\t\t\"Multiple updates for ref '%s' not allowed.\";\n+\t\t\tif (err)\n+\t\t\t\tstrbuf_addf(err, str, updates[i]->refname);\n+\n \t\t\tswitch (onerr) {\n \t\t\tcase UPDATE_REFS_MSG_ON_ERR:\n \t\t\t\terror(str, updates[i]->refname); break;\n@@ -3525,7 +3529,7 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n \n \t/* Copy, sort, and reject duplicate refs */\n \tqsort(updates, n, sizeof(*updates), ref_update_compare);\n-\tret = ref_update_reject_duplicates(updates, n, onerr);\n+\tret = ref_update_reject_duplicates(updates, n, err, onerr);\n \tif (ret)\n \t\tgoto cleanup;\n \n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244791","messageId":"1403275409-28173-17-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 16/48] refs.c: make update_ref_write update a strbuf on failure","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:42:57Z","receivedAt":"2014-06-20T14:42:57Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Change update_ref_write to also update an error strbuf on failure.\nThis makes the error available to ref_transaction_commit callers if the\ntransaction failed due to update_ref_sha1/write_ref_sha1 failures.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 9 ++++++---\n 1 file changed, 6 insertions(+), 3 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 115f143..003b313 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -3353,10 +3353,13 @@ static struct ref_lock *update_ref_lock(const char *refname,\n \n static int update_ref_write(const char *action, const char *refname,\n \t\t\t const unsigned char *sha1, struct ref_lock *lock,\n-\t\t\t enum action_on_err onerr)\n+\t\t\t struct strbuf *err, enum action_on_err onerr)\n {\n \tif (write_ref_sha1(lock, sha1, action) < 0) {\n \t\tconst char *str = \"Cannot update the ref '%s'.\";\n+\t\tif (err)\n+\t\t\tstrbuf_addf(err, str, refname);\n+\n \t\tswitch (onerr) {\n \t\tcase UPDATE_REFS_MSG_ON_ERR: error(str, refname); break;\n \t\tcase UPDATE_REFS_DIE_ON_ERR: die(str, refname); break;\n@@ -3477,7 +3480,7 @@ int update_ref(const char *action, const char *refname,\n \tlock = update_ref_lock(refname, oldval, flags, NULL, onerr);\n \tif (!lock)\n \t\treturn 1;\n-\treturn update_ref_write(action, refname, sha1, lock, onerr);\n+\treturn update_ref_write(action, refname, sha1, lock, NULL, onerr);\n }\n \n static int ref_update_compare(const void *r1, const void *r2)\n@@ -3559,7 +3562,7 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n \t\t\tret = update_ref_write(msg,\n \t\t\t\t\t update->refname,\n \t\t\t\t\t update->new_sha1,\n-\t\t\t\t\t update->lock, onerr);\n+\t\t\t\t\t update->lock, err, onerr);\n \t\t\tupdate->lock = NULL; /* freed by update_ref_write */\n \t\t\tif (ret)\n \t\t\t\tgoto cleanup;\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244794","messageId":"1403275409-28173-18-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 17/48] update-ref: use err argument to get error from ref_transaction_commit","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:42:58Z","receivedAt":"2014-06-20T14:42:58Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Call ref_transaction_commit with QUIET_ON_ERR and use the strbuf that is\nreturned to print a log message if/after the transaction fails.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n builtin/update-ref.c | 10 +++++-----\n 1 file changed, 5 insertions(+), 5 deletions(-)\n\ndiff --git a/builtin/update-ref.c b/builtin/update-ref.c\nindex 22617af..aec9004 100644\n--- a/builtin/update-ref.c\n+++ b/builtin/update-ref.c\n@@ -342,6 +342,7 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)\n \tconst char *refname, *oldval, *msg = NULL;\n \tunsigned char sha1[20], oldsha1[20];\n \tint delete = 0, no_deref = 0, read_stdin = 0, end_null = 0, flags = 0;\n+\tstruct strbuf err = STRBUF_INIT;\n \tstruct option options[] = {\n \t\tOPT_STRING( 'm', NULL, &msg, N_(\"reason\"), N_(\"reason of the update\")),\n \t\tOPT_BOOL('d', NULL, &delete, N_(\"delete the reference\")),\n@@ -359,18 +360,17 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)\n \t\tdie(\"Refusing to perform update with empty message.\");\n \n \tif (read_stdin) {\n-\t\tint ret;\n \t\ttransaction = ref_transaction_begin();\n-\n \t\tif (delete || no_deref || argc > 0)\n \t\t\tusage_with_options(git_update_ref_usage, options);\n \t\tif (end_null)\n \t\t\tline_termination = '\\0';\n \t\tupdate_refs_stdin();\n-\t\tret = ref_transaction_commit(transaction, msg, NULL,\n-\t\t\t\t\t UPDATE_REFS_DIE_ON_ERR);\n+\t\tif (ref_transaction_commit(transaction, msg, &err,\n+\t\t\t\t\t UPDATE_REFS_QUIET_ON_ERR))\n+\t\t\tdie(\"%s\", err.buf);\n \t\tref_transaction_free(transaction);\n-\t\treturn ret;\n+\t\treturn 0;\n \t}\n \n \tif (end_null)\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244803","messageId":"1403275409-28173-19-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 18/48] refs.c: remove the onerr argument to ref_transaction_commit","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:42:59Z","receivedAt":"2014-06-20T14:42:59Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Since all callers now use QUIET_ON_ERR we no longer need to provide an onerr\nargument any more. Remove the onerr argument from the ref_transaction_commit\nsignature.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n builtin/update-ref.c | 3 +--\n refs.c | 22 +++++++---------------\n refs.h | 3 +--\n 3 files changed, 9 insertions(+), 19 deletions(-)\n\ndiff --git a/builtin/update-ref.c b/builtin/update-ref.c\nindex aec9004..88ab785 100644\n--- a/builtin/update-ref.c\n+++ b/builtin/update-ref.c\n@@ -366,8 +366,7 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)\n \t\tif (end_null)\n \t\t\tline_termination = '\\0';\n \t\tupdate_refs_stdin();\n-\t\tif (ref_transaction_commit(transaction, msg, &err,\n-\t\t\t\t\t UPDATE_REFS_QUIET_ON_ERR))\n+\t\tif (ref_transaction_commit(transaction, msg, &err))\n \t\t\tdie(\"%s\", err.buf);\n \t\tref_transaction_free(transaction);\n \t\treturn 0;\ndiff --git a/refs.c b/refs.c\nindex 003b313..4f78bd9 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -3491,8 +3491,7 @@ static int ref_update_compare(const void *r1, const void *r2)\n }\n \n static int ref_update_reject_duplicates(struct ref_update **updates, int n,\n-\t\t\t\t\tstruct strbuf *err,\n-\t\t\t\t\tenum action_on_err onerr)\n+\t\t\t\t\tstruct strbuf *err)\n {\n \tint i;\n \tfor (i = 1; i < n; i++)\n@@ -3502,22 +3501,13 @@ static int ref_update_reject_duplicates(struct ref_update **updates, int n,\n \t\t\tif (err)\n \t\t\t\tstrbuf_addf(err, str, updates[i]->refname);\n \n-\t\t\tswitch (onerr) {\n-\t\t\tcase UPDATE_REFS_MSG_ON_ERR:\n-\t\t\t\terror(str, updates[i]->refname); break;\n-\t\t\tcase UPDATE_REFS_DIE_ON_ERR:\n-\t\t\t\tdie(str, updates[i]->refname); break;\n-\t\t\tcase UPDATE_REFS_QUIET_ON_ERR:\n-\t\t\t\tbreak;\n-\t\t\t}\n \t\t\treturn 1;\n \t\t}\n \treturn 0;\n }\n \n int ref_transaction_commit(struct ref_transaction *transaction,\n-\t\t\t const char *msg, struct strbuf *err,\n-\t\t\t enum action_on_err onerr)\n+\t\t\t const char *msg, struct strbuf *err)\n {\n \tint ret = 0, delnum = 0, i;\n \tconst char **delnames;\n@@ -3532,7 +3522,7 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n \n \t/* Copy, sort, and reject duplicate refs */\n \tqsort(updates, n, sizeof(*updates), ref_update_compare);\n-\tret = ref_update_reject_duplicates(updates, n, err, onerr);\n+\tret = ref_update_reject_duplicates(updates, n, err);\n \tif (ret)\n \t\tgoto cleanup;\n \n@@ -3544,7 +3534,8 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n \t\t\t\t\t (update->have_old ?\n \t\t\t\t\t\tupdate->old_sha1 : NULL),\n \t\t\t\t\t update->flags,\n-\t\t\t\t\t &update->type, onerr);\n+\t\t\t\t\t &update->type,\n+\t\t\t\t\t UPDATE_REFS_QUIET_ON_ERR);\n \t\tif (!update->lock) {\n \t\t\tif (err)\n \t\t\t\tstrbuf_addf(err, \"Cannot lock the ref '%s'.\",\n@@ -3562,7 +3553,8 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n \t\t\tret = update_ref_write(msg,\n \t\t\t\t\t update->refname,\n \t\t\t\t\t update->new_sha1,\n-\t\t\t\t\t update->lock, err, onerr);\n+\t\t\t\t\t update->lock, err,\n+\t\t\t\t\t UPDATE_REFS_QUIET_ON_ERR);\n \t\t\tupdate->lock = NULL; /* freed by update_ref_write */\n \t\t\tif (ret)\n \t\t\t\tgoto cleanup;\ndiff --git a/refs.h b/refs.h\nindex e588ff8..163b45c 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -282,8 +282,7 @@ void ref_transaction_delete(struct ref_transaction *transaction,\n * the transaction failed. The string does not end in newline.\n */\n int ref_transaction_commit(struct ref_transaction *transaction,\n-\t\t\t const char *msg, struct strbuf *err,\n-\t\t\t enum action_on_err onerr);\n+\t\t\t const char *msg, struct strbuf *err);\n \n /*\n * Free an existing transaction and all associated data.\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244788","messageId":"1403275409-28173-20-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 19/48] refs.c: change ref_transaction_update() to do error checking and return status","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:00Z","receivedAt":"2014-06-20T14:43:00Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Update ref_transaction_update() do some basic error checking and return\nnon-zero on error. Update all callers to check ref_transaction_update() for\nerror. There are currently no conditions in _update that will return error but\nthere will be in the future. Add an err argument that will be updated on\nfailure. In future patches we will start doing both locking and checking\nfor name conflicts in _update instead of _commit at which time this function\nwill start returning errors for these conditions.\n\nAlso check for BUGs during update and die(BUG:...) if we are calling\n_update with have_old but the old_sha1 pointer is NULL.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n builtin/update-ref.c | 12 +++++++-----\n refs.c | 18 ++++++++++++------\n refs.h | 14 +++++++++-----\n 3 files changed, 28 insertions(+), 16 deletions(-)\n\ndiff --git a/builtin/update-ref.c b/builtin/update-ref.c\nindex 88ab785..3067b11 100644\n--- a/builtin/update-ref.c\n+++ b/builtin/update-ref.c\n@@ -16,6 +16,7 @@ static struct ref_transaction *transaction;\n \n static char line_termination = '\\n';\n static int update_flags;\n+static struct strbuf err = STRBUF_INIT;\n \n /*\n * Parse one whitespace- or NUL-terminated, possibly C-quoted argument\n@@ -197,8 +198,9 @@ static const char *parse_cmd_update(struct strbuf *input, const char *next)\n \tif (*next != line_termination)\n \t\tdie(\"update %s: extra input: %s\", refname, next);\n \n-\tref_transaction_update(transaction, refname, new_sha1, old_sha1,\n-\t\t\t update_flags, have_old);\n+\tif (ref_transaction_update(transaction, refname, new_sha1, old_sha1,\n+\t\t\t\t update_flags, have_old, &err))\n+\t\tdie(\"%s\", err.buf);\n \n \tupdate_flags = 0;\n \tfree(refname);\n@@ -286,8 +288,9 @@ static const char *parse_cmd_verify(struct strbuf *input, const char *next)\n \tif (*next != line_termination)\n \t\tdie(\"verify %s: extra input: %s\", refname, next);\n \n-\tref_transaction_update(transaction, refname, new_sha1, old_sha1,\n-\t\t\t update_flags, have_old);\n+\tif (ref_transaction_update(transaction, refname, new_sha1, old_sha1,\n+\t\t\t\t update_flags, have_old, &err))\n+\t\tdie(\"%s\", err.buf);\n \n \tupdate_flags = 0;\n \tfree(refname);\n@@ -342,7 +345,6 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)\n \tconst char *refname, *oldval, *msg = NULL;\n \tunsigned char sha1[20], oldsha1[20];\n \tint delete = 0, no_deref = 0, read_stdin = 0, end_null = 0, flags = 0;\n-\tstruct strbuf err = STRBUF_INIT;\n \tstruct option options[] = {\n \t\tOPT_STRING( 'm', NULL, &msg, N_(\"reason\"), N_(\"reason of the update\")),\n \t\tOPT_BOOL('d', NULL, &delete, N_(\"delete the reference\")),\ndiff --git a/refs.c b/refs.c\nindex 4f78bd9..3f05e88 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -3428,19 +3428,25 @@ static struct ref_update *add_update(struct ref_transaction *transaction,\n \treturn update;\n }\n \n-void ref_transaction_update(struct ref_transaction *transaction,\n-\t\t\t const char *refname,\n-\t\t\t const unsigned char *new_sha1,\n-\t\t\t const unsigned char *old_sha1,\n-\t\t\t int flags, int have_old)\n+int ref_transaction_update(struct ref_transaction *transaction,\n+\t\t\t const char *refname,\n+\t\t\t const unsigned char *new_sha1,\n+\t\t\t const unsigned char *old_sha1,\n+\t\t\t int flags, int have_old,\n+\t\t\t struct strbuf *err)\n {\n-\tstruct ref_update *update = add_update(transaction, refname);\n+\tstruct ref_update *update;\n+\n+\tif (have_old && !old_sha1)\n+\t\tdie(\"BUG: have_old is true but old_sha1 is NULL\");\n \n+\tupdate = add_update(transaction, refname);\n \thashcpy(update->new_sha1, new_sha1);\n \tupdate->flags = flags;\n \tupdate->have_old = have_old;\n \tif (have_old)\n \t\thashcpy(update->old_sha1, old_sha1);\n+\treturn 0;\n }\n \n void ref_transaction_create(struct ref_transaction *transaction,\ndiff --git a/refs.h b/refs.h\nindex 163b45c..c5376ce 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -246,12 +246,16 @@ struct ref_transaction *ref_transaction_begin(void);\n * be deleted. If have_old is true, then old_sha1 holds the value\n * that the reference should have had before the update, or zeros if\n * it must not have existed beforehand.\n+ * Function returns 0 on success and non-zero on failure. A failure to update\n+ * means that the transaction as a whole has failed and will need to be\n+ * rolled back. On failure the err buffer will be updated.\n */\n-void ref_transaction_update(struct ref_transaction *transaction,\n-\t\t\t const char *refname,\n-\t\t\t const unsigned char *new_sha1,\n-\t\t\t const unsigned char *old_sha1,\n-\t\t\t int flags, int have_old);\n+int ref_transaction_update(struct ref_transaction *transaction,\n+\t\t\t const char *refname,\n+\t\t\t const unsigned char *new_sha1,\n+\t\t\t const unsigned char *old_sha1,\n+\t\t\t int flags, int have_old,\n+\t\t\t struct strbuf *err);\n \n /*\n * Add a reference creation to transaction. new_sha1 is the value\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244801","messageId":"1403275409-28173-21-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 20/48] refs.c: change ref_transaction_create to do error checking and return status","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:01Z","receivedAt":"2014-06-20T14:43:01Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Do basic error checking in ref_transaction_create() and make it return\nnon-zero on error. Update all callers to check the result of\nref_transaction_create(). There are currently no conditions in _create that\nwill return error but there will be in the future. Add an err argument that\nwill be updated on failure.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n builtin/update-ref.c | 4 +++-\n refs.c | 18 +++++++++++------\n refs.h | 55 +++++++++++++++++++++++++++++++++++++++++++++-------\n 3 files changed, 63 insertions(+), 14 deletions(-)\n\ndiff --git a/builtin/update-ref.c b/builtin/update-ref.c\nindex 3067b11..41121fa 100644\n--- a/builtin/update-ref.c\n+++ b/builtin/update-ref.c\n@@ -226,7 +226,9 @@ static const char *parse_cmd_create(struct strbuf *input, const char *next)\n \tif (*next != line_termination)\n \t\tdie(\"create %s: extra input: %s\", refname, next);\n \n-\tref_transaction_create(transaction, refname, new_sha1, update_flags);\n+\tif (ref_transaction_create(transaction, refname, new_sha1,\n+\t\t\t\t update_flags, &err))\n+\t\tdie(\"%s\", err.buf);\n \n \tupdate_flags = 0;\n \tfree(refname);\ndiff --git a/refs.c b/refs.c\nindex 3f05e88..c49f1c6 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -3449,18 +3449,24 @@ int ref_transaction_update(struct ref_transaction *transaction,\n \treturn 0;\n }\n \n-void ref_transaction_create(struct ref_transaction *transaction,\n-\t\t\t const char *refname,\n-\t\t\t const unsigned char *new_sha1,\n-\t\t\t int flags)\n+int ref_transaction_create(struct ref_transaction *transaction,\n+\t\t\t const char *refname,\n+\t\t\t const unsigned char *new_sha1,\n+\t\t\t int flags,\n+\t\t\t struct strbuf *err)\n {\n-\tstruct ref_update *update = add_update(transaction, refname);\n+\tstruct ref_update *update;\n+\n+\tif (!new_sha1 || is_null_sha1(new_sha1))\n+\t\tdie(\"BUG: create ref with null new_sha1\");\n+\n+\tupdate = add_update(transaction, refname);\n \n-\tassert(!is_null_sha1(new_sha1));\n \thashcpy(update->new_sha1, new_sha1);\n \thashclr(update->old_sha1);\n \tupdate->flags = flags;\n \tupdate->have_old = 1;\n+\treturn 0;\n }\n \n void ref_transaction_delete(struct ref_transaction *transaction,\ndiff --git a/refs.h b/refs.h\nindex c5376ce..33b4383 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -10,6 +10,45 @@ struct ref_lock {\n \tint force_write;\n };\n \n+/*\n+ * A ref_transaction represents a collection of ref updates\n+ * that should succeed or fail together.\n+ *\n+ * Calling sequence\n+ * ----------------\n+ * - Allocate and initialize a `struct ref_transaction` by calling\n+ * `ref_transaction_begin()`.\n+ *\n+ * - List intended ref updates by calling functions like\n+ * `ref_transaction_update()` and `ref_transaction_create()`.\n+ *\n+ * - Call `ref_transaction_commit()` to execute the transaction.\n+ * If this succeeds, the ref updates will have taken place and\n+ * the transaction cannot be rolled back.\n+ *\n+ * - At any time call `ref_transaction_free()` to discard the\n+ * transaction and free associated resources. In particular,\n+ * this rolls back the transaction if it has not been\n+ * successfully committed.\n+ *\n+ * Error handling\n+ * --------------\n+ *\n+ * On error, transaction functions append a message about what\n+ * went wrong to the 'err' argument. The message mentions what\n+ * ref was being updated (if any) when the error occurred so it\n+ * can be passed to 'die' or 'error' as-is.\n+ *\n+ * The message is appended to err without first clearing err.\n+ * This allows the caller to prepare preamble text to the generated\n+ * error message:\n+ *\n+ * strbuf_addf(&err, \"Error while doing foo-bar: \");\n+ * if (ref_transaction_update(..., &err)) {\n+ * ret = error(\"%s\", err.buf);\n+ * goto cleanup;\n+ * }\n+ */\n struct ref_transaction;\n \n /*\n@@ -248,7 +287,7 @@ struct ref_transaction *ref_transaction_begin(void);\n * it must not have existed beforehand.\n * Function returns 0 on success and non-zero on failure. A failure to update\n * means that the transaction as a whole has failed and will need to be\n- * rolled back. On failure the err buffer will be updated.\n+ * rolled back.\n */\n int ref_transaction_update(struct ref_transaction *transaction,\n \t\t\t const char *refname,\n@@ -262,11 +301,15 @@ int ref_transaction_update(struct ref_transaction *transaction,\n * that the reference should have after the update; it must not be the\n * null SHA-1. It is verified that the reference does not exist\n * already.\n+ * Function returns 0 on success and non-zero on failure. A failure to create\n+ * means that the transaction as a whole has failed and will need to be\n+ * rolled back.\n */\n-void ref_transaction_create(struct ref_transaction *transaction,\n-\t\t\t const char *refname,\n-\t\t\t const unsigned char *new_sha1,\n-\t\t\t int flags);\n+int ref_transaction_create(struct ref_transaction *transaction,\n+\t\t\t const char *refname,\n+\t\t\t const unsigned char *new_sha1,\n+\t\t\t int flags,\n+\t\t\t struct strbuf *err);\n \n /*\n * Add a reference deletion to transaction. If have_old is true, then\n@@ -282,8 +325,6 @@ void ref_transaction_delete(struct ref_transaction *transaction,\n * Commit all of the changes that have been queued in transaction, as\n * atomically as possible. Return a nonzero value if there is a\n * problem.\n- * If err is non-NULL we will add an error string to it to explain why\n- * the transaction failed. The string does not end in newline.\n */\n int ref_transaction_commit(struct ref_transaction *transaction,\n \t\t\t const char *msg, struct strbuf *err);\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244789","messageId":"1403275409-28173-22-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 21/48] refs.c: update ref_transaction_delete to check for error and return status","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:02Z","receivedAt":"2014-06-20T14:43:02Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Change ref_transaction_delete() to do basic error checking and return\nnon-zero of error. Update all callers to check the return for\nref_transaction_delete(). There are currently no conditions in _delete that\nwill return error but there will be in the future. Add an err argument that\nwill be updated on failure.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n builtin/update-ref.c | 5 +++--\n refs.c | 16 +++++++++++-----\n refs.h | 12 ++++++++----\n 3 files changed, 22 insertions(+), 11 deletions(-)\n\ndiff --git a/builtin/update-ref.c b/builtin/update-ref.c\nindex 41121fa..7c9c248 100644\n--- a/builtin/update-ref.c\n+++ b/builtin/update-ref.c\n@@ -258,8 +258,9 @@ static const char *parse_cmd_delete(struct strbuf *input, const char *next)\n \tif (*next != line_termination)\n \t\tdie(\"delete %s: extra input: %s\", refname, next);\n \n-\tref_transaction_delete(transaction, refname, old_sha1,\n-\t\t\t update_flags, have_old);\n+\tif (ref_transaction_delete(transaction, refname, old_sha1,\n+\t\t\t\t update_flags, have_old, &err))\n+\t\tdie(\"%s\", err.buf);\n \n \tupdate_flags = 0;\n \tfree(refname);\ndiff --git a/refs.c b/refs.c\nindex c49f1c6..40f04f4 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -3469,19 +3469,25 @@ int ref_transaction_create(struct ref_transaction *transaction,\n \treturn 0;\n }\n \n-void ref_transaction_delete(struct ref_transaction *transaction,\n-\t\t\t const char *refname,\n-\t\t\t const unsigned char *old_sha1,\n-\t\t\t int flags, int have_old)\n+int ref_transaction_delete(struct ref_transaction *transaction,\n+\t\t\t const char *refname,\n+\t\t\t const unsigned char *old_sha1,\n+\t\t\t int flags, int have_old,\n+\t\t\t struct strbuf *err)\n {\n-\tstruct ref_update *update = add_update(transaction, refname);\n+\tstruct ref_update *update;\n \n+\tif (have_old && !old_sha1)\n+\t\tdie(\"BUG: have_old is true but old_sha1 is NULL\");\n+\n+\tupdate = add_update(transaction, refname);\n \tupdate->flags = flags;\n \tupdate->have_old = have_old;\n \tif (have_old) {\n \t\tassert(!is_null_sha1(old_sha1));\n \t\thashcpy(update->old_sha1, old_sha1);\n \t}\n+\treturn 0;\n }\n \n int update_ref(const char *action, const char *refname,\ndiff --git a/refs.h b/refs.h\nindex 33b4383..eeababd 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -315,11 +315,15 @@ int ref_transaction_create(struct ref_transaction *transaction,\n * Add a reference deletion to transaction. If have_old is true, then\n * old_sha1 holds the value that the reference should have had before\n * the update (which must not be the null SHA-1).\n+ * Function returns 0 on success and non-zero on failure. A failure to delete\n+ * means that the transaction as a whole has failed and will need to be\n+ * rolled back.\n */\n-void ref_transaction_delete(struct ref_transaction *transaction,\n-\t\t\t const char *refname,\n-\t\t\t const unsigned char *old_sha1,\n-\t\t\t int flags, int have_old);\n+int ref_transaction_delete(struct ref_transaction *transaction,\n+\t\t\t const char *refname,\n+\t\t\t const unsigned char *old_sha1,\n+\t\t\t int flags, int have_old,\n+\t\t\t struct strbuf *err);\n \n /*\n * Commit all of the changes that have been queued in transaction, as\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244796","messageId":"1403275409-28173-23-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 22/48] refs.c: make ref_transaction_begin take an err argument","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:03Z","receivedAt":"2014-06-20T14:43:03Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Add an err argument to _begin so that on non-fatal failures in future ref\nbackends we can report a nice error back to the caller.\nWhile _begin can currently never fail for other reasons than OOM, in which\ncase we die() anyway, we may add other types of backends in the future.\nFor example, a hypothetical MySQL backend could fail in _being with\n\"Can not connect to MySQL server. No route to host\".\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n builtin/update-ref.c | 2 +-\n refs.c | 2 +-\n refs.h | 2 +-\n 3 files changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/builtin/update-ref.c b/builtin/update-ref.c\nindex 7c9c248..c6ad0be 100644\n--- a/builtin/update-ref.c\n+++ b/builtin/update-ref.c\n@@ -365,7 +365,7 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)\n \t\tdie(\"Refusing to perform update with empty message.\");\n \n \tif (read_stdin) {\n-\t\ttransaction = ref_transaction_begin();\n+\t\ttransaction = ref_transaction_begin(&err);\n \t\tif (delete || no_deref || argc > 0)\n \t\t\tusage_with_options(git_update_ref_usage, options);\n \t\tif (end_null)\ndiff --git a/refs.c b/refs.c\nindex 40f04f4..9cb7908 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -3397,7 +3397,7 @@ struct ref_transaction {\n \tsize_t nr;\n };\n \n-struct ref_transaction *ref_transaction_begin(void)\n+struct ref_transaction *ref_transaction_begin(struct strbuf *err)\n {\n \treturn xcalloc(1, sizeof(struct ref_transaction));\n }\ndiff --git a/refs.h b/refs.h\nindex eeababd..e729ea9 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -269,7 +269,7 @@ enum action_on_err {\n * Begin a reference transaction. The reference transaction must\n * be freed by calling ref_transaction_free().\n */\n-struct ref_transaction *ref_transaction_begin(void);\n+struct ref_transaction *ref_transaction_begin(struct strbuf *err);\n \n /*\n * The following functions add a reference check or update to a\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244787","messageId":"1403275409-28173-24-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 23/48] refs.c: add transaction.status and track OPEN/CLOSED/ERROR","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:04Z","receivedAt":"2014-06-20T14:43:04Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Track the status of a transaction in a new status field. Check the field for\nsanity, i.e. that status must be OPEN when _commit/_create/_delete or\n_update is called or else die(BUG:...)\n\nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 40 +++++++++++++++++++++++++++++++++++++++-\n 1 file changed, 39 insertions(+), 1 deletion(-)\n\ndiff --git a/refs.c b/refs.c\nindex 9cb7908..8c695ba 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -3387,6 +3387,25 @@ struct ref_update {\n };\n \n /*\n+ * Transaction states.\n+ * OPEN: The transaction is in a valid state and can accept new updates.\n+ * An OPEN transaction can be committed.\n+ * CLOSED: If an open transaction is successfully committed the state will\n+ * change to CLOSED. No further changes can be made to a CLOSED\n+ * transaction.\n+ * CLOSED means that all updates have been successfully committed and\n+ * the only thing that remains is to free the completed transaction.\n+ * ERROR: The transaction has failed and is no longer committable.\n+ * No further changes can be made to a CLOSED transaction and it must\n+ * be rolled back using transaction_free.\n+ */\n+enum ref_transaction_state {\n+\tREF_TRANSACTION_OPEN = 0,\n+\tREF_TRANSACTION_CLOSED = 1,\n+\tREF_TRANSACTION_ERROR = 2,\n+};\n+\n+/*\n * Data structure for holding a reference transaction, which can\n * consist of checks and updates to multiple references, carried out\n * as atomically as possible. This structure is opaque to callers.\n@@ -3395,6 +3414,8 @@ struct ref_transaction {\n \tstruct ref_update **updates;\n \tsize_t alloc;\n \tsize_t nr;\n+\tenum ref_transaction_state state;\n+\tint status;\n };\n \n struct ref_transaction *ref_transaction_begin(struct strbuf *err)\n@@ -3437,6 +3458,9 @@ int ref_transaction_update(struct ref_transaction *transaction,\n {\n \tstruct ref_update *update;\n \n+\tif (transaction->state != REF_TRANSACTION_OPEN)\n+\t\tdie(\"BUG: update called for transaction that is not open\");\n+\n \tif (have_old && !old_sha1)\n \t\tdie(\"BUG: have_old is true but old_sha1 is NULL\");\n \n@@ -3457,6 +3481,9 @@ int ref_transaction_create(struct ref_transaction *transaction,\n {\n \tstruct ref_update *update;\n \n+\tif (transaction->state != REF_TRANSACTION_OPEN)\n+\t\tdie(\"BUG: create called for transaction that is not open\");\n+\n \tif (!new_sha1 || is_null_sha1(new_sha1))\n \t\tdie(\"BUG: create ref with null new_sha1\");\n \n@@ -3477,6 +3504,9 @@ int ref_transaction_delete(struct ref_transaction *transaction,\n {\n \tstruct ref_update *update;\n \n+\tif (transaction->state != REF_TRANSACTION_OPEN)\n+\t\tdie(\"BUG: delete called for transaction that is not open\");\n+\n \tif (have_old && !old_sha1)\n \t\tdie(\"BUG: have_old is true but old_sha1 is NULL\");\n \n@@ -3532,8 +3562,13 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n \tint n = transaction->nr;\n \tstruct ref_update **updates = transaction->updates;\n \n-\tif (!n)\n+\tif (transaction->state != REF_TRANSACTION_OPEN)\n+\t\tdie(\"BUG: commit called for transaction that is not open\");\n+\n+\tif (!n) {\n+\t\ttransaction->state = REF_TRANSACTION_CLOSED;\n \t\treturn 0;\n+\t}\n \n \t/* Allocate work space */\n \tdelnames = xmalloc(sizeof(*delnames) * n);\n@@ -3595,6 +3630,9 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n \tclear_loose_ref_cache(&ref_cache);\n \n cleanup:\n+\ttransaction->state = ret ? REF_TRANSACTION_ERROR\n+\t\t: REF_TRANSACTION_CLOSED;\n+\n \tfor (i = 0; i < n; i++)\n \t\tif (updates[i]->lock)\n \t\t\tunlock_ref(updates[i]->lock);\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244792","messageId":"1403275409-28173-25-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 24/48] tag.c: use ref transactions when doing updates","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:05Z","receivedAt":"2014-06-20T14:43:05Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Change tag.c to use ref transactions for all ref updates.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n builtin/tag.c | 15 +++++++++------\n 1 file changed, 9 insertions(+), 6 deletions(-)\n\ndiff --git a/builtin/tag.c b/builtin/tag.c\nindex c6e8a71..c9bfc9a 100644\n--- a/builtin/tag.c\n+++ b/builtin/tag.c\n@@ -548,7 +548,6 @@ int cmd_tag(int argc, const char **argv, const char *prefix)\n \tstruct strbuf ref = STRBUF_INIT;\n \tunsigned char object[20], prev[20];\n \tconst char *object_ref, *tag;\n-\tstruct ref_lock *lock;\n \tstruct create_tag_options opt;\n \tchar *cleanup_arg = NULL;\n \tint annotate = 0, force = 0, lines = -1;\n@@ -556,6 +555,8 @@ int cmd_tag(int argc, const char **argv, const char *prefix)\n \tconst char *msgfile = NULL, *keyid = NULL;\n \tstruct msg_arg msg = { 0, STRBUF_INIT };\n \tstruct commit_list *with_commit = NULL;\n+\tstruct ref_transaction *transaction;\n+\tstruct strbuf err = STRBUF_INIT;\n \tstruct option options[] = {\n \t\tOPT_CMDMODE('l', \"list\", &cmdmode, N_(\"list tag names\"), 'l'),\n \t\t{ OPTION_INTEGER, 'n', NULL, &lines, N_(\"n\"),\n@@ -701,11 +702,13 @@ int cmd_tag(int argc, const char **argv, const char *prefix)\n \tif (annotate)\n \t\tcreate_tag(object, tag, &buf, &opt, prev, object);\n \n-\tlock = lock_any_ref_for_update(ref.buf, prev, 0, NULL);\n-\tif (!lock)\n-\t\tdie(_(\"%s: cannot lock the ref\"), ref.buf);\n-\tif (write_ref_sha1(lock, object, NULL) < 0)\n-\t\tdie(_(\"%s: cannot update the ref\"), ref.buf);\n+\ttransaction = ref_transaction_begin(&err);\n+\tif (!transaction ||\n+\t ref_transaction_update(transaction, ref.buf, object, prev,\n+\t\t\t\t 0, !is_null_sha1(prev), &err) ||\n+\t ref_transaction_commit(transaction, NULL, &err))\n+\t\tdie(\"%s\", err.buf);\n+\tref_transaction_free(transaction);\n \tif (force && !is_null_sha1(prev) && hashcmp(prev, object))\n \t\tprintf(_(\"Updated tag '%s' (was %s)\\n\"), tag, find_unique_abbrev(prev, DEFAULT_ABBREV));\n \n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244782","messageId":"1403275409-28173-26-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 25/48] replace.c: use the ref transaction functions for updates","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:06Z","receivedAt":"2014-06-20T14:43:06Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Update replace.c to use ref transactions for updates.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n builtin/replace.c | 15 +++++++++------\n 1 file changed, 9 insertions(+), 6 deletions(-)\n\ndiff --git a/builtin/replace.c b/builtin/replace.c\nindex 1bb491d..7528f3d 100644\n--- a/builtin/replace.c\n+++ b/builtin/replace.c\n@@ -153,7 +153,8 @@ static int replace_object_sha1(const char *object_ref,\n \tunsigned char prev[20];\n \tenum object_type obj_type, repl_type;\n \tchar ref[PATH_MAX];\n-\tstruct ref_lock *lock;\n+\tstruct ref_transaction *transaction;\n+\tstruct strbuf err = STRBUF_INIT;\n \n \tobj_type = sha1_object_info(object, NULL);\n \trepl_type = sha1_object_info(repl, NULL);\n@@ -166,12 +167,14 @@ static int replace_object_sha1(const char *object_ref,\n \n \tcheck_ref_valid(object, prev, ref, sizeof(ref), force);\n \n-\tlock = lock_any_ref_for_update(ref, prev, 0, NULL);\n-\tif (!lock)\n-\t\tdie(\"%s: cannot lock the ref\", ref);\n-\tif (write_ref_sha1(lock, repl, NULL) < 0)\n-\t\tdie(\"%s: cannot update the ref\", ref);\n+\ttransaction = ref_transaction_begin(&err);\n+\tif (!transaction ||\n+\t ref_transaction_update(transaction, ref, repl, prev,\n+\t\t\t\t 0, !is_null_sha1(prev), &err) ||\n+\t ref_transaction_commit(transaction, NULL, &err))\n+\t\tdie(\"%s\", err.buf);\n \n+\tref_transaction_free(transaction);\n \treturn 0;\n }\n \n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244781","messageId":"1403275409-28173-27-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 26/48] commit.c: use ref transactions for updates","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:07Z","receivedAt":"2014-06-20T14:43:07Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Change commit.c to use ref transactions for all ref updates.\nMake sure we pass a NULL pointer to ref_transaction_update if have_old\nis false.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n builtin/commit.c | 24 +++++++++++-------------\n 1 file changed, 11 insertions(+), 13 deletions(-)\n\ndiff --git a/builtin/commit.c b/builtin/commit.c\nindex 5e2221c..668fa6a 100644\n--- a/builtin/commit.c\n+++ b/builtin/commit.c\n@@ -1627,11 +1627,12 @@ int cmd_commit(int argc, const char **argv, const char *prefix)\n \tconst char *index_file, *reflog_msg;\n \tchar *nl;\n \tunsigned char sha1[20];\n-\tstruct ref_lock *ref_lock;\n \tstruct commit_list *parents = NULL, **pptr = &parents;\n \tstruct stat statbuf;\n \tstruct commit *current_head = NULL;\n \tstruct commit_extra_header *extra = NULL;\n+\tstruct ref_transaction *transaction;\n+\tstruct strbuf err = STRBUF_INIT;\n \n \tif (argc == 2 && !strcmp(argv[1], \"-h\"))\n \t\tusage_with_options(builtin_commit_usage, builtin_commit_options);\n@@ -1753,16 +1754,6 @@ int cmd_commit(int argc, const char **argv, const char *prefix)\n \tstrbuf_release(&author_ident);\n \tfree_commit_extra_headers(extra);\n \n-\tref_lock = lock_any_ref_for_update(\"HEAD\",\n-\t\t\t\t\t !current_head\n-\t\t\t\t\t ? NULL\n-\t\t\t\t\t : current_head->object.sha1,\n-\t\t\t\t\t 0, NULL);\n-\tif (!ref_lock) {\n-\t\trollback_index_files();\n-\t\tdie(_(\"cannot lock HEAD ref\"));\n-\t}\n-\n \tnl = strchr(sb.buf, '\\n');\n \tif (nl)\n \t\tstrbuf_setlen(&sb, nl + 1 - sb.buf);\n@@ -1771,10 +1762,17 @@ int cmd_commit(int argc, const char **argv, const char *prefix)\n \tstrbuf_insert(&sb, 0, reflog_msg, strlen(reflog_msg));\n \tstrbuf_insert(&sb, strlen(reflog_msg), \": \", 2);\n \n-\tif (write_ref_sha1(ref_lock, sha1, sb.buf) < 0) {\n+\ttransaction = ref_transaction_begin(&err);\n+\tif (!transaction ||\n+\t ref_transaction_update(transaction, \"HEAD\", sha1,\n+\t\t\t\t current_head ?\n+\t\t\t\t current_head->object.sha1 : NULL,\n+\t\t\t\t 0, !!current_head, &err) ||\n+\t ref_transaction_commit(transaction, sb.buf, &err)) {\n \t\trollback_index_files();\n-\t\tdie(_(\"cannot update HEAD ref\"));\n+\t\tdie(\"%s\", err.buf);\n \t}\n+\tref_transaction_free(transaction);\n \n \tunlink(git_path(\"CHERRY_PICK_HEAD\"));\n \tunlink(git_path(\"REVERT_HEAD\"));\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244802","messageId":"1403275409-28173-28-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 27/48] sequencer.c: use ref transactions for all ref updates","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:08Z","receivedAt":"2014-06-20T14:43:08Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Change to use ref transactions for all updates to refs.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n sequencer.c | 24 ++++++++++++++++--------\n 1 file changed, 16 insertions(+), 8 deletions(-)\n\ndiff --git a/sequencer.c b/sequencer.c\nindex 0a80c58..fd8acaf 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -272,23 +272,31 @@ static int error_dirty_index(struct replay_opts *opts)\n static int fast_forward_to(const unsigned char *to, const unsigned char *from,\n \t\t\tint unborn, struct replay_opts *opts)\n {\n-\tstruct ref_lock *ref_lock;\n+\tstruct ref_transaction *transaction;\n \tstruct strbuf sb = STRBUF_INIT;\n-\tint ret;\n+\tstruct strbuf err = STRBUF_INIT;\n \n \tread_cache();\n \tif (checkout_fast_forward(from, to, 1))\n \t\texit(1); /* the callee should have complained already */\n-\tref_lock = lock_any_ref_for_update(\"HEAD\", unborn ? null_sha1 : from,\n-\t\t\t\t\t 0, NULL);\n-\tif (!ref_lock)\n-\t\treturn error(_(\"Failed to lock HEAD during fast_forward_to\"));\n \n \tstrbuf_addf(&sb, \"%s: fast-forward\", action_name(opts));\n-\tret = write_ref_sha1(ref_lock, to, sb.buf);\n+\n+\ttransaction = ref_transaction_begin(&err);\n+\tif (!transaction ||\n+\t ref_transaction_update(transaction, \"HEAD\", to, from,\n+\t\t\t\t 0, !unborn, &err) ||\n+\t ref_transaction_commit(transaction, sb.buf, &err)) {\n+\t\tref_transaction_free(transaction);\n+\t\terror(\"%s\", err.buf);\n+\t\tstrbuf_release(&sb);\n+\t\tstrbuf_release(&err);\n+\t\treturn -1;\n+\t}\n \n \tstrbuf_release(&sb);\n-\treturn ret;\n+\tref_transaction_free(transaction);\n+\treturn 0;\n }\n \n static int do_recursive_merge(struct commit *base, struct commit *next,\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244800","messageId":"1403275409-28173-29-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 28/48] fast-import.c: change update_branch to use ref transactions","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:09Z","receivedAt":"2014-06-20T14:43:09Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Change update_branch() to use ref transactions for updates.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n fast-import.c | 24 ++++++++++++++----------\n 1 file changed, 14 insertions(+), 10 deletions(-)\n\ndiff --git a/fast-import.c b/fast-import.c\nindex 6707a66..d5206ee 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -1679,8 +1679,9 @@ found_entry:\n static int update_branch(struct branch *b)\n {\n \tstatic const char *msg = \"fast-import\";\n-\tstruct ref_lock *lock;\n+\tstruct ref_transaction *transaction;\n \tunsigned char old_sha1[20];\n+\tstruct strbuf err = STRBUF_INIT;\n \n \tif (read_ref(b->name, old_sha1))\n \t\thashclr(old_sha1);\n@@ -1689,29 +1690,32 @@ static int update_branch(struct branch *b)\n \t\t\tdelete_ref(b->name, old_sha1, 0);\n \t\treturn 0;\n \t}\n-\tlock = lock_any_ref_for_update(b->name, old_sha1, 0, NULL);\n-\tif (!lock)\n-\t\treturn error(\"Unable to lock %s\", b->name);\n \tif (!force_update && !is_null_sha1(old_sha1)) {\n \t\tstruct commit *old_cmit, *new_cmit;\n \n \t\told_cmit = lookup_commit_reference_gently(old_sha1, 0);\n \t\tnew_cmit = lookup_commit_reference_gently(b->sha1, 0);\n-\t\tif (!old_cmit || !new_cmit) {\n-\t\t\tunlock_ref(lock);\n+\t\tif (!old_cmit || !new_cmit)\n \t\t\treturn error(\"Branch %s is missing commits.\", b->name);\n-\t\t}\n \n \t\tif (!in_merge_bases(old_cmit, new_cmit)) {\n-\t\t\tunlock_ref(lock);\n \t\t\twarning(\"Not updating %s\"\n \t\t\t\t\" (new tip %s does not contain %s)\",\n \t\t\t\tb->name, sha1_to_hex(b->sha1), sha1_to_hex(old_sha1));\n \t\t\treturn -1;\n \t\t}\n \t}\n-\tif (write_ref_sha1(lock, b->sha1, msg) < 0)\n-\t\treturn error(\"Unable to update %s\", b->name);\n+\ttransaction = ref_transaction_begin(&err);\n+\tif (!transaction ||\n+\t ref_transaction_update(transaction, b->name, b->sha1, old_sha1,\n+\t\t\t\t 0, 1, &err) ||\n+\t ref_transaction_commit(transaction, msg, &err)) {\n+\t\tref_transaction_free(transaction);\n+\t\terror(\"%s\", err.buf);\n+\t\tstrbuf_release(&err);\n+\t\treturn -1;\n+\t}\n+\tref_transaction_free(transaction);\n \treturn 0;\n }\n \n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244777","messageId":"1403275409-28173-30-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 29/48] branch.c: use ref transaction for all ref updates","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:10Z","receivedAt":"2014-06-20T14:43:10Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Change create_branch to use a ref transaction when creating the new branch.\n\nThis also fixes a race condition in the old code where two concurrent\ncreate_branch could race since the lock_any_ref_for_update/write_ref_sha1\ndid not protect against the ref already existing. I.e. one thread could end up\noverwriting a branch even if the forcing flag is false.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n branch.c | 30 ++++++++++++++++--------------\n 1 file changed, 16 insertions(+), 14 deletions(-)\n\ndiff --git a/branch.c b/branch.c\nindex 660097b..c1eae00 100644\n--- a/branch.c\n+++ b/branch.c\n@@ -226,7 +226,6 @@ void create_branch(const char *head,\n \t\t int force, int reflog, int clobber_head,\n \t\t int quiet, enum branch_track track)\n {\n-\tstruct ref_lock *lock = NULL;\n \tstruct commit *commit;\n \tunsigned char sha1[20];\n \tchar *real_ref, msg[PATH_MAX + 20];\n@@ -285,15 +284,6 @@ void create_branch(const char *head,\n \t\tdie(_(\"Not a valid branch point: '%s'.\"), start_name);\n \thashcpy(sha1, commit->object.sha1);\n \n-\tif (!dont_change_ref) {\n-\t\tlock = lock_any_ref_for_update(ref.buf, NULL, 0, NULL);\n-\t\tif (!lock)\n-\t\t\tdie_errno(_(\"Failed to lock ref for update\"));\n-\t}\n-\n-\tif (reflog)\n-\t\tlog_all_ref_updates = 1;\n-\n \tif (forcing)\n \t\tsnprintf(msg, sizeof msg, \"branch: Reset to %s\",\n \t\t\t start_name);\n@@ -301,13 +291,25 @@ void create_branch(const char *head,\n \t\tsnprintf(msg, sizeof msg, \"branch: Created from %s\",\n \t\t\t start_name);\n \n+\tif (reflog)\n+\t\tlog_all_ref_updates = 1;\n+\n+\tif (!dont_change_ref) {\n+\t\tstruct ref_transaction *transaction;\n+\t\tstruct strbuf err = STRBUF_INIT;\n+\n+\t\ttransaction = ref_transaction_begin(&err);\n+\t\tif (!transaction ||\n+\t\t ref_transaction_update(transaction, ref.buf, sha1,\n+\t\t\t\t\t null_sha1, 0, !forcing, &err) ||\n+\t\t ref_transaction_commit(transaction, msg, &err))\n+\t\t\tdie(\"%s\", err.buf);\n+\t\tref_transaction_free(transaction);\n+\t}\n+\n \tif (real_ref && track)\n \t\tsetup_tracking(ref.buf + 11, real_ref, track, quiet);\n \n-\tif (!dont_change_ref)\n-\t\tif (write_ref_sha1(lock, sha1, msg) < 0)\n-\t\t\tdie_errno(_(\"Failed to write ref\"));\n-\n \tstrbuf_release(&ref);\n \tfree(real_ref);\n }\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244783","messageId":"1403275409-28173-31-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 30/48] refs.c: change update_ref to use a transaction","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:11Z","receivedAt":"2014-06-20T14:43:11Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Change the update_ref helper function to use a ref transaction internally.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 28 ++++++++++++++++++++++++----\n 1 file changed, 24 insertions(+), 4 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 8c695ba..4bdccc5 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -3524,11 +3524,31 @@ int update_ref(const char *action, const char *refname,\n \t const unsigned char *sha1, const unsigned char *oldval,\n \t int flags, enum action_on_err onerr)\n {\n-\tstruct ref_lock *lock;\n-\tlock = update_ref_lock(refname, oldval, flags, NULL, onerr);\n-\tif (!lock)\n+\tstruct ref_transaction *t;\n+\tstruct strbuf err = STRBUF_INIT;\n+\n+\tt = ref_transaction_begin(&err);\n+\tif (!t ||\n+\t ref_transaction_update(t, refname, sha1, oldval, flags,\n+\t\t\t\t !!oldval, &err) ||\n+\t ref_transaction_commit(t, action, &err)) {\n+\t\tconst char *str = \"update_ref failed for ref '%s': %s\";\n+\n+\t\tref_transaction_free(t);\n+\t\tswitch (onerr) {\n+\t\tcase UPDATE_REFS_MSG_ON_ERR:\n+\t\t\terror(str, refname, err.buf);\n+\t\t\tbreak;\n+\t\tcase UPDATE_REFS_DIE_ON_ERR:\n+\t\t\tdie(str, refname, err.buf);\n+\t\t\tbreak;\n+\t\tcase UPDATE_REFS_QUIET_ON_ERR:\n+\t\t\tbreak;\n+\t\t}\n+\t\tstrbuf_release(&err);\n \t\treturn 1;\n-\treturn update_ref_write(action, refname, sha1, lock, NULL, onerr);\n+\t}\n+\treturn 0;\n }\n \n static int ref_update_compare(const void *r1, const void *r2)\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244786","messageId":"1403275409-28173-32-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 31/48] receive-pack.c: use a reference transaction for updating the refs","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:12Z","receivedAt":"2014-06-20T14:43:12Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Wrap all the ref updates inside a transaction.\n\nSigned-off-by: Ronnie Sahlberg \n---\n builtin/receive-pack.c | 96 +++++++++++++++++++++++++++++++++-----------------\n 1 file changed, 63 insertions(+), 33 deletions(-)\n\ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex c323081..b51f8ae 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -194,7 +194,7 @@ static void write_head_info(void)\n \n struct command {\n \tstruct command *next;\n-\tconst char *error_string;\n+\tchar *error_string;\n \tunsigned int skip_update:1,\n \t\t did_not_exist:1;\n \tint index;\n@@ -468,19 +468,18 @@ static int update_shallow_ref(struct command *cmd, struct shallow_info *si)\n \treturn 0;\n }\n \n-static const char *update(struct command *cmd, struct shallow_info *si)\n+static char *update(struct command *cmd, struct shallow_info *si)\n {\n \tconst char *name = cmd->ref_name;\n \tstruct strbuf namespaced_name_buf = STRBUF_INIT;\n \tconst char *namespaced_name;\n \tunsigned char *old_sha1 = cmd->old_sha1;\n \tunsigned char *new_sha1 = cmd->new_sha1;\n-\tstruct ref_lock *lock;\n \n \t/* only refs/... are allowed */\n \tif (!starts_with(name, \"refs/\") || check_refname_format(name + 5, 0)) {\n \t\trp_error(\"refusing to create funny ref '%s' remotely\", name);\n-\t\treturn \"funny refname\";\n+\t\treturn xstrdup(\"funny refname\");\n \t}\n \n \tstrbuf_addf(&namespaced_name_buf, \"%s%s\", get_git_namespace(), name);\n@@ -498,20 +497,20 @@ static const char *update(struct command *cmd, struct shallow_info *si)\n \t\t\trp_error(\"refusing to update checked out branch: %s\", name);\n \t\t\tif (deny_current_branch == DENY_UNCONFIGURED)\n \t\t\t\trefuse_unconfigured_deny();\n-\t\t\treturn \"branch is currently checked out\";\n+\t\t\treturn xstrdup(\"branch is currently checked out\");\n \t\t}\n \t}\n \n \tif (!is_null_sha1(new_sha1) && !has_sha1_file(new_sha1)) {\n \t\terror(\"unpack should have generated %s, \"\n \t\t \"but I can't find it!\", sha1_to_hex(new_sha1));\n-\t\treturn \"bad pack\";\n+\t\treturn xstrdup(\"bad pack\");\n \t}\n \n \tif (!is_null_sha1(old_sha1) && is_null_sha1(new_sha1)) {\n \t\tif (deny_deletes && starts_with(name, \"refs/heads/\")) {\n \t\t\trp_error(\"denying ref deletion for %s\", name);\n-\t\t\treturn \"deletion prohibited\";\n+\t\t\treturn xstrdup(\"deletion prohibited\");\n \t\t}\n \n \t\tif (!strcmp(namespaced_name, head_name)) {\n@@ -526,7 +525,7 @@ static const char *update(struct command *cmd, struct shallow_info *si)\n \t\t\t\tif (deny_delete_current == DENY_UNCONFIGURED)\n \t\t\t\t\trefuse_unconfigured_deny_delete_current();\n \t\t\t\trp_error(\"refusing to delete the current branch: %s\", name);\n-\t\t\t\treturn \"deletion of the current branch prohibited\";\n+\t\t\t\treturn xstrdup(\"deletion of the current branch prohibited\");\n \t\t\t}\n \t\t}\n \t}\n@@ -544,19 +543,19 @@ static const char *update(struct command *cmd, struct shallow_info *si)\n \t\t old_object->type != OBJ_COMMIT ||\n \t\t new_object->type != OBJ_COMMIT) {\n \t\t\terror(\"bad sha1 objects for %s\", name);\n-\t\t\treturn \"bad ref\";\n+\t\t\treturn xstrdup(\"bad ref\");\n \t\t}\n \t\told_commit = (struct commit *)old_object;\n \t\tnew_commit = (struct commit *)new_object;\n \t\tif (!in_merge_bases(old_commit, new_commit)) {\n \t\t\trp_error(\"denying non-fast-forward %s\"\n \t\t\t\t \" (you should pull first)\", name);\n-\t\t\treturn \"non-fast-forward\";\n+\t\t\treturn xstrdup(\"non-fast-forward\");\n \t\t}\n \t}\n \tif (run_update_hook(cmd)) {\n \t\trp_error(\"hook declined to update %s\", name);\n-\t\treturn \"hook declined\";\n+\t\treturn xstrdup(\"hook declined\");\n \t}\n \n \tif (is_null_sha1(new_sha1)) {\n@@ -571,24 +570,32 @@ static const char *update(struct command *cmd, struct shallow_info *si)\n \t\t}\n \t\tif (delete_ref(namespaced_name, old_sha1, 0)) {\n \t\t\trp_error(\"failed to delete %s\", name);\n-\t\t\treturn \"failed to delete\";\n+\t\t\treturn xstrdup(\"failed to delete\");\n \t\t}\n \t\treturn NULL; /* good */\n \t}\n \telse {\n+\t\tstruct strbuf err = STRBUF_INIT;\n+\t\tstruct ref_transaction *transaction;\n+\n \t\tif (shallow_update && si->shallow_ref[cmd->index] &&\n \t\t update_shallow_ref(cmd, si))\n-\t\t\treturn \"shallow error\";\n-\n-\t\tlock = lock_any_ref_for_update(namespaced_name, old_sha1,\n-\t\t\t\t\t 0, NULL);\n-\t\tif (!lock) {\n-\t\t\trp_error(\"failed to lock %s\", name);\n-\t\t\treturn \"failed to lock\";\n-\t\t}\n-\t\tif (write_ref_sha1(lock, new_sha1, \"push\")) {\n-\t\t\treturn \"failed to write\"; /* error() already called */\n+\t\t\treturn xstrdup(\"shallow error\");\n+\n+\t\ttransaction = ref_transaction_begin(&err);\n+\t\tif (!transaction ||\n+\t\t ref_transaction_update(transaction, namespaced_name,\n+\t\t\t\t\t new_sha1, old_sha1, 0, 1, &err) ||\n+\t\t ref_transaction_commit(transaction, \"push\", &err)) {\n+\t\t\tchar *str = strbuf_detach(&err, NULL);\n+\t\t\tref_transaction_free(transaction);\n+\n+\t\t\trp_error(\"%s\", str);\n+\t\t\treturn str;\n \t\t}\n+\n+\t\tref_transaction_free(transaction);\n+\t\tstrbuf_release(&err);\n \t\treturn NULL; /* good */\n \t}\n }\n@@ -647,6 +654,9 @@ static void check_aliased_update(struct command *cmd, struct string_list *list)\n \tchar cmd_oldh[41], cmd_newh[41], dst_oldh[41], dst_newh[41];\n \tint flag;\n \n+\tif (cmd->error_string)\n+\t\tdie(\"BUG: check_alised_update called with failed cmd\");\n+\n \tstrbuf_addf(&buf, \"%s%s\", get_git_namespace(), cmd->ref_name);\n \tdst_name = resolve_ref_unsafe(buf.buf, sha1, 0, &flag);\n \tstrbuf_release(&buf);\n@@ -658,7 +668,7 @@ static void check_aliased_update(struct command *cmd, struct string_list *list)\n \tif (!dst_name) {\n \t\trp_error(\"refusing update to broken symref '%s'\", cmd->ref_name);\n \t\tcmd->skip_update = 1;\n-\t\tcmd->error_string = \"broken symref\";\n+\t\tcmd->error_string = xstrdup(\"broken symref\");\n \t\treturn;\n \t}\n \n@@ -684,8 +694,9 @@ static void check_aliased_update(struct command *cmd, struct string_list *list)\n \t\t cmd->ref_name, cmd_oldh, cmd_newh,\n \t\t dst_cmd->ref_name, dst_oldh, dst_newh);\n \n-\tcmd->error_string = dst_cmd->error_string =\n-\t\t\"inconsistent aliased update\";\n+\tcmd->error_string = xstrdup(\"inconsistent aliased update\");\n+\tfree(dst_cmd->error_string);\n+\tdst_cmd->error_string = xstrdup(\"inconsistent aliased update\");\n }\n \n static void check_aliased_updates(struct command *commands)\n@@ -733,7 +744,9 @@ static void set_connectivity_errors(struct command *commands,\n \t\tif (!check_everything_connected(command_singleton_iterator,\n \t\t\t\t\t\t0, &singleton))\n \t\t\tcontinue;\n-\t\tcmd->error_string = \"missing necessary objects\";\n+\t\tif (cmd->error_string) /* can't happen */\n+\t\t\tcontinue;\n+\t\tcmd->error_string = xstrdup(\"missing necessary objects\");\n \t}\n }\n \n@@ -770,9 +783,9 @@ static void reject_updates_to_hidden(struct command *commands)\n \t\tif (cmd->error_string || !ref_is_hidden(cmd->ref_name))\n \t\t\tcontinue;\n \t\tif (is_null_sha1(cmd->new_sha1))\n-\t\t\tcmd->error_string = \"deny deleting a hidden ref\";\n+\t\t\tcmd->error_string = xstrdup(\"deny deleting a hidden ref\");\n \t\telse\n-\t\t\tcmd->error_string = \"deny updating a hidden ref\";\n+\t\t\tcmd->error_string = xstrdup(\"deny updating a hidden ref\");\n \t}\n }\n \n@@ -786,8 +799,11 @@ static void execute_commands(struct command *commands,\n \tstruct iterate_data data;\n \n \tif (unpacker_error) {\n-\t\tfor (cmd = commands; cmd; cmd = cmd->next)\n-\t\t\tcmd->error_string = \"unpacker error\";\n+\t\tfor (cmd = commands; cmd; cmd = cmd->next) {\n+\t\t\tif (cmd->error_string) /* can't happen */\n+\t\t\t\tcontinue;\n+\t\t\tcmd->error_string = xstrdup(\"unpacker error\");\n+\t\t}\n \t\treturn;\n \t}\n \n@@ -800,8 +816,9 @@ static void execute_commands(struct command *commands,\n \n \tif (run_receive_hook(commands, \"pre-receive\", 0)) {\n \t\tfor (cmd = commands; cmd; cmd = cmd->next) {\n-\t\t\tif (!cmd->error_string)\n-\t\t\t\tcmd->error_string = \"pre-receive hook declined\";\n+\t\t\tif (cmd->error_string)\n+\t\t\t\tcontinue;\n+\t\t\tcmd->error_string = xstrdup(\"pre-receive hook declined\");\n \t\t}\n \t\treturn;\n \t}\n@@ -1079,7 +1096,8 @@ static void update_shallow_info(struct command *commands,\n \t\tif (is_null_sha1(cmd->new_sha1))\n \t\t\tcontinue;\n \t\tif (ref_status[cmd->index]) {\n-\t\t\tcmd->error_string = \"shallow update not allowed\";\n+\t\t\tfree(cmd->error_string);\n+\t\t\tcmd->error_string = xstrdup(\"shallow update not allowed\");\n \t\t\tcmd->skip_update = 1;\n \t\t}\n \t}\n@@ -1120,6 +1138,17 @@ static int delete_only(struct command *commands)\n \treturn 1;\n }\n \n+static void free_commands(struct command *commands)\n+{\n+\twhile (commands) {\n+\t\tstruct command *next = commands->next;\n+\n+\t\tfree(commands->error_string);\n+\t\tfree(commands);\n+\t\tcommands = next;\n+\t}\n+}\n+\n int cmd_receive_pack(int argc, const char **argv, const char *prefix)\n {\n \tint advertise_refs = 0;\n@@ -1215,5 +1244,6 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)\n \t\tpacket_flush(1);\n \tsha1_array_clear(&shallow);\n \tsha1_array_clear(&ref);\n+\tfree_commands(commands);\n \treturn 0;\n }\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244778","messageId":"1403275409-28173-33-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 32/48] fast-import.c: use a ref transaction when dumping tags","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:13Z","receivedAt":"2014-06-20T14:43:13Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Reviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n fast-import.c | 29 +++++++++++++++++++++++------\n 1 file changed, 23 insertions(+), 6 deletions(-)\n\ndiff --git a/fast-import.c b/fast-import.c\nindex d5206ee..a95e1be 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -1734,15 +1734,32 @@ static void dump_tags(void)\n {\n \tstatic const char *msg = \"fast-import\";\n \tstruct tag *t;\n-\tstruct ref_lock *lock;\n-\tchar ref_name[PATH_MAX];\n+\tstruct strbuf ref_name = STRBUF_INIT;\n+\tstruct strbuf err = STRBUF_INIT;\n+\tstruct ref_transaction *transaction;\n \n+\ttransaction = ref_transaction_begin(&err);\n+\tif (!transaction) {\n+\t\tfailure |= error(\"%s\", err.buf);\n+\t\tgoto cleanup;\n+\t}\n \tfor (t = first_tag; t; t = t->next_tag) {\n-\t\tsprintf(ref_name, \"tags/%s\", t->name);\n-\t\tlock = lock_ref_sha1(ref_name, NULL);\n-\t\tif (!lock || write_ref_sha1(lock, t->sha1, msg) < 0)\n-\t\t\tfailure |= error(\"Unable to update %s\", ref_name);\n+\t\tstrbuf_reset(&ref_name);\n+\t\tstrbuf_addf(&ref_name, \"refs/tags/%s\", t->name);\n+\n+\t\tif (ref_transaction_update(transaction, ref_name.buf, t->sha1,\n+\t\t\t\t\t NULL, 0, 0, &err)) {\n+\t\t\tfailure |= error(\"%s\", err.buf);\n+\t\t\tgoto cleanup;\n+\t\t}\n \t}\n+\tif (ref_transaction_commit(transaction, msg, &err))\n+\t\tfailure |= error(\"%s\", err.buf);\n+\n+ cleanup:\n+\tref_transaction_free(transaction);\n+\tstrbuf_release(&ref_name);\n+\tstrbuf_release(&err);\n }\n \n static void dump_marks_helper(FILE *f,\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244785","messageId":"1403275409-28173-34-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 33/48] walker.c: use ref transaction for ref updates","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:14Z","receivedAt":"2014-06-20T14:43:14Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Switch to using ref transactions in walker_fetch(). As part of the refactoring\nto use ref transactions we also fix a potential memory leak where in the\noriginal code if write_ref_sha1() would fail we would end up returning from\nthe function without free()ing the msg string.\n\nNote that this function is only called when fetching from a remote HTTP\nrepository onto the local (most of the time single-user) repository which\nlikely means that the type of collissions that the previous locking would\nprotect against and cause the fetch to fail for to be even more rare.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n walker.c | 59 +++++++++++++++++++++++++++++++++++------------------------\n 1 file changed, 35 insertions(+), 24 deletions(-)\n\ndiff --git a/walker.c b/walker.c\nindex 1dd86b8..60d9f9e 100644\n--- a/walker.c\n+++ b/walker.c\n@@ -251,39 +251,36 @@ void walker_targets_free(int targets, char **target, const char **write_ref)\n int walker_fetch(struct walker *walker, int targets, char **target,\n \t\t const char **write_ref, const char *write_ref_log_details)\n {\n-\tstruct ref_lock **lock = xcalloc(targets, sizeof(struct ref_lock *));\n+\tstruct strbuf ref_name = STRBUF_INIT;\n+\tstruct strbuf err = STRBUF_INIT;\n+\tstruct ref_transaction *transaction = NULL;\n \tunsigned char *sha1 = xmalloc(targets * 20);\n-\tchar *msg;\n-\tint ret;\n+\tchar *msg = NULL;\n \tint i;\n \n \tsave_commit_buffer = 0;\n \n-\tfor (i = 0; i < targets; i++) {\n-\t\tif (!write_ref || !write_ref[i])\n-\t\t\tcontinue;\n-\n-\t\tlock[i] = lock_ref_sha1(write_ref[i], NULL);\n-\t\tif (!lock[i]) {\n-\t\t\terror(\"Can't lock ref %s\", write_ref[i]);\n-\t\t\tgoto unlock_and_fail;\n+\tif (write_ref) {\n+\t\ttransaction = ref_transaction_begin(&err);\n+\t\tif (!transaction) {\n+\t\t\terror(\"%s\", err.buf);\n+\t\t\tgoto rollback_and_fail;\n \t\t}\n \t}\n-\n \tif (!walker->get_recover)\n \t\tfor_each_ref(mark_complete, NULL);\n \n \tfor (i = 0; i < targets; i++) {\n \t\tif (interpret_target(walker, target[i], &sha1[20 * i])) {\n \t\t\terror(\"Could not interpret response from server '%s' as something to pull\", target[i]);\n-\t\t\tgoto unlock_and_fail;\n+\t\t\tgoto rollback_and_fail;\n \t\t}\n \t\tif (process(walker, lookup_unknown_object(&sha1[20 * i])))\n-\t\t\tgoto unlock_and_fail;\n+\t\t\tgoto rollback_and_fail;\n \t}\n \n \tif (loop(walker))\n-\t\tgoto unlock_and_fail;\n+\t\tgoto rollback_and_fail;\n \n \tif (write_ref_log_details) {\n \t\tmsg = xmalloc(strlen(write_ref_log_details) + 12);\n@@ -294,19 +291,33 @@ int walker_fetch(struct walker *walker, int targets, char **target,\n \tfor (i = 0; i < targets; i++) {\n \t\tif (!write_ref || !write_ref[i])\n \t\t\tcontinue;\n-\t\tret = write_ref_sha1(lock[i], &sha1[20 * i], msg ? msg : \"fetch (unknown)\");\n-\t\tlock[i] = NULL;\n-\t\tif (ret)\n-\t\t\tgoto unlock_and_fail;\n+\t\tstrbuf_reset(&ref_name);\n+\t\tstrbuf_addf(&ref_name, \"refs/%s\", write_ref[i]);\n+\t\tif (ref_transaction_update(transaction, ref_name.buf,\n+\t\t\t\t\t &sha1[20 * i], NULL, 0, 0,\n+\t\t\t\t\t &err)) {\n+\t\t\terror(\"%s\", err.buf);\n+\t\t\tgoto rollback_and_fail;\n+\t\t}\n+\t}\n+\tif (write_ref) {\n+\t\tif (ref_transaction_commit(transaction,\n+\t\t\t\t\t msg ? msg : \"fetch (unknown)\",\n+\t\t\t\t\t &err)) {\n+\t\t\terror(\"%s\", err.buf);\n+\t\t\tgoto rollback_and_fail;\n+\t\t}\n+\t\tref_transaction_free(transaction);\n \t}\n-\tfree(msg);\n \n+\tfree(msg);\n \treturn 0;\n \n-unlock_and_fail:\n-\tfor (i = 0; i < targets; i++)\n-\t\tif (lock[i])\n-\t\t\tunlock_ref(lock[i]);\n+rollback_and_fail:\n+\tref_transaction_free(transaction);\n+\tfree(msg);\n+\tstrbuf_release(&err);\n+\tstrbuf_release(&ref_name);\n \n \treturn -1;\n }\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244798","messageId":"1403275409-28173-35-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 34/48] refs.c: make lock_ref_sha1 static","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:15Z","receivedAt":"2014-06-20T14:43:15Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"No external callers reference lock_ref_sha1 any more so lets declare it static.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 2 +-\n refs.h | 6 ------\n 2 files changed, 1 insertion(+), 7 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 4bdccc5..456acdd 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -2170,7 +2170,7 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,\n \treturn NULL;\n }\n \n-struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char *old_sha1)\n+static struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char *old_sha1)\n {\n \tchar refpath[PATH_MAX];\n \tif (check_refname_format(refname, 0))\ndiff --git a/refs.h b/refs.h\nindex e729ea9..4ac4a7d 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -177,12 +177,6 @@ extern int ref_exists(const char *);\n */\n extern int peel_ref(const char *refname, unsigned char *sha1);\n \n-/*\n- * Locks a \"refs/\" ref returning the lock on success and NULL on failure.\n- * On failure errno is set to something meaningful.\n- */\n-extern struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char *old_sha1);\n-\n /** Locks any ref (for 'HEAD' type refs). */\n #define REF_NODEREF\t0x01\n /* errno is set to something meaningful on failure */\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244784","messageId":"1403275409-28173-36-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 35/48] refs.c: remove the update_ref_lock function","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:16Z","receivedAt":"2014-06-20T14:43:16Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Since we now only call update_ref_lock with onerr==QUIET_ON_ERR we no longer\nneed this function and can replace it with just calling lock_any_ref_for_update\ndirectly.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 30 ++++++------------------------\n 1 file changed, 6 insertions(+), 24 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 456acdd..8fb0a9e 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -3333,24 +3333,6 @@ int for_each_reflog(each_ref_fn fn, void *cb_data)\n \treturn retval;\n }\n \n-static struct ref_lock *update_ref_lock(const char *refname,\n-\t\t\t\t\tconst unsigned char *oldval,\n-\t\t\t\t\tint flags, int *type_p,\n-\t\t\t\t\tenum action_on_err onerr)\n-{\n-\tstruct ref_lock *lock;\n-\tlock = lock_any_ref_for_update(refname, oldval, flags, type_p);\n-\tif (!lock) {\n-\t\tconst char *str = \"Cannot lock the ref '%s'.\";\n-\t\tswitch (onerr) {\n-\t\tcase UPDATE_REFS_MSG_ON_ERR: error(str, refname); break;\n-\t\tcase UPDATE_REFS_DIE_ON_ERR: die(str, refname); break;\n-\t\tcase UPDATE_REFS_QUIET_ON_ERR: break;\n-\t\t}\n-\t}\n-\treturn lock;\n-}\n-\n static int update_ref_write(const char *action, const char *refname,\n \t\t\t const unsigned char *sha1, struct ref_lock *lock,\n \t\t\t struct strbuf *err, enum action_on_err onerr)\n@@ -3603,12 +3585,12 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n \tfor (i = 0; i < n; i++) {\n \t\tstruct ref_update *update = updates[i];\n \n-\t\tupdate->lock = update_ref_lock(update->refname,\n-\t\t\t\t\t (update->have_old ?\n-\t\t\t\t\t\tupdate->old_sha1 : NULL),\n-\t\t\t\t\t update->flags,\n-\t\t\t\t\t &update->type,\n-\t\t\t\t\t UPDATE_REFS_QUIET_ON_ERR);\n+\t\tupdate->lock = lock_any_ref_for_update(update->refname,\n+\t\t\t\t\t\t (update->have_old ?\n+\t\t\t\t\t\t\tupdate->old_sha1 :\n+\t\t\t\t\t\t\tNULL),\n+\t\t\t\t\t\t update->flags,\n+\t\t\t\t\t\t &update->type);\n \t\tif (!update->lock) {\n \t\t\tif (err)\n \t\t\t\tstrbuf_addf(err, \"Cannot lock the ref '%s'.\",\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244780","messageId":"1403275409-28173-37-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 36/48] refs.c: remove the update_ref_write function","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:17Z","receivedAt":"2014-06-20T14:43:17Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Since we only call update_ref_write from a single place and we only call it\nwith onerr==QUIET_ON_ERR we can just as well get rid of it and just call\nwrite_ref_sha1 directly. This changes the return status for _commit from\n1 to -1 on failures when writing to the ref. Eventually we will want\n_commit to start returning more detailed error conditions than the current\nsimple success/failure. For example if the commit failed due to name\nconflicts etc.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 35 +++++++++--------------------------\n 1 file changed, 9 insertions(+), 26 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 8fb0a9e..7113843 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -3333,25 +3333,6 @@ int for_each_reflog(each_ref_fn fn, void *cb_data)\n \treturn retval;\n }\n \n-static int update_ref_write(const char *action, const char *refname,\n-\t\t\t const unsigned char *sha1, struct ref_lock *lock,\n-\t\t\t struct strbuf *err, enum action_on_err onerr)\n-{\n-\tif (write_ref_sha1(lock, sha1, action) < 0) {\n-\t\tconst char *str = \"Cannot update the ref '%s'.\";\n-\t\tif (err)\n-\t\t\tstrbuf_addf(err, str, refname);\n-\n-\t\tswitch (onerr) {\n-\t\tcase UPDATE_REFS_MSG_ON_ERR: error(str, refname); break;\n-\t\tcase UPDATE_REFS_DIE_ON_ERR: die(str, refname); break;\n-\t\tcase UPDATE_REFS_QUIET_ON_ERR: break;\n-\t\t}\n-\t\treturn 1;\n-\t}\n-\treturn 0;\n-}\n-\n /**\n * Information needed for a single ref update. Set new_sha1 to the\n * new value or to zero to delete the ref. To check the old value\n@@ -3605,14 +3586,16 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n \t\tstruct ref_update *update = updates[i];\n \n \t\tif (!is_null_sha1(update->new_sha1)) {\n-\t\t\tret = update_ref_write(msg,\n-\t\t\t\t\t update->refname,\n-\t\t\t\t\t update->new_sha1,\n-\t\t\t\t\t update->lock, err,\n-\t\t\t\t\t UPDATE_REFS_QUIET_ON_ERR);\n-\t\t\tupdate->lock = NULL; /* freed by update_ref_write */\n-\t\t\tif (ret)\n+\t\t\tret = write_ref_sha1(update->lock, update->new_sha1,\n+\t\t\t\t\t msg);\n+\t\t\tupdate->lock = NULL; /* freed by write_ref_sha1 */\n+\t\t\tif (ret) {\n+\t\t\t\tconst char *str = \"Cannot update the ref '%s'.\";\n+\n+\t\t\t\tif (err)\n+\t\t\t\t\tstrbuf_addf(err, str, update->refname);\n \t\t\t\tgoto cleanup;\n+\t\t\t}\n \t\t}\n \t}\n \n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244779","messageId":"1403275409-28173-38-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 37/48] refs.c: remove lock_ref_sha1","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:18Z","receivedAt":"2014-06-20T14:43:18Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"lock_ref_sha1 was only called from one place in refc.c and only provided\na check that the refname was sane before adding back the initial \"refs/\"\npart of the ref path name, the initial \"refs/\" that this caller had already\nstripped off before calling lock_ref_sha1.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 15 +++++----------\n 1 file changed, 5 insertions(+), 10 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 7113843..699f1f6 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -2170,15 +2170,6 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,\n \treturn NULL;\n }\n \n-static struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char *old_sha1)\n-{\n-\tchar refpath[PATH_MAX];\n-\tif (check_refname_format(refname, 0))\n-\t\treturn NULL;\n-\tstrcpy(refpath, mkpath(\"refs/%s\", refname));\n-\treturn lock_ref_sha1_basic(refpath, old_sha1, 0, NULL);\n-}\n-\n struct ref_lock *lock_any_ref_for_update(const char *refname,\n \t\t\t\t\t const unsigned char *old_sha1,\n \t\t\t\t\t int flags, int *type_p)\n@@ -2388,8 +2379,12 @@ static void try_remove_empty_parents(char *name)\n /* make sure nobody touched the ref, and unlink */\n static void prune_ref(struct ref_to_prune *r)\n {\n-\tstruct ref_lock *lock = lock_ref_sha1(r->name + 5, r->sha1);\n+\tstruct ref_lock *lock;\n+\n+\tif (check_refname_format(r->name + 5, 0))\n+\t\treturn;\n \n+\tlock = lock_ref_sha1_basic(r->name, r->sha1, 0, NULL);\n \tif (lock) {\n \t\tunlink_or_warn(git_path(\"%s\", r->name));\n \t\tunlock_ref(lock);\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244768","messageId":"1403275409-28173-39-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 38/48] refs.c: make prune_ref use a transaction to delete the ref","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:19Z","receivedAt":"2014-06-20T14:43:19Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Change prune_ref to delete the ref using a ref transaction. To do this we also\nneed to add a new flag REF_ISPRUNING that will tell the transaction that we\ndo not want to delete this ref from the packed refs. This flag is private to\nrefs.c and not exposed to external callers.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 27 ++++++++++++++++++++-------\n refs.h | 14 ++++++++++++--\n 2 files changed, 32 insertions(+), 9 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 699f1f6..3d070d5 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -25,6 +25,11 @@ static unsigned char refname_disposition[256] = {\n };\n \n /*\n+ * Used as a flag to ref_transaction_delete when a loose ref is being\n+ * pruned.\n+ */\n+#define REF_ISPRUNING\t0x0100\n+/*\n * Try to read one refname component from the front of refname.\n * Return the length of the component found, or -1 if the component is\n * not legal. It is legal if it is something reasonable to have under\n@@ -2379,17 +2384,24 @@ static void try_remove_empty_parents(char *name)\n /* make sure nobody touched the ref, and unlink */\n static void prune_ref(struct ref_to_prune *r)\n {\n-\tstruct ref_lock *lock;\n+\tstruct ref_transaction *transaction;\n+\tstruct strbuf err = STRBUF_INIT;\n \n \tif (check_refname_format(r->name + 5, 0))\n \t\treturn;\n \n-\tlock = lock_ref_sha1_basic(r->name, r->sha1, 0, NULL);\n-\tif (lock) {\n-\t\tunlink_or_warn(git_path(\"%s\", r->name));\n-\t\tunlock_ref(lock);\n-\t\ttry_remove_empty_parents(r->name);\n+\ttransaction = ref_transaction_begin(&err);\n+\tif (!transaction ||\n+\t ref_transaction_delete(transaction, r->name, r->sha1,\n+\t\t\t\t REF_ISPRUNING, 1, &err) ||\n+\t ref_transaction_commit(transaction, NULL, &err)) {\n+\t\tref_transaction_free(transaction);\n+\t\terror(\"%s\", err.buf);\n+\t\tstrbuf_release(&err);\n+\t\treturn;\n \t}\n+\tref_transaction_free(transaction);\n+\ttry_remove_empty_parents(r->name);\n }\n \n static void prune_refs(struct ref_to_prune *r)\n@@ -3599,8 +3611,9 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n \t\tstruct ref_update *update = updates[i];\n \n \t\tif (update->lock) {\n-\t\t\tdelnames[delnum++] = update->lock->ref_name;\n \t\t\tret |= delete_ref_loose(update->lock, update->type);\n+\t\t\tif (!(update->flags & REF_ISPRUNING))\n+\t\t\t\tdelnames[delnum++] = update->lock->ref_name;\n \t\t}\n \t}\n \ndiff --git a/refs.h b/refs.h\nindex 4ac4a7d..3b321c2 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -177,9 +177,19 @@ extern int ref_exists(const char *);\n */\n extern int peel_ref(const char *refname, unsigned char *sha1);\n \n-/** Locks any ref (for 'HEAD' type refs). */\n+/*\n+ * Flags controlling lock_any_ref_for_update(), ref_transaction_update(),\n+ * ref_transaction_create(), etc.\n+ * REF_NODEREF: act on the ref directly, instead of dereferencing\n+ * symbolic references.\n+ *\n+ * Flags >= 0x100 are reserved for internal use.\n+ */\n #define REF_NODEREF\t0x01\n-/* errno is set to something meaningful on failure */\n+/*\n+ * Locks any ref (for 'HEAD' type refs) and sets errno to something\n+ * meaningful on failure.\n+ */\n extern struct ref_lock *lock_any_ref_for_update(const char *refname,\n \t\t\t\t\t\tconst unsigned char *old_sha1,\n \t\t\t\t\t\tint flags, int *type_p);\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244769","messageId":"1403275409-28173-40-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 39/48] refs.c: make delete_ref use a transaction","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:20Z","receivedAt":"2014-06-20T14:43:20Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Change delete_ref to use a ref transaction for the deletion. At the same time\nsince we no longer have any callers of repack_without_ref we can now delete\nthis function.\n\nChange delete_ref to return 0 on success and 1 on failure instead of the\nprevious 0 on success either 1 or -1 on failure.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 34 +++++++++++++---------------------\n 1 file changed, 13 insertions(+), 21 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 3d070d5..92a06d4 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -2544,11 +2544,6 @@ int repack_without_refs(const char **refnames, int n, struct strbuf *err)\n \treturn ret;\n }\n \n-static int repack_without_ref(const char *refname)\n-{\n-\treturn repack_without_refs(&refname, 1, NULL);\n-}\n-\n static int delete_ref_loose(struct ref_lock *lock, int flag)\n {\n \tif (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {\n@@ -2566,24 +2561,21 @@ static int delete_ref_loose(struct ref_lock *lock, int flag)\n \n int delete_ref(const char *refname, const unsigned char *sha1, int delopt)\n {\n-\tstruct ref_lock *lock;\n-\tint ret = 0, flag = 0;\n+\tstruct ref_transaction *transaction;\n+\tstruct strbuf err = STRBUF_INIT;\n \n-\tlock = lock_ref_sha1_basic(refname, sha1, delopt, &flag);\n-\tif (!lock)\n+\ttransaction = ref_transaction_begin(&err);\n+\tif (!transaction ||\n+\t ref_transaction_delete(transaction, refname, sha1, delopt,\n+\t\t\t\t sha1 && !is_null_sha1(sha1), &err) ||\n+\t ref_transaction_commit(transaction, NULL, &err)) {\n+\t\terror(\"%s\", err.buf);\n+\t\tref_transaction_free(transaction);\n+\t\tstrbuf_release(&err);\n \t\treturn 1;\n-\tret |= delete_ref_loose(lock, flag);\n-\n-\t/* removing the loose one could have resurrected an earlier\n-\t * packed one. Also, if it was not loose we need to repack\n-\t * without it.\n-\t */\n-\tret |= repack_without_ref(lock->ref_name);\n-\n-\tunlink_or_warn(git_path(\"logs/%s\", lock->ref_name));\n-\tclear_loose_ref_cache(&ref_cache);\n-\tunlock_ref(lock);\n-\treturn ret;\n+\t}\n+\tref_transaction_free(transaction);\n+\treturn 0;\n }\n \n /*\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244766","messageId":"1403275409-28173-41-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 40/48] refs.c: add an err argument to delete_ref_loose","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:21Z","receivedAt":"2014-06-20T14:43:21Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Add an err argument to delete_loose_ref so that we can pass a descriptive\nerror string back to the caller. Pass the err argument from transaction\ncommit to this function so that transaction users will have a nice error\nstring if the transaction failed due to delete_loose_ref.\n\nAdd a new function unlink_or_err that we can call from delete_ref_loose. This\nfunction is similar to unlink_or_warn except that we can pass it an err\nargument. If err is non-NULL the function will populate err instead of\nprinting a warning().\n\nSimplify warn_if_unremovable.\n\nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 33 ++++++++++++++++++++++++++++-----\n wrapper.c | 14 ++++++--------\n 2 files changed, 34 insertions(+), 13 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 92a06d4..c7d1f3e 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -2544,16 +2544,38 @@ int repack_without_refs(const char **refnames, int n, struct strbuf *err)\n \treturn ret;\n }\n \n-static int delete_ref_loose(struct ref_lock *lock, int flag)\n+static int add_err_if_unremovable(const char *op, const char *file,\n+\t\t\t\t struct strbuf *e, int rc)\n+{\n+\tint err = errno;\n+\tif (rc < 0 && errno != ENOENT) {\n+\t\tstrbuf_addf(e, \"unable to %s %s: %s\",\n+\t\t\t op, file, strerror(errno));\n+\t\terrno = err;\n+\t\treturn -1;\n+\t}\n+\treturn 0;\n+}\n+\n+static int unlink_or_err(const char *file, struct strbuf *err)\n+{\n+\tif (err)\n+\t\treturn add_err_if_unremovable(\"unlink\", file, err,\n+\t\t\t\t\t unlink(file));\n+\telse\n+\t\treturn unlink_or_warn(file);\n+}\n+\n+static int delete_ref_loose(struct ref_lock *lock, int flag, struct strbuf *err)\n {\n \tif (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {\n \t\t/* loose */\n-\t\tint err, i = strlen(lock->lk->filename) - 5; /* .lock */\n+\t\tint res, i = strlen(lock->lk->filename) - 5; /* .lock */\n \n \t\tlock->lk->filename[i] = 0;\n-\t\terr = unlink_or_warn(lock->lk->filename);\n+\t\tres = unlink_or_err(lock->lk->filename, err);\n \t\tlock->lk->filename[i] = '.';\n-\t\tif (err && errno != ENOENT)\n+\t\tif (res)\n \t\t\treturn 1;\n \t}\n \treturn 0;\n@@ -3603,7 +3625,8 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n \t\tstruct ref_update *update = updates[i];\n \n \t\tif (update->lock) {\n-\t\t\tret |= delete_ref_loose(update->lock, update->type);\n+\t\t\tret |= delete_ref_loose(update->lock, update->type,\n+\t\t\t\t\t\terr);\n \t\t\tif (!(update->flags & REF_ISPRUNING))\n \t\t\t\tdelnames[delnum++] = update->lock->ref_name;\n \t\t}\ndiff --git a/wrapper.c b/wrapper.c\nindex bc1bfb8..740e193 100644\n--- a/wrapper.c\n+++ b/wrapper.c\n@@ -429,14 +429,12 @@ int xmkstemp_mode(char *template, int mode)\n \n static int warn_if_unremovable(const char *op, const char *file, int rc)\n {\n-\tif (rc < 0) {\n-\t\tint err = errno;\n-\t\tif (ENOENT != err) {\n-\t\t\twarning(\"unable to %s %s: %s\",\n-\t\t\t\top, file, strerror(errno));\n-\t\t\terrno = err;\n-\t\t}\n-\t}\n+\tint err;\n+\tif (rc >= 0 || errno == ENOENT)\n+\t\treturn rc;\n+\terr = errno;\n+\twarning(\"unable to %s %s: %s\", op, file, strerror(errno));\n+\terrno = err;\n \treturn rc;\n }\n \n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244776","messageId":"1403275409-28173-42-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 41/48] refs.c: pass the ref log message to _create/delete/update instead of _commit","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:22Z","receivedAt":"2014-06-20T14:43:22Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Change the reference transactions so that we pass the reflog message\nthrough to the create/delete/update function instead of the commit message.\nThis allows for individual messages for each change in a multi ref\ntransaction.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n branch.c | 4 ++--\n builtin/commit.c | 4 ++--\n builtin/fetch.c | 3 +--\n builtin/receive-pack.c | 5 +++--\n builtin/replace.c | 4 ++--\n builtin/tag.c | 4 ++--\n builtin/update-ref.c | 13 +++++++------\n fast-import.c | 8 ++++----\n refs.c | 34 +++++++++++++++++++++-------------\n refs.h | 8 ++++----\n sequencer.c | 4 ++--\n walker.c | 5 ++---\n 12 files changed, 52 insertions(+), 44 deletions(-)\n\ndiff --git a/branch.c b/branch.c\nindex c1eae00..e0439af 100644\n--- a/branch.c\n+++ b/branch.c\n@@ -301,8 +301,8 @@ void create_branch(const char *head,\n \t\ttransaction = ref_transaction_begin(&err);\n \t\tif (!transaction ||\n \t\t ref_transaction_update(transaction, ref.buf, sha1,\n-\t\t\t\t\t null_sha1, 0, !forcing, &err) ||\n-\t\t ref_transaction_commit(transaction, msg, &err))\n+\t\t\t\t\t null_sha1, 0, !forcing, msg, &err) ||\n+\t\t ref_transaction_commit(transaction, &err))\n \t\t\tdie(\"%s\", err.buf);\n \t\tref_transaction_free(transaction);\n \t}\ndiff --git a/builtin/commit.c b/builtin/commit.c\nindex 668fa6a..c499826 100644\n--- a/builtin/commit.c\n+++ b/builtin/commit.c\n@@ -1767,8 +1767,8 @@ int cmd_commit(int argc, const char **argv, const char *prefix)\n \t ref_transaction_update(transaction, \"HEAD\", sha1,\n \t\t\t\t current_head ?\n \t\t\t\t current_head->object.sha1 : NULL,\n-\t\t\t\t 0, !!current_head, &err) ||\n-\t ref_transaction_commit(transaction, sb.buf, &err)) {\n+\t\t\t\t 0, !!current_head, sb.buf, &err) ||\n+\t ref_transaction_commit(transaction, &err)) {\n \t\trollback_index_files();\n \t\tdie(\"%s\", err.buf);\n \t}\ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex 55f457c..faa1233 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -673,10 +673,9 @@ static int store_updated_refs(const char *raw_url, const char *remote_name,\n \t\t\t}\n \t\t}\n \t}\n-\n \tif (rc & STORE_REF_ERROR_DF_CONFLICT)\n \t\terror(_(\"some local refs could not be updated; try running\\n\"\n-\t\t \" 'git remote prune %s' to remove any old, conflicting \"\n+\t\t \"'git remote prune %s' to remove any old, conflicting \"\n \t\t \"branches\"), remote_name);\n \n abort:\ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex b51f8ae..0ed1ddd 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -585,8 +585,9 @@ static char *update(struct command *cmd, struct shallow_info *si)\n \t\ttransaction = ref_transaction_begin(&err);\n \t\tif (!transaction ||\n \t\t ref_transaction_update(transaction, namespaced_name,\n-\t\t\t\t\t new_sha1, old_sha1, 0, 1, &err) ||\n-\t\t ref_transaction_commit(transaction, \"push\", &err)) {\n+\t\t\t\t\t new_sha1, old_sha1, 0, 1, \"push\",\n+\t\t\t\t\t &err) ||\n+\t\t ref_transaction_commit(transaction, &err)) {\n \t\t\tchar *str = strbuf_detach(&err, NULL);\n \t\t\tref_transaction_free(transaction);\n \ndiff --git a/builtin/replace.c b/builtin/replace.c\nindex 7528f3d..df060f8 100644\n--- a/builtin/replace.c\n+++ b/builtin/replace.c\n@@ -170,8 +170,8 @@ static int replace_object_sha1(const char *object_ref,\n \ttransaction = ref_transaction_begin(&err);\n \tif (!transaction ||\n \t ref_transaction_update(transaction, ref, repl, prev,\n-\t\t\t\t 0, !is_null_sha1(prev), &err) ||\n-\t ref_transaction_commit(transaction, NULL, &err))\n+\t\t\t\t 0, !is_null_sha1(prev), NULL, &err) ||\n+\t ref_transaction_commit(transaction, &err))\n \t\tdie(\"%s\", err.buf);\n \n \tref_transaction_free(transaction);\ndiff --git a/builtin/tag.c b/builtin/tag.c\nindex c9bfc9a..74af63e 100644\n--- a/builtin/tag.c\n+++ b/builtin/tag.c\n@@ -705,8 +705,8 @@ int cmd_tag(int argc, const char **argv, const char *prefix)\n \ttransaction = ref_transaction_begin(&err);\n \tif (!transaction ||\n \t ref_transaction_update(transaction, ref.buf, object, prev,\n-\t\t\t\t 0, !is_null_sha1(prev), &err) ||\n-\t ref_transaction_commit(transaction, NULL, &err))\n+\t\t\t\t 0, !is_null_sha1(prev), NULL, &err) ||\n+\t ref_transaction_commit(transaction, &err))\n \t\tdie(\"%s\", err.buf);\n \tref_transaction_free(transaction);\n \tif (force && !is_null_sha1(prev) && hashcmp(prev, object))\ndiff --git a/builtin/update-ref.c b/builtin/update-ref.c\nindex c6ad0be..28b478a 100644\n--- a/builtin/update-ref.c\n+++ b/builtin/update-ref.c\n@@ -16,6 +16,7 @@ static struct ref_transaction *transaction;\n \n static char line_termination = '\\n';\n static int update_flags;\n+static const char *msg;\n static struct strbuf err = STRBUF_INIT;\n \n /*\n@@ -199,7 +200,7 @@ static const char *parse_cmd_update(struct strbuf *input, const char *next)\n \t\tdie(\"update %s: extra input: %s\", refname, next);\n \n \tif (ref_transaction_update(transaction, refname, new_sha1, old_sha1,\n-\t\t\t\t update_flags, have_old, &err))\n+\t\t\t\t update_flags, have_old, msg, &err))\n \t\tdie(\"%s\", err.buf);\n \n \tupdate_flags = 0;\n@@ -227,7 +228,7 @@ static const char *parse_cmd_create(struct strbuf *input, const char *next)\n \t\tdie(\"create %s: extra input: %s\", refname, next);\n \n \tif (ref_transaction_create(transaction, refname, new_sha1,\n-\t\t\t\t update_flags, &err))\n+\t\t\t\t update_flags, msg, &err))\n \t\tdie(\"%s\", err.buf);\n \n \tupdate_flags = 0;\n@@ -259,7 +260,7 @@ static const char *parse_cmd_delete(struct strbuf *input, const char *next)\n \t\tdie(\"delete %s: extra input: %s\", refname, next);\n \n \tif (ref_transaction_delete(transaction, refname, old_sha1,\n-\t\t\t\t update_flags, have_old, &err))\n+\t\t\t\t update_flags, have_old, msg, &err))\n \t\tdie(\"%s\", err.buf);\n \n \tupdate_flags = 0;\n@@ -292,7 +293,7 @@ static const char *parse_cmd_verify(struct strbuf *input, const char *next)\n \t\tdie(\"verify %s: extra input: %s\", refname, next);\n \n \tif (ref_transaction_update(transaction, refname, new_sha1, old_sha1,\n-\t\t\t\t update_flags, have_old, &err))\n+\t\t\t\t update_flags, have_old, msg, &err))\n \t\tdie(\"%s\", err.buf);\n \n \tupdate_flags = 0;\n@@ -345,7 +346,7 @@ static void update_refs_stdin(void)\n \n int cmd_update_ref(int argc, const char **argv, const char *prefix)\n {\n-\tconst char *refname, *oldval, *msg = NULL;\n+\tconst char *refname, *oldval;\n \tunsigned char sha1[20], oldsha1[20];\n \tint delete = 0, no_deref = 0, read_stdin = 0, end_null = 0, flags = 0;\n \tstruct option options[] = {\n@@ -371,7 +372,7 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)\n \t\tif (end_null)\n \t\t\tline_termination = '\\0';\n \t\tupdate_refs_stdin();\n-\t\tif (ref_transaction_commit(transaction, msg, &err))\n+\t\tif (ref_transaction_commit(transaction, &err))\n \t\t\tdie(\"%s\", err.buf);\n \t\tref_transaction_free(transaction);\n \t\treturn 0;\ndiff --git a/fast-import.c b/fast-import.c\nindex a95e1be..0876db2 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -1708,8 +1708,8 @@ static int update_branch(struct branch *b)\n \ttransaction = ref_transaction_begin(&err);\n \tif (!transaction ||\n \t ref_transaction_update(transaction, b->name, b->sha1, old_sha1,\n-\t\t\t\t 0, 1, &err) ||\n-\t ref_transaction_commit(transaction, msg, &err)) {\n+\t\t\t\t 0, 1, msg, &err) ||\n+\t ref_transaction_commit(transaction, &err)) {\n \t\tref_transaction_free(transaction);\n \t\terror(\"%s\", err.buf);\n \t\tstrbuf_release(&err);\n@@ -1748,12 +1748,12 @@ static void dump_tags(void)\n \t\tstrbuf_addf(&ref_name, \"refs/tags/%s\", t->name);\n \n \t\tif (ref_transaction_update(transaction, ref_name.buf, t->sha1,\n-\t\t\t\t\t NULL, 0, 0, &err)) {\n+\t\t\t\t\t NULL, 0, 0, msg, &err)) {\n \t\t\tfailure |= error(\"%s\", err.buf);\n \t\t\tgoto cleanup;\n \t\t}\n \t}\n-\tif (ref_transaction_commit(transaction, msg, &err))\n+\tif (ref_transaction_commit(transaction, &err))\n \t\tfailure |= error(\"%s\", err.buf);\n \n cleanup:\ndiff --git a/refs.c b/refs.c\nindex c7d1f3e..1f17a13 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -2393,8 +2393,8 @@ static void prune_ref(struct ref_to_prune *r)\n \ttransaction = ref_transaction_begin(&err);\n \tif (!transaction ||\n \t ref_transaction_delete(transaction, r->name, r->sha1,\n-\t\t\t\t REF_ISPRUNING, 1, &err) ||\n-\t ref_transaction_commit(transaction, NULL, &err)) {\n+\t\t\t\t REF_ISPRUNING, 1, NULL, &err) ||\n+\t ref_transaction_commit(transaction, &err)) {\n \t\tref_transaction_free(transaction);\n \t\terror(\"%s\", err.buf);\n \t\tstrbuf_release(&err);\n@@ -2589,8 +2589,8 @@ int delete_ref(const char *refname, const unsigned char *sha1, int delopt)\n \ttransaction = ref_transaction_begin(&err);\n \tif (!transaction ||\n \t ref_transaction_delete(transaction, refname, sha1, delopt,\n-\t\t\t\t sha1 && !is_null_sha1(sha1), &err) ||\n-\t ref_transaction_commit(transaction, NULL, &err)) {\n+\t\t\t\t sha1 && !is_null_sha1(sha1), NULL, &err) ||\n+\t ref_transaction_commit(transaction, &err)) {\n \t\terror(\"%s\", err.buf);\n \t\tref_transaction_free(transaction);\n \t\tstrbuf_release(&err);\n@@ -3367,6 +3367,7 @@ struct ref_update {\n \tint have_old; /* 1 if old_sha1 is valid, 0 otherwise */\n \tstruct ref_lock *lock;\n \tint type;\n+\tchar *msg;\n \tconst char refname[FLEX_ARRAY];\n };\n \n@@ -3414,9 +3415,10 @@ void ref_transaction_free(struct ref_transaction *transaction)\n \tif (!transaction)\n \t\treturn;\n \n-\tfor (i = 0; i < transaction->nr; i++)\n+\tfor (i = 0; i < transaction->nr; i++) {\n+\t\tfree(transaction->updates[i]->msg);\n \t\tfree(transaction->updates[i]);\n-\n+\t}\n \tfree(transaction->updates);\n \tfree(transaction);\n }\n@@ -3437,7 +3439,7 @@ int ref_transaction_update(struct ref_transaction *transaction,\n \t\t\t const char *refname,\n \t\t\t const unsigned char *new_sha1,\n \t\t\t const unsigned char *old_sha1,\n-\t\t\t int flags, int have_old,\n+\t\t\t int flags, int have_old, const char *msg,\n \t\t\t struct strbuf *err)\n {\n \tstruct ref_update *update;\n@@ -3454,13 +3456,15 @@ int ref_transaction_update(struct ref_transaction *transaction,\n \tupdate->have_old = have_old;\n \tif (have_old)\n \t\thashcpy(update->old_sha1, old_sha1);\n+\tif (msg)\n+\t\tupdate->msg = xstrdup(msg);\n \treturn 0;\n }\n \n int ref_transaction_create(struct ref_transaction *transaction,\n \t\t\t const char *refname,\n \t\t\t const unsigned char *new_sha1,\n-\t\t\t int flags,\n+\t\t\t int flags, const char *msg,\n \t\t\t struct strbuf *err)\n {\n \tstruct ref_update *update;\n@@ -3477,13 +3481,15 @@ int ref_transaction_create(struct ref_transaction *transaction,\n \thashclr(update->old_sha1);\n \tupdate->flags = flags;\n \tupdate->have_old = 1;\n+\tif (msg)\n+\t\tupdate->msg = xstrdup(msg);\n \treturn 0;\n }\n \n int ref_transaction_delete(struct ref_transaction *transaction,\n \t\t\t const char *refname,\n \t\t\t const unsigned char *old_sha1,\n-\t\t\t int flags, int have_old,\n+\t\t\t int flags, int have_old, const char *msg,\n \t\t\t struct strbuf *err)\n {\n \tstruct ref_update *update;\n@@ -3501,6 +3507,8 @@ int ref_transaction_delete(struct ref_transaction *transaction,\n \t\tassert(!is_null_sha1(old_sha1));\n \t\thashcpy(update->old_sha1, old_sha1);\n \t}\n+\tif (msg)\n+\t\tupdate->msg = xstrdup(msg);\n \treturn 0;\n }\n \n@@ -3514,8 +3522,8 @@ int update_ref(const char *action, const char *refname,\n \tt = ref_transaction_begin(&err);\n \tif (!t ||\n \t ref_transaction_update(t, refname, sha1, oldval, flags,\n-\t\t\t\t !!oldval, &err) ||\n-\t ref_transaction_commit(t, action, &err)) {\n+\t\t\t\t !!oldval, action, &err) ||\n+\t ref_transaction_commit(t, &err)) {\n \t\tconst char *str = \"update_ref failed for ref '%s': %s\";\n \n \t\tref_transaction_free(t);\n@@ -3559,7 +3567,7 @@ static int ref_update_reject_duplicates(struct ref_update **updates, int n,\n }\n \n int ref_transaction_commit(struct ref_transaction *transaction,\n-\t\t\t const char *msg, struct strbuf *err)\n+\t\t\t struct strbuf *err)\n {\n \tint ret = 0, delnum = 0, i;\n \tconst char **delnames;\n@@ -3608,7 +3616,7 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n \n \t\tif (!is_null_sha1(update->new_sha1)) {\n \t\t\tret = write_ref_sha1(update->lock, update->new_sha1,\n-\t\t\t\t\t msg);\n+\t\t\t\t\t update->msg);\n \t\t\tupdate->lock = NULL; /* freed by write_ref_sha1 */\n \t\t\tif (ret) {\n \t\t\t\tconst char *str = \"Cannot update the ref '%s'.\";\ndiff --git a/refs.h b/refs.h\nindex 3b321c2..f24b2c1 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -297,7 +297,7 @@ int ref_transaction_update(struct ref_transaction *transaction,\n \t\t\t const char *refname,\n \t\t\t const unsigned char *new_sha1,\n \t\t\t const unsigned char *old_sha1,\n-\t\t\t int flags, int have_old,\n+\t\t\t int flags, int have_old, const char *msg,\n \t\t\t struct strbuf *err);\n \n /*\n@@ -312,7 +312,7 @@ int ref_transaction_update(struct ref_transaction *transaction,\n int ref_transaction_create(struct ref_transaction *transaction,\n \t\t\t const char *refname,\n \t\t\t const unsigned char *new_sha1,\n-\t\t\t int flags,\n+\t\t\t int flags, const char *msg,\n \t\t\t struct strbuf *err);\n \n /*\n@@ -326,7 +326,7 @@ int ref_transaction_create(struct ref_transaction *transaction,\n int ref_transaction_delete(struct ref_transaction *transaction,\n \t\t\t const char *refname,\n \t\t\t const unsigned char *old_sha1,\n-\t\t\t int flags, int have_old,\n+\t\t\t int flags, int have_old, const char *msg,\n \t\t\t struct strbuf *err);\n \n /*\n@@ -335,7 +335,7 @@ int ref_transaction_delete(struct ref_transaction *transaction,\n * problem.\n */\n int ref_transaction_commit(struct ref_transaction *transaction,\n-\t\t\t const char *msg, struct strbuf *err);\n+\t\t\t struct strbuf *err);\n \n /*\n * Free an existing transaction and all associated data.\ndiff --git a/sequencer.c b/sequencer.c\nindex fd8acaf..f9906ef 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -285,8 +285,8 @@ static int fast_forward_to(const unsigned char *to, const unsigned char *from,\n \ttransaction = ref_transaction_begin(&err);\n \tif (!transaction ||\n \t ref_transaction_update(transaction, \"HEAD\", to, from,\n-\t\t\t\t 0, !unborn, &err) ||\n-\t ref_transaction_commit(transaction, sb.buf, &err)) {\n+\t\t\t\t 0, !unborn, sb.buf, &err) ||\n+\t ref_transaction_commit(transaction, &err)) {\n \t\tref_transaction_free(transaction);\n \t\terror(\"%s\", err.buf);\n \t\tstrbuf_release(&sb);\ndiff --git a/walker.c b/walker.c\nindex 60d9f9e..fd9ef87 100644\n--- a/walker.c\n+++ b/walker.c\n@@ -295,15 +295,14 @@ int walker_fetch(struct walker *walker, int targets, char **target,\n \t\tstrbuf_addf(&ref_name, \"refs/%s\", write_ref[i]);\n \t\tif (ref_transaction_update(transaction, ref_name.buf,\n \t\t\t\t\t &sha1[20 * i], NULL, 0, 0,\n+\t\t\t\t\t msg ? msg : \"fetch (unknown)\",\n \t\t\t\t\t &err)) {\n \t\t\terror(\"%s\", err.buf);\n \t\t\tgoto rollback_and_fail;\n \t\t}\n \t}\n \tif (write_ref) {\n-\t\tif (ref_transaction_commit(transaction,\n-\t\t\t\t\t msg ? msg : \"fetch (unknown)\",\n-\t\t\t\t\t &err)) {\n+\t\tif (ref_transaction_commit(transaction, &err)) {\n \t\t\terror(\"%s\", err.buf);\n \t\t\tgoto rollback_and_fail;\n \t\t}\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244770","messageId":"1403275409-28173-43-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 42/48] refs.c: pass NULL as *flags to read_ref_full","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:23Z","receivedAt":"2014-06-20T14:43:23Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"We call read_ref_full with a pointer to flags from rename_ref but since\nwe never actually use the returned flags we can just pass NULL here instead.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/refs.c b/refs.c\nindex 1f17a13..389a55f 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -2688,7 +2688,7 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms\n \t\tgoto rollback;\n \t}\n \n-\tif (!read_ref_full(newrefname, sha1, 1, &flag) &&\n+\tif (!read_ref_full(newrefname, sha1, 1, NULL) &&\n \t delete_ref(newrefname, sha1, REF_NODEREF)) {\n \t\tif (errno==EISDIR) {\n \t\t\tif (remove_empty_directories(git_path(\"%s\", newrefname))) {\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244772","messageId":"1403275409-28173-44-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 43/48] refs.c: move the check for valid refname to lock_ref_sha1_basic","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:24Z","receivedAt":"2014-06-20T14:43:24Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Move the check for check_refname_format from lock_any_ref_for_update\nto lock_ref_sha1_basic. At some later stage we will get rid of\nlock_any_ref_for_update completely.\n\nIf lock_ref_sha1_basic fails the check_refname_format test, set errno to\nEINVAL before returning NULL. This to guarantee that we will not return an\nerror without updating errno.\n\nThis leaves lock_any_ref_for_updates as a no-op wrapper which could be removed.\nBut this wrapper is also called from an external caller and we will soon\nmake changes to the signature to lock_ref_sha1_basic that we do not want to\nexpose to that caller.\n\nThis changes semantics for lock_ref_sha1_basic slightly. With this change\nit is no longer possible to open a ref that has a badly name which breaks\nany codepaths that tries to open and repair badly named refs. The normal refs\nAPI should not allow neither creating nor accessing refs with invalid names.\nIf we need such recovery code we could add it as an option to git fsck and have\ngit fsck be the only sanctioned way of bypassing the normal API and checks.\n\nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 7 +++++--\n 1 file changed, 5 insertions(+), 2 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 389a55f..bccf8c3 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -2088,6 +2088,11 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,\n \tint missing = 0;\n \tint attempts_remaining = 3;\n \n+\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n+\t\terrno = EINVAL;\n+\t\treturn NULL;\n+\t}\n+\n \tlock = xcalloc(1, sizeof(struct ref_lock));\n \tlock->lock_fd = -1;\n \n@@ -2179,8 +2184,6 @@ struct ref_lock *lock_any_ref_for_update(const char *refname,\n \t\t\t\t\t const unsigned char *old_sha1,\n \t\t\t\t\t int flags, int *type_p)\n {\n-\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL))\n-\t\treturn NULL;\n \treturn lock_ref_sha1_basic(refname, old_sha1, flags, type_p);\n }\n \n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244773","messageId":"1403275409-28173-45-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 44/48] refs.c: call lock_ref_sha1_basic directly from commit","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:25Z","receivedAt":"2014-06-20T14:43:25Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Skip using the lock_any_ref_for_update wrapper and call lock_ref_sha1_basic\ndirectly from the commit function.\n\nReviewed-by: Jonathan Nieder \nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 12 ++++++------\n 1 file changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex bccf8c3..f3fab37 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -3598,12 +3598,12 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n \tfor (i = 0; i < n; i++) {\n \t\tstruct ref_update *update = updates[i];\n \n-\t\tupdate->lock = lock_any_ref_for_update(update->refname,\n-\t\t\t\t\t\t (update->have_old ?\n-\t\t\t\t\t\t\tupdate->old_sha1 :\n-\t\t\t\t\t\t\tNULL),\n-\t\t\t\t\t\t update->flags,\n-\t\t\t\t\t\t &update->type);\n+\t\tupdate->lock = lock_ref_sha1_basic(update->refname,\n+\t\t\t\t\t\t (update->have_old ?\n+\t\t\t\t\t\t update->old_sha1 :\n+\t\t\t\t\t\t NULL),\n+\t\t\t\t\t\t update->flags,\n+\t\t\t\t\t\t &update->type);\n \t\tif (!update->lock) {\n \t\t\tif (err)\n \t\t\t\tstrbuf_addf(err, \"Cannot lock the ref '%s'.\",\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244775","messageId":"1403275409-28173-46-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 45/48] refs.c: pass a skip list to name_conflict_fn","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:26Z","receivedAt":"2014-06-20T14:43:26Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Allow passing a list of refs to skip checking to name_conflict_fn.\nThere are some conditions where we want to allow a temporary conflict and skip\nchecking those refs. For example if we have a transaction that\n1, guarantees that m is a packed refs and there is no loose ref for m\n2, the transaction will delete m from the packed ref\n3, the transaction will create conflicting m/m\n\nFor this case we want to be able to lock and create m/m since we know that the\nconflict is only transient. I.e. the conflict will be automatically resolved\nby the transaction when it deletes m.\n\nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 41 ++++++++++++++++++++++++++---------------\n 1 file changed, 26 insertions(+), 15 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex f3fab37..c6990d0 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -801,15 +801,18 @@ static int names_conflict(const char *refname1, const char *refname2)\n \n struct name_conflict_cb {\n \tconst char *refname;\n-\tconst char *oldrefname;\n \tconst char *conflicting_refname;\n+\tconst char **skip;\n+\tint skipnum;\n };\n \n static int name_conflict_fn(struct ref_entry *entry, void *cb_data)\n {\n \tstruct name_conflict_cb *data = (struct name_conflict_cb *)cb_data;\n-\tif (data->oldrefname && !strcmp(data->oldrefname, entry->name))\n-\t\treturn 0;\n+\tint i;\n+\tfor (i = 0; i < data->skipnum; i++)\n+\t\tif (!strcmp(entry->name, data->skip[i]))\n+\t\t\treturn 0;\n \tif (names_conflict(data->refname, entry->name)) {\n \t\tdata->conflicting_refname = entry->name;\n \t\treturn 1;\n@@ -822,15 +825,18 @@ static int name_conflict_fn(struct ref_entry *entry, void *cb_data)\n * conflicting with the name of an existing reference in dir. If\n * oldrefname is non-NULL, ignore potential conflicts with oldrefname\n * (e.g., because oldrefname is scheduled for deletion in the same\n- * operation).\n+ * operation). skip contains a list of refs we want to skip checking for\n+ * conflicts with.\n */\n-static int is_refname_available(const char *refname, const char *oldrefname,\n-\t\t\t\tstruct ref_dir *dir)\n+static int is_refname_available(const char *refname,\n+\t\t\t\tstruct ref_dir *dir,\n+\t\t\t\tconst char **skip, int skipnum)\n {\n \tstruct name_conflict_cb data;\n \tdata.refname = refname;\n-\tdata.oldrefname = oldrefname;\n \tdata.conflicting_refname = NULL;\n+\tdata.skip = skip;\n+\tdata.skipnum = skipnum;\n \n \tsort_ref_dir(dir);\n \tif (do_for_each_entry_in_dir(dir, 0, name_conflict_fn, &data)) {\n@@ -2077,7 +2083,8 @@ int dwim_log(const char *str, int len, unsigned char *sha1, char **log)\n /* This function should make sure errno is meaningful on error */\n static struct ref_lock *lock_ref_sha1_basic(const char *refname,\n \t\t\t\t\t const unsigned char *old_sha1,\n-\t\t\t\t\t int flags, int *type_p)\n+\t\t\t\t\t int flags, int *type_p,\n+\t\t\t\t\t const char **skip, int skipnum)\n {\n \tchar *ref_file;\n \tconst char *orig_refname = refname;\n@@ -2126,7 +2133,8 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,\n \t * name is a proper prefix of our refname.\n \t */\n \tif (missing &&\n-\t !is_refname_available(refname, NULL, get_packed_refs(&ref_cache))) {\n+\t !is_refname_available(refname, get_packed_refs(&ref_cache),\n+\t\t\t\t skip, skipnum)) {\n \t\tlast_errno = ENOTDIR;\n \t\tgoto error_return;\n \t}\n@@ -2184,7 +2192,7 @@ struct ref_lock *lock_any_ref_for_update(const char *refname,\n \t\t\t\t\t const unsigned char *old_sha1,\n \t\t\t\t\t int flags, int *type_p)\n {\n-\treturn lock_ref_sha1_basic(refname, old_sha1, flags, type_p);\n+\treturn lock_ref_sha1_basic(refname, old_sha1, flags, type_p, NULL, 0);\n }\n \n /*\n@@ -2676,10 +2684,12 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms\n \tif (!symref)\n \t\treturn error(\"refname %s not found\", oldrefname);\n \n-\tif (!is_refname_available(newrefname, oldrefname, get_packed_refs(&ref_cache)))\n+\tif (!is_refname_available(newrefname, get_packed_refs(&ref_cache),\n+\t\t\t\t &oldrefname, 1))\n \t\treturn 1;\n \n-\tif (!is_refname_available(newrefname, oldrefname, get_loose_refs(&ref_cache)))\n+\tif (!is_refname_available(newrefname, get_loose_refs(&ref_cache),\n+\t\t\t\t &oldrefname, 1))\n \t\treturn 1;\n \n \tif (log && rename(git_path(\"logs/%s\", oldrefname), git_path(TMP_RENAMED_LOG)))\n@@ -2709,7 +2719,7 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms\n \n \tlogmoved = log;\n \n-\tlock = lock_ref_sha1_basic(newrefname, NULL, 0, NULL);\n+\tlock = lock_ref_sha1_basic(newrefname, NULL, 0, NULL, NULL, 0);\n \tif (!lock) {\n \t\terror(\"unable to lock %s for update\", newrefname);\n \t\tgoto rollback;\n@@ -2724,7 +2734,7 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms\n \treturn 0;\n \n rollback:\n-\tlock = lock_ref_sha1_basic(oldrefname, NULL, 0, NULL);\n+\tlock = lock_ref_sha1_basic(oldrefname, NULL, 0, NULL, NULL, 0);\n \tif (!lock) {\n \t\terror(\"unable to lock %s for rollback\", oldrefname);\n \t\tgoto rollbacklog;\n@@ -3603,7 +3613,8 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n \t\t\t\t\t\t update->old_sha1 :\n \t\t\t\t\t\t NULL),\n \t\t\t\t\t\t update->flags,\n-\t\t\t\t\t\t &update->type);\n+\t\t\t\t\t\t &update->type,\n+\t\t\t\t\t\t delnames, delnum);\n \t\tif (!update->lock) {\n \t\t\tif (err)\n \t\t\t\tstrbuf_addf(err, \"Cannot lock the ref '%s'.\",\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244774","messageId":"1403275409-28173-47-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 46/48] refs.c: propagate any errno==ENOTDIR from _commit back to the callers","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:27Z","receivedAt":"2014-06-20T14:43:27Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"In _commit, ENOTDIR can happen in the call to lock_ref_sha1_basic, either when\nwe lstat the new refname and it returns ENOTDIR or if the name checking\nfunction reports that the same type of conflict happened. In both cases it\nmeans that we can not create the new ref due to a name conflict.\n\nFor these cases, save the errno value and abort and make sure that the caller\ncan see errno==ENOTDIR.\n\nAlso start defining specific return codes for _commit, assign -1 as a generic\nerror and -2 as the error that refers to a name conflict. Callers can (and\nshould) use that return value inspecting errno directly.\n\nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 22 +++++++++++++++-------\n refs.h | 6 ++++++\n 2 files changed, 21 insertions(+), 7 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex c6990d0..74a55b5 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -3582,7 +3582,7 @@ static int ref_update_reject_duplicates(struct ref_update **updates, int n,\n int ref_transaction_commit(struct ref_transaction *transaction,\n \t\t\t struct strbuf *err)\n {\n-\tint ret = 0, delnum = 0, i;\n+\tint ret = 0, delnum = 0, i, df_conflict = 0;\n \tconst char **delnames;\n \tint n = transaction->nr;\n \tstruct ref_update **updates = transaction->updates;\n@@ -3600,9 +3600,10 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n \n \t/* Copy, sort, and reject duplicate refs */\n \tqsort(updates, n, sizeof(*updates), ref_update_compare);\n-\tret = ref_update_reject_duplicates(updates, n, err);\n-\tif (ret)\n+\tif (ref_update_reject_duplicates(updates, n, err)) {\n+\t\tret = -1;\n \t\tgoto cleanup;\n+\t}\n \n \t/* Acquire all locks while verifying old values */\n \tfor (i = 0; i < n; i++) {\n@@ -3616,10 +3617,12 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n \t\t\t\t\t\t &update->type,\n \t\t\t\t\t\t delnames, delnum);\n \t\tif (!update->lock) {\n+\t\t\tif (errno == ENOTDIR)\n+\t\t\t\tdf_conflict = 1;\n \t\t\tif (err)\n \t\t\t\tstrbuf_addf(err, \"Cannot lock the ref '%s'.\",\n \t\t\t\t\t update->refname);\n-\t\t\tret = 1;\n+\t\t\tret = -1;\n \t\t\tgoto cleanup;\n \t\t}\n \t}\n@@ -3637,6 +3640,7 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n \n \t\t\t\tif (err)\n \t\t\t\t\tstrbuf_addf(err, str, update->refname);\n+\t\t\t\tret = -1;\n \t\t\t\tgoto cleanup;\n \t\t\t}\n \t\t}\n@@ -3647,14 +3651,16 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n \t\tstruct ref_update *update = updates[i];\n \n \t\tif (update->lock) {\n-\t\t\tret |= delete_ref_loose(update->lock, update->type,\n-\t\t\t\t\t\terr);\n+\t\t\tif (delete_ref_loose(update->lock, update->type, err))\n+\t\t\t\tret = -1;\n+\n \t\t\tif (!(update->flags & REF_ISPRUNING))\n \t\t\t\tdelnames[delnum++] = update->lock->ref_name;\n \t\t}\n \t}\n \n-\tret |= repack_without_refs(delnames, delnum, err);\n+\tif (repack_without_refs(delnames, delnum, err))\n+\t\tret = -1;\n \tfor (i = 0; i < delnum; i++)\n \t\tunlink_or_warn(git_path(\"logs/%s\", delnames[i]));\n \tclear_loose_ref_cache(&ref_cache);\n@@ -3667,6 +3673,8 @@ cleanup:\n \t\tif (updates[i]->lock)\n \t\t\tunlock_ref(updates[i]->lock);\n \tfree(delnames);\n+\tif (df_conflict)\n+\t\tret = -2;\n \treturn ret;\n }\n \ndiff --git a/refs.h b/refs.h\nindex f24b2c1..5c0543d 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -333,7 +333,13 @@ int ref_transaction_delete(struct ref_transaction *transaction,\n * Commit all of the changes that have been queued in transaction, as\n * atomically as possible. Return a nonzero value if there is a\n * problem.\n+ * If the transaction is already in failed state this function will return\n+ * an error.\n+ * Function returns 0 on success, -1 for generic failures and\n+ * UPDATE_REFS_NAME_CONFLICT (-2) if the failure was due to a name\n+ * collision (ENOTDIR).\n */\n+#define UPDATE_REFS_NAME_CONFLICT -2\n int ref_transaction_commit(struct ref_transaction *transaction,\n \t\t\t struct strbuf *err);\n \n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244771","messageId":"1403275409-28173-48-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 47/48] fetch.c: change s_update_ref to use a ref transaction","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:28Z","receivedAt":"2014-06-20T14:43:28Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Change s_update_ref to use a ref transaction for the ref update.\n\nSigned-off-by: Ronnie Sahlberg \n---\n builtin/fetch.c | 33 +++++++++++++++++++++++----------\n 1 file changed, 23 insertions(+), 10 deletions(-)\n\ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex faa1233..52f1ebc 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -375,23 +375,36 @@ static int s_update_ref(const char *action,\n {\n \tchar msg[1024];\n \tchar *rla = getenv(\"GIT_REFLOG_ACTION\");\n-\tstatic struct ref_lock *lock;\n+\tstruct ref_transaction *transaction;\n+\tstruct strbuf err = STRBUF_INIT;\n+\tint ret, df_conflict = 0;\n \n \tif (dry_run)\n \t\treturn 0;\n \tif (!rla)\n \t\trla = default_rla.buf;\n \tsnprintf(msg, sizeof(msg), \"%s: %s\", rla, action);\n-\tlock = lock_any_ref_for_update(ref->name,\n-\t\t\t\t check_old ? ref->old_sha1 : NULL,\n-\t\t\t\t 0, NULL);\n-\tif (!lock)\n-\t\treturn errno == ENOTDIR ? STORE_REF_ERROR_DF_CONFLICT :\n-\t\t\t\t\t STORE_REF_ERROR_OTHER;\n-\tif (write_ref_sha1(lock, ref->new_sha1, msg) < 0)\n-\t\treturn errno == ENOTDIR ? STORE_REF_ERROR_DF_CONFLICT :\n-\t\t\t\t\t STORE_REF_ERROR_OTHER;\n+\n+\ttransaction = ref_transaction_begin(&err);\n+\tif (!transaction ||\n+\t ref_transaction_update(transaction, ref->name, ref->new_sha1,\n+\t\t\t\t ref->old_sha1, 0, check_old, msg, &err))\n+\t\tgoto fail;\n+\n+\tret = ref_transaction_commit(transaction, &err);\n+\tif (ret == UPDATE_REFS_NAME_CONFLICT)\n+\t\tdf_conflict = 1;\n+\tif (ret)\n+\t\tgoto fail;\n+\n+\tref_transaction_free(transaction);\n \treturn 0;\n+fail:\n+\tref_transaction_free(transaction);\n+\terror(\"%s\", err.buf);\n+\tstrbuf_release(&err);\n+\treturn df_conflict ? STORE_REF_ERROR_DF_CONFLICT\n+\t\t\t : STORE_REF_ERROR_OTHER;\n }\n \n #define REFCOL_WIDTH 10\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"244767","messageId":"1403275409-28173-49-git-send-email-sahlberg@google.com","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"[PATCH v20 48/48] refs.c: make write_ref_sha1 static","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-06-20T14:43:29Z","receivedAt":"2014-06-20T14:43:29Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"No external users call write_ref_sha1 any more so lets declare it static.\n\nSigned-off-by: Ronnie Sahlberg \n---\n refs.c | 10 ++++++++--\n refs.h | 3 ---\n 2 files changed, 8 insertions(+), 5 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 74a55b5..5a0b138 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -2665,6 +2665,9 @@ static int rename_tmp_log(const char *newrefname)\n \treturn 0;\n }\n \n+static int write_ref_sha1(struct ref_lock *lock, const unsigned char *sha1,\n+\t\t\t const char *logmsg);\n+\n int rename_ref(const char *oldrefname, const char *newrefname, const char *logmsg)\n {\n \tunsigned char sha1[20], orig_sha1[20];\n@@ -2914,8 +2917,11 @@ static int is_branch(const char *refname)\n \treturn !strcmp(refname, \"HEAD\") || starts_with(refname, \"refs/heads/\");\n }\n \n-/* This function must return a meaningful errno */\n-int write_ref_sha1(struct ref_lock *lock,\n+/*\n+ * Writes sha1 into the ref specified by the lock. Makes sure that errno\n+ * is sane on error.\n+ */\n+static int write_ref_sha1(struct ref_lock *lock,\n \tconst unsigned char *sha1, const char *logmsg)\n {\n \tstatic char term = '\\n';\ndiff --git a/refs.h b/refs.h\nindex 5c0543d..db463d0 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -203,9 +203,6 @@ extern int commit_ref(struct ref_lock *lock);\n /** Release any lock taken but not written. **/\n extern void unlock_ref(struct ref_lock *lock);\n \n-/** Writes sha1 into the ref specified by the lock. **/\n-extern int write_ref_sha1(struct ref_lock *lock, const unsigned char *sha1, const char *msg);\n-\n /*\n * Setup reflog before using. Set errno to something meaningful on failure.\n */\n-- \n2.0.0.420.g181e020.dirty\n"},{"id":"245015","messageId":"53ABEDE0.9070501@gmail.com","threadId":"36972","inReplyTo":"1403275409-28173-14-git-send-email-sahlberg@google.com","subject":"Re: [PATCH v20 13/48] refs.c: make resolve_ref_unsafe set errno to something meaningful on error","fromName":"Karsten Blees","fromEmail":"karsten.blees@gmail.com","sentAt":"2014-06-26T09:54:40Z","receivedAt":"2014-06-26T09:54:40Z","isPatch":true,"sender":{"key":"karsten.blees@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1111200?v=4"},"body":"Am 20.06.2014 16:42, schrieb Ronnie Sahlberg:\n> +\t\t\terrno = ELOOP;\n\nThis fails on MinGW and MSVC < 2010. Perhaps add this to compat/mingw.h?\n\n #ifndef ELOOP\n #define ELOOP EMLINK\n #endif\n"},{"id":"245518","messageId":"53BBDA5D.7090608@alum.mit.edu","threadId":"36972","inReplyTo":"1403275409-28173-8-git-send-email-sahlberg@google.com","subject":"Re: [PATCH v20 07/48] lockfile.c: make lock_file return a meaningful errno on failurei","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2014-07-08T11:47:41Z","receivedAt":"2014-07-08T11:47:41Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 06/20/2014 04:42 PM, Ronnie Sahlberg wrote:\n> Making errno when returning from lock_file() meaningful, which should\n> fix\n> \n> * an existing almost-bug in lock_ref_sha1_basic where it assumes\n> errno==ENOENT is meaningful and could waste some work on retries\n> \n> * an existing bug in repack_without_refs where it prints\n> strerror(errno) and picks advice based on errno, despite errno\n> potentially being zero and potentially having been clobbered by\n> that point\n> [...]\n\nTypo in subject line:\n\ns/failurei/failure/\n\nMichael\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\n"},{"id":"245520","messageId":"53BBDAA4.7020701@alum.mit.edu","threadId":"36972","inReplyTo":"1403275409-28173-21-git-send-email-sahlberg@google.com","subject":"Re: [PATCH v20 20/48] refs.c: change ref_transaction_create to do error checking and return status","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2014-07-08T11:48:52Z","receivedAt":"2014-07-08T11:48:52Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"I'm in my next attempt to get through your patch series. Sorry for the\nlong hiatus.\n\nPatches 1-19 look OK aside from a minor typo that I just reported.\n\nSee below for a comment on this patch.\n\nOn 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n> Do basic error checking in ref_transaction_create() and make it return\n> non-zero on error. Update all callers to check the result of\n> ref_transaction_create(). There are currently no conditions in _create that\n> will return error but there will be in the future. Add an err argument that\n> will be updated on failure.\n> \n> Reviewed-by: Jonathan Nieder \n> Signed-off-by: Ronnie Sahlberg \n> ---\n> builtin/update-ref.c | 4 +++-\n> refs.c | 18 +++++++++++------\n> refs.h | 55 +++++++++++++++++++++++++++++++++++++++++++++-------\n> 3 files changed, 63 insertions(+), 14 deletions(-)\n> \n> diff --git a/builtin/update-ref.c b/builtin/update-ref.c\n> index 3067b11..41121fa 100644\n> --- a/builtin/update-ref.c\n> +++ b/builtin/update-ref.c\n> @@ -226,7 +226,9 @@ static const char *parse_cmd_create(struct strbuf *input, const char *next)\n> \tif (*next != line_termination)\n> \t\tdie(\"create %s: extra input: %s\", refname, next);\n> \n> -\tref_transaction_create(transaction, refname, new_sha1, update_flags);\n> +\tif (ref_transaction_create(transaction, refname, new_sha1,\n> +\t\t\t\t update_flags, &err))\n> +\t\tdie(\"%s\", err.buf);\n> \n> \tupdate_flags = 0;\n> \tfree(refname);\n> diff --git a/refs.c b/refs.c\n> index 3f05e88..c49f1c6 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -3449,18 +3449,24 @@ int ref_transaction_update(struct ref_transaction *transaction,\n> \treturn 0;\n> }\n> \n> -void ref_transaction_create(struct ref_transaction *transaction,\n> -\t\t\t const char *refname,\n> -\t\t\t const unsigned char *new_sha1,\n> -\t\t\t int flags)\n> +int ref_transaction_create(struct ref_transaction *transaction,\n> +\t\t\t const char *refname,\n> +\t\t\t const unsigned char *new_sha1,\n> +\t\t\t int flags,\n> +\t\t\t struct strbuf *err)\n> {\n> -\tstruct ref_update *update = add_update(transaction, refname);\n> +\tstruct ref_update *update;\n> +\n> +\tif (!new_sha1 || is_null_sha1(new_sha1))\n> +\t\tdie(\"BUG: create ref with null new_sha1\");\n> +\n> +\tupdate = add_update(transaction, refname);\n> \n> -\tassert(!is_null_sha1(new_sha1));\n> \thashcpy(update->new_sha1, new_sha1);\n> \thashclr(update->old_sha1);\n> \tupdate->flags = flags;\n> \tupdate->have_old = 1;\n> +\treturn 0;\n> }\n> \n> void ref_transaction_delete(struct ref_transaction *transaction,\n> diff --git a/refs.h b/refs.h\n> index c5376ce..33b4383 100644\n> --- a/refs.h\n> +++ b/refs.h\n> @@ -10,6 +10,45 @@ struct ref_lock {\n> \tint force_write;\n> };\n> \n> +/*\n> + * A ref_transaction represents a collection of ref updates\n> + * that should succeed or fail together.\n> + *\n> + * Calling sequence\n> + * ----------------\n> + * - Allocate and initialize a `struct ref_transaction` by calling\n> + * `ref_transaction_begin()`.\n> + *\n> + * - List intended ref updates by calling functions like\n> + * `ref_transaction_update()` and `ref_transaction_create()`.\n> + *\n> + * - Call `ref_transaction_commit()` to execute the transaction.\n> + * If this succeeds, the ref updates will have taken place and\n> + * the transaction cannot be rolled back.\n> + *\n> + * - At any time call `ref_transaction_free()` to discard the\n> + * transaction and free associated resources. In particular,\n> + * this rolls back the transaction if it has not been\n> + * successfully committed.\n> + *\n> + * Error handling\n> + * --------------\n> + *\n> + * On error, transaction functions append a message about what\n> + * went wrong to the 'err' argument. The message mentions what\n> + * ref was being updated (if any) when the error occurred so it\n> + * can be passed to 'die' or 'error' as-is.\n> + *\n> + * The message is appended to err without first clearing err.\n> + * This allows the caller to prepare preamble text to the generated\n> + * error message:\n> + *\n> + * strbuf_addf(&err, \"Error while doing foo-bar: \");\n> + * if (ref_transaction_update(..., &err)) {\n> + * ret = error(\"%s\", err.buf);\n> + * goto cleanup;\n> + * }\n> + */\n\nI don't have a problem with the API, but I think the idiom suggested in\nthe comment above is a bit silly. Surely one would do the following\ninstead:\n\n if (ref_transaction_update(..., &err)) {\n ret = error(\"Error while doing foo-bar: %s\", err.buf);\n goto cleanup;\n }\n\nI think it would also be helpful to document whether the error string\nthat is appended to the strbuf is terminated with a LF.\n\n> [...]\n\nMichael\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\n"},{"id":"245519","messageId":"53BBDBA0.5070308@alum.mit.edu","threadId":"36972","inReplyTo":"1403275409-28173-23-git-send-email-sahlberg@google.com","subject":"Re: [PATCH v20 22/48] refs.c: make ref_transaction_begin take an err argument","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2014-07-08T11:53:04Z","receivedAt":"2014-07-08T11:53:04Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n> Add an err argument to _begin so that on non-fatal failures in future ref\n> backends we can report a nice error back to the caller.\n> While _begin can currently never fail for other reasons than OOM, in which\n> case we die() anyway, we may add other types of backends in the future.\n> For example, a hypothetical MySQL backend could fail in _being with\n\ns/_being/_begin/\n\n> [...]\n\nMichael\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\n"},{"id":"245521","messageId":"53BBDD4F.20403@alum.mit.edu","threadId":"36972","inReplyTo":"1403275409-28173-24-git-send-email-sahlberg@google.com","subject":"Re: [PATCH v20 23/48] refs.c: add transaction.status and track OPEN/CLOSED/ERROR","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2014-07-08T12:00:15Z","receivedAt":"2014-07-08T12:00:15Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n> Track the status of a transaction in a new status field. Check the field for\n\nThe status field is not set or used anywhere. The field that you use is\n\"state\".\n\n> sanity, i.e. that status must be OPEN when _commit/_create/_delete or\n> _update is called or else die(BUG:...)\n> \n> Signed-off-by: Ronnie Sahlberg \n> ---\n> refs.c | 40 +++++++++++++++++++++++++++++++++++++++-\n> 1 file changed, 39 insertions(+), 1 deletion(-)\n> \n> diff --git a/refs.c b/refs.c\n> index 9cb7908..8c695ba 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -3387,6 +3387,25 @@ struct ref_update {\n> };\n> \n> /*\n> + * Transaction states.\n> + * OPEN: The transaction is in a valid state and can accept new updates.\n> + * An OPEN transaction can be committed.\n> + * CLOSED: If an open transaction is successfully committed the state will\n> + * change to CLOSED. No further changes can be made to a CLOSED\n> + * transaction.\n> + * CLOSED means that all updates have been successfully committed and\n> + * the only thing that remains is to free the completed transaction.\n> + * ERROR: The transaction has failed and is no longer committable.\n> + * No further changes can be made to a CLOSED transaction and it must\n> + * be rolled back using transaction_free.\n> + */\n> +enum ref_transaction_state {\n> +\tREF_TRANSACTION_OPEN = 0,\n> +\tREF_TRANSACTION_CLOSED = 1,\n> +\tREF_TRANSACTION_ERROR = 2,\n> +};\n> +\n> +/*\n> * Data structure for holding a reference transaction, which can\n> * consist of checks and updates to multiple references, carried out\n> * as atomically as possible. This structure is opaque to callers.\n> @@ -3395,6 +3414,8 @@ struct ref_transaction {\n> \tstruct ref_update **updates;\n> \tsize_t alloc;\n> \tsize_t nr;\n> +\tenum ref_transaction_state state;\n> +\tint status;\n\nThe status field should probably be deleted.\n\n> };\n> \n> struct ref_transaction *ref_transaction_begin(struct strbuf *err)\n> @@ -3437,6 +3458,9 @@ int ref_transaction_update(struct ref_transaction *transaction,\n> {\n> \tstruct ref_update *update;\n> \n> +\tif (transaction->state != REF_TRANSACTION_OPEN)\n> +\t\tdie(\"BUG: update called for transaction that is not open\");\n> +\n> \tif (have_old && !old_sha1)\n> \t\tdie(\"BUG: have_old is true but old_sha1 is NULL\");\n> \n> @@ -3457,6 +3481,9 @@ int ref_transaction_create(struct ref_transaction *transaction,\n> {\n> \tstruct ref_update *update;\n> \n> +\tif (transaction->state != REF_TRANSACTION_OPEN)\n> +\t\tdie(\"BUG: create called for transaction that is not open\");\n> +\n> \tif (!new_sha1 || is_null_sha1(new_sha1))\n> \t\tdie(\"BUG: create ref with null new_sha1\");\n> \n> @@ -3477,6 +3504,9 @@ int ref_transaction_delete(struct ref_transaction *transaction,\n> {\n> \tstruct ref_update *update;\n> \n> +\tif (transaction->state != REF_TRANSACTION_OPEN)\n> +\t\tdie(\"BUG: delete called for transaction that is not open\");\n> +\n> \tif (have_old && !old_sha1)\n> \t\tdie(\"BUG: have_old is true but old_sha1 is NULL\");\n> \n> @@ -3532,8 +3562,13 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n> \tint n = transaction->nr;\n> \tstruct ref_update **updates = transaction->updates;\n> \n> -\tif (!n)\n> +\tif (transaction->state != REF_TRANSACTION_OPEN)\n> +\t\tdie(\"BUG: commit called for transaction that is not open\");\n> +\n> +\tif (!n) {\n> +\t\ttransaction->state = REF_TRANSACTION_CLOSED;\n> \t\treturn 0;\n> +\t}\n> \n> \t/* Allocate work space */\n> \tdelnames = xmalloc(sizeof(*delnames) * n);\n> @@ -3595,6 +3630,9 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n> \tclear_loose_ref_cache(&ref_cache);\n> \n> cleanup:\n> +\ttransaction->state = ret ? REF_TRANSACTION_ERROR\n> +\t\t: REF_TRANSACTION_CLOSED;\n> +\n> \tfor (i = 0; i < n; i++)\n> \t\tif (updates[i]->lock)\n> \t\t\tunlock_ref(updates[i]->lock);\n> \n\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\nhttp://softwareswirl.blogspot.com/\n"},{"id":"245522","messageId":"53BBE2D6.2070809@alum.mit.edu","threadId":"36972","inReplyTo":"1403275409-28173-28-git-send-email-sahlberg@google.com","subject":"Re: [PATCH v20 27/48] sequencer.c: use ref transactions for all ref updates","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2014-07-08T12:23:50Z","receivedAt":"2014-07-08T12:23:50Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n> Change to use ref transactions for all updates to refs.\n> \n> Reviewed-by: Jonathan Nieder \n> Signed-off-by: Ronnie Sahlberg \n> ---\n> sequencer.c | 24 ++++++++++++++++--------\n> 1 file changed, 16 insertions(+), 8 deletions(-)\n> \n> diff --git a/sequencer.c b/sequencer.c\n> index 0a80c58..fd8acaf 100644\n> --- a/sequencer.c\n> +++ b/sequencer.c\n> @@ -272,23 +272,31 @@ static int error_dirty_index(struct replay_opts *opts)\n> static int fast_forward_to(const unsigned char *to, const unsigned char *from,\n> \t\t\tint unborn, struct replay_opts *opts)\n> {\n> -\tstruct ref_lock *ref_lock;\n> +\tstruct ref_transaction *transaction;\n> \tstruct strbuf sb = STRBUF_INIT;\n> -\tint ret;\n> +\tstruct strbuf err = STRBUF_INIT;\n> \n> \tread_cache();\n> \tif (checkout_fast_forward(from, to, 1))\n> \t\texit(1); /* the callee should have complained already */\n> -\tref_lock = lock_any_ref_for_update(\"HEAD\", unborn ? null_sha1 : from,\n> -\t\t\t\t\t 0, NULL);\n> -\tif (!ref_lock)\n> -\t\treturn error(_(\"Failed to lock HEAD during fast_forward_to\"));\n\nI think you've changed the semantics when unborn is set. Please note\nthat lock_any_ref_for_update() behaves differently if old_sha1 is NULL\n(when no check is done) vs. when it is null_sha1 (when it verifies that\nthe reference didn't previously exist). So when unborn is true, the old\ncode verifies that the reference previously didn't exist...\n\n> \n> \tstrbuf_addf(&sb, \"%s: fast-forward\", action_name(opts));\n> -\tret = write_ref_sha1(ref_lock, to, sb.buf);\n> +\n> +\ttransaction = ref_transaction_begin(&err);\n> +\tif (!transaction ||\n> +\t ref_transaction_update(transaction, \"HEAD\", to, from,\n> +\t\t\t\t 0, !unborn, &err) ||\n\n...whereas when unborn is true, the new code does no check at all. I\nthink you want\n\n ref_transaction_update(transaction, \"HEAD\",\n to, unborn ? null_sha1 : from,\n 0, 1, &err) ||\n\n> +\t ref_transaction_commit(transaction, sb.buf, &err)) {\n> +\t\tref_transaction_free(transaction);\n> +\t\terror(\"%s\", err.buf);\n> +\t\tstrbuf_release(&sb);\n> +\t\tstrbuf_release(&err);\n> +\t\treturn -1;\n> +\t}\n> \n> \tstrbuf_release(&sb);\n> -\treturn ret;\n> +\tref_transaction_free(transaction);\n> +\treturn 0;\n> }\n> \n> static int do_recursive_merge(struct commit *base, struct commit *next,\n> \n\nMichael\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\nhttp://softwareswirl.blogspot.com/\n"},{"id":"245523","messageId":"53BBE4FD.20605@alum.mit.edu","threadId":"36972","inReplyTo":"1403275409-28173-25-git-send-email-sahlberg@google.com","subject":"Re: [PATCH v20 24/48] tag.c: use ref transactions when doing updates","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2014-07-08T12:33:01Z","receivedAt":"2014-07-08T12:33:01Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n> Change tag.c to use ref transactions for all ref updates.\n> \n> Reviewed-by: Jonathan Nieder \n> Signed-off-by: Ronnie Sahlberg \n> ---\n> builtin/tag.c | 15 +++++++++------\n> 1 file changed, 9 insertions(+), 6 deletions(-)\n> \n> diff --git a/builtin/tag.c b/builtin/tag.c\n> index c6e8a71..c9bfc9a 100644\n> --- a/builtin/tag.c\n> +++ b/builtin/tag.c\n> @@ -548,7 +548,6 @@ int cmd_tag(int argc, const char **argv, const char *prefix)\n> \tstruct strbuf ref = STRBUF_INIT;\n> \tunsigned char object[20], prev[20];\n> \tconst char *object_ref, *tag;\n> -\tstruct ref_lock *lock;\n> \tstruct create_tag_options opt;\n> \tchar *cleanup_arg = NULL;\n> \tint annotate = 0, force = 0, lines = -1;\n> @@ -556,6 +555,8 @@ int cmd_tag(int argc, const char **argv, const char *prefix)\n> \tconst char *msgfile = NULL, *keyid = NULL;\n> \tstruct msg_arg msg = { 0, STRBUF_INIT };\n> \tstruct commit_list *with_commit = NULL;\n> +\tstruct ref_transaction *transaction;\n> +\tstruct strbuf err = STRBUF_INIT;\n> \tstruct option options[] = {\n> \t\tOPT_CMDMODE('l', \"list\", &cmdmode, N_(\"list tag names\"), 'l'),\n> \t\t{ OPTION_INTEGER, 'n', NULL, &lines, N_(\"n\"),\n> @@ -701,11 +702,13 @@ int cmd_tag(int argc, const char **argv, const char *prefix)\n> \tif (annotate)\n> \t\tcreate_tag(object, tag, &buf, &opt, prev, object);\n> \n> -\tlock = lock_any_ref_for_update(ref.buf, prev, 0, NULL);\n> -\tif (!lock)\n> -\t\tdie(_(\"%s: cannot lock the ref\"), ref.buf);\n> -\tif (write_ref_sha1(lock, object, NULL) < 0)\n> -\t\tdie(_(\"%s: cannot update the ref\"), ref.buf);\n> +\ttransaction = ref_transaction_begin(&err);\n> +\tif (!transaction ||\n> +\t ref_transaction_update(transaction, ref.buf, object, prev,\n> +\t\t\t\t 0, !is_null_sha1(prev), &err) ||\n\nSimilar to the error in sequencer.c a few patches later (explained in\nmore detail in my comment on that patch), here you only do a check if\n!is_null_sha1(prev), whereas the old code always did the check. I think\nyou want\n\n ref_transaction_update(transaction, ref.buf, object, prev,\n 0, 1, &err) ||\n\nPlease check whether you have made the same mistake in other patches.\n\n> [...]\n\nMichael\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\n"},{"id":"245524","messageId":"53BBE578.9030703@alum.mit.edu","threadId":"36972","inReplyTo":"1403275409-28173-26-git-send-email-sahlberg@google.com","subject":"Re: [PATCH v20 25/48] replace.c: use the ref transaction functions for updates","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2014-07-08T12:35:04Z","receivedAt":"2014-07-08T12:35:04Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n> Update replace.c to use ref transactions for updates.\n> \n> Reviewed-by: Jonathan Nieder \n> Signed-off-by: Ronnie Sahlberg \n> ---\n> builtin/replace.c | 15 +++++++++------\n> 1 file changed, 9 insertions(+), 6 deletions(-)\n> \n> diff --git a/builtin/replace.c b/builtin/replace.c\n> index 1bb491d..7528f3d 100644\n> --- a/builtin/replace.c\n> +++ b/builtin/replace.c\n> @@ -153,7 +153,8 @@ static int replace_object_sha1(const char *object_ref,\n> \tunsigned char prev[20];\n> \tenum object_type obj_type, repl_type;\n> \tchar ref[PATH_MAX];\n> -\tstruct ref_lock *lock;\n> +\tstruct ref_transaction *transaction;\n> +\tstruct strbuf err = STRBUF_INIT;\n> \n> \tobj_type = sha1_object_info(object, NULL);\n> \trepl_type = sha1_object_info(repl, NULL);\n> @@ -166,12 +167,14 @@ static int replace_object_sha1(const char *object_ref,\n> \n> \tcheck_ref_valid(object, prev, ref, sizeof(ref), force);\n> \n> -\tlock = lock_any_ref_for_update(ref, prev, 0, NULL);\n> -\tif (!lock)\n> -\t\tdie(\"%s: cannot lock the ref\", ref);\n> -\tif (write_ref_sha1(lock, repl, NULL) < 0)\n> -\t\tdie(\"%s: cannot update the ref\", ref);\n> +\ttransaction = ref_transaction_begin(&err);\n> +\tif (!transaction ||\n> +\t ref_transaction_update(transaction, ref, repl, prev,\n> +\t\t\t\t 0, !is_null_sha1(prev), &err) ||\n\nSame problem here. You need\n\n s/!is_null_sha1(prev)/1/\n\n> +\t ref_transaction_commit(transaction, NULL, &err))\n> +\t\tdie(\"%s\", err.buf);\n> \n> +\tref_transaction_free(transaction);\n> \treturn 0;\n> }\n> \n> \n\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\n"},{"id":"245526","messageId":"53BBEA04.5040103@alum.mit.edu","threadId":"36972","inReplyTo":"1403275409-28173-31-git-send-email-sahlberg@google.com","subject":"Re: [PATCH v20 30/48] refs.c: change update_ref to use a transaction","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2014-07-08T12:54:28Z","receivedAt":"2014-07-08T12:54:28Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n> Change the update_ref helper function to use a ref transaction internally.\n> \n> Reviewed-by: Jonathan Nieder \n> Signed-off-by: Ronnie Sahlberg \n> ---\n> refs.c | 28 ++++++++++++++++++++++++----\n> 1 file changed, 24 insertions(+), 4 deletions(-)\n> \n> diff --git a/refs.c b/refs.c\n> index 8c695ba..4bdccc5 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -3524,11 +3524,31 @@ int update_ref(const char *action, const char *refname,\n> \t const unsigned char *sha1, const unsigned char *oldval,\n> \t int flags, enum action_on_err onerr)\n> {\n> -\tstruct ref_lock *lock;\n> -\tlock = update_ref_lock(refname, oldval, flags, NULL, onerr);\n> -\tif (!lock)\n> +\tstruct ref_transaction *t;\n> +\tstruct strbuf err = STRBUF_INIT;\n> +\n> +\tt = ref_transaction_begin(&err);\n> +\tif (!t ||\n> +\t ref_transaction_update(t, refname, sha1, oldval, flags,\n> +\t\t\t\t !!oldval, &err) ||\n> +\t ref_transaction_commit(t, action, &err)) {\n> +\t\tconst char *str = \"update_ref failed for ref '%s': %s\";\n> +\n> +\t\tref_transaction_free(t);\n> +\t\tswitch (onerr) {\n> +\t\tcase UPDATE_REFS_MSG_ON_ERR:\n> +\t\t\terror(str, refname, err.buf);\n> +\t\t\tbreak;\n> +\t\tcase UPDATE_REFS_DIE_ON_ERR:\n> +\t\t\tdie(str, refname, err.buf);\n> +\t\t\tbreak;\n> +\t\tcase UPDATE_REFS_QUIET_ON_ERR:\n> +\t\t\tbreak;\n> +\t\t}\n> +\t\tstrbuf_release(&err);\n> \t\treturn 1;\n> -\treturn update_ref_write(action, refname, sha1, lock, NULL, onerr);\n> +\t}\n> +\treturn 0;\n> }\n> \n\nShould this function be scheduled for the \"take strbuf *err argument\"\ntreatment instead of continuing to use an action_on_err parameter?\n(Maybe you've changed this later in the patch series?)\n\nI'm not saying this change has to be part of the current patch series,\nbut let's consider it for the future.\n\nMichael\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\n"},{"id":"245528","messageId":"53BBF016.7040107@alum.mit.edu","threadId":"36972","inReplyTo":"1403275409-28173-32-git-send-email-sahlberg@google.com","subject":"Re: [PATCH v20 31/48] receive-pack.c: use a reference transaction for updating the refs","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2014-07-08T13:20:22Z","receivedAt":"2014-07-08T13:20:22Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n> Wrap all the ref updates inside a transaction.\n> \n> Signed-off-by: Ronnie Sahlberg \n> ---\n> builtin/receive-pack.c | 96 +++++++++++++++++++++++++++++++++-----------------\n> 1 file changed, 63 insertions(+), 33 deletions(-)\n> \n> diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\n> index c323081..b51f8ae 100644\n> --- a/builtin/receive-pack.c\n> +++ b/builtin/receive-pack.c\n> [...]\n> @@ -647,6 +654,9 @@ static void check_aliased_update(struct command *cmd, struct string_list *list)\n> \tchar cmd_oldh[41], cmd_newh[41], dst_oldh[41], dst_newh[41];\n> \tint flag;\n> \n> +\tif (cmd->error_string)\n> +\t\tdie(\"BUG: check_alised_update called with failed cmd\");\n\ns/check_alised_update/check_aliased_update/\n\n> [...]\n\nMichael\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\n"},{"id":"245529","messageId":"53BBF333.903@alum.mit.edu","threadId":"36972","inReplyTo":"1403275409-28173-34-git-send-email-sahlberg@google.com","subject":"Re: [PATCH v20 33/48] walker.c: use ref transaction for ref updates","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2014-07-08T13:33:39Z","receivedAt":"2014-07-08T13:33:39Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n> Switch to using ref transactions in walker_fetch(). As part of the refactoring\n> to use ref transactions we also fix a potential memory leak where in the\n> original code if write_ref_sha1() would fail we would end up returning from\n> the function without free()ing the msg string.\n> \n> Note that this function is only called when fetching from a remote HTTP\n> repository onto the local (most of the time single-user) repository which\n> likely means that the type of collissions that the previous locking would\n\ns/collissions/collisions/\n\n> protect against and cause the fetch to fail for to be even more rare.\n\nGrammatico: s/to be/are/ ?\n\n> Reviewed-by: Jonathan Nieder \n> Signed-off-by: Ronnie Sahlberg \n> ---\n> walker.c | 59 +++++++++++++++++++++++++++++++++++------------------------\n> 1 file changed, 35 insertions(+), 24 deletions(-)\n> \n> diff --git a/walker.c b/walker.c\n> index 1dd86b8..60d9f9e 100644\n> --- a/walker.c\n> +++ b/walker.c\n> @@ -251,39 +251,36 @@ void walker_targets_free(int targets, char **target, const char **write_ref)\n> int walker_fetch(struct walker *walker, int targets, char **target,\n> \t\t const char **write_ref, const char *write_ref_log_details)\n> {\n> -\tstruct ref_lock **lock = xcalloc(targets, sizeof(struct ref_lock *));\n> +\tstruct strbuf ref_name = STRBUF_INIT;\n> +\tstruct strbuf err = STRBUF_INIT;\n> +\tstruct ref_transaction *transaction = NULL;\n> \tunsigned char *sha1 = xmalloc(targets * 20);\n> -\tchar *msg;\n> -\tint ret;\n> +\tchar *msg = NULL;\n> \tint i;\n> \n> \tsave_commit_buffer = 0;\n> \n> -\tfor (i = 0; i < targets; i++) {\n> -\t\tif (!write_ref || !write_ref[i])\n> -\t\t\tcontinue;\n> -\n> -\t\tlock[i] = lock_ref_sha1(write_ref[i], NULL);\n> -\t\tif (!lock[i]) {\n> -\t\t\terror(\"Can't lock ref %s\", write_ref[i]);\n> -\t\t\tgoto unlock_and_fail;\n> +\tif (write_ref) {\n> +\t\ttransaction = ref_transaction_begin(&err);\n> +\t\tif (!transaction) {\n> +\t\t\terror(\"%s\", err.buf);\n> +\t\t\tgoto rollback_and_fail;\n> \t\t}\n> \t}\n> -\n\nIs there some reason why the transaction cannot be built up during a\nsingle iteration over targets, thereby also avoiding the need for the\nsha1[20*i] stuff? This seems like exactly the kind of situation where\ntransactions should *save* code. But perhaps I've overlooked a\ndependency between the two loops.\n\n> \tif (!walker->get_recover)\n> \t\tfor_each_ref(mark_complete, NULL);\n> \n> \tfor (i = 0; i < targets; i++) {\n> \t\tif (interpret_target(walker, target[i], &sha1[20 * i])) {\n> \t\t\terror(\"Could not interpret response from server '%s' as something to pull\", target[i]);\n> -\t\t\tgoto unlock_and_fail;\n> +\t\t\tgoto rollback_and_fail;\n> \t\t}\n> \t\tif (process(walker, lookup_unknown_object(&sha1[20 * i])))\n> -\t\t\tgoto unlock_and_fail;\n> +\t\t\tgoto rollback_and_fail;\n> \t}\n> \n> \tif (loop(walker))\n> -\t\tgoto unlock_and_fail;\n> +\t\tgoto rollback_and_fail;\n> \n> \tif (write_ref_log_details) {\n> \t\tmsg = xmalloc(strlen(write_ref_log_details) + 12);\n> @@ -294,19 +291,33 @@ int walker_fetch(struct walker *walker, int targets, char **target,\n> \tfor (i = 0; i < targets; i++) {\n> \t\tif (!write_ref || !write_ref[i])\n> \t\t\tcontinue;\n> -\t\tret = write_ref_sha1(lock[i], &sha1[20 * i], msg ? msg : \"fetch (unknown)\");\n> -\t\tlock[i] = NULL;\n> -\t\tif (ret)\n> -\t\t\tgoto unlock_and_fail;\n> +\t\tstrbuf_reset(&ref_name);\n> +\t\tstrbuf_addf(&ref_name, \"refs/%s\", write_ref[i]);\n> +\t\tif (ref_transaction_update(transaction, ref_name.buf,\n> +\t\t\t\t\t &sha1[20 * i], NULL, 0, 0,\n> +\t\t\t\t\t &err)) {\n> +\t\t\terror(\"%s\", err.buf);\n> +\t\t\tgoto rollback_and_fail;\n> +\t\t}\n> +\t}\n> +\tif (write_ref) {\n> +\t\tif (ref_transaction_commit(transaction,\n> +\t\t\t\t\t msg ? msg : \"fetch (unknown)\",\n> +\t\t\t\t\t &err)) {\n> +\t\t\terror(\"%s\", err.buf);\n> +\t\t\tgoto rollback_and_fail;\n> +\t\t}\n> +\t\tref_transaction_free(transaction);\n> \t}\n> -\tfree(msg);\n> \n> +\tfree(msg);\n> \treturn 0;\n> \n> -unlock_and_fail:\n> -\tfor (i = 0; i < targets; i++)\n> -\t\tif (lock[i])\n> -\t\t\tunlock_ref(lock[i]);\n> +rollback_and_fail:\n> +\tref_transaction_free(transaction);\n> +\tfree(msg);\n> +\tstrbuf_release(&err);\n> +\tstrbuf_release(&ref_name);\n> \n> \treturn -1;\n> }\n> \n\nMichael\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\n"},{"id":"245530","messageId":"53BBF46A.70802@alum.mit.edu","threadId":"36972","inReplyTo":"1403275409-28173-38-git-send-email-sahlberg@google.com","subject":"Re: [PATCH v20 37/48] refs.c: remove lock_ref_sha1","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2014-07-08T13:38:50Z","receivedAt":"2014-07-08T13:38:50Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n> lock_ref_sha1 was only called from one place in refc.c and only provided\n> a check that the refname was sane before adding back the initial \"refs/\"\n> part of the ref path name, the initial \"refs/\" that this caller had already\n> stripped off before calling lock_ref_sha1.\n> [...]\n\nI'm especially glad to see this ugly function disappear!\n\nMichael\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\n"},{"id":"245531","messageId":"53BBF794.1090702@alum.mit.edu","threadId":"36972","inReplyTo":"1403275409-28173-40-git-send-email-sahlberg@google.com","subject":"Re: [PATCH v20 39/48] refs.c: make delete_ref use a transaction","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2014-07-08T13:52:20Z","receivedAt":"2014-07-08T13:52:20Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n> Change delete_ref to use a ref transaction for the deletion. At the same time\n> since we no longer have any callers of repack_without_ref we can now delete\n> this function.\n> \n> Change delete_ref to return 0 on success and 1 on failure instead of the\n> previous 0 on success either 1 or -1 on failure.\n> \n> Reviewed-by: Jonathan Nieder \n> Signed-off-by: Ronnie Sahlberg \n> ---\n> refs.c | 34 +++++++++++++---------------------\n> 1 file changed, 13 insertions(+), 21 deletions(-)\n> \n> diff --git a/refs.c b/refs.c\n> index 3d070d5..92a06d4 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -2544,11 +2544,6 @@ int repack_without_refs(const char **refnames, int n, struct strbuf *err)\n> \treturn ret;\n> }\n> \n> -static int repack_without_ref(const char *refname)\n> -{\n> -\treturn repack_without_refs(&refname, 1, NULL);\n> -}\n> -\n> static int delete_ref_loose(struct ref_lock *lock, int flag)\n> {\n> \tif (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {\n> @@ -2566,24 +2561,21 @@ static int delete_ref_loose(struct ref_lock *lock, int flag)\n> \n> int delete_ref(const char *refname, const unsigned char *sha1, int delopt)\n> {\n> -\tstruct ref_lock *lock;\n> -\tint ret = 0, flag = 0;\n> +\tstruct ref_transaction *transaction;\n> +\tstruct strbuf err = STRBUF_INIT;\n> \n> -\tlock = lock_ref_sha1_basic(refname, sha1, delopt, &flag);\n\nThe old code checked that the old value of refname was sha1, regardless\nof whether sha1 was null_sha1. Presumably callers never set sha1 to\nnull_sha1...\n\n> -\tif (!lock)\n> +\ttransaction = ref_transaction_begin(&err);\n> +\tif (!transaction ||\n> +\t ref_transaction_delete(transaction, refname, sha1, delopt,\n> +\t\t\t\t sha1 && !is_null_sha1(sha1), &err) ||\n\n...But the new code explicitly skips the check if sha1 is null_sha1.\nThis shouldn't make a practical difference, because presumably callers\nnever set sha1 to null_sha1. But given that the new policy elsewhere\nfor \"delete\" updates is that it is an error for old_sha1 to equal\nnull_sha1, it seems to me that this extra check shouldn't be here. So I\nthink this should be changed to\n\n ref_transaction_delete(transaction, refname, sha1, delopt,\n !!sha1, &err) ||\n\n\n> [...]\n\nMichael\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\n"},{"id":"245532","messageId":"53BBFDFE.5060902@alum.mit.edu","threadId":"36972","inReplyTo":"1403275409-28173-41-git-send-email-sahlberg@google.com","subject":"Re: [PATCH v20 40/48] refs.c: add an err argument to delete_ref_loose","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2014-07-08T14:19:42Z","receivedAt":"2014-07-08T14:19:42Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n> Add an err argument to delete_loose_ref so that we can pass a descriptive\n> error string back to the caller. Pass the err argument from transaction\n> commit to this function so that transaction users will have a nice error\n> string if the transaction failed due to delete_loose_ref.\n> \n> Add a new function unlink_or_err that we can call from delete_ref_loose. This\n> function is similar to unlink_or_warn except that we can pass it an err\n> argument. If err is non-NULL the function will populate err instead of\n> printing a warning().\n> \n> Simplify warn_if_unremovable.\n\nThe change to warn_if_unremovable() is orthogonal to the rest of the\ncommit and should be a separate commit.\n\n> Signed-off-by: Ronnie Sahlberg \n> ---\n> refs.c | 33 ++++++++++++++++++++++++++++-----\n> wrapper.c | 14 ++++++--------\n> 2 files changed, 34 insertions(+), 13 deletions(-)\n> \n> diff --git a/refs.c b/refs.c\n> index 92a06d4..c7d1f3e 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -2544,16 +2544,38 @@ int repack_without_refs(const char **refnames, int n, struct strbuf *err)\n> \treturn ret;\n> }\n> \n> -static int delete_ref_loose(struct ref_lock *lock, int flag)\n> +static int add_err_if_unremovable(const char *op, const char *file,\n> +\t\t\t\t struct strbuf *e, int rc)\n\nThis function is only used once. Given also that its purpose is not\nthat obvious from its signature, it seems to me that the code would be\neasier to read if it were inlined.\n\n> +{\n> +\tint err = errno;\n> +\tif (rc < 0 && errno != ENOENT) {\n> +\t\tstrbuf_addf(e, \"unable to %s %s: %s\",\n> +\t\t\t op, file, strerror(errno));\n> +\t\terrno = err;\n> +\t\treturn -1;\n> +\t}\n> +\treturn 0;\n> +}\n> +\n> +static int unlink_or_err(const char *file, struct strbuf *err)\n\nThe name of this function is misleading; it sounds like it will try to\nunlink the file and if not possible call error(). Maybe a name like\n\"unlink_or_report\" would be less prejudicial.\n\nIt might also make sense to move this function to wrapper.c and\nimplement unlink_or_warn() in terms of it rather than vice versa.\n\n> +{\n> +\tif (err)\n> +\t\treturn add_err_if_unremovable(\"unlink\", file, err,\n> +\t\t\t\t\t unlink(file));\n> +\telse\n> +\t\treturn unlink_or_warn(file);\n> +}\n> +\n> +static int delete_ref_loose(struct ref_lock *lock, int flag, struct strbuf *err)\n> {\n> \tif (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {\n> \t\t/* loose */\n> -\t\tint err, i = strlen(lock->lk->filename) - 5; /* .lock */\n> +\t\tint res, i = strlen(lock->lk->filename) - 5; /* .lock */\n> \n> \t\tlock->lk->filename[i] = 0;\n> -\t\terr = unlink_or_warn(lock->lk->filename);\n> +\t\tres = unlink_or_err(lock->lk->filename, err);\n> \t\tlock->lk->filename[i] = '.';\n> -\t\tif (err && errno != ENOENT)\n> +\t\tif (res)\n> \t\t\treturn 1;\n> \t}\n> \treturn 0;\n> @@ -3603,7 +3625,8 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n> \t\tstruct ref_update *update = updates[i];\n> \n> \t\tif (update->lock) {\n> -\t\t\tret |= delete_ref_loose(update->lock, update->type);\n> +\t\t\tret |= delete_ref_loose(update->lock, update->type,\n> +\t\t\t\t\t\terr);\n> \t\t\tif (!(update->flags & REF_ISPRUNING))\n> \t\t\t\tdelnames[delnum++] = update->lock->ref_name;\n> \t\t}\n> diff --git a/wrapper.c b/wrapper.c\n> index bc1bfb8..740e193 100644\n> --- a/wrapper.c\n> +++ b/wrapper.c\n> @@ -429,14 +429,12 @@ int xmkstemp_mode(char *template, int mode)\n> \n> static int warn_if_unremovable(const char *op, const char *file, int rc)\n> {\n> -\tif (rc < 0) {\n> -\t\tint err = errno;\n> -\t\tif (ENOENT != err) {\n> -\t\t\twarning(\"unable to %s %s: %s\",\n> -\t\t\t\top, file, strerror(errno));\n> -\t\t\terrno = err;\n> -\t\t}\n> -\t}\n> +\tint err;\n> +\tif (rc >= 0 || errno == ENOENT)\n> +\t\treturn rc;\n> +\terr = errno;\n> +\twarning(\"unable to %s %s: %s\", op, file, strerror(errno));\n> +\terrno = err;\n> \treturn rc;\n> }\n> \n> \n\nMichael\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\n"},{"id":"245533","messageId":"53BC02B3.5000808@alum.mit.edu","threadId":"36972","inReplyTo":"1403275409-28173-42-git-send-email-sahlberg@google.com","subject":"Re: [PATCH v20 41/48] refs.c: pass the ref log message to _create/delete/update instead of _commit","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2014-07-08T14:39:47Z","receivedAt":"2014-07-08T14:39:47Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n> Change the reference transactions so that we pass the reflog message\n> through to the create/delete/update function instead of the commit message.\n> This allows for individual messages for each change in a multi ref\n> transaction.\n> \n> Reviewed-by: Jonathan Nieder \n> Signed-off-by: Ronnie Sahlberg \n> ---\n> [...]\n> diff --git a/refs.h b/refs.h\n> index 3b321c2..f24b2c1 100644\n> --- a/refs.h\n> +++ b/refs.h\n\nWould you please document the msg parameter in the block comment that\nprecedes these three declarations? Especially important is the fact\nthat the functions make internal copies of msg, so the caller retains\nownership of its copy. You might also mention what happens if msg is\nNULL (which, as far as I can see, is that a reflog entry is created\nanyway (except in the case of a delete) but that the entry doesn't\ncontain an explanation).\n\n> @@ -297,7 +297,7 @@ int ref_transaction_update(struct ref_transaction *transaction,\n> \t\t\t const char *refname,\n> \t\t\t const unsigned char *new_sha1,\n> \t\t\t const unsigned char *old_sha1,\n> -\t\t\t int flags, int have_old,\n> +\t\t\t int flags, int have_old, const char *msg,\n> \t\t\t struct strbuf *err);\n> \n> /*\n> @@ -312,7 +312,7 @@ int ref_transaction_update(struct ref_transaction *transaction,\n> int ref_transaction_create(struct ref_transaction *transaction,\n> \t\t\t const char *refname,\n> \t\t\t const unsigned char *new_sha1,\n> -\t\t\t int flags,\n> +\t\t\t int flags, const char *msg,\n> \t\t\t struct strbuf *err);\n\nIt is noteworthy that ref_transaction_delete() accepts a msg parameter,\neven though we currently delete a reference's entire reflog when the\nreference is deleted. I prefer to think of this as a shortcoming of the\ncurrent reference backend, from which future backends hopefully will not\nsuffer. So I like this design choice.\n\nHowever, I think it is worth noting this dichotomy in the commit message\nand perhaps also in the function documentation.\n\n> /*\n> @@ -326,7 +326,7 @@ int ref_transaction_create(struct ref_transaction *transaction,\n> int ref_transaction_delete(struct ref_transaction *transaction,\n> \t\t\t const char *refname,\n> \t\t\t const unsigned char *old_sha1,\n> -\t\t\t int flags, int have_old,\n> +\t\t\t int flags, int have_old, const char *msg,\n> \t\t\t struct strbuf *err);\n> \n> /*\n> @@ -335,7 +335,7 @@ int ref_transaction_delete(struct ref_transaction *transaction,\n> * problem.\n> */\n> int ref_transaction_commit(struct ref_transaction *transaction,\n> -\t\t\t const char *msg, struct strbuf *err);\n> +\t\t\t struct strbuf *err);\n> \n> /*\n> * Free an existing transaction and all associated data.\n> [...]\n\nMichael\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\nhttp://softwareswirl.blogspot.com/\n"},{"id":"245534","messageId":"53BC07FC.8080601@alum.mit.edu","threadId":"36972","inReplyTo":"1403275409-28173-44-git-send-email-sahlberg@google.com","subject":"Re: [PATCH v20 43/48] refs.c: move the check for valid refname to lock_ref_sha1_basic","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2014-07-08T15:02:20Z","receivedAt":"2014-07-08T15:02:20Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n> Move the check for check_refname_format from lock_any_ref_for_update\n> to lock_ref_sha1_basic. At some later stage we will get rid of\n> lock_any_ref_for_update completely.\n> \n> If lock_ref_sha1_basic fails the check_refname_format test, set errno to\n> EINVAL before returning NULL. This to guarantee that we will not return an\n> error without updating errno.\n> \n> This leaves lock_any_ref_for_updates as a no-op wrapper which could be removed.\n> But this wrapper is also called from an external caller and we will soon\n> make changes to the signature to lock_ref_sha1_basic that we do not want to\n> expose to that caller.\n> \n> This changes semantics for lock_ref_sha1_basic slightly. With this change\n> it is no longer possible to open a ref that has a badly name which breaks\n\ns/badly name/bad name,/\n\n> any codepaths that tries to open and repair badly named refs. The normal refs\n\ns/tries/try/\n\n> API should not allow neither creating nor accessing refs with invalid names.\n\ns/not allow neither/allow neither/\n\n> If we need such recovery code we could add it as an option to git fsck and have\n> git fsck be the only sanctioned way of bypassing the normal API and checks.\n\nI like the sentiment, but in the real world I'm not sure we can take\nsuch a step based only on good intentions. Which callers would be\naffected? Where is this \"git fsck\" code that would be needed to help\npeople rescue their repos?\n\nI can also imagine that we will tighten up the check_refname_format\nchecks in the future; for example, I think it would be a good idea to\nprohibit reference names that start with '-' because it is almost\nimpossible to work with them (their names look like command-line\noptions). If we ever make a change like that, we will need some amount\nof tolerance in git versions around the transition.\n\nSo...I like the idea of enforcing refname checks at the lowest level\npossible, but I think that the change you propose is too abrupt. I\nthink it needs either more careful analysis showing that it won't hurt\nanybody, or some kind of tooling or non-strict mode that people can use\nto fix their repositories.\n\nMichael\n\n> Signed-off-by: Ronnie Sahlberg \n> ---\n> refs.c | 7 +++++--\n> 1 file changed, 5 insertions(+), 2 deletions(-)\n> \n> diff --git a/refs.c b/refs.c\n> index 389a55f..bccf8c3 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -2088,6 +2088,11 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,\n> \tint missing = 0;\n> \tint attempts_remaining = 3;\n> \n> +\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n> +\t\terrno = EINVAL;\n> +\t\treturn NULL;\n> +\t}\n> +\n> \tlock = xcalloc(1, sizeof(struct ref_lock));\n> \tlock->lock_fd = -1;\n> \n> @@ -2179,8 +2184,6 @@ struct ref_lock *lock_any_ref_for_update(const char *refname,\n> \t\t\t\t\t const unsigned char *old_sha1,\n> \t\t\t\t\t int flags, int *type_p)\n> {\n> -\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL))\n> -\t\treturn NULL;\n> \treturn lock_ref_sha1_basic(refname, old_sha1, flags, type_p);\n> }\n> \n> \n\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\n"},{"id":"245535","messageId":"53BC093F.6010604@alum.mit.edu","threadId":"36972","inReplyTo":"1403275409-28173-45-git-send-email-sahlberg@google.com","subject":"Re: [PATCH v20 44/48] refs.c: call lock_ref_sha1_basic directly from commit","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2014-07-08T15:07:43Z","receivedAt":"2014-07-08T15:07:43Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n> Skip using the lock_any_ref_for_update wrapper and call lock_ref_sha1_basic\n> directly from the commit function.\n\nThis commit is obviously hostage to whether commit 43/48 is kept.\n\n> Reviewed-by: Jonathan Nieder \n> Signed-off-by: Ronnie Sahlberg \n> ---\n> refs.c | 12 ++++++------\n> 1 file changed, 6 insertions(+), 6 deletions(-)\n> \n> diff --git a/refs.c b/refs.c\n> index bccf8c3..f3fab37 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -3598,12 +3598,12 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n> \tfor (i = 0; i < n; i++) {\n> \t\tstruct ref_update *update = updates[i];\n> \n> -\t\tupdate->lock = lock_any_ref_for_update(update->refname,\n> -\t\t\t\t\t\t (update->have_old ?\n> -\t\t\t\t\t\t\tupdate->old_sha1 :\n> -\t\t\t\t\t\t\tNULL),\n> -\t\t\t\t\t\t update->flags,\n> -\t\t\t\t\t\t &update->type);\n> +\t\tupdate->lock = lock_ref_sha1_basic(update->refname,\n> +\t\t\t\t\t\t (update->have_old ?\n> +\t\t\t\t\t\t update->old_sha1 :\n> +\t\t\t\t\t\t NULL),\n> +\t\t\t\t\t\t update->flags,\n> +\t\t\t\t\t\t &update->type);\n> \t\tif (!update->lock) {\n> \t\t\tif (err)\n> \t\t\t\tstrbuf_addf(err, \"Cannot lock the ref '%s'.\",\n> \n\nMichael\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\n"},{"id":"245539","messageId":"53BC1C53.9030203@alum.mit.edu","threadId":"36972","inReplyTo":"1403275409-28173-1-git-send-email-sahlberg@google.com","subject":"Re: [PATCH v20 00/48] Use ref transactions","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2014-07-08T16:29:07Z","receivedAt":"2014-07-08T16:29:07Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 06/20/2014 04:42 PM, Ronnie Sahlberg wrote:\n> This patch series can also be found at\n> https://github.com/rsahlberg/git/tree/ref-transactions\n> \n> This patch series is based on current master and expands on the transaction\n> API. It converts all ref updates, inside refs.c as well as external, to use the\n> transaction API for updates. This makes most of the ref updates to become\n> atomic when there are failures locking or writing to a ref.\n> \n> This version completes the work to convert all ref updates to use transactions.\n> Now that all updates are through transactions I will start working on\n> cleaning up the reading of refs and to create an api for managing reflogs but\n> all that will go in a different patch series.\n> \n> Version 20:\n> - Whitespace and style changes suggested by Jun.\n\nI spent most of the day on reviewing this patch series, but now I'm out\nof time again. Here is a summary from my point of view:\n\nPatches 01-19 -- ACK mhagger\nPatches 20-42 -- I sent various comments, small to large, concerning\nthese patches\nPatch 43 -- Needs more justification if it is to be acceptable\nPatch 44 -- Depends on 43\nPatches 45-48 -- I didn't quite get to these, but...\n\nPerhaps it would be more appropriate for the rules about reference name\nconflicts to be enforced by the backend, since it is the limitations of\nthe current backend that impose the restrictions. Would that make sense?\n\nOn the other hand, removing the restrictions isn't simply a matter of\npicking a different backend, because all Git repositories have to be\nable to interact with each other.\n\nSo, I don't yet have a considered opinion on the matter.\n\n\nI think it would be good to try to merge the first part of this patch\nseries to lock in some progress while we continue iterating on the\nremainder. I'm satisfied that it is all going in the right direction\nand I am thankful to Ronnie for pushing it forward. But handling\n48-patch series is very daunting and I would welcome a split.\n\nI'm not sure whether patches 01-19 are necessarily the right split\nbetween merge-now/iterate-more; it is more or less an accident that I\nstopped after patch 19 on an earlier review. Maybe Ronnie could propose\na logical subset of the commits as being ready to be merged to next in\nthe nearish term?\n\nMichael\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\n"},{"id":"245544","messageId":"xmqqa98jadqx.fsf@gitster.dls.corp.google.com","threadId":"36972","inReplyTo":"53BC1C53.9030203@alum.mit.edu","subject":"Re: [PATCH v20 00/48] Use ref transactions","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-07-08T18:48:06Z","receivedAt":"2014-07-08T18:48:06Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Michael Haggerty writes:\n\n> Patches 01-19 -- ACK mhagger\n> Patches 20-42 -- I sent various comments, small to large, concerning\n> these patches\n> Patch 43 -- Needs more justification if it is to be acceptable\n> Patch 44 -- Depends on 43\n> Patches 45-48 -- I didn't quite get to these, but...\n>\n> Perhaps it would be more appropriate for the rules about reference name\n> conflicts to be enforced by the backend, since it is the limitations of\n> the current backend that impose the restrictions. Would that make sense?\n>\n> On the other hand, removing the restrictions isn't simply a matter of\n> picking a different backend, because all Git repositories have to be\n> able to interact with each other.\n\nI'd say that \"if you have foo/bar you cannot have foo\" may have\nstarted as an implementation limitation, but the interoperability\nrequirement with existing versions of Git and with existing\nrepositories makes it necessary to enforce it the same way as other\nrules such as \"you cannot have double-dots in name, e.g. foo..bar\"\nor \"no branches whose name begins with a dash\", neither of which\ncomes from any filesystem issues. That a rule can be loosened with\none new backend does not at all mean it is a good idea to loosen it\n\"because we can\" in the first place.\n\n> I think it would be good to try to merge the first part of this patch\n> series to lock in some progress while we continue iterating on the\n> remainder. I'm satisfied that it is all going in the right direction\n> and I am thankful to Ronnie for pushing it forward. But handling\n> 48-patch series is very daunting and I would welcome a split.\n>\n> I'm not sure whether patches 01-19 are necessarily the right split\n> between merge-now/iterate-more; it is more or less an accident that I\n> stopped after patch 19 on an earlier review. Maybe Ronnie could propose\n> a logical subset of the commits as being ready to be merged to next in\n> the nearish term?\n\nYeah, thanks for going through this, and I agree that we would be\nbetter off merging the earlier part first.\n"},{"id":"245566","messageId":"20140709050226.GA2318@sigill.intra.peff.net","threadId":"36972","inReplyTo":"xmqqa98jadqx.fsf@gitster.dls.corp.google.com","subject":"Re: [PATCH v20 00/48] Use ref transactions","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2014-07-09T05:02:26Z","receivedAt":"2014-07-09T05:02:26Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Jul 08, 2014 at 11:48:06AM -0700, Junio C Hamano wrote:\n\n> I'd say that \"if you have foo/bar you cannot have foo\" may have\n> started as an implementation limitation, but the interoperability\n> requirement with existing versions of Git and with existing\n> repositories makes it necessary to enforce it the same way as other\n> rules such as \"you cannot have double-dots in name, e.g. foo..bar\"\n> or \"no branches whose name begins with a dash\", neither of which\n> comes from any filesystem issues. That a rule can be loosened with\n> one new backend does not at all mean it is a good idea to loosen it\n> \"because we can\" in the first place.\n\nTo me it makes sense to to have it as an option, for two reasons:\n\n 1. If you want to pay the compatibility cost (e.g., because you work\n with a defined set of users on a small project and can dictate how\n they set their git options), you get the extra feature.\n\n 2. It provides a migration path if we eventually want to move to a\n default backend that allows it.\n\nI admit that I don't care _too_ much, though. My main desire for it is\nnot to store two live branches that overlap, but to store reflogs for\ndeleted branches without conflicting with live branches.\n\nAnd of course all of this is getting rather ahead of ourselves. We do\nnot have _any_ alternate backends yet, nor even yet the infrastructure\nto make them.\n\n-Peff\n"},{"id":"245989","messageId":"CAL=YDWmqXke8rt_x109NLaZU1S2d3dgpPK+kZSF9DfNxm0ECGQ@mail.gmail.com","threadId":"36972","inReplyTo":"53BC1C53.9030203@alum.mit.edu","subject":"Re: [PATCH v20 00/48] Use ref transactions","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-07-14T15:03:10Z","receivedAt":"2014-07-14T15:03:10Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"On Tue, Jul 8, 2014 at 9:29 AM, Michael Haggerty wrote:\n> On 06/20/2014 04:42 PM, Ronnie Sahlberg wrote:\n>> This patch series can also be found at\n>> https://github.com/rsahlberg/git/tree/ref-transactions\n>>\n>> This patch series is based on current master and expands on the transaction\n>> API. It converts all ref updates, inside refs.c as well as external, to use the\n>> transaction API for updates. This makes most of the ref updates to become\n>> atomic when there are failures locking or writing to a ref.\n>>\n>> This version completes the work to convert all ref updates to use transactions.\n>> Now that all updates are through transactions I will start working on\n>> cleaning up the reading of refs and to create an api for managing reflogs but\n>> all that will go in a different patch series.\n>>\n>> Version 20:\n>> - Whitespace and style changes suggested by Jun.\n>\n> I spent most of the day on reviewing this patch series,\n\nThanks!\n\n\n>but now I'm out\n> of time again. Here is a summary from my point of view:\n>\n> Patches 01-19 -- ACK mhagger\n> Patches 20-42 -- I sent various comments, small to large, concerning\n> these patches\n> Patch 43 -- Needs more justification if it is to be acceptable\n> Patch 44 -- Depends on 43\n> Patches 45-48 -- I didn't quite get to these, but...\n>\n> Perhaps it would be more appropriate for the rules about reference name\n> conflicts to be enforced by the backend, since it is the limitations of\n> the current backend that impose the restrictions. Would that make sense?\n>\n> On the other hand, removing the restrictions isn't simply a matter of\n> picking a different backend, because all Git repositories have to be\n> able to interact with each other.\n>\n> So, I don't yet have a considered opinion on the matter.\n\nI think for compatibility I would prefer to keep the same rules for\nname conflicts as for the current files implementation.\nBut we could have a configuration option to disable these checks, with\nthe caveat that this might mean that some users will\nno longer be able to access pull all the branches anymore.\n\n\n>\n>\n> I think it would be good to try to merge the first part of this patch\n> series to lock in some progress while we continue iterating on the\n> remainder. I'm satisfied that it is all going in the right direction\n> and I am thankful to Ronnie for pushing it forward. But handling\n> 48-patch series is very daunting and I would welcome a split.\n\nWill do.\n\n>\n> I'm not sure whether patches 01-19 are necessarily the right split\n> between merge-now/iterate-more; it is more or less an accident that I\n> stopped after patch 19 on an earlier review. Maybe Ronnie could propose\n> a logical subset of the commits as being ready to be merged to next in\n> the nearish term?\n>\n> Michael\n>\n> --\n> Michael Haggerty\n> mhagger@alum.mit.edu\n>\n"},{"id":"245991","messageId":"CAL=YDWkQXA=XpaV=u8MbrUTkg1DARdDuNGbY_qPjyNZOF+D4Yg@mail.gmail.com","threadId":"36972","inReplyTo":"xmqqa98jadqx.fsf@gitster.dls.corp.google.com","subject":"Re: [PATCH v20 00/48] Use ref transactions","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-07-14T16:16:38Z","receivedAt":"2014-07-14T16:16:38Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"On Tue, Jul 8, 2014 at 11:48 AM, Junio C Hamano wrote:\n> Michael Haggerty writes:\n>\n>> Patches 01-19 -- ACK mhagger\n>> Patches 20-42 -- I sent various comments, small to large, concerning\n>> these patches\n>> Patch 43 -- Needs more justification if it is to be acceptable\n>> Patch 44 -- Depends on 43\n>> Patches 45-48 -- I didn't quite get to these, but...\n>>\n>> Perhaps it would be more appropriate for the rules about reference name\n>> conflicts to be enforced by the backend, since it is the limitations of\n>> the current backend that impose the restrictions. Would that make sense?\n>>\n>> On the other hand, removing the restrictions isn't simply a matter of\n>> picking a different backend, because all Git repositories have to be\n>> able to interact with each other.\n>\n> I'd say that \"if you have foo/bar you cannot have foo\" may have\n> started as an implementation limitation, but the interoperability\n> requirement with existing versions of Git and with existing\n> repositories makes it necessary to enforce it the same way as other\n> rules such as \"you cannot have double-dots in name, e.g. foo..bar\"\n> or \"no branches whose name begins with a dash\", neither of which\n> comes from any filesystem issues. That a rule can be loosened with\n> one new backend does not at all mean it is a good idea to loosen it\n> \"because we can\" in the first place.\n\nACK.\n\n>\n>> I think it would be good to try to merge the first part of this patch\n>> series to lock in some progress while we continue iterating on the\n>> remainder. I'm satisfied that it is all going in the right direction\n>> and I am thankful to Ronnie for pushing it forward. But handling\n>> 48-patch series is very daunting and I would welcome a split.\n>>\n>> I'm not sure whether patches 01-19 are necessarily the right split\n>> between merge-now/iterate-more; it is more or less an accident that I\n>> stopped after patch 19 on an earlier review. Maybe Ronnie could propose\n>> a logical subset of the commits as being ready to be merged to next in\n>> the nearish term?\n>\n> Yeah, thanks for going through this, and I agree that we would be\n> better off merging the earlier part first.\n>\n\nOk, 01-19 is as good split as any.\nIf you merge just the 01-19 part of this series I will address\nMichael's concerns and repost patches 20-48 as a separate series.\n\nregards\nronnie sahlberg\n"},{"id":"245996","messageId":"CAL=YDWmMoFRsupE5DNPck1PkdLDPBuMtt3OdKhf6BC=8V08AEg@mail.gmail.com","threadId":"36972","inReplyTo":"53BBDAA4.7020701@alum.mit.edu","subject":"Re: [PATCH v20 20/48] refs.c: change ref_transaction_create to do error checking and return status","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-07-14T17:44:27Z","receivedAt":"2014-07-14T17:44:27Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"I have changed the comment. Thanks.\n\nOn Tue, Jul 8, 2014 at 4:48 AM, Michael Haggerty wrote:\n> I'm in my next attempt to get through your patch series. Sorry for the\n> long hiatus.\n>\n> Patches 1-19 look OK aside from a minor typo that I just reported.\n>\n> See below for a comment on this patch.\n>\n> On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n>> Do basic error checking in ref_transaction_create() and make it return\n>> non-zero on error. Update all callers to check the result of\n>> ref_transaction_create(). There are currently no conditions in _create that\n>> will return error but there will be in the future. Add an err argument that\n>> will be updated on failure.\n>>\n>> Reviewed-by: Jonathan Nieder \n>> Signed-off-by: Ronnie Sahlberg \n>> ---\n>> builtin/update-ref.c | 4 +++-\n>> refs.c | 18 +++++++++++------\n>> refs.h | 55 +++++++++++++++++++++++++++++++++++++++++++++-------\n>> 3 files changed, 63 insertions(+), 14 deletions(-)\n>>\n>> diff --git a/builtin/update-ref.c b/builtin/update-ref.c\n>> index 3067b11..41121fa 100644\n>> --- a/builtin/update-ref.c\n>> +++ b/builtin/update-ref.c\n>> @@ -226,7 +226,9 @@ static const char *parse_cmd_create(struct strbuf *input, const char *next)\n>> if (*next != line_termination)\n>> die(\"create %s: extra input: %s\", refname, next);\n>>\n>> - ref_transaction_create(transaction, refname, new_sha1, update_flags);\n>> + if (ref_transaction_create(transaction, refname, new_sha1,\n>> + update_flags, &err))\n>> + die(\"%s\", err.buf);\n>>\n>> update_flags = 0;\n>> free(refname);\n>> diff --git a/refs.c b/refs.c\n>> index 3f05e88..c49f1c6 100644\n>> --- a/refs.c\n>> +++ b/refs.c\n>> @@ -3449,18 +3449,24 @@ int ref_transaction_update(struct ref_transaction *transaction,\n>> return 0;\n>> }\n>>\n>> -void ref_transaction_create(struct ref_transaction *transaction,\n>> - const char *refname,\n>> - const unsigned char *new_sha1,\n>> - int flags)\n>> +int ref_transaction_create(struct ref_transaction *transaction,\n>> + const char *refname,\n>> + const unsigned char *new_sha1,\n>> + int flags,\n>> + struct strbuf *err)\n>> {\n>> - struct ref_update *update = add_update(transaction, refname);\n>> + struct ref_update *update;\n>> +\n>> + if (!new_sha1 || is_null_sha1(new_sha1))\n>> + die(\"BUG: create ref with null new_sha1\");\n>> +\n>> + update = add_update(transaction, refname);\n>>\n>> - assert(!is_null_sha1(new_sha1));\n>> hashcpy(update->new_sha1, new_sha1);\n>> hashclr(update->old_sha1);\n>> update->flags = flags;\n>> update->have_old = 1;\n>> + return 0;\n>> }\n>>\n>> void ref_transaction_delete(struct ref_transaction *transaction,\n>> diff --git a/refs.h b/refs.h\n>> index c5376ce..33b4383 100644\n>> --- a/refs.h\n>> +++ b/refs.h\n>> @@ -10,6 +10,45 @@ struct ref_lock {\n>> int force_write;\n>> };\n>>\n>> +/*\n>> + * A ref_transaction represents a collection of ref updates\n>> + * that should succeed or fail together.\n>> + *\n>> + * Calling sequence\n>> + * ----------------\n>> + * - Allocate and initialize a `struct ref_transaction` by calling\n>> + * `ref_transaction_begin()`.\n>> + *\n>> + * - List intended ref updates by calling functions like\n>> + * `ref_transaction_update()` and `ref_transaction_create()`.\n>> + *\n>> + * - Call `ref_transaction_commit()` to execute the transaction.\n>> + * If this succeeds, the ref updates will have taken place and\n>> + * the transaction cannot be rolled back.\n>> + *\n>> + * - At any time call `ref_transaction_free()` to discard the\n>> + * transaction and free associated resources. In particular,\n>> + * this rolls back the transaction if it has not been\n>> + * successfully committed.\n>> + *\n>> + * Error handling\n>> + * --------------\n>> + *\n>> + * On error, transaction functions append a message about what\n>> + * went wrong to the 'err' argument. The message mentions what\n>> + * ref was being updated (if any) when the error occurred so it\n>> + * can be passed to 'die' or 'error' as-is.\n>> + *\n>> + * The message is appended to err without first clearing err.\n>> + * This allows the caller to prepare preamble text to the generated\n>> + * error message:\n>> + *\n>> + * strbuf_addf(&err, \"Error while doing foo-bar: \");\n>> + * if (ref_transaction_update(..., &err)) {\n>> + * ret = error(\"%s\", err.buf);\n>> + * goto cleanup;\n>> + * }\n>> + */\n>\n> I don't have a problem with the API, but I think the idiom suggested in\n> the comment above is a bit silly. Surely one would do the following\n> instead:\n>\n> if (ref_transaction_update(..., &err)) {\n> ret = error(\"Error while doing foo-bar: %s\", err.buf);\n> goto cleanup;\n> }\n>\n> I think it would also be helpful to document whether the error string\n> that is appended to the strbuf is terminated with a LF.\n>\n>> [...]\n>\n> Michael\n>\n> --\n> Michael Haggerty\n> mhagger@alum.mit.edu\n>\n"},{"id":"245997","messageId":"CAL=YDWksQ_xf_nys1M8Ugoo8kqYZFeOmEXvePE+F1hFQQhfQ6Q@mail.gmail.com","threadId":"36972","inReplyTo":"53BBDBA0.5070308@alum.mit.edu","subject":"Re: [PATCH v20 22/48] refs.c: make ref_transaction_begin take an err argument","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-07-14T17:45:37Z","receivedAt":"2014-07-14T17:45:37Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"Thanks. Fixed.\n\nOn Tue, Jul 8, 2014 at 4:53 AM, Michael Haggerty wrote:\n> On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n>> Add an err argument to _begin so that on non-fatal failures in future ref\n>> backends we can report a nice error back to the caller.\n>> While _begin can currently never fail for other reasons than OOM, in which\n>> case we die() anyway, we may add other types of backends in the future.\n>> For example, a hypothetical MySQL backend could fail in _being with\n>\n> s/_being/_begin/\n>\n>> [...]\n>\n> Michael\n>\n> --\n> Michael Haggerty\n> mhagger@alum.mit.edu\n>\n"},{"id":"245999","messageId":"CAL=YDWmDCtDTiLeGUM+=0DQL-JWF8a+yqmK20YSoGcuvpazREQ@mail.gmail.com","threadId":"36972","inReplyTo":"53BBDD4F.20403@alum.mit.edu","subject":"Re: [PATCH v20 23/48] refs.c: add transaction.status and track OPEN/CLOSED/ERROR","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-07-14T17:55:00Z","receivedAt":"2014-07-14T17:55:00Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"I updated the comments.\n\nStatus is used in a later series to track certain errno settings. This\nused to be done here but was moved to a later series.\nI removed the status field for now and will re add it later when we\nstart using it.\n\nThanks!\n\n\nOn Tue, Jul 8, 2014 at 5:00 AM, Michael Haggerty wrote:\n> On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n>> Track the status of a transaction in a new status field. Check the field for\n>\n> The status field is not set or used anywhere. The field that you use is\n> \"state\".\n>\n>> sanity, i.e. that status must be OPEN when _commit/_create/_delete or\n>> _update is called or else die(BUG:...)\n>>\n>> Signed-off-by: Ronnie Sahlberg \n>> ---\n>> refs.c | 40 +++++++++++++++++++++++++++++++++++++++-\n>> 1 file changed, 39 insertions(+), 1 deletion(-)\n>>\n>> diff --git a/refs.c b/refs.c\n>> index 9cb7908..8c695ba 100644\n>> --- a/refs.c\n>> +++ b/refs.c\n>> @@ -3387,6 +3387,25 @@ struct ref_update {\n>> };\n>>\n>> /*\n>> + * Transaction states.\n>> + * OPEN: The transaction is in a valid state and can accept new updates.\n>> + * An OPEN transaction can be committed.\n>> + * CLOSED: If an open transaction is successfully committed the state will\n>> + * change to CLOSED. No further changes can be made to a CLOSED\n>> + * transaction.\n>> + * CLOSED means that all updates have been successfully committed and\n>> + * the only thing that remains is to free the completed transaction.\n>> + * ERROR: The transaction has failed and is no longer committable.\n>> + * No further changes can be made to a CLOSED transaction and it must\n>> + * be rolled back using transaction_free.\n>> + */\n>> +enum ref_transaction_state {\n>> + REF_TRANSACTION_OPEN = 0,\n>> + REF_TRANSACTION_CLOSED = 1,\n>> + REF_TRANSACTION_ERROR = 2,\n>> +};\n>> +\n>> +/*\n>> * Data structure for holding a reference transaction, which can\n>> * consist of checks and updates to multiple references, carried out\n>> * as atomically as possible. This structure is opaque to callers.\n>> @@ -3395,6 +3414,8 @@ struct ref_transaction {\n>> struct ref_update **updates;\n>> size_t alloc;\n>> size_t nr;\n>> + enum ref_transaction_state state;\n>> + int status;\n>\n> The status field should probably be deleted.\n>\n>> };\n>>\n>> struct ref_transaction *ref_transaction_begin(struct strbuf *err)\n>> @@ -3437,6 +3458,9 @@ int ref_transaction_update(struct ref_transaction *transaction,\n>> {\n>> struct ref_update *update;\n>>\n>> + if (transaction->state != REF_TRANSACTION_OPEN)\n>> + die(\"BUG: update called for transaction that is not open\");\n>> +\n>> if (have_old && !old_sha1)\n>> die(\"BUG: have_old is true but old_sha1 is NULL\");\n>>\n>> @@ -3457,6 +3481,9 @@ int ref_transaction_create(struct ref_transaction *transaction,\n>> {\n>> struct ref_update *update;\n>>\n>> + if (transaction->state != REF_TRANSACTION_OPEN)\n>> + die(\"BUG: create called for transaction that is not open\");\n>> +\n>> if (!new_sha1 || is_null_sha1(new_sha1))\n>> die(\"BUG: create ref with null new_sha1\");\n>>\n>> @@ -3477,6 +3504,9 @@ int ref_transaction_delete(struct ref_transaction *transaction,\n>> {\n>> struct ref_update *update;\n>>\n>> + if (transaction->state != REF_TRANSACTION_OPEN)\n>> + die(\"BUG: delete called for transaction that is not open\");\n>> +\n>> if (have_old && !old_sha1)\n>> die(\"BUG: have_old is true but old_sha1 is NULL\");\n>>\n>> @@ -3532,8 +3562,13 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n>> int n = transaction->nr;\n>> struct ref_update **updates = transaction->updates;\n>>\n>> - if (!n)\n>> + if (transaction->state != REF_TRANSACTION_OPEN)\n>> + die(\"BUG: commit called for transaction that is not open\");\n>> +\n>> + if (!n) {\n>> + transaction->state = REF_TRANSACTION_CLOSED;\n>> return 0;\n>> + }\n>>\n>> /* Allocate work space */\n>> delnames = xmalloc(sizeof(*delnames) * n);\n>> @@ -3595,6 +3630,9 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n>> clear_loose_ref_cache(&ref_cache);\n>>\n>> cleanup:\n>> + transaction->state = ret ? REF_TRANSACTION_ERROR\n>> + : REF_TRANSACTION_CLOSED;\n>> +\n>> for (i = 0; i < n; i++)\n>> if (updates[i]->lock)\n>> unlock_ref(updates[i]->lock);\n>>\n>\n>\n> --\n> Michael Haggerty\n> mhagger@alum.mit.edu\n> http://softwareswirl.blogspot.com/\n"},{"id":"246000","messageId":"CAL=YDWk=oazaCpGGS_pcO2u=JrJvwxd-CORY2SZM+PXqv8Qj0w@mail.gmail.com","threadId":"36972","inReplyTo":"53BBF333.903@alum.mit.edu","subject":"Re: [PATCH v20 33/48] walker.c: use ref transaction for ref updates","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-07-14T18:05:39Z","receivedAt":"2014-07-14T18:05:39Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"On Tue, Jul 8, 2014 at 6:33 AM, Michael Haggerty wrote:\n> On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n>> Switch to using ref transactions in walker_fetch(). As part of the refactoring\n>> to use ref transactions we also fix a potential memory leak where in the\n>> original code if write_ref_sha1() would fail we would end up returning from\n>> the function without free()ing the msg string.\n>>\n>> Note that this function is only called when fetching from a remote HTTP\n>> repository onto the local (most of the time single-user) repository which\n>> likely means that the type of collissions that the previous locking would\n>\n> s/collissions/collisions/\n>\n>> protect against and cause the fetch to fail for to be even more rare.\n>\n> Grammatico: s/to be/are/ ?\n\nThanks. Fixed.\n\n>\n>> Reviewed-by: Jonathan Nieder \n>> Signed-off-by: Ronnie Sahlberg \n>> ---\n>> walker.c | 59 +++++++++++++++++++++++++++++++++++------------------------\n>> 1 file changed, 35 insertions(+), 24 deletions(-)\n>>\n>> diff --git a/walker.c b/walker.c\n>> index 1dd86b8..60d9f9e 100644\n>> --- a/walker.c\n>> +++ b/walker.c\n>> @@ -251,39 +251,36 @@ void walker_targets_free(int targets, char **target, const char **write_ref)\n>> int walker_fetch(struct walker *walker, int targets, char **target,\n>> const char **write_ref, const char *write_ref_log_details)\n>> {\n>> - struct ref_lock **lock = xcalloc(targets, sizeof(struct ref_lock *));\n>> + struct strbuf ref_name = STRBUF_INIT;\n>> + struct strbuf err = STRBUF_INIT;\n>> + struct ref_transaction *transaction = NULL;\n>> unsigned char *sha1 = xmalloc(targets * 20);\n>> - char *msg;\n>> - int ret;\n>> + char *msg = NULL;\n>> int i;\n>>\n>> save_commit_buffer = 0;\n>>\n>> - for (i = 0; i < targets; i++) {\n>> - if (!write_ref || !write_ref[i])\n>> - continue;\n>> -\n>> - lock[i] = lock_ref_sha1(write_ref[i], NULL);\n>> - if (!lock[i]) {\n>> - error(\"Can't lock ref %s\", write_ref[i]);\n>> - goto unlock_and_fail;\n>> + if (write_ref) {\n>> + transaction = ref_transaction_begin(&err);\n>> + if (!transaction) {\n>> + error(\"%s\", err.buf);\n>> + goto rollback_and_fail;\n>> }\n>> }\n>> -\n>\n> Is there some reason why the transaction cannot be built up during a\n> single iteration over targets, thereby also avoiding the need for the\n> sha1[20*i] stuff? This seems like exactly the kind of situation where\n> transactions should *save* code. But perhaps I've overlooked a\n> dependency between the two loops.\n\nI did it this way to keep the changes minimal. But you are right that\nwith this we can do a larger refactoring and start saving some code.\nI can add changes to a later series to do that change but I want to\nkeep this change as small as possible for now.\n\nregards\nronnie sahlberg\n"},{"id":"246003","messageId":"CAL=YDWnoXBXKaEFV8F_gGEALY3A4Bx3OKmHe8h4=VQgPN+me=Q@mail.gmail.com","threadId":"36972","inReplyTo":"53BBEA04.5040103@alum.mit.edu","subject":"Re: [PATCH v20 30/48] refs.c: change update_ref to use a transaction","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-07-14T18:49:00Z","receivedAt":"2014-07-14T18:49:00Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"On Tue, Jul 8, 2014 at 5:54 AM, Michael Haggerty wrote:\n> On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n>> Change the update_ref helper function to use a ref transaction internally.\n>>\n>> Reviewed-by: Jonathan Nieder \n>> Signed-off-by: Ronnie Sahlberg \n>> ---\n>> refs.c | 28 ++++++++++++++++++++++++----\n>> 1 file changed, 24 insertions(+), 4 deletions(-)\n>>\n>> diff --git a/refs.c b/refs.c\n>> index 8c695ba..4bdccc5 100644\n>> --- a/refs.c\n>> +++ b/refs.c\n>> @@ -3524,11 +3524,31 @@ int update_ref(const char *action, const char *refname,\n>> const unsigned char *sha1, const unsigned char *oldval,\n>> int flags, enum action_on_err onerr)\n>> {\n>> - struct ref_lock *lock;\n>> - lock = update_ref_lock(refname, oldval, flags, NULL, onerr);\n>> - if (!lock)\n>> + struct ref_transaction *t;\n>> + struct strbuf err = STRBUF_INIT;\n>> +\n>> + t = ref_transaction_begin(&err);\n>> + if (!t ||\n>> + ref_transaction_update(t, refname, sha1, oldval, flags,\n>> + !!oldval, &err) ||\n>> + ref_transaction_commit(t, action, &err)) {\n>> + const char *str = \"update_ref failed for ref '%s': %s\";\n>> +\n>> + ref_transaction_free(t);\n>> + switch (onerr) {\n>> + case UPDATE_REFS_MSG_ON_ERR:\n>> + error(str, refname, err.buf);\n>> + break;\n>> + case UPDATE_REFS_DIE_ON_ERR:\n>> + die(str, refname, err.buf);\n>> + break;\n>> + case UPDATE_REFS_QUIET_ON_ERR:\n>> + break;\n>> + }\n>> + strbuf_release(&err);\n>> return 1;\n>> - return update_ref_write(action, refname, sha1, lock, NULL, onerr);\n>> + }\n>> + return 0;\n>> }\n>>\n>\n> Should this function be scheduled for the \"take strbuf *err argument\"\n> treatment instead of continuing to use an action_on_err parameter?\n> (Maybe you've changed this later in the patch series?)\n>\n> I'm not saying this change has to be part of the current patch series,\n> but let's consider it for the future.\n\nThere is a patch that does that in a later series. At that stage we\nget rid of all action_on_err arguments.\n\n\n>\n> Michael\n>\n> --\n> Michael Haggerty\n> mhagger@alum.mit.edu\n>\n"},{"id":"246004","messageId":"CAL=YDW=BNCSHGSVa5N6-GC4uZZLb6BPsiEjv6jXCsAXehT+kwg@mail.gmail.com","threadId":"36972","inReplyTo":"53BBF016.7040107@alum.mit.edu","subject":"Re: [PATCH v20 31/48] receive-pack.c: use a reference transaction for updating the refs","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-07-14T18:51:25Z","receivedAt":"2014-07-14T18:51:25Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"On Tue, Jul 8, 2014 at 6:20 AM, Michael Haggerty wrote:\n> On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n>> Wrap all the ref updates inside a transaction.\n>>\n>> Signed-off-by: Ronnie Sahlberg \n>> ---\n>> builtin/receive-pack.c | 96 +++++++++++++++++++++++++++++++++-----------------\n>> 1 file changed, 63 insertions(+), 33 deletions(-)\n>>\n>> diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\n>> index c323081..b51f8ae 100644\n>> --- a/builtin/receive-pack.c\n>> +++ b/builtin/receive-pack.c\n>> [...]\n>> @@ -647,6 +654,9 @@ static void check_aliased_update(struct command *cmd, struct string_list *list)\n>> char cmd_oldh[41], cmd_newh[41], dst_oldh[41], dst_newh[41];\n>> int flag;\n>>\n>> + if (cmd->error_string)\n>> + die(\"BUG: check_alised_update called with failed cmd\");\n>\n> s/check_alised_update/check_aliased_update/\n\nDone, thanks.\n"},{"id":"246008","messageId":"CAL=YDWmVpB3uj_was-Tyqkgxhd9-5LN6Pg0scG0JkzeWDp0pqA@mail.gmail.com","threadId":"36972","inReplyTo":"53BBF794.1090702@alum.mit.edu","subject":"Re: [PATCH v20 39/48] refs.c: make delete_ref use a transaction","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-07-14T20:50:51Z","receivedAt":"2014-07-14T20:50:51Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"On Tue, Jul 8, 2014 at 6:52 AM, Michael Haggerty wrote:\n> On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n>> Change delete_ref to use a ref transaction for the deletion. At the same time\n>> since we no longer have any callers of repack_without_ref we can now delete\n>> this function.\n>>\n>> Change delete_ref to return 0 on success and 1 on failure instead of the\n>> previous 0 on success either 1 or -1 on failure.\n>>\n>> Reviewed-by: Jonathan Nieder \n>> Signed-off-by: Ronnie Sahlberg \n>> ---\n>> refs.c | 34 +++++++++++++---------------------\n>> 1 file changed, 13 insertions(+), 21 deletions(-)\n>>\n>> diff --git a/refs.c b/refs.c\n>> index 3d070d5..92a06d4 100644\n>> --- a/refs.c\n>> +++ b/refs.c\n>> @@ -2544,11 +2544,6 @@ int repack_without_refs(const char **refnames, int n, struct strbuf *err)\n>> return ret;\n>> }\n>>\n>> -static int repack_without_ref(const char *refname)\n>> -{\n>> - return repack_without_refs(&refname, 1, NULL);\n>> -}\n>> -\n>> static int delete_ref_loose(struct ref_lock *lock, int flag)\n>> {\n>> if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {\n>> @@ -2566,24 +2561,21 @@ static int delete_ref_loose(struct ref_lock *lock, int flag)\n>>\n>> int delete_ref(const char *refname, const unsigned char *sha1, int delopt)\n>> {\n>> - struct ref_lock *lock;\n>> - int ret = 0, flag = 0;\n>> + struct ref_transaction *transaction;\n>> + struct strbuf err = STRBUF_INIT;\n>>\n>> - lock = lock_ref_sha1_basic(refname, sha1, delopt, &flag);\n>\n> The old code checked that the old value of refname was sha1, regardless\n> of whether sha1 was null_sha1. Presumably callers never set sha1 to\n> null_sha1...\n\nThey sometimes do.\n\n>\n>> - if (!lock)\n>> + transaction = ref_transaction_begin(&err);\n>> + if (!transaction ||\n>> + ref_transaction_delete(transaction, refname, sha1, delopt,\n>> + sha1 && !is_null_sha1(sha1), &err) ||\n>\n> ...But the new code explicitly skips the check if sha1 is null_sha1.\n> This shouldn't make a practical difference, because presumably callers\n> never set sha1 to null_sha1.\n\nThere are actually a few cases where callers do call delete_ref() with\nsha1 == null_sha1.\nFor example fast-import.c:update_branch() will do this is the ref can\nnot be resolved.\nIt can also happen in builtin/update-ref.c where we are passing user\nsupplied data into the call to delete_ref.\n\nSo I think the current behaviour should be ok.\n\nThere are a few options we could do:\nWe could change the semantics for ref_transaction_update|delete and\nstart allowing\n have_old==1\n old_sha1==null_sha1\nand have this behave the same way as\n have_old==0\nbut I think that would be horrible I think.\n\nWe could also change all callers to delete_ref() to be careful to only\nspecify a sha1 IFF it is not null_sha1\nbut that would just mean we require all callers to do this type of check.\nBut that would also be fragile since if/when we get new callers to\ndelete_ref we risk breaking delete_ref if we are not careful.\n\n\nI think the least bad option is to just have this check in\ndelete_ref() as now and have the semantics for delete_ref be that if\nsha1 is either NULL or null_sha1 then it means we don't care what the\nold value is.\n\n\n\nregards\nronnie sahlberg\n"},{"id":"246033","messageId":"CAL=YDWng3eBnbUAGVBoESio6d75Lz47oY3ntLFa1ktY2513bXg@mail.gmail.com","threadId":"36972","inReplyTo":"53BBE578.9030703@alum.mit.edu","subject":"Re: [PATCH v20 25/48] replace.c: use the ref transaction functions for updates","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-07-14T21:19:59Z","receivedAt":"2014-07-14T21:19:59Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"On Tue, Jul 8, 2014 at 5:35 AM, Michael Haggerty wrote:\n> On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n>> Update replace.c to use ref transactions for updates.\n>>\n>> Reviewed-by: Jonathan Nieder \n>> Signed-off-by: Ronnie Sahlberg \n>> ---\n>> builtin/replace.c | 15 +++++++++------\n>> 1 file changed, 9 insertions(+), 6 deletions(-)\n>>\n>> diff --git a/builtin/replace.c b/builtin/replace.c\n>> index 1bb491d..7528f3d 100644\n>> --- a/builtin/replace.c\n>> +++ b/builtin/replace.c\n>> @@ -153,7 +153,8 @@ static int replace_object_sha1(const char *object_ref,\n>> unsigned char prev[20];\n>> enum object_type obj_type, repl_type;\n>> char ref[PATH_MAX];\n>> - struct ref_lock *lock;\n>> + struct ref_transaction *transaction;\n>> + struct strbuf err = STRBUF_INIT;\n>>\n>> obj_type = sha1_object_info(object, NULL);\n>> repl_type = sha1_object_info(repl, NULL);\n>> @@ -166,12 +167,14 @@ static int replace_object_sha1(const char *object_ref,\n>>\n>> check_ref_valid(object, prev, ref, sizeof(ref), force);\n>>\n>> - lock = lock_any_ref_for_update(ref, prev, 0, NULL);\n>> - if (!lock)\n>> - die(\"%s: cannot lock the ref\", ref);\n>> - if (write_ref_sha1(lock, repl, NULL) < 0)\n>> - die(\"%s: cannot update the ref\", ref);\n>> + transaction = ref_transaction_begin(&err);\n>> + if (!transaction ||\n>> + ref_transaction_update(transaction, ref, repl, prev,\n>> + 0, !is_null_sha1(prev), &err) ||\n>\n> Same problem here. You need\n>\n> s/!is_null_sha1(prev)/1/\n>\n\nI think we need it.\nprev can be null_sha1 here if check_ref_valid fails to resolve the ref.\n\n\n>> + ref_transaction_commit(transaction, NULL, &err))\n>> + die(\"%s\", err.buf);\n>>\n>> + ref_transaction_free(transaction);\n>> return 0;\n>> }\n>>\n>>\n>\n>\n> --\n> Michael Haggerty\n> mhagger@alum.mit.edu\n>\n"},{"id":"246037","messageId":"CAL=YDWkaW_di822qYsf5iKHqfriTjJm6exMVTpbvBunijqgUmQ@mail.gmail.com","threadId":"36972","inReplyTo":"53BBE2D6.2070809@alum.mit.edu","subject":"Re: [PATCH v20 27/48] sequencer.c: use ref transactions for all ref updates","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-07-14T22:20:51Z","receivedAt":"2014-07-14T22:20:51Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"On Tue, Jul 8, 2014 at 5:23 AM, Michael Haggerty wrote:\n> On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n>> Change to use ref transactions for all updates to refs.\n>>\n>> Reviewed-by: Jonathan Nieder \n>> Signed-off-by: Ronnie Sahlberg \n>> ---\n>> sequencer.c | 24 ++++++++++++++++--------\n>> 1 file changed, 16 insertions(+), 8 deletions(-)\n>>\n>> diff --git a/sequencer.c b/sequencer.c\n>> index 0a80c58..fd8acaf 100644\n>> --- a/sequencer.c\n>> +++ b/sequencer.c\n>> @@ -272,23 +272,31 @@ static int error_dirty_index(struct replay_opts *opts)\n>> static int fast_forward_to(const unsigned char *to, const unsigned char *from,\n>> int unborn, struct replay_opts *opts)\n>> {\n>> - struct ref_lock *ref_lock;\n>> + struct ref_transaction *transaction;\n>> struct strbuf sb = STRBUF_INIT;\n>> - int ret;\n>> + struct strbuf err = STRBUF_INIT;\n>>\n>> read_cache();\n>> if (checkout_fast_forward(from, to, 1))\n>> exit(1); /* the callee should have complained already */\n>> - ref_lock = lock_any_ref_for_update(\"HEAD\", unborn ? null_sha1 : from,\n>> - 0, NULL);\n>> - if (!ref_lock)\n>> - return error(_(\"Failed to lock HEAD during fast_forward_to\"));\n>\n> I think you've changed the semantics when unborn is set. Please note\n> that lock_any_ref_for_update() behaves differently if old_sha1 is NULL\n> (when no check is done) vs. when it is null_sha1 (when it verifies that\n> the reference didn't previously exist). So when unborn is true, the old\n> code verifies that the reference previously didn't exist...\n>\n>>\n>> strbuf_addf(&sb, \"%s: fast-forward\", action_name(opts));\n>> - ret = write_ref_sha1(ref_lock, to, sb.buf);\n>> +\n>> + transaction = ref_transaction_begin(&err);\n>> + if (!transaction ||\n>> + ref_transaction_update(transaction, \"HEAD\", to, from,\n>> + 0, !unborn, &err) ||\n>\n> ...whereas when unborn is true, the new code does no check at all. I\n> think you want\n>\n> ref_transaction_update(transaction, \"HEAD\",\n> to, unborn ? null_sha1 : from,\n> 0, 1, &err) ||\n>\n\nRight. Fixed.\n\nThanks.\n"},{"id":"246065","messageId":"CAL=YDW=3aYhMNjCtT7BV=RDkRN2=Ac0uC_iEiQrO45i4C3zeDw@mail.gmail.com","threadId":"36972","inReplyTo":"53BC07FC.8080601@alum.mit.edu","subject":"Re: [PATCH v20 43/48] refs.c: move the check for valid refname to lock_ref_sha1_basic","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-07-15T16:40:18Z","receivedAt":"2014-07-15T16:40:18Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"On Tue, Jul 8, 2014 at 8:02 AM, Michael Haggerty wrote:\n> On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n>> Move the check for check_refname_format from lock_any_ref_for_update\n>> to lock_ref_sha1_basic. At some later stage we will get rid of\n>> lock_any_ref_for_update completely.\n>>\n>> If lock_ref_sha1_basic fails the check_refname_format test, set errno to\n>> EINVAL before returning NULL. This to guarantee that we will not return an\n>> error without updating errno.\n>>\n>> This leaves lock_any_ref_for_updates as a no-op wrapper which could be removed.\n>> But this wrapper is also called from an external caller and we will soon\n>> make changes to the signature to lock_ref_sha1_basic that we do not want to\n>> expose to that caller.\n>>\n>> This changes semantics for lock_ref_sha1_basic slightly. With this change\n>> it is no longer possible to open a ref that has a badly name which breaks\n>\n> s/badly name/bad name,/\n>\n>> any codepaths that tries to open and repair badly named refs. The normal refs\n>\n> s/tries/try/\n>\n>> API should not allow neither creating nor accessing refs with invalid names.\n>\n> s/not allow neither/allow neither/\n>\n>> If we need such recovery code we could add it as an option to git fsck and have\n>> git fsck be the only sanctioned way of bypassing the normal API and checks.\n>\n> I like the sentiment, but in the real world I'm not sure we can take\n> such a step based only on good intentions. Which callers would be\n> affected? Where is this \"git fsck\" code that would be needed to help\n> people rescue their repos?\n>\n\nI think the repair things are already busted since a while, so I don't\nthink this will make things worse,\nbut I will change it to make it better than this patch and better than\ncurrent master, please see below.\n\n\n$ cp .git/refs/heads/master .git/refs/heads/master.....@\\*@\\\\.\n$ git branch\n fatal: Reference has invalid format: 'refs/heads/master.....@*@\\.'\n$ git branch -D master.....@\\*@\\\\.\n error: branch 'master.....@*@\\.' not found.\n\ngit fsck does report that there is a problem with a broken ref\n fatal: Reference has invalid format: 'refs/heads/master.....@*@\\.'\n\nBut I don't think there is any way to fix this other than manually\ndeleting the file from the command line.\n(this is because we need to do a resolve_ref_unsafe which will not\nwork and if we can not do a resolve_ref_unsafe we can not delete the\nref,\n there is also the issue where we can not read the ref into ref-cache)\n\n\nWhat I suggest doing here is to create a new patch towards the end of\nthe series that will :\n* change the resolve_ref_unsafe(... , int reading, ...) argument to be\na bitmask of flags with\n #define RESOLVE_REF_READING 0x01 (== current flag)\n #define RESOLVE_REF_ALLOW_BAD_NAME 0x02 (== disable checking the\nrefname format during resolve)\n* then provide the flag for RESOLVE_REF_ALLOW_BAD_NAME for the cases\nwhere we try to resolve a ref in builtin/branch.c where we try to\ndelete a ref\n\n* then also pass the same flag to lock_ref_sha1_basic when we are\ndeleting a ref from transaction_commit so we can disable the \"check\nrefname\" check there too.\n\nI think that is all that is needed but I will see if there are\nadditional changes needed once I implement it.\n\n\n\nThis will mean that the semantics will become :\n* you can not create or access a branch with a bad name since both\nresolving it and locking it will fail.\n* but you can always delete a branch regardless of whether the name is\ngood or bad.\n(this will also mean that you will be able to rename a bad branch name\nto a new good name)\n\nwhich I think would be pretty sane semantics.\n\n\nI will implement these changes as a separate patch.\n\nComments?\n\n\nregards\nronnie sahlberg\n\n\n\n> I can also imagine that we will tighten up the check_refname_format\n> checks in the future; for example, I think it would be a good idea to\n> prohibit reference names that start with '-' because it is almost\n> impossible to work with them (their names look like command-line\n> options). If we ever make a change like that, we will need some amount\n> of tolerance in git versions around the transition.\n>\n> So...I like the idea of enforcing refname checks at the lowest level\n> possible, but I think that the change you propose is too abrupt. I\n> think it needs either more careful analysis showing that it won't hurt\n> anybody, or some kind of tooling or non-strict mode that people can use\n> to fix their repositories.\n>\n> Michael\n>\n>> Signed-off-by: Ronnie Sahlberg \n>> ---\n>> refs.c | 7 +++++--\n>> 1 file changed, 5 insertions(+), 2 deletions(-)\n>>\n>> diff --git a/refs.c b/refs.c\n>> index 389a55f..bccf8c3 100644\n>> --- a/refs.c\n>> +++ b/refs.c\n>> @@ -2088,6 +2088,11 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,\n>> int missing = 0;\n>> int attempts_remaining = 3;\n>>\n>> + if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n>> + errno = EINVAL;\n>> + return NULL;\n>> + }\n>> +\n>> lock = xcalloc(1, sizeof(struct ref_lock));\n>> lock->lock_fd = -1;\n>>\n>> @@ -2179,8 +2184,6 @@ struct ref_lock *lock_any_ref_for_update(const char *refname,\n>> const unsigned char *old_sha1,\n>> int flags, int *type_p)\n>> {\n>> - if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL))\n>> - return NULL;\n>> return lock_ref_sha1_basic(refname, old_sha1, flags, type_p);\n>> }\n>>\n>>\n>\n>\n> --\n> Michael Haggerty\n> mhagger@alum.mit.edu\n>\n"},{"id":"246070","messageId":"20140715180424.GJ12427@google.com","threadId":"36972","inReplyTo":"53BC07FC.8080601@alum.mit.edu","subject":"Re: [PATCH v20 43/48] refs.c: move the check for valid refname to lock_ref_sha1_basic","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2014-07-15T18:04:24Z","receivedAt":"2014-07-15T18:04:24Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Michael Haggerty wrote:\n\n> So...I like the idea of enforcing refname checks at the lowest level\n> possible, but I think that the change you propose is too abrupt. I\n> think it needs either more careful analysis showing that it won't hurt\n> anybody, or some kind of tooling or non-strict mode that people can use\n> to fix their repositories.\n\nThe recovery code has been broken for a while, so I don't see harm\nin this change.\n\nHow to take care of the recovery use case is another question. FWIW I\nalso would prefer if \"git update-ref -d\" or \"git branch -D\" could be\nused to delete corrupt refs instead of having to use fsck (since a\nfsck run can take a while), but that's a question for a later series.\n\nIn an ideal world, the low-level functions would allow *reading* and\n*deleting* poorly named refs (even without any special flag) but not\ncreating them. Is that doable? The main complication I can see is\niteration: would iteration skip poorly named refs and warn, or would\nsomething more complicated be needed?\n\nThanks,\nJonathan\n"},{"id":"246071","messageId":"20140715180709.GK12427@google.com","threadId":"36972","inReplyTo":"CAL=YDW=3aYhMNjCtT7BV=RDkRN2=Ac0uC_iEiQrO45i4C3zeDw@mail.gmail.com","subject":"Re: [PATCH v20 43/48] refs.c: move the check for valid refname to lock_ref_sha1_basic","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2014-07-15T18:07:09Z","receivedAt":"2014-07-15T18:07:09Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Ronnie Sahlberg wrote:\n\n> What I suggest doing here is to create a new patch towards the end of\n> the series that will :\n> * change the resolve_ref_unsafe(... , int reading, ...) argument to be\n> a bitmask of flags with\n> #define RESOLVE_REF_READING 0x01 (== current flag)\n> #define RESOLVE_REF_ALLOW_BAD_NAME 0x02 (== disable checking the\n> refname format during resolve)\n> * then provide the flag for RESOLVE_REF_ALLOW_BAD_NAME for the cases\n> where we try to resolve a ref in builtin/branch.c where we try to\n> delete a ref\n>\n> * then also pass the same flag to lock_ref_sha1_basic when we are\n> deleting a ref from transaction_commit so we can disable the \"check\n> refname\" check there too.\n\nYeah, that sounds lovely.\n\nI wasn't able to reproduce a regression or convince myself there is\none so I think it's okay if that happens in a separate series. But\ndoing it now would be fine, too.\n\nThanks,\nJonathan\n"},{"id":"246076","messageId":"xmqqtx6iv5cs.fsf@gitster.dls.corp.google.com","threadId":"36972","inReplyTo":"20140715180424.GJ12427@google.com","subject":"Re: [PATCH v20 43/48] refs.c: move the check for valid refname to lock_ref_sha1_basic","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-07-15T18:34:11Z","receivedAt":"2014-07-15T18:34:11Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Nieder writes:\n\n> How to take care of the recovery use case is another question. FWIW I\n> also would prefer if \"git update-ref -d\" or \"git branch -D\" could be\n> used to delete corrupt refs instead of having to use fsck (since a\n> fsck run can take a while), but that's a question for a later series.\n\nGood thinking.\n\n> In an ideal world, the low-level functions would allow *reading* and\n> *deleting* poorly named refs (even without any special flag) but not\n> creating them. Is that doable? The main complication I can see is\n> iteration: would iteration skip poorly named refs and warn, or would\n> something more complicated be needed?\n\nI somehow thought that was what we have always designed for, which\nDO_FOR_EACH_INCLUDE_BROKEN was a part of.\n"},{"id":"246086","messageId":"CAL=YDWkYAg-0h3ZwiyZGtUHFEv1KEti_uURTwgbZE9xT_P_XSQ@mail.gmail.com","threadId":"36972","inReplyTo":"20140715180424.GJ12427@google.com","subject":"Re: [PATCH v20 43/48] refs.c: move the check for valid refname to lock_ref_sha1_basic","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-07-15T19:34:08Z","receivedAt":"2014-07-15T19:34:08Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"On Tue, Jul 15, 2014 at 11:04 AM, Jonathan Nieder wrote:\n> Michael Haggerty wrote:\n>\n>> So...I like the idea of enforcing refname checks at the lowest level\n>> possible, but I think that the change you propose is too abrupt. I\n>> think it needs either more careful analysis showing that it won't hurt\n>> anybody, or some kind of tooling or non-strict mode that people can use\n>> to fix their repositories.\n>\n> The recovery code has been broken for a while, so I don't see harm\n> in this change.\n>\n> How to take care of the recovery use case is another question. FWIW I\n> also would prefer if \"git update-ref -d\" or \"git branch -D\" could be\n> used to delete corrupt refs instead of having to use fsck (since a\n> fsck run can take a while), but that's a question for a later series.\n>\n> In an ideal world, the low-level functions would allow *reading* and\n> *deleting* poorly named refs (even without any special flag) but not\n> creating them. Is that doable?\n\nThat should be doable. Ill add these repairs as 3-4 new patches at the\nend of the current patch series.\n\n> The main complication I can see is\n> iteration: would iteration skip poorly named refs and warn, or would\n> something more complicated be needed?\n\nRight now, \"git branch\" will error and abort right away when it\nfinds the first bad ref. I haven't checked the exact spot it does this\nyet but I suspect it is when building the ref-cache.\nI will solve the cases for create and delete for now.\n\n\nWhat/how to handle iterables will require more thought.\nRight now, since these refs will be filtered out and never end up in\nref-cache, either from loose refs or from packed refs\nit does mean that anyone that uses an iterator is guaranteed to only\nget refs with valid names passed back to them.\nWe would need to audit all code that uses iterators and make sure it\ncan handle the case where the callback is suddenly\ninvoked with a bad refname.\n\n>\n> Thanks,\n> Jonathan\n"},{"id":"246087","messageId":"CAL=YDWmz=yQGs8HcwO1rkg1UfBy6jEhipA2vkO+jg9Wcu1GaTA@mail.gmail.com","threadId":"36972","inReplyTo":"xmqqtx6iv5cs.fsf@gitster.dls.corp.google.com","subject":"Re: [PATCH v20 43/48] refs.c: move the check for valid refname to lock_ref_sha1_basic","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-07-15T19:35:28Z","receivedAt":"2014-07-15T19:35:28Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"On Tue, Jul 15, 2014 at 11:34 AM, Junio C Hamano wrote:\n> Jonathan Nieder writes:\n>\n>> How to take care of the recovery use case is another question. FWIW I\n>> also would prefer if \"git update-ref -d\" or \"git branch -D\" could be\n>> used to delete corrupt refs instead of having to use fsck (since a\n>> fsck run can take a while), but that's a question for a later series.\n>\n> Good thinking.\n>\n>> In an ideal world, the low-level functions would allow *reading* and\n>> *deleting* poorly named refs (even without any special flag) but not\n>> creating them. Is that doable? The main complication I can see is\n>> iteration: would iteration skip poorly named refs and warn, or would\n>> something more complicated be needed?\n>\n> I somehow thought that was what we have always designed for, which\n> DO_FOR_EACH_INCLUDE_BROKEN was a part of.\n\nI think that include broken only handles the case where the ref itself\nis bad, not when the refname is bad.\nI.e. it affects cases where the sha1 does not exist or the symref\npoints to nowhere.\n"},{"id":"246095","messageId":"CAL=YDWmc2gkw=8YavWHyLUAD4du7saPrKzPKT+dsCfdZJz1EiA@mail.gmail.com","threadId":"36972","inReplyTo":"CAL=YDWkYAg-0h3ZwiyZGtUHFEv1KEti_uURTwgbZE9xT_P_XSQ@mail.gmail.com","subject":"Re: [PATCH v20 43/48] refs.c: move the check for valid refname to lock_ref_sha1_basic","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-07-15T20:58:03Z","receivedAt":"2014-07-15T20:58:03Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"On Tue, Jul 15, 2014 at 12:34 PM, Ronnie Sahlberg wrote:\n> On Tue, Jul 15, 2014 at 11:04 AM, Jonathan Nieder wrote:\n>> Michael Haggerty wrote:\n>>\n>>> So...I like the idea of enforcing refname checks at the lowest level\n>>> possible, but I think that the change you propose is too abrupt. I\n>>> think it needs either more careful analysis showing that it won't hurt\n>>> anybody, or some kind of tooling or non-strict mode that people can use\n>>> to fix their repositories.\n>>\n>> The recovery code has been broken for a while, so I don't see harm\n>> in this change.\n>>\n>> How to take care of the recovery use case is another question. FWIW I\n>> also would prefer if \"git update-ref -d\" or \"git branch -D\" could be\n>> used to delete corrupt refs instead of having to use fsck (since a\n>> fsck run can take a while), but that's a question for a later series.\n>>\n>> In an ideal world, the low-level functions would allow *reading* and\n>> *deleting* poorly named refs (even without any special flag) but not\n>> creating them. Is that doable?\n>\n> That should be doable. Ill add these repairs as 3-4 new patches at the\n> end of the current patch series.\n>\n>> The main complication I can see is\n>> iteration: would iteration skip poorly named refs and warn, or would\n>> something more complicated be needed?\n>\n> Right now, \"git branch\" will error and abort right away when it\n> finds the first bad ref. I haven't checked the exact spot it does this\n> yet but I suspect it is when building the ref-cache.\n> I will solve the cases for create and delete for now.\n>\n>\n> What/how to handle iterables will require more thought.\n> Right now, since these refs will be filtered out and never end up in\n> ref-cache, either from loose refs or from packed refs\n> it does mean that anyone that uses an iterator is guaranteed to only\n> get refs with valid names passed back to them.\n> We would need to audit all code that uses iterators and make sure it\n> can handle the case where the callback is suddenly\n> invoked with a bad refname.\n>\n>>\n>> Thanks,\n>> Jonathan\n\nThe following seems to do the trick to allow deleting a bad ref. We\nwould need something for the iterator too.\nSince this touches the same areas that my ~100 other ref transaction\npatches that are queued up do, I\nwould like to wait applying this patch until once the next few series\nare finished and merged.\n(to avoid having to do a lot of rebases and fix legio of merge\nconflicts that this would introduce in my branches).\n\n\nTreat this as an approximation on what the fix to repair git branch -D\nwill look like once the time comes.\n\nregards\nronnie sahlberg\n\n\n\n===\n\ncommit a2514213999a192c9995a3a5e1479d7d09e2c083\nAuthor: Ronnie Sahlberg \nDate: Tue Jul 15 12:59:36 2014 -0700\n\n refs.c: fix handling of badly named refs\n\n We currently do not handle badly named refs well :\n $ cp .git/refs/heads/master .git/refs/heads/master.....@\\*@\\\\.\n $ git branch\n fatal: Reference has invalid format: 'refs/heads/master.....@*@\\.'\n $ git branch -D master.....@\\*@\\\\.\n error: branch 'master.....@*@\\.' not found.\n\n But we can not really recover from a badly named ref with less than\n manually deleting the .git/refs/heads/ file.\n\n Change resolve_ref_unsafe to take a flags field instead of a 'reading'\n boolean and update all callers that used a non-zero value for reading\n to pass the flag RESOLVE_REF_READING instead.\n Add another flag RESOLVE_REF_ALLOW_BAD_NAME that will make\n resolve_ref_unsafe skip checking the refname for sanity and use this\n from branch.c so that we will be able to call resolve_ref_unsafe on such\n refs when trying to delete it.\n Add checks for refname sanity when updating (not deleting) a ref in\n ref_transaction_update and in ref_transaction_create to make the transaction\n fail if an attempt is made to create/update a badly named ref.\n Since all ref changes will later go through the transaction layer this means\n we no longer need to check for and fail for bad refnames in\n lock_ref_sha1_basic.\n\n Change lock_ref_sha1_basic to not fail for bad refnames. Just check the\n refname, and print an error, and remember that the refname is bad so that\n we can skip calling verify_lock().\n\n Signed-off-by: Ronnie Sahlberg \n\ndiff --git a/builtin/blame.c b/builtin/blame.c\nindex 662e3fe..76340e2 100644\n--- a/builtin/blame.c\n+++ b/builtin/blame.c\n@@ -2278,7 +2278,7 @@ static struct commit\n*fake_working_tree_commit(struct diff_options *opt,\n commit->object.type = OBJ_COMMIT;\n parent_tail = &commit->parents;\n\n- if (!resolve_ref_unsafe(\"HEAD\", head_sha1, 1, NULL))\n+ if (!resolve_ref_unsafe(\"HEAD\", head_sha1, RESOLVE_REF_READING, NULL))\n die(\"no such ref: HEAD\");\n\n parent_tail = append_parent(parent_tail, head_sha1);\ndiff --git a/builtin/branch.c b/builtin/branch.c\nindex 652b1d2..5c95656 100644\n--- a/builtin/branch.c\n+++ b/builtin/branch.c\n@@ -129,7 +129,8 @@ static int branch_merged(int kind, const char *name,\n branch->merge[0] &&\n branch->merge[0]->dst &&\n (reference_name = reference_name_to_free =\n- resolve_refdup(branch->merge[0]->dst, sha1, 1, NULL)) != NULL)\n+ resolve_refdup(branch->merge[0]->dst, sha1,\n+ RESOLVE_REF_READING, NULL)) != NULL)\n reference_rev = lookup_commit_reference(sha1);\n }\n if (!reference_rev)\n@@ -233,7 +234,8 @@ static int delete_branches(int argc, const char\n**argv, int force, int kinds,\n free(name);\n\n name = mkpathdup(fmt, bname.buf);\n- target = resolve_ref_unsafe(name, sha1, 0, &flags);\n+ target = resolve_ref_unsafe(name, sha1,\n+ RESOLVE_REF_ALLOW_BAD_NAME, &flags);\n if (!target ||\n (!(flags & REF_ISSYMREF) && is_null_sha1(sha1))) {\n error(remote_branch\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex b12989d..f7307e6 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -622,7 +622,7 @@ static int checkout(void)\n if (option_no_checkout)\n return 0;\n\n- head = resolve_refdup(\"HEAD\", sha1, 1, NULL);\n+ head = resolve_refdup(\"HEAD\", sha1, RESOLVE_REF_READING, NULL);\n if (!head) {\n warning(_(\"remote HEAD refers to nonexistent ref, \"\n \"unable to checkout.\\n\"));\ndiff --git a/builtin/fmt-merge-msg.c b/builtin/fmt-merge-msg.c\nindex 3906eda..d8ab177 100644\n--- a/builtin/fmt-merge-msg.c\n+++ b/builtin/fmt-merge-msg.c\n@@ -602,7 +602,7 @@ int fmt_merge_msg(struct strbuf *in, struct strbuf *out,\n\n /* get current branch */\n current_branch = current_branch_to_free =\n- resolve_refdup(\"HEAD\", head_sha1, 1, NULL);\n+ resolve_refdup(\"HEAD\", head_sha1, RESOLVE_REF_READING, NULL);\n if (!current_branch)\n die(\"No current branch\");\n if (starts_with(current_branch, \"refs/heads/\"))\ndiff --git a/builtin/for-each-ref.c b/builtin/for-each-ref.c\nindex 4135980..a5833fd 100644\n--- a/builtin/for-each-ref.c\n+++ b/builtin/for-each-ref.c\n@@ -649,7 +649,8 @@ static void populate_value(struct refinfo *ref)\n\n if (need_symref && (ref->flag & REF_ISSYMREF) && !ref->symref) {\n unsigned char unused1[20];\n- ref->symref = resolve_refdup(ref->refname, unused1, 1, NULL);\n+ ref->symref = resolve_refdup(ref->refname, unused1,\n+ RESOLVE_REF_READING, NULL);\n if (!ref->symref)\n ref->symref = \"\";\n }\n@@ -707,7 +708,8 @@ static void populate_value(struct refinfo *ref)\n const char *head;\n unsigned char sha1[20];\n\n- head = resolve_ref_unsafe(\"HEAD\", sha1, 1, NULL);\n+ head = resolve_ref_unsafe(\"HEAD\", sha1,\n+ RESOLVE_REF_READING, NULL);\n if (!strcmp(ref->refname, head))\n v->s = \"*\";\n else\ndiff --git a/builtin/log.c b/builtin/log.c\nindex a7ba211..92db809 100644\n--- a/builtin/log.c\n+++ b/builtin/log.c\n@@ -1395,7 +1395,8 @@ int cmd_format_patch(int argc, const char\n**argv, const char *prefix)\n if (check_head) {\n unsigned char sha1[20];\n const char *ref;\n- ref = resolve_ref_unsafe(\"HEAD\", sha1, 1, NULL);\n+ ref = resolve_ref_unsafe(\"HEAD\", sha1,\n+ RESOLVE_REF_READING, NULL);\n if (ref && starts_with(ref, \"refs/heads/\"))\n branch_name = xstrdup(ref + strlen(\"refs/heads/\"));\n else\ndiff --git a/builtin/remote.c b/builtin/remote.c\nindex 401feb3..be8ebac 100644\n--- a/builtin/remote.c\n+++ b/builtin/remote.c\n@@ -568,7 +568,8 @@ static int read_remote_branches(const char *refname,\n strbuf_addf(&buf, \"refs/remotes/%s/\", rename->old);\n if (starts_with(refname, buf.buf)) {\n item = string_list_append(rename->remote_branches, xstrdup(refname));\n- symref = resolve_ref_unsafe(refname, orig_sha1, 1, &flag);\n+ symref = resolve_ref_unsafe(refname, orig_sha1,\n+ RESOLVE_REF_READING, &flag);\n if (flag & REF_ISSYMREF)\n item->util = xstrdup(symref);\n else\n@@ -704,7 +705,7 @@ static int mv(int argc, const char **argv)\n int flag = 0;\n unsigned char sha1[20];\n\n- read_ref_full(item->string, sha1, 1, &flag);\n+ read_ref_full(item->string, sha1, RESOLVE_REF_READING, &flag);\n if (!(flag & REF_ISSYMREF))\n continue;\n if (delete_ref(item->string, NULL, REF_NODEREF))\ndiff --git a/builtin/show-branch.c b/builtin/show-branch.c\nindex d873172..a9a5eb3 100644\n--- a/builtin/show-branch.c\n+++ b/builtin/show-branch.c\n@@ -727,7 +727,8 @@ int cmd_show_branch(int ac, const char **av, const\nchar *prefix)\n if (ac == 0) {\n static const char *fake_av[2];\n\n- fake_av[0] = resolve_refdup(\"HEAD\", sha1, 1, NULL);\n+ fake_av[0] = resolve_refdup(\"HEAD\", sha1,\n+ RESOLVE_REF_READING, NULL);\n fake_av[1] = NULL;\n av = fake_av;\n ac = 1;\n@@ -789,7 +790,8 @@ int cmd_show_branch(int ac, const char **av, const\nchar *prefix)\n }\n }\n\n- head_p = resolve_ref_unsafe(\"HEAD\", head_sha1, 1, NULL);\n+ head_p = resolve_ref_unsafe(\"HEAD\", head_sha1,\n+ RESOLVE_REF_READING, NULL);\n if (head_p) {\n head_len = strlen(head_p);\n memcpy(head, head_p, head_len + 1);\ndiff --git a/bundle.c b/bundle.c\nindex 1222952..8aaf5f8 100644\n--- a/bundle.c\n+++ b/bundle.c\n@@ -311,7 +311,7 @@ int create_bundle(struct bundle_header *header,\nconst char *path,\n continue;\n if (dwim_ref(e->name, strlen(e->name), sha1, &ref) != 1)\n continue;\n- if (read_ref_full(e->name, sha1, 1, &flag))\n+ if (read_ref_full(e->name, sha1, RESOLVE_REF_READING, &flag))\n flag = 0;\n display_ref = (flag & REF_ISSYMREF) ? e->name : ref;\n\ndiff --git a/cache.h b/cache.h\nindex 4ca4583..aab246d 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -948,7 +948,7 @@ extern int get_sha1_hex(const char *hex, unsigned\nchar *sha1);\n\n extern char *sha1_to_hex(const unsigned char *sha1); /* static buffer\nresult! */\n extern int read_ref_full(const char *refname, unsigned char *sha1,\n- int reading, int *flags);\n+ int flags, int *ref_flag);\n extern int read_ref(const char *refname, unsigned char *sha1);\n\n /*\n@@ -960,17 +960,17 @@ extern int read_ref(const char *refname,\nunsigned char *sha1);\n * or the input ref.\n *\n * If the reference cannot be resolved to an object, the behavior\n- * depends on the \"reading\" argument:\n+ * depends on the RESOLVE_REF_READING flag:\n *\n- * - If reading is set, return NULL.\n+ * - If RESOLVE_REF_READING is set, return NULL.\n *\n- * - If reading is not set, clear sha1 and return the name of the last\n- * reference name in the chain, which will either be a non-symbolic\n+ * - If RESOLVE_REF_READING is not set, clear sha1 and return the name of\n+ * the last reference name in the chain, which will either be a non-symbolic\n * reference or an undefined reference. If this is a prelude to\n * \"writing\" to the ref, the return value is the name of the ref\n * that will actually be created or changed.\n *\n- * If flag is non-NULL, set the value that it points to the\n+ * If ref_flag is non-NULL, set the value that it points to the\n * combination of REF_ISPACKED (if the reference was found among the\n * packed references) and REF_ISSYMREF (if the initial reference was a\n * symbolic reference).\n@@ -981,8 +981,10 @@ extern int read_ref(const char *refname, unsigned\nchar *sha1);\n *\n * errno is set to something meaningful on error.\n */\n-extern const char *resolve_ref_unsafe(const char *ref, unsigned char\n*sha1, int reading, int *flag);\n-extern char *resolve_refdup(const char *ref, unsigned char *sha1, int\nreading, int *flag);\n+#define RESOLVE_REF_READING 0x01\n+#define RESOLVE_REF_ALLOW_BAD_NAME 0x02\n+extern const char *resolve_ref_unsafe(const char *ref, unsigned char\n*sha1, int flags, int *ref_flag);\n+extern char *resolve_refdup(const char *ref, unsigned char *sha1, int\nflags, int *ref_flag);\n\n extern int dwim_ref(const char *str, int len, unsigned char *sha1, char **ref);\n extern int dwim_log(const char *str, int len, unsigned char *sha1, char **ref);\ndiff --git a/http-backend.c b/http-backend.c\nindex d2c0a62..059f790 100644\n--- a/http-backend.c\n+++ b/http-backend.c\n@@ -417,7 +417,8 @@ static int show_head_ref(const char *refname,\nconst unsigned char *sha1,\n\n if (flag & REF_ISSYMREF) {\n unsigned char unused[20];\n- const char *target = resolve_ref_unsafe(refname, unused, 1, NULL);\n+ const char *target = resolve_ref_unsafe(refname, unused,\n+ RESOLVE_REF_READING, NULL);\n const char *target_nons = strip_namespace(target);\n\n strbuf_addf(buf, \"ref: %s\\n\", target_nons);\ndiff --git a/reflog-walk.c b/reflog-walk.c\nindex 9ce8b53..d80a42a 100644\n--- a/reflog-walk.c\n+++ b/reflog-walk.c\n@@ -48,7 +48,8 @@ static struct complete_reflogs\n*read_complete_reflog(const char *ref)\n unsigned char sha1[20];\n const char *name;\n void *name_to_free;\n- name = name_to_free = resolve_refdup(ref, sha1, 1, NULL);\n+ name = name_to_free = resolve_refdup(ref, sha1,\n+ RESOLVE_REF_READING, NULL);\n if (name) {\n for_each_reflog_ent(name, read_one_reflog, reflogs);\n free(name_to_free);\ndiff --git a/refs.c b/refs.c\nindex 221d8a7..dd9d61d 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1194,7 +1194,8 @@ static void read_loose_refs(const char *dirname,\nstruct ref_dir *dir)\n hashclr(sha1);\n flag |= REF_ISBROKEN;\n }\n- } else if (read_ref_full(refname.buf, sha1, 1, &flag)) {\n+ } else if (read_ref_full(refname.buf, sha1,\n+ RESOLVE_REF_READING, &flag)) {\n hashclr(sha1);\n flag |= REF_ISBROKEN;\n }\n@@ -1346,21 +1347,21 @@ static const char\n*handle_missing_loose_ref(const char *refname,\n }\n\n /* This function needs to return a meaningful errno on failure */\n-const char *resolve_ref_unsafe(const char *refname, unsigned char\n*sha1, int reading, int *flag)\n+const char *resolve_ref_unsafe(const char *refname, unsigned char\n*sha1, int flags, int *ref_flag)\n {\n int depth = MAXDEPTH;\n ssize_t len;\n char buffer[256];\n static char refname_buffer[256];\n\n- if (flag)\n- *flag = 0;\n+ if (ref_flag)\n+ *ref_flag = 0;\n\n- if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n+ if (!(flags & RESOLVE_REF_ALLOW_BAD_NAME) &&\n+ check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n errno = EINVAL;\n return NULL;\n }\n-\n for (;;) {\n char path[PATH_MAX];\n struct stat st;\n@@ -1387,7 +1388,8 @@ const char *resolve_ref_unsafe(const char\n*refname, unsigned char *sha1, int rea\n if (lstat(path, &st) < 0) {\n if (errno == ENOENT)\n return handle_missing_loose_ref(refname, sha1,\n- reading, flag);\n+ flags & RESOLVE_REF_READING,\n+ ref_flag);\n else\n return NULL;\n }\n@@ -1407,8 +1409,8 @@ const char *resolve_ref_unsafe(const char\n*refname, unsigned char *sha1, int rea\n !check_refname_format(buffer, 0)) {\n strcpy(refname_buffer, buffer);\n refname = refname_buffer;\n- if (flag)\n- *flag |= REF_ISSYMREF;\n+ if (ref_flag)\n+ *ref_flag |= REF_ISSYMREF;\n continue;\n }\n }\n@@ -1453,21 +1455,21 @@ const char *resolve_ref_unsafe(const char\n*refname, unsigned char *sha1, int rea\n */\n if (get_sha1_hex(buffer, sha1) ||\n (buffer[40] != '\\0' && !isspace(buffer[40]))) {\n- if (flag)\n- *flag |= REF_ISBROKEN;\n+ if (ref_flag)\n+ *ref_flag |= REF_ISBROKEN;\n errno = EINVAL;\n return NULL;\n }\n return refname;\n }\n- if (flag)\n- *flag |= REF_ISSYMREF;\n+ if (ref_flag)\n+ *ref_flag |= REF_ISSYMREF;\n buf = buffer + 4;\n while (isspace(*buf))\n buf++;\n if (check_refname_format(buf, REFNAME_ALLOW_ONELEVEL)) {\n- if (flag)\n- *flag |= REF_ISBROKEN;\n+ if (ref_flag)\n+ *ref_flag |= REF_ISBROKEN;\n errno = EINVAL;\n return NULL;\n }\n@@ -1475,9 +1477,9 @@ const char *resolve_ref_unsafe(const char\n*refname, unsigned char *sha1, int rea\n }\n }\n\n-char *resolve_refdup(const char *ref, unsigned char *sha1, int\nreading, int *flag)\n+char *resolve_refdup(const char *ref, unsigned char *sha1, int flags,\nint *ref_flag)\n {\n- const char *ret = resolve_ref_unsafe(ref, sha1, reading, flag);\n+ const char *ret = resolve_ref_unsafe(ref, sha1, flags, ref_flag);\n return ret ? xstrdup(ret) : NULL;\n }\n\n@@ -1488,22 +1490,22 @@ struct ref_filter {\n void *cb_data;\n };\n\n-int read_ref_full(const char *refname, unsigned char *sha1, int\nreading, int *flags)\n+int read_ref_full(const char *refname, unsigned char *sha1, int\nflags, int *ref_flag)\n {\n- if (resolve_ref_unsafe(refname, sha1, reading, flags))\n+ if (resolve_ref_unsafe(refname, sha1, flags, ref_flag))\n return 0;\n return -1;\n }\n\n int read_ref(const char *refname, unsigned char *sha1)\n {\n- return read_ref_full(refname, sha1, 1, NULL);\n+ return read_ref_full(refname, sha1, RESOLVE_REF_READING, NULL);\n }\n\n int ref_exists(const char *refname)\n {\n unsigned char sha1[20];\n- return !!resolve_ref_unsafe(refname, sha1, 1, NULL);\n+ return !!resolve_ref_unsafe(refname, sha1, RESOLVE_REF_READING, NULL);\n }\n\n static int filter_refs(const char *refname, const unsigned char\n*sha1, int flags,\n@@ -1617,7 +1619,7 @@ int peel_ref(const char *refname, unsigned char *sha1)\n return 0;\n }\n\n- if (read_ref_full(refname, base, 1, &flag))\n+ if (read_ref_full(refname, base, RESOLVE_REF_READING, &flag))\n return -1;\n\n /*\n@@ -1783,7 +1785,7 @@ static int do_head_ref(const char *submodule,\neach_ref_fn fn, void *cb_data)\n return 0;\n }\n\n- if (!read_ref_full(\"HEAD\", sha1, 1, &flag))\n+ if (!read_ref_full(\"HEAD\", sha1, RESOLVE_REF_READING, &flag))\n return fn(\"HEAD\", sha1, flag, cb_data);\n\n return 0;\n@@ -1863,7 +1865,7 @@ int head_ref_namespaced(each_ref_fn fn, void *cb_data)\n int flag;\n\n strbuf_addf(&buf, \"%sHEAD\", get_git_namespace());\n- if (!read_ref_full(buf.buf, sha1, 1, &flag))\n+ if (!read_ref_full(buf.buf, sha1, RESOLVE_REF_READING, &flag))\n ret = fn(buf.buf, sha1, flag, cb_data);\n strbuf_release(&buf);\n\n@@ -1958,7 +1960,8 @@ int refname_match(const char *abbrev_name, const\nchar *full_name)\n static struct ref_lock *verify_lock(struct ref_lock *lock,\n const unsigned char *old_sha1, int mustexist)\n {\n- if (read_ref_full(lock->ref_name, lock->old_sha1, mustexist, NULL)) {\n+ if (read_ref_full(lock->ref_name, lock->old_sha1,\n+ mustexist ? RESOLVE_REF_READING : 0, NULL)) {\n int save_errno = errno;\n error(\"Can't verify ref %s\", lock->ref_name);\n unlock_ref(lock);\n@@ -2031,7 +2034,8 @@ int dwim_ref(const char *str, int len, unsigned\nchar *sha1, char **ref)\n\n this_result = refs_found ? sha1_from_ref : sha1;\n mksnpath(fullref, sizeof(fullref), *p, len, str);\n- r = resolve_ref_unsafe(fullref, this_result, 1, &flag);\n+ r = resolve_ref_unsafe(fullref, this_result,\n+ RESOLVE_REF_READING, &flag);\n if (r) {\n if (!refs_found++)\n *ref = xstrdup(r);\n@@ -2060,7 +2064,7 @@ int dwim_log(const char *str, int len, unsigned\nchar *sha1, char **log)\n const char *ref, *it;\n\n mksnpath(path, sizeof(path), *p, len, str);\n- ref = resolve_ref_unsafe(path, hash, 1, NULL);\n+ ref = resolve_ref_unsafe(path, hash, RESOLVE_REF_READING, NULL);\n if (!ref)\n continue;\n if (reflog_exists(path))\n@@ -2092,18 +2096,22 @@ static struct ref_lock\n*lock_ref_sha1_basic(const char *refname,\n int last_errno = 0;\n int type, lflags;\n int mustexist = (old_sha1 && !is_null_sha1(old_sha1));\n+ int resolve_flags;\n int missing = 0;\n int attempts_remaining = 3;\n-\n- if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n- errno = EINVAL;\n- return NULL;\n- }\n+ int bad_refname;\n\n lock = xcalloc(1, sizeof(struct ref_lock));\n lock->lock_fd = -1;\n\n- refname = resolve_ref_unsafe(refname, lock->old_sha1, mustexist, &type);\n+ bad_refname = check_refname_format(refname, REFNAME_ALLOW_ONELEVEL);\n+\n+ resolve_flags = RESOLVE_REF_ALLOW_BAD_NAME;\n+ if (mustexist)\n+ resolve_flags |= RESOLVE_REF_READING;\n+\n+ refname = resolve_ref_unsafe(refname, lock->old_sha1, resolve_flags,\n+ &type);\n if (!refname && errno == EISDIR) {\n /* we are trying to lock foo but we used to\n * have foo/bar which now does not exist;\n@@ -2116,7 +2124,8 @@ static struct ref_lock\n*lock_ref_sha1_basic(const char *refname,\n error(\"there are still refs under '%s'\", orig_refname);\n goto error_return;\n }\n- refname = resolve_ref_unsafe(orig_refname, lock->old_sha1, mustexist, &type);\n+ refname = resolve_ref_unsafe(orig_refname, lock->old_sha1,\n+ resolve_flags, &type);\n }\n if (type_p)\n *type_p = type;\n@@ -2180,6 +2189,8 @@ static struct ref_lock\n*lock_ref_sha1_basic(const char *refname,\n else\n unable_to_lock_index_die(ref_file, errno);\n }\n+ if (bad_refname)\n+ return lock;\n return old_sha1 ? verify_lock(lock, old_sha1, mustexist) : lock;\n\n error_return:\n@@ -2680,7 +2691,8 @@ int rename_ref(const char *oldrefname, const\nchar *newrefname, const char *logms\n if (log && S_ISLNK(loginfo.st_mode))\n return error(\"reflog for %s is a symlink\", oldrefname);\n\n- symref = resolve_ref_unsafe(oldrefname, orig_sha1, 1, &flag);\n+ symref = resolve_ref_unsafe(oldrefname, orig_sha1,\n+ RESOLVE_REF_READING, &flag);\n if (flag & REF_ISSYMREF)\n return error(\"refname %s is a symbolic ref, renaming it is not supported\",\n oldrefname);\n@@ -2704,7 +2716,7 @@ int rename_ref(const char *oldrefname, const\nchar *newrefname, const char *logms\n goto rollback;\n }\n\n- if (!read_ref_full(newrefname, sha1, 1, NULL) &&\n+ if (!read_ref_full(newrefname, sha1, RESOLVE_REF_READING, NULL) &&\n delete_ref(newrefname, sha1, REF_NODEREF)) {\n if (errno==EISDIR) {\n if (remove_empty_directories(git_path(\"%s\", newrefname))) {\n@@ -2982,7 +2994,8 @@ static int write_ref_sha1(struct ref_lock *lock,\n unsigned char head_sha1[20];\n int head_flag;\n const char *head_ref;\n- head_ref = resolve_ref_unsafe(\"HEAD\", head_sha1, 1, &head_flag);\n+ head_ref = resolve_ref_unsafe(\"HEAD\", head_sha1,\n+ RESOLVE_REF_READING, &head_flag);\n if (head_ref && (head_flag & REF_ISSYMREF) &&\n !strcmp(head_ref, lock->ref_name))\n log_ref_write(\"HEAD\", lock->old_sha1, sha1, logmsg);\n@@ -3468,6 +3481,12 @@ int ref_transaction_update(struct\nref_transaction *transaction,\n if (have_old && !old_sha1)\n die(\"BUG: have_old is true but old_sha1 is NULL\");\n\n+ if (!is_null_sha1(new_sha1) &&\n+ check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n+ strbuf_addf(err, \"Bad refname: %s\", refname);\n+ return -1;\n+ }\n+\n update = add_update(transaction, refname);\n hashcpy(update->new_sha1, new_sha1);\n update->flags = flags;\n@@ -3493,6 +3512,11 @@ int ref_transaction_create(struct\nref_transaction *transaction,\n if (!new_sha1 || is_null_sha1(new_sha1))\n die(\"BUG: create ref with null new_sha1\");\n\n+ if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n+ strbuf_addf(err, \"Bad refname: %s\", refname);\n+ return -1;\n+ }\n+\n update = add_update(transaction, refname);\n\n hashcpy(update->new_sha1, new_sha1);\ndiff --git a/remote.c b/remote.c\nindex ae04043..de84ac3 100644\n--- a/remote.c\n+++ b/remote.c\n@@ -1121,7 +1121,8 @@ static char *guess_ref(const char *name, struct ref *peer)\n struct strbuf buf = STRBUF_INIT;\n unsigned char sha1[20];\n\n- const char *r = resolve_ref_unsafe(peer->name, sha1, 1, NULL);\n+ const char *r = resolve_ref_unsafe(peer->name, sha1,\n+ RESOLVE_REF_READING, NULL);\n if (!r)\n return NULL;\n\n@@ -1182,7 +1183,8 @@ static int match_explicit(struct ref *src,\nstruct ref *dst,\n unsigned char sha1[20];\n int flag;\n\n- dst_value = resolve_ref_unsafe(matched_src->name, sha1, 1, &flag);\n+ dst_value = resolve_ref_unsafe(matched_src->name, sha1,\n+ RESOLVE_REF_READING, &flag);\n if (!dst_value ||\n ((flag & REF_ISSYMREF) &&\n !starts_with(dst_value, \"refs/heads/\")))\ndiff --git a/sequencer.c b/sequencer.c\nindex 284059b..e1419bf 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -365,7 +365,7 @@ static int is_index_unchanged(void)\n unsigned char head_sha1[20];\n struct commit *head_commit;\n\n- if (!resolve_ref_unsafe(\"HEAD\", head_sha1, 1, NULL))\n+ if (!resolve_ref_unsafe(\"HEAD\", head_sha1, RESOLVE_REF_READING, NULL))\n return error(_(\"Could not resolve HEAD commit\\n\"));\n\n head_commit = lookup_commit(head_sha1);\ndiff --git a/transport-helper.c b/transport-helper.c\nindex 84c616f..270ae28 100644\n--- a/transport-helper.c\n+++ b/transport-helper.c\n@@ -889,7 +889,7 @@ static int push_refs_with_export(struct transport\n*transport,\n int flag;\n\n /* Follow symbolic refs (mainly for HEAD). */\n- name = resolve_ref_unsafe(ref->peer_ref->name, sha1, 1, &flag);\n+ name = resolve_ref_unsafe(ref->peer_ref->name, sha1,\nRESOLVE_REF_READING, &flag);\n if (!name || !(flag & REF_ISSYMREF))\n name = ref->peer_ref->name;\n\ndiff --git a/transport.c b/transport.c\nindex 325f03e..f40e950 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -168,7 +168,8 @@ static void set_upstreams(struct transport\n*transport, struct ref *refs,\n /* Follow symbolic refs (mainly for HEAD). */\n localname = ref->peer_ref->name;\n remotename = ref->name;\n- tmp = resolve_ref_unsafe(localname, sha, 1, &flag);\n+ tmp = resolve_ref_unsafe(localname, sha,\n+ RESOLVE_REF_READING, &flag);\n if (tmp && flag & REF_ISSYMREF &&\n starts_with(tmp, \"refs/heads/\"))\n localname = tmp;\n@@ -753,7 +754,7 @@ void transport_print_push_status(const char *dest,\nstruct ref *refs,\n unsigned char head_sha1[20];\n char *head;\n\n- head = resolve_refdup(\"HEAD\", head_sha1, 1, NULL);\n+ head = resolve_refdup(\"HEAD\", head_sha1, RESOLVE_REF_READING, NULL);\n\n if (verbose) {\n for (ref = refs; ref; ref = ref->next)\n"},{"id":"246176","messageId":"CAL=YDW=kJNb0E3ARMeKpcJiybM=pbpBjO2-494QKoY+oV-5q6Q@mail.gmail.com","threadId":"36972","inReplyTo":"53BBFDFE.5060902@alum.mit.edu","subject":"Re: [PATCH v20 40/48] refs.c: add an err argument to delete_ref_loose","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-07-16T18:53:33Z","receivedAt":"2014-07-16T18:53:33Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"On Tue, Jul 8, 2014 at 7:19 AM, Michael Haggerty wrote:\n> On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:\n>> Add an err argument to delete_loose_ref so that we can pass a descriptive\n>> error string back to the caller. Pass the err argument from transaction\n>> commit to this function so that transaction users will have a nice error\n>> string if the transaction failed due to delete_loose_ref.\n>>\n>> Add a new function unlink_or_err that we can call from delete_ref_loose. This\n>> function is similar to unlink_or_warn except that we can pass it an err\n>> argument. If err is non-NULL the function will populate err instead of\n>> printing a warning().\n>>\n>> Simplify warn_if_unremovable.\n>\n> The change to warn_if_unremovable() is orthogonal to the rest of the\n> commit and should be a separate commit.\n\nDone.\nI split it into three patches.\nOne patch for the warn_if_removable change\nanother patch to add unlink_or_msg to wrapper.c\nand a final patch to change delete_loose_ref.\n\n>\n>> Signed-off-by: Ronnie Sahlberg \n>> ---\n>> refs.c | 33 ++++++++++++++++++++++++++++-----\n>> wrapper.c | 14 ++++++--------\n>> 2 files changed, 34 insertions(+), 13 deletions(-)\n>>\n>> diff --git a/refs.c b/refs.c\n>> index 92a06d4..c7d1f3e 100644\n>> --- a/refs.c\n>> +++ b/refs.c\n>> @@ -2544,16 +2544,38 @@ int repack_without_refs(const char **refnames, int n, struct strbuf *err)\n>> return ret;\n>> }\n>>\n>> -static int delete_ref_loose(struct ref_lock *lock, int flag)\n>> +static int add_err_if_unremovable(const char *op, const char *file,\n>> + struct strbuf *e, int rc)\n>\n> This function is only used once. Given also that its purpose is not\n> that obvious from its signature, it seems to me that the code would be\n> easier to read if it were inlined.\n>\n>> +{\n>> + int err = errno;\n>> + if (rc < 0 && errno != ENOENT) {\n>> + strbuf_addf(e, \"unable to %s %s: %s\",\n>> + op, file, strerror(errno));\n>> + errno = err;\n>> + return -1;\n>> + }\n>> + return 0;\n>> +}\n>> +\n>> +static int unlink_or_err(const char *file, struct strbuf *err)\n>\n> The name of this function is misleading; it sounds like it will try to\n> unlink the file and if not possible call error(). Maybe a name like\n> \"unlink_or_report\" would be less prejudicial.\n>\n> It might also make sense to move this function to wrapper.c and\n> implement unlink_or_warn() in terms of it rather than vice versa.\n>\n>> +{\n>> + if (err)\n>> + return add_err_if_unremovable(\"unlink\", file, err,\n>> + unlink(file));\n>> + else\n>> + return unlink_or_warn(file);\n>> +}\n>> +\n>> +static int delete_ref_loose(struct ref_lock *lock, int flag, struct strbuf *err)\n>> {\n>> if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {\n>> /* loose */\n>> - int err, i = strlen(lock->lk->filename) - 5; /* .lock */\n>> + int res, i = strlen(lock->lk->filename) - 5; /* .lock */\n>>\n>> lock->lk->filename[i] = 0;\n>> - err = unlink_or_warn(lock->lk->filename);\n>> + res = unlink_or_err(lock->lk->filename, err);\n>> lock->lk->filename[i] = '.';\n>> - if (err && errno != ENOENT)\n>> + if (res)\n>> return 1;\n>> }\n>> return 0;\n>> @@ -3603,7 +3625,8 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n>> struct ref_update *update = updates[i];\n>>\n>> if (update->lock) {\n>> - ret |= delete_ref_loose(update->lock, update->type);\n>> + ret |= delete_ref_loose(update->lock, update->type,\n>> + err);\n>> if (!(update->flags & REF_ISPRUNING))\n>> delnames[delnum++] = update->lock->ref_name;\n>> }\n>> diff --git a/wrapper.c b/wrapper.c\n>> index bc1bfb8..740e193 100644\n>> --- a/wrapper.c\n>> +++ b/wrapper.c\n>> @@ -429,14 +429,12 @@ int xmkstemp_mode(char *template, int mode)\n>>\n>> static int warn_if_unremovable(const char *op, const char *file, int rc)\n>> {\n>> - if (rc < 0) {\n>> - int err = errno;\n>> - if (ENOENT != err) {\n>> - warning(\"unable to %s %s: %s\",\n>> - op, file, strerror(errno));\n>> - errno = err;\n>> - }\n>> - }\n>> + int err;\n>> + if (rc >= 0 || errno == ENOENT)\n>> + return rc;\n>> + err = errno;\n>> + warning(\"unable to %s %s: %s\", op, file, strerror(errno));\n>> + errno = err;\n>> return rc;\n>> }\n>>\n>>\n>\n> Michael\n>\n> --\n> Michael Haggerty\n> mhagger@alum.mit.edu\n>\n"},{"id":"248018","messageId":"53F4B642.7020002@alum.mit.edu","threadId":"36972","inReplyTo":"CAL=YDWmc2gkw=8YavWHyLUAD4du7saPrKzPKT+dsCfdZJz1EiA@mail.gmail.com","subject":"Re: [PATCH v20 43/48] refs.c: move the check for valid refname to lock_ref_sha1_basic","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2014-08-20T14:52:50Z","receivedAt":"2014-08-20T14:52:50Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 07/15/2014 10:58 PM, Ronnie Sahlberg wrote:\n> On Tue, Jul 15, 2014 at 12:34 PM, Ronnie Sahlberg wrote:\n>> On Tue, Jul 15, 2014 at 11:04 AM, Jonathan Nieder wrote:\n>>> Michael Haggerty wrote:\n>>>\n>>>> So...I like the idea of enforcing refname checks at the lowest level\n>>>> possible, but I think that the change you propose is too abrupt. I\n>>>> think it needs either more careful analysis showing that it won't hurt\n>>>> anybody, or some kind of tooling or non-strict mode that people can use\n>>>> to fix their repositories.\n>>>\n>>> The recovery code has been broken for a while, so I don't see harm\n>>> in this change.\n>>>\n>>> How to take care of the recovery use case is another question. FWIW I\n>>> also would prefer if \"git update-ref -d\" or \"git branch -D\" could be\n>>> used to delete corrupt refs instead of having to use fsck (since a\n>>> fsck run can take a while), but that's a question for a later series.\n>>>\n>>> In an ideal world, the low-level functions would allow *reading* and\n>>> *deleting* poorly named refs (even without any special flag) but not\n>>> creating them. Is that doable?\n>>\n>> That should be doable. Ill add these repairs as 3-4 new patches at the\n>> end of the current patch series.\n>>\n>>> The main complication I can see is\n>>> iteration: would iteration skip poorly named refs and warn, or would\n>>> something more complicated be needed?\n\nI think we can get away with not including broken refnames when\niterating. After all, the main goal of tolerating them is to let them\nbe deleted, right? Then as long as iteration is not needed to implement\nthe command \"git update-ref -d\", it seems to me that it is OK if\niterating over a reference with a broken name causes a die().\n\n>> Right now, \"git branch\" will error and abort right away when it\n>> finds the first bad ref. I haven't checked the exact spot it does this\n>> yet but I suspect it is when building the ref-cache.\n>> I will solve the cases for create and delete for now.\n>>\n>>\n>> What/how to handle iterables will require more thought.\n>> Right now, since these refs will be filtered out and never end up in\n>> ref-cache, either from loose refs or from packed refs\n>> it does mean that anyone that uses an iterator is guaranteed to only\n>> get refs with valid names passed back to them.\n>> We would need to audit all code that uses iterators and make sure it\n>> can handle the case where the callback is suddenly\n>> invoked with a bad refname.\n>>\n>>>\n>>> Thanks,\n>>> Jonathan\n> \n> The following seems to do the trick to allow deleting a bad ref. We\n> would need something for the iterator too.\n> Since this touches the same areas that my ~100 other ref transaction\n> patches that are queued up do, I\n> would like to wait applying this patch until once the next few series\n> are finished and merged.\n> (to avoid having to do a lot of rebases and fix legio of merge\n> conflicts that this would introduce in my branches).\n> \n> \n> Treat this as an approximation on what the fix to repair git branch -D\n> will look like once the time comes.\n\nI'm a little worried that abandoning *all* refname checks could open us\nup to somehow trying to delete a \"reference\" with a name like\n\"../../../../etc/passwd\". Either such names have to be prohibited\nsomehow, or we have to be very sure that they can only come from trusted\nsources.\n\nMichael\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\n"},{"id":"248019","messageId":"CAL=YDW=zmqwjLzjWYLGCjwb_aQTofTJ-CrmPgoq3KKJjfHR3XQ@mail.gmail.com","threadId":"36972","inReplyTo":"53F4B642.7020002@alum.mit.edu","subject":"Re: [PATCH v20 43/48] refs.c: move the check for valid refname to lock_ref_sha1_basic","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-08-20T16:28:28Z","receivedAt":"2014-08-20T16:28:28Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"On Wed, Aug 20, 2014 at 7:52 AM, Michael Haggerty wrote:\n> On 07/15/2014 10:58 PM, Ronnie Sahlberg wrote:\n>> On Tue, Jul 15, 2014 at 12:34 PM, Ronnie Sahlberg wrote:\n>>> On Tue, Jul 15, 2014 at 11:04 AM, Jonathan Nieder wrote:\n>>>> Michael Haggerty wrote:\n>>>>\n>>>>> So...I like the idea of enforcing refname checks at the lowest level\n>>>>> possible, but I think that the change you propose is too abrupt. I\n>>>>> think it needs either more careful analysis showing that it won't hurt\n>>>>> anybody, or some kind of tooling or non-strict mode that people can use\n>>>>> to fix their repositories.\n>>>>\n>>>> The recovery code has been broken for a while, so I don't see harm\n>>>> in this change.\n>>>>\n>>>> How to take care of the recovery use case is another question. FWIW I\n>>>> also would prefer if \"git update-ref -d\" or \"git branch -D\" could be\n>>>> used to delete corrupt refs instead of having to use fsck (since a\n>>>> fsck run can take a while), but that's a question for a later series.\n>>>>\n>>>> In an ideal world, the low-level functions would allow *reading* and\n>>>> *deleting* poorly named refs (even without any special flag) but not\n>>>> creating them. Is that doable?\n>>>\n>>> That should be doable. Ill add these repairs as 3-4 new patches at the\n>>> end of the current patch series.\n>>>\n>>>> The main complication I can see is\n>>>> iteration: would iteration skip poorly named refs and warn, or would\n>>>> something more complicated be needed?\n>\n> I think we can get away with not including broken refnames when\n> iterating. After all, the main goal of tolerating them is to let them\n> be deleted, right? Then as long as iteration is not needed to implement\n> the command \"git update-ref -d\", it seems to me that it is OK if\n> iterating over a reference with a broken name causes a die().\n>\n>>> Right now, \"git branch\" will error and abort right away when it\n>>> finds the first bad ref. I haven't checked the exact spot it does this\n>>> yet but I suspect it is when building the ref-cache.\n>>> I will solve the cases for create and delete for now.\n>>>\n>>>\n>>> What/how to handle iterables will require more thought.\n>>> Right now, since these refs will be filtered out and never end up in\n>>> ref-cache, either from loose refs or from packed refs\n>>> it does mean that anyone that uses an iterator is guaranteed to only\n>>> get refs with valid names passed back to them.\n>>> We would need to audit all code that uses iterators and make sure it\n>>> can handle the case where the callback is suddenly\n>>> invoked with a bad refname.\n>>>\n>>>>\n>>>> Thanks,\n>>>> Jonathan\n>>\n>> The following seems to do the trick to allow deleting a bad ref. We\n>> would need something for the iterator too.\n>> Since this touches the same areas that my ~100 other ref transaction\n>> patches that are queued up do, I\n>> would like to wait applying this patch until once the next few series\n>> are finished and merged.\n>> (to avoid having to do a lot of rebases and fix legio of merge\n>> conflicts that this would introduce in my branches).\n>>\n>>\n>> Treat this as an approximation on what the fix to repair git branch -D\n>> will look like once the time comes.\n>\n> I'm a little worried that abandoning *all* refname checks could open us\n> up to somehow trying to delete a \"reference\" with a name like\n> \"../../../../etc/passwd\". Either such names have to be prohibited\n> somehow, or we have to be very sure that they can only come from trusted\n> sources.\n>\n\nI only set this flag from builtin/branch.c so it should only be used\nwhen a user runs 'git branch -D' from the command line.\nAll other places where we delete branches we should still be checking\nthe rename for badness.\n\nThat said, unless the \"rules for good refname\" changes in the future,\nwhich is unlikely, is should be exceptionally rare that a user ends up\nwith a bad refname in the first place.\nPerhaps my example I gave was bad since if you manually create bad\nrefs using echo > .git/refs/heads/... then you should probably know\nhow to fix it too and thus maybe we do not need this patch in the\nfirst place.\n\nDo you want me to delete this patch and resend this part of the series\n? Or is the 'only works for branch -D from the commandline' sufficient\n?\nI have no strong feelings either way so I will just follow what you decide.\n\n\nregards\nronnie sahlberg\n"},{"id":"248022","messageId":"20140820174914.GE20185@google.com","threadId":"36972","inReplyTo":"CAL=YDW=zmqwjLzjWYLGCjwb_aQTofTJ-CrmPgoq3KKJjfHR3XQ@mail.gmail.com","subject":"Re: [PATCH v20 43/48] refs.c: move the check for valid refname to lock_ref_sha1_basic","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2014-08-20T17:49:14Z","receivedAt":"2014-08-20T17:49:14Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nRonnie Sahlberg wrote:\n> On Wed, Aug 20, 2014 at 7:52 AM, Michael Haggerty wrote:\n\n>> I'm a little worried that abandoning *all* refname checks could open us\n>> up to somehow trying to delete a \"reference\" with a name like\n>> \"../../../../etc/passwd\". Either such names have to be prohibited\n>> somehow, or we have to be very sure that they can only come from trusted\n>> sources.\n>\n> I only set this flag from builtin/branch.c so it should only be used\n> when a user runs 'git branch -D' from the command line.\n> All other places where we delete branches we should still be checking\n> the rename for badness.\n\nRight, this should be safe for 'git branch -D' and 'git update-ref -d'.\n\nBut if we wanted to open it up in the future for 'git push --delete',\ntoo, then it would be a way to break out of the repository on hosts where\npeople use git-shell instead of relying on filesystem permissions. And\nthat wouldn't be good.\n\nI think elsewhere git has some checks for \"does this pathname fall in\nthis directory\". Could that be reused here, too, to make sure the\nresolved path is under the resolved $GIT_DIR/refs directory?\n\nAlternatively, when a ref being deleted doesn't meet the\n'check-ref-format' checks, would it make sense to check that it is one\nof the refs you can get by iteration?\n\nJonathan\n"},{"id":"248024","messageId":"CAL=YDW=e3Z5TqDcsY3J4-CDw+ZgQPcDTAcc1pyEeTkzAeVFcog@mail.gmail.com","threadId":"36972","inReplyTo":"20140820174914.GE20185@google.com","subject":"Re: [PATCH v20 43/48] refs.c: move the check for valid refname to lock_ref_sha1_basic","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-08-20T17:55:45Z","receivedAt":"2014-08-20T17:55:45Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"On Wed, Aug 20, 2014 at 10:49 AM, Jonathan Nieder wrote:\n> Hi,\n>\n> Ronnie Sahlberg wrote:\n>> On Wed, Aug 20, 2014 at 7:52 AM, Michael Haggerty wrote:\n>\n>>> I'm a little worried that abandoning *all* refname checks could open us\n>>> up to somehow trying to delete a \"reference\" with a name like\n>>> \"../../../../etc/passwd\". Either such names have to be prohibited\n>>> somehow, or we have to be very sure that they can only come from trusted\n>>> sources.\n>>\n>> I only set this flag from builtin/branch.c so it should only be used\n>> when a user runs 'git branch -D' from the command line.\n>> All other places where we delete branches we should still be checking\n>> the rename for badness.\n>\n> Right, this should be safe for 'git branch -D' and 'git update-ref -d'.\n>\n> But if we wanted to open it up in the future for 'git push --delete',\n> too, then it would be a way to break out of the repository on hosts where\n> people use git-shell instead of relying on filesystem permissions. And\n> that wouldn't be good.\n>\n> I think elsewhere git has some checks for \"does this pathname fall in\n> this directory\". Could that be reused here, too, to make sure the\n> resolved path is under the resolved $GIT_DIR/refs directory?\n>\n> Alternatively, when a ref being deleted doesn't meet the\n> 'check-ref-format' checks, would it make sense to check that it is one\n> of the refs you can get by iteration?\n\nGood idea! I will add such a check using the iterator.\nThat means that \"we can git branch -D anything that shows up in the\niterator regardless if the ref is badly named or unresolvable\" which\nsounds like fairly sane semantics.\n\nThanks!\n\nRonnie\n"},{"id":"248026","messageId":"53F4EA43.4020802@alum.mit.edu","threadId":"36972","inReplyTo":"CAL=YDW=zmqwjLzjWYLGCjwb_aQTofTJ-CrmPgoq3KKJjfHR3XQ@mail.gmail.com","subject":"Re: [PATCH v20 43/48] refs.c: move the check for valid refname to lock_ref_sha1_basic","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2014-08-20T18:34:43Z","receivedAt":"2014-08-20T18:34:43Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 08/20/2014 06:28 PM, Ronnie Sahlberg wrote:\n> On Wed, Aug 20, 2014 at 7:52 AM, Michael Haggerty wrote:\n>> I'm a little worried that abandoning *all* refname checks could open us\n>> up to somehow trying to delete a \"reference\" with a name like\n>> \"../../../../etc/passwd\". Either such names have to be prohibited\n>> somehow, or we have to be very sure that they can only come from trusted\n>> sources.\n> \n> I only set this flag from builtin/branch.c so it should only be used\n> when a user runs 'git branch -D' from the command line.\n> All other places where we delete branches we should still be checking\n> the rename for badness.\n> \n> That said, unless the \"rules for good refname\" changes in the future,\n> which is unlikely, is should be exceptionally rare that a user ends up\n> with a bad refname in the first place.\n> Perhaps my example I gave was bad since if you manually create bad\n> refs using echo > .git/refs/heads/... then you should probably know\n> how to fix it too and thus maybe we do not need this patch in the\n> first place.\n> \n> Do you want me to delete this patch and resend this part of the series\n> ? Or is the 'only works for branch -D from the commandline' sufficient\n> ?\n> I have no strong feelings either way so I will just follow what you decide.\n\nI think that if you run the refname through normalize_path_copy_len()\nand that function returns (1) without an error, (2) without modifying\nits argument, and (3) the result does not begin with a\nhas_dos_drive_prefix() or is_dir_sep(), then we should be safe against\ndirectory traversal attacks. I suggest doing this kind of check even if\nnot doing the full check_refname_format() check.\n\nMichael\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\n"},{"id":"248029","messageId":"xmqqk363t060.fsf@gitster.dls.corp.google.com","threadId":"36972","inReplyTo":"53F4B642.7020002@alum.mit.edu","subject":"Re: [PATCH v20 43/48] refs.c: move the check for valid refname to lock_ref_sha1_basic","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-08-20T19:45:43Z","receivedAt":"2014-08-20T19:45:43Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Michael Haggerty writes:\n\n> I think we can get away with not including broken refnames when\n> iterating. After all, the main goal of tolerating them is to let them\n> be deleted, right?\n\nOr read from a ref whose name has retroactively made invalid, in\norder to create a similar but validly named ref to serve as its\nreplacement? So at least we need to give the users some way of\nreading from them, in addition to deleting them, no?\n"},{"id":"248030","messageId":"53F500E9.6060900@alum.mit.edu","threadId":"36972","inReplyTo":"xmqqk363t060.fsf@gitster.dls.corp.google.com","subject":"Re: [PATCH v20 43/48] refs.c: move the check for valid refname to lock_ref_sha1_basic","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2014-08-20T20:11:21Z","receivedAt":"2014-08-20T20:11:21Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 08/20/2014 09:45 PM, Junio C Hamano wrote:\n> Michael Haggerty writes:\n> \n>> I think we can get away with not including broken refnames when\n>> iterating. After all, the main goal of tolerating them is to let them\n>> be deleted, right?\n> \n> Or read from a ref whose name has retroactively made invalid, in\n> order to create a similar but validly named ref to serve as its\n> replacement? So at least we need to give the users some way of\n> reading from them, in addition to deleting them, no?\n\nThe die() error message would presumably include the name of the invalid\nreference.\n\nIf we change the rules for reference names and a bunch of names become\ninvalid, then it would admittedly be cumbersome to run git N times to\nfind the N invalid names. But if we change the rules, then *at that\ntime* we could always build in iteration over broken reference names.\n\nIt's not that I have something against building it iteration over broken\nreference names now, as long as it is clearly segregated from \"normal\"\niteration to prevent illegal names from getting loose in the code.\n\nMichael\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\n"},{"id":"248034","messageId":"xmqqy4uisvlk.fsf@gitster.dls.corp.google.com","threadId":"36972","inReplyTo":"53F500E9.6060900@alum.mit.edu","subject":"Re: [PATCH v20 43/48] refs.c: move the check for valid refname to lock_ref_sha1_basic","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-08-20T21:24:23Z","receivedAt":"2014-08-20T21:24:23Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Michael Haggerty writes:\n\n> On 08/20/2014 09:45 PM, Junio C Hamano wrote:\n>> Michael Haggerty writes:\n>> \n>>> I think we can get away with not including broken refnames when\n>>> iterating. After all, the main goal of tolerating them is to let them\n>>> be deleted, right?\n>> \n>> Or read from a ref whose name has retroactively made invalid, in\n>> order to create a similar but validly named ref to serve as its\n>> replacement? So at least we need to give the users some way of\n>> reading from them, in addition to deleting them, no?\n>\n> The die() error message would presumably include the name of the invalid\n> reference.\n>\n> If we change the rules for reference names and a bunch of names become\n> invalid, then it would admittedly be cumbersome to run git N times to\n> find the N invalid names. But if we change the rules, then *at that\n> time* we could always build in iteration over broken reference names.\n>\n> It's not that I have something against building it iteration over broken\n> reference names now, as long as it is clearly segregated from \"normal\"\n> iteration to prevent illegal names from getting loose in the code.\n\nOh, I don't think it is that important for iterator to show broken\nones. If refs/heads/foo/$brokenname exists but is skipped from any\nand all iterations, nobody will get hurt until the end user wonders\nwhere foo/$brokenname went, at which time rev-parse can be used to\ngrab the value from it before update-ref -d can be used to remove it.\n\nIf refs/heads/foo/$brokenname, which is skipped from iterations,\nprevents a valid ref refs/heads/foo from getting created, that would\ngive a bit of surprise to the user who long forgot foo/$brokenname\nexisted, but the recovery procedure is exactly the same as the case\nwhere he has a branch foo/$koshername and wants to create foo; first\n'branch -D' the D/F-conflicting one and then create foo.\n\nSo the primary things I care about are when the user has a string\nthat is the name of an existing misnamed ref, the value of the ref\ncan be obtained, and the ref can be removed.\n\nThanks.\n"},{"id":"248035","messageId":"CAL=YDWkfMMqYdOVWfBJkMncPnm8GwMdd1q4ipD_Y_r-nBet+2w@mail.gmail.com","threadId":"36972","inReplyTo":"53F500E9.6060900@alum.mit.edu","subject":"Re: [PATCH v20 43/48] refs.c: move the check for valid refname to lock_ref_sha1_basic","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-08-20T21:47:08Z","receivedAt":"2014-08-20T21:47:08Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"On Wed, Aug 20, 2014 at 1:11 PM, Michael Haggerty wrote:\n> On 08/20/2014 09:45 PM, Junio C Hamano wrote:\n>> Michael Haggerty writes:\n>>\n>>> I think we can get away with not including broken refnames when\n>>> iterating. After all, the main goal of tolerating them is to let them\n>>> be deleted, right?\n>>\n>> Or read from a ref whose name has retroactively made invalid, in\n>> order to create a similar but validly named ref to serve as its\n>> replacement? So at least we need to give the users some way of\n>> reading from them, in addition to deleting them, no?\n>\n> The die() error message would presumably include the name of the invalid\n> reference.\n>\n> If we change the rules for reference names and a bunch of names become\n> invalid, then it would admittedly be cumbersome to run git N times to\n> find the N invalid names. But if we change the rules, then *at that\n> time* we could always build in iteration over broken reference names.\n>\n> It's not that I have something against building it iteration over broken\n> reference names now, as long as it is clearly segregated from \"normal\"\n> iteration to prevent illegal names from getting loose in the code.\n\nWe already have iterators that show also bad refs.\nFor example, git branch uses\nDO_FOR_EACH_INCLUDE_BROKEN\nwhich is a flag to include also broken refs that can not be resolved\nwhich allows them to be listed :\n\n\n\n$ echo \"this is not a valid sha1\" >.git/refs/heads/broken\n$ git branch\n broken\n foo\n* master\n$ git branch -D broken\nerror: branch 'broken' not found.\n\n(allowing -D to remove it is in a different patch further down my series)\n\n\nSince we already display broken/unresolvable refs, I think the most\nconsistent path is to also allow showing the refs broken/illegal-names\ntoo in the list. (when DO_FOR_EACH_INCLUDE_BROKEN is specified)\nOf course, an end user could fix this by deleting the file but since\nit is easy to add the special casing to 'git branch -D' to handle this\ncase I think this would be more userfriendly since then the user can\nuse git branch -D regardless of the reason why the ref is broken.\n\n\nRonnie\n"},{"id":"248079","messageId":"CAL=YDWmHigoTYR4ENxozVwczXqY16DWi3_b7VwacX6-iPCeyhg@mail.gmail.com","threadId":"36972","inReplyTo":"53F4EA43.4020802@alum.mit.edu","subject":"Re: [PATCH v20 43/48] refs.c: move the check for valid refname to lock_ref_sha1_basic","fromName":"Ronnie Sahlberg","fromEmail":"sahlberg@google.com","sentAt":"2014-08-21T19:42:45Z","receivedAt":"2014-08-21T19:42:45Z","isPatch":true,"sender":{"key":"sahlberg@google.com","avatar":"https://avatars.githubusercontent.com/u/7320636?v=4"},"body":"On Wed, Aug 20, 2014 at 11:34 AM, Michael Haggerty wrote:\n> On 08/20/2014 06:28 PM, Ronnie Sahlberg wrote:\n>> On Wed, Aug 20, 2014 at 7:52 AM, Michael Haggerty wrote:\n>>> I'm a little worried that abandoning *all* refname checks could open us\n>>> up to somehow trying to delete a \"reference\" with a name like\n>>> \"../../../../etc/passwd\". Either such names have to be prohibited\n>>> somehow, or we have to be very sure that they can only come from trusted\n>>> sources.\n>>\n>> I only set this flag from builtin/branch.c so it should only be used\n>> when a user runs 'git branch -D' from the command line.\n>> All other places where we delete branches we should still be checking\n>> the rename for badness.\n>>\n>> That said, unless the \"rules for good refname\" changes in the future,\n>> which is unlikely, is should be exceptionally rare that a user ends up\n>> with a bad refname in the first place.\n>> Perhaps my example I gave was bad since if you manually create bad\n>> refs using echo > .git/refs/heads/... then you should probably know\n>> how to fix it too and thus maybe we do not need this patch in the\n>> first place.\n>>\n>> Do you want me to delete this patch and resend this part of the series\n>> ? Or is the 'only works for branch -D from the commandline' sufficient\n>> ?\n>> I have no strong feelings either way so I will just follow what you decide.\n>\n> I think that if you run the refname through normalize_path_copy_len()\n> and that function returns (1) without an error, (2) without modifying\n> its argument, and (3) the result does not begin with a\n> has_dos_drive_prefix() or is_dir_sep(), then we should be safe against\n> directory traversal attacks. I suggest doing this kind of check even if\n> not doing the full check_refname_format() check.\n\nGood idea.\nLet me add this.\n\n\n>\n> Michael\n>\n> --\n> Michael Haggerty\n> mhagger@alum.mit.edu\n>\n"},{"id":"248109","messageId":"53F73A61.1010606@alum.mit.edu","threadId":"36972","inReplyTo":"CAL=YDWkfMMqYdOVWfBJkMncPnm8GwMdd1q4ipD_Y_r-nBet+2w@mail.gmail.com","subject":"Re: [PATCH v20 43/48] refs.c: move the check for valid refname to lock_ref_sha1_basic","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2014-08-22T12:41:05Z","receivedAt":"2014-08-22T12:41:05Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 08/20/2014 11:47 PM, Ronnie Sahlberg wrote:\n> [...]\n> Since we already display broken/unresolvable refs, I think the most\n> consistent path is to also allow showing the refs broken/illegal-names\n> too in the list. (when DO_FOR_EACH_INCLUDE_BROKEN is specified)\n> Of course, an end user could fix this by deleting the file but since\n> it is easy to add the special casing to 'git branch -D' to handle this\n> case I think this would be more userfriendly since then the user can\n> use git branch -D regardless of the reason why the ref is broken.\n\nMy concern with this idea is that some code relies on at least some of\nthe reference name constraints for its proper functioning; for example,\n\n* The ref caching code would likely be confused by ill-formed refnames\nlike \"refs/heads//foo\" or \"/refs/heads/foo\" or \"refs/heads/foo/\". (I\nunderstand that such references cannot exist as loose refs, but they\ncould be represented in the packed-refs file.)\n\n* Any code that might try to read or write a loose reference would\nlikely be confused by \"refs/heads//foo\" or \"refs/heads/./foo\" or\n\"refs/heads/../foo\" or \"/refs/heads/foo\" or \"refs/heads/foo/\". On\nWindows there might also be problems with \"refs/heads\\foo\" or\n\"d:refs/heads/foo\" or \"prn:refs/heads/foo\" or \"//refs/heads/foo\".\n\n* The locking code could easily be confused by a reference named\n\"refs/heads/foo.lock\".\n\nSo to the extent that we loosen the checks on refnames when they are\nread, we would have to re-vet any code that touches them to make sure\nthat it doesn't break in a horrible (and possibly security-compromising)\nway. This is why I would prefer to quarantine broken reference names in\nthe smallest possible part of the code.\n\nI *think* that the biggest problems would be related to reference names\nthat do not map straightforwardly to relative filenames, so an\nalternative would be to do some minimal checks in any case, but make it\npossible to turn off the stricter checks (those that mostly exist to\nmake reference expression parsing possible) when necessary.\n\nMichael\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\n"}]}