git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCHv4 4/6] receive-pack.c: use a single ref_transaction for atomic pushes

From
Stefan Beller <sbeller@google.com>
Date
Dec 18, 2014, 17:45 UTC
Message-ID
<1418924706-9843-1-git-send-email-sbeller@google.com>
In-Reply-To
<xmqqppbg7uxk.fsf@gitster.dls.corp.google.com>
From: Ronnie Sahlberg <sahlberg@google.com>

Update receive-pack to use an atomic transaction iff the client negotiated that it wanted atomic-push. This leaves the default behavior to be the old non-atomic one ref at a time update. This is to cause as little disruption as possible to existing clients. It is unknown if there are client scripts that depend on the old non-atomic behavior so we make it opt-in for now.

If it turns out over time that there are no client scripts that depend on the old behavior we can change git to default to use atomic pushes and instead offer an opt-out argument for people that do not want atomic pushes.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
---
v4 of the series only resends this patch as the others did not get any 
feedback to improve.
        
Notes:
    Changes v1 -> v2:
    	* update(...) assumes to be always in a transaction
    	* Caring about when to begin/commit transactions is put
    	  into execute_commands
    v2->v3:
    	* meditated about the error flow. Now we always construct a local
    	  strbuf err if required. Then the flow is easier to follow and
    	  destruction of it is performed nearby.
    	* early return in execute_commands if transaction_begin fails.
    
    v3->v4:
    	* revamp logic again. This should keep the non atomic behavior
    	  as is (in case of error say so, in non error case just free the
    	  transaction). In the atomic case we either do nothing (when no error),
    	  or abort with the goto.
    
    		if (!cmd->error_string) {
    			if (!use_atomic
    			    && ref_transaction_commit(transaction, &err)) {
    				ref_transaction_free(transaction);
    				rp_error("%s", err.buf);
    				strbuf_release(&err);
    				cmd->error_string = "failed to update ref";
    			}
    		} else if (use_atomic) {
    			goto atomic_failure;
    		} else {
    			ref_transaction_free(transaction);
    		}
    
    	 * Having the goto directly there when checking for cmd->error_string,
    	   we don't need to do it again, so the paragraph explaining the error
    	   checking is gone as well. (Previous patch had the following, this is
    	   put at the end of the function, where the goto jumps to and the comment
    	   has been dropped.
    +		/*
    +		 * update(...) may abort early (i.e. because the hook refused to
    +		 * update that ref) which then doesn't even record a transaction
    +		 * regarding that ref. Make sure all commands are without error
    +		 * and then commit atomically.
    +		 */
    +		for (cmd = commands; cmd; cmd = cmd->next)
    +			if (cmd->error_string)
    +				break;
 builtin/receive-pack.c | 83 +++++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 69 insertions(+), 14 deletions(-)
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index e76e5d5..4952d91 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -67,6 +67,7 @@ static const char *NONCE_SLOP = "SLOP";
 static const char *nonce_status;
 static long nonce_stamp_slop;
 static unsigned long nonce_stamp_slop_limit;
+static struct ref_transaction *transaction;
 
 static enum deny_action parse_deny_action(const char *var, const char *value)
 {
@@ -823,6 +824,7 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 	}
 
 	if (is_null_sha1(new_sha1)) {
+		struct strbuf err = STRBUF_INIT;
 		if (!parse_object(old_sha1)) {
 			old_sha1 = NULL;
 			if (ref_exists(name)) {
@@ -832,35 +834,36 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 				cmd->did_not_exist = 1;
 			}
 		}
-		if (delete_ref(namespaced_name, old_sha1, 0)) {
-			rp_error("failed to delete %s", name);
+		if (ref_transaction_delete(transaction,
+					   namespaced_name,
+					   old_sha1,
+					   0, old_sha1 != NULL,
+					   "push", &err)) {
+			rp_error("%s", err.buf);
+			strbuf_release(&err);
 			return "failed to delete";
 		}
+		strbuf_release(&err);
 		return NULL; /* good */
 	}
 	else {
 		struct strbuf err = STRBUF_INIT;
-		struct ref_transaction *transaction;
-
 		if (shallow_update && si->shallow_ref[cmd->index] &&
 		    update_shallow_ref(cmd, si))
 			return "shallow error";
 
-		transaction = ref_transaction_begin(&err);
-		if (!transaction ||
-		    ref_transaction_update(transaction, namespaced_name,
-					   new_sha1, old_sha1, 0, 1, "push",
-					   &err) ||
-		    ref_transaction_commit(transaction, &err)) {
-			ref_transaction_free(transaction);
-
+		if (ref_transaction_update(transaction,
+					   namespaced_name,
+					   new_sha1, old_sha1,
+					   0, 1, "push",
+					   &err)) {
 			rp_error("%s", err.buf);
 			strbuf_release(&err);
+
 			return "failed to update ref";
 		}
-
-		ref_transaction_free(transaction);
 		strbuf_release(&err);
+
 		return NULL; /* good */
 	}
 }
@@ -1052,6 +1055,7 @@ static void execute_commands(struct command *commands,
 	struct command *cmd;
 	unsigned char sha1[20];
 	struct iterate_data data;
+	struct strbuf err = STRBUF_INIT;
 
 	if (unpacker_error) {
 		for (cmd = commands; cmd; cmd = cmd->next)
@@ -1059,6 +1063,16 @@ static void execute_commands(struct command *commands,
 		return;
 	}
 
+	if (use_atomic) {
+		transaction = ref_transaction_begin(&err);
+		if (!transaction) {
+			error("%s", err.buf);
+			strbuf_release(&err);
+			for (cmd = commands; cmd; cmd = cmd->next)
+				cmd->error_string = "transaction error";
+			return;
+		}
+	}
 	data.cmds = commands;
 	data.si = si;
 	if (check_everything_connected(iterate_receive_command_list, 0, &data))
@@ -1087,7 +1101,30 @@ static void execute_commands(struct command *commands,
 		if (cmd->skip_update)
 			continue;
 
+		if (!use_atomic) {
+			transaction = ref_transaction_begin(&err);
+			if (!transaction) {
+				rp_error("%s", err.buf);
+				strbuf_release(&err);
+				cmd->error_string = "failed to start transaction";
+				continue;
+			}
+		}
 		cmd->error_string = update(cmd, si);
+		if (!cmd->error_string) {
+			if (!use_atomic
+			    && ref_transaction_commit(transaction, &err)) {
+				ref_transaction_free(transaction);
+				rp_error("%s", err.buf);
+				strbuf_release(&err);
+				cmd->error_string = "failed to update ref";
+			}
+		} else if (use_atomic) {
+			goto atomic_failure;
+		} else {
+			ref_transaction_free(transaction);
+		}
+
 		if (shallow_update && !cmd->error_string &&
 		    si->shallow_ref[cmd->index]) {
 			error("BUG: connectivity check has not been run on ref %s",
@@ -1096,10 +1133,28 @@ static void execute_commands(struct command *commands,
 		}
 	}
 
+	if (use_atomic) {
+		if (ref_transaction_commit(transaction, &err)) {
+			rp_error("%s", err.buf);
+			for (cmd = commands; cmd; cmd = cmd->next)
+				cmd->error_string = err.buf;
+		}
+		ref_transaction_free(transaction);
+	}
 	if (shallow_update && !checked_connectivity)
 		error("BUG: run 'git fsck' for safety.\n"
 		      "If there are errors, try to remove "
 		      "the reported refs above");
+	strbuf_release(&err);
+
+	return;
+
+atomic_failure:
+	for (cmd = commands; cmd; cmd = cmd->next)
+		if (!cmd->error_string)
+			cmd->error_string = "atomic push failure";
+
+	ref_transaction_free(transaction);
 }
 
 static struct command **queue_command(struct command **tail,

-- 
2.2.0.31.gad78000.dirty
Previous: Junio C HamanoNext: Eric Sunshine
Message 50 of 56 in “Add a flag to push atomically”
  1. 0/5 Add a flag to push atomicallyStefan Beller, Dec 15, 2014
  2. 1/5 receive-pack.c: add protocol support to negotiate atomic-pushStefan Beller, Dec 15, 2014
  3. Junio C HamanoDec 15, 2014
  4. Stefan BellerDec 15, 2014
  5. 2/5 send-pack.c: add an --atomic-push command line argumentStefan Beller, Dec 15, 2014
  6. Junio C HamanoDec 15, 2014
  7. 3/5 receive-pack.c: use a single ref_transaction for atomic pushesStefan Beller, Dec 15, 2014
  8. Junio C HamanoDec 15, 2014
  9. 4/5 push.c: add an --atomic-push argumentStefan Beller, Dec 15, 2014
  10. Junio C HamanoDec 15, 2014
  11. 5/5 t5543-atomic-push.sh: add basic tests for atomic pushesStefan Beller, Dec 15, 2014
  12. Junio C HamanoDec 15, 2014
  13. Junio C HamanoDec 15, 2014
  14. 1/6 receive-pack.c: add protocol support to negotiate atomic-pushStefan Beller, Dec 16, 2014
  15. 2/6 send-pack: Invert the return value of ref_update_to_be_sentStefan Beller, Dec 16, 2014
  16. Junio C HamanoDec 16, 2014
  17. 3/6 send-pack.c: add --atomic command line argumentStefan Beller, Dec 16, 2014
  18. Junio C HamanoDec 16, 2014
  19. 4/6 receive-pack.c: use a single ref_transaction for atomic pushesStefan Beller, Dec 16, 2014
  20. Eric SunshineDec 16, 2014
  21. Eric SunshineDec 16, 2014
  22. Junio C HamanoDec 16, 2014
  23. 5/6 push.c: add an --atomic-push argumentStefan Beller, Dec 16, 2014
  24. Eric SunshineDec 16, 2014
  25. Junio C HamanoDec 16, 2014
  26. Junio C HamanoDec 16, 2014
  27. 6/6 t5543-atomic-push.sh: add basic tests for atomic pushesStefan Beller, Dec 16, 2014
  28. receive-pack: refuse all commands if one fails in atomic modeStefan Beller, Dec 16, 2014
  29. Junio C HamanoDec 16, 2014
  30. Eric SunshineDec 16, 2014
  31. Junio C HamanoDec 16, 2014
  32. Stefan BellerDec 16, 2014
  33. Junio C HamanoDec 16, 2014
  34. Stefan BellerDec 16, 2014
  35. Junio C HamanoDec 16, 2014
  36. Stefan BellerDec 16, 2014
  37. Junio C HamanoDec 16, 2014
  38. 0/6 atomic pushesStefan Beller, Dec 17, 2014
  39. 1/6 receive-pack.c: add protocol support to negotiate atomicStefan Beller, Dec 17, 2014
  40. Eric SunshineDec 19, 2014
  41. 2/6 send-pack: Rename ref_update_to_be_sent to check_to_send_updateStefan Beller, Dec 17, 2014
  42. Junio C HamanoDec 17, 2014
  43. 3/6 send-pack.c: add --atomic command line argumentStefan Beller, Dec 17, 2014
  44. Junio C HamanoDec 17, 2014
  45. Eric SunshineDec 19, 2014
  46. 4/6 receive-pack.c: use a single ref_transaction for atomic pushesStefan Beller, Dec 17, 2014
  47. Junio C HamanoDec 17, 2014
  48. Stefan BellerDec 17, 2014
  49. Junio C HamanoDec 18, 2014
  50. 4/6 receive-pack.c: use a single ref_transaction for atomic pushesStefan Beller, Dec 18, 2014
  51. Eric SunshineDec 18, 2014
  52. 4/6 receive-pack.c: use a single ref_transaction for atomic pushesStefan Beller, Dec 19, 2014
  53. Eric SunshineDec 19, 2014
  54. 5/6 push.c: add an --atomic argumentStefan Beller, Dec 17, 2014
  55. Eric SunshineDec 19, 2014
  56. 6/6 t5543-atomic-push.sh: add basic tests for atomic pushesStefan Beller, Dec 17, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.