git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v20 33/48] walker.c: use ref transaction for ref updates

From
Michael Haggerty <mhagger@alum.mit.edu>
Date
Jul 8, 2014, 13:33 UTC
Message-ID
<53BBF333.903@alum.mit.edu>
In-Reply-To
<1403275409-28173-34-git-send-email-sahlberg@google.com>
On 06/20/2014 04:43 PM, Ronnie Sahlberg wrote:
Show 8 quoted lines
> Switch to using ref transactions in walker_fetch(). As part of the refactoring
> to use ref transactions we also fix a potential memory leak where in the
> original code if write_ref_sha1() would fail we would end up returning from
> the function without free()ing the msg string.
> 
> Note that this function is only called when fetching from a remote HTTP
> repository onto the local (most of the time single-user) repository which
> likely means that the type of collissions that the previous locking would
s/collissions/collisions/
> protect against and cause the fetch to fail for to be even more rare.
Grammatico: s/to be/are/ ?
Show 42 quoted lines
> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
> Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
> ---
>  walker.c | 59 +++++++++++++++++++++++++++++++++++------------------------
>  1 file changed, 35 insertions(+), 24 deletions(-)
> 
> diff --git a/walker.c b/walker.c
> index 1dd86b8..60d9f9e 100644
> --- a/walker.c
> +++ b/walker.c
> @@ -251,39 +251,36 @@ void walker_targets_free(int targets, char **target, const char **write_ref)
>  int walker_fetch(struct walker *walker, int targets, char **target,
>  		 const char **write_ref, const char *write_ref_log_details)
>  {
> -	struct ref_lock **lock = xcalloc(targets, sizeof(struct ref_lock *));
> +	struct strbuf ref_name = STRBUF_INIT;
> +	struct strbuf err = STRBUF_INIT;
> +	struct ref_transaction *transaction = NULL;
>  	unsigned char *sha1 = xmalloc(targets * 20);
> -	char *msg;
> -	int ret;
> +	char *msg = NULL;
>  	int i;
>  
>  	save_commit_buffer = 0;
>  
> -	for (i = 0; i < targets; i++) {
> -		if (!write_ref || !write_ref[i])
> -			continue;
> -
> -		lock[i] = lock_ref_sha1(write_ref[i], NULL);
> -		if (!lock[i]) {
> -			error("Can't lock ref %s", write_ref[i]);
> -			goto unlock_and_fail;
> +	if (write_ref) {
> +		transaction = ref_transaction_begin(&err);
> +		if (!transaction) {
> +			error("%s", err.buf);
> +			goto rollback_and_fail;
>  		}
>  	}
> -

Is there some reason why the transaction cannot be built up during a single iteration over targets, thereby also avoiding the need for the sha1[20*i] stuff? This seems like exactly the kind of situation where transactions should *save* code. But perhaps I've overlooked a dependency between the two loops.

Show 64 quoted lines
>  	if (!walker->get_recover)
>  		for_each_ref(mark_complete, NULL);
>  
>  	for (i = 0; i < targets; i++) {
>  		if (interpret_target(walker, target[i], &sha1[20 * i])) {
>  			error("Could not interpret response from server '%s' as something to pull", target[i]);
> -			goto unlock_and_fail;
> +			goto rollback_and_fail;
>  		}
>  		if (process(walker, lookup_unknown_object(&sha1[20 * i])))
> -			goto unlock_and_fail;
> +			goto rollback_and_fail;
>  	}
>  
>  	if (loop(walker))
> -		goto unlock_and_fail;
> +		goto rollback_and_fail;
>  
>  	if (write_ref_log_details) {
>  		msg = xmalloc(strlen(write_ref_log_details) + 12);
> @@ -294,19 +291,33 @@ int walker_fetch(struct walker *walker, int targets, char **target,
>  	for (i = 0; i < targets; i++) {
>  		if (!write_ref || !write_ref[i])
>  			continue;
> -		ret = write_ref_sha1(lock[i], &sha1[20 * i], msg ? msg : "fetch (unknown)");
> -		lock[i] = NULL;
> -		if (ret)
> -			goto unlock_and_fail;
> +		strbuf_reset(&ref_name);
> +		strbuf_addf(&ref_name, "refs/%s", write_ref[i]);
> +		if (ref_transaction_update(transaction, ref_name.buf,
> +					   &sha1[20 * i], NULL, 0, 0,
> +					   &err)) {
> +			error("%s", err.buf);
> +			goto rollback_and_fail;
> +		}
> +	}
> +	if (write_ref) {
> +		if (ref_transaction_commit(transaction,
> +					   msg ? msg : "fetch (unknown)",
> +					   &err)) {
> +			error("%s", err.buf);
> +			goto rollback_and_fail;
> +		}
> +		ref_transaction_free(transaction);
>  	}
> -	free(msg);
>  
> +	free(msg);
>  	return 0;
>  
> -unlock_and_fail:
> -	for (i = 0; i < targets; i++)
> -		if (lock[i])
> -			unlock_ref(lock[i]);
> +rollback_and_fail:
> +	ref_transaction_free(transaction);
> +	free(msg);
> +	strbuf_release(&err);
> +	strbuf_release(&ref_name);
>  
>  	return -1;
>  }
> 
Michael
-- 
Michael Haggerty
mhagger@alum.mit.edu
Previous: Ronnie SahlbergNext: Ronnie Sahlberg
Message 52 of 99 in “Use ref transactions”
  1. 00/48 Use ref transactionsRonnie Sahlberg, Jun 20, 2014
  2. 01/48 refs.c: remove ref_transaction_rollbackRonnie Sahlberg, Jun 20, 2014
  3. 02/48 refs.c: ref_transaction_commit should not free the transactionRonnie Sahlberg, Jun 20, 2014
  4. 03/48 refs.c: constify the sha arguments for ref_transaction_create|delete|updateRonnie Sahlberg, Jun 20, 2014
  5. 04/48 refs.c: allow passing NULL to ref_transaction_freeRonnie Sahlberg, Jun 20, 2014
  6. 05/48 refs.c: add a strbuf argument to ref_transaction_commit for error loggingRonnie Sahlberg, Jun 20, 2014
  7. 06/48 lockfile.c: add a new public function unable_to_lock_messageRonnie Sahlberg, Jun 20, 2014
  8. 07/48 lockfile.c: make lock_file return a meaningful errno on failureiRonnie Sahlberg, Jun 20, 2014
  9. Michael HaggertyJul 8, 2014
  10. 08/48 refs.c: add an err argument to repack_without_refsRonnie Sahlberg, Jun 20, 2014
  11. 09/48 refs.c: make sure log_ref_setup returns a meaningful errnoRonnie Sahlberg, Jun 20, 2014
  12. 10/48 refs.c: verify_lock should set errno to something meaningfulRonnie Sahlberg, Jun 20, 2014
  13. 11/48 refs.c: make remove_empty_directories always set errno to something saneRonnie Sahlberg, Jun 20, 2014
  14. 12/48 refs.c: commit_packed_refs to return a meaningful errno on failureRonnie Sahlberg, Jun 20, 2014
  15. 13/48 refs.c: make resolve_ref_unsafe set errno to something meaningful on errorRonnie Sahlberg, Jun 20, 2014
  16. Karsten BleesJun 26, 2014
  17. 14/48 refs.c: log_ref_write should try to return meaningful errnoRonnie Sahlberg, Jun 20, 2014
  18. 15/48 refs.c: make ref_update_reject_duplicates take a strbuf argument for errorsRonnie Sahlberg, Jun 20, 2014
  19. 16/48 refs.c: make update_ref_write update a strbuf on failureRonnie Sahlberg, Jun 20, 2014
  20. 17/48 update-ref: use err argument to get error from ref_transaction_commitRonnie Sahlberg, Jun 20, 2014
  21. 18/48 refs.c: remove the onerr argument to ref_transaction_commitRonnie Sahlberg, Jun 20, 2014
  22. 19/48 refs.c: change ref_transaction_update() to do error checking and return statusRonnie Sahlberg, Jun 20, 2014
  23. 20/48 refs.c: change ref_transaction_create to do error checking and return statusRonnie Sahlberg, Jun 20, 2014
  24. Michael HaggertyJul 8, 2014
  25. Ronnie SahlbergJul 14, 2014
  26. 21/48 refs.c: update ref_transaction_delete to check for error and return statusRonnie Sahlberg, Jun 20, 2014
  27. 22/48 refs.c: make ref_transaction_begin take an err argumentRonnie Sahlberg, Jun 20, 2014
  28. Michael HaggertyJul 8, 2014
  29. Ronnie SahlbergJul 14, 2014
  30. 23/48 refs.c: add transaction.status and track OPEN/CLOSED/ERRORRonnie Sahlberg, Jun 20, 2014
  31. Michael HaggertyJul 8, 2014
  32. Ronnie SahlbergJul 14, 2014
  33. 24/48 tag.c: use ref transactions when doing updatesRonnie Sahlberg, Jun 20, 2014
  34. Michael HaggertyJul 8, 2014
  35. 25/48 replace.c: use the ref transaction functions for updatesRonnie Sahlberg, Jun 20, 2014
  36. Michael HaggertyJul 8, 2014
  37. Ronnie SahlbergJul 14, 2014
  38. 26/48 commit.c: use ref transactions for updatesRonnie Sahlberg, Jun 20, 2014
  39. 27/48 sequencer.c: use ref transactions for all ref updatesRonnie Sahlberg, Jun 20, 2014
  40. Michael HaggertyJul 8, 2014
  41. Ronnie SahlbergJul 14, 2014
  42. 28/48 fast-import.c: change update_branch to use ref transactionsRonnie Sahlberg, Jun 20, 2014
  43. 29/48 branch.c: use ref transaction for all ref updatesRonnie Sahlberg, Jun 20, 2014
  44. 30/48 refs.c: change update_ref to use a transactionRonnie Sahlberg, Jun 20, 2014
  45. Michael HaggertyJul 8, 2014
  46. Ronnie SahlbergJul 14, 2014
  47. 31/48 receive-pack.c: use a reference transaction for updating the refsRonnie Sahlberg, Jun 20, 2014
  48. Michael HaggertyJul 8, 2014
  49. Ronnie SahlbergJul 14, 2014
  50. 32/48 fast-import.c: use a ref transaction when dumping tagsRonnie Sahlberg, Jun 20, 2014
  51. 33/48 walker.c: use ref transaction for ref updatesRonnie Sahlberg, Jun 20, 2014
  52. Michael HaggertyJul 8, 2014
  53. Ronnie SahlbergJul 14, 2014
  54. 34/48 refs.c: make lock_ref_sha1 staticRonnie Sahlberg, Jun 20, 2014
  55. 35/48 refs.c: remove the update_ref_lock functionRonnie Sahlberg, Jun 20, 2014
  56. 36/48 refs.c: remove the update_ref_write functionRonnie Sahlberg, Jun 20, 2014
  57. 37/48 refs.c: remove lock_ref_sha1Ronnie Sahlberg, Jun 20, 2014
  58. Michael HaggertyJul 8, 2014
  59. 38/48 refs.c: make prune_ref use a transaction to delete the refRonnie Sahlberg, Jun 20, 2014
  60. 39/48 refs.c: make delete_ref use a transactionRonnie Sahlberg, Jun 20, 2014
  61. Michael HaggertyJul 8, 2014
  62. Ronnie SahlbergJul 14, 2014
  63. 40/48 refs.c: add an err argument to delete_ref_looseRonnie Sahlberg, Jun 20, 2014
  64. Michael HaggertyJul 8, 2014
  65. Ronnie SahlbergJul 16, 2014
  66. 41/48 refs.c: pass the ref log message to _create/delete/update instead of _commitRonnie Sahlberg, Jun 20, 2014
  67. Michael HaggertyJul 8, 2014
  68. 42/48 refs.c: pass NULL as *flags to read_ref_fullRonnie Sahlberg, Jun 20, 2014
  69. 43/48 refs.c: move the check for valid refname to lock_ref_sha1_basicRonnie Sahlberg, Jun 20, 2014
  70. Michael HaggertyJul 8, 2014
  71. Ronnie SahlbergJul 15, 2014
  72. Jonathan NiederJul 15, 2014
  73. Jonathan NiederJul 15, 2014
  74. Junio C HamanoJul 15, 2014
  75. Ronnie SahlbergJul 15, 2014
  76. Ronnie SahlbergJul 15, 2014
  77. Ronnie SahlbergJul 15, 2014
  78. Michael HaggertyAug 20, 2014
  79. Ronnie SahlbergAug 20, 2014
  80. Jonathan NiederAug 20, 2014
  81. Ronnie SahlbergAug 20, 2014
  82. Michael HaggertyAug 20, 2014
  83. Ronnie SahlbergAug 21, 2014
  84. Junio C HamanoAug 20, 2014
  85. Michael HaggertyAug 20, 2014
  86. Junio C HamanoAug 20, 2014
  87. Ronnie SahlbergAug 20, 2014
  88. Michael HaggertyAug 22, 2014
  89. 44/48 refs.c: call lock_ref_sha1_basic directly from commitRonnie Sahlberg, Jun 20, 2014
  90. Michael HaggertyJul 8, 2014
  91. 45/48 refs.c: pass a skip list to name_conflict_fnRonnie Sahlberg, Jun 20, 2014
  92. 46/48 refs.c: propagate any errno==ENOTDIR from _commit back to the callersRonnie Sahlberg, Jun 20, 2014
  93. 47/48 fetch.c: change s_update_ref to use a ref transactionRonnie Sahlberg, Jun 20, 2014
  94. 48/48 refs.c: make write_ref_sha1 staticRonnie Sahlberg, Jun 20, 2014
  95. Michael HaggertyJul 8, 2014
  96. Junio C HamanoJul 8, 2014
  97. Jeff KingJul 9, 2014
  98. Ronnie SahlbergJul 14, 2014
  99. Ronnie SahlbergJul 14, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.