git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 10/17] lock_ref_sha1_basic(): on SCLD_VANISHED, retry

From
Michael Haggerty <mhagger@alum.mit.edu>
Date
Jan 7, 2014, 10:25 UTC
Message-ID
<52CBD622.3010805@alum.mit.edu>
In-Reply-To
<xmqq4n5hj8ry.fsf@gitster.dls.corp.google.com>
On 01/06/2014 06:54 PM, Junio C Hamano wrote:
Show 51 quoted lines
> Michael Haggerty <mhagger@alum.mit.edu> writes:
> 
>> If safe_create_leading_directories() fails because a file along the
>> path unexpectedly vanished, try again (up to 3 times).
>>
>> This can occur if another process is deleting directories at the same
>> time as we are trying to make them.  For example, "git pack-refs
>> --all" tries to delete the loose refs and any empty directories that
>> are left behind.  If a pack-refs process is running, then it might
>> delete a directory that we need to put a new loose reference in.
>>
>> If safe_create_leading_directories() thinks this might have happened,
>> then take its advice and try again (maximum three attempts).
>>
>> Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
>> ---
>>  refs.c | 11 ++++++++++-
>>  1 file changed, 10 insertions(+), 1 deletion(-)
>>
>> diff --git a/refs.c b/refs.c
>> index 3926136..6eb8a02 100644
>> --- a/refs.c
>> +++ b/refs.c
>> @@ -2039,6 +2039,7 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
>>  	int type, lflags;
>>  	int mustexist = (old_sha1 && !is_null_sha1(old_sha1));
>>  	int missing = 0;
>> +	int attempts = 3;
>>  
>>  	lock = xcalloc(1, sizeof(struct ref_lock));
>>  	lock->lock_fd = -1;
>> @@ -2093,7 +2094,15 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
>>  	if ((flags & REF_NODEREF) && (type & REF_ISSYMREF))
>>  		lock->force_write = 1;
>>  
>> -	if (safe_create_leading_directories(ref_file)) {
>> + retry:
>> +	switch (safe_create_leading_directories(ref_file)) {
>> +	case SCLD_OK:
>> +		break; /* success */
>> +	case SCLD_VANISHED:
>> +		if (--attempts > 0)
>> +			goto retry;
>> +		/* fall through */
> 
> Hmph.
> 
> Having no backoff/sleep at all might be OK here as long as the other
> side that removes does not retry (and I do not think the other side
> would, even though I haven't read through the series to the end yet
> ;-)).

remove_dir_recurse() only tries deleting directories once (I haven't changed that). And from a broader perspective, it would be pretty silly for any tidy-up-directories function to try deleting things more than once. So I don't think it is a problem. But even in the worst case, this function only tries three times before giving up, so it shouldn't be a disaster.

Show 10 quoted lines
> This may be just a style thing, but I find that the variable name
> "attempts" that starts out as 3 quite misleading, as its value is
> not "the number of attempts made" but "the remaining number of
> attempts allowed."  Starting it from 0 and then
> 
> 	if (attempts++ < MAX_ATTEMPTS)
> 		goto retry;
> 
> would be one way to clarify it.  Renaming it to remaining_attempts
> would be another.

I just renamed the variable to attempts_remaining. (I thought I was following your suggestion, but now I see that I put the words in the opposite order; oh well, I think it's fine either way.)

Thanks for your review! I will wait a day or so for any additional comments, and then send a v3.

Michael
-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/
Previous: Junio C HamanoNext: Michael Haggerty
Message 17 of 29 in “Fix some mkdir/rmdir races”
  1. 00/17 Fix some mkdir/rmdir racesMichael Haggerty, Jan 6, 2014
  2. 01/17 safe_create_leading_directories(): fix format of "if" chainingMichael Haggerty, Jan 6, 2014
  3. 02/17 safe_create_leading_directories(): reduce scope of local variableMichael Haggerty, Jan 6, 2014
  4. 03/17 safe_create_leading_directories(): add explicit "slash" pointerMichael Haggerty, Jan 6, 2014
  5. Junio C HamanoJan 6, 2014
  6. Michael HaggertyJan 7, 2014
  7. Junio C HamanoJan 7, 2014
  8. Sebastian SchuberthJan 19, 2014
  9. 04/17 safe_create_leading_directories(): rename local variableMichael Haggerty, Jan 6, 2014
  10. 05/17 safe_create_leading_directories(): split on first of multiple slashesMichael Haggerty, Jan 6, 2014
  11. 06/17 safe_create_leading_directories(): always restore slash at end of loopMichael Haggerty, Jan 6, 2014
  12. 07/17 safe_create_leading_directories(): introduce enum for return valuesMichael Haggerty, Jan 6, 2014
  13. 08/17 cmd_init_db(): when creating directories, handle errors conservativelyMichael Haggerty, Jan 6, 2014
  14. 09/17 safe_create_leading_directories(): add new error value SCLD_VANISHEDMichael Haggerty, Jan 6, 2014
  15. 10/17 lock_ref_sha1_basic(): on SCLD_VANISHED, retryMichael Haggerty, Jan 6, 2014
  16. Junio C HamanoJan 6, 2014
  17. Michael HaggertyJan 7, 2014
  18. 11/17 lock_ref_sha1_basic(): if locking fails with ENOENT, retryMichael Haggerty, Jan 6, 2014
  19. 12/17 remove_dir_recurse(): tighten condition for removing unreadable dirMichael Haggerty, Jan 6, 2014
  20. 13/17 remove_dir_recurse(): handle disappearing files and directoriesMichael Haggerty, Jan 6, 2014
  21. Junio C HamanoJan 6, 2014
  22. Michael HaggertyJan 7, 2014
  23. Junio C HamanoJan 7, 2014
  24. 14/17 rename_ref(): extract function rename_tmp_log()Michael Haggerty, Jan 6, 2014
  25. 15/17 rename_tmp_log(): handle a possible mkdir/rmdir raceMichael Haggerty, Jan 6, 2014
  26. 16/17 rename_tmp_log(): limit the number of remote_empty_directories() attemptsMichael Haggerty, Jan 6, 2014
  27. 17/17 rename_tmp_log(): on SCLD_VANISHED, retryMichael Haggerty, Jan 6, 2014
  28. Junio C HamanoJan 6, 2014
  29. Michael HaggertyJan 7, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.