git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 22/29] commit_packed_refs(): use a staging file separate from the lockfile

From
Michael Haggerty <mhagger@alum.mit.edu>
Date
Jun 24, 2017, 11:43 UTC
Message-ID
<8156514c-f255-5bda-2011-db8f512d6fb9@alum.mit.edu>
In-Reply-To
<20170623194610.o2eyte4xmlsscrlz@sigill.intra.peff.net>
On 06/23/2017 09:46 PM, Jeff King wrote:
Show 47 quoted lines
> On Fri, Jun 23, 2017 at 09:01:40AM +0200, Michael Haggerty wrote:
> 
>> @@ -522,10 +529,16 @@ int lock_packed_refs(struct ref_store *ref_store, int flags)
>>  		timeout_configured = 1;
>>  	}
>>  
>> +	/*
>> +	 * Note that we close the lockfile immediately because we
>> +	 * don't write new content to it, but rather to a separate
>> +	 * tempfile.
>> +	 */
>>  	if (hold_lock_file_for_update_timeout(
>>  			    &refs->lock,
>>  			    refs->path,
>> -			    flags, timeout_value) < 0)
>> +			    flags, timeout_value) < 0 ||
>> +	    close_lock_file(&refs->lock))
>>  		return -1;
> 
> I was wondering whether we'd catch a case which accidentally wrote to
> the lockfile (instead of the new tempfile, but this close() is a good
> safety check).
> 
>> -	if (commit_lock_file(&refs->lock)) {
>> -		strbuf_addf(err, "error overwriting %s: %s",
>> +	if (rename_tempfile(&refs->tempfile, refs->path)) {
>> +		strbuf_addf(err, "error replacing %s: %s",
>>  			    refs->path, strerror(errno));
>>  		goto out;
>>  	}
> 
> So our idea of committing now is the tempfile rename, and then...
> 
>> @@ -617,9 +640,10 @@ int commit_packed_refs(struct ref_store *ref_store, struct strbuf *err)
>>  	goto out;
>>  
>>  error:
>> -	rollback_lock_file(&refs->lock);
>> +	delete_tempfile(&refs->tempfile);
>>  
>>  out:
>> +	rollback_lock_file(&refs->lock);
> 
> We always rollback the lockfile regardless, because committing it would
> overwrite our new content with an empty file. There's no real safety
> against somebody calling commit_lock_file() on it, but it also seems
> like an uncommon error to make.

If this seems too dangerous, we could add a `LOCK_NO_COMMIT` flag for `hold_lock_file_for_update()` and `hold_lock_file_for_update_timeout()`, which would die() if somebody tries to commit the associated lockfile. I think we can live without it. I hope that any such bugs would be caught immediately by CI. But I admit that the prospect of renaming an empty file on top of a `packed-refs` file is quite sobering, so if anybody is worried about this, let me know and I'll implement it.

Michael
Previous: Jeff KingNext: Jeff King
Message 27 of 52 in “Create a reference backend for packed refs”
  1. 00/29 Create a reference backend for packed refsMichael Haggerty, Jun 23, 2017
  2. 01/29 t1408: add a test of stale packed refs covered by loose refsMichael Haggerty, Jun 23, 2017
  3. Jeff KingJun 23, 2017
  4. Junio C HamanoJun 23, 2017
  5. 02/29 add_packed_ref(): teach function to overwrite existing refsMichael Haggerty, Jun 23, 2017
  6. 07/29 validate_packed_ref_cache(): take a `packed_ref_store *` parameterMichael Haggerty, Jun 23, 2017
  7. 05/29 packed_ref_store: move `packed_refs_lock` member hereMichael Haggerty, Jun 23, 2017
  8. 03/29 packed_ref_store: new structMichael Haggerty, Jun 23, 2017
  9. 09/29 get_packed_refs(): take a `packed_ref_store *` parameterMichael Haggerty, Jun 23, 2017
  10. 04/29 packed_ref_store: move `packed_refs_path` hereMichael Haggerty, Jun 23, 2017
  11. 13/29 rollback_packed_refs(): take a `packed_ref_store *` parameterMichael Haggerty, Jun 23, 2017
  12. 12/29 commit_packed_refs(): take a `packed_ref_store *` parameterMichael Haggerty, Jun 23, 2017
  13. 14/29 get_packed_ref(): take a `packed_ref_store *` parameterMichael Haggerty, Jun 23, 2017
  14. 06/29 clear_packed_ref_cache(): take a `packed_ref_store *` parameterMichael Haggerty, Jun 23, 2017
  15. 17/29 packed_ref_store: support iterationMichael Haggerty, Jun 23, 2017
  16. Jeff KingJun 23, 2017
  17. 16/29 packed_peel_ref(): new function, extracted from `files_peel_ref()`Michael Haggerty, Jun 23, 2017
  18. 19/29 packed-backend: new module for handling packed referencesMichael Haggerty, Jun 23, 2017
  19. Jeff KingJun 23, 2017
  20. Stefan BellerJun 23, 2017
  21. Junio C HamanoJun 23, 2017
  22. 20/29 packed_ref_store: make class into a subclass of `ref_store`Michael Haggerty, Jun 23, 2017
  23. Jeff KingJun 23, 2017
  24. 21/29 commit_packed_refs(): report errors rather than dyingMichael Haggerty, Jun 23, 2017
  25. 22/29 commit_packed_refs(): use a staging file separate from the lockfileMichael Haggerty, Jun 23, 2017
  26. Jeff KingJun 23, 2017
  27. Michael HaggertyJun 24, 2017
  28. Jeff KingJun 24, 2017
  29. 24/29 packed_refs_lock(): report errors via a `struct strbuf *err`Michael Haggerty, Jun 23, 2017
  30. 26/29 clear_packed_ref_cache(): don't protest if the lock is heldMichael Haggerty, Jun 23, 2017
  31. Jeff KingJun 23, 2017
  32. 25/29 packed_refs_unlock(), packed_refs_is_locked(): new functionsMichael Haggerty, Jun 23, 2017
  33. 23/29 packed_refs_lock(): function renamed from lock_packed_refs()Michael Haggerty, Jun 23, 2017
  34. 28/29 repack_without_refs(): don't lock or unlock the packed refsMichael Haggerty, Jun 23, 2017
  35. Jeff KingJun 23, 2017
  36. Michael HaggertyJul 1, 2017
  37. 27/29 commit_packed_refs(): remove call to `packed_refs_unlock()`Michael Haggerty, Jun 23, 2017
  38. Jeff KingJun 23, 2017
  39. 29/29 read_packed_refs(): die if `packed-refs` contains bogus dataMichael Haggerty, Jun 23, 2017
  40. Jeff KingJun 23, 2017
  41. Michael HaggertyJul 1, 2017
  42. 11/29 lock_packed_refs(): take a `packed_ref_store *` parameterMichael Haggerty, Jun 23, 2017
  43. 10/29 add_packed_ref(): take a `packed_ref_store *` parameterMichael Haggerty, Jun 23, 2017
  44. 15/29 repack_without_refs(): take a `packed_ref_store *` parameterMichael Haggerty, Jun 23, 2017
  45. 18/29 packed_read_raw_ref(): new function, replacing `resolve_packed_ref()`Michael Haggerty, Jun 23, 2017
  46. 08/29 get_packed_ref_cache(): take a `packed_ref_store *` parameterMichael Haggerty, Jun 23, 2017
  47. Jeff KingJun 23, 2017
  48. Jeff KingJun 23, 2017
  49. Junio C HamanoJun 23, 2017
  50. Jeff KingJun 24, 2017
  51. Michael HaggertyJun 24, 2017
  52. Junio C HamanoJun 23, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.