git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 17/33] repack_without_ref(): silence errors for dangling packed refs

From
Junio C Hamano <gitster@pobox.com>
Date
Apr 22, 2013, 22:21 UTC
Message-ID
<7vip3enr4m.fsf@alter.siamese.dyndns.org>
In-Reply-To
<1366660361-21831-18-git-send-email-mhagger@alum.mit.edu>
Michael Haggerty <mhagger@alum.mit.edu> writes:
Show 35 quoted lines
> Stop emitting an error message when deleting a packed reference if we
> find another dangling packed reference that is overridden by a loose
> reference.  See the previous commit for a longer explanation of the
> issue.
>
> We have to be careful to make sure that the invalid packed reference
> really *is* overridden by a loose reference; otherwise what we have
> found is repository corruption, which we *should* report.
>
> Please note that this approach is vulnerable to a race condition
> similar to the race conditions already known to affect packed
> references [1]:
>
> * Process 1 tries to peel packed reference X as part of deleting
>   another packed reference.  It discovers that X does not refer to a
>   valid object (because the object that it referred to has been
>   garbage collected).
>
> * Process 2 tries to delete reference X.  It starts by deleting the
>   loose reference X.
>
> * Process 1 checks whether there is a loose reference X.  There is not
>   (it has just been deleted by process 2), so process 1 reports a
>   spurious error "X does not point to a valid object!"
>
> The worst case seems relatively harmless, and the fix is identical to
> the fix that will be needed for the other race conditions (namely
> holding a lock on the packed-refs file during *all* reference
> deletions), so we leave the cleaning up of all of them as a future
> project.
>
> [1] http://thread.gmane.org/gmane.comp.version-control.git/211956
>
> Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
> ---

Clean and straight-forward implementation that is very well explained. I like this much better than the previous round ;-).

Thanks, will re-queue.
Show 65 quoted lines
>  refs.c               | 37 +++++++++++++++++++++++++++++++++++--
>  t/t3210-pack-refs.sh |  2 +-
>  2 files changed, 36 insertions(+), 3 deletions(-)
>
> diff --git a/refs.c b/refs.c
> index ed54ed4..2957f6d 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -1901,8 +1901,41 @@ static int repack_without_ref_fn(struct ref_entry *entry, void *cb_data)
>  
>  	if (!strcmp(data->refname, entry->name))
>  		return 0;
> -	if (!ref_resolves_to_object(entry))
> -		return 0; /* Skip broken refs */
> +	if (entry->flag & REF_ISBROKEN) {
> +		/* This shouldn't happen to packed refs. */
> +		error("%s is broken!", entry->name);
> +		return 0;
> +	}
> +	if (!has_sha1_file(entry->u.value.sha1)) {
> +		unsigned char sha1[20];
> +		int flags;
> +
> +		if (read_ref_full(entry->name, sha1, 0, &flags))
> +			/* We should at least have found the packed ref. */
> +			die("Internal error");
> +		if ((flags & REF_ISSYMREF) || !(flags & REF_ISPACKED))
> +			/*
> +			 * This packed reference is overridden by a
> +			 * loose reference, so it is OK that its value
> +			 * is no longer valid; for example, it might
> +			 * refer to an object that has been garbage
> +			 * collected.  For this purpose we don't even
> +			 * care whether the loose reference itself is
> +			 * invalid, broken, symbolic, etc.  Silently
> +			 * omit the packed reference from the output.
> +			 */
> +			return 0;
> +		/*
> +		 * There is no overriding loose reference, so the fact
> +		 * that this reference doesn't refer to a valid object
> +		 * indicates some kind of repository corruption.
> +		 * Report the problem, then omit the reference from
> +		 * the output.
> +		 */
> +		error("%s does not point to a valid object!", entry->name);
> +		return 0;
> +	}
> +
>  	len = snprintf(line, sizeof(line), "%s %s\n",
>  		       sha1_to_hex(entry->u.value.sha1), entry->name);
>  	/* this should not happen but just being defensive */
> diff --git a/t/t3210-pack-refs.sh b/t/t3210-pack-refs.sh
> index c032d88..559f602 100755
> --- a/t/t3210-pack-refs.sh
> +++ b/t/t3210-pack-refs.sh
> @@ -142,7 +142,7 @@ test_expect_success 'delete ref with dangling packed version' '
>  	test_cmp /dev/null result
>  '
>  
> -test_expect_failure 'delete ref while another dangling packed ref' '
> +test_expect_success 'delete ref while another dangling packed ref' '
>  	git branch lamb &&
>  	git commit --allow-empty -m "future garbage" &&
>  	git pack-refs --all &&
Previous: Michael HaggertyNext: Michael Haggerty
Message 19 of 40 in “Various cleanups around reference packing and peeling”
  1. 00/33 Various cleanups around reference packing and peelingMichael Haggerty, Apr 22, 2013
  2. 01/33 refs: document flags constants REF_*Michael Haggerty, Apr 22, 2013
  3. 02/33 refs: document the fields of struct ref_valueMichael Haggerty, Apr 22, 2013
  4. 03/33 refs: document do_for_each_ref() and do_one_ref()Michael Haggerty, Apr 22, 2013
  5. 04/33 refs: document how current_ref is usedMichael Haggerty, Apr 22, 2013
  6. 05/33 refs: define constant PEELED_LINE_LENGTHMichael Haggerty, Apr 22, 2013
  7. 06/33 do_for_each_ref_in_dirs(): remove dead codeMichael Haggerty, Apr 22, 2013
  8. 07/33 get_packed_ref(): return a ref_entryMichael Haggerty, Apr 22, 2013
  9. 08/33 peel_ref(): use function get_packed_ref()Michael Haggerty, Apr 22, 2013
  10. 09/33 repack_without_ref(): use function get_packed_ref()Michael Haggerty, Apr 22, 2013
  11. 10/33 refs: extract a function ref_resolves_to_object()Michael Haggerty, Apr 22, 2013
  12. 11/33 refs: extract function peel_object()Michael Haggerty, Apr 22, 2013
  13. 12/33 peel_object(): give more specific information in return valueMichael Haggerty, Apr 22, 2013
  14. 13/33 peel_ref(): fix return value for non-peelable, not-current referenceMichael Haggerty, Apr 22, 2013
  15. 14/33 refs: extract a function peel_entry()Michael Haggerty, Apr 22, 2013
  16. 15/33 refs: change the internal reference-iteration APIMichael Haggerty, Apr 22, 2013
  17. 16/33 t3210: test for spurious error messages for dangling packed refsMichael Haggerty, Apr 22, 2013
  18. 17/33 repack_without_ref(): silence errors for dangling packed refsMichael Haggerty, Apr 22, 2013
  19. Junio C HamanoApr 22, 2013
  20. 18/33 search_ref_dir(): return an index rather than a pointerMichael Haggerty, Apr 22, 2013
  21. 19/33 refs: change how packed refs are deletedMichael Haggerty, Apr 22, 2013
  22. 20/33 t3211: demonstrate loss of peeled refs if a packed ref is deletedMichael Haggerty, Apr 22, 2013
  23. 21/33 repack_without_ref(): write peeled refs in the rewritten fileMichael Haggerty, Apr 22, 2013
  24. 22/33 refs: extract a function write_packed_entry()Michael Haggerty, Apr 22, 2013
  25. 23/33 pack-refs: rename handle_one_ref() to pack_one_ref()Michael Haggerty, Apr 22, 2013
  26. 24/33 pack-refs: merge code from pack-refs.{c,h} into refs.{c,h}Michael Haggerty, Apr 22, 2013
  27. 25/33 pack_one_ref(): rename "path" parameter to "refname"Michael Haggerty, Apr 22, 2013
  28. 26/33 refs: use same lock_file object for both ref-packing functionsMichael Haggerty, Apr 22, 2013
  29. 27/33 pack_refs(): change to use do_for_each_entry()Michael Haggerty, Apr 22, 2013
  30. 28/33 refs: inline function do_not_prune()Michael Haggerty, Apr 22, 2013
  31. 29/33 pack_one_ref(): use function peel_entry()Michael Haggerty, Apr 22, 2013
  32. 30/33 pack_one_ref(): use write_packed_entry() to do the writingMichael Haggerty, Apr 22, 2013
  33. 31/33 pack_one_ref(): do some cheap tests before a more expensive oneMichael Haggerty, Apr 22, 2013
  34. 32/33 refs: change do_for_each_*() functions to take ref_cache argumentsMichael Haggerty, Apr 22, 2013
  35. 33/33 refs: handle the main ref_cache speciallyMichael Haggerty, Apr 22, 2013
  36. Junio C HamanoApr 22, 2013
  37. fixup! t3210: test for spurious error messages for dangling packed refsMichael Haggerty, Apr 23, 2013
  38. Junio C HamanoApr 23, 2013
  39. Michael HaggertyApr 24, 2013
  40. Michael HaggertyApr 27, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.