git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 14/33] refs: extract a function peel_entry()

From
Michael Haggerty <mhagger@alum.mit.edu>
Date
Apr 16, 2013, 22:01 UTC
Message-ID
<516DCA55.8030601@alum.mit.edu>
In-Reply-To
<7vhaj6mkcb.fsf@alter.siamese.dyndns.org>
On 04/16/2013 07:55 PM, Junio C Hamano wrote:
Show 73 quoted lines
> Michael Haggerty <mhagger@alum.mit.edu> writes:
> 
>> On 04/15/2013 07:38 PM, Junio C Hamano wrote:
>>> Michael Haggerty <mhagger@alum.mit.edu> writes:
>>>
>>>>  	if (read_ref_full(refname, base, 1, &flag))
>>>>  		return -1;
>>>>  
>>>> -	if ((flag & REF_ISPACKED)) {
>>>> +	/*
>>>> +	 * If the reference is packed, read its ref_entry from the
>>>> +	 * cache in the hope that we already know its peeled value.
>>>> +	 * We only try this optimization on packed references because
>>>> +	 * (a) forcing the filling of the loose reference cache could
>>>> +	 * be expensive and (b) loose references anyway usually do not
>>>> +	 * have REF_KNOWS_PEELED.
>>>> +	 */
>>>> +	if (flag & REF_ISPACKED) {
>>>>  		struct ref_entry *r = get_packed_ref(refname);
>>>
>>> This code makes the reader wonder what happens when a new loose ref
>>> masks a stale packed ref, but the worry is unfounded because the
>>> read_ref_full() wouldn't have gave us REF_ISPACKED in the flag in
>>> such a case.
>>>
>>> But somehow the calling sequence looks like such a mistake waiting
>>> to happen.  It would be much more clear if a function that returns a
>>> "struct ref_entry *" is used instead of read_ref_full() above, and
>>> we checked (r->flag & REF_ISPACKED) in the conditional, without a
>>> separate get_packed_ref(refname).
>>
>> As I'm sure you realize, I didn't change the code that you are referring
>> to; I just added a comment.
> 
> Yes.
> 
>> But yes, I sympathize with your complaint.  Additionally, the code has
>> the drawback that get_packed_ref() is called twice: once in
>> read_ref_full() and again in the if block here.  Unfortunately, this
>> isn't so easy to fix because read_ref_full() doesn't use the loose
>> reference cache, so the reference that it returns might not even have a
>> ref_entry associated with it (specifically, unless the returned flag
>> value has REF_ISPACKED set).  So there are a couple options:
>>
>> * Always read loose references through the cache; that way there would
>> always be a ref_entry in which the return value could be presented.
>> This would not be a good idea at the moment because the loose reference
>> cache is populated one directory at a time, and reading a whole
>> directory of loose references could be expensive.  So before
>> implementing this, it would be advisable to change the code to populate
>> the loose reference cache more selectively when single loose references
>> are needed.  -> This approach would be well beyond the scope of this
>> patch series.
>>
>> * Implement a function like read_ref_full() with an additional (struct
>> ref_entry **entry) argument that is written to *in the case* that the
>> reference that was returned has a ref_entry associated with it, and NULL
>> otherwise.  This would have to be an internal function because we don't
>> want to expose the ref_entry structure outside of refs.c.
>> read_ref_full() would be implemented on top of the new function.
> 
> Isn't there another?
> 
> Instead of using read_ref_full() at this callsite, can it call a
> function, given a refname, returns a pointer to "struct ref_entry",
> using the proper "a loose ref covers the packed ref with the same
> name" semantics?
> 
> I guess that may need the same machinery for your first option to
> implement it efficiently; right now read_ref_full() goes directly to
> the single file that would hold the named ref without scanning and
> populating sibling refs in the in-core loose ref hierarchy, and
> without doing so you cannot return a struct ref_entry.  Hmm...

Yes, that's the crux. Without the ref_cache, there is no persistent ref_entry that can be returned to the caller.

Somewhere I have a prototype patch series (never submitted) that would make it reasonable to access all references via the cache. For single reference accesses, it wouldn't read all of the loose references in the whole directory but rather only create stub REF_INCOMPLETE ref_entries for the refs that are not needed immediately. These entries would be filled when they are accessed. There was also going to be a bulk read command to be used when it is expected that the whole directory will be needed. With this change we really would have a ref_entry for every reference and your suggestion would become plausible. Someday...

But for the current issue it would be totally sufficient to return an additional pointer to the ref_entry *if it is available*, because for packed refs it will always be available.

Actually, for the current patch series I don't see the urgency of changing *anything*. The code works as is and is no worse than the old code. I'd rather do this change (if I get to it) in a separate series.

Michael
-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/
Previous: Junio C HamanoNext: Michael Haggerty
Message 27 of 64 in “Various cleanups around reference packing and peeling”
  1. 00/33 Various cleanups around reference packing and peelingMichael Haggerty, Apr 14, 2013
  2. 01/33 refs: document flags constants REF_*Michael Haggerty, Apr 14, 2013
  3. 02/33 refs: document the fields of struct ref_valueMichael Haggerty, Apr 14, 2013
  4. 03/33 refs: document do_for_each_ref() and do_one_ref()Michael Haggerty, Apr 14, 2013
  5. Junio C HamanoApr 15, 2013
  6. Michael HaggertyApr 16, 2013
  7. 04/33 refs: document how current_ref is usedMichael Haggerty, Apr 14, 2013
  8. 05/33 refs: define constant PEELED_LINE_LENGTHMichael Haggerty, Apr 14, 2013
  9. 06/33 do_for_each_ref_in_dirs(): remove dead codeMichael Haggerty, Apr 14, 2013
  10. 07/33 get_packed_ref(): return a ref_entryMichael Haggerty, Apr 14, 2013
  11. 08/33 peel_ref(): use function get_packed_ref()Michael Haggerty, Apr 14, 2013
  12. 09/33 repack_without_ref(): use function get_packed_ref()Michael Haggerty, Apr 14, 2013
  13. 10/33 refs: extract a function ref_resolves_to_object()Michael Haggerty, Apr 14, 2013
  14. Junio C HamanoApr 15, 2013
  15. Michael HaggertyApr 16, 2013
  16. Junio C HamanoApr 16, 2013
  17. 11/33 refs: extract function peel_object()Michael Haggerty, Apr 14, 2013
  18. 12/33 peel_object(): give more specific information in return valueMichael Haggerty, Apr 14, 2013
  19. Junio C HamanoApr 15, 2013
  20. 13/33 peel_ref(): fix return value for non-peelable, not-current referenceMichael Haggerty, Apr 14, 2013
  21. Junio C HamanoApr 15, 2013
  22. Michael HaggertyApr 16, 2013
  23. 14/33 refs: extract a function peel_entry()Michael Haggerty, Apr 14, 2013
  24. Junio C HamanoApr 15, 2013
  25. Michael HaggertyApr 16, 2013
  26. Junio C HamanoApr 16, 2013
  27. Michael HaggertyApr 16, 2013
  28. 15/33 refs: change the internal reference-iteration APIMichael Haggerty, Apr 14, 2013
  29. Junio C HamanoApr 15, 2013
  30. Michael HaggertyApr 16, 2013
  31. Junio C HamanoApr 16, 2013
  32. 16/33 t3210: test for spurious error messages for dangling packed refsMichael Haggerty, Apr 14, 2013
  33. Junio C HamanoApr 15, 2013
  34. Michael HaggertyApr 16, 2013
  35. Junio C HamanoApr 16, 2013
  36. Jeff KingApr 16, 2013
  37. Junio C HamanoApr 17, 2013
  38. Junio C HamanoApr 17, 2013
  39. 0/2 Add documentation for new expiry option valuesMichael Haggerty, Apr 18, 2013
  40. 1/2 git-gc.txt, git-reflog.txt: document new expiry optionsMichael Haggerty, Apr 18, 2013
  41. 2/2 api-parse-options.txt: document "no-" for non-boolean optionsMichael Haggerty, Apr 18, 2013
  42. prune: introduce OPT_EXPIRY_DATE() and use itJunio C Hamano, Apr 25, 2013
  43. Michael HaggertyApr 17, 2013
  44. 17/33 repack_without_ref(): silence errors for dangling packed refsMichael Haggerty, Apr 14, 2013
  45. Junio C HamanoApr 15, 2013
  46. Michael HaggertyApr 17, 2013
  47. 18/33 search_ref_dir(): return an index rather than a pointerMichael Haggerty, Apr 14, 2013
  48. 19/33 refs: change how packed refs are deletedMichael Haggerty, Apr 14, 2013
  49. 20/33 t3211: demonstrate loss of peeled refs if a packed ref is deletedMichael Haggerty, Apr 14, 2013
  50. 21/33 repack_without_ref(): write peeled refs in the rewritten fileMichael Haggerty, Apr 14, 2013
  51. Junio C HamanoApr 15, 2013
  52. 22/33 refs: extract a function write_packed_entry()Michael Haggerty, Apr 14, 2013
  53. 23/33 pack-refs: rename handle_one_ref() to pack_one_ref()Michael Haggerty, Apr 14, 2013
  54. 24/33 pack-refs: merge code from pack-refs.{c,h} into refs.{c,h}Michael Haggerty, Apr 14, 2013
  55. 25/33 pack_one_ref(): rename "path" parameter to "refname"Michael Haggerty, Apr 14, 2013
  56. 26/33 refs: use same lock_file object for both ref-packing functionsMichael Haggerty, Apr 14, 2013
  57. 27/33 pack_refs(): change to use do_for_each_entry()Michael Haggerty, Apr 14, 2013
  58. 28/33 refs: inline function do_not_prune()Michael Haggerty, Apr 14, 2013
  59. 29/33 pack_one_ref(): use function peel_entry()Michael Haggerty, Apr 14, 2013
  60. 30/33 pack_one_ref(): use write_packed_entry() to do the writingMichael Haggerty, Apr 14, 2013
  61. 31/33 pack_one_ref(): do some cheap tests before a more expensive oneMichael Haggerty, Apr 14, 2013
  62. 32/33 refs: change do_for_each_*() functions to take ref_cache argumentsMichael Haggerty, Apr 14, 2013
  63. Junio C HamanoApr 15, 2013
  64. 33/33 refs: handle the main ref_cache speciallyMichael Haggerty, Apr 14, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.