git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 25/25] cache_ref_iterator_begin(): avoid priming unneeded directories

From
Michael Haggerty <mhagger@alum.mit.edu>
Date
May 24, 2017, 07:17 UTC
Message-ID
<33ffee8b-22da-ced4-95c6-ab68d4557f1d@alum.mit.edu>
In-Reply-To
<20170523194519.luhuej3rerzb57t7@sigill.intra.peff.net>
On 05/23/2017 09:45 PM, Jeff King wrote:
Show 49 quoted lines
> On Mon, May 22, 2017 at 04:17:55PM +0200, Michael Haggerty wrote:
> 
>> So:
>>
>> * Move the responsibility for doing the prefix check directly to
>>   `cache_ref_iterator`. This means that `cache_ref_iterator_begin()`
>>   never has to wrap its return value in a `prefix_ref_iterator`.
>>
>> * Teach `cache_ref_iterator_begin()` (and `prime_ref_dir()`) to be
>>   stricter about what they iterate over and what directories they
>>   prime.
>>
>> * Teach `cache_ref_iterator` to keep track of whether the current
>>   `cache_ref_iterator_level` is fully within the prefix. If so, skip
>>   the prefix checks entirely.
> 
> As promised, I came back to this one with a more careful eye. These
> changes all make sense to me, and the implementation matches.
> 
> My only question is:
> 
>> @@ -511,9 +582,12 @@ struct ref_iterator *cache_ref_iterator_begin(struct ref_cache *cache,
>>  	level->index = -1;
>>  	level->dir = dir;
>>  
>> -	if (prefix && *prefix)
>> -		ref_iterator = prefix_ref_iterator_begin(ref_iterator,
>> -							 prefix, 0);
>> +	if (prefix && *prefix) {
>> +		iter->prefix = xstrdup(prefix);
>> +		level->prefix_state = PREFIX_WITHIN_DIR;
>> +	} else {
>> +		level->prefix_state = PREFIX_CONTAINS_DIR;
>> +	}
> 
> Who frees the prefix? Does this need:
> 
> diff --git a/refs/ref-cache.c b/refs/ref-cache.c
> index fda3942db..a3efc5c51 100644
> --- a/refs/ref-cache.c
> +++ b/refs/ref-cache.c
> @@ -542,6 +542,7 @@ static int cache_ref_iterator_abort(struct ref_iterator *ref_iterator)
>  	struct cache_ref_iterator *iter =
>  		(struct cache_ref_iterator *)ref_iterator;
>  
> +	free(iter->prefix);
>  	free(iter->levels);
>  	base_ref_iterator_free(ref_iterator);
>  	return ITER_DONE;
Yes, you are right. Thanks for catching this.
Note: it has to be
        free((char *)iter->prefix);
because `prefix` is const.

Junio, if a reroll is not needed for other reasons, would you mind squashing this into the last patch of my series?

Michael
Previous: Jeff KingNext: Junio C Hamano
Message 25 of 31 in “Prepare to separate out a packed_ref_store”
  1. 00/25 Prepare to separate out a packed_ref_storeMichael Haggerty, May 22, 2017
  2. 01/25 t3600: clean up permissions test properlyMichael Haggerty, May 22, 2017
  3. 02/25 refs.h: clarify docstring for the ref_transaction_update()-related fnsMichael Haggerty, May 22, 2017
  4. 03/25 ref_iterator_begin_fn(): fix docstringMichael Haggerty, May 22, 2017
  5. 04/25 files-backend: use `die("BUG: ...")`, not `die("internal error: ...")`Michael Haggerty, May 22, 2017
  6. 05/25 prefix_ref_iterator: don't trim too muchMichael Haggerty, May 22, 2017
  7. 06/25 refs_ref_iterator_begin(): don't check prefixes redundantlyMichael Haggerty, May 22, 2017
  8. 07/25 refs: use `size_t` indexes when iterating over ref transaction updatesMichael Haggerty, May 22, 2017
  9. 08/25 ref_store: take a `msg` parameter when deleting referencesMichael Haggerty, May 22, 2017
  10. 10/25 files-backend: move `lock` member to `files_ref_store`Michael Haggerty, May 22, 2017
  11. 12/25 files_transaction_cleanup(): new helper functionMichael Haggerty, May 22, 2017
  12. 11/25 files_ref_store: put the packed files lock directly in this structMichael Haggerty, May 22, 2017
  13. 09/25 lockfile: add a new method, is_lock_file_locked()Michael Haggerty, May 22, 2017
  14. 14/25 ref_transaction_prepare(): new optional step for reference updatesMichael Haggerty, May 22, 2017
  15. 16/25 ref_update_reject_duplicates(): use `size_t` rather than `int`Michael Haggerty, May 22, 2017
  16. 15/25 ref_update_reject_duplicates(): expose function to whole refs moduleMichael Haggerty, May 22, 2017
  17. 17/25 ref_update_reject_duplicates(): add a sanity checkMichael Haggerty, May 22, 2017
  18. 19/25 get_packed_ref_cache(): assume "packed-refs" won't change while lockedMichael Haggerty, May 22, 2017
  19. 20/25 read_packed_refs(): do more of the work of reading packed refsMichael Haggerty, May 22, 2017
  20. 22/25 refs_ref_iterator_begin(): handle `GIT_REF_PARANOIA`Michael Haggerty, May 22, 2017
  21. 23/25 create_ref_entry(): remove `check_name` optionMichael Haggerty, May 22, 2017
  22. 24/25 ref-filter: limit traversal to prefixMichael Haggerty, May 22, 2017
  23. 25/25 cache_ref_iterator_begin(): avoid priming unneeded directoriesMichael Haggerty, May 22, 2017
  24. Jeff KingMay 23, 2017
  25. Michael HaggertyMay 24, 2017
  26. Junio C HamanoMay 24, 2017
  27. 21/25 read_packed_refs(): report unexpected fopen() failuresMichael Haggerty, May 22, 2017
  28. 13/25 ref_transaction_commit(): check for valid `transaction->state`Michael Haggerty, May 22, 2017
  29. 18/25 should_pack_ref(): new function, extracted from `files_pack_refs()`Michael Haggerty, May 22, 2017
  30. Stefan BellerMay 22, 2017
  31. Junio C HamanoMay 24, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.