git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 04/23] prefix_ref_iterator: don't trim too much

From
Michael Haggerty <mhagger@alum.mit.edu>
Date
May 17, 2017, 14:11 UTC
Message-ID
<7cec0285-fe6b-0e7e-83b4-b93c9770b55b@alum.mit.edu>
In-Reply-To
<20170517125510.qmy2qvyxuup25zw6@sigill.intra.peff.net>
On 05/17/2017 02:55 PM, Jeff King wrote:
Show 39 quoted lines
> On Wed, May 17, 2017 at 02:05:27PM +0200, Michael Haggerty wrote:
> 
>> diff --git a/refs/iterator.c b/refs/iterator.c
>> index bce1f192f7..f33d1b3a39 100644
>> --- a/refs/iterator.c
>> +++ b/refs/iterator.c
>> @@ -292,7 +292,19 @@ static int prefix_ref_iterator_advance(struct ref_iterator *ref_iterator)
>>  		if (!starts_with(iter->iter0->refname, iter->prefix))
>>  			continue;
>>  
>> -		iter->base.refname = iter->iter0->refname + iter->trim;
>> +		if (iter->trim) {
>> +			/*
>> +			 * If there wouldn't be at least one character
>> +			 * left in the refname after trimming, skip
>> +			 * over it:
>> +			 */
>> +			if (memchr(iter->iter0->refname, '\0', iter->trim + 1))
>> +				continue;
> 
> It took me a minute to figure out the logic here. You're looking for the
> end-of-string within the trim boundary, which would be an indication
> that the string itself is smaller than the boundary.
> 
> But what if it returns true, and the string really is shorter than the
> trim size? That would mean we pass a size to memchr that is longer than
> the buffer we pass. Is that legal?
> 
> I suspect it's undefined behavior according to the standard, though I'd
> guess in practice it would be fine. But if I'm understanding it
> correctly, this is the same check as:
> 
>   if (strlen(iter->iter0->refname) <= iter->trim)
> 
> which seems a lot more obvious to me and doesn't fall afoul of weird
> standard issues. The only downside I see is that it would read to the
> end of string when yours could stop at iter->trim bytes. I have no idea
> if that would be measurable (it might even be faster because strlen()
> only has one condition to loop on).

You are correct that I chose `memchr()` over `strlen()` to avoid scanning a POTENTIALLY EXTREMELY LARGE NUMBER OF CHARACTERS past the trim length, but of course in real life refnames aren't that long and `strlen()` might actually be faster.

I *think* `memchr()` is technically OK:
> Implementations shall behave as if they read the memory byte by byte

from the beginning of the bytes pointed to by s and stop at the first occurrence of c (if it is found in the initial n bytes).

But I agree that the `strlen()` version is also easier to read (I actually had that version first). So I'll change it as you have suggested.

Thanks, Michael

Previous: Jeff KingNext: Jeff King
Message 20 of 73 in “Prepare to separate out a packed_ref_store”
  1. 00/23 Prepare to separate out a packed_ref_storeMichael Haggerty, May 17, 2017
  2. 01/23 t3600: clean up permissions test properlyMichael Haggerty, May 17, 2017
  3. Jeff KingMay 17, 2017
  4. Michael HaggertyMay 17, 2017
  5. Junio C HamanoMay 18, 2017
  6. Michael HaggertyMay 19, 2017
  7. 02/23 refs.h: clarify docstring for the ref_transaction_update()-related fnsMichael Haggerty, May 17, 2017
  8. Stefan BellerMay 17, 2017
  9. Junio C HamanoMay 18, 2017
  10. 03/23 ref_iterator_begin_fn(): fix docstringMichael Haggerty, May 17, 2017
  11. 05/23 refs_ref_iterator_begin(): don't check prefixes redundantlyMichael Haggerty, May 17, 2017
  12. Jeff KingMay 17, 2017
  13. Michael HaggertyMay 17, 2017
  14. 07/23 ref_store: take `logmsg` parameter when deleting referencesMichael Haggerty, May 17, 2017
  15. Jeff KingMay 17, 2017
  16. Michael HaggertyMay 17, 2017
  17. Jeff KingMay 17, 2017
  18. 04/23 prefix_ref_iterator: don't trim too muchMichael Haggerty, May 17, 2017
  19. Jeff KingMay 17, 2017
  20. Michael HaggertyMay 17, 2017
  21. Jeff KingMay 17, 2017
  22. Junio C HamanoMay 18, 2017
  23. Michael HaggertyMay 18, 2017
  24. 09/23 files-backend: move `lock` member to `files_ref_store`Michael Haggerty, May 17, 2017
  25. Jeff KingMay 17, 2017
  26. Michael HaggertyMay 17, 2017
  27. 06/23 refs: use `size_t` indexes when iterating over ref transaction updatesMichael Haggerty, May 17, 2017
  28. Stefan BellerMay 17, 2017
  29. Michael HaggertyMay 18, 2017
  30. 10/23 files_ref_store: put the packed files lock directly in this structMichael Haggerty, May 17, 2017
  31. Jeff KingMay 17, 2017
  32. Michael HaggertyMay 17, 2017
  33. Stefan BellerMay 17, 2017
  34. Brandon WilliamsMay 18, 2017
  35. Michael HaggertyMay 19, 2017
  36. Brandon WilliamsMay 18, 2017
  37. Jeff KingMay 18, 2017
  38. Brandon WilliamsMay 18, 2017
  39. 12/23 ref_transaction_commit(): break into multiple functionsMichael Haggerty, May 17, 2017
  40. Stefan BellerMay 17, 2017
  41. Michael HaggertyMay 19, 2017
  42. 16/23 should_pack_ref(): new function, extracted from `files_pack_refs()`Michael Haggerty, May 17, 2017
  43. 17/23 get_packed_ref_cache(): assume "packed-refs" won't change while lockedMichael Haggerty, May 17, 2017
  44. Stefan BellerMay 17, 2017
  45. Jeff KingMay 18, 2017
  46. Stefan BellerMay 18, 2017
  47. 18/23 read_packed_refs(): do more of the work of reading packed refsMichael Haggerty, May 17, 2017
  48. 19/23 read_packed_refs(): report unexpected fopen() failuresMichael Haggerty, May 17, 2017
  49. Jeff KingMay 17, 2017
  50. Michael HaggertyMay 17, 2017
  51. Junio C HamanoMay 18, 2017
  52. Jeff KingMay 18, 2017
  53. 15/23 ref_update_reject_duplicates(): add a sanity checkMichael Haggerty, May 17, 2017
  54. 20/23 refs_ref_iterator_begin(): handle `GIT_REF_PARANOIA`Michael Haggerty, May 17, 2017
  55. Jeff KingMay 17, 2017
  56. 22/23 ref-filter: limit traversal to prefixMichael Haggerty, May 17, 2017
  57. Jeff KingMay 17, 2017
  58. Michael HaggertyMay 19, 2017
  59. 14/23 ref_update_reject_duplicates(): use `size_t` rather than `int`Michael Haggerty, May 17, 2017
  60. 23/23 cache_ref_iterator_begin(): avoid priming unneeded directoriesMichael Haggerty, May 17, 2017
  61. 21/23 create_ref_entry(): remove `check_name` optionMichael Haggerty, May 17, 2017
  62. 08/23 lockfile: add a new method, is_lock_file_locked()Michael Haggerty, May 17, 2017
  63. Jeff KingMay 17, 2017
  64. 11/23 files_transaction_cleanup(): new helper functionMichael Haggerty, May 17, 2017
  65. Jeff KingMay 17, 2017
  66. Michael HaggertyMay 19, 2017
  67. Stefan BellerMay 17, 2017
  68. Michael HaggertyMay 19, 2017
  69. 13/23 ref_update_reject_duplicates(): expose function to whole refs moduleMichael Haggerty, May 17, 2017
  70. Jeff KingMay 17, 2017
  71. Stefan BellerMay 17, 2017
  72. Johannes SixtMay 18, 2017
  73. Jeff KingMay 18, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.