git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 9/9] fetch-pack: drop custom loose object cache

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Nov 12, 2018, 19:32 UTC
Message-ID
<87efbq2isk.fsf@evledraar.gmail.com>
In-Reply-To
<a08d8e20-ecb5-3135-f294-56dec1730b5f@web.de>
On Mon, Nov 12 2018, René Scharfe wrote:
Show 85 quoted lines
> Am 12.11.2018 um 15:55 schrieb Jeff King:
>> Commit 024aa4696c (fetch-pack.c: use oidset to check existence of loose
>> object, 2018-03-14) added a cache to avoid calling stat() for a bunch of
>> loose objects we don't have.
>>
>> Now that OBJECT_INFO_QUICK handles this caching itself, we can drop the
>> custom solution.
>>
>> Note that this might perform slightly differently, as the original code
>> stopped calling readdir() when we saw more loose objects than there were
>> refs. So:
>>
>>   1. The old code might have spent work on readdir() to fill the cache,
>>      but then decided there were too many loose objects, wasting that
>>      effort.
>>
>>   2. The new code might spend a lot of time on readdir() if you have a
>>      lot of loose objects, even though there are very few objects to
>>      ask about.
>
> Plus the old code used an oidset while the new one uses an oid_array.
>
>> In practice it probably won't matter either way; see the previous commit
>> for some discussion of the tradeoff.
>>
>> Signed-off-by: Jeff King <peff@peff.net>
>> ---
>>  fetch-pack.c | 39 ++-------------------------------------
>>  1 file changed, 2 insertions(+), 37 deletions(-)
>>
>> diff --git a/fetch-pack.c b/fetch-pack.c
>> index b3ed7121bc..25a88f4eb2 100644
>> --- a/fetch-pack.c
>> +++ b/fetch-pack.c
>> @@ -636,23 +636,6 @@ struct loose_object_iter {
>>  	struct ref *refs;
>>  };
>>
>> -/*
>> - *  If the number of refs is not larger than the number of loose objects,
>> - *  this function stops inserting.
>> - */
>> -static int add_loose_objects_to_set(const struct object_id *oid,
>> -				    const char *path,
>> -				    void *data)
>> -{
>> -	struct loose_object_iter *iter = data;
>> -	oidset_insert(iter->loose_object_set, oid);
>> -	if (iter->refs == NULL)
>> -		return 1;
>> -
>> -	iter->refs = iter->refs->next;
>> -	return 0;
>> -}
>> -
>>  /*
>>   * Mark recent commits available locally and reachable from a local ref as
>>   * COMPLETE. If args->no_dependents is false, also mark COMPLETE remote refs as
>> @@ -670,30 +653,14 @@ static void mark_complete_and_common_ref(struct fetch_negotiator *negotiator,
>>  	struct ref *ref;
>>  	int old_save_commit_buffer = save_commit_buffer;
>>  	timestamp_t cutoff = 0;
>> -	struct oidset loose_oid_set = OIDSET_INIT;
>> -	int use_oidset = 0;
>> -	struct loose_object_iter iter = {&loose_oid_set, *refs};
>> -
>> -	/* Enumerate all loose objects or know refs are not so many. */
>> -	use_oidset = !for_each_loose_object(add_loose_objects_to_set,
>> -					    &iter, 0);
>>
>>  	save_commit_buffer = 0;
>>
>>  	for (ref = *refs; ref; ref = ref->next) {
>>  		struct object *o;
>> -		unsigned int flags = OBJECT_INFO_QUICK;
>>
>> -		if (use_oidset &&
>> -		    !oidset_contains(&loose_oid_set, &ref->old_oid)) {
>> -			/*
>> -			 * I know this does not exist in the loose form,
>> -			 * so check if it exists in a non-loose form.
>> -			 */
>> -			flags |= OBJECT_INFO_IGNORE_LOOSE;
>
> This removes the only user of OBJECT_INFO_IGNORE_LOOSE.  #leftoverbits

With this series applied there's still a use of it left in oid_object_info_extended()

Previous: René ScharfeNext: Jeff King
Message 79 of 99 in “index-pack: improve performance on NFS”
  1. index-pack: improve performance on NFSJansen, Geert, Oct 25, 2018
  2. Junio C HamanoOct 26, 2018
  3. Ævar Arnfjörð BjarmasonOct 26, 2018
  4. Junio C HamanoOct 27, 2018
  5. Jeff KingOct 27, 2018
  6. Ævar Arnfjörð BjarmasonOct 27, 2018
  7. 0/4 index-pack: optionally turn off SHA-1 collision checkingÆvar Arnfjörð Bjarmason, Oct 28, 2018
  8. Geert JansenOct 30, 2018
  9. Junio C HamanoOct 30, 2018
  10. 0/3 index-pack: test updatesÆvar Arnfjörð Bjarmason, Oct 30, 2018
  11. index-pack: add ability to disable SHA-1 collision checkÆvar Arnfjörð Bjarmason, Nov 13, 2018
  12. Junio C HamanoNov 14, 2018
  13. Ævar Arnfjörð BjarmasonNov 14, 2018
  14. 1/3 pack-objects test: modernize styleÆvar Arnfjörð Bjarmason, Oct 30, 2018
  15. 2/3 pack-objects tests: don't leave test .git corrupt at endÆvar Arnfjörð Bjarmason, Oct 30, 2018
  16. 3/3 index-pack tests: don't leave test repo dirty at endÆvar Arnfjörð Bjarmason, Oct 30, 2018
  17. 1/4 pack-objects test: modernize styleÆvar Arnfjörð Bjarmason, Oct 28, 2018
  18. 2/4 pack-objects tests: don't leave test .git corrupt at endÆvar Arnfjörð Bjarmason, Oct 28, 2018
  19. 3/4 index-pack tests: don't leave test repo dirty at endÆvar Arnfjörð Bjarmason, Oct 28, 2018
  20. 4/4 index-pack: add ability to disable SHA-1 collision checkÆvar Arnfjörð Bjarmason, Oct 28, 2018
  21. Jeff KingOct 29, 2018
  22. Jeff KingOct 29, 2018
  23. Ævar Arnfjörð BjarmasonOct 29, 2018
  24. Jeff KingOct 29, 2018
  25. Geert JansenNov 7, 2018
  26. Jeff KingNov 8, 2018
  27. Geert JansenNov 8, 2018
  28. Jeff KingNov 8, 2018
  29. Geert JansenNov 8, 2018
  30. Ævar Arnfjörð BjarmasonNov 8, 2018
  31. Ævar Arnfjörð BjarmasonNov 9, 2018
  32. Jeff KingNov 12, 2018
  33. 0/9 caching loose objectsJeff King, Nov 12, 2018
  34. 1/9 fsck: do not reuse child_process structsJeff King, Nov 12, 2018
  35. Derrick StoleeNov 12, 2018
  36. 2/9 submodule--helper: prefer strip_suffix() to ends_with()Jeff King, Nov 12, 2018
  37. Stefan BellerNov 12, 2018
  38. 3/9 rename "alternate_object_database" to "object_directory"Jeff King, Nov 12, 2018
  39. Derrick StoleeNov 12, 2018
  40. Jeff KingNov 12, 2018
  41. Ramsay JonesNov 12, 2018
  42. 4/9 sha1_file_name(): overwrite buffer instead of appendingJeff King, Nov 12, 2018
  43. Derrick StoleeNov 12, 2018
  44. 5/9 handle alternates paths the same as the main object dirJeff King, Nov 12, 2018
  45. Derrick StoleeNov 12, 2018
  46. Jeff KingNov 12, 2018
  47. Derrick StoleeNov 12, 2018
  48. 6/9 sha1-file: use an object_directory for the main object dirJeff King, Nov 12, 2018
  49. Derrick StoleeNov 12, 2018
  50. Jeff KingNov 12, 2018
  51. Stefan BellerNov 12, 2018
  52. Jeff KingNov 22, 2018
  53. Stefan BellerNov 12, 2018
  54. 7/9 object-store: provide helpers for loose_objects_cacheJeff King, Nov 12, 2018
  55. René ScharfeNov 12, 2018
  56. Jeff KingNov 12, 2018
  57. 8/9 sha1-file: use loose object cache for quick existence checkJeff King, Nov 12, 2018
  58. Derrick StoleeNov 12, 2018
  59. Ævar Arnfjörð BjarmasonNov 12, 2018
  60. Jeff KingNov 12, 2018
  61. Ævar Arnfjörð BjarmasonNov 12, 2018
  62. Ævar Arnfjörð BjarmasonNov 12, 2018
  63. Ævar Arnfjörð BjarmasonNov 13, 2018
  64. René ScharfeNov 14, 2018
  65. René ScharfeDec 2, 2018
  66. Jeff KingDec 3, 2018
  67. René ScharfeDec 4, 2018
  68. Jeff KingDec 5, 2018
  69. René ScharfeDec 5, 2018
  70. Jeff KingDec 5, 2018
  71. Jeff KingDec 5, 2018
  72. René ScharfeDec 5, 2018
  73. Jeff KingDec 5, 2018
  74. Geert JansenNov 12, 2018
  75. René ScharfeNov 27, 2018
  76. Jeff KingDec 1, 2018
  77. 9/9 fetch-pack: drop custom loose object cacheJeff King, Nov 12, 2018
  78. René ScharfeNov 12, 2018
  79. Ævar Arnfjörð BjarmasonNov 12, 2018
  80. Jeff KingNov 12, 2018
  81. René ScharfeNov 12, 2018
  82. Derrick StoleeNov 12, 2018
  83. Stefan BellerNov 12, 2018
  84. Ævar Arnfjörð BjarmasonNov 9, 2018
  85. Duy NguyenNov 9, 2018
  86. Ævar Arnfjörð BjarmasonNov 10, 2018
  87. Jeff KingNov 12, 2018
  88. Geert JansenNov 12, 2018
  89. Duy NguyenOct 27, 2018
  90. Jeff KingOct 29, 2018
  91. Junio C HamanoOct 29, 2018
  92. Jeff KingOct 29, 2018
  93. Ævar Arnfjörð BjarmasonOct 29, 2018
  94. Geert JansenOct 29, 2018
  95. Jeff KingOct 29, 2018
  96. Geert JansenOct 29, 2018
  97. Jeff KingOct 29, 2018
  98. Stefan BellerOct 29, 2018
  99. Jeff KingOct 29, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.