git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 09/16] packfile: split up responsibilities of `reprepare_packed_git()`

From
Patrick Steinhardt <ps@pks.im>
Date
Sep 2, 2025, 08:50 UTC
Message-ID
<aLav0TRRg6mH4Umw@pks.im>
In-Reply-To
<aK0XeiRz13O7j+HO@nand.local>
On Mon, Aug 25, 2025 at 10:10:02PM -0400, Taylor Blau wrote:
Show 59 quoted lines
> On Thu, Aug 21, 2025 at 09:39:07AM +0200, Patrick Steinhardt wrote:
> > diff --git a/odb.c b/odb.c
> > index 80ec6fc1fa..37ed21f53b 100644
> > --- a/odb.c
> > +++ b/odb.c
> > @@ -694,7 +694,7 @@ static int do_oid_object_info_extended(struct object_database *odb,
> >
> >  		/* Not a loose object; someone else may have just packed it. */
> >  		if (!(flags & OBJECT_INFO_QUICK)) {
> > -			reprepare_packed_git(odb->repo);
> > +			odb_reprepare(odb->repo->objects);
> >  			if (find_pack_entry(odb->repo, real, &e))
> >  				break;
> >  		}
> > @@ -1039,3 +1039,26 @@ void odb_clear(struct object_database *o)
> >
> >  	string_list_clear(&o->submodule_source_paths, 0);
> >  }
> > +
> > +void odb_reprepare(struct object_database *o)
> 
> OK; so here is the new location for the non-packfile related portions of
> the former reprepare_packed_git() function. That makes sense, but...
> 
> > +{
> > +	struct odb_source *source;
> > +
> > +	/*
> > +	 * Reprepare alt odbs, in case the alternates file was modified
> > +	 * during the course of this process. This only _adds_ odbs to
> > +	 * the linked list, so existing odbs will continue to exist for
> > +	 * the lifetime of the process.
> > +	 */
> > +	o->loaded_alternates = 0;
> > +	odb_prepare_alternates(o);
> > +
> > +	for (source = o->sources; source; source = source->next)
> > +		odb_clear_loose_cache(source);
> > +
> > +	o->approximate_object_count_valid = 0;
> > +
> > +	packfile_store_reprepare(o->packfiles);
> > +
> > +	obj_read_unlock();
> 
> ...I think I am missing where we call odb_read_lock(). The function
> packfile_store_reprepare() has a comment that it must be called under
> the odb_read_lock(), but I don't see where we acquire that lock here.
> 
> Are the callers of odb_reprepare() supposed to acquire that lock? If so,
> it seems a little awkward that the caller is supposed to acquire the
> lock, but the callee is the one to release it. Is this function missing
> a odb_read_lock() at the top?
> 
> I looked at a few callers here and none of them seem to be holding this
> lock. pthread_mutex_unlock() is supposed to check that the mutex lock is
> held for recursive and error-checking mutexes. IIRC we initialize the
> the odb mutex with PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP, so I am a
> little surprised that this did not cause a runtime error.

Oh, that's a very good catch. And yes, surprising indeed that this does not cause an error anywhere. But if you look at the code it ultimately isn't all that surprising: there is only a single command in our code base that performs concurrent reads via the ODB, namely git-grep(1). That command explicitly opts into `obj_read_use_lock` by calling `enable_obj_read_lock()`. All the other commands never perform any locking whatsoever.

In any case, I think I merely missed to add the locking call into `odb_reprepare()`. It shouldn't be the responsibility of the caller to care about locking, it should be handled automatically.

Will fix.
Patrick
Previous: Taylor BlauNext: Patrick Steinhardt
Message 70 of 181 in “packfile: carve out a new packfile store”
  1. 00/16 packfile: carve out a new packfile storePatrick Steinhardt, Aug 19, 2025
  2. 01/16 packfile: introduce a new `struct packfile_store`Patrick Steinhardt, Aug 19, 2025
  3. Karthik NayakAug 19, 2025
  4. Patrick SteinhardtAug 20, 2025
  5. Junio C HamanoAug 19, 2025
  6. Patrick SteinhardtAug 20, 2025
  7. 02/16 odb: move list of packfiles into `struct packfile_store`Patrick Steinhardt, Aug 19, 2025
  8. 03/16 odb: move initialization bit into `struct packfile_store`Patrick Steinhardt, Aug 19, 2025
  9. Karthik NayakAug 19, 2025
  10. Junio C HamanoAug 19, 2025
  11. Karthik NayakAug 20, 2025
  12. Junio C HamanoAug 22, 2025
  13. Documentation: note styling for bit fieldsKarthik Nayak, Aug 26, 2025
  14. Patrick SteinhardtAug 20, 2025
  15. Junio C HamanoAug 20, 2025
  16. 04/16 odb: move packfile map into `struct packfile_store`Patrick Steinhardt, Aug 19, 2025
  17. 05/16 odb: move MRU list of packfiles into `struct packfile_store`Patrick Steinhardt, Aug 19, 2025
  18. Karthik NayakAug 20, 2025
  19. Jeff KingAug 20, 2025
  20. Patrick SteinhardtAug 21, 2025
  21. 06/16 odb: move kept cache into `struct packfile_store`Patrick Steinhardt, Aug 19, 2025
  22. Junio C HamanoAug 19, 2025
  23. Patrick SteinhardtAug 20, 2025
  24. 07/16 packfile: reorder functions to avoid function declarationPatrick Steinhardt, Aug 19, 2025
  25. Junio C HamanoAug 19, 2025
  26. 08/16 packfile: refactor `prepare_packed_git()` to work on packfile storePatrick Steinhardt, Aug 19, 2025
  27. 09/16 packfile: split up responsibilities of `reprepare_packed_git()`Patrick Steinhardt, Aug 19, 2025
  28. Karthik NayakAug 20, 2025
  29. 10/16 packfile: refactor `install_packed_git()` to work on packfile storePatrick Steinhardt, Aug 19, 2025
  30. 11/16 packfile: always add packfiles to MRU when adding a packPatrick Steinhardt, Aug 19, 2025
  31. Karthik NayakAug 20, 2025
  32. 12/16 packfile: introduce function to load and add packfilesPatrick Steinhardt, Aug 19, 2025
  33. Karthik NayakAug 20, 2025
  34. Patrick SteinhardtAug 21, 2025
  35. 13/16 packfile: move `get_multi_pack_index()` into "midx.c"Patrick Steinhardt, Aug 19, 2025
  36. 14/16 packfile: remove `get_packed_git()`Patrick Steinhardt, Aug 19, 2025
  37. Karthik NayakAug 20, 2025
  38. Patrick SteinhardtAug 21, 2025
  39. Karthik NayakAug 20, 2025
  40. 15/16 packfile: refactor `get_all_packs()` to work on packfile storePatrick Steinhardt, Aug 19, 2025
  41. Karthik NayakAug 20, 2025
  42. Patrick SteinhardtAug 21, 2025
  43. 16/16 packfile: refactor `get_packed_git_mru()` to work on packfile storePatrick Steinhardt, Aug 19, 2025
  44. Junio C HamanoAug 19, 2025
  45. Karthik NayakAug 20, 2025
  46. 00/16 packfile: carve out a new packfile storePatrick Steinhardt, Aug 21, 2025
  47. 01/16 packfile: introduce a new `struct packfile_store`Patrick Steinhardt, Aug 21, 2025
  48. 02/16 odb: move list of packfiles into `struct packfile_store`Patrick Steinhardt, Aug 21, 2025
  49. Taylor BlauAug 25, 2025
  50. Patrick SteinhardtSep 2, 2025
  51. Taylor BlauSep 2, 2025
  52. Junio C HamanoSep 2, 2025
  53. Patrick SteinhardtSep 3, 2025
  54. Taylor BlauSep 11, 2025
  55. Patrick SteinhardtSep 15, 2025
  56. 03/16 odb: move initialization bit into `struct packfile_store`Patrick Steinhardt, Aug 21, 2025
  57. Taylor BlauAug 26, 2025
  58. 04/16 odb: move packfile map into `struct packfile_store`Patrick Steinhardt, Aug 21, 2025
  59. Taylor BlauAug 26, 2025
  60. 05/16 odb: move MRU list of packfiles into `struct packfile_store`Patrick Steinhardt, Aug 21, 2025
  61. 06/16 odb: move kept cache into `struct packfile_store`Patrick Steinhardt, Aug 21, 2025
  62. Taylor BlauAug 26, 2025
  63. Patrick SteinhardtSep 2, 2025
  64. 07/16 packfile: reorder functions to avoid function declarationPatrick Steinhardt, Aug 21, 2025
  65. Taylor BlauAug 26, 2025
  66. 08/16 packfile: refactor `prepare_packed_git()` to work on packfile storePatrick Steinhardt, Aug 21, 2025
  67. Taylor BlauAug 26, 2025
  68. 09/16 packfile: split up responsibilities of `reprepare_packed_git()`Patrick Steinhardt, Aug 21, 2025
  69. Taylor BlauAug 26, 2025
  70. Patrick SteinhardtSep 2, 2025
  71. 10/16 packfile: refactor `install_packed_git()` to work on packfile storePatrick Steinhardt, Aug 21, 2025
  72. Taylor BlauAug 26, 2025
  73. Patrick SteinhardtSep 2, 2025
  74. 11/16 packfile: always add packfiles to MRU when adding a packPatrick Steinhardt, Aug 21, 2025
  75. Taylor BlauAug 27, 2025
  76. Patrick SteinhardtSep 2, 2025
  77. 12/16 packfile: introduce function to load and add packfilesPatrick Steinhardt, Aug 21, 2025
  78. Taylor BlauAug 27, 2025
  79. 13/16 packfile: move `get_multi_pack_index()` into "midx.c"Patrick Steinhardt, Aug 21, 2025
  80. Taylor BlauAug 27, 2025
  81. 14/16 packfile: remove `get_packed_git()`Patrick Steinhardt, Aug 21, 2025
  82. Taylor BlauAug 27, 2025
  83. Patrick SteinhardtSep 2, 2025
  84. Taylor BlauSep 11, 2025
  85. Patrick SteinhardtSep 15, 2025
  86. 15/16 packfile: refactor `get_all_packs()` to work on packfile storePatrick Steinhardt, Aug 21, 2025
  87. Taylor BlauAug 27, 2025
  88. Patrick SteinhardtSep 2, 2025
  89. Taylor BlauSep 11, 2025
  90. Patrick SteinhardtSep 15, 2025
  91. 16/16 packfile: refactor `get_packed_git_mru()` to work on packfile storePatrick Steinhardt, Aug 21, 2025
  92. 00/15 packfile: carve out a new packfile storePatrick Steinhardt, Sep 2, 2025
  93. 01/15 packfile: introduce a new `struct packfile_store`Patrick Steinhardt, Sep 2, 2025
  94. Karthik NayakSep 9, 2025
  95. 02/15 odb: move list of packfiles into `struct packfile_store`Patrick Steinhardt, Sep 2, 2025
  96. Karthik NayakSep 9, 2025
  97. Patrick SteinhardtSep 9, 2025
  98. 03/15 odb: move initialization bit into `struct packfile_store`Patrick Steinhardt, Sep 2, 2025
  99. 04/15 odb: move packfile map into `struct packfile_store`Patrick Steinhardt, Sep 2, 2025
  100. Karthik NayakSep 9, 2025
  101. Patrick SteinhardtSep 9, 2025
  102. 05/15 odb: move MRU list of packfiles into `struct packfile_store`Patrick Steinhardt, Sep 2, 2025
  103. 06/15 odb: move kept cache into `struct packfile_store`Patrick Steinhardt, Sep 2, 2025
  104. 07/15 packfile: reorder functions to avoid function declarationPatrick Steinhardt, Sep 2, 2025
  105. 08/15 packfile: refactor `prepare_packed_git()` to work on packfile storePatrick Steinhardt, Sep 2, 2025
  106. 09/15 packfile: split up responsibilities of `reprepare_packed_git()`Patrick Steinhardt, Sep 2, 2025
  107. 10/15 packfile: refactor `install_packed_git()` to work on packfile storePatrick Steinhardt, Sep 2, 2025
  108. 11/15 packfile: introduce function to load and add packfilesPatrick Steinhardt, Sep 2, 2025
  109. 12/15 packfile: move `get_multi_pack_index()` into "midx.c"Patrick Steinhardt, Sep 2, 2025
  110. 13/15 packfile: remove `get_packed_git()`Patrick Steinhardt, Sep 2, 2025
  111. 14/15 packfile: refactor `get_all_packs()` to work on packfile storePatrick Steinhardt, Sep 2, 2025
  112. 15/15 packfile: refactor `get_packed_git_mru()` to work on packfile storePatrick Steinhardt, Sep 2, 2025
  113. Junio C HamanoSep 2, 2025
  114. Taylor BlauSep 11, 2025
  115. Karthik NayakSep 9, 2025
  116. 00/15 packfile: carve out a new packfile storePatrick Steinhardt, Sep 9, 2025
  117. 01/15 packfile: introduce a new `struct packfile_store`Patrick Steinhardt, Sep 9, 2025
  118. 02/15 odb: move list of packfiles into `struct packfile_store`Patrick Steinhardt, Sep 9, 2025
  119. 03/15 odb: move initialization bit into `struct packfile_store`Patrick Steinhardt, Sep 9, 2025
  120. 04/15 odb: move packfile map into `struct packfile_store`Patrick Steinhardt, Sep 9, 2025
  121. 05/15 odb: move MRU list of packfiles into `struct packfile_store`Patrick Steinhardt, Sep 9, 2025
  122. 06/15 odb: move kept cache into `struct packfile_store`Patrick Steinhardt, Sep 9, 2025
  123. 07/15 packfile: reorder functions to avoid function declarationPatrick Steinhardt, Sep 9, 2025
  124. 08/15 packfile: refactor `prepare_packed_git()` to work on packfile storePatrick Steinhardt, Sep 9, 2025
  125. 09/15 packfile: split up responsibilities of `reprepare_packed_git()`Patrick Steinhardt, Sep 9, 2025
  126. 10/15 packfile: refactor `install_packed_git()` to work on packfile storePatrick Steinhardt, Sep 9, 2025
  127. 11/15 packfile: introduce function to load and add packfilesPatrick Steinhardt, Sep 9, 2025
  128. 12/15 packfile: move `get_multi_pack_index()` into "midx.c"Patrick Steinhardt, Sep 9, 2025
  129. 13/15 packfile: remove `get_packed_git()`Patrick Steinhardt, Sep 9, 2025
  130. Taylor BlauSep 11, 2025
  131. 14/15 packfile: refactor `get_all_packs()` to work on packfile storePatrick Steinhardt, Sep 9, 2025
  132. 15/15 packfile: refactor `get_packed_git_mru()` to work on packfile storePatrick Steinhardt, Sep 9, 2025
  133. Karthik NayakSep 10, 2025
  134. Taylor BlauSep 11, 2025
  135. Taylor BlauSep 11, 2025
  136. Patrick SteinhardtSep 15, 2025
  137. 00/15 packfile: carve out a new packfile storePatrick Steinhardt, Sep 15, 2025
  138. 01/15 packfile: introduce a new `struct packfile_store`Patrick Steinhardt, Sep 15, 2025
  139. Justin ToblerSep 17, 2025
  140. Patrick SteinhardtSep 23, 2025
  141. Justin ToblerSep 24, 2025
  142. 02/15 odb: move list of packfiles into `struct packfile_store`Patrick Steinhardt, Sep 15, 2025
  143. 03/15 odb: move initialization bit into `struct packfile_store`Patrick Steinhardt, Sep 15, 2025
  144. 04/15 odb: move packfile map into `struct packfile_store`Patrick Steinhardt, Sep 15, 2025
  145. Justin ToblerSep 17, 2025
  146. Patrick SteinhardtSep 23, 2025
  147. 05/15 odb: move MRU list of packfiles into `struct packfile_store`Patrick Steinhardt, Sep 15, 2025
  148. Justin ToblerSep 17, 2025
  149. 06/15 odb: move kept cache into `struct packfile_store`Patrick Steinhardt, Sep 15, 2025
  150. 07/15 packfile: reorder functions to avoid function declarationPatrick Steinhardt, Sep 15, 2025
  151. 08/15 packfile: refactor `prepare_packed_git()` to work on packfile storePatrick Steinhardt, Sep 15, 2025
  152. 09/15 packfile: split up responsibilities of `reprepare_packed_git()`Patrick Steinhardt, Sep 15, 2025
  153. Justin ToblerSep 17, 2025
  154. Patrick SteinhardtSep 23, 2025
  155. 10/15 packfile: refactor `install_packed_git()` to work on packfile storePatrick Steinhardt, Sep 15, 2025
  156. 11/15 packfile: introduce function to load and add packfilesPatrick Steinhardt, Sep 15, 2025
  157. 12/15 packfile: move `get_multi_pack_index()` into "midx.c"Patrick Steinhardt, Sep 15, 2025
  158. 13/15 packfile: refactor `get_packed_git()` to work on packfile storePatrick Steinhardt, Sep 15, 2025
  159. 14/15 packfile: refactor `get_all_packs()` to work on packfile storePatrick Steinhardt, Sep 15, 2025
  160. 15/15 packfile: refactor `get_packed_git_mru()` to work on packfile storePatrick Steinhardt, Sep 15, 2025
  161. 00/15 packfile: carve out a new packfile storePatrick Steinhardt, Sep 23, 2025
  162. 01/15 packfile: introduce a new `struct packfile_store`Patrick Steinhardt, Sep 23, 2025
  163. 02/15 odb: move list of packfiles into `struct packfile_store`Patrick Steinhardt, Sep 23, 2025
  164. 03/15 odb: move initialization bit into `struct packfile_store`Patrick Steinhardt, Sep 23, 2025
  165. 04/15 odb: move packfile map into `struct packfile_store`Patrick Steinhardt, Sep 23, 2025
  166. 05/15 odb: move MRU list of packfiles into `struct packfile_store`Patrick Steinhardt, Sep 23, 2025
  167. 06/15 odb: move kept cache into `struct packfile_store`Patrick Steinhardt, Sep 23, 2025
  168. 07/15 packfile: reorder functions to avoid function declarationPatrick Steinhardt, Sep 23, 2025
  169. 08/15 packfile: refactor `prepare_packed_git()` to work on packfile storePatrick Steinhardt, Sep 23, 2025
  170. 09/15 packfile: split up responsibilities of `reprepare_packed_git()`Patrick Steinhardt, Sep 23, 2025
  171. 10/15 packfile: refactor `install_packed_git()` to work on packfile storePatrick Steinhardt, Sep 23, 2025
  172. 11/15 packfile: introduce function to load and add packfilesPatrick Steinhardt, Sep 23, 2025
  173. 12/15 packfile: move `get_multi_pack_index()` into "midx.c"Patrick Steinhardt, Sep 23, 2025
  174. 13/15 packfile: refactor `get_packed_git()` to work on packfile storePatrick Steinhardt, Sep 23, 2025
  175. 14/15 packfile: refactor `get_all_packs()` to work on packfile storePatrick Steinhardt, Sep 23, 2025
  176. 15/15 packfile: refactor `get_packed_git_mru()` to work on packfile storePatrick Steinhardt, Sep 23, 2025
  177. Justin ToblerSep 24, 2025
  178. Junio C HamanoSep 25, 2025
  179. Patrick SteinhardtSep 26, 2025
  180. Taylor BlauSep 28, 2025
  181. Patrick SteinhardtSep 29, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.