git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 6/6] Retain caches of submodule refs

From
Michael Haggerty <mhagger@alum.mit.edu>
Date
Aug 24, 2011, 11:33 UTC
Message-ID
<4E54E188.10802@alum.mit.edu>
In-Reply-To
<7v4o1hgemp.fsf@alter.siamese.dyndns.org>
On 08/17/2011 12:45 AM, Junio C Hamano wrote:
> All the changes except for this one made sense to me, but I am not sure
> about this one. How often do we look into different submodule refs in the
> same process over and over again?

As I've mentioned, I am not very familiar with submodules and I don't know what actions in submodules can be triggered from the top-level project. Here is the only code path that I can find in the current git code that causes submodule refs to be read at all:

setup_revisions() with opt->submodule is set,
    which can only happen when called via merge_submodule(),
    which is called from merge_file() in merge-recursive.c,
    which is called by process_renames() and merge_content(),
    which are both ultimately called from merge_trees().
I don't know how often this can happen during the lifetime of one process.

The cost of retaining the submodule ref caches is roughly 100 bytes per reference of memory. Another side effect is that the submodule caches never have to be cleaned up; this saves a list walk and one free() per cached reference if the refs for more than one submodule are accessed.

Michael
-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/
Previous: Junio C HamanoNext: Michael Haggerty
Message 15 of 54 in “Retain caches of submodule refs”
  1. 0/6 Retain caches of submodule refsMichael Haggerty, Aug 12, 2011
  2. 1/6 Extract a function clear_cached_refs()Michael Haggerty, Aug 12, 2011
  3. 2/6 Access reference caches only through new function get_cached_refs().Michael Haggerty, Aug 12, 2011
  4. Junio C HamanoAug 14, 2011
  5. Michael HaggertyAug 23, 2011
  6. 3/6 Change the signature of read_packed_refs()Michael Haggerty, Aug 12, 2011
  7. 4/6 Allocate cached_refs objects dynamicallyMichael Haggerty, Aug 12, 2011
  8. Junio C HamanoAug 14, 2011
  9. 5/6 Store the submodule name in struct cached_refs.Michael Haggerty, Aug 12, 2011
  10. 6/6 Retain caches of submodule refsMichael Haggerty, Aug 12, 2011
  11. Heiko VoigtAug 13, 2011
  12. Michael HaggertyAug 24, 2011
  13. Heiko VoigtAug 24, 2011
  14. Junio C HamanoAug 16, 2011
  15. Michael HaggertyAug 24, 2011
  16. Michael HaggertyOct 9, 2011
  17. Junio C HamanoOct 9, 2011
  18. 0/2 Provide API to invalidate refs cacheMichael Haggerty, Oct 10, 2011
  19. 1/2 invalidate_cached_refs(): take the submodule as parameterMichael Haggerty, Oct 10, 2011
  20. 2/2 invalidate_cached_refs(): expose this function in refs APIMichael Haggerty, Oct 10, 2011
  21. 0/7 Provide API to invalidate refs cacheMichael Haggerty, Oct 10, 2011
  22. 1/7 invalidate_ref_cache(): rename function from invalidate_cached_refs()Michael Haggerty, Oct 10, 2011
  23. Junio C HamanoOct 11, 2011
  24. Michael HaggertyOct 11, 2011
  25. 2/7 invalidate_ref_cache(): take the submodule as parameterMichael Haggerty, Oct 10, 2011
  26. 3/7 invalidate_ref_cache(): expose this function in refs APIMichael Haggerty, Oct 10, 2011
  27. 4/7 clear_cached_refs(): rename parameterMichael Haggerty, Oct 10, 2011
  28. 5/7 clear_cached_refs(): extract two new functionsMichael Haggerty, Oct 10, 2011
  29. 6/7 write_ref_sha1(): only invalidate the loose ref cacheMichael Haggerty, Oct 10, 2011
  30. 7/7 clear_cached_refs(): inline functionMichael Haggerty, Oct 10, 2011
  31. Junio C HamanoOct 11, 2011
  32. Michael HaggertyOct 11, 2011
  33. Julian PhillipsOct 11, 2011
  34. Junio C HamanoOct 11, 2011
  35. 0/7 Provide API to invalidate refs cacheMichael Haggerty, Oct 12, 2011
  36. 1/7 invalidate_ref_cache(): rename function from invalidate_cached_refs()Michael Haggerty, Oct 12, 2011
  37. Junio C HamanoOct 12, 2011
  38. Michael HaggertyOct 12, 2011
  39. 2/7 invalidate_ref_cache(): take the submodule as parameterMichael Haggerty, Oct 12, 2011
  40. Junio C HamanoOct 12, 2011
  41. Michael HaggertyOct 12, 2011
  42. Junio C HamanoOct 17, 2011
  43. Michael HaggertyNov 3, 2011
  44. Junio C HamanoNov 3, 2011
  45. 3/7 invalidate_ref_cache(): expose this function in refs APIMichael Haggerty, Oct 12, 2011
  46. 4/7 clear_cached_refs(): rename parameterMichael Haggerty, Oct 12, 2011
  47. 5/7 clear_cached_refs(): extract two new functionsMichael Haggerty, Oct 12, 2011
  48. 6/7 write_ref_sha1(): only invalidate the loose ref cacheMichael Haggerty, Oct 12, 2011
  49. 7/7 clear_cached_refs(): inline functionMichael Haggerty, Oct 12, 2011
  50. Junio C HamanoOct 12, 2011
  51. Heiko VoigtOct 10, 2011
  52. Michael HaggertyOct 11, 2011
  53. Heiko VoigtOct 11, 2011
  54. Heiko VoigtAug 13, 2011

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.