git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 3/8] submodule: make static functions read submodules from commits

From
Glen Choo <chooglen@google.com>
Date
Feb 11, 2022, 10:07 UTC
Message-ID
<kl6l7da1yawq.fsf@chooglen-macbookpro.roam.corp.google.com>
In-Reply-To
<20220210191533.659570-1-jonathantanmy@google.com>

Hm, after reading your feedback, I'm starting to question whether this patch makes sense in its current form.

Jonathan Tan <jonathantanmy@google.com> writes:
Show 6 quoted lines
> Glen Choo <chooglen@google.com> writes:
>> As a result, "git fetch" now reads changed submodules using the
>> `.gitmodules` and path from super_oid's tree (which is where "git fetch"
>> actually noticed the changed submodule) instead of the filesystem.
>
> Could we have a test showing what has changed?

Looking at this closer, I don't think a test is feasible, but even more importantly, I don't think this behavior change is even desirable at all..

I was confused when I wrote the commit message. Prior to this patch, "git fetch" already records the names of changed submodules by correctly reading .gitmodules from the given commit. From collect_changed_submodules_cb():

		submodule = submodule_from_path(me->repo,
						commit_oid, p->two->path);
    [...]
		item = string_list_insert(changed, name);
		if (!item->util)
			item->util = xcalloc(1, sizeof(struct changed_submodule_data));
		cs_data = item->util;
		oid_array_append(&cs_data->new_commits, &p->two->oid);

The only behavior that actually _does_ change is that we plumb super_oid through submodule_has_commits(). "git fetch" invokes this function to figure out if it already has all of the needed commits, and if so, skip the fetch.

Before plumbing super_oid, we could not check for commits in an unpopulated submodule, but now we do. We will need this when we fetch in unpopulated submodules, but as of this patch, we never fetch in unpopulated submodules anyway, so this check is just wasted effort.

And because we never fetch anyway, we can't test any meaningful behavior. We could check whether or not we check the submodule odb, but that's a lot of effort to spend on something we don't need.

So we probably don't want this behavior change. I can preserve the existing behavior by passing null_oid() instead, and pass super_oid in the actual "fetch unpopulated submodules" patch.

Show 20 quoted lines
>> @@ -1476,7 +1493,7 @@ static int get_next_submodule(struct child_process *cp,
>>  		if (!S_ISGITLINK(ce->ce_mode))
>>  			continue;
>>  
>> -		task = fetch_task_create(spf->r, ce->name);
>> +		task = fetch_task_create(spf->r, ce->name, null_oid());
>
> Hmm...is the plumbing incomplete? This code is about fetching, but we're
> not passing any superproject commit OID here. If this will be fixed in a
> future commit, maybe the distribution of what goes into each commit
> needs to be revised.
>
>> @@ -1499,7 +1516,7 @@ static int get_next_submodule(struct child_process *cp,
>>  			continue;
>>  		}
>>  
>> -		task->repo = get_submodule_repo_for(spf->r, task->sub->path);
>> +		task->repo = get_submodule_repo_for(spf->r, task->sub->path, null_oid());
>
> Same comment here.

This is intentional (but I admit that I also got confused rereading this). This should be null_oid() (i.e. read from the filesystem) because we are iterating through the index to get submodule paths. So we should pass null_oid() so that we read .gitmodules from the filesystem, not a possibly out-of-sync superproject commit.

Previous: Jonathan TanNext: Glen Choo
Message 9 of 149 in “fetch --recurse-submodules: fetch unpopulated submodules”
  1. 0/8 fetch --recurse-submodules: fetch unpopulated submodulesGlen Choo, Feb 10, 2022
  2. 1/8 submodule: inline submodule_commits() into callerGlen Choo, Feb 10, 2022
  3. 2/8 submodule: store new submodule commits oid_array in a structGlen Choo, Feb 10, 2022
  4. Jonathan TanFeb 10, 2022
  5. Junio C HamanoFeb 10, 2022
  6. 4/8 t5526: introduce test helper to assert on fetchesGlen Choo, Feb 10, 2022
  7. 3/8 submodule: make static functions read submodules from commitsGlen Choo, Feb 10, 2022
  8. Jonathan TanFeb 10, 2022
  9. Glen ChooFeb 11, 2022
  10. Glen ChooFeb 11, 2022
  11. 5/8 t5526: use grep to assert on fetchesGlen Choo, Feb 10, 2022
  12. Jonathan TanFeb 10, 2022
  13. 6/8 submodule: extract get_fetch_task()Glen Choo, Feb 10, 2022
  14. Jonathan TanFeb 10, 2022
  15. 7/8 fetch: fetch unpopulated, changed submodulesGlen Choo, Feb 10, 2022
  16. Junio C HamanoFeb 10, 2022
  17. Glen ChooFeb 11, 2022
  18. Junio C HamanoFeb 11, 2022
  19. Jonathan TanFeb 10, 2022
  20. Glen ChooFeb 14, 2022
  21. Glen ChooFeb 14, 2022
  22. Glen ChooFeb 14, 2022
  23. 8/8 submodule: fix bug and remove add_submodule_odb()Glen Choo, Feb 10, 2022
  24. Junio C HamanoFeb 10, 2022
  25. Glen ChooFeb 11, 2022
  26. Jonathan TanFeb 10, 2022
  27. Glen ChooFeb 11, 2022
  28. Junio C HamanoFeb 11, 2022
  29. Glen ChooFeb 14, 2022
  30. Junio C HamanoFeb 10, 2022
  31. Glen ChooFeb 10, 2022
  32. Junio C HamanoFeb 10, 2022
  33. Glen ChooFeb 11, 2022
  34. 0/9 fetch --recurse-submodules: fetch unpopulated submodulesGlen Choo, Feb 15, 2022
  35. 1/9 t5526: introduce test helper to assert on fetchesGlen Choo, Feb 15, 2022
  36. Ævar Arnfjörð BjarmasonFeb 15, 2022
  37. 2/9 t5526: use grep to assert on fetchesGlen Choo, Feb 15, 2022
  38. Ævar Arnfjörð BjarmasonFeb 15, 2022
  39. Glen ChooFeb 16, 2022
  40. Ævar Arnfjörð BjarmasonFeb 16, 2022
  41. Glen ChooFeb 17, 2022
  42. Ævar Arnfjörð BjarmasonFeb 17, 2022
  43. Glen ChooFeb 17, 2022
  44. 3/9 submodule: make static functions read submodules from commitsGlen Choo, Feb 15, 2022
  45. Jonathan TanFeb 15, 2022
  46. Glen ChooFeb 16, 2022
  47. Ævar Arnfjörð BjarmasonFeb 15, 2022
  48. Glen ChooFeb 16, 2022
  49. 4/9 submodule: inline submodule_commits() into callerGlen Choo, Feb 15, 2022
  50. Ævar Arnfjörð BjarmasonFeb 15, 2022
  51. 5/9 submodule: store new submodule commits oid_array in a structGlen Choo, Feb 15, 2022
  52. Ævar Arnfjörð BjarmasonFeb 15, 2022
  53. 6/9 submodule: extract get_fetch_task()Glen Choo, Feb 15, 2022
  54. 7/9 fetch: fetch unpopulated, changed submodulesGlen Choo, Feb 15, 2022
  55. Jonathan TanFeb 15, 2022
  56. Glen ChooFeb 16, 2022
  57. Glen ChooFeb 16, 2022
  58. Ævar Arnfjörð BjarmasonFeb 16, 2022
  59. Glen ChooFeb 16, 2022
  60. Ævar Arnfjörð BjarmasonFeb 15, 2022
  61. 8/9 submodule: read shallows when finding changed submodulesGlen Choo, Feb 15, 2022
  62. Jonathan TanFeb 15, 2022
  63. 9/9 submodule: fix latent check_has_commit() bugGlen Choo, Feb 15, 2022
  64. Jonathan TanFeb 15, 2022
  65. 00/10 fetch --recurse-submodules: fetch unpopulated submodulesGlen Choo, Feb 24, 2022
  66. 01/10 t5526: introduce test helper to assert on fetchesGlen Choo, Feb 24, 2022
  67. Junio C HamanoFeb 25, 2022
  68. 02/10 t5526: stop asserting on stderr literallyGlen Choo, Feb 24, 2022
  69. Ævar Arnfjörð BjarmasonFeb 24, 2022
  70. Glen ChooFeb 24, 2022
  71. Eric SunshineFeb 24, 2022
  72. Jonathan TanFeb 24, 2022
  73. Glen ChooFeb 25, 2022
  74. 04/10 submodule: make static functions read submodules from commitsGlen Choo, Feb 24, 2022
  75. 03/10 t5526: create superproject commits with test helperGlen Choo, Feb 24, 2022
  76. Jonathan TanFeb 24, 2022
  77. Glen ChooFeb 25, 2022
  78. Ævar Arnfjörð BjarmasonFeb 25, 2022
  79. Glen ChooFeb 28, 2022
  80. 05/10 submodule: inline submodule_commits() into callerGlen Choo, Feb 24, 2022
  81. 06/10 submodule: store new submodule commits oid_array in a structGlen Choo, Feb 24, 2022
  82. 07/10 submodule: extract get_fetch_task()Glen Choo, Feb 24, 2022
  83. Jonathan TanFeb 24, 2022
  84. 08/10 submodule: move logic into fetch_task_create()Glen Choo, Feb 24, 2022
  85. Jonathan TanFeb 24, 2022
  86. 10/10 submodule: fix latent check_has_commit() bugGlen Choo, Feb 24, 2022
  87. 09/10 fetch: fetch unpopulated, changed submodulesGlen Choo, Feb 24, 2022
  88. Junio C HamanoFeb 24, 2022
  89. Glen ChooFeb 25, 2022
  90. Junio C HamanoFeb 25, 2022
  91. Glen ChooFeb 25, 2022
  92. Jonathan TanFeb 25, 2022
  93. Glen ChooFeb 25, 2022
  94. Jonathan TanMar 4, 2022
  95. Glen ChooMar 5, 2022
  96. Jonathan TanMar 4, 2022
  97. Junio C HamanoFeb 26, 2022
  98. Johannes SchindelinMar 1, 2022
  99. Junio C HamanoMar 1, 2022
  100. Glen ChooMar 2, 2022
  101. Junio C HamanoMar 1, 2022
  102. 00/10 fetch --recurse-submodules: fetch unpopulated submodulesGlen Choo, Mar 4, 2022
  103. 01/10 t5526: introduce test helper to assert on fetchesGlen Choo, Mar 4, 2022
  104. Junio C HamanoMar 4, 2022
  105. Glen ChooMar 4, 2022
  106. 02/10 t5526: stop asserting on stderr literallyGlen Choo, Mar 4, 2022
  107. Junio C HamanoMar 4, 2022
  108. Jonathan TanMar 4, 2022
  109. Junio C HamanoMar 4, 2022
  110. Glen ChooMar 5, 2022
  111. 03/10 t5526: create superproject commits with test helperGlen Choo, Mar 4, 2022
  112. Jonathan TanMar 4, 2022
  113. 04/10 submodule: make static functions read submodules from commitsGlen Choo, Mar 4, 2022
  114. 05/10 submodule: inline submodule_commits() into callerGlen Choo, Mar 4, 2022
  115. 06/10 submodule: store new submodule commits oid_array in a structGlen Choo, Mar 4, 2022
  116. 07/10 submodule: extract get_fetch_task()Glen Choo, Mar 4, 2022
  117. 09/10 fetch: fetch unpopulated, changed submodulesGlen Choo, Mar 4, 2022
  118. Junio C HamanoMar 4, 2022
  119. Glen ChooMar 4, 2022
  120. Junio C HamanoMar 5, 2022
  121. Glen ChooMar 5, 2022
  122. Junio C HamanoMar 8, 2022
  123. Jonathan TanMar 4, 2022
  124. 10/10 submodule: fix latent check_has_commit() bugGlen Choo, Mar 4, 2022
  125. Junio C HamanoMar 4, 2022
  126. 08/10 submodule: move logic into fetch_task_create()Glen Choo, Mar 4, 2022
  127. Junio C HamanoMar 4, 2022
  128. 00/10 fetch --recurse-submodules: fetch unpopulated submodulesGlen Choo, Mar 8, 2022
  129. 01/10 t5526: introduce test helper to assert on fetchesGlen Choo, Mar 8, 2022
  130. 03/10 t5526: create superproject commits with test helperGlen Choo, Mar 8, 2022
  131. 02/10 t5526: stop asserting on stderr literallyGlen Choo, Mar 8, 2022
  132. 04/10 submodule: make static functions read submodules from commitsGlen Choo, Mar 8, 2022
  133. 05/10 submodule: inline submodule_commits() into callerGlen Choo, Mar 8, 2022
  134. 09/10 fetch: fetch unpopulated, changed submodulesGlen Choo, Mar 8, 2022
  135. 10/10 submodule: fix latent check_has_commit() bugGlen Choo, Mar 8, 2022
  136. 08/10 submodule: move logic into fetch_task_create()Glen Choo, Mar 8, 2022
  137. 06/10 submodule: store new submodule commits oid_array in a structGlen Choo, Mar 8, 2022
  138. 07/10 submodule: extract get_fetch_task()Glen Choo, Mar 8, 2022
  139. Junio C HamanoMar 8, 2022
  140. Glen ChooMar 8, 2022
  141. Junio C HamanoMar 9, 2022
  142. Glen ChooMar 9, 2022
  143. Junio C HamanoMar 9, 2022
  144. Glen ChooMar 9, 2022
  145. Glen ChooMar 16, 2022
  146. Junio C HamanoMar 16, 2022
  147. Glen ChooMar 16, 2022
  148. Junio C HamanoMar 16, 2022
  149. Jonathan TanMar 8, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.