git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v4 1/6] revision: add mark_tree_uninteresting_sparse

From
Derrick Stolee <stolee@gmail.com>
Date
Jan 11, 2019, 22:05 UTC
Message-ID
<5a25813e-32f6-34a4-9f24-b65415b75dda@gmail.com>
In-Reply-To
<xmqqva2vx7p2.fsf@gitster-ct.c.googlers.com>
On 1/11/2019 3:25 PM, Junio C Hamano wrote:
Show 36 quoted lines
> Junio C Hamano <gitster@pobox.com> writes:
>
> So, I assumed that the implementation was wrong, but it is the other
> way around.  You do mean to pick only already uninteresting trees
> out of "set" and mark its reachables.
>
> One thing that would make me worried is what help the callers of
> this function will get (or they may have to devise the way
> themselves) to avoid having to traverse the same tree number of
> times.  A tree can be made uninteresting after a traversal of
> another tree that contains it, but the logic in this function
>
>> +		if (tree->object.flags & UNINTERESTING) {
>> +			/*
>> +			 * Remove the flag so the next call
>> +			 * is not a no-op. The flag is added
>> +			 * in mark_tree_unintersting().
>> +			 */
>> +			tree->object.flags ^= UNINTERESTING;
>> +			mark_tree_uninteresting(r, tree);
>> +		}
> ignores the fact that it is already UNINTERESTING (in fact, in a
> sense it is even worse---it cannot be used to make a not-yet
> UNINTERESTING one into UNINTERESTING), drops the UNINTERESING bit
> and forces the traversal of that tree.  The only thing I see that
> would work as a saving grace is that mark_tree_uninteresting()
> itself would honor existing UNINTERESING bit and refuses to recurse
> into its subtrees, but that means blobs at the top-level of such a
> tree would be marked UNINTERESING while those in its subtrees can be
> left not-UNINTERESING, which sounds like inviting a mess.
>
> It does *not* immediately mean this function is misdesigned.  It
> just means that the caller needs to carefully follow whatever
> calling convention this series will establish in the later patches
> (which we haven't seen yet at this point).
>

I'm sorry that this implementation is particularly confusing. It is created only as a placeholder so we can wire up the --sparse option to use this method without being "wrong" but is then removed entirely in PATCH 4/6:

...
void mark_trees_uninteresting_sparse(struct repository *r,
  				     struct oidset *set)
  {
+	unsigned has_interesting = 0, has_uninteresting = 0;
+	struct hashmap map;
+	struct hashmap_iter map_iter;
+	struct path_and_oids_entry *entry;
  	struct object_id *oid;
  	struct oidset_iter iter;
  
  	oidset_iter_init(set, &iter);
-	while ((oid = oidset_iter_next(&iter))) {
+	while ((!has_interesting || !has_uninteresting) &&
+	       (oid = oidset_iter_next(&iter))) {
  		struct tree *tree = lookup_tree(r, oid);
  
  		if (!tree)
  			continue;
  
-		if (tree->object.flags & UNINTERESTING) {
-			/*
-			 * Remove the flag so the next call
-			 * is not a no-op. The flag is added
-			 * in mark_tree_unintersting().
-			 */
-			tree->object.flags ^= UNINTERESTING;
-			mark_tree_uninteresting(r, tree);
-		}
+		if (tree->object.flags & UNINTERESTING)
+			has_uninteresting = 1;
+		else
+			has_interesting = 1;
+	}
...
You are definitely correct that "set" is not a valuable variable name. It could instead be "tree_oids" to be slightly more informative.

Thanks, -Stolee

Previous: Junio C HamanoNext: Derrick Stolee via GitGitGadget
Message 31 of 51 in “Add a new "sparse" tree walk algorithm”
  1. 0/5 Add a new "sparse" tree walk algorithmDerrick Stolee via GitGitGadget, Nov 28, 2018
  2. 1/5 revision: add mark_tree_uninteresting_sparseDerrick Stolee via GitGitGadget, Nov 28, 2018
  3. 2/5 list-objects: consume sparse tree walkDerrick Stolee via GitGitGadget, Nov 28, 2018
  4. 3/5 pack-objects: add --sparse optionDerrick Stolee via GitGitGadget, Nov 28, 2018
  5. Stefan BellerNov 28, 2018
  6. Derrick StoleeNov 29, 2018
  7. Junio C HamanoNov 30, 2018
  8. Derrick StoleeNov 30, 2018
  9. 5/5 pack-objects: create pack.useSparse settingDerrick Stolee via GitGitGadget, Nov 28, 2018
  10. 4/5 revision: implement sparse algorithmDerrick Stolee via GitGitGadget, Nov 28, 2018
  11. Ævar Arnfjörð BjarmasonNov 28, 2018
  12. Derrick StoleeNov 29, 2018
  13. 0/6 Add a new "sparse" tree walk algorithmDerrick Stolee via GitGitGadget, Nov 29, 2018
  14. 3/6 pack-objects: add --sparse optionDerrick Stolee via GitGitGadget, Nov 29, 2018
  15. 4/6 revision: implement sparse algorithmDerrick Stolee via GitGitGadget, Nov 29, 2018
  16. 6/6 pack-objects: create GIT_TEST_PACK_SPARSEDerrick Stolee via GitGitGadget, Nov 29, 2018
  17. 5/6 pack-objects: create pack.useSparse settingDerrick Stolee via GitGitGadget, Nov 29, 2018
  18. 2/6 list-objects: consume sparse tree walkDerrick Stolee via GitGitGadget, Nov 29, 2018
  19. 1/6 revision: add mark_tree_uninteresting_sparseDerrick Stolee via GitGitGadget, Nov 29, 2018
  20. 0/6 Add a new "sparse" tree walk algorithmDerrick Stolee via GitGitGadget, Dec 10, 2018
  21. 1/6 revision: add mark_tree_uninteresting_sparseDerrick Stolee via GitGitGadget, Dec 10, 2018
  22. 3/6 pack-objects: add --sparse optionDerrick Stolee via GitGitGadget, Dec 10, 2018
  23. 2/6 list-objects: consume sparse tree walkDerrick Stolee via GitGitGadget, Dec 10, 2018
  24. 4/6 revision: implement sparse algorithmDerrick Stolee via GitGitGadget, Dec 10, 2018
  25. 6/6 pack-objects: create GIT_TEST_PACK_SPARSEDerrick Stolee via GitGitGadget, Dec 10, 2018
  26. 5/6 pack-objects: create pack.useSparse settingDerrick Stolee via GitGitGadget, Dec 10, 2018
  27. 0/6 Add a new "sparse" tree walk algorithmDerrick Stolee via GitGitGadget, Dec 14, 2018
  28. 1/6 revision: add mark_tree_uninteresting_sparseDerrick Stolee via GitGitGadget, Dec 14, 2018
  29. Junio C HamanoJan 11, 2019
  30. Junio C HamanoJan 11, 2019
  31. Derrick StoleeJan 11, 2019
  32. 2/6 list-objects: consume sparse tree walkDerrick Stolee via GitGitGadget, Dec 14, 2018
  33. Junio C HamanoJan 11, 2019
  34. 3/6 pack-objects: add --sparse optionDerrick Stolee via GitGitGadget, Dec 14, 2018
  35. Junio C HamanoJan 11, 2019
  36. Derrick StoleeJan 15, 2019
  37. Junio C HamanoJan 15, 2019
  38. 4/6 revision: implement sparse algorithmDerrick Stolee via GitGitGadget, Dec 14, 2018
  39. Ævar Arnfjörð BjarmasonDec 14, 2018
  40. Derrick StoleeDec 17, 2018
  41. Ævar Arnfjörð BjarmasonDec 17, 2018
  42. Derrick StoleeDec 17, 2018
  43. Junio C HamanoJan 11, 2019
  44. 5/6 pack-objects: create pack.useSparse settingDerrick Stolee via GitGitGadget, Dec 14, 2018
  45. 6/6 pack-objects: create GIT_TEST_PACK_SPARSEDerrick Stolee via GitGitGadget, Dec 14, 2018
  46. 0/5 Add a new "sparse" tree walk algorithmDerrick Stolee via GitGitGadget, Jan 16, 2019
  47. 2/5 list-objects: consume sparse tree walkDerrick Stolee via GitGitGadget, Jan 16, 2019
  48. 3/5 revision: implement sparse algorithmDerrick Stolee via GitGitGadget, Jan 16, 2019
  49. 4/5 pack-objects: create pack.useSparse settingDerrick Stolee via GitGitGadget, Jan 16, 2019
  50. 5/5 pack-objects: create GIT_TEST_PACK_SPARSEDerrick Stolee via GitGitGadget, Jan 16, 2019
  51. 1/5 revision: add mark_tree_uninteresting_sparseDerrick Stolee via GitGitGadget, Jan 16, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.