git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 6/7] reset: make --mixed sparse-aware

From
Elijah Newren <newren@gmail.com>
Date
Oct 6, 2021, 04:43 UTC
Message-ID
<CABPp-BHDog6U1bFKurNq+sKbNPOnmqbhB8W2wg0UeF=jb+ZScw@mail.gmail.com>
In-Reply-To
<5eaae0825af4cee84040b784c32190bb1de2f919.1633440057.git.gitgitgadget@gmail.com>

On Tue, Oct 5, 2021 at 6:21 AM Victoria Dye via GitGitGadget <gitgitgadget@gmail.com> wrote:

Show 48 quoted lines
>
> From: Victoria Dye <vdye@github.com>
>
> Sparse directory entries are "diffed" as trees in `diff_cache` (used
> internally by `reset --mixed`), following a code path separate from
> individual file handling. The use of `diff_tree_oid` there requires setting
> explicit `change` and `add_remove` functions to process the internal
> contents of a sparse directory.
>
> Additionally, the `recursive` diff option handles cases in which `reset
> --mixed` must diff/merge files that are nested multiple levels deep in a
> sparse directory.
>
> Signed-off-by: Victoria Dye <vdye@github.com>
> ---
>  builtin/reset.c                          | 30 +++++++++++++++++++++++-
>  t/t1092-sparse-checkout-compatibility.sh | 13 +++++++++-
>  2 files changed, 41 insertions(+), 2 deletions(-)
>
> diff --git a/builtin/reset.c b/builtin/reset.c
> index e1f2a2bb2c4..ceb9b122897 100644
> --- a/builtin/reset.c
> +++ b/builtin/reset.c
> @@ -175,6 +175,8 @@ static int read_from_tree(const struct pathspec *pathspec,
>                           int intent_to_add)
>  {
>         struct diff_options opt;
> +       unsigned int i;
> +       char *skip_worktree_seen = NULL;
>
>         memset(&opt, 0, sizeof(opt));
>         copy_pathspec(&opt.pathspec, pathspec);
> @@ -182,9 +184,35 @@ static int read_from_tree(const struct pathspec *pathspec,
>         opt.format_callback = update_index_from_diff;
>         opt.format_callback_data = &intent_to_add;
>         opt.flags.override_submodule_config = 1;
> +       opt.flags.recursive = 1;
>         opt.repo = the_repository;
> +       opt.change = diff_change;
> +       opt.add_remove = diff_addremove;
> +
> +       /*
> +        * When pathspec is given for resetting a cone-mode sparse checkout, it may
> +        * identify entries that are nested in sparse directories, in which case the
> +        * index should be expanded. For the sake of efficiency, this check is
> +        * overly-cautious: anything with a wildcard or a magic prefix requires
> +        * expansion, as well as literal paths that aren't in the sparse checkout
> +        * definition AND don't match any directory in the index.

s/efficiency/efficiency of checking/ ? Being overly-cautious suggests you'll expand to a full index more than is needed, and full indexes are more expensive. But perhaps the checking would be expensive too so you have a tradeoff?

Or maybe s/efficiency/simplicity/?
> +        */
> +       if (pathspec->nr && the_index.sparse_index) {
> +               if (pathspec->magic || pathspec->has_wildcard) {
> +                       ensure_full_index(&the_index);

dir.c has the notion of matching the characters preceding the wildcard characters; look for "no_wildcard_len". If the pathspec doesn't match a path up to no_wildcard_len, then the wildcard character(s) later in the pathspec can't make the pathspec match that path.

It might at least be worth mentioning this as a possible future optimization.
> +               } else {
> +                       for (i = 0; i < pathspec->nr; i++) {
> +                               if (!path_in_cone_mode_sparse_checkout(pathspec->items[i].original, &the_index) &&
> +                                   !matches_skip_worktree(pathspec, i, &skip_worktree_seen)) {

What if the pathspec corresponds to a sparse-directory in the index, but possibly without the trailing '/' character? e.g.:

   git reset HEAD~1 -- sparse-directory

One should be able to reset that directory without recursing into it...does this code handle that? Does it handle it if we add the trailing slash on the path for the reset command line?

Show 26 quoted lines
> +                                       ensure_full_index(&the_index);
> +                                       break;
> +                               }
> +                       }
> +               }
> +       }
> +
> +       free(skip_worktree_seen);
>
> -       ensure_full_index(&the_index);
>         if (do_diff_cache(tree_oid, &opt))
>                 return 1;
>         diffcore_std(&opt);
> diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh
> index e301ef5633a..4afcbc2d673 100755
> --- a/t/t1092-sparse-checkout-compatibility.sh
> +++ b/t/t1092-sparse-checkout-compatibility.sh
> @@ -804,11 +804,22 @@ test_expect_success 'sparse-index is not expanded' '
>                 ensure_not_expanded reset --hard $ref || return 1
>         done &&
>
> +       ensure_not_expanded reset --mixed base &&
>         ensure_not_expanded reset --hard update-deep &&
>         ensure_not_expanded reset --keep base &&
>         ensure_not_expanded reset --merge update-deep &&
> -       ensure_not_expanded reset --hard &&

This commit was only touching the --mixed case; why is it removing one of the tests for --hard?

Show 8 quoted lines
>
> +       ensure_not_expanded reset base -- deep/a &&
> +       ensure_not_expanded reset base -- nonexistent-file &&
> +       ensure_not_expanded reset deepest -- deep &&
> +
> +       # Although folder1 is outside the sparse definition, it exists as a
> +       # directory entry in the index, so it will be reset without needing to
> +       # expand the full index.

Ah, I think this answers one of my earlier questions. Does it also work with 'folder1/' as well as 'folder1'?

> +       ensure_not_expanded reset --hard update-folder1 &&

Wait...is update-folder1 a branch or a path? And if this commit is about --mixed, why are --hard testcases being added?

> +       ensure_not_expanded reset base -- folder1 &&
> +
> +       ensure_not_expanded reset --hard update-deep &&
another --hard testcase...was this an accidental squash by chance?
Show 6 quoted lines
>         ensure_not_expanded checkout -f update-deep &&
>         test_config -C sparse-index pull.twohead ort &&
>         (
> --
> gitgitgadget
>
Previous: Victoria Dye via GitGitGadgetNext: Victoria Dye
Message 43 of 114 in “Sparse Index: integrate with reset”
  1. 0/7 Sparse Index: integrate with resetVictoria Dye via GitGitGadget, Sep 30, 2021
  2. 1/7 reset: behave correctly with sparse-checkoutKevin Willford via GitGitGadget, Sep 30, 2021
  3. Junio C HamanoSep 30, 2021
  4. Victoria DyeOct 1, 2021
  5. 2/7 sparse-index: update command for expand/collapse testVictoria Dye via GitGitGadget, Sep 30, 2021
  6. Taylor BlauSep 30, 2021
  7. Victoria DyeSep 30, 2021
  8. Junio C HamanoSep 30, 2021
  9. Victoria DyeSep 30, 2021
  10. Junio C HamanoOct 1, 2021
  11. Victoria DyeOct 4, 2021
  12. Bagas SanjayaOct 1, 2021
  13. 3/7 reset: expand test coverage for sparse checkoutsVictoria Dye via GitGitGadget, Sep 30, 2021
  14. 4/7 reset: integrate with sparse indexVictoria Dye via GitGitGadget, Sep 30, 2021
  15. 5/7 reset: make sparse-aware (except --mixed)Victoria Dye via GitGitGadget, Sep 30, 2021
  16. 6/7 reset: make --mixed sparse-awareVictoria Dye via GitGitGadget, Sep 30, 2021
  17. Victoria DyeOct 1, 2021
  18. 7/7 unpack-trees: improve performance of next_cache_entryVictoria Dye via GitGitGadget, Sep 30, 2021
  19. 0/7 Sparse Index: integrate with resetVictoria Dye via GitGitGadget, Oct 5, 2021
  20. 1/7 reset: behave correctly with sparse-checkoutKevin Willford via GitGitGadget, Oct 5, 2021
  21. Junio C HamanoOct 5, 2021
  22. Victoria DyeOct 5, 2021
  23. Junio C HamanoOct 6, 2021
  24. Elijah NewrenOct 6, 2021
  25. Victoria DyeOct 6, 2021
  26. Bagas SanjayaOct 6, 2021
  27. 3/7 reset: expand test coverage for sparse checkoutsVictoria Dye via GitGitGadget, Oct 5, 2021
  28. Elijah NewrenOct 6, 2021
  29. 4/7 reset: integrate with sparse indexVictoria Dye via GitGitGadget, Oct 5, 2021
  30. Elijah NewrenOct 6, 2021
  31. Junio C HamanoOct 6, 2021
  32. 2/7 update-index: add --force-full-index option for expand/collapse testVictoria Dye via GitGitGadget, Oct 5, 2021
  33. Elijah NewrenOct 6, 2021
  34. Victoria DyeOct 6, 2021
  35. Elijah NewrenOct 8, 2021
  36. Junio C HamanoOct 8, 2021
  37. Bagas SanjayaOct 6, 2021
  38. 5/7 reset: make sparse-aware (except --mixed)Victoria Dye via GitGitGadget, Oct 5, 2021
  39. Elijah NewrenOct 6, 2021
  40. Victoria DyeOct 6, 2021
  41. Bagas SanjayaOct 6, 2021
  42. 6/7 reset: make --mixed sparse-awareVictoria Dye via GitGitGadget, Oct 5, 2021
  43. Elijah NewrenOct 6, 2021
  44. Victoria DyeOct 7, 2021
  45. 7/7 unpack-trees: improve performance of next_cache_entryVictoria Dye via GitGitGadget, Oct 5, 2021
  46. Bagas SanjayaOct 6, 2021
  47. Ævar Arnfjörð BjarmasonOct 5, 2021
  48. Victoria DyeOct 5, 2021
  49. Elijah NewrenOct 6, 2021
  50. 0/8 Sparse Index: integrate with resetVictoria Dye via GitGitGadget, Oct 7, 2021
  51. 1/8 reset: rename is_missing to !is_in_reset_treeVictoria Dye via GitGitGadget, Oct 7, 2021
  52. 2/8 reset: preserve skip-worktree bit in mixed resetKevin Willford via GitGitGadget, Oct 7, 2021
  53. Junio C HamanoOct 8, 2021
  54. 3/8 update-index: add --force-full-index option for expand/collapse testVictoria Dye via GitGitGadget, Oct 7, 2021
  55. Bagas SanjayaOct 8, 2021
  56. Junio C HamanoOct 8, 2021
  57. Victoria DyeOct 8, 2021
  58. Junio C HamanoOct 8, 2021
  59. Derrick StoleeOct 11, 2021
  60. Victoria DyeOct 11, 2021
  61. Junio C HamanoOct 11, 2021
  62. 4/8 reset: expand test coverage for sparse checkoutsVictoria Dye via GitGitGadget, Oct 7, 2021
  63. 5/8 reset: integrate with sparse indexVictoria Dye via GitGitGadget, Oct 7, 2021
  64. 6/8 reset: make sparse-aware (except --mixed)Victoria Dye via GitGitGadget, Oct 7, 2021
  65. Phillip WoodOct 8, 2021
  66. Victoria DyeOct 8, 2021
  67. Junio C HamanoOct 8, 2021
  68. Phillip WoodOct 9, 2021
  69. Junio C HamanoOct 10, 2021
  70. Victoria DyeOct 11, 2021
  71. Junio C HamanoOct 11, 2021
  72. Phillip WoodOct 12, 2021
  73. Junio C HamanoOct 12, 2021
  74. Phillip WoodOct 12, 2021
  75. 7/8 reset: make --mixed sparse-awareVictoria Dye via GitGitGadget, Oct 7, 2021
  76. Elijah NewrenNov 20, 2021
  77. Victoria DyeNov 22, 2021
  78. 8/8 unpack-trees: improve performance of next_cache_entryVictoria Dye via GitGitGadget, Oct 7, 2021
  79. 0/8 Sparse Index: integrate with resetVictoria Dye via GitGitGadget, Oct 11, 2021
  80. 1/8 reset: rename is_missing to !is_in_reset_treeVictoria Dye via GitGitGadget, Oct 11, 2021
  81. 2/8 reset: preserve skip-worktree bit in mixed resetVictoria Dye via GitGitGadget, Oct 11, 2021
  82. Emily ShafferOct 22, 2021
  83. Victoria DyeOct 25, 2021
  84. 3/8 sparse-index: update command for expand/collapse testVictoria Dye via GitGitGadget, Oct 11, 2021
  85. 4/8 reset: expand test coverage for sparse checkoutsVictoria Dye via GitGitGadget, Oct 11, 2021
  86. 5/8 reset: integrate with sparse indexVictoria Dye via GitGitGadget, Oct 11, 2021
  87. 7/8 reset: make --mixed sparse-awareVictoria Dye via GitGitGadget, Oct 11, 2021
  88. 6/8 reset: make sparse-aware (except --mixed)Victoria Dye via GitGitGadget, Oct 11, 2021
  89. 8/8 unpack-trees: improve performance of next_cache_entryVictoria Dye via GitGitGadget, Oct 11, 2021
  90. 0/8 Sparse Index: integrate with resetVictoria Dye via GitGitGadget, Oct 27, 2021
  91. 1/8 reset: rename is_missing to !is_in_reset_treeVictoria Dye via GitGitGadget, Oct 27, 2021
  92. 2/8 reset: preserve skip-worktree bit in mixed resetVictoria Dye via GitGitGadget, Oct 27, 2021
  93. 3/8 sparse-index: update command for expand/collapse testVictoria Dye via GitGitGadget, Oct 27, 2021
  94. 4/8 reset: expand test coverage for sparse checkoutsVictoria Dye via GitGitGadget, Oct 27, 2021
  95. 5/8 reset: integrate with sparse indexVictoria Dye via GitGitGadget, Oct 27, 2021
  96. 6/8 reset: make sparse-aware (except --mixed)Victoria Dye via GitGitGadget, Oct 27, 2021
  97. 7/8 reset: make --mixed sparse-awareVictoria Dye via GitGitGadget, Oct 27, 2021
  98. Elijah NewrenNov 20, 2021
  99. Victoria DyeNov 22, 2021
  100. 8/8 unpack-trees: improve performance of next_cache_entryVictoria Dye via GitGitGadget, Oct 27, 2021
  101. Elijah NewrenNov 20, 2021
  102. 0/8 Sparse Index: integrate with resetVictoria Dye via GitGitGadget, Nov 29, 2021
  103. 1/8 reset: rename is_missing to !is_in_reset_treeVictoria Dye via GitGitGadget, Nov 29, 2021
  104. 2/8 reset: preserve skip-worktree bit in mixed resetVictoria Dye via GitGitGadget, Nov 29, 2021
  105. 3/8 sparse-index: update command for expand/collapse testVictoria Dye via GitGitGadget, Nov 29, 2021
  106. 4/8 reset: expand test coverage for sparse checkoutsVictoria Dye via GitGitGadget, Nov 29, 2021
  107. 5/8 reset: integrate with sparse indexVictoria Dye via GitGitGadget, Nov 29, 2021
  108. 6/8 reset: make sparse-aware (except --mixed)Victoria Dye via GitGitGadget, Nov 29, 2021
  109. 7/8 reset: make --mixed sparse-awareVictoria Dye via GitGitGadget, Nov 29, 2021
  110. 8/8 unpack-trees: improve performance of next_cache_entryVictoria Dye via GitGitGadget, Nov 29, 2021
  111. Elijah NewrenNov 29, 2021
  112. Victoria DyeNov 29, 2021
  113. Elijah NewrenNov 30, 2021
  114. Ævar Arnfjörð BjarmasonDec 1, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.