git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 6/7] sparse-checkout: use new update_sparsity() function

From
Elijah Newren <newren@gmail.com>
Date
Mar 16, 2020, 17:02 UTC
Message-ID
<CABPp-BEPCpN=gW=LC7JBbJNmqhYxEhQJi6ycV21zhmuFAta3gw@mail.gmail.com>
In-Reply-To
<1c83fb68-519a-88f2-77b9-ab443ed6b4cf@gmail.com>
On Sun, Mar 15, 2020 at 9:19 AM Derrick Stolee <stolee@gmail.com> wrote:
Show 55 quoted lines
>
> On 3/14/2020 3:11 AM, Elijah Newren via GitGitGadget wrote:
> > From: Elijah Newren <newren@gmail.com>
> >
> > Remove the equivalent of 'git read-tree -mu HEAD' in the sparse-checkout
> > codepaths for setting the SKIP_WORKTREE bits and instead use the new
> > update_sparsity() function.
> >
> > Signed-off-by: Elijah Newren <newren@gmail.com>
> > ---
> >  builtin/sparse-checkout.c          | 39 +++++++-----------------------
> >  t/t1091-sparse-checkout-builtin.sh | 32 +++++++++++++++++-------
> >  2 files changed, 32 insertions(+), 39 deletions(-)
> >
> > diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c
> > index 740da4b6d54..5d3ec2e6be9 100644
> > --- a/builtin/sparse-checkout.c
> > +++ b/builtin/sparse-checkout.c
> > @@ -97,47 +97,32 @@ static int update_working_directory(struct pattern_list *pl)
> >       int result = 0;
> >       struct unpack_trees_options o;
> >       struct lock_file lock_file = LOCK_INIT;
> > -     struct object_id oid;
> > -     struct tree *tree;
> > -     struct tree_desc t;
> >       struct repository *r = the_repository;
> >
> > -     if (repo_read_index_unmerged(r))
> > -             die(_("you need to resolve your current index first"));
> > -
> > -     if (get_oid("HEAD", &oid))
> > -             return 0;
> > -
> > -     tree = parse_tree_indirect(&oid);
> > -     parse_tree(tree);
> > -     init_tree_desc(&t, tree->buffer, tree->size);
> > -
> >       memset(&o, 0, sizeof(o));
> >       o.verbose_update = isatty(2);
> > -     o.merge = 1;
> >       o.update = 1;
> > -     o.fn = oneway_merge;
>
> I'm glad this is getting much simpler.
>
> >       o.head_idx = -1;
> >       o.src_index = r->index;
> >       o.dst_index = r->index;
> >       o.skip_sparse_checkout = 0;
> >       o.pl = pl;
> > -     o.keep_pattern_list = !!pl;
>
> I'm making a mental note to see if this is necessary or not. I'm
> guessing that update_sparsity() expects a pattern list as input and
> will never free it, so this option is useless.

Yes, although your comment made me realize that I've got a memory leak if we pass o->pl==NULL to update_sparsity(), which happens whenever we pass NULL to update_working_directory(). I should fix that.

Show 18 quoted lines
> > -     resolve_undo_clear_index(r->index);
> >       setup_work_tree();
> >
> > -     cache_tree_free(&r->index->cache_tree);
> > -
> >       repo_hold_locked_index(r, &lock_file, LOCK_DIE_ON_ERROR);
> >
> > -     core_apply_sparse_checkout = 1;
> > -     result = unpack_trees(1, &t, &o);
> > +     result = update_sparsity(&o);
> >
> > -     if (!result) {
> > -             prime_cache_tree(r, r->index, tree);
> > +     if (result == 1)
> I wonder if it is appropriate to return an enum in update_sparsity()
> so this statement is clearer out-of-context. For example, I'm looking
> at this patch before looking closely at the update_sparsity() patch
> so I know how it will be used before I check its implementation.
Yes, that'd be better.
Show 124 quoted lines
> > +             /*
> > +              * We don't do any special handling of warnings from untracked
> > +              * files in the way or dirty entries that can't be removed.
> > +              */
> > +             result = 0;
> > +     if (!result)
> >               write_locked_index(r->index, &lock_file, COMMIT_LOCK);
> > -     } else
> > +     else
> >               rollback_lock_file(&lock_file);
> >
> >       return result;
> > @@ -304,8 +289,6 @@ static int sparse_checkout_init(int argc, const char **argv)
> >       };
> >
> >       repo_read_index(the_repository);
> > -     require_clean_work_tree(the_repository,
> > -                             N_("initialize sparse-checkout"), NULL, 1, 0);
> >
> >       argc = parse_options(argc, argv, NULL,
> >                            builtin_sparse_checkout_init_options,
> > @@ -560,8 +543,6 @@ static int sparse_checkout_set(int argc, const char **argv, const char *prefix,
> >       };
> >
> >       repo_read_index(the_repository);
> > -     require_clean_work_tree(the_repository,
> > -                             N_("set sparse-checkout patterns"), NULL, 1, 0);
> >
> >       argc = parse_options(argc, argv, prefix,
> >                            builtin_sparse_checkout_set_options,
> > @@ -577,8 +558,6 @@ static int sparse_checkout_disable(int argc, const char **argv)
> >       struct strbuf match_all = STRBUF_INIT;
> >
> >       repo_read_index(the_repository);
> > -     require_clean_work_tree(the_repository,
> > -                             N_("disable sparse-checkout"), NULL, 1, 0);
>
> In the three hunks above, do we still need repo_read_index()?
>
> >       memset(&pl, 0, sizeof(pl));
> >       hashmap_init(&pl.recursive_hashmap, pl_hashmap_cmp, NULL, 0);
> > diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh
> > index 8607a8e6d1a..0d93d3983e0 100755
> > --- a/t/t1091-sparse-checkout-builtin.sh
> > +++ b/t/t1091-sparse-checkout-builtin.sh
> > @@ -277,16 +277,23 @@ test_expect_success 'cone mode: add parent path' '
> >       check_files repo a deep folder1
> >  '
> >
> > -test_expect_success 'revert to old sparse-checkout on bad update' '
> > +test_expect_success 'not-up-to-date does not block rest of sparsification' '
> >       test_when_finished git -C repo sparse-checkout disable &&
> >       test_when_finished git -C repo reset --hard &&
> >       git -C repo sparse-checkout set deep &&
> > +
> >       echo update >repo/deep/deeper2/a &&
> >       cp repo/.git/info/sparse-checkout expect &&
> > -     test_must_fail git -C repo sparse-checkout set deep/deeper1 2>err &&
> > -     test_i18ngrep "cannot set sparse-checkout patterns" err &&
> > -     test_cmp repo/.git/info/sparse-checkout expect &&
> > -     check_files repo/deep a deeper1 deeper2
> > +     test_write_lines "!/deep/*/" "/deep/deeper1/" >>expect &&
> > +
> > +     git -C repo sparse-checkout set deep/deeper1 2>err &&
> > +
> > +     test_i18ngrep "Cannot update sparse checkout" err &&
> > +     test_cmp expect repo/.git/info/sparse-checkout &&
> > +     check_files repo/deep a deeper1 deeper2 &&
> > +     check_files repo/deep/deeper1 a deepest &&
> > +     check_files repo/deep/deeper1/deepest a &&
> > +     check_files repo/deep/deeper2 a
> >  '
> >
> >  test_expect_success 'revert to old sparse-checkout on empty update' '
> > @@ -316,12 +323,19 @@ test_expect_success '.gitignore should not warn about cone mode' '
> >       test_i18ngrep ! "disabling cone patterns" err
> >  '
> >
> > -test_expect_success 'sparse-checkout (init|set|disable) fails with dirty status' '
> > +test_expect_success 'sparse-checkout (init|set|disable) warns with dirty status' '
> >       git clone repo dirty &&
> >       echo dirty >dirty/folder1/a &&
> > -     test_must_fail git -C dirty sparse-checkout init &&
> > -     test_must_fail git -C dirty sparse-checkout set /folder2/* /deep/deeper1/* &&
> > -     test_must_fail git -C dirty sparse-checkout disable &&
> > +
> > +     git -C dirty sparse-checkout init 2>err &&
> > +     test_i18ngrep "warning.*Cannot update sparse checkout" err &&
> > +
> > +     git -C dirty sparse-checkout set /folder2/* /deep/deeper1/* &&
> > +     test_i18ngrep "warning.*Cannot update sparse checkout" err &&
> > +
> > +     git -C dirty sparse-checkout disable &&
> > +     test_i18ngrep "warning.*Cannot update sparse checkout" err &&
> > +
> >       git -C dirty reset --hard &&
> >       git -C dirty sparse-checkout init &&
> >       git -C dirty sparse-checkout set /folder2/* /deep/deeper1/* &&
>
> I like that the test changes here are limited to updating existing
> tests. You mentioned adding more tests, which I would also like, but
> those can come in a follow-up patch that is dedicated to all the fun
> situations we can get into, and how we should respond.
>
> Here are a few that I've tested manually:
>
> 1. Start: no sparse-checkout
>   a. Get into rebase conflict in a directory.
>   b. "git sparse-checkout init --cone"
>   c. See that the file is on-disk with conflict markers.
>   d. Resolve conflict and "git add"
>   e. "git sparse-checkout set <dir-without-conflict>"
>   f. Sparse-checkout is set as expected, file is not on disk, but still staged.
>   g. "git rebase --continue" succeeds.
>
> 2. Start: sparse-checkout set to "dir1"
>   a. Get into merge conflict in "dir2"
>   b. Resolve conflict, "git add"
>   c. "git sparse-checkout update" (available after next patch)
>   d. dir1 gone, dir1/file staged.
>   e. git merge --continue succeeds
>
> Those are the really big ones to me, but I'm sure we will find others.
> I'm really happy that this is working as we expect.
Thanks for taking a look.
Previous: Derrick StoleeNext: Derrick Stolee
Message 17 of 69 in “Sparse checkout improvements -- improved sparsity updating”
  1. 0/7 Sparse checkout improvements -- improved sparsity updatingElijah Newren via GitGitGadget, Mar 14, 2020
  2. 7/7 sparse-checkout: provide a new update subcommandElijah Newren via GitGitGadget, Mar 14, 2020
  3. Derrick StoleeMar 15, 2020
  4. Elijah NewrenMar 16, 2020
  5. Derrick StoleeMar 16, 2020
  6. Elijah NewrenMar 16, 2020
  7. 1/7 unpack-trees: fix minor typo in commentElijah Newren via GitGitGadget, Mar 14, 2020
  8. 3/7 unpack-trees: allow check_updates() to work on a different indexElijah Newren via GitGitGadget, Mar 14, 2020
  9. 4/7 unpack-trees: do not mark a dirty path with SKIP_WORKTREEElijah Newren via GitGitGadget, Mar 14, 2020
  10. Derrick StoleeMar 15, 2020
  11. 5/7 unpack-trees: add a new update_sparsity() functionElijah Newren via GitGitGadget, Mar 14, 2020
  12. Derrick StoleeMar 15, 2020
  13. Elijah NewrenMar 16, 2020
  14. 2/7 t1091: make some tests a little more defensive against failuresElijah Newren via GitGitGadget, Mar 14, 2020
  15. 6/7 sparse-checkout: use new update_sparsity() functionElijah Newren via GitGitGadget, Mar 14, 2020
  16. Derrick StoleeMar 15, 2020
  17. Elijah NewrenMar 16, 2020
  18. Derrick StoleeMar 15, 2020
  19. 00/18 Sparse checkout improvements -- improved sparsity updatingElijah Newren via GitGitGadget, Mar 21, 2020
  20. 01/18 unpack-trees: fix minor typo in commentElijah Newren via GitGitGadget, Mar 21, 2020
  21. 03/18 unpack-trees: simplify verify_absent_sparse()Elijah Newren via GitGitGadget, Mar 21, 2020
  22. 02/18 unpack-trees: remove unused error typeElijah Newren via GitGitGadget, Mar 21, 2020
  23. 04/18 unpack-trees: simplify pattern_list freeingElijah Newren via GitGitGadget, Mar 21, 2020
  24. Derrick StoleeMar 23, 2020
  25. 06/18 unpack-trees: allow check_updates() to work on a different indexElijah Newren via GitGitGadget, Mar 21, 2020
  26. 05/18 t1091: make some tests a little more defensive against failuresElijah Newren via GitGitGadget, Mar 21, 2020
  27. 08/18 unpack-trees: pull sparse-checkout pattern reading into a new functionElijah Newren via GitGitGadget, Mar 21, 2020
  28. 09/18 unpack-trees: add a new update_sparsity() functionElijah Newren via GitGitGadget, Mar 21, 2020
  29. Derrick StoleeMar 23, 2020
  30. Elijah NewrenMar 23, 2020
  31. Derrick StoleeMar 23, 2020
  32. Junio C HamanoMar 23, 2020
  33. 11/18 sparse-checkout: use improved unpack_trees porcelain messagesElijah Newren via GitGitGadget, Mar 21, 2020
  34. 13/18 unpack-trees: rename ERROR_* fields meant for warnings to WARNING_*Elijah Newren via GitGitGadget, Mar 21, 2020
  35. 10/18 sparse-checkout: use new update_sparsity() functionElijah Newren via GitGitGadget, Mar 21, 2020
  36. Derrick StoleeMar 23, 2020
  37. Elijah NewrenMar 23, 2020
  38. Derrick StoleeMar 23, 2020
  39. 15/18 unpack-trees: make sparse path messages sound like warningsElijah Newren via GitGitGadget, Mar 21, 2020
  40. 18/18 sparse-checkout: provide a new reapply subcommandElijah Newren via GitGitGadget, Mar 21, 2020
  41. Derrick StoleeMar 23, 2020
  42. 17/18 unpack-trees: failure to set SKIP_WORKTREE bits always just a warningElijah Newren via GitGitGadget, Mar 21, 2020
  43. 14/18 unpack-trees: split display_error_msgs() into twoElijah Newren via GitGitGadget, Mar 21, 2020
  44. Derrick StoleeMar 23, 2020
  45. 16/18 unpack-trees: provide warnings on sparse updates for unmerged paths tooElijah Newren via GitGitGadget, Mar 21, 2020
  46. 07/18 unpack-trees: do not mark a dirty path with SKIP_WORKTREEElijah Newren via GitGitGadget, Mar 21, 2020
  47. 12/18 unpack-trees: move ERROR_WOULD_LOSE_SUBMODULE earlierElijah Newren via GitGitGadget, Mar 21, 2020
  48. Derrick StoleeMar 23, 2020
  49. Junio C HamanoMar 23, 2020
  50. 00/18 Sparse checkout improvements -- improved sparsity updatingElijah Newren via GitGitGadget, Mar 27, 2020
  51. 01/18 unpack-trees: fix minor typo in commentElijah Newren via GitGitGadget, Mar 27, 2020
  52. 02/18 unpack-trees: remove unused error typeElijah Newren via GitGitGadget, Mar 27, 2020
  53. 06/18 unpack-trees: allow check_updates() to work on a different indexElijah Newren via GitGitGadget, Mar 27, 2020
  54. 04/18 unpack-trees: simplify pattern_list freeingElijah Newren via GitGitGadget, Mar 27, 2020
  55. 05/18 t1091: make some tests a little more defensive against failuresElijah Newren via GitGitGadget, Mar 27, 2020
  56. 03/18 unpack-trees: simplify verify_absent_sparse()Elijah Newren via GitGitGadget, Mar 27, 2020
  57. 07/18 unpack-trees: do not mark a dirty path with SKIP_WORKTREEElijah Newren via GitGitGadget, Mar 27, 2020
  58. 08/18 unpack-trees: pull sparse-checkout pattern reading into a new functionElijah Newren via GitGitGadget, Mar 27, 2020
  59. 09/18 unpack-trees: add a new update_sparsity() functionElijah Newren via GitGitGadget, Mar 27, 2020
  60. 10/18 sparse-checkout: use new update_sparsity() functionElijah Newren via GitGitGadget, Mar 27, 2020
  61. 11/18 sparse-checkout: use improved unpack_trees porcelain messagesElijah Newren via GitGitGadget, Mar 27, 2020
  62. 13/18 unpack-trees: rename ERROR_* fields meant for warnings to WARNING_*Elijah Newren via GitGitGadget, Mar 27, 2020
  63. 16/18 unpack-trees: provide warnings on sparse updates for unmerged paths tooElijah Newren via GitGitGadget, Mar 27, 2020
  64. 18/18 sparse-checkout: provide a new reapply subcommandElijah Newren via GitGitGadget, Mar 27, 2020
  65. 15/18 unpack-trees: make sparse path messages sound like warningsElijah Newren via GitGitGadget, Mar 27, 2020
  66. 14/18 unpack-trees: split display_error_msgs() into twoElijah Newren via GitGitGadget, Mar 27, 2020
  67. 17/18 unpack-trees: failure to set SKIP_WORKTREE bits always just a warningElijah Newren via GitGitGadget, Mar 27, 2020
  68. 12/18 unpack-trees: move ERROR_WOULD_LOSE_SUBMODULE earlierElijah Newren via GitGitGadget, Mar 27, 2020
  69. Derrick StoleeMar 27, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.