Re: [PATCH 2/2] sparse-checkout: clear tracked sparse dirs

From: Derrick Stolee <stolee@gmail.com>
Date: Aug 5, 2021, 01:55 UTC
Message-ID: <53d1b6ac-76a0-034b-109c-df2835c8fad6@gmail.com>
In-Reply-To: <CABPp-BE1aKAsp6yKAzM-djRHuiP=GHC_Q7xZxMWPJ=f28bOeUg@mail.gmail.com>

On 8/2/2021 12:17 PM, Elijah Newren wrote:

Show 29 quoted lines

> On Mon, Aug 2, 2021 at 8:34 AM Derrick Stolee <stolee@gmail.com> wrote:
>>
>> On 7/30/2021 9:52 AM, Elijah Newren wrote:
>>> On Thu, Jul 29, 2021 at 11:27 AM Derrick Stolee via GitGitGadget
>>> <gitgitgadget@gmail.com> wrote:
> ...
>>>> +                */
>>>> +               if (S_ISSPARSEDIR(ce->ce_mode) &&
>>>> +                   repo_file_exists(r, ce->name)) {
>>>> +                       strbuf_setlen(&path, pathlen);
>>>> +                       strbuf_addstr(&path, ce->name);
>>>> +
>>>> +                       /*
>>>> +                        * Removal is "best effort". If something blocks
>>>> +                        * the deletion, then continue with a warning.
>>>> +                        */
>>>> +                       if (remove_dir_recursively(&path, 0))
>>>> +                               warning(_("failed to remove directory '%s'"), path.buf);
>>>
>>> Um, doesn't this delete untracked files that are not ignored as well
>>> as the ignored files?  If so, was that intentional?  I'm fully on
>>> board with removing the gitignore'd files, but I'm worried removing
>>> other untracked files is dangerous.
>>
>> I believe that 'git sparse-checkout (set|add|reapply)' will fail before
>> reaching this method if there are untracked files that could potentially
>> be removed. I will double-check to ensure this is the case. It is
>> definitely my intention to protect any untracked, non-ignored files in
>> these directories by failing the sparse-checkout modification.

This is _not_ true, and I can document it with a test.

Having untracked files outside of the sparse cone is just as bad as ignored files, so I want to ensure that these get cleaned up, too.

The correct thing would be to prevent the 'git sparse-checkout (set|add|reapply)' command from making any changes to the sparse-checkout cone or the worktree if there are untracked files that would be deleted. (Right? Or is there another solution that I'm missing here?)

Show 14 quoted lines

>>> My implementation of this concept (in an external tool) was more along
>>> the lines of
>>>
>>>   * Get $LIST_OF_NON_SPARSE_DIRECTORIES by walking `git ls-files -t`
>>> output and finding common fully-sparse directories
>>>   * git clean -fX $LIST_OF_NON_SPARSE_DIRECTORIES
>>
>> I initially was running 'git clean -dfx -- <dir> ...' but that also
>> requires parsing and expanding the index (or being very careful with
>> the sparse index).
> 
> `git clean -dfx -- <dir> ...` could also be very dangerous because
> it'd delete untracked non-ignored files.  You want -X rather than -x.
> One of those cases where capitalization is critical.

Good point. I'd like to avoid using `git clean` as a subcommand, if possible, that way we have one fewer thing to do before integrating the `git sparse-checkout` builtin with the sparse index.

Thanks, -Stolee