git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 0/4] [RFC] repack: add --filter=

From
John Cai <johncai86@gmail.com>
Date
Feb 22, 2022, 18:52 UTC
Message-ID
<CFECF0B1-A94F-4372-AFC9-C0469A17E9A5@gmail.com>
In-Reply-To
<YhQHYQ9b9bYYv10r@nand.local>
Hi Taylor,
On 21 Feb 2022, at 16:42, Taylor Blau wrote:
Show 45 quoted lines
> On Mon, Feb 21, 2022 at 10:10:15PM +0100, Christian Couder wrote:
>>>> Also, to have more protection we can either
>>>>
>>>> 1. add a config value that needs to be set to true for repack to remove
>>>> objects (repack.allowDestroyFilter).
>>
>> I don't think it's of much value. We don't have such config values for
>> other possibly destructive operations.
>>
>>>> 2. --filter is dry-run by default and prints out objects that would have been removed,
>>>> and it has to be combined with another flag --destroy in order for it to actually remove
>>>> objects from the odb.
>>
>> I am not sure it's of much value either compared to naming it
>> --filter-destroy. It's likely to just make things more difficult for
>> users to understand.
>
> On this and the above, I agree with Christian.
>
>>> I share the same concern as Robert and Stolee do. But I think this issue
>>> goes deeper than just naming.
>>>
>>> Even if we called this `git repack --delete-filter` and only ran it with
>>> `--i-know-what-im-doing` flag, we would still be leaving repository
>>> corruption on the table, just making it marginally more difficult to
>>> achieve.
>>
>> My opinion on this is that the promisor object mechanism assumes by
>> design that some objects are outside a repo, and that this repo
>> shouldn't care much about these objects possibly being corrupted.
>
> For what it's worth, I am fine having a mode of repack which allows us
> to remove objects that we know are stored by a promisor remote. But this
> series doesn't do that, so users could easily run `git repack -d
> --filter=...` and find that they have irrecoverably corrupted their
> repository.
>
> I think that there are some other reasonable directions, though. One
> which Robert and I discussed was making it possible to split a
> repository into two packs, one which holds objects that match some
> `--filter` criteria, and one which holds the objects that don't match
> that filter.
>
> Another option would be to prune the repository according to objects
> that are already made available by a promisor remote.

Thanks for the discussion around the two packfile idea. Definitely interesting. However, I'm leaning towards the second option here where we ensure that objects that are about to be deleted can be retrieved via a promisor remote. That way we have an easy path to recovery.

Show 30 quoted lines
>
> An appealing quality about the above two directions is that the first
> doesn't actually remove any objects, just makes it easier to push a
> whole pack of unwanted objects off to a promsior remote. The second
> prunes the repository according to objects that are already made
> available by the promisor remote. (Yes, there is a TOCTOU race there,
> too, but it's the same prune-while-pushing race that Git already has
> today).
>
>> I am not against a name and some docs that strongly state that users
>> should be very careful when using such a command, but otherwise I
>> think such a command is perfectly ok. We have other commands that by
>> design could lead to some objects or data being lost.
>
> I can think of a handful of ways to remove objects which are unreachable
> from a repository, but I am not sure we have any ways to remove objects
> which are reachable.
>
>>> But as it stands right now, I worry that this feature is too easily
>>> misused and could result in unintended repository corruption.
>>
>> Are you worrying about the UI or about what it does?
>>
>> I am ok with improving the UI, but I think what it does is reasonable.
>
> I am more worried about the proposal's functionality than its UI,
> hopefully my concerns there are summarized above.
>
> Thanks,
> Taylor
Previous: Taylor BlauNext: Taylor Blau
Message 33 of 34 in “repack: add --filter=”
  1. 0/2 repack: add --filter=John Cai via GitGitGadget, Jan 27, 2022
  2. 1/2 pack-objects: allow --filter without --stdoutJohn Cai via GitGitGadget, Jan 27, 2022
  3. 2/2 repack: add --filter=<filter-spec> optionJohn Cai via GitGitGadget, Jan 27, 2022
  4. Derrick StoleeJan 27, 2022
  5. John CaiJan 29, 2022
  6. Christian CouderJan 30, 2022
  7. John CaiJan 30, 2022
  8. 0/4 [RFC] repack: add --filter=John Cai via GitGitGadget, Feb 9, 2022
  9. 2/4 repack: add --filter=<filter-spec> optionJohn Cai via GitGitGadget, Feb 9, 2022
  10. John CaiFeb 9, 2022
  11. 3/4 upload-pack: allow missing promisor objectsJohn Cai via GitGitGadget, Feb 9, 2022
  12. 1/4 pack-objects: allow --filter without --stdoutJohn Cai via GitGitGadget, Feb 9, 2022
  13. 4/4 tests for repack --filter modeJohn Cai via GitGitGadget, Feb 9, 2022
  14. Robert CoupFeb 17, 2022
  15. John CaiFeb 17, 2022
  16. Robert CoupFeb 16, 2022
  17. John CaiFeb 16, 2022
  18. Taylor BlauFeb 21, 2022
  19. Robert CoupFeb 21, 2022
  20. Taylor BlauFeb 21, 2022
  21. Christian CouderFeb 21, 2022
  22. Taylor BlauFeb 21, 2022
  23. Christian CouderFeb 22, 2022
  24. Taylor BlauFeb 22, 2022
  25. Robert CoupFeb 23, 2022
  26. Junio C HamanoFeb 23, 2022
  27. John CaiFeb 26, 2022
  28. Taylor BlauFeb 26, 2022
  29. John CaiFeb 26, 2022
  30. Taylor BlauFeb 26, 2022
  31. John CaiFeb 26, 2022
  32. Taylor BlauFeb 26, 2022
  33. John CaiFeb 22, 2022
  34. Taylor BlauFeb 22, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.