# [Possible Bug] --cruft option not work with git gc --prune=now 5 messages from 2024-12-24 to 2024-12-30. Participants: ZheNing Hu, Taylor Blau, Jeff King. Thread: https://gitlist.dev/t/62692 ## ZheNing Hu, 2024-12-24 12:07 Subject: [Possible Bug] --cruft option not work with git gc --prune=now Message-ID: URL: https://gitlist.dev/e/CAOLTT8Q0cH3Q%3DyPnTYKAwn8bC%2BiDW7onibsLF%3DbzVr8tGw5J0Q%40mail.gmail.com ``` I am trying to use the --cruft feature of git gc, and everything works well, but when I combine it with --prune=now, I notice that no cruft package is generated, which is very strange. So I turned on trace and checked: instead of executing `git repack --cruft`, it executed `git repack -d -l -a`. Is this normal? If `git gc --cruft --prune=now` doesn't work as expected, it won't be safe to further execute `git gc --cruft --expire-to= --prune=now`(in my another patch). I checked the code, and it appears that add_repack_all_option() first processes `--prune=now` before handling `--cruft` option. If this is a bug, I am willing to propose a patch to fix it. ˋˋˋshell GIT_TRACE=1 git gc --prune=now --cruft 19:56:54.132263 git.c:479 trace: built-in: git gc --prune=now --cruft 19:56:54.137004 run-command.c:666 trace: run_command: git pack-refs --all --prune 19:56:54.137087 run-command.c:758 trace: start_command: /Users/adl/libexec/git-core/git pack-refs --all --prune 19:56:54.152233 git.c:479 trace: built-in: git pack-refs --all --prune 19:56:54.166598 run-command.c:666 trace: run_command: git reflog expire --all 19:56:54.166705 run-command.c:758 trace: start_command: /Users/adl/libexec/git-core/git reflog expire --all 19:56:54.181965 git.c:479 trace: built-in: git reflog expire --all 19:56:54.184495 run-command.c:666 trace: run_command: git repack -d -l -a 19:56:54.184571 run-command.c:758 trace: start_command: /Users/adl/libexec/git-core/git repack -d -l -a 19:56:54.196211 git.c:479 trace: built-in: git repack -d -l -a 19:56:54.198260 run-command.c:666 trace: run_command: git pack-objects --local --delta-base-offset objects/pack/.tmp-59391-pack --keep-true-parents --honor-pack-keep --non-empty --all --reflog --indexed-objects --write-bitmap-index-quiet 19:56:54.198304 run-command.c:758 trace: start_command: /Users/adl/libexec/git-core/git pack-objects --local --delta-base-offset objects/pack/.tmp-59391-pack --keep-true-parents --honor-pack-keep --non-empty --all --reflog --indexed-objects --write-bitmap-index-quiet 19:56:54.209384 git.c:479 trace: built-in: git pack-objects --local --delta-base-offset objects/pack/.tmp-59391-pack --keep-true-parents --honor-pack-keep --non-empty --all --reflog --indexed-objects --write-bitmap-index-quiet Enumerating objects: 512819, done. Counting objects: 100% (512819/512819), done. Delta compression using up to 12 threads Compressing objects: 100% (237575/237575), done. Writing objects: 100% (512819/512819), done. Selecting bitmap commits: 23999, done. Building bitmaps: 100% (309/309), done. Total 512819 (delta 116078), reused 512819 (delta 116078), pack-reused 0 (from 0) 19:57:12.783287 run-command.c:666 trace: run_command: git prune --expire now 19:57:12.783887 run-command.c:758 trace: start_command: /Users/adl/libexec/git-core/git prune --expire now 19:57:12.799337 git.c:479 trace: built-in: git prune --expire now 19:57:12.802050 run-command.c:666 trace: run_command: git worktree prune --expire 3.months.ago 19:57:12.802115 run-command.c:758 trace: start_command: /Users/adl/libexec/git-core/git worktree prune --expire 3.months.ago 19:57:12.812977 git.c:479 trace: built-in: git worktree prune --expire 3.months.ago 19:57:12.814412 run-command.c:666 trace: run_command: git rerere gc 19:57:12.814476 run-command.c:758 trace: start_command: /Users/adl/libexec/git-core/git rerere gc 19:57:12.825732 git.c:479 trace: built-in: git rerere gc ˋˋˋ -- ZheNing Hu ``` ## Taylor Blau, 2024-12-24 15:43 Subject: Re: [Possible Bug] --cruft option not work with git gc --prune=now Message-ID: URL: https://gitlist.dev/e/CABvF%2B3Z5kWAAbLz_P%3DoL1o90-u5Mo%3DDCyi5sEv-a_bJABB%3DpKw%40mail.gmail.com In-Reply-To: ``` On Tue, Dec 24, 2024 at 08:07:16PM +0800, ZheNing Hu wrote: > I am trying to use the --cruft feature of git gc, and everything works > well, but when I combine it with --prune=now, I notice that no cruft > package is generated, which is very strange. So I turned on trace and > checked: instead of executing `git repack --cruft`, it executed `git > repack -d -l -a`. Is this normal? If `git gc --cruft --prune=now` > doesn't work as expected, it won't be safe to further execute `git gc > --cruft --expire-to= --prune=now`(in my another patch). > > I checked the code, and it appears that add_repack_all_option() first > processes `--prune=now` before handling `--cruft` option. That's right, since --prune=now will cause your repository to purge any unreachable objects immediately, regardless of their mtimes. So after a 'git gc --prune=now', your repository will only consist of reachable objects, meaning that there will be nothing to put into a cruft pack. Note that 'git gc' does not support the '--expire-to' option, but you can use it via 'git repack'. Thanks, Taylor ``` ## ZheNing Hu, 2024-12-26 06:19 Subject: Re: [Possible Bug] --cruft option not work with git gc --prune=now Message-ID: URL: https://gitlist.dev/e/CAOLTT8TraRjBObcLotNr7%2Ba9Ap7Fy8VVy7HjHLLWwEB_LvBrUQ%40mail.gmail.com In-Reply-To: ``` Taylor Blau 于2024年12月24日周二 23:39写道： > > On Tue, Dec 24, 2024 at 08:07:16PM +0800, ZheNing Hu wrote: > > I am trying to use the --cruft feature of git gc, and everything works > > well, but when I combine it with --prune=now, I notice that no cruft > > package is generated, which is very strange. So I turned on trace and > > checked: instead of executing `git repack --cruft`, it executed `git > > repack -d -l -a`. Is this normal? If `git gc --cruft --prune=now` > > doesn't work as expected, it won't be safe to further execute `git gc > > --cruft --expire-to= --prune=now`(in my another patch). > > > > I checked the code, and it appears that add_repack_all_option() first > > processes `--prune=now` before handling `--cruft` option. > > That's right, since --prune=now will cause your repository to purge any > unreachable objects immediately, regardless of their mtimes. > > So after a 'git gc --prune=now', your repository will only consist of > reachable objects, meaning that there will be nothing to put into a > cruft pack. > > Note that 'git gc' does not support the '--expire-to' option, but you > can use it via 'git repack'. > Okay, I know that gc doesn't have an --expire-to parameter, but here I was hoping to use git gc --prune=now --expire-to=, where expired and unreachable objects would be stored in for easy recovery in case of repository corruption. Moreover, it's more convenient compared to using git repack + git prune (isn't a single gc command much simpler? Plus, most users are likely familiar with gc rather than repack and prune). > Thanks, > Taylor Thanks, ZheNing Hu ``` ## Jeff King, 2024-12-26 15:23 Subject: Re: [Possible Bug] --cruft option not work with git gc --prune=now Message-ID: <20241226152347.GA68791@coredump.intra.peff.net> URL: https://gitlist.dev/e/20241226152347.GA68791%40coredump.intra.peff.net In-Reply-To: ``` On Thu, Dec 26, 2024 at 02:19:56PM +0800, ZheNing Hu wrote: > Okay, I know that gc doesn't have an --expire-to parameter, > but here I was hoping to use git gc --prune=now --expire-to=, > where expired and unreachable objects would be stored in for > easy recovery in case of repository corruption. Moreover, it's more > convenient compared to using git repack + git prune > (isn't a single gc command much simpler? Plus, most users are > likely familiar with gc rather than repack and prune). I don't think there is any reason that gc _can't_ support --expire-to as you want. But any patch to do so would have to adjust how it calls git-repack. Up until now, "--prune=now" meant it was fine to run "repack -ad", since that would just discard any unreachable packed objects without further consideration. If we're keeping any objects, then we have to tell repack what to do with them (which traditionally was just passing --unpack-unreachable, but these days can include --cruft). With --expire-to, that logic wouldn't be valid anymore; even if we are expiring all objects, we still might want to tell repack where to put the expired ones. So I think you'd want to adjust the if/else in add_repack_all_option() to pass --cruft, etc, when you see --expire-to (but still continue to "repack -ad" otherwise to keep the traditional "git gc --prune=now" as cheap as possible). -Peff ``` ## ZheNing Hu, 2024-12-30 11:58 Subject: Re: [Possible Bug] --cruft option not work with git gc --prune=now Message-ID: URL: https://gitlist.dev/e/CAOLTT8R3WRiTcNjYzHdiazK9F0ppC9RG5m%2B9gyX6KrcTEF6jyw%40mail.gmail.com In-Reply-To: <20241226152347.GA68791@coredump.intra.peff.net> ``` Jeff King 于2024年12月26日周四 23:23写道： > > On Thu, Dec 26, 2024 at 02:19:56PM +0800, ZheNing Hu wrote: > > > Okay, I know that gc doesn't have an --expire-to parameter, > > but here I was hoping to use git gc --prune=now --expire-to=, > > where expired and unreachable objects would be stored in for > > easy recovery in case of repository corruption. Moreover, it's more > > convenient compared to using git repack + git prune > > (isn't a single gc command much simpler? Plus, most users are > > likely familiar with gc rather than repack and prune). > > I don't think there is any reason that gc _can't_ support --expire-to as > you want. But any patch to do so would have to adjust how it calls > git-repack. > > Up until now, "--prune=now" meant it was fine to run "repack -ad", since > that would just discard any unreachable packed objects without further > consideration. If we're keeping any objects, then we have to tell > repack what to do with them (which traditionally was just passing > --unpack-unreachable, but these days can include --cruft). > > With --expire-to, that logic wouldn't be valid anymore; even if we are > expiring all objects, we still might want to tell repack where to put > the expired ones. So I think you'd want to adjust the if/else in > add_repack_all_option() to pass --cruft, etc, when you see --expire-to > (but still continue to "repack -ad" otherwise to keep the traditional > "git gc --prune=now" as cheap as possible). > You're right. Only when using `--cruft` and `--expire-to=` in git-gc, the behavior of `--prune=now` should change: it should pass `--cruft-expiration=now` to repack instead of passing `-a`. I will cc you in the patch for supporting --expired-to= in git gc. Thank you :) > -Peff -- ZheNing Hu ```