# [RFC] git-fetch - repack in the background after fetching 5 messages from 2006-05-30 to 2006-05-30. Participants: Martin Langhoff, Linus Torvalds, Daniel Barkalow. Thread: https://gitlist.dev/t/4349 ## Martin Langhoff, 2006-05-30 04:42 Subject: [RFC] git-fetch - repack in the background after fetching Message-ID: <11489641631558-git-send-email-martin@catalyst.net.nz> URL: https://gitlist.dev/e/11489641631558-git-send-email-martin%40catalyst.net.nz ``` Check whether we have a large set of unpacked objects and repack after the fetch, but don't for the user to wait for us. --- There's been some discussion about repacking proactively without preventing further work. But as Linus said, repacking on an active repo is _safe_, so repack in the background. If we like this approach, we should at least respect a git-repo-config entry saying core.noautorepack for users who don't want it. I don't really know if there is any convention for us to check if we are in a resource-constrained situation (aka laptops on battery). If there is, we should respect that as well. I suspect anacron and others do this already but I can't find any references. We can potentially do it on commit, merge and push as well. --- git-fetch.sh | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) 5498d015eb1062928a504af3c6b3cb9b776088e8 diff --git a/git-fetch.sh b/git-fetch.sh index 69bd810..4d64cdb 100755 --- a/git-fetch.sh +++ b/git-fetch.sh @@ -424,3 +424,9 @@ case ",$update_head_ok,$orig_head," in fi ;; esac + +if test $(git rev-list --unpacked --all | wc -l) -gt 1000 +then + echo "Repacking in the background" + nice git repack -a -d -q & +fi -- 1.3.2.g82000 ``` ## Linus Torvalds, 2006-05-30 04:51 Subject: Re: [RFC] git-fetch - repack in the background after fetching Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0605292147010.5623%40g5.osdl.org In-Reply-To: <11489641631558-git-send-email-martin@catalyst.net.nz> ``` On Tue, 30 May 2006, Martin Langhoff wrote: > > There's been some discussion about repacking proactively without > preventing further work. But as Linus said, repacking on an active > repo is _safe_ Repacking is, but "-d" is not necessarily. You really should do the prune-packed only _after_ you've repacked, and no old git programs are around. Some long-running (in git terms) git programs will look up the pack-files when they start, and if you repack after that, they won't see the new pack-file, but they _will_ notice that the unpacked files are no longer there, and will be very unhappy indeed. So the "-d" part really isn't necessarily safe. Of course, in -practice- you won't likely see this, and the archive itself is never corrupted, but concurrent git ops can fail due to it in theory, and quite frankly, that's not the kind of SCM I like to use. So either just do "git repack -a", or do things synchronously. Linus ``` ## Martin Langhoff, 2006-05-30 05:14 Subject: Re: [RFC] git-fetch - repack in the background after fetching Message-ID: <46a038f90605292214l241a86b8l31edfab010a38c5f@mail.gmail.com> URL: https://gitlist.dev/e/46a038f90605292214l241a86b8l31edfab010a38c5f%40mail.gmail.com In-Reply-To: ``` On 5/30/06, Linus Torvalds wrote: > Repacking is, but "-d" is not necessarily. Ok -- strawman knocked down. Next try... > Some long-running (in git terms) git programs will look up the pack-files > when they start, and if you repack after that, they won't see the new > pack-file, but they _will_ notice that the unpacked files are no longer > there, and will be very unhappy indeed. > > So the "-d" part really isn't necessarily safe. > > Of course, in -practice- you won't likely see this, and the archive itself > is never corrupted, but concurrent git ops can fail due to it in theory, > and quite frankly, that's not the kind of SCM I like to use. Would it be safe to repack -a && sleep 180 && git prune-packed ? > So either just do "git repack -a", or do things synchronously. Which I take to mean 'prune synchronously'. So what about... + +if test $(git rev-list --unpacked --all | wc -l) -gt 1000 +then + echo "Repacking in the background" + git prune-packed + nice git repack -a -q & +fi this would mean that at any given time there's a bit of overlap between packed and unpacked, but will be resolved over repeated commands. martin ``` ## Daniel Barkalow, 2006-05-30 06:37 Subject: Re: [RFC] git-fetch - repack in the background after fetching Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0605300207130.6713%40iabervon.org In-Reply-To: ``` On Mon, 29 May 2006, Linus Torvalds wrote: > Some long-running (in git terms) git programs will look up the pack-files > when they start, and if you repack after that, they won't see the new > pack-file, but they _will_ notice that the unpacked files are no longer > there, and will be very unhappy indeed. We should be able to fix this, right? If an object isn't found in packs or unpacked, look for new packs; if there are any, look for the object in them; if it's not there, then give up. The only tricky thing is making it possible to scan through the available packs without installing any that are already installed. I think the failure case is only a critical path in the history-walking fetch code, which should probably disable this (or defer it to after trying to download the object). -Daniel *This .sig left intentionally blank* ``` ## Linus Torvalds, 2006-05-30 14:53 Subject: Re: [RFC] git-fetch - repack in the background after fetching Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0605300752430.5623%40g5.osdl.org In-Reply-To: ``` On Tue, 30 May 2006, Daniel Barkalow wrote: > > We should be able to fix this, right? If an object isn't found in packs or > unpacked, look for new packs; if there are any, look for the object in > them; if it's not there, then give up. Yes. That sounds fine. Linus ```