{"thread":{"id":"4349","subject":"[RFC] git-fetch - repack in the background after fetching","startedAt":"2006-05-30T04:42:43Z","lastAt":"2006-05-30T14:53:14Z","messageCount":5,"participants":["Martin Langhoff","Linus Torvalds","Daniel Barkalow"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"20952","messageId":"11489641631558-git-send-email-martin@catalyst.net.nz","threadId":"4349","inReplyTo":null,"subject":"[RFC] git-fetch - repack in the background after fetching","fromName":"Martin Langhoff","fromEmail":"martin@catalyst.net.nz","sentAt":"2006-05-30T04:42:43Z","receivedAt":"2006-05-30T04:42:43Z","isPatch":false,"sender":{"key":"martin@laptop.org","avatar":null},"body":"Check whether we have a large set of unpacked objects and repack\nafter the fetch, but don't for the user to wait for us.\n\n---\n\nThere's been some discussion about repacking proactively without\npreventing further work. But as Linus said, repacking on an active\nrepo is _safe_, so repack in the background. \n\nIf we like this approach, we should at least respect a git-repo-config\nentry saying core.noautorepack for users who don't want it. I don't\nreally know if there is any convention for us to check if we are in\na resource-constrained situation (aka laptops on battery). If there\nis, we should respect that as well. I suspect anacron and others \ndo this already but I can't find any references.\n\nWe can potentially do it on commit, merge and push as well. \n---\n\n git-fetch.sh |    6 ++++++\n 1 files changed, 6 insertions(+), 0 deletions(-)\n\n5498d015eb1062928a504af3c6b3cb9b776088e8\ndiff --git a/git-fetch.sh b/git-fetch.sh\nindex 69bd810..4d64cdb 100755\n--- a/git-fetch.sh\n+++ b/git-fetch.sh\n@@ -424,3 +424,9 @@ case \",$update_head_ok,$orig_head,\" in\n \tfi\n \t;;\n esac\n+\n+if test $(git rev-list --unpacked --all | wc -l) -gt 1000\n+then\n+\techo \"Repacking in the background\"\n+\tnice git repack -a -d -q &\n+fi\n-- \n1.3.2.g82000\n"},{"id":"20954","messageId":"Pine.LNX.4.64.0605292147010.5623@g5.osdl.org","threadId":"4349","inReplyTo":"11489641631558-git-send-email-martin@catalyst.net.nz","subject":"Re: [RFC] git-fetch - repack in the background after fetching","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-05-30T04:51:00Z","receivedAt":"2006-05-30T04:51:00Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Tue, 30 May 2006, Martin Langhoff wrote:\n> \n> There's been some discussion about repacking proactively without\n> preventing further work. But as Linus said, repacking on an active\n> repo is _safe_\n\nRepacking is, but \"-d\" is not necessarily.\n\nYou really should do the prune-packed only _after_ you've repacked, and no \nold git programs are around.\n\nSome long-running (in git terms) git programs will look up the pack-files \nwhen they start, and if you repack after that, they won't see the new \npack-file, but they _will_ notice that the unpacked files are no longer \nthere, and will be very unhappy indeed.\n\nSo the \"-d\" part really isn't necessarily safe.\n\nOf course, in -practice- you won't likely see this, and the archive itself \nis never corrupted, but concurrent git ops can fail due to it in theory, \nand quite frankly, that's not the kind of SCM I like to use.\n\nSo either just do \"git repack -a\", or do things synchronously.\n\n\t\tLinus\n"},{"id":"20955","messageId":"46a038f90605292214l241a86b8l31edfab010a38c5f@mail.gmail.com","threadId":"4349","inReplyTo":"Pine.LNX.4.64.0605292147010.5623@g5.osdl.org","subject":"Re: [RFC] git-fetch - repack in the background after fetching","fromName":"Martin Langhoff","fromEmail":"martin.langhoff@gmail.com","sentAt":"2006-05-30T05:14:08Z","receivedAt":"2006-05-30T05:14:08Z","isPatch":false,"sender":{"key":"martin.langhoff@gmail.com","avatar":"https://gravatar.com/avatar/1e3f311b6c4c15836501901ca58f8c0b0667246488084ba524d8bc9867e22fd9?d=mp&s=160"},"body":"On 5/30/06, Linus Torvalds <torvalds@osdl.org> wrote:\n> Repacking is, but \"-d\" is not necessarily.\n\nOk -- strawman knocked down. Next try...\n\n> Some long-running (in git terms) git programs will look up the pack-files\n> when they start, and if you repack after that, they won't see the new\n> pack-file, but they _will_ notice that the unpacked files are no longer\n> there, and will be very unhappy indeed.\n>\n> So the \"-d\" part really isn't necessarily safe.\n>\n> Of course, in -practice- you won't likely see this, and the archive itself\n> is never corrupted, but concurrent git ops can fail due to it in theory,\n> and quite frankly, that's not the kind of SCM I like to use.\n\nWould it be safe to repack -a && sleep 180 && git prune-packed ?\n\n> So either just do \"git repack -a\", or do things synchronously.\n\nWhich I take to mean 'prune synchronously'. So what about...\n\n+\n+if test $(git rev-list --unpacked --all | wc -l) -gt 1000\n+then\n+       echo \"Repacking in the background\"\n+       git prune-packed\n+       nice git repack -a -q &\n+fi\n\nthis would mean that at any given time there's a bit of overlap\nbetween packed and unpacked, but will be resolved over repeated\ncommands.\n\n\n\n\nmartin\n"},{"id":"20959","messageId":"Pine.LNX.4.64.0605300207130.6713@iabervon.org","threadId":"4349","inReplyTo":"Pine.LNX.4.64.0605292147010.5623@g5.osdl.org","subject":"Re: [RFC] git-fetch - repack in the background after fetching","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2006-05-30T06:37:25Z","receivedAt":"2006-05-30T06:37:25Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Mon, 29 May 2006, Linus Torvalds wrote:\n\n> Some long-running (in git terms) git programs will look up the pack-files \n> when they start, and if you repack after that, they won't see the new \n> pack-file, but they _will_ notice that the unpacked files are no longer \n> there, and will be very unhappy indeed.\n\nWe should be able to fix this, right? If an object isn't found in packs or \nunpacked, look for new packs; if there are any, look for the object in \nthem; if it's not there, then give up. The only tricky thing is making it \npossible to scan through the available packs without installing any that \nare already installed. I think the failure case is only a critical path in \nthe history-walking fetch code, which should probably disable this (or \ndefer it to after trying to download the object).\n\n\t-Daniel\n*This .sig left intentionally blank*\n"},{"id":"20973","messageId":"Pine.LNX.4.64.0605300752430.5623@g5.osdl.org","threadId":"4349","inReplyTo":"Pine.LNX.4.64.0605300207130.6713@iabervon.org","subject":"Re: [RFC] git-fetch - repack in the background after fetching","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-05-30T14:53:14Z","receivedAt":"2006-05-30T14:53:14Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Tue, 30 May 2006, Daniel Barkalow wrote:\n> \n> We should be able to fix this, right? If an object isn't found in packs or \n> unpacked, look for new packs; if there are any, look for the object in \n> them; if it's not there, then give up.\n\nYes. That sounds fine.\n\n\t\tLinus\n"}]}