{"thread":{"id":"13236","subject":"Cleaning the .git directory with gc","startedAt":"2008-04-23T23:13:59Z","lastAt":"2008-04-24T21:14:58Z","messageCount":6,"participants":["Haakon Riiser","Russ Dill","David Tweed","Shawn O. Pearce"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"75118","messageId":"20080423231359.GA30913@fox","threadId":"13236","inReplyTo":null,"subject":"Cleaning the .git directory with gc","fromName":"Haakon Riiser","fromEmail":"haakon.riiser@fys.uio.no","sentAt":"2008-04-23T23:13:59Z","receivedAt":"2008-04-23T23:13:59Z","isPatch":false,"sender":{"key":"haakon.riiser@fys.uio.no","avatar":null},"body":"I've recently started using git, and while experimenting with\ngit commit --amend, I noticed that git gc does not do what I\nexpected. Example:\n\n $ mkdir foo && cd foo\n $ git init\n $ dd if=/dev/urandom bs=1k count=1000 of=rand.dat\n $ git add .\n $ git commit -a -m 'first rev'\n $ du -s .git\n 1100 .git\n\n1 MB file checked in, 1 MB repository. So far, so good.\n\n $ dd if=/dev/urandom bs=1k count=1000 of=rand.dat\n $ git commit -a -m 'replaced first rev' --amend\n $ du -s .git\n 2120 .git\n\nAt this point, I expected the --amend command to notice that\nthe amended commit contains a replacement for the old file,\nand thus that the repository didn't grow. I then figured that\nif --amend doesn't do that by itself, git gc surely will:\n\n $ git gc\n $ du -s .git\n 2104 .git\n\nSo, why doesn't gc remove the data from the first commit? Is it\nstill accessible, even though the log doesn't show it?\n\nIs it possible to actually replace the commit, i.e., to make it\nexactly like the first commit never happend at all? (Without\nmodifying the repository by hand.)\n\n-- \n Haakon\n"},{"id":"75119","messageId":"f9d2a5e10804231709v1d7e426fwf68b3d316a15081a@mail.gmail.com","threadId":"13236","inReplyTo":"20080423231359.GA30913@fox","subject":"Re: Cleaning the .git directory with gc","fromName":"Russ Dill","fromEmail":"russ.dill@gmail.com","sentAt":"2008-04-24T00:09:40Z","receivedAt":"2008-04-24T00:09:40Z","isPatch":false,"sender":{"key":"russ.dill@gmail.com","avatar":"https://gravatar.com/avatar/989b24f3fa63126a35d7c74069e2626e715a3b1a86616be9962f7ceae04ff9c5?d=mp&s=160"},"body":"On Wed, Apr 23, 2008 at 4:13 PM, Haakon Riiser wrote:\n> I've recently started using git, and while experimenting with\n> git commit --amend, I noticed that git gc does not do what I\n> expected. Example:\n\nThats a lot of work without first reading the man page:\n\n --prune\n Usually git-gc packs refs, expires old reflog entries, packs loose\n objects, and removes old rerere records. Removal of unreferenced\n loose objects is an unsafe operation while other git operations are\n in progress, so it is not done by default. Pass this option if you\n want it, and only when you know nobody else is creating new objects\n in the repository at the same time (e.g. never use this option in a\n cron script).\n"},{"id":"75120","messageId":"e1dab3980804231732x29d6d73cudd0568a910642639@mail.gmail.com","threadId":"13236","inReplyTo":"f9d2a5e10804231709v1d7e426fwf68b3d316a15081a@mail.gmail.com","subject":"Re: Cleaning the .git directory with gc","fromName":"David Tweed","fromEmail":"david.tweed@gmail.com","sentAt":"2008-04-24T00:32:20Z","receivedAt":"2008-04-24T00:32:20Z","isPatch":false,"sender":{"key":"david.tweed@gmail.com","avatar":null},"body":"On Thu, Apr 24, 2008 at 1:09 AM, Russ Dill wrote:\n> On Wed, Apr 23, 2008 at 4:13 PM, Haakon Riiser wrote:\n> > I've recently started using git, and while experimenting with\n> > git commit --amend, I noticed that git gc does not do what I\n> > expected. Example:\n>\n> Thats a lot of work without first reading the man page:\n>\n> --prune\n[snip]\n\nThere's a relatively recent change in this area. Git keeps stuff\nthat's apparently unattached for a period of, by default, 2 weeks\n(determined by gc.pruneexpire variable) after which a git gc will\nremove it. The reasoning is that even with the careful design of the\ngit updating strategy there are rare times when with a concurrent\nother git process there are files in the repo that look unattached but\nwill become attached as the other process completes. Files kept this\nway aren't propagated by clones or pulls so they're essentially\ninvisible to everything else. If you're sure you can force removal\nwith\n\ngit prune --expire now\n\nAFAICS there's no way to call \"git gc --prune\" with an --expire option\nso you've got to use the \"git prune\" command.\n\nHTH\n\n-- \ncheers, dave tweed__________________________\ndavid.tweed@gmail.com\nRm 124, School of Systems Engineering, University of Reading.\n\"while having code so boring anyone can maintain it, use Python.\" --\nattempted insult seen on slashdot\n"},{"id":"75121","messageId":"20080424005013.GP29771@spearce.org","threadId":"13236","inReplyTo":"f9d2a5e10804231709v1d7e426fwf68b3d316a15081a@mail.gmail.com","subject":"Re: Cleaning the .git directory with gc","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-04-24T00:50:13Z","receivedAt":"2008-04-24T00:50:13Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Russ Dill wrote:\n> On Wed, Apr 23, 2008 at 4:13 PM, Haakon Riiser wrote:\n> > I've recently started using git, and while experimenting with\n> > git commit --amend, I noticed that git gc does not do what I\n> > expected. Example:\n> \n> Thats a lot of work without first reading the man page:\n> \n> --prune\n> Usually git-gc packs refs, expires old reflog entries, packs loose\n> objects, and removes old rerere records. Removal of unreferenced\n> loose objects is an unsafe operation while other git operations are\n> in progress, so it is not done by default. Pass this option if you\n> want it, and only when you know nobody else is creating new objects\n> in the repository at the same time (e.g. never use this option in a\n> cron script).\n\nBut even with `git gc --prune` the old commit object will still\nbe in your repository.\n\nWhy? Both HEAD and your branch's reflog have a reference to the\nold commit. And those will remain in there for 90 days by default,\nso that you could always go back and get that if you _really_\nhad to recover it. Take a look with `git reflog show HEAD`\nor `git log -g` and you'll see what I mean.\n\nA commit is peanuts when it comes to disk space. Don't worry\nabout it. After a lot of amends and such you will be carrying\naround only a few extra MBs. In return for those few extra MBs\nyou are always able to recovery anything, up to 3 months back.\n\nIf you _really_ need to whack all of that away, make a clone\nand then discard the old one, e.g.:\n\n\tgit clone file://`pwd`/old_proj new_proj\n\nNote you need to use the file:// URI syntax to prevent Git from\njust hardlinking everything. It takes a little longer, but the\nresulting new_proj will be cruft free.\n\n-- \nShawn.\n"},{"id":"75122","messageId":"20080424005744.GR29771@spearce.org","threadId":"13236","inReplyTo":"e1dab3980804231732x29d6d73cudd0568a910642639@mail.gmail.com","subject":"Re: Cleaning the .git directory with gc","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-04-24T00:57:44Z","receivedAt":"2008-04-24T00:57:44Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"David Tweed wrote:\n> On Thu, Apr 24, 2008 at 1:09 AM, Russ Dill wrote:\n> > On Wed, Apr 23, 2008 at 4:13 PM, Haakon Riiser wrote:\n> > > I've recently started using git, and while experimenting with\n> > > git commit --amend, I noticed that git gc does not do what I\n> > > expected. Example:\n> >\n> > Thats a lot of work without first reading the man page:\n> >\n> > --prune\n> [snip]\n> \n> There's a relatively recent change in this area. Git keeps stuff\n> that's apparently unattached for a period of, by default, 2 weeks\n> (determined by gc.pruneexpire variable) after which a git gc will\n> remove it. The reasoning is that even with the careful design of the\n> git updating strategy there are rare times when with a concurrent\n> other git process there are files in the repo that look unattached but\n> will become attached as the other process completes.\n\nAlthough that's certainly true, the original poster was asking about\n`git commit --amend`. In such a case the reflog for HEAD and the\ncurrent branch are going to anchor the old commit for the reflog\nexpire period, which is 90 days. Way longer than the 2 week aging\nof loose objects.\n\n-- \nShawn.\n"},{"id":"75161","messageId":"20080424211458.GA8817@fox","threadId":"13236","inReplyTo":"20080424005013.GP29771@spearce.org","subject":"Re: Cleaning the .git directory with gc","fromName":"Haakon Riiser","fromEmail":"haakon.riiser@fys.uio.no","sentAt":"2008-04-24T21:14:58Z","receivedAt":"2008-04-24T21:14:58Z","isPatch":false,"sender":{"key":"haakon.riiser@fys.uio.no","avatar":null},"body":"[Shawn O. Pearce]\n\n> [...] \n> If you _really_ need to whack all of that away, make a clone\n> and then discard the old one, e.g.:\n> \n> \tgit clone file://`pwd`/old_proj new_proj\n> \n> Note you need to use the file:// URI syntax to prevent Git from\n> just hardlinking everything. It takes a little longer, but the\n> resulting new_proj will be cruft free.\n\nThanks for answering my question so quickly. I really wasn't that\nworried about it, but seemed strange that the --amend option only\nseemed to affect the logs. Anyway, it's nice to know I can start\nwith a clean slate if I want to by cloning from a file:// path.\n\n-- \n Haakon\n"}]}