{"thread":{"id":"17776","subject":"[PATCH] Documentation: pruning recipe for destructive filter-branch","startedAt":"2009-02-13T17:14:39Z","lastAt":"2009-02-15T19:33:37Z","messageCount":17,"participants":["Thomas Rast","Junio C Hamano","Jan Krüger","Johannes Schindelin"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"104529","messageId":"1234545279-23153-1-git-send-email-trast@student.ethz.ch","threadId":"17776","inReplyTo":null,"subject":"[PATCH] Documentation: pruning recipe for destructive filter-branch","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2009-02-13T17:14:39Z","receivedAt":"2009-02-13T17:14:39Z","isPatch":true,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Add a section about how to shrink a repository's size after running\ngit-filter-branch to remove large blobs from history.\n\nThis comes up every week or so on IRC, and the commands required to\nhandle every case are not very newbie-friendly, so hopefully writing\nthem down somewhere leads to fewer questions.\n\nThanks to doener (Björn Steinbrink) for comments and corrections.\n\nSigned-off-by: Thomas Rast \n---\n\nOr we could just add an option --prune-everything-I-have-a-backup-I-promise\nto git-filter-branch, so that users can get the same result with less effort.\n\n\n Documentation/git-filter-branch.txt | 36 +++++++++++++++++++++++++++++++++++\n 1 files changed, 36 insertions(+), 0 deletions(-)\n\ndiff --git a/Documentation/git-filter-branch.txt b/Documentation/git-filter-branch.txt\nindex 1fbbbb4..737c555 100644\n--- a/Documentation/git-filter-branch.txt\n+++ b/Documentation/git-filter-branch.txt\n@@ -339,6 +339,42 @@ git filter-branch --index-filter \\\n ---------------------------------------------------------------\n \n \n+\n+Checklist for Shrinking a Repository\n+------------------------------------\n+\n+git-filter-branch is often used to get rid of a subset of files,\n+usually with some combination of `\\--index-filter` and\n+`\\--subdirectory-filter`. If you want to physically shrink the\n+repository afterwards, you have some choices:\n+\n+* Clone it with `git clone file:///path/to/repo`. The clone will not\n+ have the removed objects. See linkgit:git-clone[1]. (Note that\n+ cloning with a plain path just hardlinks everything!)\n+\n+If you really don't want to clone it, for whatever reasons, check the\n+following points (in this order). This is a very destructive\n+approach, so *make a backup* or go back to cloning it. You have been\n+warned.\n+\n+* Make sure you really removed all variants of a filename, if a blob\n+ was moved over its lifetime. `git log \\--follow \\--all \\-- foo` can\n+ help you find renames.\n+\n+* Make sure you really filtered all refs: use `\\--tag-name-filter cat\n+ \\-- \\--all` when calling git-filter-branch.\n+\n+* Make sure you remove the original refs backed up by\n+ git-filter-branch: say `git for-each-ref \\--format=\"%(refname)\"\n+ refs/original/ | xargs -n 1 git update-ref -d`.\n+\n+* Expire all reflogs with `git reflog expire \\--expire=now \\--all`.\n+\n+* Repack to remove packed objects with `git repack -ad`.\n+\n+* Prune unpacked unreferenced objects with `git prune \\--expire=now`.\n+\n+\n Author\n ------\n Written by Petr \"Pasky\" Baudis ,\n-- \n1.6.2.rc0.274.g97213\n"},{"id":"104563","messageId":"7vr6214yi5.fsf@gitster.siamese.dyndns.org","threadId":"17776","inReplyTo":"1234545279-23153-1-git-send-email-trast@student.ethz.ch","subject":"Re: [PATCH] Documentation: pruning recipe for destructive filter-branch","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-02-14T01:07:30Z","receivedAt":"2009-02-14T01:07:30Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Thomas Rast writes:\n\n> diff --git a/Documentation/git-filter-branch.txt b/Documentation/git-filter-branch.txt\n> index 1fbbbb4..737c555 100644\n> --- a/Documentation/git-filter-branch.txt\n> +++ b/Documentation/git-filter-branch.txt\n> @@ -339,6 +339,42 @@ git filter-branch --index-filter \\\n> ---------------------------------------------------------------\n> \n> \n> +\n> +Checklist for Shrinking a Repository\n> +------------------------------------\n> +\n> +git-filter-branch is often used to get rid of a subset of files,\n> +usually with some combination of `\\--index-filter` and\n> +`\\--subdirectory-filter`. If you want to physically shrink the\n> +repository afterwards, you have some choices:\n> +\n> +* Clone it with `git clone file:///path/to/repo`. The clone will not\n> + have the removed objects. See linkgit:git-clone[1]. (Note that\n> + cloning with a plain path just hardlinks everything!)\n> +\n> +If you really don't want to clone it, for whatever reasons, check the\n> +following points (in this order). This is a very destructive\n> +approach, so *make a backup* or go back to cloning it. You have been\n> +warned.\n\nSounds sane so far and even somewhat amusing.\n\n> +* Make sure you really removed all variants of a filename, if a blob\n> + was moved over its lifetime. `git log \\--follow \\--all \\-- foo` can\n> + help you find renames.\n> +\n> +* Make sure you really filtered all refs: use `\\--tag-name-filter cat\n> + \\-- \\--all` when calling git-filter-branch.\n\nDon't these two apply equally to \"git clone\" approach? Only the remaining\nfour are specific to in-place pruning, if I am reading this correctly.\n\n> +* Make sure you remove the original refs backed up by\n> + git-filter-branch: say `git for-each-ref \\--format=\"%(refname)\"\n> + refs/original/ | xargs -n 1 git update-ref -d`.\n> +\n> +* Expire all reflogs with `git reflog expire \\--expire=now \\--all`.\n> +\n> +* Repack to remove packed objects with `git repack -ad`.\n> +\n> +* Prune unpacked unreferenced objects with `git prune \\--expire=now`.\n> +\n> +\n> Author\n> ------\n> Written by Petr \"Pasky\" Baudis ,\n> -- \n> 1.6.2.rc0.274.g97213\n>\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at http://vger.kernel.org/majordomo-info.html\n"},{"id":"104564","messageId":"20090214025115.615119bf@perceptron","threadId":"17776","inReplyTo":"1234545279-23153-1-git-send-email-trast@student.ethz.ch","subject":"Re: [PATCH] Documentation: pruning recipe for destructive filter-branch","fromName":"Jan Krüger","fromEmail":"jk@jk.gs","sentAt":"2009-02-14T01:51:15Z","receivedAt":"2009-02-14T01:51:15Z","isPatch":true,"sender":{"key":"jk@jk.gs","avatar":"https://avatars.githubusercontent.com/u/1774?v=4"},"body":"Hi,\n\nOn Fri, 13 Feb 2009 18:14:39 +0100, Thomas Rast \nwrote:\n\n> Add a section about how to shrink a repository's size after running\n> git-filter-branch to remove large blobs from history.\n\nGood idea.\n\n> +* Expire all reflogs with `git reflog expire \\--expire=now \\--all`.\n> +\n> +* Repack to remove packed objects with `git repack -ad`.\n> +\n> +* Prune unpacked unreferenced objects with `git prune \\--expire=now`.\n\nIs the --expire=now actually needed for prune? The way I read it\ngit-prune(1), it defaults to pruning everything anyway.\n\n-Jan\n"},{"id":"104591","messageId":"alpine.DEB.1.00.0902140642520.10279@pacific.mpi-cbg.de","threadId":"17776","inReplyTo":"20090214025115.615119bf@perceptron","subject":"Re: [PATCH] Documentation: pruning recipe for destructive filter-branch","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2009-02-14T05:46:18Z","receivedAt":"2009-02-14T05:46:18Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Sat, 14 Feb 2009, Jan Krüger wrote:\n\n> On Fri, 13 Feb 2009 18:14:39 +0100, Thomas Rast \n> wrote:\n> \n> > Add a section about how to shrink a repository's size after running\n> > git-filter-branch to remove large blobs from history.\n> \n> Good idea.\n\nIndeed. It is funny when some people ask, but reading it several times a \nweek in #git makes that fun go away real quick.\n\n> > +* Expire all reflogs with `git reflog expire \\--expire=now \\--all`.\n> > +\n> > +* Repack to remove packed objects with `git repack -ad`.\n> > +\n> > +* Prune unpacked unreferenced objects with `git prune \\--expire=now`.\n> \n> Is the --expire=now actually needed for prune? The way I read it\n> git-prune(1), it defaults to pruning everything anyway.\n\nI hope not. IIRC I set the default to 2 weeks.\n\n*goes to read the source* Ah, that default is only set in \"git gc\", and \n\"git prune\" still prunes everything. So you're absolutely correct.\n\nBut maybe we do not want to direct the user to use \"prune\" here, but \nrather \"gc\"?\n\nCiao,\nDscho\n"},{"id":"104601","messageId":"20090214074954.7e423dd2@perceptron","threadId":"17776","inReplyTo":"alpine.DEB.1.00.0902140642520.10279@pacific.mpi-cbg.de","subject":"Re: [PATCH] Documentation: pruning recipe for destructive filter-branch","fromName":"Jan Krüger","fromEmail":"jk@jk.gs","sentAt":"2009-02-14T06:49:54Z","receivedAt":"2009-02-14T06:49:54Z","isPatch":true,"sender":{"key":"jk@jk.gs","avatar":"https://avatars.githubusercontent.com/u/1774?v=4"},"body":"Hi,\n\nOn Sat, 14 Feb 2009 06:46:18 +0100 (CET), Johannes Schindelin\n wrote:\n\n> > Is the --expire=now actually needed for prune? The way I read it\n> > git-prune(1), it defaults to pruning everything anyway.\n> \n> [...] Ah, that default is only set in \"git gc\", and \"git prune\" still\n> prunes everything. [...] But maybe we do not want to direct the user\n> to use \"prune\" here, but rather \"gc\"?\n\nI just read the source code of gc and noticed that the only way to\ninfluence its expiration limit for prune is to set it in the config file\n(gc.pruneExpire). Thus there is no immediate way to tell gc to prune\n*all* unreachable objects and we need to stick to the instructions that\nare in the patch (minus the unnecessary --expire=now for prune). The\nalternative would be to add a command line option to gc that\ncorresponds to gc.pruneExpire. I don't really care about that, but a\npatch for it is below.\n\n-Jan\n\n-----8<-----\n\nFrom 42226c6c542be4e9fff0817662115f180033e75a Mon Sep 17 00:00:00 2001\nFrom: =?utf-8?q?Jan=20Kr=C3=BCger?= \nDate: Sat, 14 Feb 2009 07:39:23 +0100\nSubject: [PATCH] gc: add --prune-expire option equivalent to gc.pruneExpire\nMIME-Version: 1.0\nContent-Type: text/plain; charset=utf-8\nContent-Transfer-Encoding: 8bit\n\nIn semi-regular intervals, people tend to want to get rid of huge\nobjects they accidentally created in a repository. Of course it\nshouldn't be too easy to do but also not too hard. Thus we allow them to\nget rid of unreferenced objects more easily once they have managed to\nmake the objects unreferenced, by allowing them to override the prune\nexpiration limit.\n\nSigned-off-by: Jan Krüger \n---\n Documentation/git-gc.txt | 5 +++++\n builtin-gc.c | 5 +++--\n 2 files changed, 8 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex 7086eea..a4d35fb 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -59,6 +59,11 @@ are consolidated into a single pack by using the `-A` option of\n 'git-repack'. Setting `gc.autopacklimit` to 0 disables\n automatic consolidation of packs.\n \n+--prune-expire