{"thread":{"id":"16323","subject":"Re: [RFC PATCH] repack: make repack -a equivalent to repack -A and drop previous -a behavior","startedAt":"2008-11-13T23:22:36Z","lastAt":"2008-11-14T02:22:31Z","messageCount":7,"participants":["Björn Steinbrink","Brandon Casey","Theodore Tso"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"95997","messageId":"7xh1_tFsJkHTjg4Hjm-R4mGSRfYyGFmLI62OMmaNg32f86EbVIYvYrGiYpEGkvooY4pUM1e7CHk@cipher.nrlssc.navy.mil","threadId":"16323","inReplyTo":null,"subject":"[RFC PATCH] repack: make repack -a equivalent to repack -A and drop previous -a behavior","fromName":"Brandon Casey","fromEmail":"casey@nrlssc.navy.mil","sentAt":"2008-11-13T23:22:36Z","receivedAt":"2008-11-13T23:22:36Z","isPatch":true,"sender":{"key":"drafnel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/921167?v=4"},"body":"Once upon a time, repack had only a single option which began with the first\nletter of the alphabet.  Then, a second was created which would repack\nunreachable objects into the newly created pack so that git-gc --auto could\nbe invented.  But, the -a option was still necessary so that it could be\ncalled every now and then to discard the unreachable objects that were being\nrepacked over and over and over into newly generated packs. Later, -A was\nchanged so that instead of repacking the unreachable objects, it ejected\nthem from the pack so that they resided in the object store in loose form,\nto be garbage collected by prune-packed according to normal expiry rules.\n\nAnd so, -a lost its raison d'etre.\n\nSigned-off-by: Brandon Casey <casey@nrlssc.navy.mil>\n---\n\n\nThis is on top of bc/maint-keep-pack\n\n-brandon\n\n\n Documentation/git-repack.txt |   25 ++++++++++++-------------\n git-repack.sh                |    8 ++++----\n 2 files changed, 16 insertions(+), 17 deletions(-)\n\ndiff --git a/Documentation/git-repack.txt b/Documentation/git-repack.txt\nindex aaa8852..d04d5c2 100644\n--- a/Documentation/git-repack.txt\n+++ b/Documentation/git-repack.txt\n@@ -32,21 +32,20 @@ OPTIONS\n \tpack everything referenced into a single pack.\n \tEspecially useful when packing a repository that is used\n \tfor private development and there is no need to worry\n-\tabout people fetching via dumb protocols from it.  Use\n-\twith '-d'.  This will clean up the objects that `git prune`\n-\tleaves behind, but `git fsck --full` shows as\n-\tdangling.\n+\tabout people fetching via dumb protocols from it.  If used\n+\twith '-d' , then any unreachable objects in a previous pack will\n+\tbecome loose, unpacked objects, instead of being left in the\n+\told pack.  Unreachable objects are never intentionally added to\n+\ta pack, even when repacking.  This option prevents unreachable\n+\tobjects from being immediately deleted by way of being left in\n+\tthe old pack and then removed.  Instead, the loose unreachable\n+\tobjects will be pruned according to normal expiry rules\n+\twith the next 'git-gc' invocation. See linkgit:git-gc[1].\n \n -A::\n-\tSame as `-a`, unless '-d' is used.  Then any unreachable\n-\tobjects in a previous pack become loose, unpacked objects,\n-\tinstead of being left in the old pack.  Unreachable objects\n-\tare never intentionally added to a pack, even when repacking.\n-\tThis option prevents unreachable objects from being immediately\n-\tdeleted by way of being left in the old pack and then\n-\tremoved.  Instead, the loose unreachable objects\n-\twill be pruned according to normal expiry rules\n-\twith the next 'git-gc' invocation. See linkgit:git-gc[1].\n+\tSame as `-a`. Historical note: the -a and -A options used to differ\n+\tin that -a did not leave unreachable objects unpacked.  Instead,\n+\tthey were removed along with the redundant pack (when -d was used).\n \n -d::\n \tAfter packing, if the newly created packs make some\ndiff --git a/git-repack.sh b/git-repack.sh\nindex 458a497..f1e21b9 100755\n--- a/git-repack.sh\n+++ b/git-repack.sh\n@@ -7,8 +7,9 @@ OPTIONS_KEEPDASHDASH=\n OPTIONS_SPEC=\"\\\n git repack [options]\n --\n-a               pack everything in a single pack\n-A               same as -a, and turn unreachable objects loose\n+a               pack everything in a single pack, and turn unreachable objects\n+                loose\n+A               same as -a\n d               remove redundant packs, and run git-prune-packed\n f               pass --no-reuse-object to git-pack-objects\n n               do not run git-update-server-info\n@@ -29,8 +30,7 @@ while test $# != 0\n do\n \tcase \"$1\" in\n \t-n)\tno_update_info=t ;;\n-\t-a)\tall_into_one=t ;;\n-\t-A)\tall_into_one=t\n+\t-a|-A)\tall_into_one=t\n \t\tunpack_unreachable=--unpack-unreachable ;;\n \t-d)\tremove_redundant=t ;;\n \t-q)\tquiet=-q ;;\n-- \n1.6.0.3.552.g12334\n"},{"id":"95745","messageId":"20081114000219.GB5285@atjola.homenet","threadId":"16323","inReplyTo":"7xh1_tFsJkHTjg4Hjm-R4mGSRfYyGFmLI62OMmaNg32f86EbVIYvYrGiYpEGkvooY4pUM1e7CHk@cipher.nrlssc.navy.mil","subject":"Re: [RFC PATCH] repack: make repack -a equivalent to repack -A and drop previous -a behavior","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2008-11-14T00:02:19Z","receivedAt":"2008-11-14T00:02:19Z","isPatch":true,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"On 2008.11.13 17:22:36 -0600, Brandon Casey wrote:\n> Once upon a time, repack had only a single option which began with the first\n> letter of the alphabet.  Then, a second was created which would repack\n> unreachable objects into the newly created pack so that git-gc --auto could\n> be invented.  But, the -a option was still necessary so that it could be\n> called every now and then to discard the unreachable objects that were being\n> repacked over and over and over into newly generated packs. Later, -A was\n> changed so that instead of repacking the unreachable objects, it ejected\n> them from the pack so that they resided in the object store in loose form,\n> to be garbage collected by prune-packed according to normal expiry rules.\n> \n> And so, -a lost its raison d'etre.\n> \n> Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>\n> ---\n> \n> \n> This is on top of bc/maint-keep-pack\n\nI didn't check all the (proposed) commits for that branch, so just let\nme know if I'm missing anything, but doesn't this change mean that you\njust lose what \"-ad\" did?\n\nWe have:\n\t-a\tCreate a new pack, containing all reachable objects\n\t-A\tSame as -a\n\t-ad\tSame as -a, and drop all old packs and loose objects\n\t-Ad\tSama as -ad, but keep unreachable objects loose\n\n-Ad is nice regarding it's safety-net value, but eg. after a large\nfilter-branch run, when refs/original and the reflogs have been cleaned,\nyou just want to get rid of all those old unreachable objects,\nimmediately. For example after importing and massaging some large\nhistory from SVN, the -Ad behaviour is definitely _not_ what I want\nthere. Writing a few thousand loose objects just to prune them is just a\nwaste of time.\n\nBjörn\n"},{"id":"95752","messageId":"jKWdt94ZxgNW0UAgUUW-qjTtpWohpQXMfvw-AUmOXND8SD5yFw0N8w@cipher.nrlssc.navy.mil","threadId":"16323","inReplyTo":"20081114000219.GB5285@atjola.homenet","subject":"Re: [RFC PATCH] repack: make repack -a equivalent to repack -A and drop previous -a behavior","fromName":"Brandon Casey","fromEmail":"casey@nrlssc.navy.mil","sentAt":"2008-11-14T00:53:29Z","receivedAt":"2008-11-14T00:53:29Z","isPatch":true,"sender":{"key":"drafnel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/921167?v=4"},"body":"Björn Steinbrink wrote:\n> On 2008.11.13 17:22:36 -0600, Brandon Casey wrote:\n>> Once upon a time, repack had only a single option which began with the first\n>> letter of the alphabet.  Then, a second was created which would repack\n>> unreachable objects into the newly created pack so that git-gc --auto could\n>> be invented.  But, the -a option was still necessary so that it could be\n>> called every now and then to discard the unreachable objects that were being\n>> repacked over and over and over into newly generated packs. Later, -A was\n>> changed so that instead of repacking the unreachable objects, it ejected\n>> them from the pack so that they resided in the object store in loose form,\n>> to be garbage collected by prune-packed according to normal expiry rules.\n>>\n>> And so, -a lost its raison d'etre.\n>>\n>> Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>\n>> ---\n>>\n>>\n>> This is on top of bc/maint-keep-pack\n> \n> I didn't check all the (proposed) commits for that branch, so just let\n> me know if I'm missing anything, but doesn't this change mean that you\n> just lose what \"-ad\" did?\n\nyes.\n\n> We have:\n> \t-a\tCreate a new pack, containing all reachable objects\n> \t-A\tSame as -a\n> \t-ad\tSame as -a, and drop all old packs and loose objects\n\nby loose objects, I assume you mean packed unreachable objects.\n\n> \t-Ad\tSama as -ad, but keep unreachable objects loose\n> \n> -Ad is nice regarding it's safety-net value, but eg. after a large\n> filter-branch run, when refs/original and the reflogs have been cleaned,\n> you just want to get rid of all those old unreachable objects,\n> immediately. For example after importing and massaging some large\n> history from SVN, the -Ad behaviour is definitely _not_ what I want\n> there. Writing a few thousand loose objects just to prune them is just a\n> waste of time.\n\nhmm. That's a good point. Even though I think it is likely that the thousand\nloose objects that are written will be small commit objects and not blobs,\nthis use case may be enough to trump the safety benefit provided by the\nproposed change.\n\n-brandon\n"},{"id":"95755","messageId":"20081114012502.GC5285@atjola.homenet","threadId":"16323","inReplyTo":"jKWdt94ZxgNW0UAgUUW-qjTtpWohpQXMfvw-AUmOXND8SD5yFw0N8w@cipher.nrlssc.navy.mil","subject":"Re: [RFC PATCH] repack: make repack -a equivalent to repack -A and drop previous -a behavior","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2008-11-14T01:25:02Z","receivedAt":"2008-11-14T01:25:02Z","isPatch":true,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"On 2008.11.13 18:53:29 -0600, Brandon Casey wrote:\n> Björn Steinbrink wrote:\n> > I didn't check all the (proposed) commits for that branch, so just let\n> > me know if I'm missing anything, but doesn't this change mean that you\n> > just lose what \"-ad\" did?\n> \n> yes.\n> \n> > We have:\n> > \t-a\tCreate a new pack, containing all reachable objects\n> > \t-A\tSame as -a\n> > \t-ad\tSame as -a, and drop all old packs and loose objects\n> \n> by loose objects, I assume you mean packed unreachable objects.\n\nNo, actually I just totally ignored the fact that -a of course already\ndeletes the loose objects. The packed unreachable objects are in the old\npacks, so they're already included in the first half of my sentence ;-)\n\n> > \t-Ad\tSama as -ad, but keep unreachable objects loose\n> > \n> > -Ad is nice regarding it's safety-net value, but eg. after a large\n> > filter-branch run, when refs/original and the reflogs have been cleaned,\n> > you just want to get rid of all those old unreachable objects,\n> > immediately. For example after importing and massaging some large\n> > history from SVN, the -Ad behaviour is definitely _not_ what I want\n> > there. Writing a few thousand loose objects just to prune them is just a\n> > waste of time.\n> \n> hmm. That's a good point. Even though I think it is likely that the thousand\n> loose objects that are written will be small commit objects and not blobs,\n\nWhen you only fix up merge commits, author information and such things,\nthen yes, most objects will be commits. And then it's not even that bad.\n\nBut a more interesting case is when in your old SCM you had multiple\nprojects in one repo, and you can't sanely separate them before the\nimport. So you might end up using the subdirectory filter a few times,\nor even just drop a bunch of branches in each copy of your import.\n\nAnd another one is when you had accidently commited some huge, useless\nfiles, and as you're switching to git now anyway, you want to get rid of\nthem, so you use an index-filter to drop them.\n\nFor those two cases, -Ad vs -ad can make a huge difference. I remember\nsomeone on #git using a subdirectory filter on some project and trying\nto get the repo to a sane size afterwards. -Ad took basically forever,\nwhile -ad finished in 5 seconds or so.\n\n> this use case may be enough to trump the safety benefit provided by the\n> proposed change.\n\nIMHO, \"git gc\" already provides enough safety. I tend to see \"gc\" as the\nregular \"just use it\" tool, while repack gives me more control over how\nI want things to be done, without forcing me to use the real plumbing or\nto fumble around with the configuration for gc. And when I want control,\nI'm generally prepared to shoot myself in the foot.\n\nBjörn\n"},{"id":"95756","messageId":"Dei99j9xFoGAh0A3LCO0zx3LqCb4lViSZpGvv_KGn38@cipher.nrlssc.navy.mil","threadId":"16323","inReplyTo":"20081114012502.GC5285@atjola.homenet","subject":"Re: [RFC PATCH] repack: make repack -a equivalent to repack -A and drop previous -a behavior","fromName":"Brandon Casey","fromEmail":"casey@nrlssc.navy.mil","sentAt":"2008-11-14T01:36:45Z","receivedAt":"2008-11-14T01:36:45Z","isPatch":true,"sender":{"key":"drafnel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/921167?v=4"},"body":"Björn Steinbrink wrote:\n> On 2008.11.13 18:53:29 -0600, Brandon Casey wrote:\n>> Björn Steinbrink wrote:\n>>> I didn't check all the (proposed) commits for that branch, so just let\n>>> me know if I'm missing anything, but doesn't this change mean that you\n>>> just lose what \"-ad\" did?\n>> yes.\n>>\n>>> We have:\n>>> \t-a\tCreate a new pack, containing all reachable objects\n>>> \t-A\tSame as -a\n>>> \t-ad\tSame as -a, and drop all old packs and loose objects\n>> by loose objects, I assume you mean packed unreachable objects.\n> \n> No, actually I just totally ignored the fact that -a of course already\n> deletes the loose objects.\n\nActually, I had forgotten that repack deletes any loose objects at all.\nIt does call prune-packed, but only when -d is used.\n\n> IMHO, \"git gc\" already provides enough safety. I tend to see \"gc\" as the\n> regular \"just use it\" tool, while repack gives me more control over how\n> I want things to be done, without forcing me to use the real plumbing or\n> to fumble around with the configuration for gc. And when I want control,\n> I'm generally prepared to shoot myself in the foot.\n\nI think you're right. Thanks for providing an example of a real use case.\n\n-brandon\n"},{"id":"95757","messageId":"20081114014828.GD5285@atjola.homenet","threadId":"16323","inReplyTo":"Dei99j9xFoGAh0A3LCO0zx3LqCb4lViSZpGvv_KGn38@cipher.nrlssc.navy.mil","subject":"Re: [RFC PATCH] repack: make repack -a equivalent to repack -A and drop previous -a behavior","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2008-11-14T01:48:28Z","receivedAt":"2008-11-14T01:48:28Z","isPatch":true,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"On 2008.11.13 19:36:45 -0600, Brandon Casey wrote:\n> Björn Steinbrink wrote:\n> > On 2008.11.13 18:53:29 -0600, Brandon Casey wrote:\n> >> Björn Steinbrink wrote:\n> >>> We have:\n> >>> \t-a\tCreate a new pack, containing all reachable objects\n> >>> \t-A\tSame as -a\n> >>> \t-ad\tSame as -a, and drop all old packs and loose objects\n> >> by loose objects, I assume you mean packed unreachable objects.\n> > \n> > No, actually I just totally ignored the fact that -a of course already\n> > deletes the loose objects.\n> \n> Actually, I had forgotten that repack deletes any loose objects at all.\n> It does call prune-packed, but only when -d is used.\n\nUgh, right. -a does not delete loose objects without -d. So, ignoring\nthe .keep stuff, my initial description was even right and I just\nconfused myself afterwards :-/\n\nThanks,\nBjörn\n"},{"id":"95759","messageId":"20081114022231.GB25117@mit.edu","threadId":"16323","inReplyTo":"jKWdt94ZxgNW0UAgUUW-qjTtpWohpQXMfvw-AUmOXND8SD5yFw0N8w@cipher.nrlssc.navy.mil","subject":"Re: [RFC PATCH] repack: make repack -a equivalent to repack -A and drop previous -a behavior","fromName":"Theodore Tso","fromEmail":"tytso@mit.edu","sentAt":"2008-11-14T02:22:31Z","receivedAt":"2008-11-14T02:22:31Z","isPatch":true,"sender":{"key":"tytso@mit.edu","avatar":"https://avatars.githubusercontent.com/u/51416?v=4"},"body":"On Thu, Nov 13, 2008 at 06:53:29PM -0600, Brandon Casey wrote:\n> > -Ad is nice regarding it's safety-net value, but eg. after a large\n> > filter-branch run, when refs/original and the reflogs have been cleaned,\n> > you just want to get rid of all those old unreachable objects,\n> > immediately. For example after importing and massaging some large\n> > history from SVN, the -Ad behaviour is definitely _not_ what I want\n> > there. Writing a few thousand loose objects just to prune them is just a\n> > waste of time.\n> \n> hmm. That's a good point. Even though I think it is likely that the thousand\n> loose objects that are written will be small commit objects and not blobs,\n> this use case may be enough to trump the safety benefit provided by the\n> proposed change.\n\nThe problem is even small commit objects take a full 4k (or whatever\nyour filesystem block size is) when they are ejected as loose objects.\nAs a result, the current \"git gc\" defaults can end up requiring far\n*more* disk space than before, certainly while it is running, and\nsometimes even after the \"git gc\" completes.  (I then end up running\n\"git prune\" to complete deletion of the ejected objects.)\n\nSometimes this gets so annoying that I'll run the individual commands\nrun by git-gc by hand, except I use git repack -ad instead of git\nrepack -A.  If we are going to get rid of the distinction between git\nrepack -a and git repack -A, perhaps there can be a config option to\nforce the immediate ejection of the unreachable objects, instead of\ncreating loose objects?  \n\nIf the goal is safety, it would be nice if git repack could create a\nseparate pack that only contained unreachable objects, and then have\ngit prune be able to remove a pack if it only contains unreachable\nobjects.\n\n\t\t\t\t\t\t- Ted\n"}]}