{"thread":{"id":"8608","subject":"git-repack made my pack 317x larger...","startedAt":"2007-06-15T08:50:57Z","lastAt":"2007-06-15T15:53:25Z","messageCount":5,"participants":["linux@horizon.com","Nicolas Pitre","Benjamin Sergeant"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"45126","messageId":"20070615085057.5706.qmail@science.horizon.com","threadId":"8608","inReplyTo":null,"subject":"git-repack made my pack 317x larger...","fromName":"","fromEmail":"linux@horizon.com","sentAt":"2007-06-15T08:50:57Z","receivedAt":"2007-06-15T08:50:57Z","isPatch":false,"sender":{"key":"linux@horizon.com","avatar":null},"body":"I was grabbing a copy of the hwmon-2.6 git tree to play with:\n\n> ~/tmp$ git-clone git://lm-sensors.org/kernel/mhoffman/hwmon-2.6.git\n> Initialized empty Git repository in /home/linux/tmp/hwmon-2.6/.git/\n> remote: Generating pack...\n> remote: Done counting 496311 objects.\n> remote: Deltifying 496311 objects.\n> remote:  100% (496311/496311) done\n> Indexing 496311 objects...\n> remote: Total 496311 (delta 400999), reused 496151 (delta 400839)\n>  100% (496311/496311) done\n> Resolving 400999 deltas...\n>  100% (400999/400999) done\n> Checking 22409 files out...\n>  100% (22409/22409) done\n> ~/tmp$ ls -l hwmon-2.6/git/objects/pack\n> -r--r--r-- 1 linux users  11912528 Jun 15 03:35 pack-d4ec0478f7530345ff466805ab4fe63efc9251df.idx\n> -r--r--r-- 1 linux users 166721991 Jun 15 03:35 pack-d4ec0478f7530345ff466805ab4fe63efc9251df.pack\n\nThen I noticed that it's a Linux kernel... oops!\nSorry for wasting the network bandwidth, but I can save myself\nthe disk space.\n\n> ~/tmp$ rm -r hwmon-2.6/*\n> ~/tmp$ mv hwmon-2.6 hwmon-2.6.old\n> ~/tmp$ git clone --reference /usr/src/linux -n hwmon-2.6.old hwmon-2.6\n> Initialized empty Git repository in /home/linux/tmp/hwmon-2.6/.git/\n> remote: Generating pack...\n> remote: Done counting 124 objects.\n> remote: Deltifying 124 objects...\n> remote:  100% (124/124) done\n> Indexing 124 objects...\n> remote: Total 124 (delta 83), reused 97 (delta 75)\n>  100% (124/124) done\n> Resolving 83 deltas...\n>  100% (83/83) done\n> ~/tmp$ rm -rf hwmon-2.6.olf\n> ~/tmp$ cd hwmon-2.6\n> ~/tmp/hwmon-2.6$ ls -l .git/objects/pack/\n> total 168\n> -r--r--r-- 1 linux users   4040 Jun 15 03:44 pack-e4de475aa6c82099b4a4a8f6f410dcc316c7cf19.idx\n> -r--r--r-- 1 linux users 161548 Jun 15 03:44 pack-e4de475aa6c82099b4a4a8f6f410dcc316c7cf19.pack\n\nMuch better!  But since that's only a few objects, let's repack them even tighter...\n\n> ~/tmp/hwmon-2.6$ git-repack -a -d -f --window=50\n> Generating pack...\n> Done counting 18090 objects.\n> Deltifying 18090 objects...\n>  100% (18090/18090) done\n> Writing 18090 objects...\n>  100% (18090/18090) done\n> Total 18090 (delta 2465), reused 7876 (delta 0)\n> Pack pack-1af4ab6620f71733f17b16d69a89b6741e2c7fe3 created.\n> Removing unused objects 100%...\n> Done.\n> ~/tmp/hwmon-2.6$ ls -l .git/objects/pack/\n> total 50620\n> -r--r--r-- 1 linux users   435224 Jun 15 04:00 pack-1af4ab6620f71733f17b16d69a89b6741e2c7fe3.idx\n> -r--r--r-- 1 linux users 51333891 Jun 15 04:00 pack-1af4ab6620f71733f17b16d69a89b6741e2c7fe3.pack\n\nUh... what happened?  It's not a full kernel clone, but it's a lot more\nobjects than I expected.  Where did all the extra objects come from?\n"},{"id":"45132","messageId":"alpine.LFD.0.99.0706150935530.5651@xanadu.home","threadId":"8608","inReplyTo":"20070615085057.5706.qmail@science.horizon.com","subject":"Re: git-repack made my pack 317x larger...","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2007-06-15T13:39:19Z","receivedAt":"2007-06-15T13:39:19Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Fri, 15 Jun 2007, linux@horizon.com wrote:\n\n> > ~/tmp$ git clone --reference /usr/src/linux -n hwmon-2.6.old hwmon-2.6\n> > Initialized empty Git repository in /home/linux/tmp/hwmon-2.6/.git/\n> > remote: Generating pack...\n> > remote: Done counting 124 objects.\n> > remote: Deltifying 124 objects...\n> > remote:  100% (124/124) done\n> > Indexing 124 objects...\n> > remote: Total 124 (delta 83), reused 97 (delta 75)\n> >  100% (124/124) done\n> > Resolving 83 deltas...\n> >  100% (83/83) done\n> > ~/tmp$ rm -rf hwmon-2.6.olf\n> > ~/tmp$ cd hwmon-2.6\n> > ~/tmp/hwmon-2.6$ ls -l .git/objects/pack/\n> > total 168\n> > -r--r--r-- 1 linux users   4040 Jun 15 03:44 pack-e4de475aa6c82099b4a4a8f6f410dcc316c7cf19.idx\n> > -r--r--r-- 1 linux users 161548 Jun 15 03:44 pack-e4de475aa6c82099b4a4a8f6f410dcc316c7cf19.pack\n> \n> Much better!  But since that's only a few objects, let's repack them even tighter...\n> \n> > ~/tmp/hwmon-2.6$ git-repack -a -d -f --window=50\n> > Generating pack...\n> > Done counting 18090 objects.\n> > Deltifying 18090 objects...\n> >  100% (18090/18090) done\n> > Writing 18090 objects...\n> >  100% (18090/18090) done\n> > Total 18090 (delta 2465), reused 7876 (delta 0)\n> > Pack pack-1af4ab6620f71733f17b16d69a89b6741e2c7fe3 created.\n> > Removing unused objects 100%...\n> > Done.\n> > ~/tmp/hwmon-2.6$ ls -l .git/objects/pack/\n> > total 50620\n> > -r--r--r-- 1 linux users   435224 Jun 15 04:00 pack-1af4ab6620f71733f17b16d69a89b6741e2c7fe3.idx\n> > -r--r--r-- 1 linux users 51333891 Jun 15 04:00 pack-1af4ab6620f71733f17b16d69a89b6741e2c7fe3.pack\n> \n> Uh... what happened?  It's not a full kernel clone, but it's a lot more\n> objects than I expected.  Where did all the extra objects come from?\n\nMaybe you want to add -l as well to your git-repack invocation.\n\n\nNicolas\n"},{"id":"45136","messageId":"20070615145433.22970.qmail@science.horizon.com","threadId":"8608","inReplyTo":"alpine.LFD.0.99.0706150935530.5651@xanadu.home","subject":"Re: git-repack made my pack 317x larger...","fromName":"","fromEmail":"linux@horizon.com","sentAt":"2007-06-15T14:54:33Z","receivedAt":"2007-06-15T14:54:33Z","isPatch":false,"sender":{"key":"linux@horizon.com","avatar":null},"body":">> Uh... what happened?  It's not a full kernel clone, but it's a lot more\n>> objects than I expected.  Where did all the extra objects come from?\n\n> Maybe you want to add -l as well to your git-repack invocation.\n\nAh.  Thank you.  Indeed, this is another example of git documentation\ndisease.  git-repack refers to git-pack-objects, which gives a very\ntechnical explanation of what it does, but nowhere is it mentioned that\nlist of objects suppled to git-pack-object's stdin includes objects\nborrowed from alternates.\n\nOf course, reading the description of git-rev-list --objects, you might\nget the impression, but it's not exactly hugely obvious.\n\nGiven that \"git-repack -f\" is a not uncommon command, could I suggest\nthat the default is wrong, and there should be a special flag for\n\"suck in alternates, so this repository is no longer dependent\non any others\".\n\nMentally, git-repack is a \"reduce space consumption\" command, not an\nincrease one.  Having to remember that this repository uses alternates\nand add an extra flag to avoid having a space explosion is distinctly\nannoying.\n\n(I might complan a little less if git-repack would take -adlf rather\nthan insisting on -a -d -l -f.  Is that a deliberate choice or has just\nnobody stepped up to revamp the option parsing?)\n"},{"id":"45138","messageId":"1621f9fa0706150810m1d1bc31ua485db72f6e0bca1@mail.gmail.com","threadId":"8608","inReplyTo":"20070615145433.22970.qmail@science.horizon.com","subject":"Re: git-repack made my pack 317x larger...","fromName":"Benjamin Sergeant","fromEmail":"bsergean@gmail.com","sentAt":"2007-06-15T15:10:09Z","receivedAt":"2007-06-15T15:10:09Z","isPatch":false,"sender":{"key":"bsergean@gmail.com","avatar":null},"body":"> (I might complan a little less if git-repack would take -adlf rather\n> than insisting on -a -d -l -f.  Is that a deliberate choice or has just\n> nobody stepped up to revamp the option parsing?)\n\ngetopt is our friend, indeed.\n\n(http://www.hmug.org/man/1/getopt.php)\n\nThe following code fragment shows how one might process the arguments for\n     a command that can take the options -a and -b, and the option -o, which\n     requires an argument.\n\n           args=`getopt abo: $*`\n           # you should not use `getopt abo: \"$@\"` since that would parse\n           # the arguments differently from what the set command below does.\n           if [ $? != 0 ]\n           then\n                   echo 'Usage: ...'\n                   exit 2\n           fi\n           set -- $args\n           # You cannot use the set command with a backquoted getopt directly,\n           # since the exit code from getopt would be shadowed by those of set,\n           # which is zero by definition.\n           for i\n           do\n                   case \"$i\"\n                   in\n                           -a|-b)\n                                   echo flag $i set; sflags=\"${i#-}$sflags\";\n                                   shift;;\n                           -o)\n                                   echo oarg is \"'\"$2\"'\"; oarg=\"$2\"; shift;\n                                   shift;;\n                           --)\n                                   shift; break;;\n                   esac\n           done\n           echo single-char flags: \"'\"$sflags\"'\"\n           echo oarg is \"'\"$oarg\"'\"\n\n     This code will accept any of the following as equivalent:\n\n           cmd -aoarg file file\n           cmd -a -o arg file file\n           cmd -oarg -a file file\n           cmd -a -oarg -- file file\n"},{"id":"45142","messageId":"alpine.LFD.0.99.0706151143590.5651@xanadu.home","threadId":"8608","inReplyTo":"20070615145433.22970.qmail@science.horizon.com","subject":"Re: git-repack made my pack 317x larger...","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2007-06-15T15:53:25Z","receivedAt":"2007-06-15T15:53:25Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Fri, 15 Jun 2007, linux@horizon.com wrote:\n\n> >> Uh... what happened?  It's not a full kernel clone, but it's a lot more\n> >> objects than I expected.  Where did all the extra objects come from?\n> \n> > Maybe you want to add -l as well to your git-repack invocation.\n> \n> Ah.  Thank you.  Indeed, this is another example of git documentation\n> disease.  git-repack refers to git-pack-objects, which gives a very\n> technical explanation of what it does, but nowhere is it mentioned that\n> list of objects suppled to git-pack-object's stdin includes objects\n> borrowed from alternates.\n\nAt some point it is necessary for people like you who are not so \nintimate with the packing code, and therefore to whom this doesn't look \nobvious, to raise those issues, and ideally provide patches.\n\n> Given that \"git-repack -f\" is a not uncommon command, could I suggest\n> that the default is wrong, and there should be a special flag for\n> \"suck in alternates, so this repository is no longer dependent\n> on any others\".\n\nWell, I tend to disagree here.  I don't think using -f _should_ be that \ncommon.  It is a really expensive operation and you usualy should have a \ngood reason to use it.\n\n> Mentally, git-repack is a \"reduce space consumption\" command, not an\n> increase one.  Having to remember that this repository uses alternates\n> and add an extra flag to avoid having a space explosion is distinctly\n> annoying.\n\nWhy don't you use git-gc then?  Its mental model and actual \nimplementation is really about reducing space, maybe even more than \ngit-repack is, and it does call git-repack with -l.\n\n\nNicolas\n"}]}