{"thread":{"id":"11036","subject":"Removing old data without disturbing tree?","startedAt":"2007-11-27T19:39:55Z","lastAt":"2007-11-27T21:10:06Z","messageCount":3,"participants":["David Brown","Nicolas Pitre"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"61116","messageId":"20071127193955.GA16585@old.davidb.org","threadId":"11036","inReplyTo":null,"subject":"Removing old data without disturbing tree?","fromName":"David Brown","fromEmail":"git@davidb.org","sentAt":"2007-11-27T19:39:55Z","receivedAt":"2007-11-27T19:39:55Z","isPatch":false,"sender":{"key":"git@davidb.org","avatar":"https://gravatar.com/avatar/94c86a2938470a74c2eac5e2b69afc0871f79a660295c02219597aba8cb101c1?d=mp&s=160"},"body":"An upstream tree I'm mirroring with git-p4 has decided to start checking\nin large tarballs (150MB) periodically.  It's basically a prebuild version\nof some firmware needed to run the rest of the software.\n\nGit doesn't seem to have any problem with these tarballs (and is using a\nlot less space than P4), but I have a feeling we might start running into\nproblems when things get real big.  Does anyone have experience with packs\ngrowing beyong several GB?\n\nAside from that, is there an easy way to prune out the old history from a\nworking tree?  I'd like something like what 'git clone --depth n' would\nproduce, and I suppose I could do the clone and then pivot the trees.  I\nmainly don't want to be rewriting history, just making parts inaccessible.\n\nThanks,\nDavid Brown\n"},{"id":"61120","messageId":"alpine.LFD.0.99999.0711271459520.9605@xanadu.home","threadId":"11036","inReplyTo":"20071127193955.GA16585@old.davidb.org","subject":"Re: Removing old data without disturbing tree?","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2007-11-27T20:06:45Z","receivedAt":"2007-11-27T20:06:45Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Tue, 27 Nov 2007, David Brown wrote:\n\n> An upstream tree I'm mirroring with git-p4 has decided to start checking\n> in large tarballs (150MB) periodically.  It's basically a prebuild version\n> of some firmware needed to run the rest of the software.\n> \n> Git doesn't seem to have any problem with these tarballs (and is using a\n> lot less space than P4), but I have a feeling we might start running into\n> problems when things get real big.  Does anyone have experience with packs\n> growing beyong several GB?\n\nIt should just work.  It was tested with artificial data sets but that's \nabout it.\n\nNow if those tarballs are actually multiple revisions of the same \npackage, you might consider storing them uncompressed and let Git delta \ncompress them against each other which will produce an even more \nsignificant space saving.\n\n\nNicolas\n"},{"id":"61122","messageId":"20071127211006.GA22242@old.davidb.org","threadId":"11036","inReplyTo":"alpine.LFD.0.99999.0711271459520.9605@xanadu.home","subject":"Re: Removing old data without disturbing tree?","fromName":"David Brown","fromEmail":"git@davidb.org","sentAt":"2007-11-27T21:10:06Z","receivedAt":"2007-11-27T21:10:06Z","isPatch":false,"sender":{"key":"git@davidb.org","avatar":"https://gravatar.com/avatar/94c86a2938470a74c2eac5e2b69afc0871f79a660295c02219597aba8cb101c1?d=mp&s=160"},"body":"On Tue, Nov 27, 2007 at 03:06:45PM -0500, Nicolas Pitre wrote:\n>On Tue, 27 Nov 2007, David Brown wrote:\n>\n>> An upstream tree I'm mirroring with git-p4 has decided to start checking\n>> in large tarballs (150MB) periodically.  It's basically a prebuild version\n>> of some firmware needed to run the rest of the software.\n>> \n>> Git doesn't seem to have any problem with these tarballs (and is using a\n>> lot less space than P4), but I have a feeling we might start running into\n>> problems when things get real big.  Does anyone have experience with packs\n>> growing beyong several GB?\n>\n>It should just work.  It was tested with artificial data sets but that's \n>about it.\n>\n>Now if those tarballs are actually multiple revisions of the same \n>package, you might consider storing them uncompressed and let Git delta \n>compress them against each other which will produce an even more \n>significant space saving.\n\nI did manage to talk them into leaving them uncompressed.  But, they are\nlarge, and don't seem to delta compress all that well.  Maybe as more come,\nthe compression will be better.\n\nI guess this will be a good test case...  It will probably take months or\neven a year or so for the repo to get up to several GB.\n\nDavid\n"}]}