Re: Packfile can't be mapped
- From
Shawn Pearce <spearce@spearce.org>
- Date
- Aug 28, 2006, 05:33 UTC
- Message-ID
- <20060828053301.GA25285@spearce.org>
- In-Reply-To
- <Pine.LNX.4.64.0608280014190.3683@localhost.localdomain>
Nicolas Pitre <nico@cam.org> wrote:
Show 16 quoted lines
> On Sun, 27 Aug 2006, Shawn Pearce wrote: > > > I'm going to try to get tree deltas written to the pack sometime this > > week. That should compact this intermediate pack down to something > > that git-pack-objects would be able to successfully mmap into a > > 32 bit address space. A complete repack with no delta reuse will > > hopefully generate a pack closer to 400 MB in size. But I know > > Jon would like to get that pack even smaller. :) > > One thing to consider in your code (if you didn't implement that > already) is to _not_ attempt any delta on any object whose size is > smaller than 50 bytes, and then limit the maximum delta size to > object_size/2 - 20 (use that for the last argument to diff-delta() and > store the undeltified object when diff-delta returns NULL). This way > you'll avoid creating delta objects that are most likely to end up being > _larger_ than the undeltified object.
I haven't tried this. Should be trivial to implement. Thanks for the suggestion.
Show 8 quoted lines
> > I should point out that the input stream to fast-import was 20 GB > > (completely decompressed revisions from RCS) plus all commit data. > > The original CVS ,v files are around 3 GB. An archive .tar.gz'ing > > the ,v files is around 550 MB. Going to only 1.7 GB without tree > > or commit deltas is certainly pretty good. :) > > Good job indeed. Oh and you probably should not bother trying to > deltify commit objects at all since that would be a waste of time.
I wasn't going to bother even trying to delta the commits. In this import the 200k commits isn't a very large percentage of the data. As I'm sure you are well aware its pretty much a waste time to try with the commits, especially with an "intermediate" pack such as this one.
-- Shawn.