{"thread":{"id":"5284","subject":"Fwd: Git SCM and zlib dictionaries","startedAt":"2006-08-15T15:19:59Z","lastAt":"2006-08-15T15:19:59Z","messageCount":1,"participants":["Jon Smirl"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"25359","messageId":"9e4733910608150819l6ba602e2q2f52a5693a2bac4d@mail.gmail.com","threadId":"5284","inReplyTo":"00B40C71-72B6-499B-806B-64A140136944@alumni.caltech.edu","subject":"Fwd: Git SCM and zlib dictionaries","fromName":"Jon Smirl","fromEmail":"jonsmirl@gmail.com","sentAt":"2006-08-15T15:19:59Z","receivedAt":"2006-08-15T15:19:59Z","isPatch":false,"sender":{"key":"jonsmirl@gmail.com","avatar":"https://gravatar.com/avatar/cff3bf5bfdfa6708b905712ff91f0f9b8aaca161659f38c02b787920d5d28b7e?d=mp&s=160"},"body":"---------- Forwarded message ----------\nFrom: Mark Adler <madler@alumni.caltech.edu>\nDate: Aug 15, 2006 10:43 AM\nSubject: Re: Git SCM and zlib dictionaries\nTo: Jon Smirl <jonsmirl@gmail.com>\nCc: Jean-loup Gailly <jloup@gzip.org>\n\n\nOn Aug 15, 2006, at 6:11 AM, Jon Smirl wrote:\n> What we are doing is similar to full-text\n> search indexing.\n\nIf the point of very small (1Kish) compressed chunks is for random\naccess and individual decompression of those pieces, then there are\nother approaches.  You can for example compress many of them together\nfor better compression (say 32), and accept some speed degradation by\nhaving to decompress on average half (16) of them to get to the one\nyou want.\n\n------------------------------\nWe have delta runs of about 20 revsisions, compress those 20 blobs as\na group instead of individually. The pack index would point all 20\nsha1's to the same blob with a different type code. You had to load\nand unzip most of these objects anyway to compute the revision off\nfrom the diffs. Putting them into a single zip means that they share a\nsingle compression table.\n-------------------------------\n\nOr you can process the whole thing to create a custom coding scheme,\nas illustrated in \"Managing Gigabytes\":\n\n     http://www.cs.mu.oz.au/mg/\n\nmark\n\n\n\n-- \nJon Smirl\njonsmirl@gmail.com\n"}]}