{"thread":{"id":"20303","subject":"[PATCHv2] Gitweb support for XZ compressed snapshots","startedAt":"2009-07-31T02:12:03Z","lastAt":"2009-07-31T02:12:03Z","messageCount":1,"participants":["Mark A Rada"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"119207","messageId":"896CD516-F3F0-4CA1-907A-6C7EF6D9789B@uwaterloo.ca","threadId":"20303","inReplyTo":null,"subject":"[PATCHv2] Gitweb support for XZ compressed snapshots","fromName":"Mark A Rada","fromEmail":"marada@uwaterloo.ca","sentAt":"2009-07-31T02:12:03Z","receivedAt":"2009-07-31T02:12:03Z","isPatch":false,"sender":{"key":"marada@uwaterloo.ca","avatar":"https://avatars.githubusercontent.com/u/38430?v=4"},"body":"Ok, I've fixed things up a bit, and done a bit of benchmarking, let me  \nknow if something/wrong.\n\n\n--\nMark A Rada (ferrous26)\nmarada@uwaterloo.ca\n\n\n------------------->8-------------------\nFrom: Mark Rada <marada@uwaterloo.ca>\nDate: Thu, 30 Jul 2009 08:56:42 -0400\nSubject: [PATCH] Gitweb support for XZ compressed snapshots\n\nThe XZ compression format uses the LZMA compression algorithm, which\ngenerally is capable of yielding higher compression ratios than both\nGZip and BZip2 at the cost of using more CPU time and RAM (lots more).\nIt is relevant to note that while LZMA is the slowest of the mentioned\nalgorithms for compression, it is much faster than BZip2 for\ndecompression (but still slower than GZip).\n\nYou can enable XZ compressed snapshots by adding 'txz' to the list of\ndefault options for snapshots in $GITWEB_CONFIG or adding txz to an\nindividual repository using the gitweb.snapshot variable for the config\nfile.\n\nI did some simple benchmarks, starting with an already tarballed\narchive of the repos listed below. Memory usage seemed to be consistent\nfor any given algorithm. All tests done at default compression level.\n\nCPU: AMD Sempron 3400+ (1 core @ 1.8GHz with 256K L2 cache)\n\n\t\t\tVirtual Memory\n\t\t\tGZip: \t 4152K\n\t\t\tBZip2: \t13352K\n\t\t\tXZ: \t  102M\n\nLinux kernel 2.6 series (f5886c7f96f2542382d3a983c5f13e03d7fc5259)\ngzip \t23.70s user    \t0.47s system  \t99% cpu  \t24.227 total\ngunzip \t3.74s user     \t0.74s system  \t94% cpu  \t4.741 total\nbzip2 \t130.96s user   \t0.53s system  \t99% cpu  \t2:11.97 total\nbunzip2 31.05s user    \t1.02s system  \t99% cpu  \t32.355 total\nxz \t448.78s user \t0.91s system  \t99% cpu  \t7:31.28 total\nunxz \t7.67s user     \t0.80s system  \t98% cpu  \t8.607 total\n\nGit (0a53e9ddeaddad63ad106860237bbf53411d11a7)\ngzip \t0.77s user \t0.03s system \t99% cpu \t0.792 total\ngunzip \t0.12s user \t0.02s system \t98% cpu \t0.142 total\nbzip2 \t3.42s user \t0.02s system \t99% cpu \t3.454 total\nbunzip2 0.95s user \t0.03s system \t99% cpu \t0.984 total\nxz \t12.88s user \t0.14s system \t98% cpu \t13.239 total\nunxz \t0.27s user \t0.03s system \t99% cpu \t0.298 total\n\nXZ (669413bb2db954bbfde3c4542fddbbab53891eb4)\nxz \t1.62s user \t0.03s system \t99% cpu \t1.652 total\nunxz   \t0.05s user \t0.00s system \t99% cpu \t0.058 total\nbzip2  \t1.28s user \t0.01s system \t99% cpu \t1.298 total\nbunzip2 0.15s user \t0.01s system \t100% cpu \t0.157 total\ngzip  \t0.12s user \t0.00s system \t95% cpu \t0.132 total\ngunzip \t0.02s user \t0.00s system \t97% cpu \t0.027 total\n\nI don't think it should be the default format, at least not right now,\nsimply because the XZ format is still fairly new (the format was\ndeclared stable about 6 months ago), and there have been no \"stable\"\nreleases of the utils yet.\n\nSigned-off-by: Mark Rada <marada@uwaterloo.ca>\n---\n  gitweb/gitweb.perl |    8 ++++++++\n  1 files changed, 8 insertions(+), 0 deletions(-)\n\ndiff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl\nindex 7fbd5ff..3398163 100755\n--- a/gitweb/gitweb.perl\n+++ b/gitweb/gitweb.perl\n@@ -176,6 +176,13 @@ our %known_snapshot_formats = (\n  \t\t'format' => 'tar',\n  \t\t'compressor' => ['bzip2']},\n\n+\t'txz' => {\n+\t\t'display' => 'tar.xz',\n+\t\t'type' => 'application/x-xz',\n+\t\t'suffix' => '.tar.xz',\n+\t\t'format' => 'tar',\n+\t\t'compressor' => ['xz']},\n+\n  \t'zip' => {\n  \t\t'display' => 'zip',\n  \t\t'type' => 'application/x-zip',\n@@ -188,6 +195,7 @@ our %known_snapshot_formats = (\n  our %known_snapshot_format_aliases = (\n  \t'gzip'  => 'tgz',\n  \t'bzip2' => 'tbz2',\n+\t'xz'    => 'txz',\n\n  \t# backward compatibility: legacy gitweb config support\n  \t'x-gzip' => undef, 'gz' => undef,\n-- \n1.6.4\n"}]}