# Add Gitweb support for LZMA compressed snapshots 17 messages from 2009-07-30 to 2009-08-01. Participants: Mark A Rada, Johannes Schindelin, Jakub Narebski, J.H., Felipe Contreras, Alex Riesen, Erik Faye-Lund, Dmitry Potapov, André Goddard Rosa, Jim Meyering. Thread: https://gitlist.dev/t/20293 ## Mark A Rada, 2009-07-30 05:48 Subject: Add Gitweb support for LZMA compressed snapshots Message-ID: URL: https://gitlist.dev/e/E0C39B59-E2C5-4C28-9570-D33FEA2A44EB%40uwaterloo.ca ``` Hi, I thought I would submit this little patch I made to my gitweb. I am on a relatively slow connection, and so LZMA compression time is less of a concern than bandwidth---I'm guessing that I am not the only person who suffers from slow internet connection syndrome. -- Mark A Rada (ferrous26) marada@uwaterloo.ca From a080ce5e8467317a3f20c6de8814549e5c4f46f2 Mon Sep 17 00:00:00 2001 From: Mark Rada Date: Wed, 29 Jul 2009 21:27:32 -0400 Subject: [PATCH] Add Gitweb support for LZMA compressed snapshots Signed-off-by: Mark Rada --- gitweb/gitweb.perl | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl index 38492d0..e131ea4 100755 --- a/gitweb/gitweb.perl +++ b/gitweb/gitweb.perl @@ -184,6 +184,13 @@ our %known_snapshot_formats = ( 'format' => 'tar', 'compressor' => ['bzip2']}, + 'txz' => { + 'display' => 'tar.lzma', + 'type' => 'application/x-lzma', + 'suffix' => '.lzma', + 'format' => 'tar', + 'compressor' => ['lzma']}, + 'zip' => { 'display' => 'zip', 'type' => 'application/x-zip', @@ -196,6 +203,7 @@ our %known_snapshot_formats = ( our %known_snapshot_format_aliases = ( 'gzip' => 'tgz', 'bzip2' => 'tbz2', + 'lzma' => 'txz', # backward compatibility: legacy gitweb config support 'x-gzip' => undef, 'gz' => undef, -- 1.6.4 ``` ## J.H., 2009-07-30 07:44 Subject: Re: Add Gitweb support for LZMA compressed snapshots Message-ID: <4A714F5C.70000@eaglescrag.net> URL: https://gitlist.dev/e/4A714F5C.70000%40eaglescrag.net In-Reply-To: ``` Mark A Rada wrote: > Hi, > > I thought I would submit this little patch I made to my gitweb. I am on > a relatively slow > connection, and so LZMA compression time is less of a concern than > bandwidth---I'm > guessing that I am not the only person who suffers from slow internet > connection > syndrome. > > > -- > Mark A Rada (ferrous26) > marada@uwaterloo.ca Don't use 'lzma' the command, use 'xz' ( http://tukaani.org/xz/ ) as it uses the lzma2 format which is, by far, preferable to what 'lzma' outputs. Same compression (lzma) just different file format. - John 'Warthog9' Hawley ``` ## Johannes Schindelin, 2009-07-30 07:52 Subject: Re: Add Gitweb support for LZMA compressed snapshots Message-ID: URL: https://gitlist.dev/e/alpine.DEB.1.00.0907300950180.4589%40intel-tinevez-2-302 In-Reply-To: ``` Hi, On Thu, 30 Jul 2009, Mark A Rada wrote: > I thought I would submit this little patch I made to my gitweb. I am on > a relatively slow connection, and so LZMA compression time is less of a > concern than bandwidth---I'm guessing that I am not the only person who > suffers from slow internet connection syndrome. That's great! FWIW on this mailing list, we prefer to have patches inlined so that it is easier for reviewers to comment (it is almost impossible to get any patch into git.git without review, and because reviewing is such an ungratifying job you want to make it easier). The patch is pretty straight-forward, but I'd love to see some information in the commit message about: - performance numbers on the _server_ (i.e. how does the RAM and CPU load compare to, say, bzip2?) - how to disable/enable it (yes, it's in the documentation, but you can make things easier still) - whether it is turned on by default, and why. Thanks, Dscho ``` ## Jakub Narebski, 2009-07-30 08:31 Subject: Re: Add Gitweb support for LZMA compressed snapshots Message-ID: URL: https://gitlist.dev/e/m3r5vy1siq.fsf%40localhost.localdomain In-Reply-To: ``` Mark A Rada writes: > I thought I would submit this little patch I made to my gitweb. I am > on a relatively slow connection, and so LZMA compression time is > less of a concern than bandwidth---I'm guessing that I am not the > only person who suffers from slow internet connection syndrome. First, Documentation/SubmittingPatches states that we prefer inline patches, with additional comments (like the one above) either between "---\n" line and diffstat, or like I did here before patch with patch separated by comment by e.g. "-- >8 --\n" (scissors) line. If posting patch inline isn't possible, and you have to use attachement (for example because mailer you use wraps lines), please at least use 'text/plain' mimetype so patch can be viewed without need to save it in separate file (this might require changing suffix from '.patch' to '.txt'). Second, if you are using 'txz' as internal name for new snapshot format, why not use 'XZ utils' instead of 'LZMA Utils', and use 'display' => 'txz', 'type' => 'application/x-xz' and 'suffix' => '.txz' (BTW. shouldn't suffix in your case be '.tar.lzma' and not only '.lzma'?) and of course 'compressor' => ['xz']? BTW. I wonder if it would be good idea to add support for this format directly to git-archive... OTOH it would mean additional dependency. -- Jakub Narebski Git User's Survey 2009: http://tinyurl.com/GitSurvey2009 -- >8 -- From: Mark Rada Subject: [PATCH] Add Gitweb support for LZMA compressed snapshots Signed-off-by: Mark Rada --- gitweb/gitweb.perl | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl index 38492d0..e131ea4 100755 --- a/gitweb/gitweb.perl +++ b/gitweb/gitweb.perl @@ -184,6 +184,13 @@ our %known_snapshot_formats = ( 'format' => 'tar', 'compressor' => ['bzip2']}, + 'txz' => { + 'display' => 'tar.lzma', + 'type' => 'application/x-lzma', + 'suffix' => '.lzma', + 'format' => 'tar', + 'compressor' => ['lzma']}, + 'zip' => { 'display' => 'zip', 'type' => 'application/x-zip', @@ -196,6 +203,7 @@ our %known_snapshot_formats = ( our %known_snapshot_format_aliases = ( 'gzip' => 'tgz', 'bzip2' => 'tbz2', + 'lzma' => 'txz', # backward compatibility: legacy gitweb config support 'x-gzip' => undef, 'gz' => undef, -- 1.6.4 ``` ## Johannes Schindelin, 2009-07-30 08:43 Subject: Re: Add Gitweb support for LZMA compressed snapshots Message-ID: URL: https://gitlist.dev/e/alpine.DEB.1.00.0907301042300.7467%40intel-tinevez-2-302 In-Reply-To: ``` Hi, On Thu, 30 Jul 2009, Jakub Narebski wrote: > BTW. I wonder if it would be good idea to add support for this format > directly to git-archive... OTOH it would mean additional dependency. I don't think it would be a good idea; we do not have bzip2 support either. The only reason we have inbuilt gzip and zip support is because the format is so similar to Git's own compression. Ciao, Dscho ``` ## Felipe Contreras, 2009-07-31 15:45 Subject: Re: Add Gitweb support for LZMA compressed snapshots Message-ID: <94a0d4530907310845k51dbafbfy8e240467113dfc16@mail.gmail.com> URL: https://gitlist.dev/e/94a0d4530907310845k51dbafbfy8e240467113dfc16%40mail.gmail.com In-Reply-To: ``` On Thu, Jul 30, 2009 at 8:43 AM, Johannes Schindelin wrote: > Hi, > > On Thu, 30 Jul 2009, Jakub Narebski wrote: > >> BTW. I wonder if it would be good idea to add support for this format >> directly to git-archive...  OTOH it would mean additional dependency. > > I don't think it would be a good idea; we do not have bzip2 support > either. bzip2 has no advantages whatsoever. AFAIK xz is superior to other formats and it would be nice to see git make a technological stance encouraging xz. > The only reason we have inbuilt gzip and zip support is because the format > is so similar to Git's own compression. Personally I don't see the point of having zip support. -- Felipe Contreras ``` ## Alex Riesen, 2009-08-01 07:43 Subject: Re: Add Gitweb support for LZMA compressed snapshots Message-ID: <81b0412b0908010043y51680392r69a0090aba520b13@mail.gmail.com> URL: https://gitlist.dev/e/81b0412b0908010043y51680392r69a0090aba520b13%40mail.gmail.com In-Reply-To: <4A714F5C.70000@eaglescrag.net> ``` On Thu, Jul 30, 2009 at 09:44, J.H. wrote: > Don't use 'lzma' the command, use 'xz' ( http://tukaani.org/xz/ ) as it uses The host seems to be down. > the lzma2 format which is, by far, preferable to what 'lzma' outputs.  Same > compression (lzma) just different file format. And the whole project seems to be very young (listen on ohloh since 2007). And it is not in current stable distributions. Exotic thing. ``` ## Johannes Schindelin, 2009-08-01 12:34 Subject: Re: Add Gitweb support for LZMA compressed snapshots Message-ID: URL: https://gitlist.dev/e/alpine.DEB.1.00.0908011431060.8306%40pacific.mpi-cbg.de In-Reply-To: <94a0d4530907310845k51dbafbfy8e240467113dfc16@mail.gmail.com> ``` Hi, On Fri, 31 Jul 2009, Felipe Contreras wrote: > On Thu, Jul 30, 2009 at 8:43 AM, Johannes > Schindelin wrote: > > Hi, > > > > On Thu, 30 Jul 2009, Jakub Narebski wrote: > > > >> BTW. I wonder if it would be good idea to add support for this format > >> directly to git-archive...  OTOH it would mean additional dependency. > > > > I don't think it would be a good idea; we do not have bzip2 support > > either. > > bzip2 has no advantages whatsoever. Bzzzzt. Wrong. Just because you cannot see them does not mean it has no advantages. > AFAIK xz is superior to other formats and it would be nice to see git > make a technological stance encouraging xz. Bzzzt. Wrong again. Git's mission in life is not to encourage one compression over another. If at all, the only compression Git actually does promote in a sense is zlib compression. > > The only reason we have inbuilt gzip and zip support is because the > > format is so similar to Git's own compression. > > Personally I don't see the point of having zip support. Personally, I see the point of having zip support. It makes things easy for Windows users. And it's an established format, much more so than tar.gz. Ciao, Dscho ``` ## Felipe Contreras, 2009-08-01 13:10 Subject: Re: Add Gitweb support for LZMA compressed snapshots Message-ID: <94a0d4530908010610n31261414yc08060f3de9c115f@mail.gmail.com> URL: https://gitlist.dev/e/94a0d4530908010610n31261414yc08060f3de9c115f%40mail.gmail.com In-Reply-To: ``` On Sat, Aug 1, 2009 at 12:34 PM, Johannes Schindelin wrote: > Hi, > > On Fri, 31 Jul 2009, Felipe Contreras wrote: > >> On Thu, Jul 30, 2009 at 8:43 AM, Johannes >> Schindelin wrote: >> > Hi, >> > >> > On Thu, 30 Jul 2009, Jakub Narebski wrote: >> > >> >> BTW. I wonder if it would be good idea to add support for this format >> >> directly to git-archive...  OTOH it would mean additional dependency. >> > >> > I don't think it would be a good idea; we do not have bzip2 support >> > either. >> >> bzip2 has no advantages whatsoever. > > Bzzzzt.  Wrong.  Just because you cannot see them does not mean it has no > advantages. All right, most the time you compress once, and multiple people uncompress multiple times. Therefore the path that should be optimized is the decompression, that leaves bzip2 out of the picture. You can think of advantages of bzip2 in more esoteric cases where the compression time is also important. I don't think that's the case here. Actually it might be if gitweb is not caching the tarballs, but if that's the case I wouldn't say it's an advantage of bzip2. >> AFAIK xz is superior to other formats and it would be nice to see git >> make a technological stance encouraging xz. > > Bzzzt. Wrong again.  Git's mission in life is not to encourage one > compression over another. Git's mission in life is not promoting asciidot either, but it's doing that unintentionally by merely using it. > If at all, the only compression Git actually does promote in a sense is > zlib compression. > >> > The only reason we have inbuilt gzip and zip support is because the >> > format is so similar to Git's own compression. >> >> Personally I don't see the point of having zip support. > > Personally, I see the point of having zip support.  It makes things easy > for Windows users.  And it's an established format, much more so than > tar.gz. Windows can't extract .tar.gz? -- Felipe Contreras ``` ## Erik Faye-Lund, 2009-08-01 14:04 Subject: Re: Add Gitweb support for LZMA compressed snapshots Message-ID: <40aa078e0908010704k7581b656h1ed98a8852efa445@mail.gmail.com> URL: https://gitlist.dev/e/40aa078e0908010704k7581b656h1ed98a8852efa445%40mail.gmail.com In-Reply-To: <94a0d4530908010610n31261414yc08060f3de9c115f@mail.gmail.com> ``` On Sat, Aug 1, 2009 at 3:10 PM, Felipe Contreras wrote: > Windows can't extract .tar.gz? Not without installing 3. party software. Zip-support is built into the operating system. -- Erik "kusma" Faye-Lund kusmabite@gmail.com (+47) 986 59 656 ``` ## Dmitry Potapov, 2009-08-01 14:13 Subject: Re: Add Gitweb support for LZMA compressed snapshots Message-ID: <20090801141304.GB10264@dpotapov.dyndns.org> URL: https://gitlist.dev/e/20090801141304.GB10264%40dpotapov.dyndns.org In-Reply-To: <94a0d4530907310845k51dbafbfy8e240467113dfc16@mail.gmail.com> ``` On Fri, Jul 31, 2009 at 03:45:02PM +0000, Felipe Contreras wrote: > > bzip2 has no advantages whatsoever. AFAIK xz is superior to other > formats and it would be nice to see git make a technological stance > encouraging xz. git is not about encouraging any compression. If you have not noticed, git-archive produces tar and not tgz (though it would not be difficult to implement it, because we use zlib anyway). If you like xz and it is installed on your system, you can always compress tar with it: git archive HEAD | xz > my-archive.tar.xz So, I see no reason to have this compression coupled with git. > > > The only reason we have inbuilt gzip and zip support is because the format > > is so similar to Git's own compression. > > Personally I don't see the point of having zip support. zip is still a popular format on Windows. If git archive was not able to produce then creating would require creating tar first, extracting it to some temporary directory and then creating a zip file. It is not very efficient process and requires extra efforts, so because we use zlib compression anyway, it makes perfect sense that we can create zip file directly. Dmitry ``` ## Dmitry Potapov, 2009-08-01 14:34 Subject: Re: Add Gitweb support for LZMA compressed snapshots Message-ID: <20090801143403.GC10264@dpotapov.dyndns.org> URL: https://gitlist.dev/e/20090801143403.GC10264%40dpotapov.dyndns.org In-Reply-To: <81b0412b0908010043y51680392r69a0090aba520b13@mail.gmail.com> ``` On Sat, Aug 01, 2009 at 09:43:39AM +0200, Alex Riesen wrote: > > And the whole project seems to be very young (listen on ohloh since 2007). > And it is not in current stable distributions. Exotic thing. The project seems to be it is developed together with Igor Pavlov, who is the author of 7-Zip (and now 7-Zip also supports LZMA2 and XZ format). Accordingly to Igor Pavlov: LZMA2 provides the following advantages over LZMA: 1) Better compression ratio for data than can't be compressed. It can store such blocks of data in uncompressed form. Also it decompresses such data faster. 2) Better multithreading support. If you compress big file, LZMA2 can split that file to chunks and compress these chunks in multiple threads. Clearly XZ is not very popular yet, but I would not call it as exotic. At least, I expect it to be supported by future distributives. Dmitry ``` ## André Goddard Rosa, 2009-08-01 14:38 Subject: Re: Add Gitweb support for LZMA compressed snapshots Message-ID: URL: https://gitlist.dev/e/b8bf37780908010738y305b67c7v3c4e8f2190bfdc93%40mail.gmail.com In-Reply-To: <81b0412b0908010043y51680392r69a0090aba520b13@mail.gmail.com> ``` On Sat, Aug 1, 2009 at 4:43 AM, Alex Riesen wrote: > On Thu, Jul 30, 2009 at 09:44, J.H. wrote: >> Don't use 'lzma' the command, use 'xz' ( http://tukaani.org/xz/ ) as it uses > > The host seems to be down. > You can clone it here: git://ctrl.tukaani.org/xz.git Best regards, Andre ``` ## Jim Meyering, 2009-08-01 14:58 Subject: Re: Add Gitweb support for LZMA compressed snapshots Message-ID: <87tz0rinrv.fsf@meyering.net> URL: https://gitlist.dev/e/87tz0rinrv.fsf%40meyering.net In-Reply-To: <81b0412b0908010043y51680392r69a0090aba520b13@mail.gmail.com> ``` Alex Riesen wrote: > On Thu, Jul 30, 2009 at 09:44, J.H. wrote: >> Don't use 'lzma' the command, use 'xz' ( http://tukaani.org/xz/ ) as it uses > > The host seems to be down. > >> the lzma2 format which is, by far, preferable to what 'lzma' outputs.  Same >> compression (lzma) just different file format. > > And the whole project seems to be very young (listen on ohloh since 2007). > And it is not in current stable distributions. Exotic thing. XZ is young, but worthy, imho. It was added to Fedora 11 in the last few days. ``` ## Mark A Rada, 2009-08-01 16:07 Subject: Re: Add Gitweb support for LZMA compressed snapshots Message-ID: URL: https://gitlist.dev/e/CC0DDAD1-D61F-465F-9533-95A101C61522%40uwaterloo.ca In-Reply-To: <40aa078e0908010704k7581b656h1ed98a8852efa445@mail.gmail.com> ``` On 1-Aug-09, at 10:04 AM, Erik Faye-Lund wrote: > On Sat, Aug 1, 2009 at 3:10 PM, Felipe > Contreras wrote: >> Windows can't extract .tar.gz? > > Not without installing 3. party software. Zip-support is built into > the operating system. > > -- > Erik "kusma" Faye-Lund > kusmabite@gmail.com > (+47) 986 59 656 > -- > To unsubscribe from this list: send the line "unsubscribe git" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html FWIW, Windows doesn't seem to come with a whole lot, and you need 3rd party software for a number of things, BUT that is also one of the primary reasons that I need to be able to quickly get snapshots from my Git repositories at home, and using zip format is convenient if I am in an extremely restrictive environment and cannot get a smaller XZ compressed snapshot :). -- Mark A Rada (ferrous26) marada@uwaterloo.ca ``` ## Alex Riesen, 2009-08-01 18:51 Subject: Re: Add Gitweb support for LZMA compressed snapshots Message-ID: <81b0412b0908011151p7206226fi8bea97c7b2c5f876@mail.gmail.com> URL: https://gitlist.dev/e/81b0412b0908011151p7206226fi8bea97c7b2c5f876%40mail.gmail.com In-Reply-To: <87tz0rinrv.fsf@meyering.net> ``` On Sat, Aug 1, 2009 at 16:58, Jim Meyering wrote: > Alex Riesen wrote: >> And it is not in current stable distributions. Exotic thing. > > XZ is young, but worthy, imho. > It was added to Fedora 11 in the last few days. Well, then maybe we should do by its good example and support xz. In addition to .lzma, .7z, .cpio, etc. ``` ## Erik Faye-Lund, 2009-08-01 21:39 Subject: Re: Add Gitweb support for LZMA compressed snapshots Message-ID: <40aa078e0908011439q78a4cdefqa0b05955cc6a5fde@mail.gmail.com> URL: https://gitlist.dev/e/40aa078e0908011439q78a4cdefqa0b05955cc6a5fde%40mail.gmail.com In-Reply-To: ``` On Sat, Aug 1, 2009 at 6:07 PM, Mark A Rada wrote: > FWIW, Windows doesn't seem to come with a whole lot, and you need > 3rd party software for a number of things Tell that to the US Department of Justice ;) -- Erik "kusma" Faye-Lund kusmabite@gmail.com (+47) 986 59 656 ```