# How to prevent Git from compressing certain files? 5 messages from 2009-07-22 to 2009-07-23. Participants: Dirk Süsserott, Jakub Narebski, Jeff King. Thread: https://gitlist.dev/t/20197 ## Dirk Süsserott, 2009-07-22 19:49 Subject: How to prevent Git from compressing certain files? Message-ID: <4A676D4B.7040004@dirk.my1.cc> URL: https://gitlist.dev/e/4A676D4B.7040004%40dirk.my1.cc ``` Hi, I'm (ab)using Git to store my media files, i.e. digicam pictures (*.jpg) and the like. This way I can e.g. comment a series of pictures without installing and learning a special purpose "Photo Archiving" tool. Gitk shows the roadmap! Somewhere I read that Git isn't supposed to efficiently handle binary files. Of course, I don't want to merge my files, just store them with their history and git-push them to some "safe place". I figured that pushing and git gc'ing both try to compress those files (or differences) really hard. Works great for "regular" files, but is pointless with jpegs. Question: Is there a way to prevent Git from trying to compress certain files based on their extension? -- Dirk ``` ## Jakub Narebski, 2009-07-22 20:44 Subject: Re: How to prevent Git from compressing certain files? Message-ID: URL: https://gitlist.dev/e/m3vdlkbg8x.fsf%40localhost.localdomain In-Reply-To: <4A676D4B.7040004@dirk.my1.cc> ``` Dirk Süsserott writes: > I'm (ab)using Git to store my media files, i.e. digicam pictures (*.jpg) > and the like. This way I can e.g. comment a series of pictures without > installing and learning a special purpose "Photo Archiving" tool. Gitk > shows the roadmap! > > Somewhere I read that Git isn't supposed to efficiently handle binary > files. Of course, I don't want to merge my files, just store them with > their history and git-push them to some "safe place". > > I figured that pushing and git gc'ing both try to compress those files > (or differences) really hard. Works great for "regular" files, but is > pointless with jpegs. > > Question: Is there a way to prevent Git from trying to compress certain > files based on their extension? There is _undocumented_ "delta" gitattribute, introduced in a74db82 (Teach "delta" attribute to pack-objects., 2007-05-19), which you have to unset. Like this: == .gitattributes == *.jpg -delta -- Jakub Narebski Poland ShadeHawk on #git ``` ## Jakub Narebski, 2009-07-22 20:46 Subject: Re: How to prevent Git from compressing certain files? Message-ID: URL: https://gitlist.dev/e/m3r5w8bg69.fsf%40localhost.localdomain In-Reply-To: <4A676D4B.7040004@dirk.my1.cc> ``` Dirk Süsserott writes: > I'm (ab)using Git to store my media files, i.e. digicam pictures (*.jpg) > and the like. This way I can e.g. comment a series of pictures without > installing and learning a special purpose "Photo Archiving" tool. Gitk > shows the roadmap! > > Somewhere I read that Git isn't supposed to efficiently handle binary > files. Of course, I don't want to merge my files, just store them with > their history and git-push them to some "safe place". > > I figured that pushing and git gc'ing both try to compress those files > (or differences) really hard. Works great for "regular" files, but is > pointless with jpegs. > > Question: Is there a way to prevent Git from trying to compress certain > files based on their extension? There is _undocumented_ "delta" gitattribute, introduced by Junio C Hamano in a74db82 (Teach "delta" attribute to pack-objects., 2007-05-19), which you would have to unset. Like this: == .gitattributes == *.jpg -delta -- Jakub Narebski Poland ShadeHawk on #git ``` ## Jeff King, 2009-07-23 10:12 Subject: Re: How to prevent Git from compressing certain files? Message-ID: <20090723101227.GA4247@coredump.intra.peff.net> URL: https://gitlist.dev/e/20090723101227.GA4247%40coredump.intra.peff.net In-Reply-To: <4A676D4B.7040004@dirk.my1.cc> ``` On Wed, Jul 22, 2009 at 09:49:31PM +0200, Dirk Süsserott wrote: > Somewhere I read that Git isn't supposed to efficiently handle binary > files. Of course, I don't want to merge my files, just store them with > their history and git-push them to some "safe place". Git handles binary files better than many systems. The downsides are: - you can't do file-level diffing and merging very well, for obvious reasons (though actually, git is better than most; it makes it easy to look at both sides individually and pick the one you want). - really _big_ files can give lousy performance. Git assumes single files can fit into memory, which means files in the gigabyte range (or hundreds of megabytes if your machine is old :) ) can be awful. It also means that things like inexact rename detection and finding delta candidates can be slow. > I figured that pushing and git gc'ing both try to compress those files > (or differences) really hard. Works great for "regular" files, but is > pointless with jpegs. > > Question: Is there a way to prevent Git from trying to compress certain > files based on their extension? There are actually two types of compression that git uses: delta compression between similar objects in packs, and zlib compression of loose objects and objects within packs. You almost certainly don't want zlib compression on your jpegs, as they are already compressed. You can turn off zlib compression entirely by setting core.compression to 0. Unfortunately, this turns off compression for _all_ objects, which means in a mixed-use repo you won't be compressing your text (and even in a photos-only repo, you are not compressing your commit messages). Delta compression between two jpegs, or between two versions of a jpeg where the image data itself was modified, is unlikely to be useful. However, if you use EXIF metadata in the file, then you will save a lot of space between versions with the same image data, but different metadata. So it's worth leaving delta compression on in that case, and probably turning it off otherwise. As Jakub mentioned, you can use the delta gitattribute for just your jpegs. You can also turn off deltas entirely by setting pack.window to 0, though you may be losing some benefit on your non-blob objects. -Peff ``` ## Dirk Süsserott, 2009-07-23 18:51 Subject: Re: How to prevent Git from compressing certain files? Message-ID: <4A68B143.1070902@dirk.my1.cc> URL: https://gitlist.dev/e/4A68B143.1070902%40dirk.my1.cc In-Reply-To: <20090723101227.GA4247@coredump.intra.peff.net> ``` Jeff, Jakub, thanks for the answers and explanation of how and when Git compresses data. Setting both core.compression and pack.window to 0 sped up my git-gc :-) Dirk ```