{"thread":{"id":"20197","subject":"How to prevent Git from compressing certain files?","startedAt":"2009-07-22T19:49:31Z","lastAt":"2009-07-23T18:51:47Z","messageCount":5,"participants":["Dirk Süsserott","Jakub Narebski","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"118479","messageId":"4A676D4B.7040004@dirk.my1.cc","threadId":"20197","inReplyTo":null,"subject":"How to prevent Git from compressing certain files?","fromName":"Dirk Süsserott","fromEmail":"newsletter@dirk.my1.cc","sentAt":"2009-07-22T19:49:31Z","receivedAt":"2009-07-22T19:49:31Z","isPatch":false,"sender":{"key":"newsletter@dirk.my1.cc","avatar":null},"body":"Hi,\n\nI'm (ab)using Git to store my media files, i.e. digicam pictures (*.jpg)\nand the like. This way I can e.g. comment a series of pictures without\ninstalling and learning a special purpose \"Photo Archiving\" tool. Gitk\nshows the roadmap!\n\nSomewhere I read that Git isn't supposed to efficiently handle binary\nfiles. Of course, I don't want to merge my files, just store them with\ntheir history and git-push them to some \"safe place\".\n\nI figured that pushing and git gc'ing both try to compress those files\n(or differences) really hard. Works great for \"regular\" files, but is\npointless with jpegs.\n\nQuestion: Is there a way to prevent Git from trying to compress certain\nfiles based on their extension?\n\n-- Dirk\n"},{"id":"118480","messageId":"m3vdlkbg8x.fsf@localhost.localdomain","threadId":"20197","inReplyTo":"4A676D4B.7040004@dirk.my1.cc","subject":"Re: How to prevent Git from compressing certain files?","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2009-07-22T20:44:39Z","receivedAt":"2009-07-22T20:44:39Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Dirk Süsserott writes:\n\n> I'm (ab)using Git to store my media files, i.e. digicam pictures (*.jpg)\n> and the like. This way I can e.g. comment a series of pictures without\n> installing and learning a special purpose \"Photo Archiving\" tool. Gitk\n> shows the roadmap!\n> \n> Somewhere I read that Git isn't supposed to efficiently handle binary\n> files. Of course, I don't want to merge my files, just store them with\n> their history and git-push them to some \"safe place\".\n> \n> I figured that pushing and git gc'ing both try to compress those files\n> (or differences) really hard. Works great for \"regular\" files, but is\n> pointless with jpegs.\n> \n> Question: Is there a way to prevent Git from trying to compress certain\n> files based on their extension?\n\nThere is _undocumented_ \"delta\" gitattribute, introduced in a74db82\n(Teach \"delta\" attribute to pack-objects., 2007-05-19), which you have\nto unset. \n\nLike this:\n\n == .gitattributes ==\n *.jpg -delta\n\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"},{"id":"118481","messageId":"m3r5w8bg69.fsf@localhost.localdomain","threadId":"20197","inReplyTo":"4A676D4B.7040004@dirk.my1.cc","subject":"Re: How to prevent Git from compressing certain files?","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2009-07-22T20:46:07Z","receivedAt":"2009-07-22T20:46:07Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Dirk Süsserott writes:\n\n> I'm (ab)using Git to store my media files, i.e. digicam pictures (*.jpg)\n> and the like. This way I can e.g. comment a series of pictures without\n> installing and learning a special purpose \"Photo Archiving\" tool. Gitk\n> shows the roadmap!\n> \n> Somewhere I read that Git isn't supposed to efficiently handle binary\n> files. Of course, I don't want to merge my files, just store them with\n> their history and git-push them to some \"safe place\".\n> \n> I figured that pushing and git gc'ing both try to compress those files\n> (or differences) really hard. Works great for \"regular\" files, but is\n> pointless with jpegs.\n> \n> Question: Is there a way to prevent Git from trying to compress certain\n> files based on their extension?\n\nThere is _undocumented_ \"delta\" gitattribute, introduced by Junio C Hamano\nin a74db82 (Teach \"delta\" attribute to pack-objects., 2007-05-19), which\nyou would have to unset.\n\nLike this:\n\n == .gitattributes ==\n *.jpg -delta\n\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"},{"id":"118573","messageId":"20090723101227.GA4247@coredump.intra.peff.net","threadId":"20197","inReplyTo":"4A676D4B.7040004@dirk.my1.cc","subject":"Re: How to prevent Git from compressing certain files?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-07-23T10:12:27Z","receivedAt":"2009-07-23T10:12:27Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Jul 22, 2009 at 09:49:31PM +0200, Dirk Süsserott wrote:\n\n> Somewhere I read that Git isn't supposed to efficiently handle binary\n> files. Of course, I don't want to merge my files, just store them with\n> their history and git-push them to some \"safe place\".\n\nGit handles binary files better than many systems. The downsides are:\n\n - you can't do file-level diffing and merging very well, for obvious\n reasons (though actually, git is better than most; it makes it easy\n to look at both sides individually and pick the one you want).\n\n - really _big_ files can give lousy performance. Git assumes single\n files can fit into memory, which means files in the gigabyte range\n (or hundreds of megabytes if your machine is old :) ) can be awful.\n It also means that things like inexact rename detection and finding\n delta candidates can be slow.\n\n> I figured that pushing and git gc'ing both try to compress those files\n> (or differences) really hard. Works great for \"regular\" files, but is\n> pointless with jpegs.\n> \n> Question: Is there a way to prevent Git from trying to compress certain\n> files based on their extension?\n\nThere are actually two types of compression that git uses: delta\ncompression between similar objects in packs, and zlib compression of\nloose objects and objects within packs.\n\nYou almost certainly don't want zlib compression on your jpegs, as they\nare already compressed. You can turn off zlib compression entirely by\nsetting core.compression to 0. Unfortunately, this turns off compression\nfor _all_ objects, which means in a mixed-use repo you won't be\ncompressing your text (and even in a photos-only repo, you are not\ncompressing your commit messages).\n\nDelta compression between two jpegs, or between two versions of a jpeg\nwhere the image data itself was modified, is unlikely to be useful.\nHowever, if you use EXIF metadata in the file, then you will save a lot\nof space between versions with the same image data, but different\nmetadata. So it's worth leaving delta compression on in that case, and\nprobably turning it off otherwise.\n\nAs Jakub mentioned, you can use the delta gitattribute for just your\njpegs. You can also turn off deltas entirely by setting pack.window to\n0, though you may be losing some benefit on your non-blob objects.\n\n-Peff\n"},{"id":"118595","messageId":"4A68B143.1070902@dirk.my1.cc","threadId":"20197","inReplyTo":"20090723101227.GA4247@coredump.intra.peff.net","subject":"Re: How to prevent Git from compressing certain files?","fromName":"Dirk Süsserott","fromEmail":"newsletter@dirk.my1.cc","sentAt":"2009-07-23T18:51:47Z","receivedAt":"2009-07-23T18:51:47Z","isPatch":false,"sender":{"key":"newsletter@dirk.my1.cc","avatar":null},"body":"Jeff, Jakub, thanks for the answers and explanation of how and when Git \ncompresses data. Setting both core.compression and pack.window to 0 sped \nup my git-gc :-)\n\n\tDirk\n"}]}