{"thread":{"id":"18292","subject":"Transparently encrypt repository contents with GPG","startedAt":"2009-03-12T21:19:48Z","lastAt":"2012-06-18T20:03:36Z","messageCount":18,"participants":["Matthias Nothhaft","Sverre Rabbelier","Michael J Gruber","Thomas Rast","Jeff King","Junio C Hamano","bigbear","lalebarde"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"107884","messageId":"978bdee00903121419o61cd7a87rb55809796bd257d7@mail.gmail.com","threadId":"18292","inReplyTo":null,"subject":"Transparently encrypt repository contents with GPG","fromName":"Matthias Nothhaft","fromEmail":"matthias.nothhaft@googlemail.com","sentAt":"2009-03-12T21:19:48Z","receivedAt":"2009-03-12T21:19:48Z","isPatch":false,"sender":{"key":"matthias.nothhaft@googlemail.com","avatar":"https://gravatar.com/avatar/09d556aa6f5dd18cd55fd4c9f4babed400e35545353f65bc6a2161a776e0cff4?d=mp&s=160"},"body":"Hi,\n\nI'm new to Git but I really already love it. ;-)\n\nI would like to have repository that transparently encrypts and\ndecrypts all files using GPG.\n\nWhat I need is a way to automatically modify each file\n\na) before it is written in the repository\nb) after it is read from the repository\n\nIs there a way to get this work somehow? Can someone give me some\nhints where I need to begin?\n\nregards,\nMatthias\n"},{"id":"107885","messageId":"fabb9a1e0903121434u4a3d71bdi6277071f54557a7e@mail.gmail.com","threadId":"18292","inReplyTo":"978bdee00903121419o61cd7a87rb55809796bd257d7@mail.gmail.com","subject":"Re: Transparently encrypt repository contents with GPG","fromName":"Sverre Rabbelier","fromEmail":"srabbelier@gmail.com","sentAt":"2009-03-12T21:34:54Z","receivedAt":"2009-03-12T21:34:54Z","isPatch":false,"sender":{"key":"srabbelier@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3098?v=4"},"body":"Heya,\n\nOn Thu, Mar 12, 2009 at 22:19, Matthias Nothhaft\n > What I need is a way to\nautomatically modify each file\n>\n> a) before it is written in the repository\n> b) after it is read from the repository\n\nHave a look at smudging, you might not need to touch the git source\ncode at all ;).\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"107926","messageId":"49BA39A1.9090203@drmicha.warpmail.net","threadId":"18292","inReplyTo":"fabb9a1e0903121434u4a3d71bdi6277071f54557a7e@mail.gmail.com","subject":"Re: Transparently encrypt repository contents with GPG","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2009-03-13T10:46:57Z","receivedAt":"2009-03-13T10:46:57Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Sverre Rabbelier venit, vidit, dixit 12.03.2009 22:34:\n> Heya,\n> \n> On Thu, Mar 12, 2009 at 22:19, Matthias Nothhaft\n> > What I need is a way to\n> automatically modify each file\n>>\n>> a) before it is written in the repository\n>> b) after it is read from the repository\n> \n> Have a look at smudging, you might not need to touch the git source\n> code at all ;).\n> \n\nAnd people asked me not to be cryptic... even though the OP explicitely\nasked for encryption, of course ;)\n\n\"git help attributes\" may help: look for filter and set attributes and\nconfig (filter.$name.{clean,smudge}) accordingly. smudge should probably\ndecrypt, clean should encrypt.\n\nBTW: Why not use an encrypted file system? That way your work tree would\nbe encrypted also.\n\nCheers,\nMichael\n"},{"id":"107931","messageId":"fabb9a1e0903130351w592ee33dm499f10306d565aab@mail.gmail.com","threadId":"18292","inReplyTo":"49BA39A1.9090203@drmicha.warpmail.net","subject":"Re: Transparently encrypt repository contents with GPG","fromName":"Sverre Rabbelier","fromEmail":"srabbelier@gmail.com","sentAt":"2009-03-13T10:51:43Z","receivedAt":"2009-03-13T10:51:43Z","isPatch":false,"sender":{"key":"srabbelier@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3098?v=4"},"body":"Heya,\n\nOn Fri, Mar 13, 2009 at 11:46, Michael J Gruber\n wrote:\n> And people asked me not to be cryptic... even though the OP explicitely\n> asked for encryption, of course ;)\n\nI wasn't being cryptic, I just don't remember the details of smudge,\njust that it exists, and that it allows you to perform operations on a\nfile on checkout and on add.\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"107932","messageId":"200903131215.49336.trast@student.ethz.ch","threadId":"18292","inReplyTo":"49BA39A1.9090203@drmicha.warpmail.net","subject":"Re: Transparently encrypt repository contents with GPG","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2009-03-13T11:15:43Z","receivedAt":"2009-03-13T11:15:43Z","isPatch":false,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Michael J Gruber wrote:\n> \"git help attributes\" may help: look for filter and set attributes and\n> config (filter.$name.{clean,smudge}) accordingly. smudge should probably\n> decrypt, clean should encrypt.\n\nWouldn't this trip over the randomness included in all encryption [to\navoid generating the same cyphertext for two separate identical\nmessages, which gives away some information], which would let git\nthink the file has been changed as soon as its stat info has changed\n(or is just racy)?\n\nNot to mention that this makes most source-oriented features such as\ndiff, blame, merge, etc., rather useless.\n\n-- \nThomas Rast\ntrast@{inf,student}.ethz.ch\n"},{"id":"107933","messageId":"fabb9a1e0903130417x36121bd5ya8b323e0a80bbd8f@mail.gmail.com","threadId":"18292","inReplyTo":"200903131215.49336.trast@student.ethz.ch","subject":"Re: Transparently encrypt repository contents with GPG","fromName":"Sverre Rabbelier","fromEmail":"srabbelier@gmail.com","sentAt":"2009-03-13T11:17:37Z","receivedAt":"2009-03-13T11:17:37Z","isPatch":false,"sender":{"key":"srabbelier@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3098?v=4"},"body":"Heya,\n\nOn Fri, Mar 13, 2009 at 12:15, Thomas Rast wrote:\n> Not to mention that this makes most source-oriented features such as\n> diff, blame, merge, etc., rather useless.\n\nI would assume that smudge takes care of this somehow, it'd seem like\na rather useless feature otherwise :).\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"107949","messageId":"49BA6606.1070403@fastmail.fm","threadId":"18292","inReplyTo":"fabb9a1e0903130417x36121bd5ya8b323e0a80bbd8f@mail.gmail.com","subject":"Re: Transparently encrypt repository contents with GPG","fromName":"Michael J Gruber","fromEmail":"michaeljgruber+gmane@fastmail.fm","sentAt":"2009-03-13T13:56:22Z","receivedAt":"2009-03-13T13:56:22Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Sverre Rabbelier venit, vidit, dixit 13.03.2009 12:17:\n> Heya,\n> \n> On Fri, Mar 13, 2009 at 12:15, Thomas Rast \n> wrote:\n>> Not to mention that this makes most source-oriented features such\n>> as diff, blame, merge, etc., rather useless.\n> \n> I would assume that smudge takes care of this somehow, it'd seem\n> like a rather useless feature otherwise :).\n\nSverre was being prophetic with the somehow. Here's a working setup\n(though I still don't know why not to use luks):\n\nIn .gitattributes (or.git/info/a..) use\n\n* filter=gpg diff=gpg\n\nIn your config:\n\n[filter \"gpg\"]\n smudge = gpg -d -q --batch --no-tty\n clean = gpg -ea -q --batch --no-tty -r C920A124\n[diff \"gpg\"]\n textconv = decrypt\n\nThis gives you textual diffs even in log! You want use gpg-agent here.\n\nNow for Sverre's prophecy and the helper I haven't shown you yet: It\nturns out that blobs are not smudged before they are fed to textconv!\n[Also, it seems that the textconv config does allow parameters, bit I\nhaven't checked thoroughly.]\n\nThis means that e.g. when diffing work tree with HEAD textconv is called\ntwice: once is with a smudged file (from the work tree) and once with a\ncleaned file (from HEAD). That's why I needed a small helper script\n\"decrypt\" which does nothing but\n\n#!/bin/sh\ngpg -d -q --batch --no-tty \"$1\" || cat $1\n\nYeah, this assumes gpg errors out because it's fed something unencrypted\n(and not encrypted with the wrong key) etc. It's only proof of concept\nquality.\n\nMe thinks it's not right that diff is failing to call smudge here, isn't it?\n\nMichael\n"},{"id":"107951","messageId":"fabb9a1e0903130719w6c5ddb3dre040091cfb373de1@mail.gmail.com","threadId":"18292","inReplyTo":"49BA6606.1070403@fastmail.fm","subject":"Re: Transparently encrypt repository contents with GPG","fromName":"Sverre Rabbelier","fromEmail":"srabbelier@gmail.com","sentAt":"2009-03-13T14:19:24Z","receivedAt":"2009-03-13T14:19:24Z","isPatch":false,"sender":{"key":"srabbelier@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3098?v=4"},"body":"Heya,\n\nOn Fri, Mar 13, 2009 at 14:56, Michael J Gruber\n wrote:\n> Sverre was being prophetic with the somehow. Here's a working setup\n> (though I still don't know why not to use luks):\n\nGlad to hear I was right ;). Also awesome that you looked into this\nand shared your findings, thanks!\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"107970","messageId":"20090313171312.GB16504@sigill.intra.peff.net","threadId":"18292","inReplyTo":"49BA6606.1070403@fastmail.fm","subject":"Re: Transparently encrypt repository contents with GPG","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-03-13T17:13:12Z","receivedAt":"2009-03-13T17:13:12Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Mar 13, 2009 at 02:56:22PM +0100, Michael J Gruber wrote:\n\n> Sverre was being prophetic with the somehow. Here's a working setup\n> (though I still don't know why not to use luks):\n> \n> In .gitattributes (or.git/info/a..) use\n> \n> * filter=gpg diff=gpg\n> \n> In your config:\n> \n> [filter \"gpg\"]\n> smudge = gpg -d -q --batch --no-tty\n> clean = gpg -ea -q --batch --no-tty -r C920A124\n> [diff \"gpg\"]\n> textconv = decrypt\n> \n> This gives you textual diffs even in log! You want use gpg-agent here.\n\nThis is not going to work very well in general. Smudging and cleaning\nis about putting the canonical version of a file in the git repo, and\nmunging it for the working tree. Trying to go backwards is going to lead\nto problems, including:\n\n 1. Git sometimes wants to look at content of special files inside\n trees, like .gitignore. Now it can't.\n\n 2. Git uses timestamps and inodes to decide whether files need to be\n looked at all to determine if they are different. So when you do\n a checkout and \"git diff\", everything will look OK. But when it\n does actually look at file contents, it compares canonical\n versions. And your canonical versions are going to be _different_\n everytime you encrypt, even if the content is the same:\n\n echo content >file\n git add file\n git diff ;# no output\n touch file\n git diff ;# looks like file is totally rewritten\n\n So you will probably end up with extra cruft in your commits if you\n ever touch files.\n\n> Now for Sverre's prophecy and the helper I haven't shown you yet: It\n> turns out that blobs are not smudged before they are fed to textconv!\n> [Also, it seems that the textconv config does allow parameters, bit I\n> haven't checked thoroughly.]\n\nI don't think they should be smudged. Smudging is about converting for\nthe working tree, and the diff is operating on canonical formats. If\nanything, I think the error is that we feed smudged data from the\nworking tree to textconv; we should always be handing it clean data (and\nthis goes for external diff, too, which I suspect behaves the same way).\n\nI haven't looked, but it probably is a result of the optimization to\nreuse worktree files.\n\n-Peff\n\nPS If it isn't obvious, I don't think this smudge/filter technique is\nthe right way to go about this. But one final comment if you did want to\npursue this: you are using asymmetric encryption in your GPG invocation,\nwhich is going to be a lot slower and the result will take up more\nspace. Try using a symmetric cipher.\n"},{"id":"107981","messageId":"7vy6v9f9zn.fsf@gitster.siamese.dyndns.org","threadId":"18292","inReplyTo":"49BA6606.1070403@fastmail.fm","subject":"Re: Transparently encrypt repository contents with GPG","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-03-13T20:23:08Z","receivedAt":"2009-03-13T20:23:08Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Michael J Gruber writes:\n\n> In .gitattributes (or.git/info/a..) use\n>\n> * filter=gpg diff=gpg\n>\n> In your config:\n>\n> [filter \"gpg\"]\n> smudge = gpg -d -q --batch --no-tty\n> clean = gpg -ea -q --batch --no-tty -r C920A124\n> [diff \"gpg\"]\n> textconv = decrypt\n>\n> This gives you textual diffs even in log! You want use gpg-agent here.\n\nDon't do this.\n\nThink why the smudge/clean pair exists.\n\nThe version controlled data, the contents, may not be suitable for\nconsumption in the work tree in its verbatim form. For example, a cross\nplatform project would want to consistently use LF line termination inside\na repository, but on a platform whose tools expect CRLF line endings, the\ncontents cannot be used verbatim. We \"smudge\" the contents running\nunix2dos when checking things out on such platforms, and \"clean\" the\nplatform specific CRLF line endings by running dos2unix when checking\nthings in. By doing so, you can see what really got changed between\nversions without getting distracted, and more importantly, \"you\" in this\nsentence is not limited to the human end users alone.\n\ngit internally runs diff and xdelta to see what was changed, so that:\n\n * it can reduce storage requirement when it runs pack-objects;\n\n * it can check what path in the preimage was similar to what other path\n in the postimage, to deduce a rename;\n\n * it can check what blocks of lines in the postimage came from what other\n blocks of lines in the preimage, to pass blames across file boundaries.\n\nIf your \"clean\" encrypts and \"smudge\" decrypts, it means you are refusing\nall the benifit git offers. You are making a pair of similar \"smudged\"\ncontents totally dissimilar in their \"clean\" counterparts. That is simply\nbackwards.\n\nAs the sole raison d'etre of diff.textconv is to allow potentially lossy\nconversion (e.g. msword-to-text) applied to the preimage and postimage\npair of contents (that are supposed to be \"clean\") before giving a textual\ndiff to human consumption, the above config may appear to work, but if you\nreally want an encrypted repository, you should be using an encrypting\nfilesystem. That would give an added benefit that the work tree\nassociated with your repository would also be encrypted.\n"},{"id":"108004","messageId":"49BB920A.20301@drmicha.warpmail.net","threadId":"18292","inReplyTo":"7vy6v9f9zn.fsf@gitster.siamese.dyndns.org","subject":"Re: Transparently encrypt repository contents with GPG","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2009-03-14T11:16:26Z","receivedAt":"2009-03-14T11:16:26Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Junio C Hamano venit, vidit, dixit 13.03.2009 21:23:\n> Michael J Gruber writes:\n> \n>> In .gitattributes (or.git/info/a..) use\n>>\n>> * filter=gpg diff=gpg\n>>\n>> In your config:\n>>\n>> [filter \"gpg\"]\n>> smudge = gpg -d -q --batch --no-tty\n>> clean = gpg -ea -q --batch --no-tty -r C920A124\n>> [diff \"gpg\"]\n>> textconv = decrypt\n>>\n>> This gives you textual diffs even in log! You want use gpg-agent here.\n> \n> Don't do this.\n> \n> Think why the smudge/clean pair exists.\n> \n> The version controlled data, the contents, may not be suitable for\n> consumption in the work tree in its verbatim form. For example, a cross\n> platform project would want to consistently use LF line termination inside\n> a repository, but on a platform whose tools expect CRLF line endings, the\n> contents cannot be used verbatim. We \"smudge\" the contents running\n> unix2dos when checking things out on such platforms, and \"clean\" the\n> platform specific CRLF line endings by running dos2unix when checking\n> things in. By doing so, you can see what really got changed between\n> versions without getting distracted, and more importantly, \"you\" in this\n> sentence is not limited to the human end users alone.\n> \n> git internally runs diff and xdelta to see what was changed, so that:\n> \n> * it can reduce storage requirement when it runs pack-objects;\n> \n> * it can check what path in the preimage was similar to what other path\n> in the postimage, to deduce a rename;\n> \n> * it can check what blocks of lines in the postimage came from what other\n> blocks of lines in the preimage, to pass blames across file boundaries.\n> \n> If your \"clean\" encrypts and \"smudge\" decrypts, it means you are refusing\n> all the benifit git offers. You are making a pair of similar \"smudged\"\n> contents totally dissimilar in their \"clean\" counterparts. That is simply\n> backwards.\n> \n> As the sole raison d'etre of diff.textconv is to allow potentially lossy\n> conversion (e.g. msword-to-text) applied to the preimage and postimage\n> pair of contents (that are supposed to be \"clean\") before giving a textual\n> diff to human consumption, the above config may appear to work, but if you\n> really want an encrypted repository, you should be using an encrypting\n> filesystem. That would give an added benefit that the work tree\n> associated with your repository would also be encrypted.\n\nExactly. This is why I suggested using cryptfs/luks in my first response\nalready.\n\nBut I don't know the OP's requirements, which is why I also told him how\nto do what he wanted, even though it has the drawbacks you and Jeff (and\nmaybe I) mentioned. Maybe it's an attempt at hosting a semi-private repo\non a public (free) server?\n\nBesides the non-text nature of encrypted content, the problem here is\nthat d(e(x))=x for all x but e(d(x)) differs from x most probably, and\nhopefully randomly, unless you use the right version of debian's openssl\nof course ;)\n\nThat being said:\ngit diff calls textconv filters with smudged as well as cleaned files\n(when diffing work tree files to blobs), and this does not seem right. I\nhope this is not happening with the internal diff, nor with crlf!\n\nSince both the cleaned and the smudged version are supposed to be\n\"authoritative\" (as opposed to the textconv'ed one) one may argue either\nway what's the right approach. For internal use comparing the cleaned\nversions may make more sense, for displaying diff's the checked-out\nform, i.e. smudged versions make more sense.\n\nBut that is another topic which would need to be substantiated with\ntests. It's not completely unlikely I may come up with some, but don't\ncount on it...\n\nCheers,\nMichael\n"},{"id":"108015","messageId":"7viqmcaqov.fsf@gitster.siamese.dyndns.org","threadId":"18292","inReplyTo":"49BB920A.20301@drmicha.warpmail.net","subject":"Re: Transparently encrypt repository contents with GPG","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-03-14T18:45:52Z","receivedAt":"2009-03-14T18:45:52Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Michael J Gruber writes:\n\n> Since both the cleaned and the smudged version are supposed to be\n> \"authoritative\" (as opposed to the textconv'ed one) one may argue either\n> way what's the right approach.\n\nSmudged one can never be authoritative. That is the whole point of smudge\nfilter and in general the whole convert_to_working_tree() infrastructure.\nIt changes depending on who you are (e.g. on what platform you are on).\nSo running comparison between two clean versions is the only sane thing to\ndo.\n\nYou could argue textconv should work on smudged contents or on clean\ncontents before smudging. As long as it is done consistently, I do not\ncare either way too deeply, as its output is not supposed to be used for\nanything but human consumption. Two equally sane arrangement would be:\n\n (1) Start from two clean contents (run convert_to_git() if contents were\n obtained from the work tree), run textconv, run diff, and output the\n result literally; or\n\n (2) Start from two smudged contents (run convert_to_working_tree() for\n contents taken from the repository), run textconv, run diff, and\n run clean before sending the result to the output.\n\nThe former assumes a textconv filter that wants to work on clean\ncontents, the latter for a one that expects smudged input. I probably\nwould suggest going the former approach, as it is consistent with the\ngeneral principle in other parts of the system (the internal processing\nhappens on clean contents).\n\nBoth of the above two assumes that the output should come in clean form;\nit is consistent with the way normal diff is generated for consumption by\ngit-apply. You can certainly argue that the final output should be in\nsmudged form when textconv is used, as it is purely for human consumption,\nand is not even supposed to be fed to apply.\n"},{"id":"108096","messageId":"49BE77DD.5020407@drmicha.warpmail.net","threadId":"18292","inReplyTo":"7viqmcaqov.fsf@gitster.siamese.dyndns.org","subject":"Re: Transparently encrypt repository contents with GPG","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2009-03-16T16:01:33Z","receivedAt":"2009-03-16T16:01:33Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Junio C Hamano venit, vidit, dixit 14.03.2009 19:45:\n> Michael J Gruber writes:\n> \n>> Since both the cleaned and the smudged version are supposed to be\n>> \"authoritative\" (as opposed to the textconv'ed one) one may argue either\n>> way what's the right approach.\n> \n> Smudged one can never be authoritative. That is the whole point of smudge\n> filter and in general the whole convert_to_working_tree() infrastructure.\n> It changes depending on who you are (e.g. on what platform you are on).\n> So running comparison between two clean versions is the only sane thing to\n> do.\n\nYes. I guess I'm being too much of a mathematician here: if clean is a\nwell-defined function, then clean(x) is well defined by specifying x. In\nthat sense x is equally authoritative.\nAgain, if smudge is the inverse of clean, i.e. smudge and clean are\nbijective, then x differs from y iff clean(x) differs from clean(y).\n\n> You could argue textconv should work on smudged contents or on clean\n> contents before smudging. As long as it is done consistently, I do not\n> care either way too deeply, as its output is not supposed to be used for\n> anything but human consumption. Two equally sane arrangement would be:\n> \n> (1) Start from two clean contents (run convert_to_git() if contents were\n> obtained from the work tree), run textconv, run diff, and output the\n> result literally; or\n> \n> (2) Start from two smudged contents (run convert_to_working_tree() for\n> contents taken from the repository), run textconv, run diff, and\n> run clean before sending the result to the output.\n> \n> The former assumes a textconv filter that wants to work on clean\n> contents, the latter for a one that expects smudged input. I probably\n> would suggest going the former approach, as it is consistent with the\n> general principle in other parts of the system (the internal processing\n> happens on clean contents).\n> \n> Both of the above two assumes that the output should come in clean form;\n> it is consistent with the way normal diff is generated for consumption by\n> git-apply. You can certainly argue that the final output should be in\n> smudged form when textconv is used, as it is purely for human consumption,\n> and is not even supposed to be fed to apply.\n\nAlso, I don't expect clean to be necessarily meaningful when applied to\nthe result of textconv, and even less so to the output of diff.\n\nNow, a simple test shows that git diff obviously does this when diffing\nHEAD to worktree:\n\ndiff between HEAD and clean(worktree)\n\nWhich is the right thing. It just seems so that textconv is not even\ncalled \"in the wrong place of the chain\", but messes the diff up in this\nway:\n\ndiff between textconv(HEAD) and textconv(worktree)\n\n(I expected clean(textconv(worktree)) first, which would be wrong, too).\nI.e., the clean filter is ignored completely in the presence of textconv.\n\nOK, I'll stop bugging you, until I checked the existing tests and the\ncode...\n\nMichael\n"},{"id":"108187","messageId":"20090317074054.GA18475@coredump.intra.peff.net","threadId":"18292","inReplyTo":"49BE77DD.5020407@drmicha.warpmail.net","subject":"Re: Transparently encrypt repository contents with GPG","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-03-17T07:40:54Z","receivedAt":"2009-03-17T07:40:54Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Mar 16, 2009 at 05:01:33PM +0100, Michael J Gruber wrote:\n\n> Now, a simple test shows that git diff obviously does this when diffing\n> HEAD to worktree:\n> \n> diff between HEAD and clean(worktree)\n> \n> Which is the right thing. It just seems so that textconv is not even\n> called \"in the wrong place of the chain\", but messes the diff up in this\n> way:\n> \n> diff between textconv(HEAD) and textconv(worktree)\n> \n> (I expected clean(textconv(worktree)) first, which would be wrong, too).\n> I.e., the clean filter is ignored completely in the presence of textconv.\n\nYeah, I think this should probably be textconv(clean(worktree)) to match\nthe regular HEAD/worktree diff (if it isn't already). Can you put\ntogether a test that shows the breakage?\n\n-Peff\n"},{"id":"108192","messageId":"20090317082239.GF18475@coredump.intra.peff.net","threadId":"18292","inReplyTo":"7vy6v9f9zn.fsf@gitster.siamese.dyndns.org","subject":"Re: Transparently encrypt repository contents with GPG","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-03-17T08:22:39Z","receivedAt":"2009-03-17T08:22:39Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Mar 13, 2009 at 01:23:08PM -0700, Junio C Hamano wrote:\n\n> As the sole raison d'etre of diff.textconv is to allow potentially lossy\n> conversion (e.g. msword-to-text) applied to the preimage and postimage\n> pair of contents (that are supposed to be \"clean\") before giving a textual\n> diff to human consumption, the above config may appear to work, but if you\n> really want an encrypted repository, you should be using an encrypting\n> filesystem. That would give an added benefit that the work tree\n> associated with your repository would also be encrypted.\n\nI can think of one reason that having git do the encryption might be\nbeneficial: pushing to an untrusted source.\n\nIf you encrypted all blobs but kept trees and commits in plaintext, you\ncould retain (some of) the benefits of git's incremental push. The\ndownsides, though, are:\n\n 1. You are revealing the hashes of your blobs' plaintext. Which means\n I can try brute-forcing your blobs by checking against a hash\n function.\n\n 2. The remote can't actually look at the blobs. The most obvious\n problem with this is that you can't send it thin packs, since it\n can't actually resolve deltas.\n\nAnd given the ensuing mess that it would make of the code to\nconditionally say \"Oh, we have this object, but you're not allowed to\nread it\", it is almost certainly not worth it.\n\nBut maybe somebody can prove me wrong and design a system that allows\nefficient encrypted pushing to a non-trusted remote and also doesn't\nsuck.\n\n-Peff\n"},{"id":"189808","messageId":"1335029110871-7487506.post@n2.nabble.com","threadId":"18292","inReplyTo":"978bdee00903121419o61cd7a87rb55809796bd257d7@mail.gmail.com","subject":"Re: Transparently encrypt repository contents with GPG","fromName":"bigbear","fromEmail":"urs.rau@gmail.com","sentAt":"2012-04-21T17:25:10Z","receivedAt":"2012-04-21T17:25:10Z","isPatch":false,"sender":{"key":"urs.rau@gmail.com","avatar":null},"body":"\nMatthias Nothhaft wrote\n> \n> Hi,\n> \n> I'm new to Git but I really already love it. ;-)\n> \n> I would like to have repository that transparently encrypts and\n> decrypts all files using GPG.\n> \n> What I need is a way to automatically modify each file\n> \n> a) before it is written in the repository\n> b) after it is read from the repository\n> \n> Is there a way to get this work somehow? Can someone give me some\n> hints where I need to begin?\n> \n> regards,\n> Matthias\n> \n> \n\nHave come across this on my own search for an encrypted git repo. Matthias\nit looks as if somebody has come up with a \"working\" system that uses the\n'smudge & clean' filter features of git. \nSeems to me that to use it for storing the repo on a non trusted or possibly\npublic git repo with some private content in the files this seems to be a\nworkable solution.\n\nTransparent Git Encryption\nhttps://gist.github.com/873637\nand/or possibly \nhttps://github.com/shadowhand/git-encrypt\n\nThe way to do this is to use git's \"smudge\" and \"clean\" filters, but it's\nnot necessarily recommended for reasons that are explained here by Junio C\nHamano, the maintainer of git:\n\n http://article.gmane.org/gmane.comp.version-control.git/113221\n\n\n\n\n\n\n--\nView this message in context: http://git.661346.n2.nabble.com/Transparently-encrypt-repository-contents-with-GPG-tp2470145p7487506.html\nSent from the git mailing list archive at Nabble.com.\n"},{"id":"193759","messageId":"1339918412381-7561644.post@n2.nabble.com","threadId":"18292","inReplyTo":"1335029110871-7487506.post@n2.nabble.com","subject":"Re: Transparently encrypt repository contents with GPG","fromName":"lalebarde","fromEmail":"l.alebarde@free.fr","sentAt":"2012-06-17T07:33:32Z","receivedAt":"2012-06-17T07:33:32Z","isPatch":false,"sender":{"key":"l.alebarde@free.fr","avatar":"https://gravatar.com/avatar/fda8e675abcdda4aa434c0ba02e5765afadb490c4017a9e9d03cc63f0fa290f0?d=mp&s=160"},"body":"Hi,\nI am puzzled from the \nhttp://article.gmane.org/gmane.comp.version-control.git/113221\nrecommandation of Junio C Hamano , the maintainer of git, to not encrypt\nfiles before pushing them :\n\nJunio C Hamano wrote\n> If your \"clean\" encrypts and \"smudge\" decrypts, it means you are refusing\n> all the benifit git offers.\n\nJunio C Hamano wrote\n> the above config may appear to work\n*So, does it work or not, or partially ? And if partially, what does not\nwork ?*\n\nAnother issue is the use of the cypher ECB by \nhttps://github.com/shadowhand/git-encrypt git-encrypt . \nhttp://stackoverflow.com/questions/1220751/how-to-choose-an-aes-encryption-mode-cbc-ecb-ctr-ocb-cfb\nSome argue it is bad (cf also \nhttp://en.wikipedia.org/wiki/Block_cipher_modes_of_operation#Electronic_codebook_.28ECB.29\nthat ). \n\nSo I made some experiments, tacking a 15Mb pdf :\n\n/$ openssl enc -base64 -aes-256-ecb -S 1762851 -k a5G4juy64VVBgfq4\nWileyE1\n$ openssl enc -base64 -aes-256-ecb -S 1762851 -k a5G4juy64VVBgfq4 WileyE2\n$ md5sum WileyE1\nd43058d8443777aea871350245d9865b WileyE1\n$ md5sum WileyE2\nd43058d8443777aea871350245d9865b WileyE2\n\n$ openssl enc -base64 -aes-256-ofb -S 1762851 -k a5G4juy64VVBgfq4 WileyE1\n$ openssl enc -base64 -aes-256-ofb -S 1762851 -k a5G4juy64VVBgfq4 WileyE2\n503d82849ad53652268d1abdcfbce9de WileyE1\n503d82849ad53652268d1abdcfbce9de WileyE2\n\n$ openssl enc -base64 -aes-256-cbc -S 1762851 -k a5G4juy64VVBgfq4 WileyE1\n$ openssl enc -base64 -aes-256-cbc -S 1762851 -k a5G4juy64VVBgfq4 WileyE2\ne726431cbd9ff8780946ddfad775600a WileyE1\ne726431cbd9ff8780946ddfad775600a WileyE2/\n\n*As the hash are identical from one run to another, I don't understand why\nwe should stick to the ECB cypher.*\n\nCan some one clarify the two points please ?\n\n\n--\nView this message in context: http://git.661346.n2.nabble.com/Transparently-encrypt-repository-contents-with-GPG-tp2470145p7561644.html\nSent from the git mailing list archive at Nabble.com.\n"},{"id":"193800","messageId":"1340049816370-7561687.post@n2.nabble.com","threadId":"18292","inReplyTo":"CAL1Gx-Ufs8TNVeeefAXBnX-eCnEk_DC1w6oJVRPcMcStdL_+-Q@mail.gmail.com","subject":"Re: Transparently encrypt repository contents with GPG","fromName":"lalebarde","fromEmail":"l.alebarde@free.fr","sentAt":"2012-06-18T20:03:36Z","receivedAt":"2012-06-18T20:03:36Z","isPatch":false,"sender":{"key":"l.alebarde@free.fr","avatar":"https://gravatar.com/avatar/fda8e675abcdda4aa434c0ba02e5765afadb490c4017a9e9d03cc63f0fa290f0?d=mp&s=160"},"body":"Thanks for your clarifications ! stars 2 & 3 are still not clear for me.\nProbably because I am new to git.\n\nDo you think that if a solution is found, in the hypothesis it respects both\ngit & strong cryptography, it would have success ? My analyse is that small\nenterprises that do not have many servers nor premises may need git hosting.\nEven big companies with their own networks if they want more security. \n\nTrueCrypt or encrypted file system on the host is not feasible off the\nshelves. One have to settle its own dedicated server at the host.\n\nOn my side, I am afraid to push my projects in clear into a host. But\npossibly I am too much paranoïde. Do you have an idea of the risk ?\n\n--\nView this message in context: http://git.661346.n2.nabble.com/Transparently-encrypt-repository-contents-with-GPG-tp2470145p7561687.html\nSent from the git mailing list archive at Nabble.com.\n"}]}