# [Q] Encrypted GIT? 17 messages from 2008-03-13 to 2008-03-13. Participants: Alexander Gladysh, Miklos Vajna, Johannes Schindelin, Theodore Tso, Jeff King, Thomas Harning, David Brown, Luke Lu. Thread: https://gitlist.dev/t/12665 ## Alexander Gladysh, 2008-03-13 08:48 Subject: [Q] Encrypted GIT? Message-ID: URL: https://gitlist.dev/e/c6c947f60803130148w7981a3f0r718c0801343c7b78%40mail.gmail.com ``` Hi, list! I want to create a private GIT repo (without working copy) on a machine in external data-center. While I do not actually believe that it is possible that someone who has physical access to a machine would be interested in peeking into my repo, I'd like to play safe and to have this issue covered. Please advise what is the best way to do it. Are there any existing solutions? Thanks, Alexander. ``` ## Miklos Vajna, 2008-03-13 11:47 Subject: Re: [Q] Encrypted GIT? Message-ID: <20080313114738.GC2414@genesis.frugalware.org> URL: https://gitlist.dev/e/20080313114738.GC2414%40genesis.frugalware.org In-Reply-To: ``` On Thu, Mar 13, 2008 at 11:48:53AM +0300, Alexander Gladysh wrote: > I want to create a private GIT repo (without working copy) on a > machine in external data-center. While I do not actually believe that > it is possible that someone who has physical access to a machine would > be interested in peeking into my repo, I'd like to play safe and to > have this issue covered. > > Please advise what is the best way to do it. Are there any existing solutions? i don't think but you can write a wrapper around git receive/upload-pack and use (for example) tar+gpg to keep your repo encrypted on the disc. ``` ## Johannes Schindelin, 2008-03-13 11:55 Subject: Re: [Q] Encrypted GIT? Message-ID: URL: https://gitlist.dev/e/alpine.LSU.1.00.0803131254580.1656%40racer.site In-Reply-To: <20080313114738.GC2414@genesis.frugalware.org> ``` Hi, On Thu, 13 Mar 2008, Miklos Vajna wrote: > On Thu, Mar 13, 2008 at 11:48:53AM +0300, Alexander Gladysh wrote: > > I want to create a private GIT repo (without working copy) on a > > machine in external data-center. While I do not actually believe that > > it is possible that someone who has physical access to a machine would > > be interested in peeking into my repo, I'd like to play safe and to > > have this issue covered. > > > > Please advise what is the best way to do it. Are there any existing > > solutions? > > i don't think but you can write a wrapper around git receive/upload-pack > and use (for example) tar+gpg to keep your repo encrypted on the disc. The problem is: you cannot decrypt on the remote side, otherwise you will lose all the security. But if you do not decrypt on the remote side, you cannot store deltified objects (you lose all the benefits of Git's efficient storage), neither can you update incrementally (you lose all the benefits of Git's efficient transport). The latter can be remedied (somewhat) by encrypting each object individually. In that case, .gitattributes can help (you should be able to find a mail to that extent, which I sent no more than 2 weeks ago). However, you must make sure that the encryption is repeatable, i.e. two different encryption runs _must_ result in _identical_ output. If it is only a single file containing all your secrets, it can also make sense to just encrypt it, and track the _encrypted_ file directly (without clean/smudge filters). Hth, Dscho ``` ## Miklos Vajna, 2008-03-13 12:16 Subject: Re: [Q] Encrypted GIT? Message-ID: <20080313121644.GD2414@genesis.frugalware.org> URL: https://gitlist.dev/e/20080313121644.GD2414%40genesis.frugalware.org In-Reply-To: ``` On Thu, Mar 13, 2008 at 12:55:11PM +0100, Johannes Schindelin wrote: > The latter can be remedied (somewhat) by encrypting each object > individually. In that case, .gitattributes can help (you should be able > to find a mail to that extent, which I sent no more than 2 weeks ago). > However, you must make sure that the encryption is repeatable, i.e. two > different encryption runs _must_ result in _identical_ output. afaik, this is not the case for gpg. ``` ## Theodore Tso, 2008-03-13 12:58 Subject: Re: [Q] Encrypted GIT? Message-ID: <20080313125853.GA12927@mit.edu> URL: https://gitlist.dev/e/20080313125853.GA12927%40mit.edu In-Reply-To: <20080313121644.GD2414@genesis.frugalware.org> ``` On Thu, Mar 13, 2008 at 01:16:44PM +0100, Miklos Vajna wrote: > On Thu, Mar 13, 2008 at 12:55:11PM +0100, Johannes Schindelin wrote: > > The latter can be remedied (somewhat) by encrypting each object > > individually. In that case, .gitattributes can help (you should be able > > to find a mail to that extent, which I sent no more than 2 weeks ago). > > However, you must make sure that the encryption is repeatable, i.e. two > > different encryption runs _must_ result in _identical_ output. > > afaik, this is not the case for gpg. No, and you wouldn't want to use gpg because of the overhead it adds around an encrypted message. You would need to use a raw encryption algorithm, or one with very minimal wrapping. It's normally at this point that that you'd need to bring in a security expert to ask a whole lot of questions about your exact use scenario, do a formal threat analysis, since there are all sorts of unanswered questions about what kind of key management solution you really need for your situation. It's usually not as simple as "just encrypt it". How many people need to have access to the to the repository? Do you need to revoke access to the repository later? Who is allowed to give a new person access to the repository? etc., etc., etc. - Ted ``` ## Alexander Gladysh, 2008-03-13 13:27 Subject: Re: [Q] Encrypted GIT? Message-ID: URL: https://gitlist.dev/e/c6c947f60803130627r45629099g33a0741f319dc99c%40mail.gmail.com In-Reply-To: <20080313125853.GA12927@mit.edu> ``` > It's normally at this point that that you'd need to bring in a security expert to ask a > whole lot of questions about your exact use scenario, do a formal > threat analysis, since there are all sorts of unanswered questions > about what kind of key management solution you really need for your > situation. Uh. This is for kind of hobbyist noncommercial usage, so there are not that much resources for bringing in security experts. :-) Also I do not expect this data to be protected from determined (payed) professional attack -- a determined professional would probably be able to find some weaker spot elsewhere. However I do want such attack to cost enough to ward off idle amateurs and bored professionals. :-) > It's usually not as simple as "just encrypt it". > How many people need to have access to the to the repository? Well, 2-5, up to ten, I guess. In immediate future -- two persons only. :-) > Do you need to revoke access to the repository later? Probably. But restricting remote access should be enough. > Who is allowed to give a new person access to the repository? To keep things simple, me myself only. > etc., etc., etc. Thank you, Alexander. ``` ## Johannes Schindelin, 2008-03-13 15:21 Subject: Re: [Q] Encrypted GIT? Message-ID: URL: https://gitlist.dev/e/alpine.LSU.1.00.0803131620270.1656%40racer.site In-Reply-To: <20080313125853.GA12927@mit.edu> ``` Hi, On Thu, 13 Mar 2008, Theodore Tso wrote: > On Thu, Mar 13, 2008 at 01:16:44PM +0100, Miklos Vajna wrote: > > On Thu, Mar 13, 2008 at 12:55:11PM +0100, Johannes Schindelin wrote: > > > The latter can be remedied (somewhat) by encrypting each object > > > individually. In that case, .gitattributes can help (you should be > > > able to find a mail to that extent, which I sent no more than 2 > > > weeks ago). However, you must make sure that the encryption is > > > repeatable, i.e. two different encryption runs _must_ result in > > > _identical_ output. > > > > afaik, this is not the case for gpg. > > No, and you wouldn't want to use gpg because of the overhead it adds > around an encrypted message. To the contrary: if your files are small (which they are most likely), you _want_ the overhead, in order to make the encryption harder to crack. AFAICT gpg is a good all-round encryption tool, and reinventing the wheel just for encrypting things in a git repository just does not cut it. Ciao, Dscho ``` ## Jeff King, 2008-03-13 15:53 Subject: Re: [Q] Encrypted GIT? Message-ID: <20080313155322.GA30847@coredump.intra.peff.net> URL: https://gitlist.dev/e/20080313155322.GA30847%40coredump.intra.peff.net In-Reply-To: <20080313125853.GA12927@mit.edu> ``` On Thu, Mar 13, 2008 at 08:58:54AM -0400, Theodore Tso wrote: > No, and you wouldn't want to use gpg because of the overhead it adds > around an encrypted message. You would need to use a raw encryption > algorithm, or one with very minimal wrapping. It's normally at this Well, "raw encryption algorithm" is a bit vague here. :) I thought about this a while ago and come to a few conclusions: - encrypting before git sees content sucks, because you are either sacrificing security (content X always encrypts to Y) or system stability (git doesn't know that Y and Y' are really the same thing) - encrypting at the object level (when we do zlib) sucks, because we still want to name contents by their hash, which means the object database index contains information about what's in your content. There's also some per-object overhead. Plus any system without the key can't do deltas. - encrypting whole packfiles sucks for local storage, since you lose the random access property (unless you go with something static like an ECB mode, but then you are sacrificing security). - encrypting whole packfiles is a bit better for transport. The key-holding repo does the deltas and just treats the remote repo as dumb storage (it can't be smart, since that would involve looking at the data). Storage overhead is minimal if packfiles are a reasonable size. So I think the last makes the most sense, where your local repo is totally unprotected, but you efficiently push git objects to a remote untrusted repo. You could probably do something totally external to git using bundles as the primitive. Store an encrypted index on the remote that says "here are the packs I have, and the objects they contain." Whenever you push, pull the index (which is of course more network-intensive than regular git protocol, but not as bad as pulling all the data) and calculate a thin-pack bundle yourself. Encrypt the bundle and store remotely. > point that that you'd need to bring in a security expert to ask a > whole lot of questions about your exact use scenario, do a formal > threat analysis, since there are all sorts of unanswered questions > about what kind of key management solution you really need for your > situation. I don't know if a formal thread analysis is necessary. I think most people are interested in "if the contents of remote storage X are known, how much do people know about the _contents_ of my repo stored on X?" and they don't care about masking the size, time of updates, etc. That's a fairly straightforward application of cryptography. The tricky part is doing it in a way that can still leverage some of git's efficiencies. > It's usually not as simple as "just encrypt it". How many people need > to have access to the to the repository? Do you need to revoke access > to the repository later? Who is allowed to give a new person access > to the repository? etc., etc., etc. Sure, those are all interesting questions for a complete system. But I think it makes sense to incrementally try the basics first. -Peff ``` ## Jeff King, 2008-03-13 16:00 Subject: Re: [Q] Encrypted GIT? Message-ID: <20080313160016.GB30847@coredump.intra.peff.net> URL: https://gitlist.dev/e/20080313160016.GB30847%40coredump.intra.peff.net In-Reply-To: ``` On Thu, Mar 13, 2008 at 04:21:44PM +0100, Johannes Schindelin wrote: > > No, and you wouldn't want to use gpg because of the overhead it adds > > around an encrypted message. > > To the contrary: if your files are small (which they are most likely), you > _want_ the overhead, in order to make the encryption harder to crack. Not necessarily. Using random IVs, random salts, and random padding does increase security. Adding headers to every object that tell which algorithm and parameters were used are nice for interoperability, but don't help with security. Doing per-object asymmetric encryptions (gpg --encrypt without --symmetric) is performance insanity. > AFAICT gpg is a good all-round encryption tool, and reinventing the wheel > just for encrypting things in a git repository just does not cut it. Keep in mind that in the example you posted before, you were not using 99% of gpg. You were just asking it to do a symmetric CBC cipher using a passphrase. So it is overkill for that, but at the same time not actually very flexible for doing those sorts of low-level things. OpenSSL provides a much better toolkit for that. -Peff ``` ## Jeff King, 2008-03-13 16:01 Subject: Re: [Q] Encrypted GIT? Message-ID: <20080313160154.GC30847@coredump.intra.peff.net> URL: https://gitlist.dev/e/20080313160154.GC30847%40coredump.intra.peff.net In-Reply-To: <20080313155322.GA30847@coredump.intra.peff.net> ``` On Thu, Mar 13, 2008 at 11:53:22AM -0400, Jeff King wrote: > You could probably do something totally external to git using bundles as > the primitive. Store an encrypted index on the remote that says "here > are the packs I have, and the objects they contain." Whenever you push, > pull the index (which is of course more network-intensive than regular > git protocol, but not as bad as pulling all the data) and calculate a > thin-pack bundle yourself. Encrypt the bundle and store remotely. Oh, and a scheme like this generalizes well from "there is one key" to "N asymmetric keyholders". > I don't know if a formal thread analysis is necessary. I think most That should of course be "threa_t_ analysis". I of course want to formally analyze this thread. ;) -Peff ``` ## Thomas Harning, 2008-03-13 16:10 Subject: Re: [Q] Encrypted GIT? Message-ID: <20080313121027.5f51f852@gmail.com> URL: https://gitlist.dev/e/20080313121027.5f51f852%40gmail.com In-Reply-To: ``` On Thu, 13 Mar 2008 11:48:53 +0300 "Alexander Gladysh" wrote: > Hi, list! > > I want to create a private GIT repo (without working copy) on a > machine in external data-center. While I do not actually believe that > it is possible that someone who has physical access to a machine would > be interested in peeking into my repo, I'd like to play safe and to > have this issue covered. > > Please advise what is the best way to do it. Are there any existing > solutions? > Potential solution to store arbitrary data in a safe manner: mkdir remote_git_raw remote_git sshfs : $PWD/remote_git_raw encfs $PWD/remote_git_raw $PWD/remote_git This will lock your data in a remote encfs volume. (Uses FUSE) Not quite sure about the implications on performance... but this will certainly keep your data safe on that remote location. ``` ## Theodore Tso, 2008-03-13 16:12 Subject: Re: [Q] Encrypted GIT? Message-ID: <20080313161201.GA31653@mit.edu> URL: https://gitlist.dev/e/20080313161201.GA31653%40mit.edu In-Reply-To: <20080313155322.GA30847@coredump.intra.peff.net> ``` On Thu, Mar 13, 2008 at 11:53:22AM -0400, Jeff King wrote: > - encrypting whole packfiles is a bit better for transport. The > key-holding repo does the deltas and just treats the remote repo as > dumb storage (it can't be smart, since that would involve looking at > the data). Storage overhead is minimal if packfiles are a reasonable > size. > > So I think the last makes the most sense, where your local repo is > totally unprotected, but you efficiently push git objects to a remote > untrusted repo. If the main goal is primarily backup of your repository to an untrusted remote server, yes, that makes perfect sense. If you assume multiple trusted developers would actually be *operating* on an encrypted repo, the life gets much harder, as you've pointed out. > - encrypting before git sees content sucks, because you are either > sacrificing security (content X always encrypts to Y) or system > stability (git doesn't know that Y and Y' are really the same thing) It's not clear that "content X always encrypts to Y" is a fatal flaw, by the way. Yes, it leaks a bit of information, but in a source code management situation, it may not matter. If you do absolutely care, tough, it might be that the simplest solution is to store the entire repository and working tree under cryptofs. After all, what's the point of encrypting the local repo if the checked-out working tree is unprotected for all to see? :-) - Ted ``` ## Jeff King, 2008-03-13 16:19 Subject: Re: [Q] Encrypted GIT? Message-ID: <20080313161919.GA2050@coredump.intra.peff.net> URL: https://gitlist.dev/e/20080313161919.GA2050%40coredump.intra.peff.net In-Reply-To: <20080313161201.GA31653@mit.edu> ``` On Thu, Mar 13, 2008 at 12:12:01PM -0400, Theodore Tso wrote: > If the main goal is primarily backup of your repository to an > untrusted remote server, yes, that makes perfect sense. > > If you assume multiple trusted developers would actually be > *operating* on an encrypted repo, the life gets much harder, as you've > pointed out. Well, it depends on the meaning of "operate". :) I think you could still use it as a rendezvous point as you would any bare repository. Pushing and pulling would have a little larger network overhead, and a lot more CPU overhead. But yes, that scheme is horrible for a working repo. > > - encrypting before git sees content sucks, because you are either > > sacrificing security (content X always encrypts to Y) or system > > stability (git doesn't know that Y and Y' are really the same thing) > > It's not clear that "content X always encrypts to Y" is a fatal flaw, > by the way. Yes, it leaks a bit of information, but in a source code Agreed (I actually recommended in Dscho's original thread "you can do it by eliminating the salt, if you accept the consequences..."). So after my saying "no formal threat analysis is necessary" you have clearly called me on making a bunch of usage assumptions. Oops. :) > management situation, it may not matter. If you do absolutely care, > tough, it might be that the simplest solution is to store the entire > repository and working tree under cryptofs. After all, what's the > point of encrypting the local repo if the checked-out working tree is > unprotected for all to see? :-) Yes. And it doesn't involve any git-specific code at all. :) -Peff ``` ## David Brown, 2008-03-13 17:43 Subject: Re: [Q] Encrypted GIT? Message-ID: <20080313174328.GA3783@old.davidb.org> URL: https://gitlist.dev/e/20080313174328.GA3783%40old.davidb.org In-Reply-To: ``` On Thu, Mar 13, 2008 at 12:55:11PM +0100, Johannes Schindelin wrote: >The latter can be remedied (somewhat) by encrypting each object >individually. In that case, .gitattributes can help (you should be able >to find a mail to that extent, which I sent no more than 2 weeks ago). >However, you must make sure that the encryption is repeatable, i.e. two >different encryption runs _must_ result in _identical_ output. Any decent file encryption program will never have this characteristic. It's normally a bad idea from a security perspective. David ``` ## Luke Lu, 2008-03-13 18:36 Subject: Re: [Q] Encrypted GIT? Message-ID: <51ED164C-2269-48B7-B8C9-0E819BFD63EC@vicaya.com> URL: https://gitlist.dev/e/51ED164C-2269-48B7-B8C9-0E819BFD63EC%40vicaya.com In-Reply-To: ``` On Mar 13, 2008, at 1:48 AM, Alexander Gladysh wrote: > Hi, list! > > I want to create a private GIT repo (without working copy) on a > machine in external data-center. While I do not actually believe that > it is possible that someone who has physical access to a machine would > be interested in peeking into my repo, I'd like to play safe and to > have this issue covered. > > Please advise what is the best way to do it. Are there any existing > solutions? An obvious and easy solution: use an encrypted partition on the remote server and ssh as transport. Last time I checked, git on encrypted volumes is plenty fast. __Luke ``` ## Thomas Harning, 2008-03-13 19:15 Subject: Re: [Q] Encrypted GIT? Message-ID: <20080313151532.20d72b14@gmail.com> URL: https://gitlist.dev/e/20080313151532.20d72b14%40gmail.com In-Reply-To: <51ED164C-2269-48B7-B8C9-0E819BFD63EC@vicaya.com> ``` On Thu, 13 Mar 2008 11:36:46 -0700 Luke Lu wrote: > An obvious and easy solution: use an encrypted partition on the > remote server and ssh as transport. Last time I checked, git on > encrypted volumes is plenty fast. If its an encrypted partition on the remote server... then its visible @ that server.. which I don't think is desired in the situation. An encrypted partition is fairly useless on a remote server unless the remote server is expected to be physically removed/powered down... otherwise anything can get into that data while its alive (pending permissions, lack-of-holes, etc..) The encfs solution makes sure that nothing is ever revealed remote-side... all data is prevented from even going over ssh in its unencrypted form. ``` ## Luke Lu, 2008-03-13 20:06 Subject: Re: [Q] Encrypted GIT? Message-ID: <8F9BA906-777F-4B7D-BA19-D0848D1886B3@vicaya.com> URL: https://gitlist.dev/e/8F9BA906-777F-4B7D-BA19-D0848D1886B3%40vicaya.com In-Reply-To: <20080313151532.20d72b14@gmail.com> ``` On Mar 13, 2008, at 12:15 PM, Thomas Harning wrote: > On Thu, 13 Mar 2008 11:36:46 -0700 > Luke Lu wrote: > >> An obvious and easy solution: use an encrypted partition on the >> remote server and ssh as transport. Last time I checked, git on >> encrypted volumes is plenty fast. > > If its an encrypted partition on the remote server... then its visible > @ that server.. which I don't think is desired in the situation. > > An encrypted partition is fairly useless on a remote server unless the > remote server is expected to be physically removed/powered down... > otherwise anything can get into that data while its alive (pending > permissions, lack-of-holes, etc..) > > The encfs solution makes sure that nothing is ever revealed > remote-side... all data is prevented from even going over ssh in its > unencrypted form. Yes encfs over an sshfs is probably the safest. But it is intolerably slow if you need any kind of random access of data, which git does all the time. You can mount the encrypted partition using a key over ssh per git push or pull to minimize exposure while get the performance you want. __Luke ```