{"thread":{"id":"44452","subject":"[RFC] Add way to make Git credentials accessible from clean/smudge filter","startedAt":"2016-11-10T11:52:44Z","lastAt":"2016-11-12T13:58:05Z","messageCount":9,"participants":["Lars Schneider","Matthieu Moy","Jeff King","Dennis Kaarsemaker"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"305677","messageId":"4C8C5650-7221-4F62-A9CC-81AE01EF6DC7@gmail.com","threadId":"44452","inReplyTo":null,"subject":"[RFC] Add way to make Git credentials accessible from clean/smudge filter","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2016-11-10T11:52:36Z","receivedAt":"2016-11-10T11:52:44Z","isPatch":false,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"Hi,\n\nwe just implemented the first \"real-world\" user of the new clean/smudge \n\"filter protocol\" interface (see \"convert: add filter..process option\"\nedcc858 for details) and the results are fantastic. Filtering 12,000 files in\nmy artificial test repo is more than 60x faster (depending on the platform). \nOn Windows that means the clean/smudge operations runs in 57 seconds instead \nof 55 minutes [1]!\n\nI have a number of ideas to improve the protocol even further and I am seeking\nearly feedback on the following - possibly most controversial - idea:\n\nSome filters might want to perform additional network interactions and these\nfilters would like to use the Git credentials to perform these actions. If\nsuch a filter is configured with \"offerCredentials = true\" then the filter \ncould request the current Git credentials via the \"filter-protocol\".\n\nA configuration could look like this:\n------------------------\n[filter \"myfilter\"]\n process = my-filter-process\n required = true\n offerCredentials = true\n------------------------\n\nThe default would, of course, be \"offerCredentials = false\".\n\nI haven't looked at an implemenation approach at all. I wonder if this could\nbe OK from a conceptional point of view or if there are obvious security \nproblems that I am missing.\n\nThanks,\nLars\n\n\n[1] https://github.com/github/git-lfs/pull/1617#issuecomment-259411850"},{"id":"305678","messageId":"vpqoa1n1qom.fsf@anie.imag.fr","threadId":"44452","inReplyTo":"4C8C5650-7221-4F62-A9CC-81AE01EF6DC7@gmail.com","subject":"Re: [RFC] Add way to make Git credentials accessible from clean/smudge filter","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@grenoble-inp.fr","sentAt":"2016-11-10T12:10:17Z","receivedAt":"2016-11-10T12:10:35Z","isPatch":false,"sender":{"key":"matthieu.moy@grenoble-inp.fr","avatar":"https://gravatar.com/avatar/72c8a2705971a25dfaff23cece15130d405685845d911aedd5667ace277f3fc5?d=mp&s=160"},"body":"Lars Schneider writes:\n\n> I haven't looked at an implemenation approach at all. I wonder if this could\n> be OK from a conceptional point of view or if there are obvious security \n> problems that I am missing.\n\nDid you consider just running \"git credential\" from the filter? It may\nnot be the perfect solution, but it should work. I already used it to\nget credential from a remote-helper (git-remote-mediawiki). When\nprompting credentials interactively, it grabs the terminal directly, so\nit work even if stdin/stdout are used for the protocol.\n\nAsking the main git process to get the credentials probably has added\nvalue like the ability to prompt once and use the same for several\nfilter processes.\n\n-- \nMatthieu Moy\nhttp://www-verimag.imag.fr/~moy/\n"},{"id":"305679","messageId":"20161110160809.2gvf67rlnvounulf@sigill.intra.peff.net","threadId":"44452","inReplyTo":"vpqoa1n1qom.fsf@anie.imag.fr","subject":"Re: [RFC] Add way to make Git credentials accessible from clean/smudge filter","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-11-10T16:08:09Z","receivedAt":"2016-11-10T16:10:01Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Nov 10, 2016 at 01:10:17PM +0100, Matthieu Moy wrote:\n\n> Lars Schneider writes:\n> \n> > I haven't looked at an implemenation approach at all. I wonder if this could\n> > be OK from a conceptional point of view or if there are obvious security \n> > problems that I am missing.\n> \n> Did you consider just running \"git credential\" from the filter? It may\n> not be the perfect solution, but it should work. I already used it to\n> get credential from a remote-helper (git-remote-mediawiki). When\n> prompting credentials interactively, it grabs the terminal directly, so\n> it work even if stdin/stdout are used for the protocol.\n\nYeah, that is the solution I was going to suggest. The credentials are\ntotally orthogonal to the filters, and I would rather not shove them\ninto the protocol. It's an extra process, but with the new multi-use\nsmudge filter, it's one per git invocation, not one per file.\n\n-Peff\n"},{"id":"305767","messageId":"77603924-3552-4146-9C9E-A106CFA96D7A@gmail.com","threadId":"44452","inReplyTo":"20161110160809.2gvf67rlnvounulf@sigill.intra.peff.net","subject":"Re: [RFC] Add way to make Git credentials accessible from clean/smudge filter","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2016-11-11T09:28:56Z","receivedAt":"2016-11-11T09:29:07Z","isPatch":false,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\nOn 10 Nov 2016, at 17:08, Jeff King wrote:\n\n> On Thu, Nov 10, 2016 at 01:10:17PM +0100, Matthieu Moy wrote:\n> \n>> Lars Schneider writes:\n>> \n>>> I haven't looked at an implemenation approach at all. I wonder if this could\n>>> be OK from a conceptional point of view or if there are obvious security \n>>> problems that I am missing.\n>> \n>> Did you consider just running \"git credential\" from the filter? It may\n>> not be the perfect solution, but it should work. I already used it to\n>> get credential from a remote-helper (git-remote-mediawiki). When\n>> prompting credentials interactively, it grabs the terminal directly, so\n>> it work even if stdin/stdout are used for the protocol.\n> \n> Yeah, that is the solution I was going to suggest. The credentials are\n> totally orthogonal to the filters, and I would rather not shove them\n> into the protocol. It's an extra process, but with the new multi-use\n> smudge filter, it's one per git invocation, not one per file.\n\nThe trouble with \"git credential\" is that it works only if the credential \nhelper is setup correctly. Although I assume that most people have setup this, \nI have also worked with a number of people who prefer to enter their passwords \nevery time Git makes a network connection.\n\n- Lars"},{"id":"305768","messageId":"20161111093122.osbdwmze5x5t742v@sigill.intra.peff.net","threadId":"44452","inReplyTo":"77603924-3552-4146-9C9E-A106CFA96D7A@gmail.com","subject":"Re: [RFC] Add way to make Git credentials accessible from clean/smudge filter","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-11-11T09:31:22Z","receivedAt":"2016-11-11T09:31:29Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Nov 11, 2016 at 10:28:56AM +0100, Lars Schneider wrote:\n\n> > Yeah, that is the solution I was going to suggest. The credentials are\n> > totally orthogonal to the filters, and I would rather not shove them\n> > into the protocol. It's an extra process, but with the new multi-use\n> > smudge filter, it's one per git invocation, not one per file.\n> \n> The trouble with \"git credential\" is that it works only if the credential \n> helper is setup correctly. Although I assume that most people have setup this, \n> I have also worked with a number of people who prefer to enter their passwords \n> every time Git makes a network connection.\n\nAre you sure about that? If I do:\n\n echo url=https://example.com/repo.git |\n git credential fill\n\nI get prompted for a username and password.\n\n-Peff\n"},{"id":"305769","messageId":"4060C484-B42E-42AB-BB42-4753373F0E7F@gmail.com","threadId":"44452","inReplyTo":"20161111093122.osbdwmze5x5t742v@sigill.intra.peff.net","subject":"Re: [RFC] Add way to make Git credentials accessible from clean/smudge filter","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2016-11-11T09:40:14Z","receivedAt":"2016-11-11T09:40:25Z","isPatch":false,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\nOn 11 Nov 2016, at 10:31, Jeff King wrote:\n\n> On Fri, Nov 11, 2016 at 10:28:56AM +0100, Lars Schneider wrote:\n> \n>>> Yeah, that is the solution I was going to suggest. The credentials are\n>>> totally orthogonal to the filters, and I would rather not shove them\n>>> into the protocol. It's an extra process, but with the new multi-use\n>>> smudge filter, it's one per git invocation, not one per file.\n>> \n>> The trouble with \"git credential\" is that it works only if the credential \n>> helper is setup correctly. Although I assume that most people have setup this, \n>> I have also worked with a number of people who prefer to enter their passwords \n>> every time Git makes a network connection.\n> \n> Are you sure about that? If I do:\n> \n> echo url=https://example.com/repo.git |\n> git credential fill\n> \n> I get prompted for a username and password.\n\nHm.. either I don't understand you or I expressed myself unclear. \n\nLet's say a user runs:\n\n$ git clone https://myrepo.git\n\nIf no credential helper is setup, then Git asks the user for credentials.\nAfterwards Git starts downloading stuff. At some point Git will run my\nsmudge filter on some files and in my case the smudge filter needs the\nGit credentials. AFAIK, the smudge filter has no way to get the credentials \nfrom Git at this point - not even by invoking \"git credential\". \nIs this correct?\n\n- Lars"},{"id":"305808","messageId":"1478894572.12153.1.camel@kaarsemaker.net","threadId":"44452","inReplyTo":"4060C484-B42E-42AB-BB42-4753373F0E7F@gmail.com","subject":"Re: [RFC] Add way to make Git credentials accessible from clean/smudge filter","fromName":"Dennis Kaarsemaker","fromEmail":"dennis@kaarsemaker.net","sentAt":"2016-11-11T20:02:52Z","receivedAt":"2016-11-11T20:03:01Z","isPatch":false,"sender":{"key":"dennis@kaarsemaker.net","avatar":"https://avatars.githubusercontent.com/u/200649?v=4"},"body":"On Fri, 2016-11-11 at 10:40 +0100, Lars Schneider wrote:\n> On 11 Nov 2016, at 10:31, Jeff King wrote:\n> \n> > On Fri, Nov 11, 2016 at 10:28:56AM +0100, Lars Schneider wrote:\n> > \n> > > > Yeah, that is the solution I was going to suggest. The credentials are\n> > > > totally orthogonal to the filters, and I would rather not shove them\n> > > > into the protocol. It's an extra process, but with the new multi-use\n> > > > smudge filter, it's one per git invocation, not one per file.\n> > > \n> > > \n> > > The trouble with \"git credential\" is that it works only if the credential \n> > > helper is setup correctly. Although I assume that most people have setup this, \n> > > I have also worked with a number of people who prefer to enter their passwords \n> > > every time Git makes a network connection.\n> > \n> > \n> > Are you sure about that? If I do:\n> > \n> > echo url=https://example.com/repo.git |\n> > git credential fill\n> > \n> > I get prompted for a username and password.\n> \n> \n> Hm.. either I don't understand you or I expressed myself unclear. \n> \n> Let's say a user runs:\n> \n> $ git clone https://myrepo.git\n> \n> If no credential helper is setup, then Git asks the user for credentials.\n> Afterwards Git starts downloading stuff. At some point Git will run my\n> smudge filter on some files and in my case the smudge filter needs the\n> Git credentials. AFAIK, the smudge filter has no way to get the credentials \n> from Git at this point - not even by invoking \"git credential\". \n> Is this correct?\n\nI think that's correct, but the same argument goes both ways: unless I\nuse a credential helper, or explicitely give a filter application my\ncredentials, I don't want a helper to be able to get to those\ncredentials. I'd consider that a security bug.\n\nD.\n\n"},{"id":"305811","messageId":"20161111202710.7zpalypsnorqeclq@sigill.intra.peff.net","threadId":"44452","inReplyTo":"1478894572.12153.1.camel@kaarsemaker.net","subject":"Re: [RFC] Add way to make Git credentials accessible from clean/smudge filter","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-11-11T20:27:10Z","receivedAt":"2016-11-11T20:27:28Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Nov 11, 2016 at 09:02:52PM +0100, Dennis Kaarsemaker wrote:\n\n> > > Are you sure about that? If I do:\n> > > \n> > > echo url=https://example.com/repo.git |\n> > > git credential fill\n> > > \n> > > I get prompted for a username and password.\n> > \n> > \n> > Hm.. either I don't understand you or I expressed myself unclear. \n> > \n> > Let's say a user runs:\n> > \n> > $ git clone https://myrepo.git\n> > \n> > If no credential helper is setup, then Git asks the user for credentials.\n> > Afterwards Git starts downloading stuff. At some point Git will run my\n> > smudge filter on some files and in my case the smudge filter needs the\n> > Git credentials. AFAIK, the smudge filter has no way to get the credentials \n> > from Git at this point - not even by invoking \"git credential\". \n> > Is this correct?\n> \n> I think that's correct, but the same argument goes both ways: unless I\n> use a credential helper, or explicitely give a filter application my\n> credentials, I don't want a helper to be able to get to those\n> credentials. I'd consider that a security bug.\n\nYeah, agreed. They are logically two separate operations, so I think it\nis a feature that they do not implicitly share credentials.\n\nI think the only place where we implicitly share credentials is when\nserving an HTTP fetch or push requires multiple HTTP requests. And there\nit seems pretty sane to do so.\n\n-Peff\n"},{"id":"305859","messageId":"83393B7B-E429-46AE-BFE6-F1D5C2A6D4C5@gmail.com","threadId":"44452","inReplyTo":"20161111202710.7zpalypsnorqeclq@sigill.intra.peff.net","subject":"Re: [RFC] Add way to make Git credentials accessible from clean/smudge filter","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2016-11-12T13:57:56Z","receivedAt":"2016-11-12T13:58:05Z","isPatch":false,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\n> On 11 Nov 2016, at 21:27, Jeff King wrote:\n> \n> On Fri, Nov 11, 2016 at 09:02:52PM +0100, Dennis Kaarsemaker wrote:\n> \n>>>> Are you sure about that? If I do:\n>>>> \n>>>> echo url=https://example.com/repo.git |\n>>>> git credential fill\n>>>> \n>>>> I get prompted for a username and password.\n>>> \n>>> \n>>> Hm.. either I don't understand you or I expressed myself unclear. \n>>> \n>>> Let's say a user runs:\n>>> \n>>> $ git clone https://myrepo.git\n>>> \n>>> If no credential helper is setup, then Git asks the user for credentials.\n>>> Afterwards Git starts downloading stuff. At some point Git will run my\n>>> smudge filter on some files and in my case the smudge filter needs the\n>>> Git credentials. AFAIK, the smudge filter has no way to get the credentials \n>>> from Git at this point - not even by invoking \"git credential\". \n>>> Is this correct?\n>> \n>> I think that's correct, but the same argument goes both ways: unless I\n>> use a credential helper, or explicitely give a filter application my\n>> credentials, I don't want a helper to be able to get to those\n>> credentials. I'd consider that a security bug.\n> \n> Yeah, agreed. They are logically two separate operations, so I think it\n> is a feature that they do not implicitly share credentials.\n> \n> I think the only place where we implicitly share credentials is when\n> serving an HTTP fetch or push requires multiple HTTP requests. And there\n> it seems pretty sane to do so.\n\nAgreed. Thanks for your thoughts on this!\n\n- Lars"}]}