{"thread":{"id":"48449","subject":"[Best Practices Request] clean/smudge configuration","startedAt":"2018-05-09T22:08:59Z","lastAt":"2018-05-11T19:25:44Z","messageCount":6,"participants":["Randall S. Becker","Bryan Turner","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"347142","messageId":"004d01d3e7e2$517f78d0$f47e6a70$@nexbridge.com","threadId":"48449","inReplyTo":null,"subject":"[Best Practices Request] clean/smudge configuration","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2018-05-09T22:08:48Z","receivedAt":"2018-05-09T22:08:59Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"Hi Git Team,\n\nI'm trying to work out some best practices for managing clean/smudge filters\nand hit a bump. The situation is that I have an environment where the\npossible clean/smudge filter configuration can change over time and needs to\nbe versioned with the product being managed in a git repository. Part of the\nconfiguration is no problem in .gitattributes, but the other bits are in\n.git/config. I get that the runnable part of the filters need to be strictly\nplatform independent in principle, but I can abstract that part in this\nsituation.\n\nThe question: what is the best practice for versioning the parts of\nclean/smudge filters that are in .git/config given that only some users in\nmy environment will be cloning the repository in question and that I really\ncan't put the entries in /etc/gitconfig or ~/.gitconfig because of potential\nconflicts with other repositories that might also have clean/smudge\ndefinitions.\n\nThanks,\nRandall \n\n\n\n\n"},{"id":"347143","messageId":"CAGyf7-GJgpCZ=wEorLxYOm_ABaDamWKkTnOtaAwti35DJ_zS7g@mail.gmail.com","threadId":"48449","inReplyTo":"004d01d3e7e2$517f78d0$f47e6a70$@nexbridge.com","subject":"Re: [Best Practices Request] clean/smudge configuration","fromName":"Bryan Turner","fromEmail":"bturner@atlassian.com","sentAt":"2018-05-09T22:39:00Z","receivedAt":"2018-05-09T22:39:15Z","isPatch":false,"sender":{"key":"bturner@atlassian.com","avatar":"https://gravatar.com/avatar/16bcf3167981c1ef7c804e502642366d888a35b0d0b0a4ca01fdc442aa1acb1e?d=mp&s=160"},"body":"On Wed, May 9, 2018 at 3:09 PM Randall S. Becker <rsbecker@nexbridge.com>\nwrote:\n\n> The question: what is the best practice for versioning the parts of\n> clean/smudge filters that are in .git/config given that only some users in\n> my environment will be cloning the repository in question and that I\nreally\n> can't put the entries in /etc/gitconfig or ~/.gitconfig because of\npotential\n> conflicts with other repositories that might also have clean/smudge\n> definitions.\n\nDepending on level of trust, one approach might be to use an [include] in\n.git/config to include a file that's in the repository. Something like:\n\n[include]\n     path = ../path/to/config\n\nThat way that part of the configuration can be derived from the\nrepository's contents. If you checkout a revision where the file doesn't\nexist, then Git just ignores the include.\n\nYou're trusting the repository's contents, though, at that point; whatever\nis in that file will be included in your overall config and honored by Git.\n\nBryan\n"},{"id":"347144","messageId":"006201d3e7e9$17c1a800$4744f800$@nexbridge.com","threadId":"48449","inReplyTo":"CAGyf7-GJgpCZ=wEorLxYOm_ABaDamWKkTnOtaAwti35DJ_zS7g@mail.gmail.com","subject":"RE: [Best Practices Request] clean/smudge configuration","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2018-05-09T22:57:18Z","receivedAt":"2018-05-09T22:57:30Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On May 9, 2018 6:39 PM, Bryan Turner wrote\n> \n> On Wed, May 9, 2018 at 3:09 PM Randall S. Becker\n> <rsbecker@nexbridge.com>\n> wrote:\n> \n> > The question: what is the best practice for versioning the parts of\n> > clean/smudge filters that are in .git/config given that only some\n> > users in my environment will be cloning the repository in question and\n> > that I\n> really\n> > can't put the entries in /etc/gitconfig or ~/.gitconfig because of\n> potential\n> > conflicts with other repositories that might also have clean/smudge\n> > definitions.\n> \n> Depending on level of trust, one approach might be to use an [include] in\n> .git/config to include a file that's in the repository. Something like:\n> \n> [include]\n>      path = ../path/to/config\n\nIt's a possibility, but I don't like the implications. Files that are subject to the clean/smudge would need to be reprocessed manually. In the scenario:\n\n1. A checkout is done, changing ../path/to/config.\n2. The clean/smudge configuration changes in ../path/to/config, but the files impacted by it do not.\n3. git does not look like it would not be aware of the change until after the checkout, which is too late.\n4. The work tree is now inconsistent with the idempotency of the clean/smudge rules, basically because nothing happened. (not blaming git here, just timing).\n\nAs far as I understand, this is a bit of a chicken-and-egg problem because the clean/smudge config needs to be there before the checkout. Correct?\n\nCheers,\nRandall\n\n"},{"id":"347307","messageId":"001401d3e899$60cf3930$226dab90$@nexbridge.com","threadId":"48449","inReplyTo":"CAGyf7-GJgpCZ=wEorLxYOm_ABaDamWKkTnOtaAwti35DJ_zS7g@mail.gmail.com","subject":"RE: [Best Practices Request] clean/smudge configuration","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2018-05-10T19:59:12Z","receivedAt":"2018-05-10T19:59:22Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"\n\nOn May 9, 2018 6:39 PM, Bryan Turner wrote:\n> On Wed, May 9, 2018 at 3:09 PM Randall S. Becker\n> <rsbecker@nexbridge.com>\n> wrote:\n> \n> > The question: what is the best practice for versioning the parts of\n> > clean/smudge filters that are in .git/config given that only some\n> > users in my environment will be cloning the repository in question and\n> > that I\n> really\n> > can't put the entries in /etc/gitconfig or ~/.gitconfig because of\n> potential\n> > conflicts with other repositories that might also have clean/smudge\n> > definitions.\n> \n> Depending on level of trust, one approach might be to use an [include] in\n> .git/config to include a file that's in the repository. Something like:\n> \n> [include]\n>      path = ../path/to/config\n> \n\nWhat if we create a ../.gitconfig like ../.gitattributes, that is loaded\nbefore .git/config? With loads of warnings in the documentation about what\n*NOT* to put in here, any platform specifics and your own risk. The code in\nconfig.c would look like the following, with obvious updates to\ndocumentation and the test suite, so it's not fully baked yet. So far, I\ndon't have a solution to the chicken-and-egg problem, other than this.\nHowever, if I'm barking up the wrong ballpark...\n\ndiff --git a/config.c b/config.c\nindex b0c20e6cb..75d5288ff 100644\n--- a/config.c\n+++ b/config.c\n@@ -1555,11 +1555,15 @@ static int do_git_config_sequence(const struct\nconfig_options *opts,\n        char *xdg_config = xdg_config_home(\"config\");\n        char *user_config = expand_user_path(\"~/.gitconfig\", 0);\n        char *repo_config;\n+       char *repo_config_versioned;\n\n-       if (opts->commondir)\n+       if (opts->commondir) {\n                repo_config = mkpathdup(\"%s/config\", opts->commondir);\n-       else\n+               repo_config_versioned = mkpathdup(\"%s/../.gitconfig\",\nopts->commondir);\n+       } else {\n                repo_config = NULL;\n+               repo_config_versioned = NULL;\n+       }\n\n        current_parsing_scope = CONFIG_SCOPE_SYSTEM;\n        if (git_config_system() && !access_or_die(git_etc_gitconfig(), R_OK,\n0))\n@@ -1574,6 +1578,8 @@ static int do_git_config_sequence(const struct\nconfig_options *opts,\n                ret += git_config_from_file(fn, user_config, data);\n\n        current_parsing_scope = CONFIG_SCOPE_REPO;\n+       if (repo_config_versioned && !access_or_die(repo_config_versioned,\nR_OK, 0))\n+               ret += git_config_from_file(fn, repo_config_versioned,\ndata);\n        if (repo_config && !access_or_die(repo_config, R_OK, 0))\n                ret += git_config_from_file(fn, repo_config, data);\n\n@@ -1585,6 +1591,7 @@ static int do_git_config_sequence(const struct\nconfig_options *opts,\n        free(xdg_config);\n        free(user_config);\n        free(repo_config);\n+       free(repo_config_versioned);\n        return ret;\n }\n\n\n"},{"id":"347326","messageId":"xmqq7eobhqpi.fsf@gitster-ct.c.googlers.com","threadId":"48449","inReplyTo":"001401d3e899$60cf3930$226dab90$@nexbridge.com","subject":"Re: [Best Practices Request] clean/smudge configuration","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-05-11T02:27:21Z","receivedAt":"2018-05-11T02:27:27Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Randall S. Becker\" <rsbecker@nexbridge.com> writes:\n\n> What if we create a ../.gitconfig like ../.gitattributes, that is loaded\n> before .git/config?\n\nYou should not forget one of the two reasons why clean/smudge/diff\netc. drivers must be given with a step of redirection, split between\nattributes and config.  \"This path holds text from ancient macs\" at\nthe logical level may be expressed in .gitattributes, and then \"when\nchecking out text from ancient macs, process the blob data with this\nprogram to turn it into a form suitable for working tree\" is given\nas a smudge filter program, that is (1) suitable for the platform\n_you_ as the writer of the config file is using *AND* (2) something\nyou are confortable running on behalf of you.\n\nWe *deliberately* lack a mechanism to pay attention to in-tree\nconfig that gets propagated to those who get them via \"git clone\",\nexactly because otherwise your upstream may not just specify a\n\"smudge\" program that your platform would be unable to run, but can\nspecify a \"smudge\" program that pretends to be a smudger, but is\nactually a program that installs a keylogger and posts your login\npassword and bank details to this mailing list ;-)\n\nSo, no, I do not think in-tree configuration will fly.\n"},{"id":"347379","messageId":"001601d3e95d$d76272a0$862757e0$@nexbridge.com","threadId":"48449","inReplyTo":"xmqq7eobhqpi.fsf@gitster-ct.c.googlers.com","subject":"RE: [Best Practices Request] clean/smudge configuration","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2018-05-11T19:25:32Z","receivedAt":"2018-05-11T19:25:44Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"\n\nOn May 10, 2018 10:27 PM, Junio C Hamano wrote:\n> \"Randall S. Becker\" <rsbecker@nexbridge.com> writes:\n> \n> > What if we create a ../.gitconfig like ../.gitattributes, that is\n> > loaded before .git/config?\n> \n> You should not forget one of the two reasons why clean/smudge/diff etc.\n> drivers must be given with a step of redirection, split between attributes and\n> config.  \"This path holds text from ancient macs\" at the logical level may be\n> expressed in .gitattributes, and then \"when checking out text from ancient\n> macs, process the blob data with this program to turn it into a form suitable\n> for working tree\" is given as a smudge filter program, that is (1) suitable for\n> the platform _you_ as the writer of the config file is using *AND* (2)\n> something you are confortable running on behalf of you.\n> \n> We *deliberately* lack a mechanism to pay attention to in-tree config that\n> gets propagated to those who get them via \"git clone\", exactly because\n> otherwise your upstream may not just specify a \"smudge\" program that your\n> platform would be unable to run, but can specify a \"smudge\" program that\n> pretends to be a smudger, but is actually a program that installs a keylogger\n> and posts your login password and bank details to this mailing list ;-)\n> \n> So, no, I do not think in-tree configuration will fly.\n\nI agree, which is why I asked the original question instead of posting it as a formal patch. We would probably get a brand new CVE implementing the proposed proto-changes even if the original intent was internal trusted repositories only. That's why I asked this as a \"Best Practices\" type question, which I think I have a better idea on now. The actual situation is rather cool from a DevOps point of view, and whatever the ultimate solution is, might make for a nice presentation at some future conference 😉.\n\nCheers and thanks,\nRandall\n\n"}]}