{"thread":{"id":"23651","subject":"How to manage parameter files and code separately using git?","startedAt":"2010-05-01T12:57:31Z","lastAt":"2010-05-02T15:52:14Z","messageCount":4,"participants":["Tilo Schwarz","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"140678","messageId":"op.vb0195s1a8ed4e@dellschleppa","threadId":"23651","inReplyTo":null,"subject":"How to manage parameter files and code separately using git?","fromName":"Tilo Schwarz","fromEmail":"tilo@tilo-schwarz.de","sentAt":"2010-05-01T12:57:31Z","receivedAt":"2010-05-01T12:57:31Z","isPatch":false,"sender":{"key":"tilo@tilo-schwarz.de","avatar":null},"body":"Hi all,\n\nmaybe someone has an idea how to do what I'd like to do using git:\n\nI use git for \"normal\" c coding, i.e. having branches like master, test,  \nfoo, etc. Built are executables (Linux), which need parameter text files  \nto work. What I did up to now was to check in the parameter files in the  \nsame way I check in the code: when I change a parameter file, I do a  \ncommit on it, this way I always have a history of my parameter files.\n\nThis has one drawback: If I check out an older version, I also get the old  \nparameter file, which is not what I want, because the parameters are  \ndetermined by hardware settings. I.e., I would like to checkout an old  \ncommit, but still have the last version of the parameter file.\n\nI tried to solve the problem by\n\n1. creating a branch 'parameters', which contains only the parameter files\n2. ignore the parameters files with extension *.m in .gitignore in the  \ncode branches (master etc.)\n\nThis seemed to work at first, but when I switch back from the parameters  \nbranch to master, all *.m files are removed, although .gitignore of master  \ncontains '*.m'.\n\nSo apparently git removes during branch switch first the tracked files and  \nthan populates the working dir with the new files of master.\n\nNow my questions are\n\n1. Is there a way to work around it?\n2. Is there a maybe totally different way to solve my initial problem  \n(separate code history from parameter file history) without using two git  \nrepos?\n\n\nThanks a lot!\n\n     Tilo\n"},{"id":"140691","messageId":"7vmxwj5ym8.fsf@alter.siamese.dyndns.org","threadId":"23651","inReplyTo":"op.vb0195s1a8ed4e@dellschleppa","subject":"Re: How to manage parameter files and code separately using git?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-05-01T17:18:55Z","receivedAt":"2010-05-01T17:18:55Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Tilo Schwarz\" <tilo@tilo-schwarz.de> writes:\n\n> I use git for \"normal\" c coding, i.e. having branches like master, test,  \n> foo, etc. Built are executables (Linux), which need parameter text files  \n> to work. What I did up to now was to check in the parameter files in the  \n> same way I check in the code: when I change a parameter file, I do a  \n> commit on it, this way I always have a history of my parameter files.\n>\n> This has one drawback: If I check out an older version, I also get the old  \n> parameter file, which is not what I want, because the parameters are  \n> determined by hardware settings. I.e., I would like to checkout an old  \n> commit, but still have the last version of the parameter file.\n\nThis design constraint makes the issue not a version control problem but\nmore of a software engineering problem.  The changes to your parameter\nfile have its own history (e.g. you upgraded your machine last week with\ndifferent hardware---the software didn't change but the parameter has to)\nand the history is largely independent of the software that reads the\nfile.  In that sense, you should _never_ tie its history with the history\nof your software.\n\nAt the same time, the history of the parameter file is not completely\nindependent of that of the software.  You may have added a new feature and\ncode to read a new setting in the parameter file today and older parameter\nfile would not have a setting for that variable.\n\nThe method I used in my day-job project is something like this:\n\n - Include a \"config.sample\" file and track it in the history of the\n   source code; the values it gives to the variables are the same as the\n   default ones.  This file mostly serves as a documentation.\n\n - Design your parameter mechanism in a way that:\n\n   - a parameter file can include another;\n\n   - an unrecognized parameter setting is ignored;\n\n   - a variable definition that happens later overrides an earlier\n     definition.\n\n - Have a real parameter file that is _not_ tracked in the history of the\n   source code.  Have it include parameter.sample and then define machine\n   specific settings after that to override.\n\nIt is only required for the \"real\" parameter file not to be tracked in the\nhistory of the source code---it is perfectly fine to track it in a\nseparate history.  The easiest way to do this would be to keep a parameter\nworking tree next door, and \"ln -s ../params/config config\" in the working\ntree for the source code.  You could keep this symlink tracked in the\nsource history, but you do not have to.\n\nYou will try to read from ./config, which will include ./config.sample in\norder to read the default, and then read the customized setting that\nappear later in ./config.  Because of the symlink, you will actually be\nreading from ../params/config next door.\n"},{"id":"140747","messageId":"op.vb2ms4r8a8ed4e@dellschleppa","threadId":"23651","inReplyTo":"7vmxwj5ym8.fsf@alter.siamese.dyndns.org","subject":"Re: How to manage parameter files and code separately using git?","fromName":"Tilo Schwarz","fromEmail":"tilo@tilo-schwarz.de","sentAt":"2010-05-02T09:18:30Z","receivedAt":"2010-05-02T09:18:30Z","isPatch":false,"sender":{"key":"tilo@tilo-schwarz.de","avatar":null},"body":"On Sat, 01 May 2010 19:18:55 +0200, Junio C Hamano <gitster@pobox.com>  \nwrote:\n\nThank you for the detailed explanation!\n\n> This design constraint makes the issue not a version control problem but\n> more of a software engineering problem.  The changes to your parameter\n> file have its own history (e.g. you upgraded your machine last week with\n> different hardware---the software didn't change but the parameter has to)\n> and the history is largely independent of the software that reads the\n> file.  In that sense, you should _never_ tie its history with the history\n> of your software.\n\nOk.\n\n> At the same time, the history of the parameter file is not completely\n> independent of that of the software.  You may have added a new feature  \n> and code to read a new setting in the parameter file today and older  \n> parameter file would not have a setting for that variable.\n\nExactly.\n\n> The method I used in my day-job project is something like this:\n>\n>  - Include a \"config.sample\" file and track it in the history of the\n>    source code; the values it gives to the variables are the same as the\n>    default ones.  This file mostly serves as a documentation.\n\nI actually did exactly that already (but forgot to mention it in my first  \nmail, sorry!). The problem with this \"one\" config.sample is, that I need  \nabout three really different config files because of really different  \nsensor setups on about five different computers. So when I push on a USB  \nstick, walk to another computer, then pull and try to run the software -  \nbummer, forgot to copy the current config file - walk back, copy ...\nBut maybe the solution below solves this too. Or I write a little script  \ndoing the push and parameter file copy.\n\n>  - Design your parameter mechanism in a way that:\n>\n>    - a parameter file can include another;\n\nCan't do that yet.\n\n>    - an unrecognized parameter setting is ignored;\n\nCan do.\n\n>    - a variable definition that happens later overrides an earlier\n>      definition.\n\nCan do.\n\n>  - Have a real parameter file that is _not_ tracked in the history of the\n>    source code.  Have it include parameter.sample and then define machine\n>    specific settings after that to override.\n\nGood idea.\n\n> It is only required for the \"real\" parameter file not to be tracked in  \n> the history of the source code---it is perfectly fine to track it in a\n> separate history.  The easiest way to do this would be to keep a  \n> parameter working tree next door, and \"ln -s ../params/config config\" in  \n> the working tree for the source code.  You could keep this symlink  \n> tracked in the\n> source history, but you do not have to.\n\nI see. So if I want parameter file history the only proper solution is to  \nhave a separate parameter file git repo, right?\n\n> You will try to read from ./config, which will include ./config.sample in\n> order to read the default, and then read the customized setting that\n> appear later in ./config.  Because of the symlink, you will actually be\n> reading from ../params/config next door.\n\nOk, will give it a try.\n\n-- \nRegards,\n\n\tTilo\n"},{"id":"140780","messageId":"7vmxwiuwr5.fsf@alter.siamese.dyndns.org","threadId":"23651","inReplyTo":"op.vb2ms4r8a8ed4e@dellschleppa","subject":"Re: How to manage parameter files and code separately using git?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-05-02T15:52:14Z","receivedAt":"2010-05-02T15:52:14Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Tilo Schwarz\" <tilo@tilo-schwarz.de> writes:\n\n> I see. So if I want parameter file history the only proper solution is to  \n> have a separate parameter file git repo, right?\n\nI wouldn't say it is \"the\" \"only\" proper solution, but under your design\nconstraint that dictates that the part that is left in \"config\" (which\nincludes the tracked-and-tied-to-the-software-version \"config.sample\")\nmust have a history that is independent from the software history, it is\none workable solution that would be the easiest.  I can imagine a more\nelaborate implementation that stores the history of that config file on a\nseparate branch in the same repository and manipulate that file with\ncustom scripts, and that also would be another workable solution.\n"}]}