How to manage parameter files and code separately using git?

Hi all,

maybe someone has an idea how to do what I'd like to do using git:

I use git for "normal" c coding, i.e. having branches like master, test, foo, etc. Built are executables (Linux), which need parameter text files to work. What I did up to now was to check in the parameter files in the same way I check in the code: when I change a parameter file, I do a commit on it, this way I always have a history of my parameter files.

This has one drawback: If I check out an older version, I also get the old parameter file, which is not what I want, because the parameters are determined by hardware settings. I.e., I would like to checkout an old commit, but still have the last version of the parameter file.

I tried to solve the problem by

1. creating a branch 'parameters', which contains only the parameter files
2. ignore the parameters files with extension *.m in .gitignore in the  
code branches (master etc.)

This seemed to work at first, but when I switch back from the parameters branch to master, all *.m files are removed, although .gitignore of master contains '*.m'.

So apparently git removes during branch switch first the tracked files and than populates the working dir with the new files of master.

Now my questions are

1. Is there a way to work around it?
2. Is there a maybe totally different way to solve my initial problem  
(separate code history from parameter file history) without using two git  
repos?

Thanks a lot!

     Tilo

"Tilo Schwarz" <tilo@tilo-schwarz.de> writes:

This design constraint makes the issue not a version control problem but more of a software engineering problem. The changes to your parameter file have its own history (e.g. you upgraded your machine last week with different hardware---the software didn't change but the parameter has to) and the history is largely independent of the software that reads the file. In that sense, you should _never_ tie its history with the history of your software.

At the same time, the history of the parameter file is not completely independent of that of the software. You may have added a new feature and code to read a new setting in the parameter file today and older parameter file would not have a setting for that variable.

The method I used in my day-job project is something like this:

 - Include a "config.sample" file and track it in the history of the
   source code; the values it gives to the variables are the same as the
   default ones.  This file mostly serves as a documentation.

 - Design your parameter mechanism in a way that:

   - a parameter file can include another;

   - an unrecognized parameter setting is ignored;

   - a variable definition that happens later overrides an earlier
     definition.

 - Have a real parameter file that is _not_ tracked in the history of the
   source code.  Have it include parameter.sample and then define machine
   specific settings after that to override.

It is only required for the "real" parameter file not to be tracked in the history of the source code---it is perfectly fine to track it in a separate history. The easiest way to do this would be to keep a parameter working tree next door, and "ln -s ../params/config config" in the working tree for the source code. You could keep this symlink tracked in the source history, but you do not have to.

You will try to read from ./config, which will include ./config.sample in order to read the default, and then read the customized setting that appear later in ./config. Because of the symlink, you will actually be reading from ../params/config next door.

On Sat, 01 May 2010 19:18:55 +0200, Junio C Hamano <gitster@pobox.com> wrote:

Thank you for the detailed explanation!

Ok.

Exactly.

I actually did exactly that already (but forgot to mention it in my first mail, sorry!). The problem with this "one" config.sample is, that I need about three really different config files because of really different sensor setups on about five different computers. So when I push on a USB stick, walk to another computer, then pull and try to run the software - bummer, forgot to copy the current config file - walk back, copy ... But maybe the solution below solves this too. Or I write a little script doing the push and parameter file copy.

Can't do that yet.

Can do.

Can do.

Good idea.

I see. So if I want parameter file history the only proper solution is to have a separate parameter file git repo, right?

Ok, will give it a try.

"Tilo Schwarz" <tilo@tilo-schwarz.de> writes:

I wouldn't say it is "the" "only" proper solution, but under your design constraint that dictates that the part that is left in "config" (which includes the tracked-and-tied-to-the-software-version "config.sample") must have a history that is independent from the software history, it is one workable solution that would be the easiest. I can imagine a more elaborate implementation that stores the history of that config file on a separate branch in the same repository and manipulate that file with custom scripts, and that also would be another workable solution.