Project organisation and structure

Hello, I'm new to git, and I hope this is the right mailinglist for what I'm going to ask; it was the only one I found on http://git.or.cz. Sorry if it's not or my question is remarkably stupid. I've read the git documentation, of course, but it wasn't always easy to wrap my head around all the concepts, so I might have missed an obvious solution to my problem.

Currently, I've got a tree of self-written C++ source code for which I don't use any SCM, and I'd like to start using git on it. However, I've run into issues with my directory layout I cannot resolve on my own. I have a collection of libraries and programs, which aren't coupled too much, but aren't really independent either; for example, library X depends on Y, program Z on lib A and so forth. Since all of these libs and programs are logically connected to each other in some way, I would like to have them all in one central repository. So I heard that I could create kind of a "meta-repository" for the whole project, and have submodules for each library or program, which sounds exactly like what I want. However, I've been told that my current directory layout will not work this way with git.

This is how my project root directory (say it's ~) looks like: ~/build - Build files which apply for the whole project, like doxygen build files. ~/include/$submodule - Each library/program has an own directory here, in which all header files go. ~/projects/$submodule - Files related to an individual project, like makefiles etc. ~/src/$submodule - Like the above two, but for source files. ~/test/$submodule - Again, a directory for each library/program, containing files for (unit)tests. ~/doc - Documentation files for the project as a whole. ~/doc/$submodule - Documentation files for individual libraries/programs.

These are the directories I would like git to track for me. There are some other dirs not listed here (e.g. for object files,) but I can easily add them to .gitignore.

So, the git structure as I imagine it would look like this: ~/.git contains the "meta-repository," in which all the submodules reside. Each submodule would have its .git directory in ~/projects/$submodule. Alternatively, ~/.git for the meta-repo and ~/.git-$subproject or something like this would be okay too.

However, I have been told I cannot go to ~/projects/$submodule and do a "git add ../../{include,src,test,doc}/$submodule" there. I could add symlinks in ~/projects/$submodule to each of these dirs and add that link, but this would not work with windows, I guess (would it work with git, anyway?)

So, how can I manage my code without having to restructure my tree?

I'd like to keep it that way, because it is convenient to have only one directory to pass to the compiler as an additional include dir, and can include my header files as "submodule/someheader.hpp" which is, in my opinion, helpful in organizing the code.

If there should be no way I can keep my directory setup the way it is with the current git, would a feature request be a good idea (and possibly implemented), or does my directory tree have a serious flaw I haven't stumbled across yet, which makes it unusable anyways?

Thank you in advance.

k wayne wrote:

This is the right place, and no you didn't miss anything obvious.

Right. That would break things rather horribly as you're trying to add files below the root of your repository.

It would work with git, and it will work on windows, sort of. Git knows that the symlink is a symlink, but it will check out a proper copy of the actual file in its place on windows. You just have to be careful when updating it, as tests and stuff before you've committed to git might break with one file being the old revision still.

Short answer; Lots of symlinks.

Longer answer: Are you really sure that's the best structure for your code? I usually find it absolutely simplest to bundle tests, buildrules and source all in one directory (possibly with tests and documentation in a subdir), but otherwise keep it all jumbled together. There's a very important reason for this, and it's that your .cpp files shouldn't have to know what some other buildsystem is doing with its header files, so you should always be able to #include "header_for_this_api.hpp" and be done with it.

If I were you, I'd do something like this: /: top-level buildstuff /doc: collection-wide documentation ("this suite of programs blahblah") /t: integration tests /lib/$library_subproject - holds library source/build/unit-tests/doc /src/$program_subproject - holds program source/build/unit-tests/doc

After all, you're mostly updating one library or program at a time, right? If you're not and all the API's are in wild flux, you should probably just keep it jumbled in a single git repository for the moment and split them later when things have calmed down a bit.

With the layout above, you just pass -I./lib as include dir and use
	#include <$library/header.hpp>
in your cpp files, which is presumably what you'd use if you'd installed
the headers in the normal location too.

A feature request to be able to track files outside the repository would require some major surgery to the most basic of git objects, so I assume such a feature wouldn't be too well received.

I'd call it unusable to have to hop to several different directories to add a single public function to a library, but perhaps that's just me?

Perhaps your programs contain mini-libraries that are shared at link-time between applications? My advice in that case is to turn them into libraries and make proper API's from them. That way the above directory structure will give you everything you want while maintaining each individual component separated in its own repository.

http://people.redhat.com/drepper/goodpractice.pdf holds a very nice and concise writeup on how to create API's. I follow to the letter in most of my library-like projects.

k wayne wrote:

Your directory structure is common and intuitive in the sense that thats what happens, you start coding then add stuff along the way. You don't know from the start what will the tree look like.

Git is well suited for your tree and will eat it with out solt

Do not use subprojects these are used with two separately developed projects that are dependent (one on the other not two ways). For example I have a GUI app that uses wxWidgets. Instead of instructing my users on where to get it and what exact version to use, with what version of my program, I mirror it in a submodule and this is all done automatically for me and my cloners.

What you need is topic branches. Not all branches need have all the same files. In fact they can have nothing in common. The only thing with git is that at a certain checkout in time only these files will be present on disk.

So lets say you have:

/include/lib4foobar /include/lib4bar /src/lib4foobar /src/lib4bar /src/foo /src/bar

Lets complicate it and say lib4bar is also dependent on lib4foobar then you have these branches: lib4foobar lib4bar foo bar

When you work on a library/app you take your foo-maintainer hat and push patches to your subtree branch. When it is time for integration, checkout master and merge all the branches in.

Simple as hell. And faster then you ever imagine.

Additionally you can have working branches that, lets say did-of-the-day, where you save work to 3 topics /src/bar /src/lib4bar /src/lib4foobar. Once you are happy. You can push them through the project branches, and delete the working branch, or you can merge that directly into master and when you pull master into the topic branches after a big merge, they will have it automatically.

This is all very similar to Linux. With the subject maintainers and all, just with fewer people.

PS: Don't forget to have a very first patch with nothing but
a common root file like GPLv2 or README. The first patch is
hard to work with.

Just my $0.017 Boaz