{"thread":{"id":"15567","subject":"Project organisation and structure","startedAt":"2008-09-17T13:49:19Z","lastAt":"2008-09-18T07:30:05Z","messageCount":3,"participants":["k wayne","Boaz Harrosh","Andreas Ericsson"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"90943","messageId":"bcf9fe2c0809170649x3f377c7erecc9f69a4a664d52@mail.gmail.com","threadId":"15567","inReplyTo":null,"subject":"Project organisation and structure","fromName":"k wayne","fromEmail":"pleexed@gmail.com","sentAt":"2008-09-17T13:49:19Z","receivedAt":"2008-09-17T13:49:19Z","isPatch":false,"sender":{"key":"pleexed@gmail.com","avatar":null},"body":"Hello,\nI'm new to git, and I hope this is the right mailinglist for what I'm\ngoing to ask; it was the only one I found on http://git.or.cz. Sorry\nif it's not or my question is remarkably stupid. I've read the git\ndocumentation, of course, but it wasn't always easy to wrap my head\naround all the concepts, so I might have missed an obvious solution to\nmy problem.\n\nCurrently, I've got a tree of self-written C++ source code for which I\ndon't use any SCM, and I'd like to start using git on it. However,\nI've run into issues with my directory layout I cannot resolve on my\nown.\nI have a collection of libraries and programs, which aren't coupled\ntoo much, but aren't really independent either; for example, library X\ndepends on Y, program Z on lib A and so forth. Since all of these libs\nand programs are logically connected to each other in some way, I\nwould like to have them all in one central repository.\nSo I heard that I could create kind of a \"meta-repository\" for the\nwhole project, and have submodules for each library or program, which\nsounds exactly like what I want. However, I've been told that my\ncurrent directory layout will not work this way with git.\n\nThis is how my project root directory (say it's ~) looks like:\n~/build - Build files which apply for the whole project, like doxygen\nbuild files.\n~/include/$submodule - Each library/program has an own directory here,\nin which all header files go.\n~/projects/$submodule - Files related to an individual project, like\nmakefiles etc.\n~/src/$submodule - Like the above two, but for source files.\n~/test/$submodule - Again, a directory for each library/program,\ncontaining files for (unit)tests.\n~/doc - Documentation files for the project as a whole.\n~/doc/$submodule - Documentation files for individual libraries/programs.\n\nThese are the directories I would like git to track for me. There are\nsome other dirs not listed here (e.g. for object files,) but I can\neasily add them to .gitignore.\n\nSo, the git structure as I imagine it would look like this: ~/.git\ncontains the \"meta-repository,\" in which all the submodules reside.\nEach submodule would have its .git directory in ~/projects/$submodule.\nAlternatively, ~/.git for the meta-repo and ~/.git-$subproject or\nsomething like this would be okay too.\n\nHowever, I have been told I cannot go to ~/projects/$submodule and do\na \"git add ../../{include,src,test,doc}/$submodule\" there. I could add\nsymlinks in ~/projects/$submodule to each of these dirs and add that\nlink, but this would not work with windows, I guess (would it work\nwith git, anyway?)\n\nSo, how can I manage my code without having to restructure my tree?\n\nI'd like to keep it that way, because it is convenient to have only\none directory to pass to the compiler as an additional include dir,\nand can include my header files as \"submodule/someheader.hpp\" which\nis, in my opinion, helpful in organizing the code.\n\nIf there should be no way I can keep my directory setup the way it is\nwith the current git, would a feature request be a good idea (and\npossibly implemented), or does my directory tree have a serious flaw I\nhaven't stumbled across yet, which makes it unusable anyways?\n\nThank you in advance.\n"},{"id":"91489","messageId":"48D137FB.70502@op5.se","threadId":"15567","inReplyTo":"bcf9fe2c0809170649x3f377c7erecc9f69a4a664d52@mail.gmail.com","subject":"Re: Project organisation and structure","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2008-09-17T17:01:47Z","receivedAt":"2008-09-17T17:01:47Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"k wayne wrote:\n> Hello,\n> I'm new to git, and I hope this is the right mailinglist for what I'm\n> going to ask; it was the only one I found on http://git.or.cz. Sorry\n> if it's not or my question is remarkably stupid. I've read the git\n> documentation, of course, but it wasn't always easy to wrap my head\n> around all the concepts, so I might have missed an obvious solution to\n> my problem.\n> \n\nThis is the right place, and no you didn't miss anything obvious.\n\n\n> I have a collection of libraries and programs, which aren't coupled\n> too much, but aren't really independent either; for example, library X\n> depends on Y, program Z on lib A and so forth. Since all of these libs\n> and programs are logically connected to each other in some way, I\n> would like to have them all in one central repository.\n> So I heard that I could create kind of a \"meta-repository\" for the\n> whole project, and have submodules for each library or program, which\n> sounds exactly like what I want. However, I've been told that my\n> current directory layout will not work this way with git.\n> \n> This is how my project root directory (say it's ~) looks like:\n> ~/build - Build files which apply for the whole project, like doxygen\n> build files.\n> ~/include/$submodule - Each library/program has an own directory here,\n> in which all header files go.\n> ~/projects/$submodule - Files related to an individual project, like\n> makefiles etc.\n> ~/src/$submodule - Like the above two, but for source files.\n> ~/test/$submodule - Again, a directory for each library/program,\n> containing files for (unit)tests.\n> ~/doc - Documentation files for the project as a whole.\n> ~/doc/$submodule - Documentation files for individual libraries/programs.\n> \n> These are the directories I would like git to track for me. There are\n> some other dirs not listed here (e.g. for object files,) but I can\n> easily add them to .gitignore.\n> \n> So, the git structure as I imagine it would look like this: ~/.git\n> contains the \"meta-repository,\" in which all the submodules reside.\n> Each submodule would have its .git directory in ~/projects/$submodule.\n> Alternatively, ~/.git for the meta-repo and ~/.git-$subproject or\n> something like this would be okay too.\n> \n> However, I have been told I cannot go to ~/projects/$submodule and do\n> a \"git add ../../{include,src,test,doc}/$submodule\" there.\n\nRight. That would break things rather horribly as you're trying to add\nfiles below the root of your repository.\n\n> I could add\n> symlinks in ~/projects/$submodule to each of these dirs and add that\n> link, but this would not work with windows, I guess (would it work\n> with git, anyway?)\n> \n\nIt would work with git, and it will work on windows, sort of. Git knows\nthat the symlink is a symlink, but it will check out a proper copy of\nthe actual file in its place on windows. You just have to be careful\nwhen updating it, as tests and stuff before you've committed to git\nmight break with one file being the old revision still.\n\n> So, how can I manage my code without having to restructure my tree?\n> \n\nShort answer; Lots of symlinks.\n\nLonger answer:\nAre you really sure that's the best structure for your code? I usually\nfind it absolutely simplest to bundle tests, buildrules and source\nall in one directory (possibly with tests and documentation in a subdir),\nbut otherwise keep it all jumbled together. There's a very important\nreason for this, and it's that your .cpp files shouldn't have to know\nwhat some other buildsystem is doing with its header files, so you\nshould always be able to #include \"header_for_this_api.hpp\" and be\ndone with it.\n\nIf I were you, I'd do something like this:\n/: top-level buildstuff\n/doc: collection-wide documentation (\"this suite of programs blahblah\")\n/t: integration tests\n/lib/$library_subproject - holds library source/build/unit-tests/doc\n/src/$program_subproject - holds program source/build/unit-tests/doc\n\nAfter all, you're mostly updating one library or program at a time, right?\nIf you're not and all the API's are in wild flux, you should probably\njust keep it jumbled in a single git repository for the moment and split\nthem later when things have calmed down a bit.\n\n> I'd like to keep it that way, because it is convenient to have only\n> one directory to pass to the compiler as an additional include dir,\n> and can include my header files as \"submodule/someheader.hpp\" which\n> is, in my opinion, helpful in organizing the code.\n> \n\nWith the layout above, you just pass -I./lib as include dir and use\n\t#include <$library/header.hpp>\nin your cpp files, which is presumably what you'd use if you'd installed\nthe headers in the normal location too.\n\n> If there should be no way I can keep my directory setup the way it is\n> with the current git, would a feature request be a good idea (and\n> possibly implemented),\n\nA feature request to be able to track files outside the repository would\nrequire some major surgery to the most basic of git objects, so I assume\nsuch a feature wouldn't be too well received.\n\n> or does my directory tree have a serious flaw I\n> haven't stumbled across yet, which makes it unusable anyways?\n> \n\nI'd call it unusable to have to hop to several different directories to\nadd a single public function to a library, but perhaps that's just me?\n\nPerhaps your programs contain mini-libraries that are shared at link-time\nbetween applications? My advice in that case is to turn them into libraries\nand make proper API's from them. That way the above directory structure\nwill give you everything you want while maintaining each individual\ncomponent separated in its own repository.\n\nhttp://people.redhat.com/drepper/goodpractice.pdf holds a very nice and\nconcise writeup on how to create API's. I follow to the letter in most\nof my library-like projects.\n\n-- \nAndreas Ericsson                   andreas.ericsson@op5.se\nOP5 AB                             www.op5.se\nTel: +46 8-230225                  Fax: +46 8-230231\n"},{"id":"91007","messageId":"48D2037D.5040301@panasas.com","threadId":"15567","inReplyTo":"bcf9fe2c0809170649x3f377c7erecc9f69a4a664d52@mail.gmail.com","subject":"Re: Project organisation and structure","fromName":"Boaz Harrosh","fromEmail":"bharrosh@panasas.com","sentAt":"2008-09-18T07:30:05Z","receivedAt":"2008-09-18T07:30:05Z","isPatch":false,"sender":{"key":"bharrosh@panasas.com","avatar":"https://gravatar.com/avatar/347e426f8ca0409f6891ceeefd4c3c2d8769608382323057efeaa362b4c3dbf3?d=mp&s=160"},"body":"k wayne wrote:\n> Hello,\n> I'm new to git, and I hope this is the right mailinglist for what I'm\n> going to ask; it was the only one I found on http://git.or.cz. Sorry\n> if it's not or my question is remarkably stupid. I've read the git\n> documentation, of course, but it wasn't always easy to wrap my head\n> around all the concepts, so I might have missed an obvious solution to\n> my problem.\n> \n> Currently, I've got a tree of self-written C++ source code for which I\n> don't use any SCM, and I'd like to start using git on it. However,\n> I've run into issues with my directory layout I cannot resolve on my\n> own.\n> I have a collection of libraries and programs, which aren't coupled\n> too much, but aren't really independent either; for example, library X\n> depends on Y, program Z on lib A and so forth. Since all of these libs\n> and programs are logically connected to each other in some way, I\n> would like to have them all in one central repository.\n> So I heard that I could create kind of a \"meta-repository\" for the\n> whole project, and have submodules for each library or program, which\n> sounds exactly like what I want. However, I've been told that my\n> current directory layout will not work this way with git.\n> \n> This is how my project root directory (say it's ~) looks like:\n> ~/build - Build files which apply for the whole project, like doxygen\n> build files.\n> ~/include/$submodule - Each library/program has an own directory here,\n> in which all header files go.\n> ~/projects/$submodule - Files related to an individual project, like\n> makefiles etc.\n> ~/src/$submodule - Like the above two, but for source files.\n> ~/test/$submodule - Again, a directory for each library/program,\n> containing files for (unit)tests.\n> ~/doc - Documentation files for the project as a whole.\n> ~/doc/$submodule - Documentation files for individual libraries/programs.\n> \n> These are the directories I would like git to track for me. There are\n> some other dirs not listed here (e.g. for object files,) but I can\n> easily add them to .gitignore.\n> \n> So, the git structure as I imagine it would look like this: ~/.git\n> contains the \"meta-repository,\" in which all the submodules reside.\n> Each submodule would have its .git directory in ~/projects/$submodule.\n> Alternatively, ~/.git for the meta-repo and ~/.git-$subproject or\n> something like this would be okay too.\n> \n> However, I have been told I cannot go to ~/projects/$submodule and do\n> a \"git add ../../{include,src,test,doc}/$submodule\" there. I could add\n> symlinks in ~/projects/$submodule to each of these dirs and add that\n> link, but this would not work with windows, I guess (would it work\n> with git, anyway?)\n> \n> So, how can I manage my code without having to restructure my tree?\n> \n> I'd like to keep it that way, because it is convenient to have only\n> one directory to pass to the compiler as an additional include dir,\n> and can include my header files as \"submodule/someheader.hpp\" which\n> is, in my opinion, helpful in organizing the code.\n> \n> If there should be no way I can keep my directory setup the way it is\n> with the current git, would a feature request be a good idea (and\n> possibly implemented), or does my directory tree have a serious flaw I\n> haven't stumbled across yet, which makes it unusable anyways?\n> \n> Thank you in advance.\n> --\n\nYour directory structure is common and intuitive in the sense that thats\nwhat happens, you start coding then add stuff along the way. You don't\nknow from the start what will the tree look like.\n\nGit is well suited for your tree and will eat it with out solt\n\nDo not use subprojects these are used with two separately developed\nprojects that are dependent (one on the other not two ways).\nFor example I have a GUI app that uses wxWidgets. Instead of instructing\nmy users on where to get it and what exact version to use, with what \nversion of my program, I mirror it in a submodule and this is all done\nautomatically for me and my cloners.\n\nWhat you need is topic branches. Not all branches need have all the\nsame files. In fact they can have nothing in common. The only thing\nwith git is that at a certain checkout in time only these files will\nbe present on disk.\n\nSo lets say you have:\n\n/include/lib4foobar\n/include/lib4bar\n/src/lib4foobar\n/src/lib4bar\n/src/foo\n/src/bar\n\nLets complicate it and say lib4bar is also dependent on lib4foobar\nthen you have these branches: lib4foobar lib4bar foo bar\n\nWhen you work on a library/app you take your foo-maintainer hat and\npush patches to your subtree branch. When it is time for integration,\ncheckout master and merge all the branches in.\n\nSimple as hell. And faster then you ever imagine.\n\nAdditionally you can have working branches that, lets say\ndid-of-the-day, where you save work to 3 topics /src/bar\n/src/lib4bar /src/lib4foobar. Once you are happy. You can\npush them through the project branches, and delete the\nworking branch, or you can merge that directly into master\nand when you pull master into the topic branches after a big\nmerge, they will have it automatically.\n\nThis is all very similar to Linux. With the subject maintainers\nand all, just with fewer people.\n\nPS: Don't forget to have a very first patch with nothing but\na common root file like GPLv2 or README. The first patch is\nhard to work with.\n\nJust my $0.017\nBoaz\n"}]}