{"thread":{"id":"39923","subject":"How to organize multiple small reusable components with Git?","startedAt":"2015-07-23T11:20:25Z","lastAt":"2015-07-25T18:31:50Z","messageCount":3,"participants":["Bing Tian","Adam Kurkiewicz"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"266654","messageId":"CAMDZmAEf4afqhR4WT4e-dXjY5OWUSoE_WVGj1ztW-pA=ym9e3A@mail.gmail.com","threadId":"39923","inReplyTo":null,"subject":"How to organize multiple small reusable components with Git?","fromName":"Bing Tian","fromEmail":"tianbing1212@gmail.com","sentAt":"2015-07-23T11:20:25Z","receivedAt":"2015-07-23T11:20:25Z","isPatch":false,"sender":{"key":"tianbing1212@gmail.com","avatar":null},"body":"I am using git to manage some circuit components.\nEach component is small and I plan to create a \"component\" project in\nGit to hold all the small components.\nEach component may have several released version, such as\nComonent1_V1, Component1_V2, Comonent2_V1, Component2_V2.\nAnd for future reuse, I may use Component1_V1+Component2_V2, or\nComponent1_V2+Component2_V1 in my local directory.\nSo, I plan to create several directories, each for one version of one\ncomponent. I looks like followings:\nComponents\n |------Component1\n | |------------------Version1(Dir)\n | |------------------Version2(Dir)\n |-------Component2\n |------------------Version1(Dir)\n |------------------Version2(Dir)\n\nI want to know, is the above a suitable way in Git? Are there some\nrisks for my project and Git?\nAny suggestions to cover this case in Git?\n"},{"id":"266655","messageId":"CAA2SmNdD1W6VMe8JnfKpEdsy=WdbKja0F8i1yQ7gKUYLjemBPg@mail.gmail.com","threadId":"39923","inReplyTo":"CAMDZmAEf4afqhR4WT4e-dXjY5OWUSoE_WVGj1ztW-pA=ym9e3A@mail.gmail.com","subject":"Re: How to organize multiple small reusable components with Git?","fromName":"Adam Kurkiewicz","fromEmail":"popantrop@gmail.com","sentAt":"2015-07-23T13:05:39Z","receivedAt":"2015-07-23T13:05:39Z","isPatch":false,"sender":{"key":"popantrop@gmail.com","avatar":null},"body":"Hi Bing,\nYou seem to be describing a very interesting problem, similar to what\nI often run into in my day job. In my day job I'm composing multiple\njava microservices into a single application. We have about 7\nmicroservices, but this number will probably grow to more than 10\nbefore the application is feature-complete. Because microservices are\nmutually independent and unaware of each other (however, they _do_\nshare knowledge of message formats exchanged on the message queue), we\nhave a very acute problem of composing entire system together. The\nproblem is two-fold:\n\n1) how do you know which set of the versions of components gives you a\nworking system?\n2) how do you keep track of a record of [the sets of the versions of\ncomponents] which give you a working system?\n\nPart 1) is much more difficult than part 2). Although part 1) lies\noutside of the scope of version control I'd like to pay your attention\nto it nonetheless. In principle with 10 versions of 10 components you\nhave 10 billion possibly valid configurations of the system. In such\ncircumstance it is very helpful to use such good SE practices like\ndepcrecation warnings, semantic versioning and clean distinction\nbetween interface and its implementation. You can also do it the linux\nkernel way by simply never introducing backwards-incompatible changes.\nIf you want to read up about semantic versioning, here's the link to\nthe standard:\nhttp://semver.org/\n\nPart 2) is really what git can help you with:\n\nOne way to deal with this is to manage each component in a separate\ngit repository (thus version each component separately) and have a\nthin parent project, which imports all other projects as submodules.\nThe way submodules work is through storing each submodule as a\nseparate directory. You can go to each of the submodule directories,\ncheckout a particular commit which you want to be marked as a part of\nyour system and mark each component with usual command \"git add\n\". That obviously assumes that you've already solved\nPart 1) and you know what versions of components give you a working\nsystem. You can repeat this process for each submodule and once you're\nhappy commit the result in the parent project and push to the remote.\nYou can also do usual git things such as tags for example.\n\nI'm not sure how much I like this solution though: having multiple\nrepositories seems to be an overkill for most cases. Behaviour of\nsubmodules has been startling on a few occasions (for example `git\nsubmodule update` can actually take you back in time rather than\nforward). I've also experienced several peculiar and difficult to\npinpoint issues on windows. Although I haven't used it much, git\nsubtrees seems to be much nicer alternative. You can read about them\nhere:\nhttp://blogs.atlassian.com/2013/05/alternatives-to-git-submodule-git-subtree/\n\nAs to your original idea about having multiple separate directories,\nI'm not sure I like it at all. By splitting the versions of your\ncomponents into separate directories you loose a lot of benefits of\nversion control. With multiple directories you won't be able to apply\nhotfixes to multiple versions of the component for example without\nrewriting the logic of cherry-pick.\n\nCheers,\nAdam\n\nOn 23 July 2015 at 12:20, Bing Tian wrote:\n> I am using git to manage some circuit components.\n> Each component is small and I plan to create a \"component\" project in\n> Git to hold all the small components.\n> Each component may have several released version, such as\n> Comonent1_V1, Component1_V2, Comonent2_V1, Component2_V2.\n> And for future reuse, I may use Component1_V1+Component2_V2, or\n> Component1_V2+Component2_V1 in my local directory.\n> So, I plan to create several directories, each for one version of one\n> component. I looks like followings:\n> Components\n> |------Component1\n> | |------------------Version1(Dir)\n> | |------------------Version2(Dir)\n> |-------Component2\n> |------------------Version1(Dir)\n> |------------------Version2(Dir)\n>\n> I want to know, is the above a suitable way in Git? Are there some\n> risks for my project and Git?\n> Any suggestions to cover this case in Git?\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at http://vger.kernel.org/majordomo-info.html\n"},{"id":"266772","messageId":"CAA2SmNdXUdcTrLLg9iMMW6+N6h_XgA02w6XWMvncWYp=pdX61A@mail.gmail.com","threadId":"39923","inReplyTo":"CAMDZmAGBdpjnjJtNH9Nw9E_pATtgnOY_tq3QbQqUas-BM=ss=g@mail.gmail.com","subject":"Re: How to organize multiple small reusable components with Git?","fromName":"Adam Kurkiewicz","fromEmail":"popantrop@gmail.com","sentAt":"2015-07-25T18:31:50Z","receivedAt":"2015-07-25T18:31:50Z","isPatch":false,"sender":{"key":"popantrop@gmail.com","avatar":null},"body":"Hi Bing,\nI don't think the approach you're outlining in your most recent e-mail\nis going to work particularly well for you. See my replies in-line.\n\nI have devised a solution that should work much better for you, it's a\ntiny bash script and a set of naming conventions for repository\nbranches. I've described it in detail and implemented a toy example on\ngithub:\n\nhttps://github.com/picrin/git-components/blob/master/README.md\n\nFYI, when you click reply button, you have to click reply all or\notherwise your e-mail doesn't go to the mailing list but just to me\npersonally.\n\nInline responses follow:\n\n> Thanks for your kindly reply, which helps a lot.\n> I plans a neutral approach for my case now.\n> (1) create a \"Components\" git repo to hold all kinds of components,\n> each component in a directory.\n> As for each component, use regular git way to manage the version.\n> Accordingly there's not Version1 (Dir) or Version2(Dir), to follow Git\n> practice.\n\nThis way you can't version each component independently -- you have\none global history, which means a lot of noise per component.\n\n> (2) create another publication (release) server for all versions of\n> all components. (the server is not managed in git)\n> The publication server is read only, and every one can get\n> component(s) they need for further use.\n\nIt's not clear to me how your publication server would be able to pick\nup from your messy history the correct versions of multiple\ncomponents. Also, some of your clients might want to sit on an older\n(or more stable, or otherwise customised) version of a given\ncomponent, or want a narrower set of components, so\none-history-for-all clearly doesn't work for you.\n\nBesides, for publications git has already got something built in, read\nup about git-archive.\n\nAdam\n\n>\n>\n>\n> 2015-07-23 20:57 GMT+08:00 Adam Kurkiewicz :\n>> Hi Bing,\n>> You seem to be describing a very interesting problem, similar to what I\n>> often run into in my day job. In my day job I'm composing multiple java\n>> microservices into a single application. We have about 7 microservices, but\n>> this number will probably grow to more than 10 before the application is\n>> feature-complete. Because microservices are mutually independent and unaware\n>> of each other (however, they _do_ share knowledge of message formats\n>> exchanged on the message queue), we have a very acute problem of composing\n>> entire system together. The problem is two-fold:\n>>\n>> 1) how do you know which set of the versions of components gives you a\n>> working system?\n>> 2) how do you keep track of a record of [the sets of the versions of\n>> components] which give you a working system?\n>>\n>> Part 1) is much more difficult than part 2). Although part 1) lies outside\n>> of the scope of version control I'd like to pay your attention to it\n>> nonetheless. In principle with 10 versions of 10 components you have 10\n>> billion possibly valid configurations of the system. In such circumstance it\n>> is very helpful to use such good SE practices like depcrecation warnings,\n>> semantic versioning and clean distinction between interface and its\n>> implementation. You can also do it the linux kernel way by simply never\n>> introducing backwards-incompatible changes. If you want to read up about\n>> semantic versioning, here's the link to the standard:\n>> http://semver.org/\n>>\n>> Part 2) is really what git can help you with:\n>>\n>> One way to deal with this is to manage each component in a separate git\n>> repository (thus version each component separately) and have a thin parent\n>> project, which imports all other projects as submodules. The way submodules\n>> work is through storing each submodule as a separate directory. You can go\n>> to each of the submodule directories, checkout a particular commit which you\n>> want to be marked as a part of your system and mark each component with\n>> usual command \"git add \". That obviously assumes that you've\n>> already solved Part 1) and you know what versions of components give you a\n>> working system. You can repeat this process for each submodule and once\n>> you're happy commit the result in the parent project and push to the remote.\n>> You can also do usual git things such as tags for example.\n>>\n>> I'm not sure how much I like this solution though: having multiple\n>> repositories seems to be an overkill for most cases. Behaviour of submodules\n>> has been startling on a few occasions (for example `git submodule update`\n>> can actually take you to back in time rather than forward). I've also\n>> experienced several peculiar and difficult to pinpoint issues on windows.\n>> Although I haven't used it much, git subtrees seems to be much nicer\n>> alternative. You can read about them here:\n>> http://blogs.atlassian.com/2013/05/alternatives-to-git-submodule-git-subtree/\n>>\n>> As to your original idea about having multiple separate directories, I'm not\n>> sure I like it at all. By splitting the versions of your components into\n>> separate directories you loose a lot of benefits of version control. With\n>> multiple directories you won't be able to apply hotfixes to multiple\n>> versions of the component for example without rewriting the logic of\n>> cherry-pick.\n>>\n>> Cheers,\n>> Adam\n>>\n>>\n>> 2015-07-23 12:20 GMT+01:00 Bing Tian :\n>>>\n>>> I am using git to manage some circuit components.\n>>> Each component is small and I plan to create a \"component\" project in\n>>> Git to hold all the small components.\n>>> Each component may have several released version, such as\n>>> Comonent1_V1, Component1_V2, Comonent2_V1, Component2_V2.\n>>> And for future reuse, I may use Component1_V1+Component2_V2, or\n>>> Component1_V2+Component2_V1 in my local directory.\n>>> So, I plan to create several directories, each for one version of one\n>>> component. I looks like followings:\n>>> Components\n>>> |------Component1\n>>> | |------------------Version1(Dir)\n>>> | |------------------Version2(Dir)\n>>> |-------Component2\n>>> |------------------Version1(Dir)\n>>> |------------------Version2(Dir)\n>>>\n>>> I want to know, is the above a suitable way in Git? Are there some\n>>> risks for my project and Git?\n>>> Any suggestions to cover this case in Git?\n>>> --\n>>> To unsubscribe from this list: send the line \"unsubscribe git\" in\n>>> the body of a message to majordomo@vger.kernel.org\n>>> More majordomo info at http://vger.kernel.org/majordomo-info.html\n>>\n>>\n"}]}