{"thread":{"id":"39074","subject":"How to combine git repos with similar code and keep all branches and tags?","startedAt":"2015-04-14T16:44:13Z","lastAt":"2015-04-14T17:31:38Z","messageCount":5,"participants":["Jose de Leon","Matthieu Moy","Junio C Hamano","Max Horn"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"259356","messageId":"BBAA9EAE77388248BCAB045DA762C5A83EFE242F@EXBE01-ENSIM.ms.ensim.com","threadId":"39074","inReplyTo":null,"subject":"How to combine git repos with similar code and keep all branches and tags?","fromName":"Jose de Leon","fromEmail":"jdeleon@ensim.com","sentAt":"2015-04-14T16:44:13Z","receivedAt":"2015-04-14T16:44:13Z","isPatch":false,"sender":{"key":"jdeleon@ensim.com","avatar":null},"body":"Hi All,\n\n\nI've got an interesting problem and the possible solutions I've found from searching google don't seem to work for us. In a nutshell, I need to combine multiple git repositories into single repository and preserve all history, branches and tags from each repository. \n\nDoes anybody have a solution for this, how do this kind of thing, is it even possible?\n\nFor some unknown reason to me, our developers started a git project, called Ver1, this was the first version. Then sometime later, they created a new git repository called Ver2, the initial commit for Ver2 was essentially a copy of the code in Ver1 from the master. They didn't clone it, they just copied the code at the latest point. This wasn't done by forking. Then they repeated this for Ver3 and Ver4, etc. On top of all that, they have been adding new code to Ver1, Ver2, etc. that has caused each to divert from the other and each have their own branch and tag sets. They have similar directory structure and similar file names, but there are some slight differences in structure.\n\nWell, this has become unmanageable and now we want to combine them into one single git repository. \n\nLogically, these are the same project but at different versions, basically, Ver1 and Ver2, etc, should be simply branches at different times if these were combined into a single repository.\n"},{"id":"259358","messageId":"vpqr3rm8wn6.fsf@anie.imag.fr","threadId":"39074","inReplyTo":"BBAA9EAE77388248BCAB045DA762C5A83EFE242F@EXBE01-ENSIM.ms.ensim.com","subject":"Re: How to combine git repos with similar code and keep all branches and tags?","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@grenoble-inp.fr","sentAt":"2015-04-14T17:09:17Z","receivedAt":"2015-04-14T17:09:17Z","isPatch":false,"sender":{"key":"matthieu.moy@grenoble-inp.fr","avatar":"https://gravatar.com/avatar/72c8a2705971a25dfaff23cece15130d405685845d911aedd5667ace277f3fc5?d=mp&s=160"},"body":"Jose de Leon writes:\n\n> For some unknown reason to me, our developers started a git project,\n> called Ver1, this was the first version. Then sometime later, they\n> created a new git repository called Ver2, the initial commit for Ver2\n> was essentially a copy of the code in Ver1 from the master. They\n> didn't clone it, they just copied the code at the latest point.\n\nThis is why \"graft points\" were created, and then superseeded by \"git\nreplace\".\n\nSee http://git-scm.com/blog/2010/03/17/replace.html and\nhttp://git-scm.com/docs/git-replace\n\n-- \nMatthieu Moy\nhttp://www-verimag.imag.fr/~moy/\n"},{"id":"259363","messageId":"13B30361-1BCD-4481-A33B-E5C65C8169F9@quendi.de","threadId":"39074","inReplyTo":"BBAA9EAE77388248BCAB045DA762C5A83EFE242F@EXBE01-ENSIM.ms.ensim.com","subject":"Re: How to combine git repos with similar code and keep all branches and tags?","fromName":"Max Horn","fromEmail":"max@quendi.de","sentAt":"2015-04-14T17:14:36Z","receivedAt":"2015-04-14T17:14:36Z","isPatch":false,"sender":{"key":"max@quendi.de","avatar":"https://avatars.githubusercontent.com/u/241512?v=4"},"body":"Hi Jose,\n\nOn 14.04.2015, at 18:44, Jose de Leon wrote:\n\n> Hi All,\n> \n> \n> I've got an interesting problem and the possible solutions I've found from searching google don't seem to work for us. In a nutshell, I need to combine multiple git repositories into single repository and preserve all history, branches and tags from each repository. \n> \n> Does anybody have a solution for this, how do this kind of thing, is it even possible?\n> \n> For some unknown reason to me, our developers started a git project, called Ver1, this was the first version. Then sometime later, they created a new git repository called Ver2, the initial commit for Ver2 was essentially a copy of the code in Ver1 from the master. They didn't clone it, they just copied the code at the latest point. This wasn't done by forking. Then they repeated this for Ver3 and Ver4, etc. On top of all that, they have been adding new code to Ver1, Ver2, etc. that has caused each to divert from the other and each have their own branch and tag sets. They have similar directory structure and similar file names, but there are some slight differences in structure.\n> \n> Well, this has become unmanageable and now we want to combine them into one single git repository. \n> \n> Logically, these are the same project but at different versions, basically, Ver1 and Ver2, etc, should be simply branches at different times if these were combined into a single repository.\n\nHere is one possible way to go about this using grafts (I used something\nsimilar in the past);\n\n1) Get all the data into a single git repository.\n\n Since everything is already in a git repositories, you could e.g. create a\n clone of Ver1; then add remotes for Ver2 ... VerN, and fetch them all,\n along with tags. If there are conflicts between branch or tag names, deal\n with them at this point.\n\n2) Create graft points to tie the history together.\n\n Identify the commit in Ver1 at which Ver2 branched off. Then, graft that\n as parent for the initial commits of Ver2. See here for some basic\n instructions or feel\n free to ask for details (or Google, or... :)\n\n Repeat vor Ver2+Ver3, Ver3+Ver4, etc.\n\n3) Finally, you can get rid of the graft points, and turn everything into a\n\"proper\" history, by running \"git filter-branch\". Something like\n\n git filter-branch -- --all\n\nought to do it, but I might be forgetting something (I am sure somebody will\ncorrect me soon in that case, though ;-). Best to have a look at\n for yourself, though.\n\n\nThis all is under the assumption that you want to stay as close to how\nthings really were (usually a good idea). But sometimes it may be desirable\nto make further adjustments. E.g. you may wish to adjust committer names,\nrearrange some stuff (though usually git is quite good at doing the right\nthing automatically, etc. How to do that of course depends on what exactly\nyou want to do, but in many cases, filter-branch is your friend.\n\nHope that helps!\n\nMax\n"},{"id":"259361","messageId":"xmqqfv8239n1.fsf@gitster.dls.corp.google.com","threadId":"39074","inReplyTo":"vpqr3rm8wn6.fsf@anie.imag.fr","subject":"Re: How to combine git repos with similar code and keep all branches and tags?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-04-14T17:25:06Z","receivedAt":"2015-04-14T17:25:06Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Matthieu Moy writes:\n\n> Jose de Leon writes:\n>\n>> For some unknown reason to me, our developers started a git project,\n>> called Ver1, this was the first version. Then sometime later, they\n>> created a new git repository called Ver2, the initial commit for Ver2\n>> was essentially a copy of the code in Ver1 from the master. They\n>> didn't clone it, they just copied the code at the latest point.\n>\n> This is why \"graft points\" were created, and then superseeded by \"git\n> replace\".\n>\n> See http://git-scm.com/blog/2010/03/17/replace.html and\n> http://git-scm.com/docs/git-replace\n\nAfter setting up either grafts or replaces, I'd strongly recommend\nrunning filter-branch or bfg to rewrite the history of the combined\nresult, and have the developers use that rewritten history _after_\nremoving the grafts (or replaces).\n\nAnd if you are going to go that route, then graft is sufficient and\na lot more light-weight. The only advantage replace has over grafts\nis that they can be transferred using \"git push\" and \"git fetch\"\n(while grafts can be transferred with some file transfer mechanism\noutside Git) but if you are rewriting the history, that advantage is\nlost.\n"},{"id":"259364","messageId":"BBAA9EAE77388248BCAB045DA762C5A83EFE251A@EXBE01-ENSIM.ms.ensim.com","threadId":"39074","inReplyTo":"13B30361-1BCD-4481-A33B-E5C65C8169F9@quendi.de","subject":"RE: How to combine git repos with similar code and keep all branches and tags?","fromName":"Jose de Leon","fromEmail":"jdeleon@ensim.com","sentAt":"2015-04-14T17:31:38Z","receivedAt":"2015-04-14T17:31:38Z","isPatch":false,"sender":{"key":"jdeleon@ensim.com","avatar":null},"body":"Thank you!\n\n-----Original Message-----\nFrom: Max Horn [mailto:max@quendi.de] \nSent: Tuesday, April 14, 2015 10:15 AM\nTo: Jose de Leon\nCc: git@vger.kernel.org\nSubject: Re: How to combine git repos with similar code and keep all branches and tags?\n\nHi Jose,\n\nOn 14.04.2015, at 18:44, Jose de Leon wrote:\n\n> Hi All,\n> \n> \n> I've got an interesting problem and the possible solutions I've found from searching google don't seem to work for us. In a nutshell, I need to combine multiple git repositories into single repository and preserve all history, branches and tags from each repository. \n> \n> Does anybody have a solution for this, how do this kind of thing, is it even possible?\n> \n> For some unknown reason to me, our developers started a git project, called Ver1, this was the first version. Then sometime later, they created a new git repository called Ver2, the initial commit for Ver2 was essentially a copy of the code in Ver1 from the master. They didn't clone it, they just copied the code at the latest point. This wasn't done by forking. Then they repeated this for Ver3 and Ver4, etc. On top of all that, they have been adding new code to Ver1, Ver2, etc. that has caused each to divert from the other and each have their own branch and tag sets. They have similar directory structure and similar file names, but there are some slight differences in structure.\n> \n> Well, this has become unmanageable and now we want to combine them into one single git repository. \n> \n> Logically, these are the same project but at different versions, basically, Ver1 and Ver2, etc, should be simply branches at different times if these were combined into a single repository.\n\nHere is one possible way to go about this using grafts (I used something similar in the past);\n\n1) Get all the data into a single git repository.\n\n Since everything is already in a git repositories, you could e.g. create a\n clone of Ver1; then add remotes for Ver2 ... VerN, and fetch them all,\n along with tags. If there are conflicts between branch or tag names, deal\n with them at this point.\n\n2) Create graft points to tie the history together.\n\n Identify the commit in Ver1 at which Ver2 branched off. Then, graft that\n as parent for the initial commits of Ver2. See here for some basic\n instructions or feel\n free to ask for details (or Google, or... :)\n\n Repeat vor Ver2+Ver3, Ver3+Ver4, etc.\n\n3) Finally, you can get rid of the graft points, and turn everything into a \"proper\" history, by running \"git filter-branch\". Something like\n\n git filter-branch -- --all\n\nought to do it, but I might be forgetting something (I am sure somebody will correct me soon in that case, though ;-). Best to have a look at for yourself, though.\n\n\nThis all is under the assumption that you want to stay as close to how things really were (usually a good idea). But sometimes it may be desirable to make further adjustments. E.g. you may wish to adjust committer names, rearrange some stuff (though usually git is quite good at doing the right thing automatically, etc. How to do that of course depends on what exactly you want to do, but in many cases, filter-branch is your friend.\n\nHope that helps!\n\nMax\n"}]}