{"thread":{"id":"14056","subject":"Importing non-version controlled bits and pieces to Git","startedAt":"2008-06-20T09:42:22Z","lastAt":"2008-06-24T06:46:14Z","messageCount":10,"participants":["Peter Karlsson","Miklos Vajna","Johannes Schindelin","Johannes Sixt","Karl Hasselström"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"80451","messageId":"Pine.LNX.4.64.0806201036440.24013@ds9.cixit.se","threadId":"14056","inReplyTo":null,"subject":"Importing non-version controlled bits and pieces to Git","fromName":"Peter Karlsson","fromEmail":"peter@softwolves.pp.se","sentAt":"2008-06-20T09:42:22Z","receivedAt":"2008-06-20T09:42:22Z","isPatch":false,"sender":{"key":"peter@softwolves.pp.se","avatar":"https://avatars.githubusercontent.com/u/990764?v=4"},"body":"Hi!\n\nI am looking at moving a project that has not been version-controlled\nin the regular sense into Git. I have found it to currently consist of\n40 directories with copies of the set of files in various shapes.\n\nI have reconstructed something that looks like a probable revision tree\nfor these 40 directories, and I want to put this into Git so that I can\nexamine what changes have been made where, and merge the various\nversions back together so that there can be one version with all the\nvarious fixes.\n\nI was looking for something that could do this for me, but found nothing\nmentioned on the Wiki, and my Google karma was not good enough to find\nsomething, either. The closest thing I know of is the import-tars.perl\nscript that I have used before, when importing something with a\n(near-)linear history, but this is too far off from a linear history to\ntry to tar up the directories and import that way, I believe.\n\nHas anyone written such an import tool, or should I start hacking?\n\n-- \n\\\\// Peter, digital archeologist\n"},{"id":"80463","messageId":"20080620115102.GW29404@genesis.frugalware.org","threadId":"14056","inReplyTo":"Pine.LNX.4.64.0806201036440.24013@ds9.cixit.se","subject":"Re: Importing non-version controlled bits and pieces to Git","fromName":"Miklos Vajna","fromEmail":"vmiklos@frugalware.org","sentAt":"2008-06-20T11:51:02Z","receivedAt":"2008-06-20T11:51:02Z","isPatch":false,"sender":{"key":"vmiklos@frugalware.org","avatar":"https://gravatar.com/avatar/401c1cbbb3a5d13e650c691a2c71d6fd0b80df1a01bc74d9f1972675dd58f2bd?d=mp&s=160"},"body":"On Fri, Jun 20, 2008 at 10:42:22AM +0100, Peter Karlsson <peter@softwolves.pp.se> wrote:\n> I am looking at moving a project that has not been version-controlled\n> in the regular sense into Git. I have found it to currently consist of\n> 40 directories with copies of the set of files in various shapes.\n> \n> I have reconstructed something that looks like a probable revision tree\n> for these 40 directories, and I want to put this into Git so that I can\n> examine what changes have been made where, and merge the various\n> versions back together so that there can be one version with all the\n> various fixes.\n\nMaybe contrib/fast-import/import-tars.perl?\n\nIt imports tar.{gz,bz2,Z} files, but you can tar up your trees or it can\njust serve as an example and you can write your own script that does the\nsame with directories.\n"},{"id":"80468","messageId":"alpine.DEB.1.00.0806201323390.6439@racer","threadId":"14056","inReplyTo":"20080620115102.GW29404@genesis.frugalware.org","subject":"Re: Importing non-version controlled bits and pieces to Git","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-06-20T12:25:51Z","receivedAt":"2008-06-20T12:25:51Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Fri, 20 Jun 2008, Miklos Vajna wrote:\n\n> On Fri, Jun 20, 2008 at 10:42:22AM +0100, Peter Karlsson <peter@softwolves.pp.se> wrote:\n> > I am looking at moving a project that has not been version-controlled \n> > in the regular sense into Git. I have found it to currently consist of \n> > 40 directories with copies of the set of files in various shapes.\n> > \n> > I have reconstructed something that looks like a probable revision \n> > tree for these 40 directories, and I want to put this into Git so that \n> > I can examine what changes have been made where, and merge the various \n> > versions back together so that there can be one version with all the \n> > various fixes.\n> \n> Maybe contrib/fast-import/import-tars.perl?\n\nBTW this is a relatively simple script showing how to import commits into \na Git repository using git-fast-import.\n\nYou should be able to use git-fast-import from a simple script yourself; \nif you would like to see an example how to feed fast-import, just export a \nfew commits with \"git fast-export HEAD~5..\".\n\nHth,\nDscho\n"},{"id":"80470","messageId":"Pine.LNX.4.64.0806201339330.4004@ds9.cixit.se","threadId":"14056","inReplyTo":"alpine.DEB.1.00.0806201323390.6439@racer","subject":"Re: Importing non-version controlled bits and pieces to Git","fromName":"Peter Karlsson","fromEmail":"peter@softwolves.pp.se","sentAt":"2008-06-20T12:42:33Z","receivedAt":"2008-06-20T12:42:33Z","isPatch":false,"sender":{"key":"peter@softwolves.pp.se","avatar":"https://avatars.githubusercontent.com/u/990764?v=4"},"body":"Johannes Schindelin:\n\n> On Fri, 20 Jun 2008, Miklos Vajna wrote:\n> > Maybe contrib/fast-import/import-tars.perl?\n\nAs I noted, I have used that before, and it works best with linear\nhistories, even if I managed to hack it to create a branch for a\nproject that had a simple branching structure (an update was created\nfor version n.x after version n+1.x was released).\n\n> You should be able to use git-fast-import from a simple script\n> yourself; if you would like to see an example how to feed\n> fast-import, just export a few commits with \"git fast-export\n> HEAD~5..\".\n\nYeah, that is what I am currently planning on doing. I was just trying\nto be lazy and see if someone already had solved the problem for me :-)\n\n\nOne complicating factor is that I am working with DOS/Windows source\ncode, so all text files have CRLF line-endings, which means converting\nbefore feeding to fast-import, or manually doing \"git add\" with\ncore.autocrlf == true. fast-import can't be set up to do CRLF\nauto-conversion, can it?\n\n-- \n\\\\// Peter - http://www.softwolves.pp.se/\n"},{"id":"80473","messageId":"485BA94F.4090805@viscovery.net","threadId":"14056","inReplyTo":"Pine.LNX.4.64.0806201339330.4004@ds9.cixit.se","subject":"Re: Importing non-version controlled bits and pieces to Git","fromName":"Johannes Sixt","fromEmail":"j.sixt@viscovery.net","sentAt":"2008-06-20T12:57:51Z","receivedAt":"2008-06-20T12:57:51Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Peter Karlsson schrieb:\n> One complicating factor is that I am working with DOS/Windows source\n> code, so all text files have CRLF line-endings, which means converting\n> before feeding to fast-import, or manually doing \"git add\" with\n> core.autocrlf == true. fast-import can't be set up to do CRLF\n> auto-conversion, can it?\n\nNo, but you can put `tr -d '\\015'` in the pipeline in front of\nfast-import. (Unless, of course, you also have to feed binary data.)\n\n-- Hannes\n"},{"id":"80475","messageId":"alpine.DEB.1.00.0806201359320.6439@racer","threadId":"14056","inReplyTo":"Pine.LNX.4.64.0806201339330.4004@ds9.cixit.se","subject":"Re: Importing non-version controlled bits and pieces to Git","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-06-20T13:00:22Z","receivedAt":"2008-06-20T13:00:22Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Fri, 20 Jun 2008, Peter Karlsson wrote:\n\n> One complicating factor is that I am working with DOS/Windows source \n> code, so all text files have CRLF line-endings, which means converting \n> before feeding to fast-import, or manually doing \"git add\" with \n> core.autocrlf == true. fast-import can't be set up to do CRLF \n> auto-conversion, can it?\n\nWhy not import everything with a CR/LF, and then use filter-branch with a \nreally simple tree-filter?  It is slow, alright, but it is safe, too.\n\nCiao,\nDscho\n"},{"id":"80678","messageId":"Pine.LNX.4.64.0806230732120.31319@ds9.cixit.se","threadId":"14056","inReplyTo":"alpine.DEB.1.00.0806201359320.6439@racer","subject":"Re: Importing non-version controlled bits and pieces to Git","fromName":"Peter Karlsson","fromEmail":"peter@softwolves.pp.se","sentAt":"2008-06-23T06:35:14Z","receivedAt":"2008-06-23T06:35:14Z","isPatch":false,"sender":{"key":"peter@softwolves.pp.se","avatar":"https://avatars.githubusercontent.com/u/990764?v=4"},"body":"Johannes Schindelin:\n\n> Why not import everything with a CR/LF, and then use filter-branch\n> with a really simple tree-filter? It is slow, alright, but it is\n> safe, too.\n\nThat might work. My problem is the non-linearity of the data I want to\nimport. But I might be able to massage the import-tars output before I\nfeed it to git-fast-import to describe the history I need it to.\n\n-- \n\\\\// Peter - http://www.softwolves.pp.se/\n"},{"id":"80706","messageId":"Pine.LNX.4.64.0806231543580.31319@ds9.cixit.se","threadId":"14056","inReplyTo":"Pine.LNX.4.64.0806230732120.31319@ds9.cixit.se","subject":"Re: Importing non-version controlled bits and pieces to Git","fromName":"Peter Karlsson","fromEmail":"peter@softwolves.pp.se","sentAt":"2008-06-23T14:46:11Z","receivedAt":"2008-06-23T14:46:11Z","isPatch":false,"sender":{"key":"peter@softwolves.pp.se","avatar":"https://avatars.githubusercontent.com/u/990764?v=4"},"body":"Peter Karlsson:\n\n> That might work. My problem is the non-linearity of the data I want\n> to import. But I might be able to massage the import-tars output\n> before I feed it to git-fast-import to describe the history I need it\n> to.\n\nI ended up making a script that converted a description file that\ndescribed the hierarchy of versions and generated a fast-import file\nfrom it. Good to have so that I could move stuff around a bit to make\nup a version tree that looked somewhat reasonable (a lot of\ncross-directory copying has been going on here, and no-one knows which\nversion is which).\n\nI will try to make the script available.\n\n-- \n\\\\// Peter - http://www.softwolves.pp.se/\n"},{"id":"80830","messageId":"20080624051204.GA1760@diana.vm.bytemark.co.uk","threadId":"14056","inReplyTo":"Pine.LNX.4.64.0806231543580.31319@ds9.cixit.se","subject":"Re: Importing non-version controlled bits and pieces to Git","fromName":"Karl Hasselström","fromEmail":"kha@treskal.com","sentAt":"2008-06-24T05:12:04Z","receivedAt":"2008-06-24T05:12:04Z","isPatch":false,"sender":{"key":"kha@treskal.com","avatar":"https://gravatar.com/avatar/f0120c734b5279b345075a28521e1ac66acb20c9913ffe9bf6ae97e53f7f3f13?d=mp&s=160"},"body":"On 2008-06-23 15:46:11 +0100, Peter Karlsson wrote:\n\n> Peter Karlsson:\n>\n> > That might work. My problem is the non-linearity of the data I\n> > want to import. But I might be able to massage the import-tars\n> > output before I feed it to git-fast-import to describe the history\n> > I need it to.\n>\n> I ended up making a script that converted a description file that\n> described the hierarchy of versions and generated a fast-import file\n> from it. Good to have so that I could move stuff around a bit to\n> make up a version tree that looked somewhat reasonable (a lot of\n> cross-directory copying has been going on here, and no-one knows\n> which version is which).\n\nA bit too late now, but ... another thing you could have done is make\na straight linear import à la import-tars, and then modified the\nparentage with grafts until it looked good; and then finally use\nfilter-branch to make the grafts part of the \"real\" history.\n\nThe advantage of this approach is that you can edit the parentage\nalmost interactively, which should be the best approach given that you\nhave a small number of commits and have to guess their relationships.\n\n-- \nKarl Hasselström, kha@treskal.com\n      www.treskal.com/kalle\n"},{"id":"80838","messageId":"Pine.LNX.4.64.0806240742240.5039@ds9.cixit.se","threadId":"14056","inReplyTo":"20080624051204.GA1760@diana.vm.bytemark.co.uk","subject":"Re: Importing non-version controlled bits and pieces to Git","fromName":"Peter Karlsson","fromEmail":"peter@softwolves.pp.se","sentAt":"2008-06-24T06:46:14Z","receivedAt":"2008-06-24T06:46:14Z","isPatch":false,"sender":{"key":"peter@softwolves.pp.se","avatar":"https://avatars.githubusercontent.com/u/990764?v=4"},"body":"Karl Hasselström:\n\n> A bit too late now, but ...\n\nNot really, I'm on to the next project to import :-)\n\n> another thing you could have done is make a straight linear import à\n> la import-tars, and then modified the parentage with grafts until it\n> looked good; and then finally use filter-branch to make the grafts\n> part of the \"real\" history.\n\nYeah, I was considering this approach, but then I was half-way\ndesigning the script in my head, plus that the import is so quick that\nit was easy enough to re-write the configuration file and re-import to\ntry something else. I love that aspect of Git -- the preparing of the\nfile to import takes longer than the actual git-fast-import call...\n\n> The advantage of this approach is that you can edit the parentage\n> almost interactively, which should be the best approach given that\n> you have a small number of commits and have to guess their\n> relationships.\n\n\"Almost\" being the magic word :-) I achieved more or less the same by\nre-importing. Plus that the job of preparing tarballs of everything\nwould have been some work as well, since I had to choose which files to\nimport from each directory (and do CRLF transform).\n\n-- \n\\\\// Peter - http://www.softwolves.pp.se/\n"}]}