# Importing non-version controlled bits and pieces to Git 10 messages from 2008-06-20 to 2008-06-24. Participants: Peter Karlsson, Miklos Vajna, Johannes Schindelin, Johannes Sixt, Karl Hasselström. Thread: https://gitlist.dev/t/14056 ## Peter Karlsson, 2008-06-20 09:42 Subject: Importing non-version controlled bits and pieces to Git Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0806201036440.24013%40ds9.cixit.se ``` Hi! I am looking at moving a project that has not been version-controlled in the regular sense into Git. I have found it to currently consist of 40 directories with copies of the set of files in various shapes. I have reconstructed something that looks like a probable revision tree for these 40 directories, and I want to put this into Git so that I can examine what changes have been made where, and merge the various versions back together so that there can be one version with all the various fixes. I was looking for something that could do this for me, but found nothing mentioned on the Wiki, and my Google karma was not good enough to find something, either. The closest thing I know of is the import-tars.perl script that I have used before, when importing something with a (near-)linear history, but this is too far off from a linear history to try to tar up the directories and import that way, I believe. Has anyone written such an import tool, or should I start hacking? -- \\// Peter, digital archeologist ``` ## Miklos Vajna, 2008-06-20 11:51 Subject: Re: Importing non-version controlled bits and pieces to Git Message-ID: <20080620115102.GW29404@genesis.frugalware.org> URL: https://gitlist.dev/e/20080620115102.GW29404%40genesis.frugalware.org In-Reply-To: ``` On Fri, Jun 20, 2008 at 10:42:22AM +0100, Peter Karlsson wrote: > I am looking at moving a project that has not been version-controlled > in the regular sense into Git. I have found it to currently consist of > 40 directories with copies of the set of files in various shapes. > > I have reconstructed something that looks like a probable revision tree > for these 40 directories, and I want to put this into Git so that I can > examine what changes have been made where, and merge the various > versions back together so that there can be one version with all the > various fixes. Maybe contrib/fast-import/import-tars.perl? It imports tar.{gz,bz2,Z} files, but you can tar up your trees or it can just serve as an example and you can write your own script that does the same with directories. ``` ## Johannes Schindelin, 2008-06-20 12:25 Subject: Re: Importing non-version controlled bits and pieces to Git Message-ID: URL: https://gitlist.dev/e/alpine.DEB.1.00.0806201323390.6439%40racer In-Reply-To: <20080620115102.GW29404@genesis.frugalware.org> ``` Hi, On Fri, 20 Jun 2008, Miklos Vajna wrote: > On Fri, Jun 20, 2008 at 10:42:22AM +0100, Peter Karlsson wrote: > > I am looking at moving a project that has not been version-controlled > > in the regular sense into Git. I have found it to currently consist of > > 40 directories with copies of the set of files in various shapes. > > > > I have reconstructed something that looks like a probable revision > > tree for these 40 directories, and I want to put this into Git so that > > I can examine what changes have been made where, and merge the various > > versions back together so that there can be one version with all the > > various fixes. > > Maybe contrib/fast-import/import-tars.perl? BTW this is a relatively simple script showing how to import commits into a Git repository using git-fast-import. You should be able to use git-fast-import from a simple script yourself; if you would like to see an example how to feed fast-import, just export a few commits with "git fast-export HEAD~5..". Hth, Dscho ``` ## Peter Karlsson, 2008-06-20 12:42 Subject: Re: Importing non-version controlled bits and pieces to Git Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0806201339330.4004%40ds9.cixit.se In-Reply-To: ``` Johannes Schindelin: > On Fri, 20 Jun 2008, Miklos Vajna wrote: > > Maybe contrib/fast-import/import-tars.perl? As I noted, I have used that before, and it works best with linear histories, even if I managed to hack it to create a branch for a project that had a simple branching structure (an update was created for version n.x after version n+1.x was released). > You should be able to use git-fast-import from a simple script > yourself; if you would like to see an example how to feed > fast-import, just export a few commits with "git fast-export > HEAD~5..". Yeah, that is what I am currently planning on doing. I was just trying to be lazy and see if someone already had solved the problem for me :-) One complicating factor is that I am working with DOS/Windows source code, so all text files have CRLF line-endings, which means converting before feeding to fast-import, or manually doing "git add" with core.autocrlf == true. fast-import can't be set up to do CRLF auto-conversion, can it? -- \\// Peter - http://www.softwolves.pp.se/ ``` ## Johannes Sixt, 2008-06-20 12:57 Subject: Re: Importing non-version controlled bits and pieces to Git Message-ID: <485BA94F.4090805@viscovery.net> URL: https://gitlist.dev/e/485BA94F.4090805%40viscovery.net In-Reply-To: ``` Peter Karlsson schrieb: > One complicating factor is that I am working with DOS/Windows source > code, so all text files have CRLF line-endings, which means converting > before feeding to fast-import, or manually doing "git add" with > core.autocrlf == true. fast-import can't be set up to do CRLF > auto-conversion, can it? No, but you can put `tr -d '\015'` in the pipeline in front of fast-import. (Unless, of course, you also have to feed binary data.) -- Hannes ``` ## Johannes Schindelin, 2008-06-20 13:00 Subject: Re: Importing non-version controlled bits and pieces to Git Message-ID: URL: https://gitlist.dev/e/alpine.DEB.1.00.0806201359320.6439%40racer In-Reply-To: ``` Hi, On Fri, 20 Jun 2008, Peter Karlsson wrote: > One complicating factor is that I am working with DOS/Windows source > code, so all text files have CRLF line-endings, which means converting > before feeding to fast-import, or manually doing "git add" with > core.autocrlf == true. fast-import can't be set up to do CRLF > auto-conversion, can it? Why not import everything with a CR/LF, and then use filter-branch with a really simple tree-filter? It is slow, alright, but it is safe, too. Ciao, Dscho ``` ## Peter Karlsson, 2008-06-23 06:35 Subject: Re: Importing non-version controlled bits and pieces to Git Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0806230732120.31319%40ds9.cixit.se In-Reply-To: ``` Johannes Schindelin: > Why not import everything with a CR/LF, and then use filter-branch > with a really simple tree-filter? It is slow, alright, but it is > safe, too. That might work. My problem is the non-linearity of the data I want to import. But I might be able to massage the import-tars output before I feed it to git-fast-import to describe the history I need it to. -- \\// Peter - http://www.softwolves.pp.se/ ``` ## Peter Karlsson, 2008-06-23 14:46 Subject: Re: Importing non-version controlled bits and pieces to Git Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0806231543580.31319%40ds9.cixit.se In-Reply-To: ``` Peter Karlsson: > That might work. My problem is the non-linearity of the data I want > to import. But I might be able to massage the import-tars output > before I feed it to git-fast-import to describe the history I need it > to. I ended up making a script that converted a description file that described the hierarchy of versions and generated a fast-import file from it. Good to have so that I could move stuff around a bit to make up a version tree that looked somewhat reasonable (a lot of cross-directory copying has been going on here, and no-one knows which version is which). I will try to make the script available. -- \\// Peter - http://www.softwolves.pp.se/ ``` ## Karl Hasselström, 2008-06-24 05:12 Subject: Re: Importing non-version controlled bits and pieces to Git Message-ID: <20080624051204.GA1760@diana.vm.bytemark.co.uk> URL: https://gitlist.dev/e/20080624051204.GA1760%40diana.vm.bytemark.co.uk In-Reply-To: ``` On 2008-06-23 15:46:11 +0100, Peter Karlsson wrote: > Peter Karlsson: > > > That might work. My problem is the non-linearity of the data I > > want to import. But I might be able to massage the import-tars > > output before I feed it to git-fast-import to describe the history > > I need it to. > > I ended up making a script that converted a description file that > described the hierarchy of versions and generated a fast-import file > from it. Good to have so that I could move stuff around a bit to > make up a version tree that looked somewhat reasonable (a lot of > cross-directory copying has been going on here, and no-one knows > which version is which). A bit too late now, but ... another thing you could have done is make a straight linear import à la import-tars, and then modified the parentage with grafts until it looked good; and then finally use filter-branch to make the grafts part of the "real" history. The advantage of this approach is that you can edit the parentage almost interactively, which should be the best approach given that you have a small number of commits and have to guess their relationships. -- Karl Hasselström, kha@treskal.com www.treskal.com/kalle ``` ## Peter Karlsson, 2008-06-24 06:46 Subject: Re: Importing non-version controlled bits and pieces to Git Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0806240742240.5039%40ds9.cixit.se In-Reply-To: <20080624051204.GA1760@diana.vm.bytemark.co.uk> ``` Karl Hasselström: > A bit too late now, but ... Not really, I'm on to the next project to import :-) > another thing you could have done is make a straight linear import à > la import-tars, and then modified the parentage with grafts until it > looked good; and then finally use filter-branch to make the grafts > part of the "real" history. Yeah, I was considering this approach, but then I was half-way designing the script in my head, plus that the import is so quick that it was easy enough to re-write the configuration file and re-import to try something else. I love that aspect of Git -- the preparing of the file to import takes longer than the actual git-fast-import call... > The advantage of this approach is that you can edit the parentage > almost interactively, which should be the best approach given that > you have a small number of commits and have to guess their > relationships. "Almost" being the magic word :-) I achieved more or less the same by re-importing. Plus that the job of preparing tarballs of everything would have been some work as well, since I had to choose which files to import from each directory (and do CRLF transform). -- \\// Peter - http://www.softwolves.pp.se/ ```