threads / discuss / 13705

parsecvs losing files

Subject: parsecvs losing files

## tl;dr

4 messages between May 28, 2008 and May 30, 2008.

replies: 3people: 2as markdown or json

Alex Bennee· May 28, 2008, 14:50 UTC · lore
Hi,

I've been using a daily cron job running parsecvs to convert an uber CVS tree into git. It works quite well and is substantially faster than all the other methods I have tried. It also re-creates the same history every time which is something I haven't managed to do with the other tools.

Anyway today I noticed it had failed to import a sub-directory of the project (not a bit I usually build). However looking at the import log from parsecvs I see that the file was read by parsecvs. It looks as though it should have made it into the git repo. The only thing that seems different from the other modules is that the files where imported once and don't seem to have been touched since. This may of caused parsecvs to get confused?

Questions:
1. Has anyone seen this behaviour with parsecvs?
2. Is there any way to list all the tree objects in a repo so I can
search to see if the files exist in a tree but just not this branch?
-- 
Alex, homepage: http://www.bennee.com/~alex/
Alex Bennee· May 28, 2008, 16:53 UTC · re: Alex Bennee · lore

Re: parsecvs losing files

On Wed, May 28, 2008 at 3:50 PM, Alex Bennee <kernel-hacker@bennee.com> wrote:
Show 9 quoted lines
> Hi,
> <snip>
> Anyway today I noticed it had failed to import a sub-directory of the
> project (not a bit I usually build). However looking at the import log
> from parsecvs I see that the file was read by parsecvs. It looks as
> though it should have made it into the git repo. The only thing that
> seems different from the other modules is that the files where
> imported once and don't seem to have been touched since. This may of
> caused parsecvs to get confused?

Well in answer to myself parsecvs does get confused. In an example failed to import file:

Load:                          third-party/libxml/runtest.c,v   8207 of 79070
/export/git/master.cvs/third-party/libxml/runtest.c,v spliced:
	 1.1.1.1
	 1.1
Sorted heads for /export/git/master.cvs/third-party/libxml/runtest.c,v
	master 1.1
	master > BRANCH-3-5-branch 1.1.1.1.0.2
	master > BRANCH-3-5-16-branch 1.1.1.1.0.4
building branches for /export/git/master.cvs/third-party/libxml/runtest.c,v
file sha1: b694d565caf10fedbc7566f2bf15b893c57d5a19
file sha1: b694d565caf10fedbc7566f2bf15b893c57d5a19
file has 2 revisions

An lo, looking at the branches mentioned these missing files are there. Trouble is the files should be in a number of branches, looking at the ,v file in question:

head	1.1;
branch	1.1.1;
access;
symbols
	BRANCH-3-5-26-1:1.1.1.1
	BRANCH-3-5-25-1:1.1.1.1
	BRANCH-3-5-24-1:1.1.1.1
	BRANCH-3-5-22-3:1.1.1.1
	BRANCH-3-5-22-1:1.1.1.1
	BRANCH-3-5-21-1:1.1.1.1
	BRANCH-3-5-20-1:1.1.1.1
	BRANCH-3-5-16-7:1.1.1.1
	BRANCH-3-5-19-1:1.1.1.1
	BRANCH-3-5-16-6:1.1.1.1
	BRANCH-3-5-16-5:1.1.1.1
	BRANCH-3-5-18-1:1.1.1.1
	BRANCH-3-5-16-4:1.1.1.1
	BRANCH-3-5-16-branch:1.1.1.1.0.4
	BRANCH-3-5-17-1:1.1.1.1
	BRANCH-3-5-16-3:1.1.1.1
	BRANCH-3-5-16-2:1.1.1.1
	BRANCH-3-5-16-1:1.1.1.1
	post-merge-of-ajz-post-3-5-branch:1.1.1.1
	pre-merge-of-ajz-post-3-5-branch:1.1.1.1
	BRANCH-3-5-15-1:1.1.1.1
	BRANCH-3-5-14-1:1.1.1.1
	BRANCH-3-5-branch:1.1.1.1.0.2
	BRANCH-3-5-13-1:1.1.1.1
	BRANCH-3-5-12-1:1.1.1.1
	BRANCH-3-5-11-1:1.1.1.1
	BRANCH-3-5-10-1:1.1.1.1
	BRANCH-3-5-9-2:1.1.1.1
	BRANCH-3-3-20-red-e1-opt-branch:1.1.1.1
	GNOME_LIBXML2_2_6_29:1.1.1.1
	GNOME:1.1.1;
locks; strict;
comment	@ * @;
1.1
date	2007.08.16.15.59.35;	author jbpn;	state Exp;
branches
	1.1.1.1;
next	;

1.1.1.1 date 2007.08.16.15.59.35; author jbpn; state Exp; branches; next ;

desc @@

1.1 log @Initial revision <snip rest of file>

I'm not sure why parsecvs got this wrong, I'm digging through it but I'm a little fuzzy where lex/bison/whatever is involved. I notice looking at the log for some of the files that did make it that the CVS revisions for all the branches have a .0.[something] suffix which the missing branches in this case don't have. However CVS is very sure the file exists in these other branches.

Could this be something to do with explicit branches and sticky branch tags? Or just a straight bug in parsecvs?

-- 
Alex, homepage: http://www.bennee.com/~alex/
Pierre Habouzit· May 28, 2008, 17:03 UTC · re: Alex Bennee · lore

Re: parsecvs losing files

  Sorry for the top posting but the git list isn't really the upstream
for parsecvs. I'm now Cc-ing keithp who is the author :)
On Wed, May 28, 2008 at 04:53:30PM +0000, Alex Bennee wrote:
Show 109 quoted lines
> On Wed, May 28, 2008 at 3:50 PM, Alex Bennee <kernel-hacker@bennee.com> wrote:
> > Hi,
> > <snip>
> > Anyway today I noticed it had failed to import a sub-directory of the
> > project (not a bit I usually build). However looking at the import log
> > from parsecvs I see that the file was read by parsecvs. It looks as
> > though it should have made it into the git repo. The only thing that
> > seems different from the other modules is that the files where
> > imported once and don't seem to have been touched since. This may of
> > caused parsecvs to get confused?
> 
> Well in answer to myself parsecvs does get confused. In an example
> failed to import file:
> 
> Load:                          third-party/libxml/runtest.c,v   8207 of 79070
> /export/git/master.cvs/third-party/libxml/runtest.c,v spliced:
> 	 1.1.1.1
> 	 1.1
> Sorted heads for /export/git/master.cvs/third-party/libxml/runtest.c,v
> 	master 1.1
> 	master > BRANCH-3-5-branch 1.1.1.1.0.2
> 	master > BRANCH-3-5-16-branch 1.1.1.1.0.4
> building branches for /export/git/master.cvs/third-party/libxml/runtest.c,v
> file sha1: b694d565caf10fedbc7566f2bf15b893c57d5a19
> file sha1: b694d565caf10fedbc7566f2bf15b893c57d5a19
> file has 2 revisions
> 
> An lo, looking at the branches mentioned these missing files are
> there. Trouble is the files should be in a number of branches, looking
> at the ,v file in question:
> 
> head	1.1;
> branch	1.1.1;
> access;
> symbols
> 	BRANCH-3-5-26-1:1.1.1.1
> 	BRANCH-3-5-25-1:1.1.1.1
> 	BRANCH-3-5-24-1:1.1.1.1
> 	BRANCH-3-5-22-3:1.1.1.1
> 	BRANCH-3-5-22-1:1.1.1.1
> 	BRANCH-3-5-21-1:1.1.1.1
> 	BRANCH-3-5-20-1:1.1.1.1
> 	BRANCH-3-5-16-7:1.1.1.1
> 	BRANCH-3-5-19-1:1.1.1.1
> 	BRANCH-3-5-16-6:1.1.1.1
> 	BRANCH-3-5-16-5:1.1.1.1
> 	BRANCH-3-5-18-1:1.1.1.1
> 	BRANCH-3-5-16-4:1.1.1.1
> 	BRANCH-3-5-16-branch:1.1.1.1.0.4
> 	BRANCH-3-5-17-1:1.1.1.1
> 	BRANCH-3-5-16-3:1.1.1.1
> 	BRANCH-3-5-16-2:1.1.1.1
> 	BRANCH-3-5-16-1:1.1.1.1
> 	post-merge-of-ajz-post-3-5-branch:1.1.1.1
> 	pre-merge-of-ajz-post-3-5-branch:1.1.1.1
> 	BRANCH-3-5-15-1:1.1.1.1
> 	BRANCH-3-5-14-1:1.1.1.1
> 	BRANCH-3-5-branch:1.1.1.1.0.2
> 	BRANCH-3-5-13-1:1.1.1.1
> 	BRANCH-3-5-12-1:1.1.1.1
> 	BRANCH-3-5-11-1:1.1.1.1
> 	BRANCH-3-5-10-1:1.1.1.1
> 	BRANCH-3-5-9-2:1.1.1.1
> 	BRANCH-3-3-20-red-e1-opt-branch:1.1.1.1
> 	GNOME_LIBXML2_2_6_29:1.1.1.1
> 	GNOME:1.1.1;
> locks; strict;
> comment	@ * @;
> 
> 
> 1.1
> date	2007.08.16.15.59.35;	author jbpn;	state Exp;
> branches
> 	1.1.1.1;
> next	;
> 
> 1.1.1.1
> date	2007.08.16.15.59.35;	author jbpn;	state Exp;
> branches;
> next	;
> 
> 
> desc
> @@
> 
> 
> 
> 1.1
> log
> @Initial revision
> <snip rest of file>
> 
> I'm not sure why parsecvs got this wrong, I'm digging through it but
> I'm a little fuzzy where lex/bison/whatever is involved. I notice
> looking at the log for some of the files that did make it that the CVS
> revisions for all the branches have a .0.[something] suffix which the
> missing branches in this case don't have. However CVS is very sure the
> file exists in these other branches.
> 
> Could this be something to do with explicit branches and sticky branch
> tags? Or just a straight bug in parsecvs?
> 
> 
> -- 
> Alex, homepage: http://www.bennee.com/~alex/
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org
Alex Bennee· May 30, 2008, 10:08 UTC · re: Pierre Habouzit · lore

Re: [WORKAROUND] parsecvs losing files

On Wed, May 28, 2008 at 6:03 PM, Pierre Habouzit <madcoder@debian.org> wrote:
>  Sorry for the top posting but the git list isn't really the upstream
> for parsecvs. I'm now Cc-ing keithp who is the author :)
I shall keep CC'ing git@ for visibility if anyone else comes across this :-)
Show 7 quoted lines
>
> On Wed, May 28, 2008 at 04:53:30PM +0000, Alex Bennee wrote:
>> On Wed, May 28, 2008 at 3:50 PM, Alex Bennee <kernel-hacker@bennee.com> wrote:
>> > Hi,
>> > <snip>
>> > Anyway today I noticed it had failed to import a sub-directory of the
>> > project (not a bit I usually build).
<snip>
Show 20 quoted lines
>>
>> Well in answer to myself parsecvs does get confused. In an example
>> failed to import file:
>>
>> Load:                          third-party/libxml/runtest.c,v   8207 of 79070
>> /export/git/master.cvs/third-party/libxml/runtest.c,v spliced:
>>        1.1.1.1
>>        1.1
>> Sorted heads for /export/git/master.cvs/third-party/libxml/runtest.c,v
>>       master 1.1
>>       master > BRANCH-3-5-branch 1.1.1.1.0.2
>>       master > BRANCH-3-5-16-branch 1.1.1.1.0.4
>> building branches for /export/git/master.cvs/third-party/libxml/runtest.c,v
>> file sha1: b694d565caf10fedbc7566f2bf15b893c57d5a19
>> file sha1: b694d565caf10fedbc7566f2bf15b893c57d5a19
>> file has 2 revisions
>>
>> An lo, looking at the branches mentioned these missing files are
>> there. Trouble is the files should be in a number of branches, looking
>> at the ,v file in question:
<snip>
>>       BRANCH-3-3-20-red-e1-opt-branch:1.1.1.1
<snip>
>> I notice
>> looking at the log for some of the files that did make it that the CVS
>> revisions for all the branches have a .0.[something] suffix which the
>> missing branches in this case don't have.
<snip>

So I understand how this has happened. This particular module was imported directly into the working branch at the time (BRANCH-3-3-20-red-e1-opt-branch) where as other modules where imported into the CVS HEAD and then branched into the current working branch. As a result the branch tag didn't have the magic 0 in it.

We where able to work around the import failure by deleting the branch tag from the module and then branching it again. The new branch tag became:

BRANCH-3-3-20-red-e1-opt-branch:1.1.1.1.0.6

which parsecvs was able to correctly parse and assign to the correct GIT branch on import. I'm guessing this is a corner case that could do with better handling but in our case it was solved by tweaking our CVS repository.

-- 
Alex, homepage: http://www.bennee.com/~alex/

← back to recent threads