gitweb wishlist

  Hello,

  I would be very happy if you could extend the gitweb scripts a little.
Basically, what I need is to have ability to create a permanent link to
a given file in the repository, which stays same across revisions (as
long as the file stays with the given name, obviously).

  E.g. I would like to have something like

	http://www.kernel.org/git/gitweb.cgi?p=cogito%2Fcogito.git;n=contrib/ciabot.pl

for file contrib/ciabot.pl in the latest Cogito tree, and

	http://www.kernel.org/git/gitweb.cgi?p=cogito%2Fcogito.git;n=contrib

for the list of the contrib/ directory in the latest Cogito tree.

  I think I would prefer the link from the repository index to go not to
the log page, but some "summary" page, which would have some short
information about the repository (owner, description, list of branches
if gitweb supports that, list of tags, link to the latest tree and link
to the log).

  BTW, why are people using ';' in the URIs for separators instead of
good old '&'s? Hmm, actually it just occurred to me that it might be to
workaround entity string problems?

  Thanks,

In article <20050511012626.GL26384@pasky.ji.cz> (at Wed, 11 May 2005 03:26:26 +0200), Petr Baudis <pasky@ucw.cz> says:

                                                     ~~~ why not /?

--yoshfuji

Dear diary, on Wed, May 11, 2005 at 03:49:07AM CEST, I got a letter where "YOSHIFUJI Hideaki / ?$B5HF#1QL@" <yoshfuji@linux-ipv6.org> told me that...

This was a simple cut'n'paste. I don't care either way personally, but I think it's better to be conservative when generating links and escape this. OTOH it should obviously also accept a URL containing a slash instead.

On Wed, 2005-05-11 at 03:26 +0200, Petr Baudis wrote:

With the next round I will try to introduce a real browser through the tree instead of the current damn simple tree-object list. It would be nice if it can show the the current path and provide a "up to parent" way. Along with that it should be easy to add the stuff you asked for.

Yes, but it's "bad old" not "good". :)
  http://www.w3.org/TR/html4/appendix/notes.html#h-B.2.2

Thanks, Kay

On Wed, 2005-05-11 03:26:26 +0200, Petr Baudis <pasky@ucw.cz> wrote:

I've got another one. I'd like to see st_mtime on the file lists to see when a file was last touched...

MfG, JBG

On Wed, 2005-05-11 at 11:30 +0200, Jan-Benedict Glaw wrote:

We don't have that information in the trees. The date is only available in the commits and reading all commits or storing that information in a index or a database is beyond the scope of that simple cgi.

Thanks, Kay

* [Previous page] [Next page] would be nice in addition to last
  10, day, week, etc.

* Putting the commit headline and "X hour"s ago in a separate
  div or span next to each other, so that a long commit headline
  wraps properly and does not start the second line just under
  the "X hours ago" timestamp would be nicer (you can see what I
  mean easily by narrowing the browser window).

On Thu, 2005-05-12 at 13:07 -0700, Junio C Hamano wrote:

That should be easy to do with the parameters we have now for the git-rev-list. I will first finish the new browser through the trees/files, then the project overview page and after that try the pager,

Block elements (div) are not allowed inside an a-tag in XHTML/Strict - don't know how to do this, cause the whole headline should be a link without the use of javascript. :)

Thanks, Kay

KS> Block elements (div) are not allowed inside an a-tag in XHTML/Strict - KS> don't know how to do this,...

Wouldn't splitting the commit headline and "X hours ago" into two separate elements and wrap them individually inside an a-tag each pointing at the same destination good enough then?

KS> On Thu, 2005-05-12 at 13:07 -0700, Junio C Hamano wrote:

KS> That should be easy to do with the parameters we have now for the KS> git-rev-list. I will first finish the new browser through the KS> trees/files, then the project overview page and after that try the KS> pager,

Is this still on your list of things to do?

I don't know if this is intentional, but it looks like gitweb discards everything after the first line starting with 'Signed-off-by:' on the log page. This will in some cases remove valuable log information when a second author or the committer adds additional comments to a commit:

        <log message by author>

        Signed-off-by: <author>

        <log message by committer>

        Signed-off-by: <committer>

The commit page gets it right, which is why I suspect it might just be a way to trim the amount of text on the log page.

I also noticed that there is a 'faulty' signed-off-by line in commit 14ebb908e10f068dc1901d35f4b716bc69143d19 in case the above is intentional. Dunno if that should be matched by relaxing the regexp a little.

On Fri, 2005-05-13 at 14:06 +0200, Jonas Fonseca wrote:

All that should work now.

Thanks, Kay

On Wed, 2005-05-11 at 03:26 +0200, Petr Baudis wrote:

http://www.kernel.org/git/gitweb.cgi?p=cogito/cogito.git;a=blob;f=contrib/ciabot.pl

http://www.kernel.org/git/gitweb.cgi?p=cogito/cogito.git;a=tree;f=contrib

Sounds reasonable, I will try that with the next round.

Thanks, Kay

Kay Sievers <kay.sievers@vrfy.org> wrote Sat, May 14, 2005:

How about support for getting the 'raw' files and diffs without all the HTML markup?

On Wed, 2005-05-11 at 03:26 +0200, Petr Baudis wrote:

I did this now. The top-link shows now the repository listing with a nice "last change" field. The default link points to an overview page which also list the tags. (The owner filed in that list is not correct until now, cause the cron-job needs to be adapted.)

Thanks, Kay

Dear diary, on Wed, May 18, 2005 at 04:55:51AM CEST, I got a letter where Kay Sievers <kay.sievers@vrfy.org> told me that...

Thanks, this is exactly how I envisioned it. :-)

On Wed, 18 May 2005, Kay Sievers wrote:

In the summary page, could we get authorship information too? Right now it looks like

	recent commits

	15 minutes ago	[PATCH] Add tests for diff-tree
	31 minutes ago	diff-tree: use new base_name_compare() helper function
	34 minutes ago	Introduce "base_name_compare()" helper function 
	...

and wouldn't it be nice if it told you who had written these things, like

	recent commits

	15 minutes ago	Junio C Hamano	[PATCH] Add tests for diff-tree
	31 minutes ago	Linus Torvalds	diff-tree: use new base_name_compare() helper function
	34 minutes ago	Linus Torvalds	Introduce "base_name_compare()" helper function 
	...

(limit the name to the first 20 characters or something to make things line up).

If the lines get too long, you could changes "minutes" to "min", and maybe limit the name to the first 10 characters (or even first name, but there's a lot of David's around, while 10-12 characters tends to get enough of the last name to be reasonable unique).

		Linus

LT> If the lines get too long, you could changes "minutes" to "min", and maybe LT> limit the name to the first 10 characters (or even first name, but there's LT> a lot of David's around, while 10-12 characters tends to get enough of LT> the last name to be reasonable unique).

Or abbreviate the first names.  L Torvalds, JC Hamano, etc.

On Fri, 20 May 2005, Junio C Hamano wrote:

Technically, yes. Except I end up at least personally going by either first names (or _possibly_ email addresses), not by last name. Maybe Linux is fairly unique in that, but dammit, I'm not "Torvalds", I'm "Linus". And like it or not, you're either Junio or junkio at least to me ;)

		Linus

On Fri, 2005-05-20 at 09:54 -0700, Linus Torvalds wrote:

Something like that: :)
  http://www.kernel.org/git/?p=cogito/cogito.git;a=summary

Kay

On Fri, 20 May 2005, Kay Sievers wrote:

Looking good.

I still think "minutes" is an awfully long word to waste vertical space with, especially since it's also the only one that will have up to three digits associated with it. It seems kind of silly to give more (or even equal) space to the simplified time field than to the person who did something.

(To see minutes, you need to use "git/git.git" instead of cogito.

So I still think you should change "minutes" to "min", which is the common abbreviation.

(That "ago" is also very repetitive, but removing it would seem to make it unreadable, so I guess it's needed).

				Linus

On Fri, 20 May 2005, Linus Torvalds wrote:

Oh, dang, while I'm at it, why not ask for the "commmitdiff" thing to have the commit message in it too, ie basically look like a prettified version of "git-diff-tree -v -M <cmitname>"

You already do the first line of it, so there's not much missing. Right now there is no place where everything important from one commit is "brought together" (ie no single place where you can see both what the diff is all about, and what it actually does).

		Linus

On Fri, 2005-05-20 at 11:28 -0700, Linus Torvalds wrote:

Somehting like this?:
  http://kernel.org/git/?p=git/git.git;a=commitdiff;h=de809dbbce497e0d107562615c1d85ff35b4e0c5

Kay

Hello Kay, I would like to see that I can klick on the file instead of the seperate 'blob' link in a directory view, becasue that is more intuitive and you can already an klick on directories:

http://www.kernel.org/git/?p=git/git.git;a=tree;h=665a48af9e192ed84d2707c95d4c0d9c45eb45ad;hb=411746940f02f6fb90c4b6b97c6f07cee599c2e1

Thanks for this great tool!

Sincerely,
	Thomas

On Fri, 20 May 2005, Kay Sievers wrote:

Yes, except I don't think you should repeat the part of the commit message that you used as a header (so in your example, since you already used "Fix up previous commit" in the header, don't show it below).

Btw, any chance to see gitweb itself under gitweb on kernel.org?

		Linus

On Fri, 20 May 2005, Kay Sievers wrote:

Btw, at least for me, this looks much more interesting than the "commit" thing, and maybe it would make sense to make the summary links be to the "commitdiff" instead of the "commit"?

Or is it just so much more expensive to generate, that we want to not have people go there normally? (hpa cc'd, since he may have some insight into whether this is likely to be an issue or not? It's not like git-diff-tree is that expensive, but it _does_ end up doing a "diff" against each changed file, of course, modulo any caching of results).

		Linus

Linus Torvalds wrote:

What I ended up doing for the diff viewer on kernel.org is that every page that's generated gets stuffed in a cache (locklessly indexed by a SHA-1 of a canonicalized form of the query); the pages people actually see are then simply pulled from the cache. This caching was a just enormous win. In the case of the diff viewer, the header is generated each time, since I allow the user to select a custom style sheet (and don't want to cache versions for each style sheet), but that's a trivial detail.

	-hpa

On Fri, 20 May 2005, H. Peter Anvin wrote:

Ok. That still leaves the bandwidth issue (the full diffs are bigger than the commit object), but usually the diffs in individual commits aren't _that_ large, so maybe it's a non-issue.

Oh, btw, I notice that you moved klibc over to git - care to share your cvs->git script (I assume you scripted it ;)? That would seem to be an obvious addition to the core stuff..

		Linus

Linus Torvalds wrote:

Actually, Kay did the conversion... the scripts are clearly very cantankerous, because if *I* run them -- I tried -- they don't work! Since it's Kay's work, I'll leave them to him, but I would definitely love to move more of my CVS repos over to git, especially syslinux.

	-hpa

Hello, I imported the mutt-cvs for the 1.5 branch into GIT using the following script. But it is a hack. I also think that I will use something like that to build a CVS->GIT vendortracking.

cvsps -x -z 10 -b HEAD -g -p ../../patches/

And using the attached script to import the patches in GIT. It works quiet well.

See also msgid: 1115080139.21105.18.camel@localhost.localdomain there are the scripts which he used to convert the CVS to GIT for HPA. My scripts are based on his work.

	Thomas

#!/usr/bin/perl

use strict; use warnings; use File::Temp qw/ tempfile tempdir /;

# --------------------- # PatchSet 1 # Date: 2002/07/23 07:41:30 # Author: hpa # Branch: HEAD # Tag: (none) # Log: # Initial revision # # Members: # klibc.cvsroot/snprintf.c:INITIAL->1.1 # klibc.cvsroot/vsnprintf.c:INITIAL->1.1 # klibc.cvsroot/klibc/Makefile:INITIAL->1.1 # klibc.cvsroot/klibc/snprintf.c:INITIAL->1.1 # klibc.cvsroot/klibc/vsnprintf.c:INITIAL->1.1 # # --- /dev/null 2005-04-30 18:00:24.840397008 +0200 # +++ klibc/klibc.cvsroot/snprintf.c 2005-05-02 19:57:42.879913000 +0200 # @@ -0,0 +1,19 @@ # +/*

my $patch = $ARGV[0];

my %committer = (
	brendan  => [ 'Brendan Cully',   'brendan@kublai.com' ],
	me       => [ 'Michael Elkins',  'me@sigpipe.org' ],
	roessler => [ 'Thomas Roessler', 'roessler@does-not-exist.org' ]
);

my @log = ();

$ENV{GIT_AUTHOR_EMAIL} = ""; $ENV{GIT_COMMITTER_EMAIL} = "";

open (my $fd, $patch);
while (my $line = <$fd>) {
	if ($line =~ m/^Date: (.*)/) {
		$ENV{GIT_AUTHOR_DATE} = $1;

	} elsif ($line =~ m/^Author: (.*)/) {
		if (defined($committer{$1})) {
			$ENV{GIT_COMMITTER_NAME}  = @{$committer{$1}}[0];
			$ENV{GIT_COMMITTER_EMAIL} = @{$committer{$1}}[1];
			$ENV{GIT_AUTHOR_NAME}         = @{$committer{$1}}[0];
			$ENV{GIT_AUTHOR_EMAIL}        = @{$committer{$1}}[1];
		} else {
			$ENV{GIT_COMMITTER_NAME} = $1;
			$ENV{GIT_AUTHOR_NAME}        = $1;
		}

	} elsif ($line =~ m/^Log:/) {
		while (my $line = <$fd>) {
			if ($line =~ m/^Members: $/) {
				pop(@log);
				last;
			} elsif ($line =~ /^From: (.+) <([^>]+@[^>]+)>$/) {
				$ENV{GIT_AUTHOR_NAME}  = $1;
				$ENV{GIT_AUTHOR_EMAIL} = $2;
			}
			push @log, $line;
		}
	}
}

close($fd);

my ($fh, $logfile) = tempfile(CLEANUP => 1); print $fh @log; system("git patch $patch < $logfile"); close($fh);

On Fri, 2005-05-20 at 13:50 -0700, H. Peter Anvin wrote:

Here we go;

These scripts are just a quick hack, I just wanted to know how nice the stupid cvs file history can be converted to git-committs.

It exports the CVS repo with the help of the nice cvsps to individual patches. (Every patch contains something like a "ChangeSet" by searching for file revisions with the same checkin-date)

Then the patches with the header are split into individual files for committing it into git (similar to Linus' git-mbox-tools).

If we reach a CVS tag with a patch during sequential patching, the script throws away the whole current working tree and checks the revision out of CVS. This way we make sure, that the git-tag matches tree CVS has tagged. (I've encountered two mismatches in the "patch-chain" with the CVS revision-tag. These corrections are hardcoded into the script. :)

For every CVS revision-tag a git-tag without any content except the name is created.

And the klibc-repo was created with a patched git-commit to fake the commit date with the author date. :)

Good luck with it, Kay

Kay Sievers wrote:

In fact, I kind of wish we'd also made committer == author.

Since this whole thing is an import from another revision control system, one really wants that. It's one of those very rare situations in which fudging the commit date is not only fully legitimate, but darn near required.

	-hpa

On Sat, 21 May 2005, Kay Sievers wrote:

Ugh, indeed.

Is it a cvsps bug or what that causes you to have to re-order the patches? Or is it that you don't handle branches or something in CVS? The fact that you also remove one of the tags "suppress ash-branch" in that same number sequence that you had to fix up by re-ordering seems to imply that the breakage has something to do with branching.

Does anybody have any suggestions for a nice and smallish CVS project that has branches that I should look at?

		Linus

On Fri, 2005-05-20 at 12:22 -0700, Linus Torvalds wrote:

How about this:
  http://www.kernel.org/git/?p=git/git.git;a=summary

The default link is still the same, but you can use the link at the end.

Kay

On Fri, 2005-05-20 at 11:16 -0700, Linus Torvalds wrote:

Changed to "min".

Kay

Hi, Kay Sievers wrote:

Cool.

More feature requests:  ;-)
- Alternate white and almost-white backgrounds in the lists (all of them ;-)
  so that wide-screened people like me don't lose context when their eyes
  travel the long road from left to right edge of the screen. ;-)

- Merges currently don't have diff links. It'd be nice to have one for
  each parent.

- File diffs have the "diff" link on the *parent*, not on the child.
  That's counter-intuitive -- if I want to see what the Foo patch changes,
  I should be able to click on the "diff" link on _that_ line, not the one
  below it. Example:
http://www.kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=history;h=9636273dae265b9354b861b373cd43cd76a6d0fe;f=MAINTAINERS

On Sat, 2005-05-21 at 09:29 +0200, Matthias Urlichs wrote:

Done!

Done! But I'm not sure if that is really useful. It may create a very very big diff. :)

No, the "diff" link is a diff against the current commit not an incremental one from revision to revision. Me may change that, I'm not sure what's the best here.

Kay

[ Thomas added to cc, since he seems to have also worked on this ]

On Fri, 20 May 2005, H. Peter Anvin wrote:

Ok, "cvsps" output scares me. I wonder what

	WARNING: Invalid PatchSet 775, Tag syslinux-2_12-pre7:
	    memdisk/init32.asm:1.3=after, memdisk/Makefile:1.26=before. Treated as 'before'
	WARNING: Invalid PatchSet 775, Tag syslinux-2_12-pre7:
	    memdisk/init32.asm:1.3=after, memdisk/e820test.c:1.7=before. Treated as 'before'
	...

means..

Also, your syslinux repo is interesting and shows another thing: doing a

	cvsps -g -p separate

ends badly with

	Directing PatchSet 938 to file separate/938.patch
	cvs rdiff: failed to read diff file header /tmp/cvso8PswZ for mdiskchk.com,v: end of file
	system command returned non-zero exit status: 1: aborting

which doesn't look very promising and causes an empty diff for mdiskck.com. Trying with --cvs-direct shows the reason:

	Index: syslinux/sample/mdiskchk.com
	===================================================================
	RCS file: 
	/home/torvalds/src/osscvs/cvsroot/syslinux/sample/mdiskchk.com,v
	retrieving revision 1.1
	retrieving revision 1.2
	diff -u -r1.1 -r1.2
	Binary files /tmp/cvsU6MGU0 and /tmp/cvsiskFVR differ

which shows that anything that bases itself of diffs (ie uses "-g" with cvsps) is just doomed to failure, since there's no good way to handle binary data. Both Kay's and Thomas' scripts try to do the "-g" thing, that's just not right.

So the cvs->git thing would need to be based on the actual objects, which obviously fits git quite well, but I was really hoping to have cvsps give some nice intermediate format..

So it looks like we should avoid the diff format, and instead use

	cvsps -p separate

and then just parse the "Members" thing and turning each of them either into a "delete" (for ->.*DEAD) or "cvs checkout -rxxx" (for ".*->xxx").

Handling branches by literally treating them as different heads in git sounds quite simple, and indeed it looks like the basic logic for cvs->git translation would be

	for-each-patch-from-cvsps
	do
		git-read-tree -m branchname-from-patch
		git-update-cache -f -u -q -a
		for-each-member-in-patch
		do
			if [ DEAD ]; then
				rm member
				git-update-cache --remove member
			else
				cvs co -rREV member
				git-update-cache --add member
			fi
			cat commit-message-from-patch | 
				git-commit-tree $(git-write-tree) -p branchname-from-patch > .git/revs/heads/branchname-from-patch
		done
	done

which looks like it should work, and handle binary files right.

There seems to be two questions:

 - what to do about branch creation (ie a branch name we haven't seen
   before): it looks like cvsps doesn't tell you what the _originating_
   branch was for a new branch (that may be my confusion - maybe you can't
   create branches off branches in CVS?)

   For syslinux, it looks like you can always base it on HEAD, or possibly 
   just the previous patch (which looks like it is always HEAD). The above 
   pseudo-script will actually do that automatically, simply by virtue of
   the "git-read-tree -m" at the top of the loop failing when the
   branchname doesn't exist yet.

 - whether to bother to create merge entries for when somebody tried to 
   merge a branch back or forth in CVS. 

   CVS fundamentally doesn't have the notion of such a thing, and cvsps 
   can't either. But we could try to guess, based on the commit message, 
   perhaps.

   NOTE! Such a "merge" would not have any real GIT merge functionality 
   what-so-ever. It would just introduce a second parent into the commit, 
   nothing more.

Bah. What crud.

		Linus

Hi, Linus Torvalds wrote:

I have an old CVS->BK merge script lying around which does all of this reasonably correctly. If somebody wants to git-ify it, be my guest (I won't have time in the foreseeable futurre :-( ); it's in the rsync://server.smurf.noris.de/sourcemgr.git/#main repository as bin/b.cvs{,.pl}.

Hi Linus,

Linus Torvalds wrote:

Ok. I'll tell you. It means that the committer uses bad practices in tagging ;-) It generally means that force tag (cvs tag -F <file>) was used on a specific file. Here's the scenario:

cvsps is trying to associate a tag to a specific commit. But in the cvs world this is not always at all possible. If, for example, a commit made and all files are tagged. Now some random file is modified and committed. Then, a bug is found in a file from the previously tagged set, say the file 'memdisk/init32.asm'. The bug is fixed, committed and the tag is MOVED for _just that file_ forward to the new version. Now there is no commit that can be associated with the tag. In this case, cvsps believes this to be a 'FUNKY' tag. There is a more pathological case having to do with 'INVALID' tags... It's enough to make a grown man cry.

I accept patches ;-) Honestly, handling binary data should be trivial I just haven't had the interest, and surprisingly noone else on the internet ever has. The only binary file in the kernel appears to be the logo.gif, according to Ingo.

[ discussion on working around broken handling of binary files in cvsps]

See attached patch to cvsps.c which displays 'Ancestor branch' when this differs from Branch.

Hey, a polished turd is only so shiny... cvsps is a 99% solution [to the problem of extracting metatdata from cvs] only and cvs makes the other 1% impossible.

David

--- cvsps.c~ 2003-04-11 10:06:01.000000000 -0400 +++ cvsps.c 2005-05-23 23:26:12.110231536 -0400

David Mansfield wrote:

This is only pathological if the tag now represents a state that never actually existed in the history of the repository. I don't believe there are any such cases in the syslinux repository; I could be wrong, but I am *highly* sceptical.

Actually, as long as we can create the tree that exists between each changeset, we should be OK.

No sh*t...

	-hpa

H. Peter Anvin wrote:

I didn't mean that YOUR repository had more pathological stuff in it, just that SOME do. 'FUNKY' tags are not really that bad, it's just that there is not a single commit to assign them to (i.e. at no point were all of the objects in the repository at that state simultaneously), which makes the import of such a tag difficult into a more commit oriented system.

Another way to reach 'funky'ness is to modify a file, commit and tag, without having done a 'cvs update' first (and a colleague has done a commit since your last 'cvs update')

David

David Mansfield wrote:

Not sure, sounds more likely.

Either which way, I guess there are two ways to deal with them in 'git'; either as standalone trees (tags pointing to tree objects), or probably more sensical, as impromptu branches if one can find a sane origin object.

	-hpa

On Mon, 23 May 2005, David Mansfield wrote:

The "what crud" refers to cvs. cvsps seems to be a great way to make a tool to migrate away from CVS (or if forced to use CVS, at least show it in a sane manner). So don't take it the wrong way.

I've gotten side-tracked with purely git issues, and since I don't actually have any CVS archives, the cvs->git translation will be on the back-burner for a while, but your "Ancestor branch" patch seems to at least solve the problem that cvsps didn't show all the information that was there. So now I know how to do branches, even if I don't think I'd ever _really_ merge them back (which is as much info as CVS contains).

They'd just be dangling references, ie you could get to them if you wanted to, for historical reasons, and they could be merged merged by hand, of course. Some day..

		Linus

On Mon, 23 May 2005, Linus Torvalds wrote:

Naff.

I just checked in a "cvs2git.c" file in the "tools" project (which has my patch application stuff).

It's still buggy, and it's hacky as hell, but you can basically do something like this:

	cvsps | cvs2git > script

with the normal setup for "cvsps", and "cvs2git" needs one additional stage, namely it wants to know the RCSDIR where to find the RCS files (that should be basically "$CVSROOT/module").

That _script_ then creates a git archive. Very hacky. So after you've successfully created the conversion script, check it to see that it looks sane, and then do

	sh script

and the end result is a git'ified version of your CVS repo (and a corrupted working directory, btw, so look out. It _shouldn't_ corrupt your old CVS repo, though, so it should be ok).

It has the logic for branches, but it doesn't work, and I'm fed up enough with CVS and RCS for the moment that I'm not going to work on it any more tonight. I don't know what stupid bug I have (I've had about a million of them on this silly program), but it's at a point where I think others might find it interesting, and it's probably/hopefully some really embarrassing typo or something and easily fixed.

It converted Peter's "syslinux" repository in a couple of minutes, resulting in 1038 commits (it _should_ have resulted in 1046 commits, that's the branch thing afaik) and most of it looks sane:

	diff-tree cfb715c827e19226a446d47c98a7460fd94633ff (from a809559323f1b370717e475dd252b24686f97727)
	Author: hpa <hpa>
	Date:   Thu May 19 22:30:50 2005 -0700
	    
	    gcc4 compilation fix
	
	
	diff-tree a809559323f1b370717e475dd252b24686f97727 (from 4d65331b50a7b5ce858bb55a58f37b17ebc26c72)
	Author: hpa <hpa>
	Date:   Sun May 8 22:47:03 2005 -0700
	    
	    New Multiboot module; increase command line limit to 1023
	
	
	diff-tree 4d65331b50a7b5ce858bb55a58f37b17ebc26c72 (from e88244753d528f695790adc96f0542d20dc33882)
	Author: hpa <hpa>
	Date:   Fri Apr 29 07:08:03 2005 -0700
	    
	    Don't clobber live registers, it's not nice
	
	
	diff-tree e88244753d528f695790adc96f0542d20dc33882 (from a49e189e35d208648a0d0b52ff652a5f3f8a707e)
	Author: hpa <hpa>
	Date:   Fri Apr 29 07:05:52 2005 -0700
	    
	    Use the correct register

	...
	...
	...

	diff-tree 350772d45425a85dae86ec721d6bd3fde5595d50 (from 47ee894e7821f50cb83ea14b08132337577b2a1e)
	Author: hpa <hpa>
	Date:   Sat Jan 31 13:24:35 1998 -0800
	    
	    Slightly less ugly Id tag.
	
	
	diff-tree 47ee894e7821f50cb83ea14b08132337577b2a1e (from a8b52f1c31055049b276d14c67436d06dd7757aa)
	Author: hpa <hpa>
	Date:   Sat Jan 31 13:22:38 1998 -0800
	    
	    Added Id tags.
	
	
	diff-tree b924672aadb2c3b7f3cac1aaf52fbb4a1ed86b8d (from root)
	Author: hpa <hpa>
	Date:   Sat Jan 31 13:16:05 1998 -0800
	    
	    Initial revision

And btw, it's definitely cvsps that does all the heavy lifting here. "cvs2git" itself is 255 lines of horrid crud, and should have been written in perl, except I only do C..

		Linus

On Tue, 24 May 2005, Linus Torvalds wrote:

I'm back, and yes, it was a really stupid thing.

However, David, I need more help deciphering "cvsps" output..

Fixing the branch handling shows that cvsps does some really strange things with the newly added "Ancestor grpah". Here's one example:

	---------------------
	PatchSet 372 
	Date: 2002/02/03 21:37:50
	Author: hpa
	Branch: syslinux-1_6x-1
	Ancestor branch: HEAD
	Tag: syslinux-1_67 
	Log:
	New mailing list information
	
	Members: 
	        syslinux.doc:1.48->1.48.2.1 
	
	---------------------
	PatchSet 373 
	Date: 2002/02/11 23:08:47
	Author: hpa
	Branch: HEAD
	Tag: (none) 
	Log:
	tftpd32 needs version 2.11 or later.
	
	Members: 
	        pxelinux.doc:1.28->1.29 
	
	---------------------
	PatchSet 374 
	Date: 2002/02/18 23:43:43
	Author: hpa
	Branch: syslinux-1_6x-1
	Ancestor branch: HEAD
	Tag: syslinux-1_6x-merge-2 
	Log:
	Actually make the -o option work properly.
	
	Members: 
	        syslinux.c:1.13->1.13.2.1 
	
	---------------------

note how both 372 _and_ 374 claim to have HEAD as their ancestor, and are on the "syslinux-1_6x-1" branch. What's up with that? Right now this causes my git archive to first create 372 as a branch off HEAD, and then overwrite that with 374, resulting in a dangling branch for 372 that _exists_, but it's not reachable any more, because the branch name that it used has been overwritten by the _new_ branch off HEAD.

Side note: cvs2git is pretty robust since it doesn't rely on patches anywhere, so the head of the branch likely ends up being correct, if that "syslinux.doc" file has been modified anywhere else in the branch. So this _usually_ just results in (a) git-fsck-cache complaining about unreachable commits and (b) possible history being hard to find.

Maybe this cvs2git behaviour is the right thing to do, and what really happened was that the changes described by PatchSet 372 aren't really available any more even in CVS, unless you go back by date or something like that.

However, I suspect it's a cvsps bug in the "ancestor branch" thing. I could work around it by just saying "if I have already seen this branch, I'll ignore the ancestor information".

So I'd like to know whether this is a cvsps issue or whether I actually ended up doing the right thing and it really should be a dangling branch-name that got re-used...

(And if it's a cvsps issue, I'd obviously prefer to get a cvsps patch instead of having a questionable workaround in cvs2git).

"Davi-Mansfieldobi, you're our only hope.."

		Linus

On Tue, 24 May 2005, Linus Torvalds wrote:

Ahh, looking at cvsps source, I think I see what's going on. 

It's deciding the "previous branch" by looking at what the previous branch for the first individual file in the PatchSet was, which fails because in this case, PatchSet 372 was changing "syslinux.doc", and Patchset 374 was changing "syslinux.c", and thus the previous version of the individual _files_ were both in the HEAD branch.

So it does look like I should just ignore the "Ancestor branch" information if the new branch already existed.

Of course, some semantics will never be translatable when trying to treat CVS as a sane system (ie treating CVS as if it was changeset-based is always going to cause strange corner cases since it really is file-based), but that should most likely give the best approximation of what a conversion should do.

		Linus

Linus Torvalds wrote:

I now consider all files in a commit, and all commits in a branch to determine the ancestor, and only report it in the first commit on the branch.

Strangely, you have to look at (potentially) all commits on a branch to find the 'true' ancestor branch.

The problem is for branch-off-branch branches where the first commit on the new branch modifies only files never modified on the branch-off-HEAD branch. This is because cvs only REALLY creates the branch when the first commit is made (for that file) on the branch. Before that, it is just a 'potential' branch...

But I have code now which (seems to) works, but needs a bit more checking.

Yes.

David

Linus Torvalds wrote:

I've attached what I just committed. The previous 'show ancestor' patch needs to be reversed and this applied. It works for me on a half-dozen repos including syslinux.

You no longer should need to work around multiple reporting of the ancestor for a given branch, though it couldn't hurt.

I'm going to finish getting some of Thomas's patches in and make an actual release so people won't have to scour the lists.

David

---------------------
PatchSet 176 
Date: 2005/05/24 19:57:37
Author: david
Branch: HEAD
Tag: (none) 
Log:
show branch ancestry

Members: 
	cvsps.c:4.99->4.100 
	cvsps_types.h:4.9->4.10 

David Mansfield wrote:

Oops. I forgot to metion I made the tracking of branch ancestry an option because it potentially increases the cpu time a fair margin (though here it seemed trivial). You need to pass '-A' as an additional argument.

David

Linus Torvalds wrote:

Yes. While not falling asleep last night I realized that the quick-and-dirty approach was bogus. I need to track what the ancestor is as I'm building up the data structure, not while outputting it. So I'm working on a correct version which puts ancestor_branch into the PatchSet structure itself.

It's completely done now except for that it segfaults instantly.

BTW where did you get the cvsroot for syslinux? Could I get a copy somewhere?

David

On Tue, 24 May 2005, David Mansfield wrote:

Yes.

Very good. Are you going to also make a new release at some point, so that we don't have strange random patches floating around?

Peter sent it in private email, I don't know how public that is (it probably is perfectly public and he just didn't want to spam the mailing list or run afoul of size limits, but I just don't know for sure, so..), but I bet he'll happily send it to you too.

		Linus

Linus Torvalds wrote:

Already sent... I haven't looked it over to make sure there isn't anything that shouldn't be in there yet, so if you need to distribute it please give me a warning so I can look it over first.

	-hpa

Linus Torvalds wrote:

I'll take a look. One problem is that many folks use non-local cvs... not sure if that will be an issue. I'll look to cleaning this up if necessary.

I actually found an issue with the 30-second ancestor branch patch I sent and I'm doing that one properly now. Once that's done I can look at the branch capture logic in cvs2git and see if anything pops out.

Really cool.  What's 8 commits between friends?

David

Hello Linus, I tried cvs2git and have the following problem:

	---------------------
	PatchSet 26
	Date: 1998/06/20 03:53:44
	Author: roessler
	Branch: mutt-0-93
	Ancestor branch: HEAD
	Tag: (none)
	Log:
	documenting alias-path

	Members:
		doc/manual.sgml:1.2->1.2.2.1

	---------------------

And your script does that:

	export GIT_COMMITTER_NAME=roessler
	export GIT_COMMITTER_EMAIL=roessler
	export GIT_AUTHOR_NAME=roessler
	export GIT_AUTHOR_EMAIL=roessler
	export GIT_AUTHOR_DATE='1998/06/20 03:53:44'
	ln -sf refs/heads/'master' .git/HEAD
	git-read-tree -m HEAD
	git-checkout-cache -f -u -a
	mkdir -p doc
	co -p -r1.2.2.1 '/home/cip/adm/sithglan/work/mutt/cvsrepository/doc/Attic/manual.sgml,v' > 'doc/manual.sgml'
	git-update-cache --add -- 'doc/manual.sgml'
	tree=$(git-write-tree)
	cat > .cmitmsg <<EOFMSG
	documenting alias-path
	EOFMSG
	commit=$(cat .cmitmsg | git-commit-tree $tree -p HEAD)
	echo $commit > .git/HEAD

The problem might be that this is the first commit in the branch. But I thought it should end up in refs/heads/mutt-0-93. The problem is that this ends up a empty file and next time the script is working on it, it fails because the branch is empty:

	+ export GIT_COMMITTER_NAME=roessler
	+ GIT_COMMITTER_NAME=roessler
	+ export GIT_COMMITTER_EMAIL=roessler
	+ GIT_COMMITTER_EMAIL=roessler
	+ export GIT_AUTHOR_NAME=roessler
	+ GIT_AUTHOR_NAME=roessler
	+ export GIT_AUTHOR_EMAIL=roessler
	+ GIT_AUTHOR_EMAIL=roessler
	+ export 'GIT_AUTHOR_DATE=1998/06/20 07:12:32'
	+ GIT_AUTHOR_DATE=1998/06/20 07:12:32
	+ ln -sf refs/heads/mutt-0-93 .git/HEAD
	+ git-read-tree -m HEAD
	usage: git-read-tree (<sha> | -m <sha1> [<sha2> <sha3>])
	+ git-checkout-cache -f -u -a
	+ co -p -r1.1.1.1.2.2 /home/cip/adm/sithglan/work/mutt/cvsrepository/handler.c,v
	/home/cip/adm/sithglan/work/mutt/cvsrepository/handler.c,v  -->  standard output
	revision 1.1.1.1.2.2
	+ git-update-cache --add -- handler.c
	++ git-write-tree
	+ tree=9e4d085838e4e62a8c4236a6713a7dd8d7b07b4e
	+ cat
	++ cat .cmitmsg
	++ git-commit-tree 9e4d085838e4e62a8c4236a6713a7dd8d7b07b4e -p HEAD
	usage: git-commit-tree <sha1> [-p <sha1>]* < changelog
	+ commit=
	+ echo

	Thomas

On Tue, 24 May 2005, Thomas Glanzmann wrote:

Yes, you're using the cvs2git from yesterday, which didn't write the new commit to the right branch. This is part of the branch fixing I've done.

Wait another few minutes and I'll commit my fix the problem with cvsps branch handling (and I need to escape '$' in <<EOFMSG handling).

		Linus

On Tue, 24 May 2005, Linus Torvalds wrote:

Ok, committed. It takes a few minutes for the mirroring to pick it up, but you should soon see a commit that says

    cvs2git: escape <<EOF messages, and work around cvsps branch handling
    
    This escapes '$' characters in <<-handling, and gives preference to
    the new branch when cvsps incorrectly reports a commit as originating
    on an old branch.

and once you do, you should have something that works. 

Of course, I've still only tested it on syslinux, but it converts a syslinux CVS repo in 64 seconds for me, and now the result really _does_ look correct at least superficially. Ie I can see 1029 commits on HEAD, which is exactly what cvsps also reports.

And I see four different branches (HEAD is called "master" as per the normal naming):

	torvalds@ppc970:~/src/osscvs/syslinux> ll .git/refs/heads/
	total 16K
	-rw-rw-r--  1 torvalds torvalds 41 May 24 09:36 branch-1_xx
	-rw-rw-r--  1 torvalds torvalds 41 May 24 09:37 master
	-rw-rw-r--  1 torvalds torvalds 41 May 24 09:36 syslinux
	-rw-rw-r--  1 torvalds torvalds 41 May 24 09:36 syslinux-1_6x-1

and doing a 

	git-rev-tree branch-1_xx master syslinux syslinux-1_6x-1 | wc -l

reports 1046 total revisions (which also matches cvsps exactly).

So things look ok, but I haven't actually checked the _contents_ of the tree, except to look that the pathces that "git-whatchanged -p" reports look sane.

There's two remaining bad things:

 - name translation doesn't exist (so all of Peters changesets get 
   reported as author "hpa <hpa>")

 - the commit time will be the conversion time, not the original commit 
   time (but the _author_ time will be correct). I suspect that for a 
   conversion like this, we really should add support for GIT_COMMIT_DATE. 

   That would also make the archive conversion 100% reproducible, ie
   everybody should get the exact same objects (and thus the exact same
   SHA1 values) which is good.

I'll add the GIT_COMMIT_DATE thing, but the name translation is for somebody else.

		Linus

On Tue, 24 May 2005, Linus Torvalds wrote:

Done. I've also fixed the timezone to "+0000", so that it doesn't matter where you do the conversion, you should always get the same results (again, I just pushed that out, it might not have hit the public mirrors yet).

To get GIT_COMMITTER_DATE (note: COMMITTER, not COMMIT, to illogically match the name/email ones) you obviously also need a new git. So to have it all working right, you should have the top commits in git-tools and git be

    cvs2git: set timezone info to UTC, the way CVS does

and

    git-commit-tree: allow overriding of commit date

respectively.

And if this doesn't work for you, point me to the CVS archive that causes you trouble.

		Linus

Hello,

you should try the mutt cvs repository[1].

I have the following issues all seem easy to fix:

	- PatchSet 1 depends on PatchSet 2 (but cvsps gets the ordering wrong;
	  should be easy fixable) (I just swichted the two before
	  running cvs2git)

	- Some Shell escapes (I didn't looked into them yet)

		(faui02new) [/var/tmp/sithglan/mutt-cvs] bash ~/work/cvsps/sane
		defaulting to local storage area
		Committing initial tree 7e68fd9a5104b61192a7da7357549d95b3a0620c
		Ignoring path .cvsignore
		...
		/home/cip/adm/sithglan/work/cvsps/sane: line 1: ...: command not found
		/home/cip/adm/sithglan/work/cvsps/sane: command substitution: line 1: unexpected EOF while looking for matching `''
		/home/cip/adm/sithglan/work/cvsps/sane: command substitution: line 4: syntax error: unexpected end of file
		/home/cip/adm/sithglan/work/cvsps/sane: command substitution: line 1: unexpected EOF while looking for matching `''
		/home/cip/adm/sithglan/work/cvsps/sane: command substitution: line 5: syntax error: unexpected end of file
		/home/cip/adm/sithglan/work/cvsps/sane: command substitution: line 1: unexpected EOF while looking for matching `''
		/home/cip/adm/sithglan/work/cvsps/sane: command substitution: line 2: syntax error: unexpected end of file
		/home/cip/adm/sithglan/work/cvsps/sane: command substitution: line 1: unexpected EOF while looking for matching `''
		/home/cip/adm/sithglan/work/cvsps/sane: command substitution: line 3: syntax error: unexpected end of file
		/home/cip/adm/sithglan/work/cvsps/sane: command substitution: line 1: unexpected EOF while looking for matching `''
		/home/cip/adm/sithglan/work/cvsps/sane: command substitution: line 26: syntax error: unexpected end of file

But hey this looks really good: :-))))))

(faui02new) [/var/tmp/sithglan/mutt-cvs] git parent ~/work/mutt/git/mutt-cvs (faui02new) [/var/tmp/sithglan/mutt-cvs] git parentdiff (faui02new) [/var/tmp/sithglan/mutt-cvs]

I think I will run my 'import patch by patch script again' and check the changesets against the cvs2git tree, but it looks fine for me.

	Thomas

[1] To make it reproducable for you:

I used the attached patch against cvsps-2.0rc1 which fixes date covnersion problems and of course includes the ancestor thing.

rsync -r rsync://cvs.gnupg.org/mutt-cvs-rep mutt-cvs-rep

On Tue, 24 May 2005, Thomas Glanzmann wrote:

Ok, this seems to be a cvsps bug, and I'll treat it as such. David, any ideas? It seems to be because of how cvsps sorts things by date, which is obviously bogus.

The cvs2git thing wouldn't normally even _care_ (ie would happily re-order the thing), but for the fact that it causes problems with branches that are used before they are created in this case.

cvsps really should do some kind of topo-sort. Probably doesn't need a lot (ie it probably doesn't even need to be topological, but the "order" should be based on trivial dependencies first, and time second. For example, once David does the per-commit branch handling, I suspect enough of an ordering to keep git happy falls out of that).

Ok, I'll check it out. I didn't figure out what characters are shell-expanded by "<<EOF" handling, and only did '$' because that showed up in the syslinux archives.

In theory, they should give the exact same results, no? At least if there are no binary objects. Of course, you'd have to update your import script to do the times the same way.

		Linus

Linus Torvalds wrote:

Nothing is expanded when you quote the EOF-word:

cat <<"EOF" `foo` $PATH bar \ baz EOF

gives:

`foo` $PATH bar \ baz

Ciao, ET.

On Wed, 25 May 2005, Edgar Toernig wrote:

Oh, wow.

It's even documented in the bash man-page, now that I understand what to look for.

Anyway, that doc also tells me that I'm quoting the right characters, so now it's not worth fixing any more.

		Linus

LT> On Wed, 25 May 2005, Edgar Toernig wrote:

LT> Oh, wow.

Test scripts in your t/ uses <<\EOF all over the place.

On Tue, 24 May 2005, Junio C Hamano wrote:

Yeah, yeah, rub it in. I suck at shell.

The sad part is that I don't even have an excuse like "I use perl all the time". No, shell is my _forte_ when it comes to scripting.

		Linus "crawl under a rock" Torvalds

LT> The sad part is that I don't even have an excuse like "I use LT> perl all the time". No, shell is my _forte_ when it comes to LT> scripting.

You are allowed to suck at scripting when you are so good at C ;-).

By the by, are you still having problem with "whatchanged -s"? I see a few of my patches still not in your tree, and I do not care too much about the last one (rename/copy similarity estimator update) after you made it clear that -M and -C are of lower priority, I would like to know if the "diff-tree -s" fix I sent you was missing the point, and if so how. Also the test scripts to verify your fix to the "checkout-cache --prefix pointing at some directory via symlink" along with your fix is something I feel ready to go.

Hoping it would be easy to look for a message by message-ID, here they are. The first one is the proposed "diff-tree -s" fix.

Subject: Re: [PATCH 3/3] Diff overhaul, adding the other half...
Message-ID: <7vr7fxuh8b.fsf@assigned-by-dhcp.cox.net>

Subject: Squelch compiler warning
Message-ID: <7vis18v4ea.fsf_-_@assigned-by-dhcp.cox.net>

Subject: [PATCH] Allow symlinks in the leading path in checkout-cache --prefix=
Message-ID: <7vacmlvwfk.fsf_-_@assigned-by-dhcp.cox.net>

Subject: [PATCH] Update rename/copy similarity estimator.
Message-ID: <7vbr70v3tf.fsf@assigned-by-dhcp.cox.net>

Thomas Glanzmann wrote:

Sounds good. I'll give it a try. I'm testing the branch ancestor logic, which seems to be working better now. The version I sent to the list yesterday was pretty bogus for some cases, as well as reporting the ancestor multiple times for any give branch.

There is something strange about 'cvs import' I believe which causes various bizarre things to happen to the first cvsps patchset. I haven't looked at mutt cvs yet, but this could be the cause. If you see a lot of version numbers 1.1.1.1 then this is indeed the problem.

I'll look at taking these patches upstream. The 'MT' fix is already in my cvs of cvsps, and the rest looks pretty good.

Do you know where I can get attribution information for these changes? Are they all from you? (I'm not familiar with debian at all)

David

Hello,

yes, that is happening. But it should be fairly easy to fix that. Because the second one says INITIAL->1.1 and the first says 1.1->1.1.1.1 a lot.

Good. :-)

none of them is from me, they're all from Debian. Here are a few URLs how to get the attribution:

http://packages.qa.debian.org/c/cvsps.html
	-> You can get the source files from there: DSC are
	metainformation, ORIG is your upstream version and DIFF are the
	debian changes against your upstream version as patch. There is
	also a Changelog in the diff for the package. You should find in
	there everything you need.

http://bugs.debian.org/cgi-bin/pkgreport.cgi?pkg=cvsps
	-> The bug tracking system of debian could also be of help.

	Thomas

On Tue, 24 May 2005, Thomas Glanzmann wrote:

Ok, that's a lot bigger and slower than syslinux. It seems to be importing about 9.5 changesets per second, and there's 3757 patchsets, so it looks like about 6 minutes.

Oh, done.

And yes, there's a few problems. It seems to be the fault of a frowning "smiley" - the '\' followed by newline in this:

	[unstable] Re-add in-reply-to.  This time with a suitable default. #-\

and one back-tick.

Will fix. This will take another six minutes of testing ;)

		Linus

On Tue, 24 May 2005, Linus Torvalds wrote:

Almost eight minutes. Still, the final average was 8 changesets per second, which sounds pretty damn good to me, actually.

Anyway, I've checked in the fix for the quoting, and I now get the right number of revisions, ie

	git-rev-tree $(ls .git/refs/heads/) | wc -l

returns the same "3757" that cvsps reports. 

However, "git-fsck-cache --unreachable" reports 102 unreachable blobs, which worries me. It's really blobs only, which is strange: it implies that we did the "git-update-cache" but not a "git-write-tree" (or that the git-write-tree failed for some reason, but that sounds even stranger, since we did successfully do all the commits)

The only way I can see the unreachable blobs happening is if one of he ChangeSet entries in cvsps mentions the _same_ pathname twice for a single ChangeSet. David, is that possible?

Exactly because it's only blobs, it really does smell like a cvsps issue. My scripts always use "git-update-cache --add -- filename", so it never creates any blobs _except_ when it adds them to the index (and thus write-tree should always pick them up, unless we update the index again before the next write-tree happens).

			Linus

Linus Torvalds wrote:

Sounds possible. Unfortunately, the 'uniqueness' of a commit actually doesn't exist. It's all smoke-and-mirrors. In order to disallow this (which I think need to do) I'd need to use some commit member information, and add some heuristic: if this file is already in the commit, then this MUST be a different commit. Unfortunately, it's possible that the 'member' already in the commit is the wrong one and this is the right one, which just sounds horribly ugly to me.

I'll think on it.

David

On Tue, 24 May 2005, David Mansfield wrote:

I think it's a fundamentally hard problem to fix, but it may be that the fix is to give hints about command line options and in particular the time fuzz thing to try.

So maybe just _detection_ logic in cvsps, along with a warning like

	"time fuzz is 600 seconds, and the time difference between the two
	 commits of this file was 431 seconds. You may want to try a lower
	 -z argument"

or something.

It might also be possible to try to sort all the names by date of commit first, and see if they "bunch up" into groups of low fuzz with much bigger fuzz in between groups..

		Linus

Hello,

yes, it is. ;-)

Nice! :-)

btw:

For the mutt tree there are a few 'empty commits' eg were the parent tree is the same as the current. This is because git ignores .cvsignore and they commited some .cvsignore files without any other deltas. I don't know if you want to handle this. Just a note.

Yes, it is, I had that problem before. For example tlr commtis the changelog seperate with '# changlog' or so log message and cvsps thinks because of the 'time fuzz which defaults to a way to high value' that three changelog commits are all one delta. And that it adds three entries. And what annoys me most in the wrong direction. So if you would apply them as patches they don't apply because of the wrong ordering.

Reference:
		PatchSet 3005
		Date: 2002/12/07 19:19:42
		Author: roessler
		Branch: HEAD
		Tag: (none)
		Log:
		# changelog commit

		Members:
			ChangeLog:3.7->3.8
			ChangeLog:3.6->3.7
			ChangeLog:3.5->3.6

Just call cvsps with -z "20" for the mutt repository also -z 1 should work because the timestamps of one 'commit' are all set to the same value.

	Thomas

On Tue, 24 May 2005, Thomas Glanzmann wrote:

Ahh, the mutt people really use something else for development, and this is just an export into CVS (like the Linux bkcvs tree)? Or do they just have fast machines and no networking? Or are there good versions of CVS around that re-use the same time across one whole commit?

		Linus

Hello,

I did one sampling and though it would be representative which it isn't. What I don't understand why noone ever fixed this? cvs has its own rcs implementation anyway to speed things up, hasn't it?

	Thomas

On Tue, 24 May 2005, Thomas Glanzmann wrote:

CVS has so many warts, that people can't even be bothered to fix things like this. It's file-based, and that's that.

Using "-z 1" with cvsps doesn't seem to work well for me, but "-z 5" seems ok, and together with the new git that allows .cvsignore, it doesn't generate any warnings, nor any unreachable blobs. It does generate 300 new changesets, and how many of those are required, I dunno. Clearly 102 of them were, to disambiguate those changelog things.

Maybe "-z 10" would have generated a better thing with fewer changes yet still unique changesets without dup files.

That's a bit irritating, that there's this magic tweakable that generates different trees. Oh, well.

Anyway, what worries me more is that cvsps might have re-ordered other changesets than just the first two. It probably doesn't _matter_, but still...

		Linus

I think I have an idea to prevent this incorrect ordering of patchsets. 
  We'll see in the morning how it works out.

David

On Tue, 24 May 2005, Thomas Glanzmann wrote:

I don't like source repositories with dot-files, and I thought it was a good idea to disallow them, but on the other hand I'd like it even less if some CVS-weenie goes and says "I can't convert my project to git without potentially losing information".

So in the name of furthering humanity through allowing people to migrate away from CVS, I'm considering making the git dot-file check be more specific to "." ".." and ".git". After all, project-specific rules might have their own porcelain-related ignore-files that cause dot-files to never appear..

Hmm.

		Linus

Linus Torvalds wrote:

There is another good reason to allow dot files: since git handles sparse trees well, and only needs metadata at the top level, it's pretty ideal for keeping track of people's account profiles across accounts, and that's mostly dot files.

I was considering using CVS for that at one point, but the way CVS distributes its metadata and recurses makes that insane. For git, it would be trivial.

	-hpa

Hello,

I have a shared environment, and handle it this way:

	- I edit all files on my machine on university or at least put
	  it there per scp when I am done modifing them local.

	- I have file called .env in my home which contains a list with
	  all files I want to 'distribute'.

	- There is a script called envup, which updates all files which
	  are not up2date. It uses perl because perl is everywhere I
	  have an account on. After that I call a perl preprocessor to
	  generate ssh config files (there are floating so much ssh
	  versions out there) and one to adopt my .screenrc (for the
	  machine local screen) to my needs.

my .bash_profile does a lot of testing to add paths and sets aliases based on the environment of the machine and not based on host/domain names. Files with lot of deltas like my .fvwm2rc or .bash_profile are all version controled using rcs on a per file basis of course. This works very well for me on Linux, Solaris, OpenBSD, FreeBSD, NetBSD, OSF, AIX, MacOSX, name it ...

Oh and to bootstrap my environment on a 'new' machine:

tar cfz env.tgz `cat .env`, move it over, unpack and re login.

A while ago I also distributed binaries (bash, rar, fvwm) via envup for my environment, but I adopted now to something that works everywhere, and if it doesn't I do that manual. So envup only transfers stuff which is new. (I have some accounts on low bandwidth sites). One known side effect. If envup doesn't find md5sum binary it just transfers all files.

Oh and a friend and colleague has a similliar script only that he pushes his environment out.

	Thomas

#!/usr/bin/perl -w

use strict; use integer;

my $command  = '';

my %local = (); my %remote = ();

my @sums = (); my @transfer = ();

if ( -f '/home/cip/adm/sithglan/bin/envup.server'
  || -f "$ENV{HOME}/bin/envup.server" ) {
	die "NEVER EVER RUN THIS IN THE CIP POOL: YOU SUCK :-)";
	exit; # way to secureredundant
}

# [remote] ------------------------------------------ @sums = `ssh 131.188.30.105 bin/envup.server`;

foreach $_ (@sums) {
	chomp;
	if (/(^[a-f0-9]{32})[ ]{2}([^\s]+)$/) {
		$remote{$2} = $1;
		# print "$1 -> $2\n";
	} else {
		print "remote: can't match: <$_>\n";
	}
}

# [local] ------------------------------------------

$command = 'md5sum ' . join(' ', keys(%remote));

@sums = qx($command 2> /dev/null);

foreach $_ (@sums) {
	chomp;
	if (/(^[a-f0-9]{32})[ ]{2}([^\s]+)$/) {
		$local{$2} = $1;
		#print "$1 -> $2\n";

	} else {
		print "local can't match: <$_>\n";
	}
}

foreach my $file (keys(%remote)) {
	if (exists($local{$file})) {
		if ($remote{$file} ne $local{$file}) {
			push(@transfer, $file);
		}

	} else {
		push(@transfer, $file);
	}
}

if (@transfer) {
        $command = "ssh 131.188.30.105 -- tar cf - " . join(' ', @transfer) . " | tar xf -";
        system($command);
}

#!/bin/bash

md5sum $(find `cat $HOME/.env` -type f)

Hello,

Allowing dot files is a good thing, I think. But there is another issue I want to hear your comment on. My git frontend uses regular expressions from .git/ignore to filter orphan files I know about when calling 'git status' or 'git orhpan'. However, I thought to add this file to versioning. Because I think it belongs there. However I have to think that through.

So if there would be a .git/etc or something were I can put files which stay under revision control and maybe add some options to the frontend or git core to don't diff against them per default. But this sounds just ugly. Maybe I should just retrieve the ignore list when doing push/pulls. But I dislike this idea to. However I have to think a bit longer over this.

Comments, anyone?

	Thomas

On Tue, May 24, 2005 at 02:13:26PM -0700, Linus Torvalds wrote:

What about allowing to put some file inside of .git/ under revision-control too? Wouldn't it be nice to have something like an "ignore" file or other repository meta-data managed by git itself.

Kay

On Tue, 24 May 2005, Kay Sievers wrote:

Put them into ".git-ignore" if so..

		Linus

I was browsing www.kernel.org/git and noticed that it shows only files that exist at the tip. How do I get history of a file that does not exist anymore at the tip?

For example, diff-helper.c history is (quite correctly) truncated somewhere close to where diff-tree-helper.c was renamed to it. From the commit log, humans can easily tell that it used to be called diff-tree-helper.c. I could not find an easy way to see the history of diff-tree-helper.c file.

On probably a bit different topic (but I do not know who is updating the copy on www.kernel.org, sorry). Could somebody update http://www.kernel.org/pub/software/scm/git/docs/ to rename git-diff-tree-helper to git-diff-helper please? git.git/Documentation/git.txt has been corrected quite some time ago [*1*] and I do not know how the updates are propagated to the web version of the documentation; is it a manual process?

[Footnotes]

*1* The following command tells me it was done about 10 days ago.

    git-rev-list 0 |
    git-diff-tree -S'diff-tree-helper' --stdin -v -p Documentation/git.txt

On Tue, 24 May 2005, Junio C Hamano wrote:

The only sane interface I can think of is to expose the subdirectory history and then pick from that. Otherwise you'd have to actually type in the name, which is a bit against the notion of a graphical browsing interface.

		Linus

LT> The only sane interface I can think of is to expose the subdirectory LT> history and then pick from that. Otherwise you'd have to actually type in LT> the name, which is a bit against the notion of a graphical browsing LT> interface.

Knowing to type "merge-tree.c" you need to be an old timer ;-).

Since I asked that question I found out that each commit has a link to the diff and the tree, so if I know when merge-tree.c disappeared, I can go backwards from there.

I think what is useful, from software archaeologist point of
view, would be to give a way to the web users to use pickaxe.
Type piece of code in the textbox and have the CGI run "rev-log
| diff-tree -S'<that piece of the code>'".

On Tue, May 24, 2005 at 07:23:39PM -0700, Junio C Hamano wrote:

I will add a search function when the git binaries on kernel.org get an update. Currently it's cogito-0.10.

David Greaves can write to that directory. David?

Kay

Yep. It is a manual rsync process.

I last ran it on the 22nd as soon as Linus committed my latest doc patches.

It's my bad though - I didn't update index.html which should be a copy of git.html

 http://www.kernel.org/pub/software/scm/git/docs/git.html
is correct.

Anyway, done now, give it time to replicate.

David

DG> Anyway, done now, give it time to replicate.

Thanks. Would something like the following (1) easy to arrange and (2) make your life easier?

    - make index.html symlink to git.html
    - have cron job to build and install those pages at
      www.kernel.org

Junio C Hamano wrote:

Hmm - sounds complicated ;)

On Tue, May 24, 2005 at 07:23:39PM -0700, Junio C Hamano wrote:

If git-diff-tree is given the -M:
  git-rev-list HEAD | git-diff-tree -r -M --stdin diff-helper.c

and it would print:
  99665af5c0be0fe4319b39183e84917993153576 (from 13ab4462d2aefb252d7c916bd537151856b7c967)
  :100644 100644 51bb658be4f73c00016b4ecb82f09d30941998a4 51bb658be4f73c00016b4ecb82f09d30941998a4 R10000 diff-tree-helper.c diff-helper.c

instead of:
  99665af5c0be0fe4319b39183e84917993153576 (from 13ab4462d2aefb252d7c916bd537151856b7c967)
  :000000 100644 0000000000000000000000000000000000000000 51bb658be4f73c00016b4ecb82f09d30941998a4 N      diff-helper.c

gitweb could follow the old filename and show the whole history. :)

Thanks, Kay

On Wed, May 25, 2005 at 02:35:44PM +0200, Kay Sievers wrote:

                                                                                                      ^^^^^
Btw:
Can we add a leading '0' to have 3-digit values every time? It's pretty
stupid to get the '100' out ot that field instead of '10'. :)

Kay

Stupidly, I use "%d" without precision to output score, and worse yet the score is not scaled (meaning it would change when definition of MAX_SCORE changes, exposing internal scale that rename/copy uses).

For the sake of parsability, I think I should make that number always 3 digit and per-cent; so 86% hit would be "C086" and the bogus R10000 you pointed out would become "R100".

KS> gitweb could follow the old filename and show the whole history. :)

Yes, it would work for diff-helper.c, but not for merge-tree.c, which does not have any equivalent in the tip of the tree.

On Tue, 24 May 2005, Linus Torvalds wrote:

Looking at the contents of these files, all but one of them are changelog files, which would be consistent with this theory - if gitps ends up "smushing together" two separate commits (and mutt seems to have the bad habit of having just a simple "# changelog commit" as the commit message, so it would likely trigger the "same commit message" logic), you'd get exactly this.

The one non-changelog file looks like some kind of message translation thing:

	# This file was prepared by (in alphabetical order):
	#
	#   Alexey Vyskubov (alexey@pepper.spb.ru)
	#   Andrew W. Nosenko (awn@bcs.zp.ua)
	#   Michael Sobolev (mss@transas.com)
	#   Vsevolod Volkov (vvv@mutt.org.ua)
	#
	# To contact translators, please use mutt-ru mailing list:
	#   http://woe.spb.ru/mailman/listinfo/mutt-ru
	#
	msgid ""
	msgstr ""
	"Project-Id-Version: mutt-1.4i\n"
	"POT-Creation-Date: 2002-05-02 01:08+0200\n"
	"PO-Revision-Date: 2002-05-03 22:53+0300\n"

	...

	#: alias.c:280
	#, c-format
	msgid "[%s = %s] Accept?"
	msgstr "[%s = %s] ðÒÉÎÑÔØ?"

	...

and it looks like it is "po/ru.po". Indeed, that's a big clue:

	---------------------
	PatchSet 2869 
	Date: 2002/05/13 21:17:48
	Author: roessler
	Branch: mutt-1-4-stable
	Tag: (none) 
	Log:
	From: Vsevolod Volkov <vvv@mutt.org.ua>
	
	update
	
	Members: 
	        po/ru.po:1.129.2.5->1.129.2.6 
	        po/ru.po:1.129.2.4->1.129.2.5 
	
	---------------------

and I thus rest my case. cvs2git is doing the right thing, and this is something that needs to be fixed in cvsps in case anybody cares.

		Linus

Hello,

I pick that up.

Thanks. I wanted to send patches or at least ask you this for ages, but never did. :-)

	Thomas

On Tue, 24 May 2005, Thomas Glanzmann wrote:

Note that one advantage of the unconverted output is that while it's unreadable and not very helpful, it _is_ the raw output from CVS. Again, that means that everybody will convert the same CVS archive into exactly the same git tree, and that means (among other things) that you can then immediately merge between the trees.

Perhaps more interestingly, it should also mean that you can _continue_ to use CVS, then re-convert it at a later date, and I think you should be able to merge with somebody who has been using git in the meantime.

In contrast, if you have fancy name translation, the converted tree will obviously depend on your translation rules.

I dunno. Maybe the advantages of having nice names outweigh the disadvantage of possibly generating incompatible trees.

			Linus

Hello,

I see your point. But it depends on the usage scenario. For me for example I would like to vendortrack a few CVS repositories. And I use it only to maintain a few patches (branches and the merging facilities of git come handy in here), not in a distributed environment were I need that much reproducability. So having the option to use them is fine for me and when I need reproducability than I simple don't. When I think about this scenario ... it comes in my mind that it maybe would be helpful to have a helper applications like git-merge-base which looks at the treeids and not the commit ids to find the merge base, or is that just bullshit?

	Thomas

LT> Perhaps more interestingly, it should also mean that you can _continue_ to LT> use CVS, then re-convert it at a later date, and I think you should be LT> able to merge with somebody who has been using git in the meantime.

LT> In contrast, if you have fancy name translation, the converted tree will LT> obviously depend on your translation rules.

LT> I dunno. Maybe the advantages of having nice names outweigh the LT> disadvantage of possibly generating incompatible trees.

LT> 			Linus

Porcelain layers should be capable of mapping author/committer names taken out of the commit object, just like they already convert the human unreadable unixtime value into something human readable. I'd vote for keeping the original value taken from CVS for this particular "conversion" application.

What _all_ Porcelain layer implementation would benefit from is if we had a common output format routine that is similar to the spirit of show_date() function. Have format_commit_fancy() function that takes a commit object and have it do the mapping. Then everybody can use it for their own Porcelain.

The diff-tree header generation can use it when (and only when) it is operating under a new flag (--map-author-names), to prettyprint the author names. I'd also suggest to have a flag to reduce the prettyprinting it does in the current output (like omitting committer information) to make its output be usable for reproducing the commit history exactly. diff-tree with recent enhancement you did (I am talking about single commit output and --stdin, not my diffcore stuff) has become quite useful tool for this kind of thing.

JCH> What _all_ Porcelain layer implementation would benefit from is JCH> if we had a common output format routine that is similar to the JCH> spirit of show_date() function. Have format_commit_fancy() JCH> function that takes a commit object and have it do the mapping.

Here is a small script called "whodunnit.sh", and its output can be cleaned up if we had a git-format-commit command that used format_commit_fancy(), that massages author/committer names (and probably some other prettyprinting), instead of plain old "git-cat-file commit".

#!/bin/sh
git-rev-list ${1:-HEAD} |
while read commit
do
	git-cat-file commit $commit |
	sed -ne '/^author \([^>]*>\).*/{s//\1/p;q;}'
done | sort | uniq -c | sort -n

The lines it currently spits out looks like this:

      6 ...
      8 Linus Torvalds <torvalds@ppc970.osdl.org.(none)>
        ...
    155 ...
    229 Linus Torvalds <torvalds@ppc970.osdl.org>

My suggestion for Thomas is not to volunteer changing cvsps-to-git to munge names at conversion time, but instead to volunteer doing the format_commit_fancy() on the core-ish side. It would read from $GIT_DIR/author-names which would be a plain text file that is a sequence of:

    "bogus name" TAB "good name" LF

where you would put ".(none)" version to "bogus" side and the corrected one on the "good" side.

On 5/24/05, David Mansfield <david@cobite.com> wrote:

Projects that branch on release (and maintain a long-lived stable branch following the release) often use a floating MERGED branch to keep track of what bugfixes have been merged back into HEAD. This practice, broken as it is, is the recommended approach AFAIK.

It would be a good thing to be able to tell cvsps to ignore certain tags (by name or by regex).

martin

Hello,

actually I think this is the broken upstream version. It can't parse
dates right. Just look at the exported patches and see if them all from
1970. However the debian package has a patch in which solves it:

maybe you should try with the attached patch or with the version that comes with debian sarge. I also reported this problem a while back to the original author.

	Thomas

--- cvsps-2.0rc1.orig/util.c +++ cvsps-2.0rc1/util.c

Thomas Glanzmann wrote:

I was about to apply this and I already had in it my cvs tree! Funny how these things go. I must have gotten it before, applied it and never released a new version. Funny that this one hase the tm.tm_isdst = 0 that is missing from the version I applied (and fixes an important bug).

Anyway, I'm about to release a new version cumulative with all this, a fixed ancestor version, correct ordering for those pesky import commits, and a couple other annoying fixes.

BTW: the above warnings are actually legit in this case.

David