Errors GITtifying GCC and Binutils

Hi!

I'm currently working a lot with Binutils and GCC and wanted to import those two projects into GIT trees, but both of them failed. If anybody wants to have access to the half-finished GIT trees, please let me know:

This is with GIT as of yesterday evening:

Binutils ~~~~~~~~ $ /home/jbglaw/bin/git cvsimport -v -d :pserver:anoncvs@sourceware.org:/cvs/src -C src src [...] Update gdb/testsuite/ChangeLog: 230971 bytes Fetching gdb/testsuite/gdb.base/annota1.exp v 1.1.1.2 Update gdb/testsuite/gdb.base/annota1.exp: 14656 bytes Fetching gdb/testsuite/gdb.base/annota2.cc v 1.1.1.3 Update gdb/testsuite/gdb.base/annota2.cc: 238 bytes Fetching gdb/testsuite/gdb.base/annota2.exp v 1.1.1.2 Update gdb/testsuite/gdb.base/annota2.exp: 8639 bytes Fetching sim/mcore/ChangeLog v 1.1.1.5 Update sim/mcore/ChangeLog: 1325 bytes Fetching sim/mcore/interp.c v 1.1.1.5 Update sim/mcore/interp.c: 47368 bytes Tree ID 640f8ff80e14257bc73fa0c3344504c4db960655 Parent ID ac9471abd876476724dc0205272b0565c18b1a7c Committed patch 102 (SNAPSHOT 1999-05-25 18:00:33) Commit ID 5847cc095ce0445eb6bf9df1203ccdad35f501fe Created tag 'gdb-1999-05-25' on 'SNAPSHOT' Switching from SNAPSHOT to origin Fetching bfd/ChangeLog v 1.19 Update bfd/ChangeLog: 118751 bytes Fetching bfd/elf32-arm.h v 1.4 Update bfd/elf32-arm.h: 90491 bytes Tree ID 5100b5c49ef78e1027b476f4dbdebeca0b01f113 Parent ID 8d35c43246f8d9693203ef6a87aabf378593bceb Committed patch 103 (origin 1999-05-26 08:27:37) Commit ID bb18137d43506ccc7f78da63b18d9f6b6749264d Fetching include/opcode/ChangeLog v 1.4 Update include/opcode/ChangeLog: 63395 bytes Fetching include/opcode/hppa.h v 1.2 Update include/opcode/hppa.h: 23271 bytes Tree ID 7b211af5cb99cbe307d02d72c1e144f3c1aac7e9 Parent ID bb18137d43506ccc7f78da63b18d9f6b6749264d Committed patch 104 (origin 1999-05-26 16:04:10) Commit ID 4da3ef4d311fc62a1a0bba8b680595ad824a7e2c Fetching ld/ChangeLog v 1.11 Update ld/ChangeLog: 328968 bytes Fetching ld/emulparams/armelf_oabi.sh v 1.2 Update ld/emulparams/armelf_oabi.sh: 585 bytes Tree ID fcc05a0f9c66693e5b26a5fb4821f6bbeadf4c31 Parent ID 4da3ef4d311fc62a1a0bba8b680595ad824a7e2c Committed patch 105 (origin 1999-05-26 17:23:31) Commit ID ddaaceeede7f9bb938188c4f72ddce420d08efad Switching from origin to gdb-4_18-branch usage: git-read-tree (<sha> | -m [--aggressive] [-u | -i] <sha1> [<sha2> [<sha3>]]) read-tree failed: 33024

$ /home/jbglaw/bin/git cvsimport -v -d :pserver:anoncvs@sourceware.org:/cvs/src -C src src usage: git-read-tree (<sha> | -m [--aggressive] [-u | -i] <sha1> [<sha2> [<sha3>]]) read-tree failed: 33024

GCC
~~~
$ /home/jbglaw/bin/git svnimport -C gcc -v svn://gcc.gnu.org/svn/gcc
[...]
... 3930 trunk/gcc/combine.c ...
Tree ID 2af6a2f6e28835fec0aaf60d278e356d12d9ae5f
Merge parent branch: 587bbd00ea5b0f8c18d8ca58bbfcaa4b6b62ff92
Committed change 3930:/ 1993-03-30 20:37:29)
Commit ID 3ed4cebd349a2c98c7a06d6333b014bd4fe23d6d
Writing to refs/heads/origin
DONE: 3930 origin 3ed4cebd349a2c98c7a06d6333b014bd4fe23d6d
... 3931 trunk/gcc/config/mips/mips.h ...
Tree ID f173f836adf4d8efb42c180d7cd5a786a3acf361
Merge parent branch: 3ed4cebd349a2c98c7a06d6333b014bd4fe23d6d
Committed change 3931:/ 1993-03-30 21:50:50)
Commit ID 7c132f1464d27bd8198c51913b17a0534376e3e6
Writing to refs/heads/origin
DONE: 3931 origin 7c132f1464d27bd8198c51913b17a0534376e3e6
... 3932 trunk/gcc/config/pa/pa.md ...
Tree ID a88f5523f982dbac1334b78c47fd56a685640a46
Merge parent branch: 7c132f1464d27bd8198c51913b17a0534376e3e6
Committed change 3932:/ 1993-03-30 22:02:05)
Commit ID 9a8ddfa77f6735c8ba87ab98bc3ac7b056a09d96
Writing to refs/heads/origin
DONE: 3932 origin 9a8ddfa77f6735c8ba87ab98bc3ac7b056a09d96
... 3933 trunk/gcc/real.c ...
Tree ID 7d25a64013b301e0e575587a5adfd8b540b4c5a7
Merge parent branch: 9a8ddfa77f6735c8ba87ab98bc3ac7b056a09d96
Committed change 3933:/ 1993-03-31 05:30:24)
Commit ID 70cc7d27f3a4e5117df4e4d2f6e5a2aa4c4a89a6
Writing to refs/heads/origin
DONE: 3933 origin 70cc7d27f3a4e5117df4e4d2f6e5a2aa4c4a89a6
... 3934 trunk/gcc/real.h ...
Tree ID fce52e724f0f1543abd5bd430a11822c3977e803
Merge parent branch: 70cc7d27f3a4e5117df4e4d2f6e5a2aa4c4a89a6
Committed change 3934:/ 1993-03-31 05:39:37)
Commit ID f8dad6fb5af69122823800ca974e16d4f09906c2
Writing to refs/heads/origin
DONE: 3934 origin f8dad6fb5af69122823800ca974e16d4f09906c2
... 3935 trunk/gcc/Makefile.in ...
Tree ID 7c2cd41d922407ab1df47954fb82964ad7511328
Merge parent branch: f8dad6fb5af69122823800ca974e16d4f09906c2
Committed change 3935:/ 1993-03-31 05:41:37)
Commit ID eb99aa0ad37564c2071fe4b2f539fe072a72ad4c
Writing to refs/heads/origin
DONE: 3935 origin eb99aa0ad37564c2071fe4b2f539fe072a72ad4c
... 3936 trunk/gcc/c-lex.c ...
Tree ID 90547e4ef900e45f4354aa036f216a3bff93c8dd
Merge parent branch: eb99aa0ad37564c2071fe4b2f539fe072a72ad4c
Committed change 3936:/ 1993-03-31 05:44:03)
Commit ID ceff85145f8671fb2a9d826a761cedc2a507bd1e
Writing to refs/heads/origin
DONE: 3936 origin ceff85145f8671fb2a9d826a761cedc2a507bd1e
... 3937 trunk/gcc/final.c ...
Can't fork at /home/jbglaw/bin/git-svnimport line 379.

$ /home/jbglaw/bin/git svnimport -C gcc -v svn://gcc.gnu.org/svn/gcc Use of uninitialized value in system at /home/jbglaw/bin/git-svnimport line 295. usage: git-read-tree (<sha> | -m [--aggressive] [-u | -i] <sha1> [<sha2> [<sha3>]]) read-tree failed: 33024

MfG, JBG

On Wed, 22 Mar 2006, Jan-Benedict Glaw wrote:

Well, I can re-create your error..

That's a horrible error message, and the reason for it is that no "gdb-4_18-branch" exists.

There's a _tag_ called "gdb-4_18", and there's a branch called "GDB_4_18", and they actually point to two _different_ commits (both "Initial creation of sourceware repository"). The commits actually have identical trees, but they're different, because they have different times - by 50 seconds.

Gaah.

Looking at cvsps output (from

	cvsps --norc -u -A -v -d --cvsdirect
		--root :pserver:anoncvs@sourceware.org:/cvs/src
		src > cvsps.out 2> cvsps.err

it's "PatchSet 104" (well, for me it is, I have a hacked cvsps, so it might not be that for you), which creates the "gdb-4_18-branch", but it appears that cvsps hasn't actually figured out any "Ancestor branch" for that commit.

What a crock.

Anyway, it's clearly a cvsps bug (mentioning a new branch without the _source_ of that branch). Equally clearly, "git cvsimport" is being an ass about then failing so totally on it.

I'll try to take a look at why cvsps does that.

		Linus

On Wed, 22 Mar 2006, Linus Torvalds wrote:

It _seems_ that the reason for that is that cvsps considers a revision number of 1.1.1.1 to have a "dot depth" of 0, for some really strange reason (it's a total special case).

And that will currently not compare as a "greater" dot depth than not having any revision number at all for the ancestor, so such a revision will never be considered an ancestor branch.

This one-liner to cvsps.c seems to make sure we have an ancestor branch for that "gdb-4.18-branch" branch, at least according to the cvsps output. I'm re-running "git cvsimport" with this cvsps to see if it gets us past that one point, but I need to go pick up Patricia from school, so I won't have time to actually check the result. If somebody wants to play with this, go wild.

(The point of this patch is to make sure that if the head PatchSet doesn't have an ancestor, we'll consider _any_ valid ancestor to be better than that).

		Linus

---

On Wed, 22 Mar 2006, Linus Torvalds wrote:

The "git cvsimport" is still running, but at least it seems to be happily running further past the point it broke earlier.

Will send the patch over to David Mansfield.

		Linus

On Wed, 2006-03-22 17:28:23 -0800, Linus Torvalds <torvalds@osdl.org> wrote:

I've started it once again, too, with the one-liner added to Debian unstable's version of cvsps:

Fetching gas/ChangeLog v 1.479 Update gas/ChangeLog: 250329 bytes Tree ID a6b48ebac02a4158d37bab17c54c667223ecd971 Parent ID 4cabd2962031fd7ec6416580d84fb30a304969f3 Committed patch 3742 (origin 2000-07-29 03:23:31) Commit ID 1910c20a44455db916a5c040663716a7389219bc Fetching winsup/cygwin/fhandler.h v 1.16 Update winsup/cygwin/fhandler.h: 25992 bytes Fetching winsup/cygwin/include/cygwin/cygwin_dll.h v 1.2 Update winsup/cygwin/include/cygwin/cygwin_dll.h: 3050 bytes Fetching winsup/cygwin/lib/cygwin_crt0.c v 1.5 Update winsup/cygwin/lib/cygwin_crt0.c: 926 bytes Tree ID 0c2c7e9d0846e5f42b0bebad8b27ce439ddefb73 Parent ID 1910c20a44455db916a5c040663716a7389219bc Committed patch 3743 (origin 2000-07-29 04:19:24) Commit ID a15aac16f12061fbfef1be8f21b80a5076c8d605 Fetching winsup/cygwin/ChangeLog v 1.235 Update winsup/cygwin/ChangeLog: 75391 bytes Fetching winsup/cygwin/dtable.cc v 1.11 Update winsup/cygwin/dtable.cc: 14399 bytes Fetching winsup/cygwin/environ.cc v 1.17 Update winsup/cygwin/environ.cc: 17190 bytes Fetching winsup/cygwin/winsup.h v 1.22 Update winsup/cygwin/winsup.h: 15828 bytes Tree ID f777977c2b138952bc5a9bc431eec3de99a5f7db Parent ID a15aac16f12061fbfef1be8f21b80a5076c8d605 Committed patch 3744 (origin 2000-07-29 16:01:23) Commit ID 38b0ed94ef1c402b7b78ac6cad6c89ce189cd223 Switching from origin to #CVSPS_NO_BRANCH usage: git-read-tree (<sha> | -m [--aggressive] [-u | -i] <sha1> [<sha2> [<sha3>]]) read-tree failed: 33024

It seems there's a patch like http://www.gelato.unsw.edu.au/archives/git/0602/16278.html is missing? ...or we need a better cvsps. Shall I add it and try again / try to continue, or give up on it for now? Though it would be nice to have these two large and important source trees under GIT control :-)

MfG, JBG

On Thu, 23 Mar 2006, Jan-Benedict Glaw wrote:

Ahh. I had some other changes to my cvsps, so I never saw this. I just turned the #CVSPS_NO_BRANCH thing into HEAD.

		Linus

On Thu, Mar 23, 2006 at 09:03:06PM +0100, Jan-Benedict Glaw wrote:

You make want to try the cvsps patch I attached to the email here:

http://www.gelato.unsw.edu.au/archives/git/0511/11812.html

Good luck!

-chris

On Thu, 2006-03-23 at 19:39 -0500, Chris Shoemaker wrote:

I'm busy writing a new import tool to get X into git; I've got it generating complete revision trees in memory, and dumping them in graphviz form. I'd sure be interested to see how well this works with other ancient and broken CVS trees. Once I've got it dealing with current X.org CVS correctly (or, at least, reasonably), I'll finish it up and get it to actually generate the repository.

git://git.freedesktop.org/~keithp/parsecvs

Usage is completely lame at present -- a list of ,v file names either on the command line or via stdin (one name per line). The .dot file is output on stdout, with some diagnostics on stderr.

Pipe this through 'dot -Tsvg' and you'll get a .svg file which can be viewed with inkscape. They're generally immense...

On Thu, 2006-03-23 19:39:44 -0500, Chris Shoemaker <c.shoemaker@cox.net> wrote:

[...] cvs rlog: Logging src/winsup/w32api/include/ddk cvs rlog: Logging src/winsup/w32api/include/directx cvs rlog: Logging src/winsup/w32api/lib cvs rlog: Logging src/winsup/w32api/lib/ddk cvs rlog: Logging src/winsup/w32api/lib/directx invalid initial_branch for file bfd/po/BLD-POTFILES.in, probably from old cache, run with -x. DONE; creating master branch fatal: refs/heads/origin: not a valid SHA1 fatal: master: not a valid SHA1 fatal: 'HEAD': No such file or directory usage: git-read-tree (<sha> | -m [--aggressive] [-u | -i] <sha1> [<sha2> [<sha3>]]) checkout failed: 256

So it fails pretty early this time :)

MfG, JBG

On Fri, Mar 24, 2006 at 08:52:29AM +0100, Jan-Benedict Glaw wrote:

I guess that error message wasn't quite as obvious as I intended.

That means you have old cvsps cache state hanging around. You can either run cvsps with -x or delete the cache file manually. Those are the files in ~/.cvsps.

Incidentally, I'd recommend doing this in two stages during trouble-shooting. Run cvsps first and verify that you can produce a valid ancestry tree. If it's not-quite-right you can even edit the cvsps output to reparent the incorrect branches. Then run git-cvsimport after you're satisfied with the ancestry.

-chris

Linus Torvalds wrote:

Probably because in 99% of all cases, revision 1.1.1.1 is the result of a "cvs import".

	-hpa

On Wed, 2006-03-22 at 22:09 -0800, H. Peter Anvin wrote:

All odd branches are imports. Internal branches are even. So, 1.1.3.1 would be the first import along the second vendor branch from the trunk.

Note that vendor branches are always made from the first revision along a branch, independent of when they occur, so you'll get 1.1.3.1 even if the head revision along the trunk is 1.246.

On Thu, 23 Mar 2006, Keith Packard wrote:

I have to say, that one thing I've learnt during this whole git thing is that other SCM's are DAMN CONFUSED.

I used to think that git was potentially hard to understand. Not so. git is an absolute paragon of logic and easy-to-understand concepts.

Compared to SVN (can anybody sat "trunk/branch/tag confusion") and CVS, git is not only a hell of a lot more capable, it's just more logical.

We will hereby start scouring the net for people who say git is hard to understand and use, and just kill them. They clearly are just polluting the gene pool.

		Linus

On Thu, 23 Mar 2006 08:01:14 -0800 (PST) Linus Torvalds <torvalds@osdl.org> wrote:

lol, that sounds like a really good plan. Perhaps as a two pronged effort its worth changing the notion that git is primarily "plumbing". Adding some of the nice features of cogito and other "porcelains" into the core git might go a ways toward converting the few naysayers we don't kill.

Sean

On Thu, 23 Mar 2006, sean wrote:

Actually, as far as I can tell, git already has a hell of a lot more porcelain than pretty much any non-IDE type traditional SCM. Certainly more than CVS.

Yeah, I'm not counting things like Eclipse etc. I'm talking about "plain SCM" environments, ie just basic SVN or CVS. What are we missing in that department? (The only thing I can think of is a diff colorizer, which some prople seem to really want).

		Linus

Linus Torvalds <torvalds@osdl.org> wrote:

A pretty native point-and-click Windows GUI so Windows users can use GIT without knowing how to actually use their computer. :-)

I'm not trying to bash Windows users. I'm just saying that there's definately a large user base for SCMs such as CVS who just want to check in the latest version of a file they have to maintain. Many of these people are afraid of a command prompt. Asking them to install Cygwin just to check in a file is a difficult challenge.

And even if a user is perfectly comfortable with a command prompt and could write one-line scripts faster than anyone else, sometimes users just prefer a GUI interface.

qgit probably comes close in this department but hasn't been packaged up into a pretty Windows installer. :-)

But your definately right; once the blame/annotate war settles out GIT will have pretty much everything one might need - except a good distributed bug/issue tracking type system. :-)

On Thu, Mar 23, 2006 at 03:48:25PM -0500, Shawn Pearce wrote:

Junio, where do we stand on this?

I've been a bit busy to finish the bug fixing I need to do on annotate[1], and frankly, I have no strong feelings either way.

I must admit, I find annotate easier to read, but I wrote it, so I'm a bit biased. Maybe it is just that in C you sometimes get lost in the details.

1 - The only bug I'm aware of, at the moment, is the fact that merges sometimes get assigned poorly, which I have a plan to fix, I just haven't had the time to sort it out yet.

Ryan Anderson <ryan@michonline.com> writes:

As far as I am concerned, you two are still on the starting line. Admittably, Fredrik took a bit more time to come to there while you were still waiting there. But since then I do not think either of you moved much.

I have not seen anybody to come up with a test history, compare what the two algorithms do on that test history, and argue why one is more correct than the other. I've been too busy to start that myself, and honestly speaking I am not that interested in the details myself in that area.

To me, the only reason annotate/blame exist in git is to support the cvs server emulation, so obviously I want at least _one_ of them working correctly to be usable by git-cvsserver, but beyond that, which one to pick is not really what I am interested in. When I am working in a git repository, I'd rather use combination of whatchanged and pickaxe not annotate/blame myself anyway.

I suspect there are people a lot more interested in having a better annotate/blame, and I've been sort-of hoping somebody would really start that blame/annotate war.

Shawn Pearce <spearce@spearce.org> writes:

Export your git repository via git-cvsserver and have them use TortoiseCVS. Such "maintain the tip and that is the only thing what interest me" people do not even need to know the backend is git.

Hi,

On Thu, 23 Mar 2006, Junio C Hamano wrote:

Now if I could only find a way to tell TortoiseCVS which CVS_SERVER to use...

Ciao, Dscho

Shawn Pearce <spearce@spearce.org> wrote:

There ought to be such a thing.  And I hope it gets called `bugger'.

-- [mdw]

Mark Wooding wrote:

I'm working (slowly) on integrating it with Mantis (www.mantisbt.org), which we use at work. It shouldn't be difficult to reuse that code with Bugzilla and other similar trackers.

The recognition thing is done in the update-script, looking for a hash followed by a number (the bug-id) and then sending that commit to another program, so it's simply a matter of including the bug-id, prefixed with a hash, and the bug-topic somewhere in the commit message, which is a fairly good practice anyways.

From: Linus Torvalds <torvalds@osdl.org>
Date: Thu, 23 Mar 2006 12:38:33 -0800 (PST)

gitk does color the diffs already, or are we talking about some "side-by-side" multiple window thing showing "before" on the left and "after" on the right?

Given that the gitk author has also written diff colorizers in the past, I don't see providing this as being much of a problem :)

On Thu, 23 Mar 2006, David S. Miller wrote:

I think we're just talking about 

	git diff | git colorize

like this (and then integrating it a bit better).

		Linus

--- colorize.c --- #include "cache.h"

#define BUFSIZE 128

#define NORMAL "" #define BOLD "\033[1m" #define UL "\033[4m"

#define GRAYBG	"\033[47m"

#define BLACK "\033[30m" #define RED "\033[31m" #define GREEN "\033[32m" #define YELLOW "\033[33m" #define BLUE "\033[34m" #define MAGENTA "\033[35m" #define CYAN "\033[36m" #define WHITE "\033[37m" #define RESET "\033[m"

enum linetype {
	UNKNOWN,
	HEAD,
	FRAGHEAD,
	ADD,
	REMOVE,
	UNCHANGED,
};

static const char *tput[][2] = {
	[UNKNOWN] =	{ NORMAL, NORMAL "\n" },
	[HEAD] =	{ BOLD,   RESET "\n" },
	[FRAGHEAD] =	{ GRAYBG, RESET "\n" },
	[ADD] =		{ GREEN,  RESET "\n" },
	[REMOVE] =	{ RED,    RESET "\n" },
	[UNCHANGED] =	{ NORMAL, NORMAL "\n" },
};

static void colorize(FILE *in, FILE *out)
{
	char buffer[BUFSIZE];
	const char *end = NULL;
	int in_fragment = 0;

	while (fgets(buffer, BUFSIZE, in)) {
		int type = UNKNOWN;
		int eoln, len = strlen(buffer);
		const char *begin;

		if (!len)
			break;
		eoln = buffer[len-1] == '\n';
		if (eoln)
			buffer[--len] = 0;

		/* Did we have a partial line from before? */
		if (end) {
			fputs(buffer, out);
			if (!eoln)
				continue;
			fputs(end, out);
			end = NULL;
			continue;
		}

		if (in_fragment) {
			in_fragment--;
			switch (buffer[0]) {
			case '+':
				type = ADD;
				break;
			case '-':
				type = REMOVE;
				break;
			case ' ':
				type = UNCHANGED;
				break;
			case '\\':
				type = UNCHANGED;
				break;
			default:
				in_fragment = 0;
			}
		}
		if (type == UNKNOWN) {
			if (!strncmp(buffer, "@@ ", 3)) {
				type = FRAGHEAD;
				/* We should parse the line numbers */
				in_fragment = INT_MAX;
			}
			if (!strncmp(buffer, "+++ ", 4) ||
			    !strncmp(buffer, "--- ", 4) ||
			    !strncmp(buffer, "diff ", 5))
				type = HEAD;
		}

		begin = tput[type][0];
		end = tput[type][1];
		fputs(begin, out);
		fputs(buffer, out);
		if (!eoln)
			continue;
		fputs(end, out);
		end = NULL;
	}
	if (end)
		fputs(end, out);
}

int main(int argc, char **argv)
{
	colorize(stdin, stdout);
	return 0;
}

On Thu, 23 Mar 2006 13:31:20 -0800 (PST) "David S. Miller" <davem@davemloft.net> wrote:

Colorized "git diff", like cg-diff. Vim users can use vimpager instead of less.

On Thu, 23 Mar 2006 12:38:33 -0800 (PST) Linus Torvalds <torvalds@osdl.org> wrote:

Yeah, i was thinking more along the lines of the way cogito handles commit message editing for example, where you can change which files are committed by editing the file list in place. Maybe the colorized git log viewer would be worth pulling into core as well, etc.

It's been a long time since i've looked at cogito but perhaps there are other things in it that have proven useful and deserve to be pushed into core.

I guess my original comment was made because I always cringe when i see git described as "plumbing" and only having porcelain-"ish" commands included.

Sean

On Thu, Mar 23, 2006 at 12:38:33PM -0800, Linus Torvalds wrote:

I'd like sunglasses with that diff colouriser, please ;-)

For my various hacking projects and archiving needs git has done me alot of good and it's pretty close to the answer to the question for life, universe and everything. But a few rough areas (I'm currently using git 1.2.4 btw.) remain:

 o During the debugging phase before a new kernel release I put anything
   that isn't appropriate for the master branch on a queue branch which
   I am rebasing frequently to ensure things will work right in the
   "patch bombing" phase before the next -rc1 when I'm sending everything
   on the queue branch upstream.
   The problem: users pull such a branch, create their own branch starting
   somewhere on my queue branch.  So eventually when they pull again
   after I rebased the branch things blow up spectactularly.  This needs a
   simple solution.
 o git rebase had no reasonable handling of conflicts last I ran into a
   rebase conflict.
 o If a file is modified in a user's tree and a non-conflicting patch is
   being pull users seem to expect the old CVS behaviour which is trying
   to merge into the checked out tree, worst case adding conflict markers.
   Git just refuses the operation.
 o I had people piling up over 2GB in their $GIT_DIR/objects/pack/
   directory because they were using the rsync method for updating.
 o Git is a dramatically more powerful and for most operations better
   performing SCM than CVS - but CVS is what people know, it's easy to
   learn and handling special cases like conflicts is sort of obvious
   because CVS expects the user to cleanup the mess and does not try to
   compete with the users in that.
 o A Git for Dummies book would be helpful.
 o When users have problems with git I found it useful to explain them
   how git internally works so they get a better understanding of what
   actually is going on.  Dominic Sweetman which is an excellent
   technical writer has made a similar experience and started writing
   a bit about git in the wiki at http://www.linux-mips.org/wiki/WhatIsGit
   May somebody wants to extend this?
   (Dominic unfortunately is currently deeply burried in writing the
   2nd issue of See MIPS Run, so can't really contribute ...)

  Ralf

Ralf Baechle wrote:

See how Junio does with next and pu and recommend your users to do the same. There's no way of pulling a rebased branch, because the rebasing destroys ancestry information, meaning the original commits other people have cease to exist in your repository.

"git rerere" might be of service here. Other than that, it's merging that goes, unless we come up with a way of patching the delta on the fly when such things are encountered. Unfortunately that is beyond me, but perhaps there are other takers on the list. It would indeed be very nice to have.

A pull (fetch + merge) requires a pristine index. If the user has done "git update-index" on the files, but not committed the merge *should* fail every time. If they have changes in the working tree but not in the index, the fetch should work, but the final phase (checking out the updated head) should fail, since the working tree has un-committed changes.

This most likely happens because you're doing too large packs. You can do incremental packing, or ask users to switch to using the git:// protocol, which is much faster for incremental updates.

git doesn't compete with the user either, but it doesn't touch the working tree unless the merge succeeds, which is sane imo but surprising for CVS users where the action is done in the working tree and the result is put under rcs control.

The tutorial is fairly complete.

http://www.kernel.org/pub/software/scm/git/docs/tutorial.html

Good to know. Unfortunately I don't know git internals half as well as I would like. I can sometimes answer questions, but starting from scratch and explain it is beyond me.

On Fri, 24 Mar 2006 13:59:02 +0100, Andreas Ericsson wrote:

But the "other people" still have those commits, so it should be rather straightforward for a tool to also perform a rebase for them when doing this kind of "rebased pull". I think there's just a single arc of data missing showing where a rebased commit object came from.

So this sounds solvable, and it is something I would very much enjoy having, (call me funny, but I prefer to rebase and avoid a merge commit when looking at independent lines of development for which logically there shouldn't be any "merge" required).

-Carl

Carl Worth wrote:

Yes they do, but you don't, so their tip won't match yours, meaning their git will try a merge, which will fail since lots of commits are already applied. Perhaps it would be possible to try the blobs against each other, if anyone's interested.

For the cases where no merge is required you could rebase several branches on top of one and simply publish that one. If that's the case, git would need the ability to know what branches are exported and which arne't, which should be a lot simpler than implementing a rebased-merge strategy.

On Thu, Mar 23, 2006 at 08:01:14AM -0800, Linus Torvalds wrote:

This might be somewhat controversial, and I haven't done any research to confirm my impression, but you might be just seeing the symptoms of different ways of looking at the problem.

Scott Collins (QT evangelist, incredibly smart guy) commented to me sometime over the summer, that every new SCM was born out of someone's desire to implement a new merge algorithm. While I think that's too simple, I think there have been an awful lot of academic SCMs out there.

Git has taken a very pragmatic approach, in that the goal has been "What is the smallest number of concepts we can create that let us solve the problem, even if we occassionally have to make some tradeoffs?" (Thinking of rename detection there, mostly.)

So, really, I guess the comment I'm trying to make here is that Occam was right.

On Thu, 23 Mar 2006, Ryan Anderson wrote:

The exact details are lost in antiquity, but I'm sure one of the defining moments in time for CVS was Dick Grune saying "Merges? We don't need no steenking merges! We'll just make branching difficult! Yeah, that's it! Mwhahahhahhaaaa!".

The rest is history.

[ Really, the sad part is that you're probably right even when it comes to 
  CVS. The #1 feature of CVS as defined by Brian Berliner in his CVS II 
  paper was 'Concurrent access and conflict-resolution algorithms to 
  guarantee that source changes are not "lost"'. ]

			Linus

Ryan Anderson <ryan@michonline.com> writes:

I do not see it as a tradeoff not to record renames. It _is_ a feature.

On the other hand, rename detection is an eye candy, which is sometimes useful but only sometimes. If you look at the history of a real project, content movement across multiple files is a norm, and content movement between two files, one of which disappears and the other appears, is a rather narrow special case. If you think in terms of "renames", you can only talk about that special case, and rename detection also can only deal with that special case.

Two good examples were discussed some time ago on the list. One was about "where did {powerpc,pcc64}/Makefile come from?" and the answer was "content migrated over time across multiple commits, and you cannot really say this Makefile is renamed from somewhere". The other was the comment by Linus on how revision.c evolved in our project.

I am reasonably happy with how our rename detection turned out to be, but we should keep in mind that detecting file renames is scratching only a narrowly defined subset of the problem space.

Pickaxe was an attempt to help tracking other forms of content movement, and it is minimally useful as a building block, but if we really want to track content movement across file boundaries, like Linus originally envisioned in

	http://article.gmane.org/gmane.comp.version-control.git/217

we need to have a bit more Porcelain around it.

On Thu, Mar 23, 2006 at 03:51:51PM -0800, Junio C Hamano wrote:

Oh, I don't disagree.

What I was getting at was that not recording renames means we've traded off a little bit of speed and maybe accuracy, when we care about renames, for a simpler, better implementation.

It's a tradeoff, but one that was very much the right decision, IMO.

Ryan Anderson <ryan@michonline.com> writes:

Well, that is like arguing that we do not autocommit every time the user makes any change in the working tree -- which means git cannot be used to go back to _any_ time in history -- but we are making that tradeoff and instead letting the user to decide explicitly when to make commits.

Recording every keystrokes _is_ a wrong feature and not supporting a wrong feature _is_ a feature. It is not a tradeoff.

When I said "not recording renames _is_ a feature", I really meant it that way.

On Wed, Mar 22, 2006 at 03:39:00PM -0800, Linus Torvalds wrote:

Last I converted CVS trees to git found that about half of the branches were branching off from the wrong commit of the parent branch. At that time I decieded to just move the branch using a quick script instead of diving into cvsps.

  Ralf

On Wed, 2006-03-22 14:33:37 +0100, Jan-Benedict Glaw <jbglaw@lug-owl.de> wrote:

Since it seems nobody looked at the GCC import run (which means to use the svnimport), I ran it again, under strace control:

... 4279 trunk/gcc/config/i386/xm-sco.h ...

This time it broke at a different revision, so I guess it's not a SVN or git / git-svnimport problem, but rather a problem of my Perl installation or the kernel itself?

Tree ID 5b04fbc98f8dc9d50506b6dbc8f31567eea2e225
Committed change 4279:/ 1993-04-29 21:13:46)
Merge parent branch: eeb742d8ffd78d58f05d0b9c80bb55e1dc25ad13
Commit ID e85129f5e8af0b93a41d5bf294f17a9c9bf9fa21
Writing to refs/heads/origin
DONE: 4279 origin e85129f5e8af0b93a41d5bf294f17a9c9bf9fa21
... 4280 trunk/gcc/config/mips/mips.h ...
Tree ID 3feb45ec3ee93e8a6d75b8ce552281e0ed2d7215
Committed change 4280:/ 1993-04-30 00:53:35)
Merge parent branch: e85129f5e8af0b93a41d5bf294f17a9c9bf9fa21
Commit ID 34b473ffc0e05419c50be848d5349592b7c48ee3
Writing to refs/heads/origin
DONE: 4280 origin 34b473ffc0e05419c50be848d5349592b7c48ee3
readline() on closed filehandle H at /home/jbglaw/bin/git-svnimport line 562.
4281: cannot find commit 'origin'!
readline() on closed filehandle H at /home/jbglaw/bin/git-svnimport line 562.
4282: cannot find commit 'origin'!
readline() on closed filehandle H at /home/jbglaw/bin/git-svnimport line 562.
4283: cannot find commit 'origin'!
readline() on closed filehandle H at /home/jbglaw/bin/git-svnimport line 562.
4284: cannot find commit 'origin'!
readline() on closed filehandle H at /home/jbglaw/bin/git-svnimport line 562.
4285: cannot find commit 'origin'!
... 4286 trunk/gcc/fixincludes ...
Can't fork at /home/jbglaw/bin/git-svnimport line 379.

strace of this:

read(3, "rintf decl"..., 4096) = 2896 write(6, "superfluou"..., 4096) = 4096 read(3, "$file ${LI"..., 4096) = 1448 read(3, "m -f ${LIB"..., 4096) = 1448 read(3, "LIB}/machi"..., 4096) = 1448 write(6, " 2>/dev/nu"..., 4096) = 4096 read(3, "memory\\.h"..., 4096) = 2896 read(3, "h>\") > ${"..., 4096) = 1448 write(6, "&& [ ! -r "..., 4096) = 4096 read(3, " char *__n"..., 4096) = 3438 write(6, "claim to h"..., 4096) = 4096 write(6, "ymbolic no"..., 446) = 446 close(6) = 0 pipe([6, 7]) = 0 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb7ddf708) = -1 ENOMEM (Cannot allocate memory) close(6) = 0 close(7) = 0 write(2, "Can\'t for"..., 55) = 55 close(4) = 0 close(3) = 0

What are possible reasons for clone() to fail with -ENOMEN? I have to admit that the box _is_ loaded a bit all the time:

jbglaw@bixie:~/vax/git-conversion$ uptime
 19:23:58 up 136 days,  7:46, 20 users,  load average: 4.45, 4.25, 3.05
jbglaw@bixie:~/vax/git-conversion$ free
             total       used       free     shared    buffers     cached
Mem:        507308     501760       5548          0       2184      16900
-/+ buffers/cache:     482676      24632
Swap:      2441872    1295512    1146360

MfG, JBG

Jan-Benedict Glaw wrote:

If you send me a bzipped tar-ball of the repo you're trying to import, preferrably with all the patches to cvsps you've tried, I'll see what I can do over the weekend.

On Fri, 2006-03-24 20:10:55 +0100, Andreas Ericsson <ae@op5.se> wrote:

It's the regular SVN-based sources of GCC at their upstream location. In their CVS timeline, they had the repository rsync'able, but I guess with SVN we only get access through the SVN server.

MfG, JBG

Jan-Benedict Glaw <jbglaw@lug-owl.de> writes:

I have the same (?) problem with one of my svn repository. It worked before (I've redone the import with the -r flag), so I bisected it. The problematic commit seems to be:

diff-tree 4802426... (from 525c0d7...)
Author: Karl  Hasselström <kha@treskal.com>
Date:   Sun Feb 26 06:11:27 2006 +0100

    svnimport: Convert executable flag

    Convert the svn:executable property to file mode 755 when converting
    an SVN repository to GIT.

    Signed-off-by: Karl Hasselström <kha@treskal.com>
    Signed-off-by: Junio C Hamano <junkio@cox.net>

:100755 100755 ee2940f... 6603b96... M  git-svnimport.perl

I think it has a memory leak, it used up to 140m of memory.

$ git reset --hard 4802426^ $ time ../git-svnimport.perl file:///path/ Use of uninitialized value in string eq at ../git-svnimport.perl line 463. Use of uninitialized value in substitution (s///) at ../git-svnimport.perl line 466. real 0m55.801s user 0m30.578s sys 0m23.084s

$ git reset --hard 4802426 $ time ../git-svnimport.perl file:///path/ Use of uninitialized value in string eq at ../git-svnimport.perl line 463. Use of uninitialized value in substitution (s///) at ../git-svnimport.perl line 466. Can't fork at /home/santi/usr/src/scm/git/git-svnimport.perl line 331. real 6m2.163s user 0m20.332s sys 0m50.180s

and it didn't finished. Hope it helps.

Santi

Jan-Benedict Glaw <jbglaw@lug-owl.de> wrote:

If you don't care for automated branch handling, how about trying git-svn? under the contrib/ directory in git.git

	git-svn init svn://gcc.gnu.org/svn/gcc
	git-svn fetch

I've known of SVN library bindings leaking memory in the past, but I thought it's been solved. Afaik, any memory allocated by the Perl interpreter is never released back to the kernel, either. (At least that seems to be the case with my setup (Debian unstable, Perl 5.8.8, 2.6 kernel, x86 machine).

Some importers (my own git-svn included) aren't amazingly efficient when handling lots of history which gcc has. It looks like (from what I understand of the SVN api used in git-svnimport) is that the entire log for the 100k+ revisions in the tree is slurped down into memory before any processing is done.

git-svn does this too, but by parsing the output of the svn binary instead of using the library, so at least it won't have issues with the svn bindings and libraries to worry about.

My git-svn process running on the SVN tree just finished parsing the svn log output, and it's maxed out at 74M RSS (on a 32-bit x86). It'll probably take a while to import it all (which I won't do), but I could have just as easily done the following to reduce memory usage by ~half:

	git-svn fetch -r0:50000		# import the first 50000k
	git-svn fetch			# now import the remaining

Afaik, there's no way to do something like the above with git-svnimport for memory-starved setups.

We should be safely able to import histories with thousands of revisions without hogging up lots of memory.

With this, we lose the ability to autocorrect mistakes when people specify revisions in reverse, but it's probably no longer a problem since we only have one method of log parsing nowadays.

I've added an extra check to ensure that revision numbers do increment.

Also, increment the version number to 0.11.0. I really should just call it 1.0 soon...

Signed-off-by: Eric Wong <normalperson@yhbt.net>

---

 contrib/git-svn/git-svn.perl |  109 ++++++++++++++++++++++++------------------
 1 files changed, 63 insertions(+), 46 deletions(-)

c76df6617116a7d330a3110230bc3b01eaf9c66d

Isn't gcc in svn nowadays?

I'd try something like:

rsync://gcc.gnu.org/gcc-svn gcc-svn git-svnimport -C gcc-git -i -v file:///$(pwd)/gcc-svn

unless you have write access, in which case you may prefer:

mkdir gcc-svn && cd gcc-svn git-svn init svn+ssh://username@gcc.gnu.org/svn/gcc/trunk git-svn fetch

-JimC