How to sync changes in GIT to ClearCase or How to get the list of files changed in GIT for a specific branch

Hi Everyone, 

Thanks for looking into this thread.

This is a help on GIT to ClearCase sync and am using UNIX session and not Windows

I am intending to use ClearCase in the background and GIT in the front. For syncing files from ClearCase to GIT i checkout the git branch use rsync command to get the changed files in ClearCase and then i do the 'git add' and 'git commit'

This works out well, but i want to get a list of files changed in GIt so that i can bring that to ClearCase and checkout and checkin files.

Can someone provide me the commands to

1. Get list of files changed in GIT, after i do commit?
2. Get list of files changed in GIT after commit, but files changed after a
tag (baseline).
3. Sync these files to ClearCase?

Hope you gurus can help me....am in desperate need of these commands. Appreciate your help in this.

Thanks ask4thunder

On  0, ask4thunder <ask4thunder@gmail.com> wrote:

To handle files that have been removed make sure to use
	rsync --exclude=.git --delete ...

and
	git add -u

to remove any deleted files from git.

One possibility:

move to the root of the clearcase project

set GIT_DIR to point to the .git directory (which, presumably is outside the clearcase project)

and some combination of:
	git diff --name-only
	git diff-index --name-status HEAD

If clearcase wasn't so painful you would ideally just run 'git checkout -f' to make clearcase match git, but clearcase is horrible and you have to explicitly 'clearcase checkout' each file.

	git diff --name-only <the-commit>^!

	git diff --name-only <the-tag>..HEAD

(or diff-index, etc)

	clearcase checkout the <path>
	git checkout HEAD -- <path>

	(assuming you've set GIT_DIR accordingly
	 and are sitting at the root of the project)

If you get this stuff working with native clearcase commands, then others could benefit from your work.

bi-directional git<->cc can be tricky (and importing all of branches can be even trickier), but even being able to import one branch is better than nothing, no?

The reason I replied is that I, too, once wrote a similar tool to suck cc projects into git. One project was so huge (and cc sucks so bad) that it literally took a few weeks for it to import.

Our organization also ran vob backups at 2am everynight, so I couldn't just let it run continuously since clearcase would lock the script out at 2am.

It ran incrementally, and lots of other site-specific
hacks.  I don't know clearcase at all (our site used a bunch
of in-house wrapper commands) so I never bothered to make
it work with raw cc commands. That's why I have nothing
to share.  It was also a total hack and once I got the stuff
I wanted out of it, you can bet that I never looked back
(which is why the bi-directional part was not important
 to me at all).

clearcase.. wow <shudders>

All it does is remind me of this horrible, horrible cc mvfs kernel module bug that was going around deleting automounts...

okay.. enough scary thoughts.

kudos for fighting the good fight,

On Thu, Apr 23, 2009 at 11:59:20AM -0700, ask4thunder wrote:

This is what I used to work with clearcase in a former job:

Use cc_import.sh to import your changes and later export your changes back to CC with export2cc.pl <path_to_CC_repo> <LABEL> <from> <to>

Greetings, Peter

cc_import.sh: Importing files from the clearcase checkout

#!/bin/sh

# ATTENTION: This will create a branch named 'cc' in your git import # Make sure that your actual checkout is NEVER on that branch if # running this script!!!!

########################### # Adapt this to your needs

BRANCH=refs/heads/cc

export GIT_WORK_TREE="/place/where/your/clearcase/checkout/is" MY_GIT_IMPORT="/place/to/your/git-import"

###########################

export GIT_INDEX_FILE="${MY_GIT_IMPORT}/.git/index.clearcase"

# save your HEAD cp .git/HEAD .git/HEAD.local

# switch to the clearcase import branch git symbolic-ref HEAD "$BRANCH"

git add -u git add .

# use the date of the checkin as commit message datum=$(date +'%Y.%m.%d %T') git commit -m"$datum"

# switch HEAD back to your previous one cp .git/HEAD.local .git/HEAD

========================

export2cc.pl: exporting git revisions to CC

#!/usr/bin/perl

# TODO's: # - The commit msg might be extracted multiple times # - it has a hardcoded path name (right now: /d/msg) # - There is a proplem adding a deep hierarchy of files # (e.g. newdir1/newdir2/newfile1} # - No rename support

use File::Basename; use strict;

my $verbose = 1; my $upstream; my $from;

my $cc_repo; my $CC_LABEL;

$cc_repo = shift @ARGV; $CC_LABEL = shift @ARGV;

print "CC repo: $cc_repo\n"; print "Label: $cc_repo\n";

sub run_cmd_pipe {
	if ($^O eq 'MSWin32' || $^O eq 'msys') {
		my @invalid = grep {m/[":*]/} @_;
		die "$^O does not support: @invalid\n" if @invalid;
		my @args = map { m/ /o ? "\"$_\"": $_ } @_;
		return qx{@args};
	} else {
		my $fh = undef;
		open($fh, '-|', @_) or die;
		return <$fh>;
	}
}

if (scalar(@ARGV) >= 1) {
	$upstream = $ARGV[0];

	$from = ((scalar(@ARGV) >= 2) ? "$ARGV[1]" : "HEAD");
}

my @revisions; push @revisions, "$ARGV[0]";

doit($revisions[0]);

sub doit($) {
	my $rev = shift;

	my @difftree_result = qx(git diff-tree -r --name-status $rev^ $rev -- );

	my @added_files;	# A
	my @copied_files;	# C
	my @deleted_files;	# D
	my @renamed_files;	# R

	my @modified_files;	# M

	foreach (@difftree_result) {
		my ($status, $file) = split /\t/;
		chomp $file;

		if ($status =~ /A/) {
			print "adding file \'$file\'\n" if ($verbose);
			push @added_files, "$file";
		}

		if ($status =~ /C/) {
			print "copied file '$file'\n" if ($verbose);
			push @copied_files, "$file";
		}

		if ($status =~ /D/) {
			print "deleted file '$file'\n" if ($verbose);
			push @deleted_files, "$file";
		}

		if ($status =~ /R/) {
			print "renamed file '$file'\n" if ($verbose);
			push @renamed_files, "$file";
		}

		if ($status =~ /M/) {
			print "modify file '$file'\n" if ($verbose);
			push @modified_files, "$file";
		}
	}

	# sort the files
	@added_files   = sort @added_files;
	@copied_files  = sort @copied_files;
	@deleted_files = sort @deleted_files;
	@renamed_files = sort @renamed_files;
	@modified_files= sort @modified_files;

	my $commit_msg = extract_commitmsg_from_git($rev);
	print $commit_msg;

	foreach (@renamed_files, @copied_files) {
		print "- $_" . "\n";
		die "can't handle renames";
	}

	foreach (@deleted_files) {
		my ($filename, $dirname) = fileparse($_);

		die "The directory $cc_repo/$dirname doesn't exist" if (! -d "$cc_repo/$dirname");

		cc_checkout($cc_repo, $dirname, "") || die "Can't checkout $dirname\n";
		cc_rmname($cc_repo, $_, $commit_msg) || die "Can't delete $_\n";
		cc_checkin($cc_repo, $dirname, "") || die "Can't checkin $dirname\n";

		# lable file and directory
		cc_mklabel($cc_repo, $dirname, $CC_LABEL) || die "Can't label $_\n";
	}
	
	foreach (@added_files) {
		print "N $_" . "\t(add the file manually)\n";
		
		my ($filename, $dirname) = fileparse($_);

		die "The directory $cc_repo/$dirname doesn't exist" if (! -d "$cc_repo/$dirname");

		cc_checkout($cc_repo, $dirname, "") || die "Can't checkout $dirname\n";
		extract_file_from_git($cc_repo, $_, $rev);
		cc_mkelem($cc_repo, $_, $commit_msg);
		cc_checkin($cc_repo, $_, "") || die "Can't checkin $_\n";
		cc_checkin($cc_repo, $dirname, "") || die "Can't checkin $dirname\n";

		# lable file and directory
		cc_mklabel($cc_repo, $_, $CC_LABEL) || die "Can't label $_\n";
		cc_mklabel($cc_repo, $dirname, $CC_LABEL) || die "Can't label $_\n";
	}

	foreach (@modified_files) {
		print "M $_" . "\n";
		cc_checkout($cc_repo, $_, $commit_msg) || die "Can't checkout $_\n";
		extract_file_from_git($cc_repo, $_, $rev);
		cc_checkin($cc_repo, $_, $commit_msg) || die "Can't checkin $_\n";
		cc_mklabel($cc_repo, $_, $CC_LABEL);
	}
}

sub cc_checkout ($$$) {
	my $path_to_repo = shift;
	my $file = shift;
	my $commit_msg = shift;

	my $result;
	my $errcode = 0;

	if ($commit_msg eq "") {
		$result = qx(cleartool co -res -ncomment "$path_to_repo/$file");
		$errcode = ($? >> 8);
	} else {
		my $msg_file = "/d/msg.txt";
		open(MSGFILE, ">$msg_file")
			or die "can't export the commit msg";
	
		print MSGFILE $commit_msg;
		close(MSGFILE);
	
		#print "cleartool co -res -unr -cfi $msg_file $path_to_repo/$file\n";
		$result = qx(cleartool co -res -unr -cfi $msg_file "$path_to_repo/$file");
		$errcode = ($? >> 8);
	
		qx(rm $msg_file);
	}

	print $result;
	return !$errcode;
}

sub cc_checkin ($$$) {
	my $path_to_repo = shift;
	my $file = shift;
	my $commit_msg = shift;

	my $result;
	my $errcode = 0;

	if ($commit_msg eq "") {
		$result = qx(cleartool ci -nc "$path_to_repo/$file");
		$errcode = ($? >> 8);
	} else {
		my $msg_file = "/d/msg.txt";
		open(MSGFILE, ">$msg_file")
			or die "can't export the commit msg";

		print MSGFILE $commit_msg;
		close(MSGFILE);

		$result = qx(cleartool ci -cfi $msg_file "$path_to_repo/$file");
		$errcode = ($? >> 8);

		qx(rm $msg_file);
	}

	print $result;

	return !$errcode;
}

##
# Add a new file/directory to clearcase
#
# @preconditon: the underlying directory is already checked out
sub cc_mkelem($$$) {
	my $path_to_repo = shift;
	my $file = shift;
	my $commit_msg = shift;

	my $msg_file = "/d/msg.txt";
	open(MSGFILE, ">$msg_file")
		or die "can't export the commit msg";

	print MSGFILE $commit_msg;
	close(MSGFILE);

	my $result = qx(cleartool mkelem -cfi $msg_file "$path_to_repo/$file");
	die "mkelem $path_to_repo/$file failed" if ($?);
	
	print $result;

	qx(rm $msg_file);
}

##
# Delete a file from clearcase
#
# @precondition: the underlying directory is already checked out
sub cc_rmname($$$) {
	my $path_to_repo = shift;
	my $file = shift;
	my $commit_msg = shift;

	my $errcode = 0;

	my $msg_file = "/d/msg.txt";
	open(MSGFILE, ">$msg_file")
		or die "can't export the commit msg";

	print MSGFILE $commit_msg;
	close(MSGFILE);

	my $result = qx(cleartool rmname -f -cfi $msg_file "$path_to_repo/$file");
	die "rmname $path_to_repo/$file failed" if ($?);
	$errcode = ($? >> 8);
	
	print $result;

	qx(rm $msg_file);
	return !$errcode;
}

##
#
sub cc_mklabel($$$) {
	my $path_to_repo = shift;
	my $file = shift;
	my $label = shift;
	
	my $result = qx(cleartool mklabel -replace "$label" "$path_to_repo/$file");
	die "mklabel $path_to_repo/$file failed" if ($?);
	print $result;
}

sub extract_file_from_git($$$) {
	my $cc_repo_path = shift;
	my $file = shift;
	my $rev = shift;

	my $result = qx(git show $rev:$file > $cc_repo_path/$file);
	die "extract_file_from_git $cc_repo_path/$file failed" if ($?);
}

sub extract_commitmsg_from_git($) {
	my $revision = shift;

	my $result = qx(git log -1 --pretty=format:'%s%n%b' $revision --);
	die "extract_commitmsg_from_git $revision failed" if ($?);
	return $result;
}

A Clearcase/Git bridge I wrote for my current job. It also pulls in history as best it can.

http://github.com/charleso/git-cc/tree/master

Charles

On Fri, May 8, 2009 at 7:04 AM, Senthil Kumar <ask4thunder@gmail.com> wrote:

Like I say not exactly an expert. But it looks like when run in this manner 'git pull' is unable to figure out the location of the .git directory. Try something like.

system("GIT_DIR=$data_path.git; git pull");

There were some other replies to your original post that promising for the cc-to-git-and-back aspect. Looking into them may save you a lot of time.

Good luck.