threads / discuss / 9262

Can you do this with GIT?

Subject: Can you do this with GIT?

## tl;dr

7 messages between Jul 27, 2007 and Jul 27, 2007.

replies: 6people: 4as markdown or json

CPD· Jul 27, 2007, 18:02 UTC · lore
Hi All,

I hope this is the right forum, it's all I could find. Sincere apologies in advance if I in the wrong place.

I set up a source control system for the company around CVS, but GIT has some very attractive features and I'd like to migrate if it can do some other things that we need.

We produce variations based on a (mostly) common codebase. In CVS I set up "environment" modules for each platform, then when you are working on that platform, you simply check out the correct environment and build. Only the needed code and tools are exposed in that environment (this is important as clients must NOT see each other's code and most customers have some customization). I do this by defining and renaming modules in the CVSROOT modules file.

Does GIT support anything like this? Or another way to acheive the same end?
Thanks muchly, in advance, Colin
-- 
View this message in context: http://www.nabble.com/Can-you-do-this-with-GIT--tf4159350.html#a11834063
Sent from the git mailing list archive at Nabble.com.
Johannes Schindelin· Jul 27, 2007, 19:05 UTC · re: CPD · lore

Re: Can you do this with GIT?

Hi,
On Fri, 27 Jul 2007, CPD wrote:
> I hope this is the right forum, it's all I could find. Sincere apologies 
> in advance if I in the wrong place.

You might be interested in http://git.or.cz: there is a lot of useful information to find; amongst others what is the right forum...

So yes, this is the right forum.
Show 11 quoted lines
> I set up a source control system for the company around CVS, but GIT has
> some very attractive features and I'd like to migrate if it can do some
> other things that we need.
> 
> We produce variations based on a (mostly) common codebase. In CVS I set 
> up "environment" modules for each platform, then when you are working on 
> that platform, you simply check out the correct environment and build. 
> Only the needed code and tools are exposed in that environment (this is 
> important as clients must NOT see each other's code and most customers 
> have some customization). I do this by defining and renaming modules in 
> the CVSROOT modules file.

I would use branches for that. A base branch with the common code, and the customisations in all the branches, which merge from the base branch.

If you have an interesting change in a custom branch, you can percolate that back into the base branch, by checking out that base branch and cherry-picking the commit you want to have.

Of course, you can automate this merging (or even rebasing, if you are interested in keeping your customisations nice and tidy) with a script. With a git alias even.

Hth, Dscho

Ray Lehtiniemi· Jul 27, 2007, 19:45 UTC · re: Johannes Schindelin · lore

Re: Can you do this with GIT?

On Friday 27 July 2007 13:05, Johannes Schindelin wrote:
> On Fri, 27 Jul 2007, CPD wrote:
Show 10 quoted lines
> > We produce variations based on a (mostly) common codebase. In CVS I set
> > up "environment" modules for each platform, then when you are working on
> > that platform, you simply check out the correct environment and build.
> > Only the needed code and tools are exposed in that environment (this is
> > important as clients must NOT see each other's code and most customers
> > have some customization). I do this by defining and renaming modules in
> > the CVSROOT modules file.
>
> I would use branches for that.  A base branch with the common code, and
> the customisations in all the branches, which merge from the base branch.

this would break down if there were client-specific modules in the base branch, though... how could those be hidden from the other clients?

ray
Johannes Schindelin· Jul 27, 2007, 19:49 UTC · re: Ray Lehtiniemi · lore

Re: Can you do this with GIT?

Hi,
On Fri, 27 Jul 2007, Ray Lehtiniemi wrote:
Show 16 quoted lines
> On Friday 27 July 2007 13:05, Johannes Schindelin wrote:
> > On Fri, 27 Jul 2007, CPD wrote:
> 
> > > We produce variations based on a (mostly) common codebase. In CVS I set
> > > up "environment" modules for each platform, then when you are working on
> > > that platform, you simply check out the correct environment and build.
> > > Only the needed code and tools are exposed in that environment (this is
> > > important as clients must NOT see each other's code and most customers
> > > have some customization). I do this by defining and renaming modules in
> > > the CVSROOT modules file.
> >
> > I would use branches for that.  A base branch with the common code, and
> > the customisations in all the branches, which merge from the base branch.
> 
> this would break down if there were client-specific modules in the base 
> branch, though... how could those be hidden from the other clients?

Umm. Don't put the client-specific modules in the base branch, then? The base branch is the common code, the code that every client may look at. Nothing else.

Maybe I did not get the whole picture... do you want your _clients_ to access your main repo with Git?

Ciao, Dscho

Ray Lehtiniemi· Jul 27, 2007, 20:08 UTC · re: Johannes Schindelin · lore

Re: Can you do this with GIT?

On Friday 27 July 2007 13:49, Johannes Schindelin wrote:
Show 23 quoted lines
> Hi,
>
> On Fri, 27 Jul 2007, Ray Lehtiniemi wrote:
> > On Friday 27 July 2007 13:05, Johannes Schindelin wrote:
> > > On Fri, 27 Jul 2007, CPD wrote:
> > > > We produce variations based on a (mostly) common codebase. In CVS I
> > > > set up "environment" modules for each platform, then when you are
> > > > working on that platform, you simply check out the correct
> > > > environment and build. Only the needed code and tools are exposed in
> > > > that environment (this is important as clients must NOT see each
> > > > other's code and most customers have some customization). I do this
> > > > by defining and renaming modules in the CVSROOT modules file.
> > >
> > > I would use branches for that.  A base branch with the common code, and
> > > the customisations in all the branches, which merge from the base
> > > branch.
> >
> > this would break down if there were client-specific modules in the base
> > branch, though... how could those be hidden from the other clients?
>
> Umm.  Don't put the client-specific modules in the base branch, then?  The
> base branch is the common code, the code that every client may look at.
> Nothing else.

yes, i don;t think there's any other way to do it with branches (and just be careful not to merge the private branches back! :-)

> Maybe I did not get the whole picture...  do you want your _clients_ to
> access your main repo with Git?

not in my case, anyway... although if you define "client" to include "subcontractor", then yes, i'd be interested in going down that road... sometimes there are pieces which they aren't licensed to see, but if we could somehow track their work in a separate repo and then easily merge it back into the fully licensed tree, that would have great value.

from the original message, i also keyed on "Only the needed code and tools are exposed in that environment". this is something i'd like to see too... i'm working in a software eco-system with hundreds of standalone components, and would like the ability to pick and choose a small handful of those for any given project, without pulling in reams of history on the stuff i'm not using...

thanks ray

>
> Ciao,
> Dscho
Linus Torvalds· Jul 27, 2007, 19:45 UTC · re: CPD · lore

Re: Can you do this with GIT?

On Fri, 27 Jul 2007, CPD wrote:
Show 10 quoted lines
> 
> We produce variations based on a (mostly) common codebase. In CVS I set up
> "environment" modules for each platform, then when you are working on that
> platform, you simply check out the correct environment and build. Only the
> needed code and tools are exposed in that environment (this is important as
> clients must NOT see each other's code and most customers have some
> customization). I do this by defining and renaming modules in the CVSROOT
> modules file.
> 
> Does GIT support anything like this? Or another way to acheive the same end?

Depending on how big the individual pieces are, you can do it either with branches or subprojects.

If the "mostly common" codebase means that 90% of it really ends up being common, and what you have is a few config things and perhaps a library or two that either shows up or not, git branches would work really well.

And branches are totally independent of each other, so it's quite possible to have even "secret" things in one branch, and it would never get exposed to anybody else. The biggest worry you'd need to make sure of is that you don't want to merge such private branches directly with each other - since then the merged branch would see everything that was in the history of the other branch, even if the merge itself ended up not picking the "secret stuff".

So with branches that have private information, you'd have to have one "common" branch that doesn't have any of the private stuff at all, and do the general development on that, and then you can merge that common stuff into all the individual private branches (and if you want to go the other way, moving something from a private branch to the "common area", you'd have to cherry-pick just that particular detail into the common branch).

If you have bigger building blocks, and some of the whole building blocks need to be private, you'd probably want to use the subproject support, which would make for more of a CVS "modules" kind of thing and in that sense perhaps closer to what you do now, but I suspect that just using branches would actually end up being much simpler for you. It would depend mostly on how big those building blocks that you need to keep separate are.

		Linus
Ray Lehtiniemi· Jul 27, 2007, 19:52 UTC · re: CPD · lore

Re: Can you do this with GIT?

On Friday 27 July 2007 12:02, CPD wrote:
Show 7 quoted lines
> We produce variations based on a (mostly) common codebase. In CVS I set up 
> "environment" modules for each platform, then when you are working on that
> platform, you simply check out the correct environment and build. Only the
> needed code and tools are exposed in that environment (this is important as
> clients must NOT see each other's code and most customers have some
> customization). I do this by defining and renaming modules in the CVSROOT
> modules file.
this sounds very close to what i'm trying to achieve at the moment.
> Does GIT support anything like this? Or another way to acheive the same
> end?

i couldn;t find any ready way to do it.... but i've hacked up a few preliminary thoughts on how this might work in git. comments and feedback are welcome :-)

the basic idea is to import each snapshot of each vendor release as a tagged root commit, then push all these tagged root commits into a central "library" repository. each tagged root commit stands alone and functions as a cvs module.

library repos are set up as remotes under git, and a simple script will import a set of snapshots into a brand new project as a baseline for customization.

the three shell scripts, and the two control files used by them, are included here:

==> vendor-tracking.sh <==
###########################################################
#
# Import a vendor snapshot into a "library" repository.
#
# To import a vendor snapshot, unzip the snapshot into
# a new folder.  Then go into the folder and type
#
#  xxx-import <library> <tag>
#
# This will create an empty git repo in the current
# snapshot folder and create a root commit in 
# it which holds the snapshot.  This root commit will be
# tagged as "tag" and then pushed into the "library" repo.
#
# No error checking is done.
#
###########################################################
xxx-import ()
{(
	repo=$1
	tag=$2
	export GIT_DIR=/tmp/$$.$tag
	rm -rf $GIT_DIR
	mkdir -p $GIT_DIR
	git init
	git add .
	git commit -s -m "Import $tag"
	git tag $tag master
	git push $repo refs/tags/$tag:refs/tags/$tag
	rm -rf $GIT_DIR
)}
###########################################################
#
# Use a vendor release in the current project.
#
# To use a copy of some vendor snapshot in the current
# project, go to the top level of your current project,
# then type
#
#  xxx-use <library> <tag> <path>
#
# The tagged snapshot will be fetched from the library
# and deposited into the <path> subdirectory, then committed
# to the project.
# 
# No error checking is done.
#
###########################################################
xxx-use ()
{(
	repo=$1
	tag=$2
	path=$3
	git fetch $repo refs/tags/$tag:refs/tags/$tag
	git read-tree --prefix=$path/ $tag
	git commit -s -m "Use $tag as $path"
)}
###########################################################
#
# Create a new project from a set of vendor components.
#
# To start a new project with a known array of vendor
# components, create a remotes file describing the libraries
# and an manifest file describing the components and how they
# will be used. Then, in a new folder, type
#
#  xxx-start <remotes> <manifest>
#
# This will create a new Git project in the current folder,
# add a remote for each library repository, then proceed to
# fetch and commit each vendor component into the new project.
# The new project is now ready to be customized.
# 
# No error checking is done.
#
###########################################################
xxx-start ()
{(
	remotes=$1
	manifest=$2
	git init
	cat $remotes |
	while read name url
	do
		git remote add $name $url
	done
	cat $manifest |
	while read path remote tag
	do
		xxx-use $remote $tag $path
	done
	git checkout -f master
)}

==> remotes <== lib1 ssh://me@home.com/home/me/work/library lib2 //filesrv/git/library

==> manifest <== path/to/A lib1 vendorA/componentA/version1 path/to/B lib2 vendorB/componentB/version2

i would prefer to import and tag raw tree objects, since i get some of my snapshots as pre-assembled collections of (modified copies of) several components. it would therefore be convenient to have several tags per snapshot pointing at the subtrees corresponding to each component. however, fetch and push seem to prefer commits over trees....

as it stands, the vendor snapshots have no "history" to them. each snapshot stands in isolation. there was an interesting thread a few days ago about importing ancient kernel history as a series of trees, then "stitching together" a "synthetic history" from a collection of trees. i would see something like that as the basis for "vendor tracking branches", which could be restitched if an older snapshot is received "out of order".

hope it helps ray

← back to recent threads