Centralized processes in git

Hello,
     My company is looking at git to replace subversion as our scm.  
We have a small team of developers which need to publish to a  
centralized repository. One thing we would like is to have an  
automated bug fix merge ability. So if I make a bug fix to master we  
want that fix to be merged into other branches that we set. If there  
is a conflict we want that person  to be notified by email or some  
other way. I would like to know if anyone on the list has a good way  
to implement this?

On Mon, Aug 20, 2007 at 13:30:47 -0500, Robert Boone wrote:

Normal way of pushing changes to the central repo is over the ssh protocol, which invokes git (specifically git-receive-pack) on the server and that in turn invokes pre-receive, update, post-receive and post-update hooks. The first two can prevent the refs to be updated, the later two are for arbitrary post-processing.

To do the automatic merging like you describe, you'd probably put up a script to run from post-receive and start whatever merges you want to do.

That script gets a list of updated refs along with new and old version names on standard input, so you just look for your mainline name (refs/heads/master probably) and if you find, run the appropriate merges. You can of course program whatever notifications you want in that script.

Commit would undo any changes pushed between checking out a branch and commiting, so you should probably do the merges on a separate repository, that will only be managed by the scripts. You can use the alternates mechanism to avoid duplicating the data if that repository will be on the central server.

I'd recommend the hook script to write the things to just write the instructions what to merge somewhere (into a named pipe, probably -- or socket, which could be extended to separate server) and process that with a continually running "daemon" script, to avoid synchronization issues.

In git distribution is a pair of scripts in contrib/continuous directory, that use the notifying hook + daemon approach to run tests against each commit, so you can use that for inspiration. It's in perl and being universal complicates it quite a bit though; you can get a lot shorter with shell and everything hardcoded.

Jan Hudec wrote:

That makes this kind of operation ten times more complicated than it ought to be, IMO.

I wonder if it makes sense to expose a repository locking mechanism for this kind of application. The builtin git commands would test for the lock and block (waiting up to some configurable timeout) until it went away, but wouldn't necessarily ever actually lock things themselves. Or maybe a shared/exclusive lock (aka an rwlock) would be appropriate here; the repository-altering commands would grab a shared lock.

A lock-and-block primitive eliminates the need for a separate work queue manager for stuff like this: you just make sure you exclusive-lock the repo before you start your postprocessing (and make sure your postprocessing handles the case where another commit landed before you got launched, of course). Then you know that nothing else will screw with the repo while you're working, and that your execution will be serialized. If you don't need serialized operation like that, you just never grab the exclusive lock and things continue to work as today.

Stupid idea? The wrinkle, of course, is that you need to run git commands from within your script, so *those* can't block. I can think of a few easy ways around that, though, e.g., use an environment variable to identify yourself as the holder of the lock, perhaps by putting your PID in the lockfile and setting the variable to your PID.

-Steve

Steven Grimm wrote:

I guess that's a bit confusing -- what I mean to say is that the repository-altering commands that today do not lock the repository would grab a shared lock such that, in the absence of something else grabbing the exclusive lock, they will continue to effectively not lock the repository. IOW, by grabbing the shared lock they will continue to exhibit their current behavior under normal conditions.

-Steve

On Tue, Aug 21, 2007 at 08:36:40AM +0800, Steven Grimm wrote:

For this use case, I don't think you need to touch the core git commands at all. Since we're just talking about a repo that people are pushing into, why not just grab the lock in a hook before accepting the push?

That serializes the push, but each push can do arbitrary work while holding the lock.

-Peff