Re: Trying to sync two svn repositories with git-svn (repost)

From: Avery Pennarun <apenwarr@gmail.com>
Date: Apr 29, 2009, 03:19 UTC
Message-ID: <32541b130904282019n4b930f80g1ed22b2dde22510a@mail.gmail.com>
In-Reply-To: <20090428223728.GE15420@raven.wolf.lan>

On Tue, Apr 28, 2009 at 6:37 PM, Josef Wolf <jw@raven.inka.de> wrote:

Show 17 quoted lines

> Currently, there exist multiple independent repositories (for security
> reasons).  In the past, the repositories were "synchronized" manually.
> So technically, the repositories have no common history (at least not
> in svn's metadata).  But the contents are actually rather "similar",
> since they were synchronized multiple times in the past.
>
> In the long term, I'd like to move everything completely to git.  That
> would make it much easier to move changes from one repos to the other
> while keeping the (intended) differences in the policy.
>
> So my first goal is to bring the contents into sync.  The next step would
> be to create a "reference" (the official) git repository, which can be
> cloned by the administrations to create their localized repositories.
>
> In the meantime, I need a way to synchronize the contents from time to
> time.  I guess it will take some time to create the official repos and
> get used to the work flow.

Okay, I think I'm following you. And I think the difficulty of your solution will depend on how important it is to cherry-pick each individual commit from each repo vs. just merging everything as a batch.

At Versabanq, we're using git for a bunch of stuff including our autobuilder (http://github.com/apenwarr/gitbuilder) and my own branching/merging. However, for historical reasons, everything needs to also go into an svn repository, which some people use.

Yes, it is possible to rebase everything from git onto an svn branch, and then git svn dcommit it. However, in my experience, this is fairly nasty (and it also tries to linearize non-linear history, which is just messy). What we've been doing lately is just merging all changes from git into the svn branch as a single commit:

   git checkout git-svn
   git merge --no-ff mybranch   # --no-ff prevents git-svn from
crazily linearizing things
   git svn dcommit

   git checkout mybranch
   git merge git-svn

As long as you "git config merge.summary true" (to make the merge commit list all the commits it's merging) and you merge frequently enough, this is reasonably painless. You end up with a lot of merge commits, but the git history is recording everything fully, so if you want to throw away svn someday, you can just go ahead.

Now, your problem is a little more complex, because it sounds like people are checking in two types of things on both sides: private things and public things. So if you want *only* the private things, you're going to have to cherry-pick, and cherry-picking is going to confuse your merging.

If you could convince the people using svn to use two branches: one for private stuff and one for public stuff, then your life would be easier. You could just merge the public stuff in git, and ignore the private stuff.

If that's not an option, you *can* combine cherry-pick with -s ours as you suggest, though it's kind of nasty. The trick is to merge -s ours in *both* directions at the right time, so you can avoid conflicts.

    git checkout git-svn
    git merge mybranch

    git checkout mybranch
    [git cherry-pick or git merge *everything* you're missing from git-svn...]
    git merge -s ours --no-ff git-svn
      # future merges from git-svn will ignore everything in mybranch up to now

    git checkout git-svn
      # we know git-svn is already up to date, because of the first merge above
    git merge -s ours --no-ff mybranch
      # future merges from mybranch will ignore everything in git-svn up to now
    git svn dcommit

After these steps (WARNING: I didn't actually run them, so I might have made a mistake), you should have both branches in sync, and you *should* be able to merge in both directions whenever you want (make sure you use --no-ff), until the next time someone commits something private and screws you over again.

If you have more than one svn server, the above method should be extensible; just use another svn branch in place of 'mybranch' or keep cross-merging across all the branches.

Good luck :)

Avery