git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Trying to sync two svn repositories with git-svn (repost)

From
JWJosef Wolf <jw@raven.inka.de>
Date
Apr 29, 2009, 16:01 UTC
Message-ID
<20090429160129.GF15420@raven.wolf.lan>
In-Reply-To
<32541b130904282019n4b930f80g1ed22b2dde22510a@mail.gmail.com>
Thanks for your answer, Avery!
On Tue, Apr 28, 2009 at 11:19:51PM -0400, Avery Pennarun wrote:
Show 23 quoted lines
> On Tue, Apr 28, 2009 at 6:37 PM, Josef Wolf <jw@raven.inka.de> wrote:
> > Currently, there exist multiple independent repositories (for security
> > reasons).  In the past, the repositories were "synchronized" manually.
> > So technically, the repositories have no common history (at least not
> > in svn's metadata).  But the contents are actually rather "similar",
> > since they were synchronized multiple times in the past.
> >
> > In the long term, I'd like to move everything completely to git.  That
> > would make it much easier to move changes from one repos to the other
> > while keeping the (intended) differences in the policy.
> >
> > So my first goal is to bring the contents into sync.  The next step would
> > be to create a "reference" (the official) git repository, which can be
> > cloned by the administrations to create their localized repositories.
> >
> > In the meantime, I need a way to synchronize the contents from time to
> > time.  I guess it will take some time to create the official repos and
> > get used to the work flow.
> 
> Okay, I think I'm following you.  And I think the difficulty of your
> solution will depend on how important it is to cherry-pick each
> individual commit from each repo vs. just merging everything as a
> batch.
I've already done the cherry-picking.  Basically, I've done this:
  # first, move patches from second-svn to first-svn
  git checkout first-svn
  git svn rebase
  git cherry-pick sha1 # repeat as needed
  git merge -s ours second-svn
  # Now, the other way around
  git checkout second-svn
  git svn rebase
  git cherry-pick sha1 # repeat as needed
  git merge -s ours first-svn

The first git-svn-rebase after the merge causes all the (already picked) commits from the other branch to be pulled into the current branch. Adding the --no-ff option does not help. Omitting the cherry-picking does not help, either.

To be honest, I do not understand this behavior at all. I thought "-s ours" should mark the other branch as "already merged". IMHO, this should prevent future merges from pulling those commits again. But this seems not to be true: git-svn-rebase tries to apply them _again_, causing almost everything to conflict.

> At Versabanq, we're using git for a bunch of stuff including our
> autobuilder (http://github.com/apenwarr/gitbuilder) and my own

Interesting project. One question: the README mentions that the gitbuilder as capable to update itself. But I can not actually see this functionality in the scripts. Is that just a typo or am I missing something?

> Yes, it is possible to rebase everything from git onto an svn branch,
> and then git svn dcommit it.
AFAICS, this is the preferred work flow of git-svn.
Show 12 quoted lines
> However, in my experience, this is
> fairly nasty (and it also tries to linearize non-linear history, which
> is just messy).  What we've been doing lately is just merging all
> changes from git into the svn branch as a single commit:
> 
>    git checkout git-svn
>    git merge --no-ff mybranch   # --no-ff prevents git-svn from
> crazily linearizing things
>    git svn dcommit
> 
>    git checkout mybranch
>    git merge git-svn

How would this be changed if the commits are coming from the other svn repo instead of directly from git?

So should I replace all my "git cherry-pick sha1" by corresponding "git merge -s ??? --no-ff sha1" commands?

> As long as you "git config merge.summary true" (to make the merge
> commit list all the commits it's merging)

How does this option influence the merge operation? Or is this meant to provide additional information to the person who does the next merge?

> and you merge frequently
> enough, this is reasonably painless.  You end up with a lot of merge
> commits, but the git history is recording everything fully, so if you
> want to throw away svn someday, you can just go ahead.

Sounds good, but I still don't get it :-) Can you provide a more verbose example of the workflow?

> Now, your problem is a little more complex, because it sounds like
> people are checking in two types of things on both sides: private
> things and public things.  So if you want *only* the private things,

I want both. The difference is that I (usually) want to pull the public things unmodified, while I want to generalize/localize the private things. So when merging the private part, I would not want to pick the specific entries. But I still want to pick the _structure_ (possibly removing or modifying the localized entries).

Show 25 quoted lines
> you're going to have to cherry-pick, and cherry-picking is going to
> confuse your merging.
> 
> If you could convince the people using svn to use two branches: one
> for private stuff and one for public stuff, then your life would be
> easier.  You could just merge the public stuff in git, and ignore the
> private stuff.
> 
> If that's not an option, you *can* combine cherry-pick with -s ours as
> you suggest, though it's kind of nasty.  The trick is to merge -s ours
> in *both* directions at the right time, so you can avoid conflicts.
> 
>     git checkout git-svn
>     git merge mybranch
> 
>     git checkout mybranch
>     [git cherry-pick or git merge *everything* you're missing from git-svn...]
>     git merge -s ours --no-ff git-svn
>       # future merges from git-svn will ignore everything in mybranch up to now
> 
>     git checkout git-svn
>       # we know git-svn is already up to date, because of the first merge above
>     git merge -s ours --no-ff mybranch
>       # future merges from mybranch will ignore everything in git-svn up to now
>     git svn dcommit
Does that mean I should do a normal merge _before_ I go cherry-pick?
Show 9 quoted lines
> After these steps (WARNING: I didn't actually run them, so I might
> have made a mistake), you should have both branches in sync, and you
> *should* be able to merge in both directions whenever you want (make
> sure you use --no-ff), until the next time someone commits something
> private and screws you over again.
> 
> If you have more than one svn server, the above method should be
> extensible; just use another svn branch in place of 'mybranch' or keep
> cross-merging across all the branches.

Maybe I got confused about what I should do on which branch. Currently, I have five branches:

  first-svn/trunk    # svn-remote branch of my repos
  second-svn/trunk   # svn-remote branch of their repos
  first-svn          # created by "git checkout -b first-svn/trunk"
  second-svn         # created by "git checkout -b second-svn/trunk"
  master             # I don't use this one yet
For this, I have the following configuration:
  [svn-remote "first-svn"]
          url = file:///var/tmp/builds/git-sync/svn/first-svn
          fetch = trunk:refs/remotes/first-svn/trunk
          branches = branches/*:refs/remotes/first-svn/*
          tags = tags/*:refs/remotes/first-svn/tags/*
  [svn-remote "second-svn"]
          url = file:///var/tmp/builds/git-sync/svn/second-svn
          fetch = trunk:refs/remotes/second-svn/trunk
          branches = branches/*:refs/remotes/second-svn/*
          tags = tags/*:refs/remotes/second-svn/tags/*
Previous: Avery PennarunNext: Avery Pennarun
Message 6 of 31 in “Trying to sync two svn repositories with git-svn (repost)”
  1. Josef WolfApr 27, 2009
  2. Josef WolfApr 28, 2009
  3. Avery PennarunApr 28, 2009
  4. Josef WolfApr 28, 2009
  5. Avery PennarunApr 29, 2009
  6. Josef WolfApr 29, 2009
  7. Avery PennarunApr 29, 2009
  8. Josef WolfApr 29, 2009
  9. Avery PennarunApr 30, 2009
  10. Josef WolfApr 30, 2009
  11. Avery PennarunApr 30, 2009
  12. Josef WolfMay 1, 2009
  13. Avery PennarunMay 1, 2009
  14. Josef WolfMay 2, 2009
  15. Avery PennarunMay 4, 2009
  16. Josef WolfMay 4, 2009
  17. Josef WolfMay 6, 2009
  18. Avery PennarunMay 6, 2009
  19. Josef WolfMay 6, 2009
  20. Avery PennarunMay 8, 2009
  21. Josef WolfMay 8, 2009
  22. Josef WolfMay 13, 2009
  23. Avery PennarunMay 13, 2009
  24. Josef WolfMay 13, 2009
  25. Avery PennarunMay 14, 2009
  26. Josef WolfMay 14, 2009
  27. Avery PennarunMay 14, 2009
  28. Josef WolfMay 15, 2009
  29. Avery PennarunMay 15, 2009
  30. Josef WolfMay 17, 2009
  31. Josef WolfMay 20, 2009

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.