{"thread":{"id":"19090","subject":"Trying to sync two svn repositories with git-svn (repost)","startedAt":"2009-04-27T20:12:51Z","lastAt":"2009-05-20T16:40:14Z","messageCount":31,"participants":["Josef Wolf","Avery Pennarun"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"112457","messageId":"20090427201251.GC15420@raven.wolf.lan","threadId":"19090","inReplyTo":null,"subject":"Trying to sync two svn repositories with git-svn (repost)","fromName":"Josef Wolf","fromEmail":"jw@raven.inka.de","sentAt":"2009-04-27T20:12:51Z","receivedAt":"2009-04-27T20:12:51Z","isPatch":false,"sender":{"key":"jw@raven.inka.de","avatar":null},"body":"Hello,\n\nI have two subversion repositories which I would like to synchronize via\ngit-svn. For this, I have set up a git repository and configured two\nbranches to track the subversion repositories via git-svn:\n\n mkdir test-sync\n cd test-sync\n git svn init --stdlayout file://$REPOSDIR/svn-first\n\n for repos in svn-first svn-second; do\n git config svn-remote.$repos.url file://$REPOSDIR/$repos\n git config svn-remote.$repos.fetch trunk:refs/remotes/$repos/trunk\n git config svn-remote.$repos.branches branches/*:refs/remotes/$repos/*\n git config svn-remote.$repos.tags tags/*:refs/remotes/$repos/tags/*\n git svn fetch -R $repos\n git checkout -b $repos $repos/trunk\n done\n git gc\n\nThis gives me two remote and two local branches:\n\n master\n svn-first\n * svn-second\n svn-first/trunk\n svn-second/trunk\n\nAs a first step, I tried to \"mirror\" the manual \"merges\" that were done\nbetween the subversion repositories in the past:\n\n git checkout svn-first\n git cherry-pick svn-second-sha1 .... # repeat as needed\n\n git checkout svn-second\n git cherry-pick svn-first-sha1 .... # repeat as needed\n\nSo I've spent almost 4 weeks to cherry-pick and resolve all the conflicts.\nLooks good so far, since\n\n git diff svn-first svn-second\n git diff svn-first/trunk svn-first/trunk\n git diff svn-second/trunk svn-second/trunk\n\ngive me the desired outputs. Now I do\n\n git checkout svn-first\n git merge -s ours svn-second\n git checkout svn-second\n git merge -s ours svn-first\n\nto tell git that the branches are in sync.\n\nBut now, when I try to\n\n git checkout svn-second\n git svn rebase\n\nI get lots of conflicts. When I inspect the .git/rebase-apply directory\nand the conflicts, it looks like \"git svn rebase\" tries to re-apply all\nthe commits from svn-first. When I omit the \"git merge -s ours svn-first\"\ncommand, it does not re-apply those commits. So it looks like the\n\"git merge -s ours\" wipes some information that git-svn needs to know\nwhat was already merged.\n\nWhat am I missing? I thought the \"ours\" strategy is meant to tell git\nthat everything from that branch was merged, either manually or by\ncherry-pick.\n\nAny hints how to track this down?\n\nBTW: this is git version 1.6.0.2\n"},{"id":"112553","messageId":"20090428203040.GD15420@raven.wolf.lan","threadId":"19090","inReplyTo":"20090427201251.GC15420@raven.wolf.lan","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Josef Wolf","fromEmail":"jw@raven.inka.de","sentAt":"2009-04-28T20:30:40Z","receivedAt":"2009-04-28T20:30:40Z","isPatch":false,"sender":{"key":"jw@raven.inka.de","avatar":null},"body":"On Mon, Apr 27, 2009 at 10:12:51PM +0200, Josef Wolf wrote:\n\n[ ... ]\n> give me the desired outputs. Now I do\n> \n> git checkout svn-first\n> git merge -s ours svn-second\n> git checkout svn-second\n> git merge -s ours svn-first\n> \n> to tell git that the branches are in sync.\n> \n> But now, when I try to\n> \n> git checkout svn-second\n> git svn rebase\n> \n> I get lots of conflicts. When I inspect the .git/rebase-apply directory\n> and the conflicts, it looks like \"git svn rebase\" tries to re-apply all\n> the commits from svn-first. When I omit the \"git merge -s ours svn-first\"\n> command, it does not re-apply those commits. So it looks like the\n> \"git merge -s ours\" wipes some information that git-svn needs to know\n> what was already merged.\n> \n> What am I missing? I thought the \"ours\" strategy is meant to tell git\n> that everything from that branch was merged, either manually or by\n> cherry-pick.\n\nAfter lots of RTFM, I get the impression, that cherry-pick is the only\noperation I can do to sync a git-svn branch with other (git or git-svn)\nbranches. merge/pull should be avoided.\n\nBut then, how do I mark cherry-picked commits as \"already synched\"?\n"},{"id":"112555","messageId":"32541b130904281353k572fed5la468de65f73ccd19@mail.gmail.com","threadId":"19090","inReplyTo":"20090427201251.GC15420@raven.wolf.lan","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2009-04-28T20:53:52Z","receivedAt":"2009-04-28T20:53:52Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Mon, Apr 27, 2009 at 4:12 PM, Josef Wolf wrote:\n> I have two subversion repositories which I would like to synchronize via\n> git-svn.  [...]\n\nWhat you're attempting is rather complicated. I guess I'd suggest we\nback up a step: why do you want to do this? In what way does a \"pure\nsvn\" tool like svnsync\n(http://svnbook.red-bean.com/en/1.5/svn.ref.svnsync.html) not do what\nyou want?\n\nAre you making changes to *both* svn repositories and then want to\nsynchronize their histories? This is basically impossible, since svn\nonly has a linear history. If you add commit A to one repo, and\ncommit B to the other, you will never make the histories identical in\nboth repos; one will necessarily have A and then B, and the other will\nhave B and then A. That's not really desirable, since one of those\ntwo histories is a lie. If commit B breaks A, but it looks like A was\ncommitted *after* B, then the person who wrote A will be blamed for\nthe error.\n\nSo, what is it you're *really* trying to do?\n\nHave fun,\n\nAvery\n"},{"id":"112569","messageId":"20090428223728.GE15420@raven.wolf.lan","threadId":"19090","inReplyTo":"32541b130904281353k572fed5la468de65f73ccd19@mail.gmail.com","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Josef Wolf","fromEmail":"jw@raven.inka.de","sentAt":"2009-04-28T22:37:28Z","receivedAt":"2009-04-28T22:37:28Z","isPatch":false,"sender":{"key":"jw@raven.inka.de","avatar":null},"body":"Thanks for your answer, Avery!\n\nOn Tue, Apr 28, 2009 at 04:53:52PM -0400, Avery Pennarun wrote:\n> On Mon, Apr 27, 2009 at 4:12 PM, Josef Wolf wrote:\n> > I have two subversion repositories which I would like to synchronize via\n> > git-svn.  [...]\n> \n> What you're attempting is rather complicated. I guess I'd suggest we\n> back up a step: why do you want to do this? In what way does a \"pure\n> svn\" tool like svnsync\n> (http://svnbook.red-bean.com/en/1.5/svn.ref.svnsync.html) not do what\n> you want?\n\nThe explanation to that git somewhat longish, sorry for that.\n\nThe project is about configuring networks. If you think about cfengine,\nyou get pretty close to what it is about. It started out as a single\nrepository. But since configuring networks requires lots of\n\"localization\", it was soon split into two parts: One part are the\n\"mechanics\": generic libraries/scripts. The other part is the\n\"policy\": the configuration that specifies how the mechanics should\ndo their task. The reason for the split was to allow multiple\npolicies (for independent administrations) to share one common set of\nmechanics (as svn:externals). For security reasons, the mechanics were\nimported back into the (multiple) policies later on.\n\nCurrently, there exist multiple independent repositories (for security\nreasons). In the past, the repositories were \"synchronized\" manually.\nSo technically, the repositories have no common history (at least not\nin svn's metadata). But the contents are actually rather \"similar\",\nsince they were synchronized multiple times in the past.\n\nIn the long term, I'd like to move everything completely to git. That\nwould make it much easier to move changes from one repos to the other\nwhile keeping the (intended) differences in the policy.\n\nSo my first goal is to bring the contents into sync. The next step would\nbe to create a \"reference\" (the official) git repository, which can be\ncloned by the administrations to create their localized repositories.\n\nIn the meantime, I need a way to synchronize the contents from time to\ntime. I guess it will take some time to create the official repos and\nget used to the work flow.\n\n> Are you making changes to *both* svn repositories\n\nYes.\n\n> and then want to synchronize their histories?\n\nSince synchronization was done manually in the past, I do not (yet) care\nabout the history very much. My first goal was to get the contents into\na sane state: I've done lots of criss-cross cherry-picking. Now that the\ncontents are in sync (within git branches), I'd like to do two things:\n\n - Set \"markers\" to indicate which commits are already synchronized, so\n the next synchronization will be easier. I thought \"git-merge -s ours\"\n would be the correct way to do that. But as I already wrote: this\n causes the next \"git-svn rebase\" to apply all the commits of the other\n branch (which I already cherry-picked)\n\n - Feed the cherry-picked commits back to the svn repositories. I've not\n tried that yet, since I think I should \"git-svn rebase\" first.\n\n> This is basically impossible, since svn only has a linear history.\n> If you add commit A to one repo, and\n> commit B to the other, you will never make the histories identical in\n> both repos;\n\nYeah, I see. But I don't really care about that, as long as the relative\norder of the commits is kept when they are moved to the other repos.\nSince the svn repositories will die once the migration is done, this\nis not a big deal.\n\n> one will necessarily have A and then B, and the other will\n> have B and then A. That's not really desirable, since one of those\n> two histories is a lie.\n\nThe current situation is an even bigger lie.\n\n> So, what is it you're *really* trying to do?\n\nI hope my explanation was not too boring...\n\n> Have fun,\n\nYeah, I am having _lots_ of fun =8)\n"},{"id":"112586","messageId":"32541b130904282019n4b930f80g1ed22b2dde22510a@mail.gmail.com","threadId":"19090","inReplyTo":"20090428223728.GE15420@raven.wolf.lan","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2009-04-29T03:19:51Z","receivedAt":"2009-04-29T03:19:51Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Tue, Apr 28, 2009 at 6:37 PM, Josef Wolf wrote:\n> Currently, there exist multiple independent repositories (for security\n> reasons).  In the past, the repositories were \"synchronized\" manually.\n> So technically, the repositories have no common history (at least not\n> in svn's metadata).  But the contents are actually rather \"similar\",\n> since they were synchronized multiple times in the past.\n>\n> In the long term, I'd like to move everything completely to git.  That\n> would make it much easier to move changes from one repos to the other\n> while keeping the (intended) differences in the policy.\n>\n> So my first goal is to bring the contents into sync.  The next step would\n> be to create a \"reference\" (the official) git repository, which can be\n> cloned by the administrations to create their localized repositories.\n>\n> In the meantime, I need a way to synchronize the contents from time to\n> time.  I guess it will take some time to create the official repos and\n> get used to the work flow.\n\nOkay, I think I'm following you. And I think the difficulty of your\nsolution will depend on how important it is to cherry-pick each\nindividual commit from each repo vs. just merging everything as a\nbatch.\n\nAt Versabanq, we're using git for a bunch of stuff including our\nautobuilder (http://github.com/apenwarr/gitbuilder) and my own\nbranching/merging. However, for historical reasons, everything needs\nto also go into an svn repository, which some people use.\n\nYes, it is possible to rebase everything from git onto an svn branch,\nand then git svn dcommit it. However, in my experience, this is\nfairly nasty (and it also tries to linearize non-linear history, which\nis just messy). What we've been doing lately is just merging all\nchanges from git into the svn branch as a single commit:\n\n git checkout git-svn\n git merge --no-ff mybranch # --no-ff prevents git-svn from\ncrazily linearizing things\n git svn dcommit\n\n git checkout mybranch\n git merge git-svn\n\nAs long as you \"git config merge.summary true\" (to make the merge\ncommit list all the commits it's merging) and you merge frequently\nenough, this is reasonably painless. You end up with a lot of merge\ncommits, but the git history is recording everything fully, so if you\nwant to throw away svn someday, you can just go ahead.\n\nNow, your problem is a little more complex, because it sounds like\npeople are checking in two types of things on both sides: private\nthings and public things. So if you want *only* the private things,\nyou're going to have to cherry-pick, and cherry-picking is going to\nconfuse your merging.\n\nIf you could convince the people using svn to use two branches: one\nfor private stuff and one for public stuff, then your life would be\neasier. You could just merge the public stuff in git, and ignore the\nprivate stuff.\n\nIf that's not an option, you *can* combine cherry-pick with -s ours as\nyou suggest, though it's kind of nasty. The trick is to merge -s ours\nin *both* directions at the right time, so you can avoid conflicts.\n\n git checkout git-svn\n git merge mybranch\n\n git checkout mybranch\n [git cherry-pick or git merge *everything* you're missing from git-svn...]\n git merge -s ours --no-ff git-svn\n # future merges from git-svn will ignore everything in mybranch up to now\n\n git checkout git-svn\n # we know git-svn is already up to date, because of the first merge above\n git merge -s ours --no-ff mybranch\n # future merges from mybranch will ignore everything in git-svn up to now\n git svn dcommit\n\nAfter these steps (WARNING: I didn't actually run them, so I might\nhave made a mistake), you should have both branches in sync, and you\n*should* be able to merge in both directions whenever you want (make\nsure you use --no-ff), until the next time someone commits something\nprivate and screws you over again.\n\nIf you have more than one svn server, the above method should be\nextensible; just use another svn branch in place of 'mybranch' or keep\ncross-merging across all the branches.\n\nGood luck :)\n\nAvery\n"},{"id":"112647","messageId":"20090429160129.GF15420@raven.wolf.lan","threadId":"19090","inReplyTo":"32541b130904282019n4b930f80g1ed22b2dde22510a@mail.gmail.com","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Josef Wolf","fromEmail":"jw@raven.inka.de","sentAt":"2009-04-29T16:01:30Z","receivedAt":"2009-04-29T16:01:30Z","isPatch":false,"sender":{"key":"jw@raven.inka.de","avatar":null},"body":"Thanks for your answer, Avery!\n\nOn Tue, Apr 28, 2009 at 11:19:51PM -0400, Avery Pennarun wrote:\n> On Tue, Apr 28, 2009 at 6:37 PM, Josef Wolf wrote:\n> > Currently, there exist multiple independent repositories (for security\n> > reasons).  In the past, the repositories were \"synchronized\" manually.\n> > So technically, the repositories have no common history (at least not\n> > in svn's metadata).  But the contents are actually rather \"similar\",\n> > since they were synchronized multiple times in the past.\n> >\n> > In the long term, I'd like to move everything completely to git.  That\n> > would make it much easier to move changes from one repos to the other\n> > while keeping the (intended) differences in the policy.\n> >\n> > So my first goal is to bring the contents into sync.  The next step would\n> > be to create a \"reference\" (the official) git repository, which can be\n> > cloned by the administrations to create their localized repositories.\n> >\n> > In the meantime, I need a way to synchronize the contents from time to\n> > time.  I guess it will take some time to create the official repos and\n> > get used to the work flow.\n> \n> Okay, I think I'm following you. And I think the difficulty of your\n> solution will depend on how important it is to cherry-pick each\n> individual commit from each repo vs. just merging everything as a\n> batch.\n\nI've already done the cherry-picking. Basically, I've done this:\n\n # first, move patches from second-svn to first-svn\n git checkout first-svn\n git svn rebase\n git cherry-pick sha1 # repeat as needed\n git merge -s ours second-svn\n\n # Now, the other way around\n git checkout second-svn\n git svn rebase\n git cherry-pick sha1 # repeat as needed\n git merge -s ours first-svn\n\nThe first git-svn-rebase after the merge causes all the (already picked)\ncommits from the other branch to be pulled into the current branch.\nAdding the --no-ff option does not help. Omitting the cherry-picking\ndoes not help, either.\n\nTo be honest, I do not understand this behavior at all. I thought\n\"-s ours\" should mark the other branch as \"already merged\". IMHO, this\nshould prevent future merges from pulling those commits again. But\nthis seems not to be true: git-svn-rebase tries to apply them _again_,\ncausing almost everything to conflict.\n\n> At Versabanq, we're using git for a bunch of stuff including our\n> autobuilder (http://github.com/apenwarr/gitbuilder) and my own\n\nInteresting project. One question: the README mentions that the\ngitbuilder as capable to update itself. But I can not actually see\nthis functionality in the scripts. Is that just a typo or am I missing\nsomething?\n\n> Yes, it is possible to rebase everything from git onto an svn branch,\n> and then git svn dcommit it.\n\nAFAICS, this is the preferred work flow of git-svn.\n\n> However, in my experience, this is\n> fairly nasty (and it also tries to linearize non-linear history, which\n> is just messy). What we've been doing lately is just merging all\n> changes from git into the svn branch as a single commit:\n> \n> git checkout git-svn\n> git merge --no-ff mybranch # --no-ff prevents git-svn from\n> crazily linearizing things\n> git svn dcommit\n> \n> git checkout mybranch\n> git merge git-svn\n\nHow would this be changed if the commits are coming from the other svn\nrepo instead of directly from git?\n\nSo should I replace all my \"git cherry-pick sha1\" by corresponding\n\"git merge -s ??? --no-ff sha1\" commands?\n\n> As long as you \"git config merge.summary true\" (to make the merge\n> commit list all the commits it's merging)\n\nHow does this option influence the merge operation? Or is this meant\nto provide additional information to the person who does the next merge?\n\n> and you merge frequently\n> enough, this is reasonably painless. You end up with a lot of merge\n> commits, but the git history is recording everything fully, so if you\n> want to throw away svn someday, you can just go ahead.\n\nSounds good, but I still don't get it :-) Can you provide a more\nverbose example of the workflow?\n\n> Now, your problem is a little more complex, because it sounds like\n> people are checking in two types of things on both sides: private\n> things and public things. So if you want *only* the private things,\n\nI want both. The difference is that I (usually) want to pull the public\nthings unmodified, while I want to generalize/localize the private things.\nSo when merging the private part, I would not want to pick the specific\nentries. But I still want to pick the _structure_ (possibly removing or\nmodifying the localized entries).\n\n> you're going to have to cherry-pick, and cherry-picking is going to\n> confuse your merging.\n> \n> If you could convince the people using svn to use two branches: one\n> for private stuff and one for public stuff, then your life would be\n> easier. You could just merge the public stuff in git, and ignore the\n> private stuff.\n> \n> If that's not an option, you *can* combine cherry-pick with -s ours as\n> you suggest, though it's kind of nasty. The trick is to merge -s ours\n> in *both* directions at the right time, so you can avoid conflicts.\n> \n> git checkout git-svn\n> git merge mybranch\n> \n> git checkout mybranch\n> [git cherry-pick or git merge *everything* you're missing from git-svn...]\n> git merge -s ours --no-ff git-svn\n> # future merges from git-svn will ignore everything in mybranch up to now\n> \n> git checkout git-svn\n> # we know git-svn is already up to date, because of the first merge above\n> git merge -s ours --no-ff mybranch\n> # future merges from mybranch will ignore everything in git-svn up to now\n> git svn dcommit\n\nDoes that mean I should do a normal merge _before_ I go cherry-pick?\n\n> After these steps (WARNING: I didn't actually run them, so I might\n> have made a mistake), you should have both branches in sync, and you\n> *should* be able to merge in both directions whenever you want (make\n> sure you use --no-ff), until the next time someone commits something\n> private and screws you over again.\n> \n> If you have more than one svn server, the above method should be\n> extensible; just use another svn branch in place of 'mybranch' or keep\n> cross-merging across all the branches.\n\nMaybe I got confused about what I should do on which branch. Currently,\nI have five branches:\n\n first-svn/trunk # svn-remote branch of my repos\n second-svn/trunk # svn-remote branch of their repos\n first-svn # created by \"git checkout -b first-svn/trunk\"\n second-svn # created by \"git checkout -b second-svn/trunk\"\n master # I don't use this one yet\n\nFor this, I have the following configuration:\n\n [svn-remote \"first-svn\"]\n url = file:///var/tmp/builds/git-sync/svn/first-svn\n fetch = trunk:refs/remotes/first-svn/trunk\n branches = branches/*:refs/remotes/first-svn/*\n tags = tags/*:refs/remotes/first-svn/tags/*\n [svn-remote \"second-svn\"]\n url = file:///var/tmp/builds/git-sync/svn/second-svn\n fetch = trunk:refs/remotes/second-svn/trunk\n branches = branches/*:refs/remotes/second-svn/*\n tags = tags/*:refs/remotes/second-svn/tags/*\n"},{"id":"112653","messageId":"32541b130904291113p6f99a82ft824cd3c482447117@mail.gmail.com","threadId":"19090","inReplyTo":"20090429160129.GF15420@raven.wolf.lan","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2009-04-29T18:13:29Z","receivedAt":"2009-04-29T18:13:29Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Wed, Apr 29, 2009 at 12:01 PM, Josef Wolf wrote:\n> On Tue, Apr 28, 2009 at 11:19:51PM -0400, Avery Pennarun wrote:\n>> Okay, I think I'm following you.  And I think the difficulty of your\n>> solution will depend on how important it is to cherry-pick each\n>> individual commit from each repo vs. just merging everything as a\n>> batch.\n>\n> I've already done the cherry-picking.\n\nSo you're saying that from now on, *all* changes from *both* branches\nneed to be integrated in both directions? If so, you're done with\ncherry-picking. If not, you're not.\n\n>  Basically, I've done this:\n>\n>  # first, move patches from second-svn to first-svn\n>  git checkout first-svn\n>  git svn rebase\n>  git cherry-pick sha1 # repeat as needed\n>  git merge -s ours second-svn\n>\n>  # Now, the other way around\n>  git checkout second-svn\n>  git svn rebase\n>  git cherry-pick sha1 # repeat as needed\n>  git merge -s ours first-svn\n>\n> The first git-svn-rebase after the merge causes all the (already picked)\n> commits from the other branch to be pulled into the current branch.\n> Adding the --no-ff option does not help.  Omitting the cherry-picking\n> does not help, either.\n\nHmm, I don't see any 'git svn dcommit' in there. The steps I listed\nreferred to dcommit, but explicitly left out calls to 'git svn\nrebase'.\n\nI think it's likely that your problems stem from this. The git svn\ndocumentation refers to the 'git svn rebase' operation a lot, but it's\nonly really useful for one thing: linearizing history to make it look\n(to svn) like git was never involved. This is handy for people who\nwant to use git at work without their boss knowing about it, but it\n*loses information* and will mess up future merges.\n\nIn general, 'git svn rebase' should be avoided for all the same\nreasons that 'git rebase' should be avoided. They're both great when\nused carefully, but they shouldn't be your main day-to-day activity.\nUnfortunately git svn encourages you to use rebase in your day-to-day\nactivity... but the workflow I'm talking about actually avoids this\nproblem completely. What you want most of the time is really just\n'git svn fetch'. and 'git svn dcommit'.\n\nI think I was also a bit too offhand in my previous email when\nexpanding my suggestion to work with multiple svn hosts. The clearest\nway to do this is with three branches:\n\n - 1 remote branch: git-svn-1\n - 1 remote branch: git-svn-2\n - 1 local branch: master\n\nSo the steps are something like this. (Again, WARNING: I'm not\nrunning these as I type them, so I could be screwing up just about\nanything.)\n\nGetting started:\n\n git checkout master\n ... Use 'git svn fetch' to update git-svn-1 and git-svn-2 ...\n ... git merge/cherry-pick what you want from git-svn-1 and\ngit-svn-2. ALWAYS use --no-ff if using git merge\n git merge --no-ff -s ours git-svn-1\n git merge --no-ff -s ours git-svn-2\n # now master has everything from both svn repositories\n\nFrom now on:\n\n # Update git-svn-1 with the latest master\n git checkout git-svn-1\n # since git-svn-1 is a remote branch, you now have a detached HEAD\n git merge --no-ff master\n git svn dcommit\n\n # Update git-svn-2 with the latest master\n git checkout git-svn-2\n # since git-svn-2 is a remote branch, you now have a *different*\ndetached HEAD\n git merge --no-ff master\n git svn dcommit\n\n # Update master with the latest svn\n git checkout master\n # HEAD is now attached to master\n git merge --no-ff git-svn-1\n git merge --no-ff git-svn-2\n # no need for '-s ours' in the above merge, as no rebasing means\nno merge history was lost\n\n>> At Versabanq, we're using git for a bunch of stuff including our\n>> autobuilder (http://github.com/apenwarr/gitbuilder) and my own\n>\n> Interesting project.  One question: the README mentions that the\n> gitbuilder as capable to update itself.  But I can not actually see\n> this functionality in the scripts.  Is that just a typo or am I missing\n> something?\n\nI guess you're reading the line that says, \"Now that your gitbuilder\nis working, you probably want to have it continue\nto update itself automatically.\" This is actually talking about\n*running* itself automatically, as in \"upating the build results to\nthe latest copy of your project.\" I can see how it's a very unclear\nword to use there. Thanks for the feedback.\n\n>> As long as you \"git config merge.summary true\" (to make the merge\n>> commit list all the commits it's merging)\n>\n> How does this option influence the merge operation?  Or is this meant\n> to provide additional information to the person who does the next merge?\n\nWhen you *merge* (as opposed to rebase or cherry-pick) into an svn\nbranch, you only create a *single* svn commit that contains *all* the\nchanges. The above config setting just makes the merge commit contain\na list of all the commits it contains.\n\n>> Now, your problem is a little more complex, because it sounds like\n>> people are checking in two types of things on both sides: private\n>> things and public things.  So if you want *only* the private things,\n>\n> I want both.  The difference is that I (usually) want to pull the public\n> things unmodified, while I want to generalize/localize the private things.\n> So when merging the private part, I would not want to pick the specific\n> entries.  But I still want to pick the _structure_ (possibly removing or\n> modifying the localized entries).\n\nIf you're going to be mangling things so thoroughly, then you might\njust have to resort to cherry-picking everything one by one from one\nbranch to the other. It doesn't sound very fun, but if other people\nare being so uncooperative by mixing public and private stuff in their\nrepositories, there's no way I can see to automate it anyhow.\n\nIf you're using cherry-pick for everything, there's no reason to use\ntricks like 'merge -s ours'. Just leave out the merging entirely and\ndon't pretend that what you're doing is merging; it isn't. (You still\ndon't need 'git svn rebase' for anything. Just checkout the branch\nyou want to change, cherry-pick stuff into it, and 'git svn dcommit'\nif appropriate.)\n\nIf the situation ever changes, you can always do one last 'merge -s\nours' and mark the histories as combined. Then future merges will\nbring in any future changes.\n\nGood luck.\n\nAvery\n"},{"id":"112714","messageId":"20090429223747.GG15420@raven.wolf.lan","threadId":"19090","inReplyTo":"32541b130904291113p6f99a82ft824cd3c482447117@mail.gmail.com","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Josef Wolf","fromEmail":"jw@raven.inka.de","sentAt":"2009-04-29T22:37:47Z","receivedAt":"2009-04-29T22:37:47Z","isPatch":false,"sender":{"key":"jw@raven.inka.de","avatar":null},"body":"Thanks for your great explanations, Avery!\n\nOn Wed, Apr 29, 2009 at 02:13:29PM -0400, Avery Pennarun wrote:\n> On Wed, Apr 29, 2009 at 12:01 PM, Josef Wolf wrote:\n> > On Tue, Apr 28, 2009 at 11:19:51PM -0400, Avery Pennarun wrote:\n> >> Okay, I think I'm following you.  And I think the difficulty of your\n> >> solution will depend on how important it is to cherry-pick each\n> >> individual commit from each repo vs. just merging everything as a\n> >> batch.\n> >\n> > I've already done the cherry-picking.\n> \n> So you're saying that from now on, *all* changes from *both* branches\n> need to be integrated in both directions?\n\nExactly. Those three commands:\n\n git diff first-svn second-svn # this should be the \"private\" diff\n git diff first-svn/trunk first-svn # what my cherry-picking has changed\n # (and waits for push) in first-svn\n git diff second-svn/trunk second-svn # what my cherry-picking has changed\n # (and waits for push) in second-svn\n\nshow me _exactly_ what I want them to be. The manual synchronizations\nwhich were done in the past are resolved now. But I can't find the way\nhow to put the result of this cherry-picking back into the svn repositories\n\n> If so, you're done with cherry-picking. If not, you're not.\n\nYeah, I think I'm done with it. That's why I tried to \"git merge -s ours\"\nto tell git about the good news ;-)\n\n> >  Basically, I've done this:\n> >\n> >  # first, move patches from second-svn to first-svn\n> >  git checkout first-svn\n> >  git svn rebase\n> >  git cherry-pick sha1 # repeat as needed\n> >  git merge -s ours second-svn\n> >\n> >  # Now, the other way around\n> >  git checkout second-svn\n> >  git svn rebase\n> >  git cherry-pick sha1 # repeat as needed\n> >  git merge -s ours first-svn\n> >\n> > The first git-svn-rebase after the merge causes all the (already picked)\n> > commits from the other branch to be pulled into the current branch.\n> > Adding the --no-ff option does not help.  Omitting the cherry-picking\n> > does not help, either.\n> \n> Hmm, I don't see any 'git svn dcommit' in there. The steps I listed\n> referred to dcommit, but explicitly left out calls to 'git svn\n> rebase'.\n\nAh! I thought I _have_ to \"git svn rebase\" before I dcommit, like I need\nto \"svn update\" before I can do \"svn commit\".\n\n> I think it's likely that your problems stem from this. The git svn\n> documentation refers to the 'git svn rebase' operation a lot, but it's\n> only really useful for one thing: linearizing history to make it look\n> (to svn) like git was never involved. This is handy for people who\n> want to use git at work without their boss knowing about it, but it\n> *loses information* and will mess up future merges.\n\nOK, I'll try without rebase.\n\n> In general, 'git svn rebase' should be avoided for all the same\n> reasons that 'git rebase' should be avoided. They're both great when\n> used carefully, but they shouldn't be your main day-to-day activity.\n\nUnfortunately, all the howto's I could find recommend exactly that:\ngit-svn-rebase for getting commits from svn and dcommit for sending\ncommits to svn.\n\n> Unfortunately git svn encourages you to use rebase in your day-to-day\n> activity... but the workflow I'm talking about actually avoids this\n> problem completely. What you want most of the time is really just\n> 'git svn fetch'. and 'git svn dcommit'.\n> \n> I think I was also a bit too offhand in my previous email when\n> expanding my suggestion to work with multiple svn hosts. The clearest\n> way to do this is with three branches:\n> \n> - 1 remote branch: git-svn-1\n> - 1 remote branch: git-svn-2\n> - 1 local branch: master\n\nI will try this one. But this will take a while, since my\ncherry-picking was done criss-cross. Thus, I need to \"rebase\"\nthe cherries now to get them onto a single branch. Is there\na simple way to do that or do I have to redo the cherry-picking from\nscratch?\n\n> So the steps are something like this. (Again, WARNING: I'm not\n> running these as I type them, so I could be screwing up just about\n> anything.)\n> \n> Getting started:\n> \n> git checkout master\n> ... Use 'git svn fetch' to update git-svn-1 and git-svn-2 ...\n> ... git merge/cherry-pick what you want from git-svn-1 and\n> git-svn-2. ALWAYS use --no-ff if using git merge\n> git merge --no-ff -s ours git-svn-1\n> git merge --no-ff -s ours git-svn-2\n> # now master has everything from both svn repositories\n\nI guess I need\n\n git checkout git-svn-1; git svn dcommit\n git checkout git-svn-2; git svn dcommit\n\nat this point to push the synchronization work to the svn repositories?\n\n> >From now on:\n> \n> # Update git-svn-1 with the latest master\n> git checkout git-svn-1\n> # since git-svn-1 is a remote branch, you now have a detached HEAD\n> git merge --no-ff master\n> git svn dcommit\n> \n> # Update git-svn-2 with the latest master\n> git checkout git-svn-2\n> # since git-svn-2 is a remote branch, you now have a *different*\n> detached HEAD\n> git merge --no-ff master\n> git svn dcommit\n> \n> # Update master with the latest svn\n> git checkout master\n> # HEAD is now attached to master\n> git merge --no-ff git-svn-1\n> git merge --no-ff git-svn-2\n> # no need for '-s ours' in the above merge, as no rebasing means\n> no merge history was lost\n\nLooks reasonable.\n\n> >> At Versabanq, we're using git for a bunch of stuff including our\n> >> autobuilder (http://github.com/apenwarr/gitbuilder) and my own\n> >\n> > Interesting project.  One question: the README mentions that the\n> > gitbuilder as capable to update itself.  But I can not actually see\n> > this functionality in the scripts.  Is that just a typo or am I missing\n> > something?\n> \n> I guess you're reading the line that says, \"Now that your gitbuilder\n> is working, you probably want to have it continue\n> to update itself automatically.\"\n\nExactly.\n\n> This is actually talking about\n> *running* itself automatically, as in \"upating the build results to\n> the latest copy of your project.\" I can see how it's a very unclear\n> word to use there. Thanks for the feedback.\n\nOK..\n\n> >> As long as you \"git config merge.summary true\" (to make the merge\n> >> commit list all the commits it's merging)\n> >\n> > How does this option influence the merge operation?  Or is this meant\n> > to provide additional information to the person who does the next merge?\n> \n> When you *merge* (as opposed to rebase or cherry-pick) into an svn\n> branch, you only create a *single* svn commit that contains *all* the\n> changes. The above config setting just makes the merge commit contain\n> a list of all the commits it contains.\n\nBut git will not use this information in any way, AFAIK. So this information\nis only for the person who will do the next merge?\n\n> >> Now, your problem is a little more complex, because it sounds like\n> >> people are checking in two types of things on both sides: private\n> >> things and public things.  So if you want *only* the private things,\n> >\n> > I want both.  The difference is that I (usually) want to pull the public\n> > things unmodified, while I want to generalize/localize the private things.\n> > So when merging the private part, I would not want to pick the specific\n> > entries.  But I still want to pick the _structure_ (possibly removing or\n> > modifying the localized entries).\n> \n> If you're going to be mangling things so thoroughly, then you might\n> just have to resort to cherry-picking everything one by one from one\n> branch to the other. It doesn't sound very fun, but if other people\n> are being so uncooperative by mixing public and private stuff in their\n> repositories, there's no way I can see to automate it anyhow.\n\nThe people are not uncooperative. It is just that there's no way to\ncompletely separate the public and private content. For example, the\nprivate part of my apache config looks like this (somewhat simplified):\n\n &set_conf (apache => {\n servername => \"my.host.org\",\n davlock => \"/m/a/dav/lock\",\n confdir => \"/m/a/etc/apache\",\n httpdir => \"/m/b/httpd\",\n docroot => \"/m/b/httpd/htdocs\",\n svndir => \"/m/b/repos/svn\",\n gitdir => \"/m/b/repos/git\",\n vhosts => {\n \"*:80\"=> {\n downloads => {\n \"my-debian-repos\"=>[\"/m/b/lib/my-debian\",\n },\n },\n \"*:443\"=> {\n docroot => \"/m/b/httpd/htdocs\",\n downloads => {\n \"kdb\" => [\"/m/b/lib/kdb/SRC\", \"Kdb download\"],\n \"kdbdemo\" => [\"/m/b/lib/kdbdemo\", \"Kdbdemo download\"],\n \"pictures\" => [\"/m/b/Pictures\", \"Pictures\"],\n },\n cgis => {\n \"svn\" => [\"/m/l/svn/cgi\", \"Svn repos administration\"],\n \"misc\" => [\"/m/a/cgi\", \"Misc cgi scripts\"],\n },\n svn => { # FIXME: svn, repos\n \"ab\" => [\"Ab repository\"],\n \"misc\" => [\"Subversion Repository\"],\n \"pmisc\" => [\"Private Subversion Repository\"],\n },\n git => {\n \"test\" => [\"Git test repos\"],\n },\n revproxies => {\n \"/test/\" => [\"http://localhost:3000/railstest/\"],\n \"/foobar/\" => [\"http://foo.bar.org:8001/\"],\n }\n },\n },\n });\n\nFrom this information, the public part knows how to generate the apache\nconfig. Other people are not interested which locations are defined\nhere, But they _are_ interested in the layout how this (perl) structure \nlooks like. So I need to strip my localizations and provide the\nstructure as a template for other people to fill in. Of course, this\nleads to more conflicts in the future every time the layout is changed.\n\n> If you're using cherry-pick for everything, there's no reason to use\n> tricks like 'merge -s ours'. Just leave out the merging entirely and\n> don't pretend that what you're doing is merging; it isn't. (You still\n> don't need 'git svn rebase' for anything. Just checkout the branch\n> you want to change, cherry-pick stuff into it, and 'git svn dcommit'\n> if appropriate.)\n\nBut then I have to do the book-keeping (what was already picked in which\ndirection) by myself?\n\n> If the situation ever changes, you can always do one last 'merge -s\n> ours' and mark the histories as combined. Then future merges will\n> bring in any future changes.\n\nOK\n"},{"id":"112729","messageId":"32541b130904291907q4003ad86v4728c5b2ba0aacb7@mail.gmail.com","threadId":"19090","inReplyTo":"20090429223747.GG15420@raven.wolf.lan","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2009-04-30T02:07:31Z","receivedAt":"2009-04-30T02:07:31Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Wed, Apr 29, 2009 at 6:37 PM, Josef Wolf wrote:\n> On Wed, Apr 29, 2009 at 02:13:29PM -0400, Avery Pennarun wrote:\n>> So you're saying that from now on, *all* changes from *both* branches\n>> need to be integrated in both directions?\n>\n> Exactly.  Those three commands:\n>\n>  git diff first-svn        second-svn  # this should be the \"private\" diff\n>  git diff first-svn/trunk  first-svn   # what my cherry-picking has changed\n>                                        # (and waits for push) in first-svn\n>  git diff second-svn/trunk second-svn  # what my cherry-picking has changed\n>                                        # (and waits for push) in second-svn\n>\n> show me _exactly_ what I want them to be.  The manual synchronizations\n> which were done in the past are resolved now.  But I can't find the way\n> how to put the result of this cherry-picking back into the svn repositories\n\nOkay, I think perhaps you're missing something that took me a long\ntime to figure out about git-svn, but once I understood it, my life\nwent a lot more smoothly.\n\nBasically, 'git svn fetch' updates just a *remote* branch. (Remote\nbranches are in .git/refs/remotes/* and you can't change them\nyourself, because you can't attach your HEAD to them. If you try to\ncheck them out, you get the latest revision in that branch, but your\nHEAD becomes detached.)\n\nI'm not actually sure which of the above branches you're referring to\nis remote and which is local. Let's guess that first-svn/trunk is\nremote, and first-svn is a local copy of it, onto which you\ncherry-picked some extra patches.\n\nSome people like to think of just making a copy of the git-svn remote\nbranch, doing stuff to it, and then doing git-svn dcommit and/or\nrebase on that branch. This sounds good, but it makes a mess *if*\nyou're doing any kind of git merging (which many git-svn users never\ndo, but which it seems you'd like to do extensively). Remember: git\nmerge is 100% totally incompatible with rebasing.\n\nWhat I'm suggesting is that you think of your local branch (first-svn\nin this case, I guess) as *not* an svn branch at all. You never do\nany git-svn operations directly on this branch. In fact, rename it to\nmaster so you aren't tempted to get yourself into trouble.\n\nNow, you've merged from first-svn/trunk and cherry-picked some extra\nstuff onto this branch, right? Good. Now you want to *merge* this\nbranch into first-svn/trunk, producing just *one* new commit, and\ndcommit that into svn.\n\n git checkout first-svn/trunk\n # detaches the HEAD\n git merge master\n # produces a merge commit on the detached HEAD\n git svn dcommit\n # produces a *different* commit object on the first-svn/trunk branch\n # ...and moves HEAD to it.\n\nThe newly-produced commit tells git that first-svn/trunk is now\nup-to-date with master. Note that the interim merge commit (produced\nby 'git merge') is never shared with *anyone*, so it's perfectly okay\nthat we replace it with the next command.\n\nWhat you probably thought you should do, given that the existing\ngit-svn documentation says to do it, is more like this:\n\n # WRONG\n git checkout first-svn\n git cherry-pick some stuff\n git merge [perhaps -s ours] second-svn/trunk\n git svn dcommit\n\nBut the above will *change* every single commit you put on first-svn,\nbecause dcommit needs to *regenerate* all the commits after putting\nthem into svn and getting them back again. This is essentially a\nrebase, and disrupts any merges you might have made from this branch\nto another one. Next time you merge, you'll get a zillion conflicts.\n\n> Ah!  I thought I _have_ to \"git svn rebase\" before I dcommit, like I need\n> to \"svn update\" before I can do \"svn commit\".\n\nThis is true and yet not true. The reason I don't have to ever use\n'git svn rebase' is that 'git svn fetch' updates my first-svn/trunk\nbranch, and then I quickly do a merge-then-dcommit on that branch. If\nI was to do a 'git svn rebase' first, nothing would happen, because\nsvn doesn't change.\n\nThis is important, since 'git svn dcommit' actually *does* do a 'git\nsvn rebase' for you automatically, trying to be helpful.\n\n>> In general, 'git svn rebase' should be avoided for all the same\n>> reasons that 'git rebase' should be avoided.  They're both great when\n>> used carefully, but they shouldn't be your main day-to-day activity.\n>\n> Unfortunately, all the howto's I could find recommend exactly that:\n> git-svn-rebase for getting commits from svn and dcommit for sending\n> commits to svn.\n\nYeah, they're trying to keep things simple, at the cost of preventing\nyou from doing anything complicated. I'm not smart enough to do both,\nso I'm just making things complicated for you here ;)\n\nAs it happens, I wrote the git-svn chapter for the\nvery-nearly-available new O'Reilly book \"Version Control with Git.\" I\ngave the complicated solution there too. I'm eagerly awaiting the\ngiant flames from people who actually wrote git-svn (and its\ndocumentation) and therefore are highly qualified to disagree with me.\n\n>>   - 1 remote branch: git-svn-1\n>>   - 1 remote branch: git-svn-2\n>>   - 1 local branch: master\n>\n> I will try this one.  But this will take a while, since my\n> cherry-picking was done criss-cross.  Thus, I need to \"rebase\"\n> the cherries now to get them onto a single branch.  Is there\n> a simple way to do that or do I have to redo the cherry-picking from\n> scratch?\n\nNo no! Stop rebasing!\n\nYou have a branch that looks the way you want, right? That means\nyou're 99% of the way there. You just have to convince git that this\nbranch and the svn branch are related to each other in the way they\nactually are.\n\nTo do that, you just need to do is make a single merge commit on your\nsvn remote branch that looks the way you want and merges from your\nexisting branch, then do a single 'git svn dcommit'. Here's one way\n(assuming you want to make svn look like your new local branch):\n\n git checkout my-local-branch\n git merge -s ours svn-branch\n git checkout svn-branch\n git merge --no-ff my-local-branch\n git svn dcommit\n\n(If the occasionally-suggested '-s theirs' merge strategy existed, you\ncould just do the last three steps using git merge -s theirs.)\n\n>> >> As long as you \"git config merge.summary true\" (to make the merge\n>> >> commit list all the commits it's merging)\n>>\n>> When you *merge* (as opposed to rebase or cherry-pick) into an svn\n>> branch, you only create a *single* svn commit that contains *all* the\n>> changes.  The above config setting just makes the merge commit contain\n>> a list of all the commits it contains.\n>\n> But git will not use this information in any way, AFAIK.  So this information\n> is only for the person who will do the next merge?\n\nIn fact, it *only* affects the svn log. Otherwise svn log ends up\nwith a useless commit that says \"Merged from commit (giant hex\nstring)\", and you can't actually do anything with the giant hex string\nbecause svn doesn't know what it is.\n\nIf nobody looks at your svn changelog, it's irrelevant.\n\n> The people are not uncooperative.  It is just that there's no way to\n> completely separate the public and private content.\n\nThere is, if you're willing to do it. The usual way is two have two\nbranches: public and private.\n\nWhenever you make a change that you want to be public, you commit it\non the public branch, then merge (git merge or svn merge, it doesn't\nmatter) from public to private. If you want to make a private change,\nyou just commit it directly to private.\n\nThis way, you will always have the two sets of changes isolated, you\nnever have to cherry-pick anything, and \"git diff public private\" is\nalways a sensible thing to do.\n\n(In fact, when I do this, I often don't share the private branch with\nanyone at all, which means it's safe to rebase. That means I can keep\na clean set of patches against the public branch, and sort and\nrearrange or share them whenever I feel like it. This is useful in\nsome cases. Rebasing isn't *always* bad :))\n\n>> If you're using cherry-pick for everything, there's no reason to use\n>> tricks like 'merge -s ours'.  Just leave out the merging entirely and\n>> don't pretend that what you're doing is merging; it isn't.  (You still\n>> don't need 'git svn rebase' for anything.  Just checkout the branch\n>> you want to change, cherry-pick stuff into it, and 'git svn dcommit'\n>> if appropriate.)\n>\n> But then I have to do the book-keeping (what was already picked in which\n> direction) by myself?\n\nOn branch b, 'git merge x' will always merge all the changes from the\nmost recent merge of x into b (which might be a \"-s ours\" merge if you\nwant), up to the tip of x. So if you don't commit any *new* private\nstuff to x, you can use merge. If you're intermixing the changes,\nyou'll need to use cherry-pick. git won't attempt to track the\ncherry-picks for you (like eg. svnmerge will).\n\nHave fun,\n\nAvery\n"},{"id":"112792","messageId":"20090430222808.GH15420@raven.wolf.lan","threadId":"19090","inReplyTo":"32541b130904291907q4003ad86v4728c5b2ba0aacb7@mail.gmail.com","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Josef Wolf","fromEmail":"jw@raven.inka.de","sentAt":"2009-04-30T22:28:09Z","receivedAt":"2009-04-30T22:28:09Z","isPatch":false,"sender":{"key":"jw@raven.inka.de","avatar":null},"body":"On Wed, Apr 29, 2009 at 10:07:31PM -0400, Avery Pennarun wrote:\n> On Wed, Apr 29, 2009 at 6:37 PM, Josef Wolf wrote:\n> > On Wed, Apr 29, 2009 at 02:13:29PM -0400, Avery Pennarun wrote:\n> >> So you're saying that from now on, *all* changes from *both* branches\n> >> need to be integrated in both directions?\n> >\n> > Exactly.  Those three commands:\n> >\n> >  git diff first-svn        second-svn  # this should be the \"private\" diff\n> >  git diff first-svn/trunk  first-svn   # what my cherry-picking has changed\n> >                                        # (and waits for push) in first-svn\n> >  git diff second-svn/trunk second-svn  # what my cherry-picking has changed\n> >                                        # (and waits for push) in second-svn\n> >\n> > show me _exactly_ what I want them to be.  The manual synchronizations\n> > which were done in the past are resolved now.  But I can't find the way\n> > how to put the result of this cherry-picking back into the svn repositories\n> \n> Okay, I think perhaps you're missing something that took me a long\n> time to figure out about git-svn, but once I understood it, my life\n> went a lot more smoothly.\n> \n> Basically, 'git svn fetch' updates just a *remote* branch. (Remote\n> branches are in .git/refs/remotes/* and you can't change them\n> yourself, because you can't attach your HEAD to them. If you try to\n> check them out, you get the latest revision in that branch, but your\n> HEAD becomes detached.)\n> \n> I'm not actually sure which of the above branches you're referring to\n> is remote and which is local. Let's guess that first-svn/trunk is\n> remote, and first-svn is a local copy of it, onto which you\n> cherry-picked some extra patches.\n\nYeah, your guess is correct:\n\n first-svn/trunk remote branch of the fist svn repos\n second-svn/trunk remote branch of the second svn repos\n first-svn local branch where first-svn/trunk is to be tracked\n second-svn local branch where second-svn/trunk is to be tracked\n\n> What I'm suggesting is that you think of your local branch (first-svn\n> in this case, I guess) as *not* an svn branch at all. You never do\n> any git-svn operations directly on this branch. In fact, rename it to\n> master so you aren't tempted to get yourself into trouble.\n\nI see. But currently, I have _two_ local branches and not only one, as\nyou suggested in your last post.\n\n> Now, you've merged from first-svn/trunk and cherry-picked some extra\n> stuff onto this branch, right? Good. Now you want to *merge* this\n> branch into first-svn/trunk, producing just *one* new commit, and\n> dcommit that into svn.\n> \n> git checkout first-svn/trunk\n> # detaches the HEAD\n> git merge master\n> # produces a merge commit on the detached HEAD\n> git svn dcommit\n> # produces a *different* commit object on the first-svn/trunk branch\n> # ...and moves HEAD to it.\n> \n> The newly-produced commit tells git that first-svn/trunk is now\n> up-to-date with master. Note that the interim merge commit (produced\n> by 'git merge') is never shared with *anyone*, so it's perfectly okay\n> that we replace it with the next command.\n\nFinally, this seems to work. Luckily, I've done the cherry-picking\nnot directly, but instead wrote a perl-script with instructions which\ncommits to pick and how to resolve the conflicts for specific commits.\nI use copies of the repositories for this, so I can re-play the scenario\nuntil the result fits the expectations.\n\nWith your explanation, I finally arrived at the _first_ work flow\nthat is able to push the results of the cherry-picking back to the\nsvn repositories. Having the branches described above, it goes\nlike this:\n\n # first, retrieve what is available\n #\n git svn fetch first-svn\n git svn fetch second-svn\n\n # cherry-pick from second-svn to first-svn\n #\n git svn checkout first-svn\n git cherry-pick sha1-from-second-svn # repeat as needed\n git checkout first-svn/trunk\n git merge --no-ff first-svn\n git diff first-svn/trunk first-svn >changes.diff\n git svn dcommit\n\n # now do the same the other way around\n #\n git svn checkout second-svn\n git cherry-pick sha1-from-first-svn # repeat as needed\n git checkout second-svn/trunk\n git merge --no-ff second-svn\n git diff second-svn/trunk second-svn >changes.diff\n git svn dcommit\n\nBut I am still somewhat confused:\n\n git log -1 first-svn/trunk\n\nsays \"Merge branch first-svn into HEAD\". But this does not reflect\nwhat I've actually done: I've picked _from_ second-svn and committed\nthat _to_ first-svn.\n\n> What you probably thought you should do, given that the existing\n> git-svn documentation says to do it, is more like this:\n> \n> # WRONG\n> git checkout first-svn\n> git cherry-pick some stuff\n> git merge [perhaps -s ours] second-svn/trunk\n> git svn dcommit\n\nAlmost... In addition, I was trying to \"git svn rebase\" before the\ndcommit\n\n> But the above will *change* every single commit you put on first-svn,\n> because dcommit needs to *regenerate* all the commits after putting\n> them into svn and getting them back again. This is essentially a\n> rebase, and disrupts any merges you might have made from this branch\n> to another one. Next time you merge, you'll get a zillion conflicts.\n> \n> > Ah!  I thought I _have_ to \"git svn rebase\" before I dcommit, like I need\n> > to \"svn update\" before I can do \"svn commit\".\n> \n> This is true and yet not true. The reason I don't have to ever use\n> 'git svn rebase' is that 'git svn fetch' updates my first-svn/trunk\n> branch, and then I quickly do a merge-then-dcommit on that branch. If\n> I was to do a 'git svn rebase' first, nothing would happen, because\n> svn doesn't change.\n> \n> This is important, since 'git svn dcommit' actually *does* do a 'git\n> svn rebase' for you automatically, trying to be helpful.\n\nWhat would happen if somebody else creates a new commit just after I\n\"git svn fetch\" but before I dcommit? Guess, svn will not accept this\ncommit, because it is based on an outdated revision. How would I\nget out from this situation?\n\n> >> In general, 'git svn rebase' should be avoided for all the same\n> >> reasons that 'git rebase' should be avoided.  They're both great when\n> >> used carefully, but they shouldn't be your main day-to-day activity.\n> >\n> > Unfortunately, all the howto's I could find recommend exactly that:\n> > git-svn-rebase for getting commits from svn and dcommit for sending\n> > commits to svn.\n> \n> Yeah, they're trying to keep things simple, at the cost of preventing\n> you from doing anything complicated. I'm not smart enough to do both,\n> so I'm just making things complicated for you here ;)\n\nAt least I could finally submit the cherries to the svn repositories.\nThat's a big step forward, although I still don't fully understand all\nthe details.\n\n> As it happens, I wrote the git-svn chapter for the\n> very-nearly-available new O'Reilly book \"Version Control with Git.\" I\n> gave the complicated solution there too.\n\nInteresting. Do you have any information when it will be available?\n\n> I'm eagerly awaiting the\n> giant flames from people who actually wrote git-svn (and its\n> documentation) and therefore are highly qualified to disagree with me.\n> \n> >>   - 1 remote branch: git-svn-1\n> >>   - 1 remote branch: git-svn-2\n> >>   - 1 local branch: master\n> >\n> > I will try this one.  But this will take a while, since my\n> > cherry-picking was done criss-cross.  Thus, I need to \"rebase\"\n> > the cherries now to get them onto a single branch.  Is there\n> > a simple way to do that or do I have to redo the cherry-picking from\n> > scratch?\n> \n> No no! Stop rebasing!\n> \n> You have a branch that looks the way you want, right?\n\nUmmm, no.. I have _two_ branches:\n\n first-svn: contains the cherries that I picked from second-svn. This\n branch looks the way first-svn/trunk should be\n second-svn: contains the cherries that I picked from first-svn. This\n looks the way second-svn/trunk should be\n\nDon't I need to rebase at least one of them if I want to \"merge\" those\ntwo branches into a single one?\n\nI have a hard time to adopt my mental model to the one-branch method for\nsome reason. OTOH, I can easily understand the multiple-branch method:\nfor every remote branch, I have a local branch on which I collect the\ncommits that should go to this remote.\n\n> That means\n> you're 99% of the way there. You just have to convince git that this\n> branch and the svn branch are related to each other in the way they\n> actually are.\n> \n> To do that, you just need to do is make a single merge commit on your\n> svn remote branch that looks the way you want and merges from your\n> existing branch, then do a single 'git svn dcommit'. Here's one way\n> (assuming you want to make svn look like your new local branch):\n> \n> git checkout my-local-branch\n> git merge -s ours svn-branch\n> git checkout svn-branch\n> git merge --no-ff my-local-branch\n> git svn dcommit\n> \n> (If the occasionally-suggested '-s theirs' merge strategy existed, you\n> could just do the last three steps using git merge -s theirs.)\n> \n> >> >> As long as you \"git config merge.summary true\" (to make the merge\n> >> >> commit list all the commits it's merging)\n> >>\n> >> When you *merge* (as opposed to rebase or cherry-pick) into an svn\n> >> branch, you only create a *single* svn commit that contains *all* the\n> >> changes.  The above config setting just makes the merge commit contain\n> >> a list of all the commits it contains.\n> >\n> > But git will not use this information in any way, AFAIK.  So this information\n> > is only for the person who will do the next merge?\n> \n> In fact, it *only* affects the svn log. Otherwise svn log ends up\n> with a useless commit that says \"Merged from commit (giant hex\n> string)\", and you can't actually do anything with the giant hex string\n> because svn doesn't know what it is.\n\nOK. I just noticed the list is limited to 22 entries. Can this be\nconfigured somehow to contain the complete list?\n\n> > The people are not uncooperative.  It is just that there's no way to\n> > completely separate the public and private content.\n> \n> There is, if you're willing to do it. The usual way is two have two\n> branches: public and private.\n\nWell, my plan was to have one (generic) public repository that contains\ntemplates instead of the localized information. Separating the\nrepositories is a security measure here.\n\nWhether separate repositories or only different branches, conflicts\n_are_ to be expected in this area. \n\n> Whenever you make a change that you want to be public, you commit it\n> on the public branch, then merge (git merge or svn merge, it doesn't\n> matter) from public to private. If you want to make a private change,\n> you just commit it directly to private.\n> \n> This way, you will always have the two sets of changes isolated, you\n> never have to cherry-pick anything, and \"git diff public private\" is\n> always a sensible thing to do.\n\nYeah, I see. OTOH, I'd rather avoid doing two steps in one go.\nCurrently, I have a hard time to get _one_ branch in sync. I'll come\nback to the multiple-branch-thing as soon as I have mastered the\none-branch-thing ;-)\n\n> (In fact, when I do this, I often don't share the private branch with\n> anyone at all, which means it's safe to rebase. That means I can keep\n> a clean set of patches against the public branch, and sort and\n> rearrange or share them whenever I feel like it. This is useful in\n> some cases. Rebasing isn't *always* bad :))\n> \n> >> If you're using cherry-pick for everything, there's no reason to use\n> >> tricks like 'merge -s ours'.  Just leave out the merging entirely and\n> >> don't pretend that what you're doing is merging; it isn't.  (You still\n> >> don't need 'git svn rebase' for anything.  Just checkout the branch\n> >> you want to change, cherry-pick stuff into it, and 'git svn dcommit'\n> >> if appropriate.)\n> >\n> > But then I have to do the book-keeping (what was already picked in which\n> > direction) by myself?\n> \n> On branch b, 'git merge x' will always merge all the changes from the\n> most recent merge of x into b (which might be a \"-s ours\" merge if you\n> want), up to the tip of x. So if you don't commit any *new* private\n> stuff to x, you can use merge. If you're intermixing the changes,\n> you'll need to use cherry-pick. git won't attempt to track the\n> cherry-picks for you (like eg. svnmerge will).\n"},{"id":"112794","messageId":"32541b130904301559w329bdd4bo6f2736a505b7235f@mail.gmail.com","threadId":"19090","inReplyTo":"20090430222808.GH15420@raven.wolf.lan","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2009-04-30T22:59:50Z","receivedAt":"2009-04-30T22:59:50Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Thu, Apr 30, 2009 at 6:28 PM, Josef Wolf wrote:\n>  # cherry-pick from second-svn to first-svn\n>  #\n>  git svn checkout first-svn\n>  git cherry-pick sha1-from-second-svn # repeat as needed\n>  git checkout first-svn/trunk\n>  git merge --no-ff first-svn\n>  git diff first-svn/trunk first-svn >changes.diff\n>  git svn dcommit\n[...]\n> But I am still somewhat confused:\n>\n>  git log -1 first-svn/trunk\n>\n> says \"Merge branch first-svn into HEAD\".  But this does not reflect\n> what I've actually done: I've picked _from_ second-svn and committed\n> that _to_ first-svn.\n\nThe most recent commit to first-svn/trunk was \"git merge --no-ff\nfirst-svn\", which creates the merge commit you're seeing here. (HEAD\n== first-svn/trunk). So this sounds right to me.\n\n\"git log -1 first-svn\" would give you the first cherry-pick. But\nremember, it's a completely different branch.\n\n>> What you probably thought you should do, given that the existing\n>> git-svn documentation says to do it, is more like this:\n>>\n>>    # WRONG\n>>    git checkout first-svn\n>>    git cherry-pick some stuff\n>>    git merge [perhaps -s ours] second-svn/trunk\n>>    git svn dcommit\n>\n> Almost... In addition, I was trying to \"git svn rebase\" before the\n> dcommit\n\n'git svn dcommit' implies 'git svn rebase' first anyway, so it's the same.\n\n> What would happen if somebody else creates a new commit just after I\n> \"git svn fetch\" but before I dcommit?  Guess, svn will not accept this\n> commit, because it is based on an outdated revision.  How would I\n> get out from this situation?\n\nAFAIK, it will attempt to do \"git svn rebase\" first, and if that\nsucceeds, it will do the commit.\n\nIn such a case, the rebase should be okay, because it's only changing\ncommits (in fact, just one commit: the merge commit) that don't exist\non any other branch. Thus it won't mangle any other merges.\n\n>> As it happens, I wrote the git-svn chapter for the\n>> very-nearly-available new O'Reilly book \"Version Control with Git.\"  I\n>> gave the complicated solution there too.\n>\n> Interesting.  Do you have any information when it will be available?\n\nMid-May, as I understand it.\n\n> Ummm, no.. I have _two_ branches:\n>\n>  first-svn:  contains the cherries that I picked from second-svn. This\n>              branch looks the way first-svn/trunk should be\n>  second-svn: contains the cherries that I picked from first-svn. This\n>              looks the way second-svn/trunk should be\n\nOkay, if you want to end up with two different remote branches, it\nmakes sense to have two different local branches.\n\n> Don't I need to rebase at least one of them if I want to \"merge\" those\n> two branches into a single one?\n\nI don't think so. If you merge them together, what do you *want* it\nto look like? And what do you want to do with that branch afterwards?\n It's hard for me to guess, but it seems unlikely that rebasing things\nwill get you there.\n\nIf what you want is \"one central branch that currently looks like\nfirst-svn/trunk or second-svn/trunk or maybe something else, but we'll\nbe merging future changes to first-svn and second-svn into it in the\nfuture\", then you would do:\n\n git checkout -b one-true-branch\n ...make it look however you want...\n # now mark it as up-to-date with svn, but don't change anything\n git merge -s ours first-svn/trunk\n git merge -s ours second-svn/trunk\n\nAnd then in the future, whenever first-svn/trunk or second-svn/trunk\nchange, you would do:\n\n git merge first-svn/trunk\n git merge second-svn/trunk\n\netc.\n\n> I have a hard time to adopt my mental model to the one-branch method for\n> some reason.  OTOH, I can easily understand the multiple-branch method:\n> for every remote branch, I have a local branch on which I collect the\n> commits that should go to this remote.\n\nIt's indeed pretty complicated. I had much better luck once I finally\nseparated the two concepts in my mind. In general, you want to name\nlocal branches after what they *do*; you almost never have a 1:1\nmapping between local and remote branches, particularly when using\ngit-svn (at least, when using git-svn with merges instead of rebases).\n\nMaybe think of it like this: you're not \"collecting commits.\" You're\nsimply merging changes from one place to another, and testing them\nout. Once you're happy with what's on your local branch, you want to\nmerge it (--no-ff) into the remote branch and git svn dcommit it. svn\nwill never see the individual commits; it only sees the merge commits.\n\n>> >> >> As long as you \"git config merge.summary true\" (to make the merge\n>> >> >> commit list all the commits it's merging)\n[...]\n>> In fact, it *only* affects the svn log.  Otherwise svn log ends up\n>> with a useless commit that says \"Merged from commit (giant hex\n>> string)\", and you can't actually do anything with the giant hex string\n>> because svn doesn't know what it is.\n>\n> OK.  I just noticed the list is limited to 22 entries.  Can this be\n> configured somehow to contain the complete list?\n\nAh, that. I don't think there's an obvious way. I forgot I did it\nuntil now, but my copy of git is patched to \"fix\" this. My patch is\nattached below.\n\n>> > The people are not uncooperative.  It is just that there's no way to\n>> > completely separate the public and private content.\n>>\n>> There is, if you're willing to do it.  The usual way is two have two\n>> branches: public and private.\n>\n> Well, my plan was to have one (generic) public repository that contains\n> templates instead of the localized information.  Separating the\n> repositories is a security measure here.\n>\n> Whether separate repositories or only different branches, conflicts\n> _are_ to be expected in this area.\n\nConflicts are normal. But you can simply resolve them, using normal\ngit mechanisms, when merging from public-svn/trunk to\nsecond-svn/trunk, or whatever. At least you shouldn't have to\ncherry-pick or rebase anything.\n\nHave fun,\n\nAvery\n\n\ndiff --git a/builtin-fmt-merge-msg.c b/builtin-fmt-merge-msg.c\nindex df18f40..96c42ff 100644\n--- a/builtin-fmt-merge-msg.c\n+++ b/builtin-fmt-merge-msg.c\n@@ -255,7 +255,7 @@ static void shortlog(const char *name, unsigned char *sha1,\n }\n\n int fmt_merge_msg(int merge_summary, struct strbuf *in, struct strbuf *out) {\n- int limit = 20, i = 0, pos = 0;\n+ int limit = 2000, i = 0, pos = 0;\n char line[1024];\n char *p = line, *sep = \"\";\n unsigned char head_sha1[20];\n"},{"id":"112823","messageId":"20090501142811.GI15420@raven.wolf.lan","threadId":"19090","inReplyTo":"32541b130904301559w329bdd4bo6f2736a505b7235f@mail.gmail.com","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Josef Wolf","fromEmail":"jw@raven.inka.de","sentAt":"2009-05-01T14:28:11Z","receivedAt":"2009-05-01T14:28:11Z","isPatch":false,"sender":{"key":"jw@raven.inka.de","avatar":null},"body":"On Thu, Apr 30, 2009 at 06:59:50PM -0400, Avery Pennarun wrote:\n> On Thu, Apr 30, 2009 at 6:28 PM, Josef Wolf wrote:\n> >  # cherry-pick from second-svn to first-svn\n> >  #\n> >  git svn checkout first-svn\n> >  git cherry-pick sha1-from-second-svn # repeat as needed\n> >  git checkout first-svn/trunk\n> >  git merge --no-ff first-svn\n> >  git diff first-svn/trunk first-svn >changes.diff\n> >  git svn dcommit\n> [...]\n> > But I am still somewhat confused:\n> >\n> >  git log -1 first-svn/trunk\n> >\n> > says \"Merge branch first-svn into HEAD\".  But this does not reflect\n> > what I've actually done: I've picked _from_ second-svn and committed\n> > that _to_ first-svn.\n> \n> The most recent commit to first-svn/trunk was \"git merge --no-ff\n> first-svn\", which creates the merge commit you're seeing here. (HEAD\n> == first-svn/trunk). So this sounds right to me.\n> \n> \"git log -1 first-svn\" would give you the first cherry-pick. But\n> remember, it's a completely different branch.\n\nI can see why this happens, but I still find it confusing. Maybe I\nshould help with the -m option?\n\n> >> What you probably thought you should do, given that the existing\n> >> git-svn documentation says to do it, is more like this:\n> >>\n> >>    # WRONG\n> >>    git checkout first-svn\n> >>    git cherry-pick some stuff\n> >>    git merge [perhaps -s ours] second-svn/trunk\n> >>    git svn dcommit\n> >\n> > Almost... In addition, I was trying to \"git svn rebase\" before the\n> > dcommit\n> \n> 'git svn dcommit' implies 'git svn rebase' first anyway, so it's the same.\n> \n> > What would happen if somebody else creates a new commit just after I\n> > \"git svn fetch\" but before I dcommit?  Guess, svn will not accept this\n> > commit, because it is based on an outdated revision.  How would I\n> > get out from this situation?\n> \n> AFAIK, it will attempt to do \"git svn rebase\" first, and if that\n> succeeds, it will do the commit.\n> \n> In such a case, the rebase should be okay, because it's only changing\n> commits (in fact, just one commit: the merge commit) that don't exist\n> on any other branch. Thus it won't mangle any other merges.\n\nYeah, that's the simple case. But what if the rebase don't succeed?\n\n> > Ummm, no.. I have _two_ branches:\n> >\n> >  first-svn:  contains the cherries that I picked from second-svn. This\n> >              branch looks the way first-svn/trunk should be\n> >  second-svn: contains the cherries that I picked from first-svn. This\n> >              looks the way second-svn/trunk should be\n> \n> Okay, if you want to end up with two different remote branches, it\n> makes sense to have two different local branches.\n\nWell, I _have_ two different remotes because I have two svn repositories.\n\n> > Don't I need to rebase at least one of them if I want to \"merge\" those\n> > two branches into a single one?\n> \n> I don't think so. If you merge them together, what do you *want* it\n> to look like? And what do you want to do with that branch afterwards?\n> It's hard for me to guess, but it seems unlikely that rebasing things\n> will get you there.\n\nThen I have to keep both local branches. But I still wonder why you\nsuggested to go with _one_ local branch.\n\n> If what you want is \"one central branch that currently looks like\n> first-svn/trunk or second-svn/trunk or maybe something else, but we'll\n> be merging future changes to first-svn and second-svn into it in the\n> future\", then you would do:\n> \n> git checkout -b one-true-branch\n> ...make it look however you want...\n> # now mark it as up-to-date with svn, but don't change anything\n> git merge -s ours first-svn/trunk\n> git merge -s ours second-svn/trunk\n> \n> And then in the future, whenever first-svn/trunk or second-svn/trunk\n> change, you would do:\n> \n> git merge first-svn/trunk\n> git merge second-svn/trunk\n> \n> etc.\n\nI guess that might be interesting for the generic branch, where all the\nlocalizations are replaced by templates.\n"},{"id":"112834","messageId":"32541b130905011217x7f339d41x696fedee7298e3a4@mail.gmail.com","threadId":"19090","inReplyTo":"20090501142811.GI15420@raven.wolf.lan","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2009-05-01T19:17:14Z","receivedAt":"2009-05-01T19:17:14Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Fri, May 1, 2009 at 10:28 AM, Josef Wolf wrote:\n> On Thu, Apr 30, 2009 at 06:59:50PM -0400, Avery Pennarun wrote:\n>> \"git log -1 first-svn\" would give you the first cherry-pick.  But\n>> remember, it's a completely different branch.\n>\n> I can see why this happens, but I still find it confusing.  Maybe I\n> should help with the -m option?\n\nI don't know what -m does. Maybe try looking at the graph with gitk;\nthat might give some clues.\n\n>> AFAIK, it will attempt to do \"git svn rebase\" first, and if that\n>> succeeds, it will do the commit.\n>>\n>> In such a case, the rebase should be okay, because it's only changing\n>> commits (in fact, just one commit: the merge commit) that don't exist\n>> on any other branch.  Thus it won't mangle any other merges.\n>\n> Yeah, that's the simple case.  But what if the rebase don't succeed?\n\nThen you'll get a conflict, and you'll have to fix it first before you\ncan dcommit.\n\n>> Okay, if you want to end up with two different remote branches, it\n>> makes sense to have two different local branches.\n>\n> Well, I _have_ two different remotes because I have two svn repositories.\n\nRight. I was just wondering whether you wanted the two branches'\ncontents to be *different* or identical. I guess different.\n\n> Then I have to keep both local branches.  But I still wonder why you\n> suggested to go with _one_ local branch.\n\nFor my own purposes, I try not to create a 1:1 mapping between local\nbranches and remote branches; this just ends up being confusing,\nbecause I can have commits in my local branch that aren't in the\nremote one, and vice versa. So it's not very useful to create a local\nbranch *just* because I have a corresponding remote branch.\n\nIn your case, you might want to have just a single local branch for\nyour \"public\" stuff. You would then merge changes from the two svn\nremote branches into your local branch, and you'd also merge from your\nlocal branch into your remote branches (using a disconnected HEAD and\nsvn dcommit).\n\nHave fun,\n\nAvery\n"},{"id":"112889","messageId":"20090502215852.GJ15420@raven.wolf.lan","threadId":"19090","inReplyTo":"32541b130905011217x7f339d41x696fedee7298e3a4@mail.gmail.com","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Josef Wolf","fromEmail":"jw@raven.inka.de","sentAt":"2009-05-02T21:58:52Z","receivedAt":"2009-05-02T21:58:52Z","isPatch":false,"sender":{"key":"jw@raven.inka.de","avatar":null},"body":"On Fri, May 01, 2009 at 03:17:14PM -0400, Avery Pennarun wrote:\n> On Fri, May 1, 2009 at 10:28 AM, Josef Wolf wrote:\n> > On Thu, Apr 30, 2009 at 06:59:50PM -0400, Avery Pennarun wrote:\n> >> \"git log -1 first-svn\" would give you the first cherry-pick.  But\n> >> remember, it's a completely different branch.\n> >\n> > I can see why this happens, but I still find it confusing.  Maybe I\n> > should help with the -m option?\n> \n> I don't know what -m does. Maybe try looking at the graph with gitk;\n> that might give some clues.\n\nOption -m lets me set the log message explicitly :)\n\n> >> Okay, if you want to end up with two different remote branches, it\n> >> makes sense to have two different local branches.\n> >\n> > Well, I _have_ two different remotes because I have two svn repositories.\n> \n> Right. I was just wondering whether you wanted the two branches'\n> contents to be *different* or identical. I guess different.\n\nThey have to stay different, because they are localized.\n\n> > Then I have to keep both local branches.  But I still wonder why you\n> > suggested to go with _one_ local branch.\n> \n> For my own purposes, I try not to create a 1:1 mapping between local\n> branches and remote branches; this just ends up being confusing,\n> because I can have commits in my local branch that aren't in the\n> remote one, and vice versa. So it's not very useful to create a local\n> branch *just* because I have a corresponding remote branch.\n> \n> In your case, you might want to have just a single local branch for\n> your \"public\" stuff. You would then merge changes from the two svn\n> remote branches into your local branch, and you'd also merge from your\n> local branch into your remote branches (using a disconnected HEAD and\n> svn dcommit).\n\nBut I am working not only on the \"public\" stuff. Additionally, I am\nworking on _multiple_ localized stuff. Thus, I have multiple remote\nrepositories.\n\nSomehow, I still can't get it work. This is what I do:\n\n # create the repos\n #\n git svn init --stdlayout file:///var/tmp/builds/git-sync/svn/svn-1\n git config merge.stat true\n\n # add configuration for svn-1 repos\n #\n git config svn-remote.svn-1.url file:///var/tmp/builds/git-sync/svn/svn-1\n git config svn-remote.svn-1.fetch trunk:refs/remotes/svn-1/trunk\n git config svn-remote.svn-1.branches branches/*:refs/remotes/svn-1/*\n git config svn-remote.svn-1.tags tags/*:refs/remotes/svn-1/tags/*\n\n # add configuration for svn-2 repos\n #\n git config svn-remote.svn-2.url file:///var/tmp/builds/git-sync/svn/svn-2\n git config svn-remote.svn-2.fetch trunk:refs/remotes/svn-2/trunk\n git config svn-remote.svn-2.branches branches/*:refs/remotes/svn-2/*\n git config svn-remote.svn-2.tags tags/*:refs/remotes/svn-2/tags/*\n\n # fetch the commits from svn repositories\n #\n git svn fetch -R svn-1\n git svn fetch -R svn-2\n\n # create local tracking branches\n #\n git checkout -b svn-1 svn-1/trunk\n git checkout -b svn-2 svn-2/trunk\n\n # just to see what we've done\n #\n git tag svn-1-orig svn-1\n git tag svn-2-orig svn-2\n\n # move stuff from svn-2 to svn-1\n #\n git svn fetch svn-2\n git checkout svn-1\n git cherry-pick 05b964\n [ continue cherry-picking ]\n git merge --no-ff -s ours svn-1\n\n # check what I have done\n #\n git diff svn-1-orig svn-1/trunk # shows what I expect\n\n # move the result to svn-1\n #\n git checkout svn-1/trunk\n git merge --no-ff svn-1\n git svn dcommit\n\n # move stuff from svn-1 to svn-2\n #\n git svn fetch svn-1\n git checkout svn-2\n git cherry-pick -n c9dae\n [ continue cherry-picking ]\n git merge --no-ff -s ours svn-2\n\n # check what I have done\n #\n git diff svn-2-orig svn-2/trunk # shows what I expect\n\n # move the result to svn-2\n #\n git checkout svn-2/trunk\n git merge --no-ff svn-2\n git svn dcommit\n\nAt this point, we should be synchronized.\n\n git checkout svn-2/trunk\n git svn fetch svn-1\n git merge --no-ff svn-1\n\nBOOM. Although no new commits were fetched, we get a lot of conflicts\nhere. So git is not fully aware about the fact that we are synchronized.\n"},{"id":"112993","messageId":"32541b130905040858v2a0b7a6br5b056a365fcb6855@mail.gmail.com","threadId":"19090","inReplyTo":"20090502215852.GJ15420@raven.wolf.lan","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2009-05-04T15:58:20Z","receivedAt":"2009-05-04T15:58:20Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Sat, May 2, 2009 at 5:58 PM, Josef Wolf wrote:\n> Somehow, I still can't get it work.  This is what I do:\n[...]\n>  # move stuff from svn-2 to svn-1\n>  #\n>  git svn fetch svn-2\n>  git checkout svn-1\n>  git cherry-pick 05b964\n>  [  continue cherry-picking ]\n>  git merge --no-ff -s ours svn-1\n\nNote that you probably should be merging '-s ours svn-2' here, not\nsvn-1. svn-1 already contains svn-1 (of course) so that merge didn't\ndo anything. It most especially doesn't mark svn-1 as being\nup-to-date with svn-2, and that's probably going to make trouble\nlater.\n\n>  # check what I have done\n>  #\n>  git diff svn-1-orig svn-1/trunk # shows what I expect\n\nThis is unsurprising, since you haven't changed either branch during the above.\n\n>  # move the result to svn-1\n>  #\n>  git checkout svn-1/trunk\n>  git merge --no-ff svn-1\n>  git svn dcommit\n\nThis looks ok.\n\n>  # move stuff from svn-1 to svn-2\n>  #\n>  git svn fetch svn-1\n>  git checkout svn-2\n>  git cherry-pick -n c9dae\n>  [ continue cherry-picking ]\n>  git merge --no-ff -s ours svn-2\n\nAgain, you seem to have merged in the wrong branch here. This one\nshould be svn-1.\n\n>  # check what I have done\n>  #\n>  git diff svn-2-orig svn-2/trunk # shows what I expect\n\nAgain, these branches haven't changed, so no surprise here either.\n\n>  # move the result to svn-2\n>  #\n>  git checkout svn-2/trunk\n>  git merge --no-ff svn-2\n>  git svn dcommit\n\nThis seems ok.\n\n> At this point, we should be synchronized.\n\nYes, although there are no merges between svn-1 and svn-2, so the next\nattempt at merging will merge *everything*, causing conflicts.\n\n>  git checkout svn-2/trunk\n>  git svn fetch svn-1\n>  git merge --no-ff svn-1\n>\n> BOOM.  Although no new commits were fetched, we get a lot of conflicts\n> here.  So git is not fully aware about the fact that we are synchronized.\n\nYou seem to almost have it. Fix the -s ours merges above and I think\nyou'll be in business.\n\nHave fun,\n\nAvery\n"},{"id":"113005","messageId":"20090504211423.GK15420@raven.wolf.lan","threadId":"19090","inReplyTo":"32541b130905040858v2a0b7a6br5b056a365fcb6855@mail.gmail.com","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Josef Wolf","fromEmail":"jw@raven.inka.de","sentAt":"2009-05-04T21:14:23Z","receivedAt":"2009-05-04T21:14:23Z","isPatch":false,"sender":{"key":"jw@raven.inka.de","avatar":null},"body":"On Mon, May 04, 2009 at 11:58:20AM -0400, Avery Pennarun wrote:\n> On Sat, May 2, 2009 at 5:58 PM, Josef Wolf wrote:\n> > Somehow, I still can't get it work.  This is what I do:\n> [...]\n> >  # move stuff from svn-2 to svn-1\n> >  #\n> >  git svn fetch svn-2\n> >  git checkout svn-1\n> >  git cherry-pick 05b964\n> >  [  continue cherry-picking ]\n> >  git merge --no-ff -s ours svn-1\n> \n> Note that you probably should be merging '-s ours svn-2' here, not\n> svn-1. svn-1 already contains svn-1 (of course) so that merge didn't\n> do anything. It most especially doesn't mark svn-1 as being\n> up-to-date with svn-2, and that's probably going to make trouble\n> later.\n\nYeah, you're right. That was a typo.\n\nFixing this, the \"getting started\" seems to work now: the cherry-picked\ncommits end up in the svn repositories. But the synchronization after\nthe \"getting started\" does not seem to work yet. Here's what I've done:\n\n git tag svn-1-orig svn-1\n git tag svn-2-orig svn-2\n\n # move cherries to svn-2\n #\n git svn fetch svn-1\n git checkout svn-2\n git cherry-pick c9da\n [ ... ]\n git merge --no-ff -s ours svn-1\n git checkout svn-2/trunk\n git merge --no-ff svn-2\n git svn dcommit\n git diff svn-2-orig svn-2/trunk # check what I've done\n\n # move cherries to svn-1\n #\n git svn fetch svn-2\n git checkout svn-1\n git cherry-pick 05b9\n [ ... ]\n git merge --no-ff -s ours svn-2\n git checkout svn-1/trunk\n git merge --no-ff svn-1\n git svn dcommit\n git diff svn-1-orig svn-1/trunk # check what I've done\n\n git diff svn-1/trunk svn-2/trunk # shows the diffs I want to keep\n\n # now try a synchronization\n #\n git checkout svn-2/trunk\n git svn fetch svn-1 # nothing new was checked in yet\n git merge --no-ff svn-1\n git svn dcommit\n\nSince no new commits were made in svn, those four commands should be\nalmost a no-op.\n\nBut Instead of merging only the changes that were done after the last\nsynchronization, the last dcommit makes svn-2/trunk identical to svn-1.\nThis effectively wipes all the differences which I would like to keep.\n"},{"id":"113113","messageId":"20090506185224.GM15420@raven.wolf.lan","threadId":"19090","inReplyTo":"20090504211423.GK15420@raven.wolf.lan","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Josef Wolf","fromEmail":"jw@raven.inka.de","sentAt":"2009-05-06T18:52:24Z","receivedAt":"2009-05-06T18:52:24Z","isPatch":false,"sender":{"key":"jw@raven.inka.de","avatar":null},"body":"On Mon, May 04, 2009 at 11:14:23PM +0200, Josef Wolf wrote:\n\nI am still trying to understand what is going on here, so I tried to\ndraw the history-graph. It turns out that after the \"getting-started\"\ncommands is not what I would have expected:\n\n> git tag svn-1-orig svn-1\n> git tag svn-2-orig svn-2\n> \n> # move cherries to svn-2\n> #\n> git svn fetch svn-1\n> git checkout svn-2\n> git cherry-pick c9da\n> [ ... ]\n> git merge --no-ff -s ours svn-1\n> git checkout svn-2/trunk\n> git merge --no-ff svn-2\n> git svn dcommit\n> \n> # move cherries to svn-1\n> #\n> git svn fetch svn-2\n> git checkout svn-1\n> git cherry-pick 05b9\n> [ ... ]\n> git merge --no-ff -s ours svn-2\n> git checkout svn-1/trunk\n> git merge --no-ff svn-1\n> git svn dcommit\n\nHere's what I have at this point:\n\n ------------------S1TRUNK\n / /\n --hs1--O1--c2...c2-------S1\n \\ /\n `+++++++. /\n \\ /\n --hs2--O2--c1...c1--S2\n \\ \\\n -------------S2TRUNK\n\n hs1, hs2: history imported from svn-1 and svn2, respectively\n O1, O2: the svn-1-orig and svn-2-orig tags\n c1, c2: cherries picked from hs1 and hs2, respectively\n S1, S2: svn-1 and svn-2, the local tracking branches\n S1TRUNK, S2TRUNK: the remotes/svn-X/trunk branches\n\nI would have expected a symmetrical diagram. But it turns out that the\nconnection marked with plusses is still at O1 instead of S1. So it takes\nno wonder that the c2 cherries get re-applied to the s2 branch on the\nnext merge.\n\nIs this understanding somewhat plausible? Any ideas how to get this\nthis fixed?\n"},{"id":"113115","messageId":"32541b130905061223h7efddeecvcc52a369093a6b50@mail.gmail.com","threadId":"19090","inReplyTo":"20090506185224.GM15420@raven.wolf.lan","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2009-05-06T19:23:40Z","receivedAt":"2009-05-06T19:23:40Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Wed, May 6, 2009 at 2:52 PM, Josef Wolf wrote:\n> Here's what I have at this point:\n>\n>             ------------------S1TRUNK\n>            /                 /\n>   --hs1--O1--c2...c2-------S1\n>            \\              /\n>             `+++++++.    /\n>                      \\  /\n>   --hs2--O2--c1...c1--S2\n>            \\            \\\n>             -------------S2TRUNK\n>\n>   hs1, hs2:         history imported from svn-1 and svn2, respectively\n>   O1, O2:           the svn-1-orig and svn-2-orig tags\n>   c1, c2:           cherries picked from hs1 and hs2, respectively\n>   S1, S2:           svn-1 and svn-2, the local tracking branches\n>   S1TRUNK, S2TRUNK: the remotes/svn-X/trunk branches\n>\n> I would have expected a symmetrical diagram.  But it turns out that the\n> connection marked with plusses is still at O1 instead of S1.  So it takes\n> no wonder that the c2 cherries get re-applied to the s2 branch on the\n> next merge.\n\nThat's a well-drawn diagram, but unfortunately I'm still confused.\nWhat is the \"connection marked with plusses\" and does it have a name?\nIt *looks* to me like both S1TRUNK and S2TRUNK should be okay, but\nit's hard to tell what has actually happened here.\n\nIf you could post a screenshot of 'gitk --all' it might help.\n\nAvery\n"},{"id":"113144","messageId":"20090506225042.GN15420@raven.wolf.lan","threadId":"19090","inReplyTo":"32541b130905061223h7efddeecvcc52a369093a6b50@mail.gmail.com","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Josef Wolf","fromEmail":"jw@raven.inka.de","sentAt":"2009-05-06T22:50:42Z","receivedAt":"2009-05-06T22:50:42Z","isPatch":false,"sender":{"key":"jw@raven.inka.de","avatar":null},"body":"On Wed, May 06, 2009 at 03:23:40PM -0400, Avery Pennarun wrote:\n> On Wed, May 6, 2009 at 2:52 PM, Josef Wolf wrote:\n> > Here's what I have at this point:\n> >\n> >             ------------------S1TRUNK\n> >            /                 /\n> >   --hs1--O1--c2...c2-------S1\n> >            \\              /\n> >             `+++++++.    /\n> >                      \\  /\n> >   --hs2--O2--c1...c1--S2\n> >            \\            \\\n> >             -------------S2TRUNK\n> >\n> >   hs1, hs2:         history imported from svn-1 and svn2, respectively\n> >   O1, O2:           the svn-1-orig and svn-2-orig tags\n> >   c1, c2:           cherries picked from hs1 and hs2, respectively\n> >   S1, S2:           svn-1 and svn-2, the local tracking branches\n> >   S1TRUNK, S2TRUNK: the remotes/svn-X/trunk branches\n> >\n> > I would have expected a symmetrical diagram.  But it turns out that the\n> > connection marked with plusses is still at O1 instead of S1.  So it takes\n> > no wonder that the c2 cherries get re-applied to the s2 branch on the\n> > next merge.\n> \n> That's a well-drawn diagram, but unfortunately I'm still confused.\n> What is the \"connection marked with plusses\" and does it have a name?\n\nWell, the whole history (including this connection) was created by the\ncommands I posted. The only exception are hs1 and hs2, which were\nimported from the svn repositories (but they are linear).\n\nAFAICS, this connection was created by the very first merge after the\ncherry-picking from hs1 (the cherries marked as c1..c1). After importing\nthe svn repositories, I've done this:\n\n git checkout svn-2 # S2 in the diagram above, but used\n # to be identical to O2 at that time\n [ cherry-picking c1...c1 ] # S2 now moved to its place in diagram\n git merge --no-ff -s ours svn-1 # S1 in the diagram above, but used\n # to be identical to O1 at the\n # time of that merge, This merge\n # creates the mystical\n # \"connection marked with plusses\"\n\nThen we've done the \"detached head\" merge, that created the S2TRUNK\n\n git checkout svn-2/trunk # S2TRUNK was at O2 at that time\n git merge --no-ff svn-2\n git svn dcommit # moves S2TRUNK to the place in the diagram\n\nNow take care of the other direction:\n\n git checkout svn-1 # S1 in the diagram above, but still\n # identical to O1 at that time\n [ cherry-picking c2...c2 ] # S1 now moved to its place in diagram\n git merge --no-ff -s ours svn-2 # S2 in the diagram above. Unlike S1,\n # S2 already _is_ at the place where\n # it is drawn in the diagram. So\n # this merge creates the connection\n # S2->S1\n\nNow we do the \"detached head\" merge, that creates the S1TRUNK\n\n git checkout svn-1/trunk # S1TRUNK was at O1 at that time\n git merge --no-ff svn-1\n git svn dcommit # moves S1TRUNK to the place in the diagram\n\nSo the \"connection marked with plusses\" is basically the counterpart of\nthe \"S2->S1\" connection. But while \"S2->S1\" got its proper position at\nthe time it was created, the plus-connection was created before the c2\ncherries. And it was never adjusted. AFAICS, those two connections\nshould be symmetrical: \"S1->S2\" and \"S2->S1\".\n\n> It *looks* to me like both S1TRUNK and S2TRUNK should be okay, but\n> it's hard to tell what has actually happened here.\n\nYes, the trunks (and the svn repositories) look pretty good at _that_\npoint in time. But the next merge on S2TRUNK moves all the modifications\ndone by the c2 cherries down to S2TRUNK.\n\n> If you could post a screenshot of 'gitk --all' it might help.\n\nIMHO, Screenshots are not of much help here. That's why I posted\n\n http://www.spinics.net/lists/git/msg102609.html\n\nThe svn histories are about 1250 commits each. The cherry-pickings are\nabout 350 commits each. This gives histories running in parallel for\nlong distances. Add to this gitk's tendency to change lanes at every\noccasion: There's no chance to get multiple screen shots (the interesting\nbranch/merge-points, as I described in the thread referenced above) in\nsync. There's many opportunities to get confused. At least for me, as a\nnewbie to git.\n"},{"id":"113369","messageId":"32541b130905081344m634a78d5l984f4903ec1515eb@mail.gmail.com","threadId":"19090","inReplyTo":"20090506225042.GN15420@raven.wolf.lan","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2009-05-08T20:44:39Z","receivedAt":"2009-05-08T20:44:39Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Wed, May 6, 2009 at 6:50 PM, Josef Wolf wrote:\n> The svn histories are about 1250 commits each.  The cherry-pickings are\n> about 350 commits each.  This gives histories running in parallel for\n> long distances.  Add to this gitk's tendency to change lanes at every\n> occasion:  There's no chance to get multiple screen shots (the interesting\n> branch/merge-points, as I described in the thread referenced above) in\n> sync.  There's many opportunities to get confused.  At least for me, as a\n> newbie to git.\n\nI agree that gitk's lane-changing can be a bit confusing. Could you\ntry making a slightly modified version of your script, where you only\ncherry-pick one or two commits in each direction? That should be\nfunctionally identical, but a much simpler diagram.\n\nAvery\n"},{"id":"113383","messageId":"20090508235821.GO15420@raven.wolf.lan","threadId":"19090","inReplyTo":"32541b130905081344m634a78d5l984f4903ec1515eb@mail.gmail.com","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Josef Wolf","fromEmail":"jw@raven.inka.de","sentAt":"2009-05-08T23:58:22Z","receivedAt":"2009-05-08T23:58:22Z","isPatch":false,"sender":{"key":"jw@raven.inka.de","avatar":null},"body":"On Fri, May 08, 2009 at 04:44:39PM -0400, Avery Pennarun wrote:\n> I agree that gitk's lane-changing can be a bit confusing. Could you\n> try making a slightly modified version of your script, where you only\n> cherry-pick one or two commits in each direction? That should be\n> functionally identical, but a much simpler diagram.\n\nSomehow I can't keep git-svn from fetching all the svn revisions, thus\nit's still split over a long distance.\n\nBut in the meantime, I've hacked a quick-n-dirty script to show only\nthe 'interesting' commits with an optional context. This helped me a\nlot to get a better understanding what's going on. I have appended the\nresult at the end of this mail. Please convert it with \"dot -Tps\" to\npostscript to view the result.\n\nSo here I go again with my attempt to analyze what happens. I attach\nthe sha1 of created commits as a comment to the command that creates the\ncommit.\n\n # Create tags so we can see later what we have done\n #\n git tag svn-1-orig svn-1\n git tag svn-2-orig svn-2\n\n # move cherries from svn-1 to svn-2\n #\n git svn fetch svn-1\n git checkout svn-2\n [ cherry-picking, creates 67446..0a742 ]\n git merge --no-ff -s ours svn-1 -m 'merge ours svn-1 to svn-2' # 5d9a0\n git checkout svn-2/trunk\n git merge --no-ff svn-2 -m 'merge svn-1 to svn-2' # f80d2\n git svn dcommit\n\n # check the results\n #\n git diff svn-2-orig svn-2/trunk\n git diff svn-2-orig svn-2\n\n # move cherries from svn-1 to svn-2\n #\n git svn fetch svn-2\n git checkout svn-1\n [ cherry-picking, creates a5cf3..c3ff2 ]\n git merge --no-ff -s ours svn-2 -m 'merge ours svn-2 to svn-1' # 2379d\n git checkout -q svn-1/trunk\n git merge --no-ff svn-1 -m 'merge svn-2 to svn-1' # 693fa\n git svn dcommit --no-rebase\n\n # again, check the results\n #\n git diff svn-1-orig svn-1/trunk\n git diff svn-1-orig svn-1\n\nAt this time, I made the graph attached below. Two things are\ninteresting in this graph:\n- svn-1 has all the imported commits and all the cherries as parent\n svn-2 does _not_ have a5cf3..c3ff2 as parent\n- same thing happens for remotes/svn-2/trunk vs. remotes/svn-2/trunk\n\n\nNow, when I do\n\n git checkout svn-2 # same thing happens when svn-2/trunk is used\n git merge --no-ff svn-1\n\nAnd here I'm completely barfed. The tree is set _identical_ to the tree\nin 2379d. All the differences which should be kept are lost here.\nI can easily see (although I don't know how to avoid it) why a5cf3..c3ff2\nmight be applied to svn-2 and svn-2/trunk though it should not be applied.\n\nBut I completely fail to see why the tree is set identical to 2379d.\n\n\nHere is an overview of the created commits:\n\n# b8bf1, 8536f..09393, d0f29 imported from svn-1 repository\n# 7b397, 17156..e0772, 05eb1 imported from svn-2 repository\n# a5cf3..c3ff2 cherries from svn-2 to svn-1\n# 67446..0a742 cherries from svn-1 to svn-2\n# 5d9a0 merge ours svn-1 to svn-2\n# 693fa merge svn-2 to svn-1\n# 2379d merge ours svn-2 to svn-1\n# f80d2 merge svn-1 to svn-2\n\nAnd here's the .dot graph. Please pipe it through \"dot -Tps\" to create\na postscript file of the graph.\n\nstrict digraph G {\n size = \"7,10\"\n \"8536f\" [label=\"8536f\"] ;\n \"b8bf1\"->\"8536f\" ;\n \"693fa\" [label=\"693fa\\nremotes/svn-1/trunk\"] ;\n \"d0f29\"->\"693fa\" ;\n \"2379d\"->\"693fa\" ;\n \"5d9a0\" [label=\"5d9a0\\nsvn-2\"] ;\n \"d0f29\"->\"5d9a0\" ;\n \"0a742\"->\"5d9a0\" ;\n \"f80d2\" [label=\"f80d2\\nremotes/svn-2/trunk\"] ;\n \"05eb1\"->\"f80d2\" ;\n \"5d9a0\"->\"f80d2\" ;\n \"b8bf1\" [label=\"b8bf1\"] ;\n \"09393\" [label=\"09393\"] ;\n \"8536f\"->\"09393\" [style=\"dotted\"] ;\n \"05eb1\" [label=\"05eb1\\nsvn-2-orig\"] ;\n \"e0772\"->\"05eb1\" ;\n \"67446\" [label=\"67446\"] ;\n \"05eb1\"->\"67446\" ;\n \"d0f29\" [label=\"d0f29\\nmaster\\nsvn-1-orig\"] ;\n \"09393\"->\"d0f29\" ;\n \"a5cf3\" [label=\"a5cf3\"] ;\n \"d0f29\"->\"a5cf3\" ;\n \"17156\" [label=\"17156\"] ;\n \"7b397\"->\"17156\" ;\n \"2379d\" [label=\"2379d\\nsvn-1\"] ;\n \"c3ff2\"->\"2379d\" ;\n \"5d9a0\"->\"2379d\" ;\n \"0a742\" [label=\"0a742\"] ;\n \"67446\"->\"0a742\" [style=\"dotted\"] ;\n \"c3ff2\" [label=\"c3ff2\"] ;\n \"a5cf3\"->\"c3ff2\" [style=\"dotted\"] ;\n \"7b397\" [label=\"7b397\"] ;\n \"e0772\" [label=\"e0772\"] ;\n \"17156\"->\"e0772\" [style=\"dotted\"] ;\n}\n"},{"id":"113779","messageId":"20090513120922.GP15420@raven.wolf.lan","threadId":"19090","inReplyTo":"20090508235821.GO15420@raven.wolf.lan","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Josef Wolf","fromEmail":"jw@raven.inka.de","sentAt":"2009-05-13T12:09:22Z","receivedAt":"2009-05-13T12:09:22Z","isPatch":false,"sender":{"key":"jw@raven.inka.de","avatar":null},"body":"On Sat, May 09, 2009 at 01:58:22AM +0200, Josef Wolf wrote:\n\nAfter lots of trial-and-error, I guess I've located the reason of\none of the failures. Please see below: \n\n> # Create tags so we can see later what we have done\n> #\n> git tag svn-1-orig svn-1\n> git tag svn-2-orig svn-2\n> \n> # move cherries from svn-1 to svn-2\n> #\n> git svn fetch svn-1\n> git checkout svn-2\n> [ cherry-picking, creates 67446..0a742 ]\n> git merge --no-ff -s ours svn-1 -m 'merge ours svn-1 to svn-2' # 5d9a0\n> git checkout svn-2/trunk\n> git merge --no-ff svn-2 -m 'merge svn-1 to svn-2' # f80d2\n> git svn dcommit\n> \n> # check the results\n> #\n> git diff svn-2-orig svn-2/trunk\n> git diff svn-2-orig svn-2\n> \n> # move cherries from svn-1 to svn-2\n> #\n> git svn fetch svn-2\n> git checkout svn-1\n> [ cherry-picking, creates a5cf3..c3ff2 ]\n> git merge --no-ff -s ours svn-2 -m 'merge ours svn-2 to svn-1' # 2379d\n> git checkout -q svn-1/trunk\n> git merge --no-ff svn-1 -m 'merge svn-2 to svn-1' # 693fa\n> git svn dcommit --no-rebase\n> \n> # again, check the results\n> #\n> git diff svn-1-orig svn-1/trunk\n> git diff svn-1-orig svn-1\n> \n> At this time, I made the graph attached below. Two things are\n> interesting in this graph:\n> - svn-1 has all the imported commits and all the cherries as parent\n> svn-2 does _not_ have a5cf3..c3ff2 as parent\n> - same thing happens for remotes/svn-2/trunk vs. remotes/svn-2/trunk\n\nSo, in order to mark a5cf3..c3ff2 as ancestors of svn-2, I've inserted\nfollowing step at this place:\n\n git checkout svn-2\n git merge --no-ff -s ours svn-1\n\nThis works fine, a5cf3..c3ff2 are now recorded as ancestors of svn-2 and\nwill no longer be picked on future merges.\n\n git checkout svn-2/trunk\n git merge --no-ff -s ours svn-1\n git svn dcommit\n\nNow here's the problem: This last dcommit does simply a reset, because\nnothing has changed since the last dcommit. So a5cf3..c3ff2 are _not_\nmarked as ancestors of svn-2/trunk, causing those cherries to be rebased\nat the next dcommit with real changes.\n\nUnfortunately, dcommit doesn't seem to have an option to force rebase\ninstead of resetting.\n\nAny ideas how to mark those commits as ancestors of svn-2/trunk?\n\n> Here is an overview of the created commits:\n> \n> # b8bf1, 8536f..09393, d0f29 imported from svn-1 repository\n> # 7b397, 17156..e0772, 05eb1 imported from svn-2 repository\n> # a5cf3..c3ff2 cherries from svn-2 to svn-1\n> # 67446..0a742 cherries from svn-1 to svn-2\n> # 5d9a0 merge ours svn-1 to svn-2\n> # 693fa merge svn-2 to svn-1\n> # 2379d merge ours svn-2 to svn-1\n> # f80d2 merge svn-1 to svn-2\n> \n> And here's the .dot graph. Please pipe it through \"dot -Tps\" to create\n> a postscript file of the graph.\n> \n> strict digraph G {\n> size = \"7,10\"\n> \"8536f\" [label=\"8536f\"] ;\n> \"b8bf1\"->\"8536f\" ;\n> \"693fa\" [label=\"693fa\\nremotes/svn-1/trunk\"] ;\n> \"d0f29\"->\"693fa\" ;\n> \"2379d\"->\"693fa\" ;\n> \"5d9a0\" [label=\"5d9a0\\nsvn-2\"] ;\n> \"d0f29\"->\"5d9a0\" ;\n> \"0a742\"->\"5d9a0\" ;\n> \"f80d2\" [label=\"f80d2\\nremotes/svn-2/trunk\"] ;\n> \"05eb1\"->\"f80d2\" ;\n> \"5d9a0\"->\"f80d2\" ;\n> \"b8bf1\" [label=\"b8bf1\"] ;\n> \"09393\" [label=\"09393\"] ;\n> \"8536f\"->\"09393\" [style=\"dotted\"] ;\n> \"05eb1\" [label=\"05eb1\\nsvn-2-orig\"] ;\n> \"e0772\"->\"05eb1\" ;\n> \"67446\" [label=\"67446\"] ;\n> \"05eb1\"->\"67446\" ;\n> \"d0f29\" [label=\"d0f29\\nmaster\\nsvn-1-orig\"] ;\n> \"09393\"->\"d0f29\" ;\n> \"a5cf3\" [label=\"a5cf3\"] ;\n> \"d0f29\"->\"a5cf3\" ;\n> \"17156\" [label=\"17156\"] ;\n> \"7b397\"->\"17156\" ;\n> \"2379d\" [label=\"2379d\\nsvn-1\"] ;\n> \"c3ff2\"->\"2379d\" ;\n> \"5d9a0\"->\"2379d\" ;\n> \"0a742\" [label=\"0a742\"] ;\n> \"67446\"->\"0a742\" [style=\"dotted\"] ;\n> \"c3ff2\" [label=\"c3ff2\"] ;\n> \"a5cf3\"->\"c3ff2\" [style=\"dotted\"] ;\n> \"7b397\" [label=\"7b397\"] ;\n> \"e0772\" [label=\"e0772\"] ;\n> \"17156\"->\"e0772\" [style=\"dotted\"] ;\n> }\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at http://vger.kernel.org/majordomo-info.html\n"},{"id":"113811","messageId":"32541b130905131028i5c4b1a31j7f760f8157507df6@mail.gmail.com","threadId":"19090","inReplyTo":"20090513120922.GP15420@raven.wolf.lan","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2009-05-13T17:28:04Z","receivedAt":"2009-05-13T17:28:04Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Wed, May 13, 2009 at 8:09 AM, Josef Wolf wrote:\n> Now here's the problem:  This last dcommit does simply a reset, because\n> nothing has changed since the last dcommit.  So a5cf3..c3ff2 are _not_\n> marked as ancestors of svn-2/trunk, causing those cherries to be rebased\n> at the next dcommit with real changes.\n\nI find this a *bit* curious, since each dcommit should be adding the\ncherry-picked changes you just now picked from the opposite branch,\nright? If you weren't going to change anything, then you wouldn't\nhave needed to do the cherry picks at all; you could have just done a\nmerge -s ours in both directions in the first place.\n\nAnyway, regardless of the above, AFAIK there's no way to force svn to\nmake an empty commit, which is a problem in this case. You can make a\nnonempty commit, though; I've done this in the past by just adding a\nnewline to the end of some arbitrary file. Basically:\n\ngit merge -s ours whatever\necho >>Makefile\ngit add Makefile\ngit commit --amend\ngit svn dcommit\n\nSilly, but it works.\n\n> Unfortunately, dcommit doesn't seem to have an option to force rebase\n> instead of resetting.\n\nWell, in fact it *is* rebasing, which throws away the extra commit\nbecause it thinks that commit didn't do anything. I've experienced\nthis problem a few times in the past, but I knew what was happening\nand I figured my case was too rare to matter. Perhaps not.\n\nThis could be considered a bug in git-svn, so I cc:'d Eric Wong, who I\nthink is the main git-svn developer. Anyway, try the workaround\nabove.\n\nGood luck,\n\nAvery\n"},{"id":"113856","messageId":"20090513222243.GQ15420@raven.wolf.lan","threadId":"19090","inReplyTo":"32541b130905131028i5c4b1a31j7f760f8157507df6@mail.gmail.com","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Josef Wolf","fromEmail":"jw@raven.inka.de","sentAt":"2009-05-13T22:22:43Z","receivedAt":"2009-05-13T22:22:43Z","isPatch":false,"sender":{"key":"jw@raven.inka.de","avatar":null},"body":"Thanks for your patience, Avery! I would be completely lost here without\nyour help..\n\nOn Wed, May 13, 2009 at 01:28:04PM -0400, Avery Pennarun wrote:\n> On Wed, May 13, 2009 at 8:09 AM, Josef Wolf wrote:\n> > Now here's the problem:  This last dcommit does simply a reset, because\n> > nothing has changed since the last dcommit.  So a5cf3..c3ff2 are _not_\n> > marked as ancestors of svn-2/trunk, causing those cherries to be rebased\n> > at the next dcommit with real changes.\n> \n> I find this a *bit* curious, since each dcommit should be adding the\n> cherry-picked changes you just now picked from the opposite branch,\n> right?\n\nThey _are_ already added. I try to outline what happens based on the\ngraph I posted in my previous mail:\n\n # move cherries from svn-1 to svn-2\n #\n git svn fetch svn-1\n git checkout svn-2\n [ cherry-picking, creates 67446..0a742 ]\n git merge --no-ff -s ours svn-1 -m 'merge ours svn-1 to svn-2' # 5d9a0\n git checkout svn-2/trunk\n git merge --no-ff svn-2 -m 'merge svn-1 to svn-2' # f80d2\n git svn dcommit\n\nThis sequence moves cherries 67446..0a742 to svn-2/trunk and creates\n5d9a0+f80d2\n67446..0a742 are now ancestors of svn-2/trunk\na5cf3..c3ff2 are _not_ ancestors of svn-2/trunk (they don't even exist yet)\n\n # move cherries from svn-1 to svn-2\n #\n git svn fetch svn-2\n git checkout svn-1\n [ cherry-picking, creates a5cf3..c3ff2 ]\n git merge --no-ff -s ours svn-2 -m 'merge ours svn-2 to svn-1' # 2379d\n git checkout -q svn-1/trunk\n git merge --no-ff svn-1 -m 'merge svn-2 to svn-1' # 693fa\n git svn dcommit --no-rebase\n\nThis sequence moves cherries a5cf3..c3ff2 to svn-1/trunk and creates\n2379d+693fa\na5cf3..c3ff2 are now ancestors of svn-1/trunk\n67446..0a742 are also ancestors of svn-1/trunk since 5d9a0 pulls them in.\n\nPlease notice the asymmetry here. If I try to merge another change to\nsvn-2 at this point, dcommit tries to pull the cherries a5cf3..c3ff2.\nSince those cherries came initially from 17156..e0772, they are already\nincluded literally in the current tree and I get a lot of conflicts,\nTo tell git about the fact that those cherries are already available in\nsvn-2/trunk, I try yet another merge set of commands:\n\n git checkout svn-2\n git merge --no-ff -s ours svn-1\n\nThis works fine, a5cf3..c3ff2 are now recorded as ancestors of svn-2 and\nwill no longer be picked on future merges.\n\n git checkout svn-2/trunk\n git merge --no-ff -s ours svn-1\n git svn dcommit\n\nAnd _this_ is the dcommit I was talking about in the paragraph you cited\nabove. This dcommit notices that the resulting tree is identical to the\ntree already in svn, since it was merged with \"-s ours\" and no real change\nwas done in the mean time. So dcommit just resets and a5cf3..c3ff2 are\nstill not marked as ancestors of svn-2/trunk and would be pulled again\nat the next merge attempt, resulting in conflicts.\n\n> If you weren't going to change anything, then you wouldn't\n> have needed to do the cherry picks at all; you could have just done a\n> merge -s ours in both directions in the first place.\n\nThe cherries _are_ moved in both directions. But the ancestry is not yet\nadopted because at the time of the move from svn-1 to svn-2 the cherries\nthat were picked from svn-2 did not exist. Therefore dcommit wants to\nmove them to svn-2 (where the textual contents of those cherries already\nare).\n\n> Anyway, regardless of the above, AFAIK there's no way to force svn to\n> make an empty commit, which is a problem in this case. You can make a\n> nonempty commit, though; I've done this in the past by just adding a\n> newline to the end of some arbitrary file. Basically:\n> \n> git merge -s ours whatever\n> echo >>Makefile\n> git add Makefile\n> git commit --amend\n> git svn dcommit\n\nAh, I see :-) You can do a _lot_ with git if you know _how_ to do it ;-)\n\nYes, that helps a little bit: all the cherries are now ancestors of both\nremote branches and both local branches. But after this, all dcommits\ncomplain about outdated transactions although there were no commits\nto the svn repositories in the meantime:\n\n $ git merge --no-ff -s ours svn-1\n Merge made by ours. \n $ echo >>Makefile\n $ git add Makefile\n $ git commit --amend -m 'Force merge ours svn-1 to svn-2/trunk'\n Created commit ae455ca: Force merge ours svn-1 to svn-2/trunk\n $ git svn dcommit\n Committing to file:///var/tmp/builds/git-sync/svn/svn-2/trunk ...\n M Makefile \n Committed r1260 \n M Makefile \n r1260 = 372579ff221a151f026eef42213e52e1b9bb9d47 (svn-2/trunk) \n No changes between current HEAD and refs/remotes/svn-2/trunk \n Resetting to the latest refs/remotes/svn-2/trunk \n $ git checkout svn-2\n Previous HEAD position was 372579f... Force merge ours svn-1 to svn-2/trunk\n Switched to branch \"svn-2\" \n $ git svn fetch svn-1\n $ git merge --no-ff svn-1\n Already up-to-date. \n $ git svn dcommit\n Committing to file:///var/tmp/builds/git-sync/svn/svn-2/trunk ...\n Transaction is out of date: File '/trunk/policy.pl' is out of date at /usr/lib64/git/git-svn line 469\n\nGitk shows that svn-2 is no longer an ancestor of svn-2/trunk. Might this\nbe the reason for the \"transaction out of date\"? How do I recover from that?\n\n> > Unfortunately, dcommit doesn't seem to have an option to force rebase\n> > instead of resetting.\n> \n> Well, in fact it *is* rebasing, which throws away the extra commit\n> because it thinks that commit didn't do anything. I've experienced\n> this problem a few times in the past, but I knew what was happening\n> and I figured my case was too rare to matter. Perhaps not.\n> \n> This could be considered a bug in git-svn, so I cc:'d Eric Wong, who I\n> think is the main git-svn developer. Anyway, try the workaround\n> above.\n\nI am not sure this is a bug. I have still the feeling that I am doing\nsomething wrong. Maybe I should not try to throw two svn remotes onto\na single git repository? Maybe I should create a separate repository\nfor every direction?\n"},{"id":"113889","messageId":"32541b130905132335t3cbd0e7wf29577ee15ba0bac@mail.gmail.com","threadId":"19090","inReplyTo":"20090513222243.GQ15420@raven.wolf.lan","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2009-05-14T06:35:18Z","receivedAt":"2009-05-14T06:35:18Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Wed, May 13, 2009 at 6:22 PM, Josef Wolf wrote:\n> Thanks for your patience, Avery!  I would be completely lost here without\n> your help..\n\nOkay, well, I think I've been making things worse instead of better :(\n\nFundamentally, my claim that merging symmetrically between two svn\nbranches \"ought to be easy\" was incorrect. The problem is that when\ngit does a merge from branch A to B and then back to A, it really\n*really* wants the two branches to end up identical. All of git's\nmerge machinery works on the assumption that this is what you want.\n\nNow, you can still bypass this by using various clever tricks. The\nsolution we were working with turned out to be *almost* right, and I\ndid get it working in my test environment, but it got so convoluted\nthat I couldn't even explain to myself why it was correct. That's\nusually a bad sign. So I threw that one away.\n\nBy far the sanest thing you could possibly do is to create a central\n\"public\" branch that contains all the common commits, then merge from\nthat public branch to the site-specific branches, but never merge in\nthe opposite direction. In case you happen to make some changes on\nthe site-specific branches that you want to share, you can just\ncherry-pick them; the resulting conflicts when merging back are likely\nto be fairly minor. This would be entirely consistent with git's\nnormal operations, and would be easy:\n\n git checkout public\n git cherry-pick stuff # as rarely as possible; do the work\ndirectly on public if you can\n\n git checkout svn-1\n git merge --no-ff public\n git svn dcommit\n\n git checkout svn-2\n git merge --no-ff public\n git svn dcommit\n\nNo criss-cross merges, no insanity, no question about whether it's correct.\n\nMore as an academic exercise than anything, I did find a way that will\nlet you do criss-cross merging of all changes on A and B. I still\ndon't *really* recommend you use it, because it's extremely error\nprone, and there are lots of places where you could get merge\nconflicts and then end up in trouble. (The above simple method, in\ncontrast, might get conflicts sometimes, but you can just fix them as\nyou encounter them and be done with it.)\n\nThe script below demonstrates how to take branches remote-ab and\nremote-ac, and auto-pick their changes (as they happen) into a new\n(automatically managed) branch public. Then it merges public back\ninto each branch, while avoiding conflicts. The magic itself happens\nin extract() and crossmerge().\n\nIf nothing else, this method makes the gitk output far more sane than\nthe original method. This is because it doesn't include the history\nof 'public' in the site-specific branches. That was the fundamental\nflaw in the method I had identified originally. You can trick that\noriginal method into working too, but it's stunningly complex. This\nis much more sane, albeit still not really sane.\n\nEnjoy!\n\nHave fun,\n\nAvery\n\nP.S. Sorry for the mess. I suppose I should have broken down and\nwritten (or asked for :)) a minimal test case earlier, as it quickly\nrevealed the problem.\n\n\n#!/bin/bash -x\nset -e\nrm -rf tt\nmkdir tt\ncd tt\ngit init\n\ncount=100\nnewfile()\n{\n\tcount=$(($count + 1))\n\techo $count >$1\n\tgit add $1\n\tgit commit -m \"$1\"\n}\n\nnewfile .gitignore\ngit checkout -b public\nnewfile a\ngit checkout -b remote-ab public\nnewfile b\ngit checkout -b remote-ac public\nnewfile c\n\n# We've simulated two remote branches (perhaps svn repositories), remote-ab\n# and remote-ac. They contain one identical file (a) and one different file\n# (b vs. c). We've arranged for the common part to end up in 'public'.\n\ngit tag -f remote-ab-lastmerge remote-ab\ngit tag -f remote-ac-lastmerge remote-ac\n\nextract()\n{\n\tlast_public=\"$(git merge-base $1-lastmerge public)\"\n\tif false; then\n\t\t# use this if you want each patch separately\n\t\tgit branch -f $1-public $1\n\t\tgit checkout $1-public\n\t\tgit rebase --onto \"$last_public\" $1-lastmerge\n\telse\n\t\t# use this if you want changes squashed into one patch\n\t\tgit branch -f $1-public \"$last_public\"\n\t\tgit checkout $1-public\n\t\tgit diff --binary $1-lastmerge $1 -- | git apply --index\n\t\t(\n\t\t\techo \"merged from $1\"\n\t\t\techo\n\t\t\tgit log $1-lastmerge..$1\n\t\t) | git commit -F -\n\tfi\n\t\n\tgit checkout $1\n\tgit merge -s ours -m 'no-op' $1-public\n}\n\ncrossmerge()\n{\n\tbranches=\"remote-ab remote-ac\"\n\t\n\tfor b in $branches; do\n\t\t# extract the most recent changes from $b-lastmerge..$b\n\t\t# The changes can be found as public..$b-public\n\t\textract $b\n\t\t\n\t\t# Merge those completed changes into public\n\t\tgit checkout public\n\t\tgit merge $b-public\n\t\tgit branch -d $b-public\n\t\tgit tag -f $b-lastmerge $b # to reduce problems if this script dies\nhalfway through\n\tdone\n\t\n\t# merge changes from public back into each branch.\n\t# changes that originated in each branch won't be re-merged, because\n\t# we already merged back $b-public into each $b.\n\tfor b in $branches; do\n\t\tgit checkout $b\n\t\tgit merge --no-ff public\n\t\tgit tag -f $b-lastmerge $b\n\tdone\n}\n\niterate()\n{\n\t# Some changes have arrived in the remote repos:\n\t#git svn fetch ab\n\t#git svn fetch ac\n\tgit checkout remote-ab\n\tnewfile x$1\n\tgit checkout remote-ac\n\tnewfile y$1\n\n\tcrossmerge\n}\n\niterate 1\ngit tag remote-ab-1 remote-ab\ngit tag remote-ac-1 remote-ac\n\niterate 2\ngit tag remote-ab-2 remote-ab\ngit tag remote-ac-2 remote-ac\n\niterate 3\n\ngitk --all\n"},{"id":"113965","messageId":"20090514214120.GR15420@raven.wolf.lan","threadId":"19090","inReplyTo":"32541b130905132335t3cbd0e7wf29577ee15ba0bac@mail.gmail.com","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Josef Wolf","fromEmail":"jw@raven.inka.de","sentAt":"2009-05-14T21:41:20Z","receivedAt":"2009-05-14T21:41:20Z","isPatch":false,"sender":{"key":"jw@raven.inka.de","avatar":null},"body":"On Thu, May 14, 2009 at 02:35:18AM -0400, Avery Pennarun wrote:\n\n> By far the sanest thing you could possibly do is to create a central\n> \"public\" branch that contains all the common commits, then merge from\n> that public branch to the site-specific branches, but never merge in\n> the opposite direction. In case you happen to make some changes on\n> the site-specific branches that you want to share, you can just\n> cherry-pick them; the resulting conflicts when merging back are likely\n> to be fairly minor. This would be entirely consistent with git's\n> normal operations, and would be easy:\n> \n> git checkout public\n> git cherry-pick stuff # as rarely as possible; do the work\n> directly on public if you can\n> \n> git checkout svn-1\n> git merge --no-ff public\n> git svn dcommit\n> \n> git checkout svn-2\n> git merge --no-ff public\n> git svn dcommit\n> \n> No criss-cross merges, no insanity, no question about whether it's correct.\n\nIndeed, this looks pretty simple. But AFAICS, this works only when\nstarting out with a virgin repository. In my situation, public is\ncurrently empty and have to be constructed from scratch by picking\nfrom the privates.\n\nSo it seems I have to sync the privates in a first step and build the\npublic from that in a second step.\n\nSo here's my second plan:\n1. instead of doing the cherry-picking in a single repository, it might\n be helpful to do it in separate repositories: one repository for each\n direction. While there are still two remote svn repositories in each\n svn repository, there is no need for criss-cross anymore. The flow\n of the data is in one direction and it seems (at least at first glance)\n I can use git-svn-rebase to get a linear history.\n2. After the synchronization is done, I would merge the two repositories\n into a third one to create the public repository. Since this will be\n a pure git environment, I hope that the problems that are caused svn's\n lack of merge support will vanish.\n3. Once the public repository exists, create the privates based on that\n public.\n\nHere's my first attempt for the first step:\n\n # setup a repository template for the synchronization and configure the\n # svn remotes\n mkdir -p svn-sync.templ\n (\n cd svn-sync.templ\n git svn init --stdlayout file:///svn/svn-1\n git config merge.stat true\n for remote in svn-1 svn-2; do\n git config svn-remote.$remote.url file:///svn/$remote\n git config svn-remote.$remote.fetch trunk:refs/remotes/$remote/trunk\n git config svn-remote.$remote.branches branches/*:refs/remotes/$remote/*\n git config svn-remote.$remote.tags tags/*:refs/remotes/$remote/tags/*\n git svn fetch -R $remote\n git checkout -b $remote $remote/trunk\n git tag $remote-orig $remote\n done\n git gc\n )\n \n # now copy the template to create the repositories where the actual\n # synchronization will be done\n cp -a svn-sync.templ to-svn-1\n cp -a svn-sync.templ to-svn-2\n \n # move cherries from svn-1 to svn-2 in the to-svn-2 repository\n (\n cd to-svn-2\n git svn fetch svn-1\n git checkout svn-2\n [ pick cherries ]\n git svn dcommit\n git tag -f svn-1-lastmerge svn-1\n )\n \n # move cherries from svn-2 to svn-1 in the to-svn-1 repository\n (\n cd to-svn-1\n git svn fetch svn-2\n git checkout svn-1\n [ pick cherries ]\n git svn dcommit\n git tag -f svn-2-lastmerge svn-2\n )\n \n # time passes\n \n # Move new commits from svn-1 to svn-2\n (\n cd to-svn-2\n git checkout svn-1\n git svn rebase\n git checkout svn-2\n git svn rebase svn-1\n [ more cherries ]\n git svn dcommit\n git tag -f svn-1-lastmerge svn-1\n )\n \n # Move new commits from svn-2 to svn-1\n (\n cd to-svn-1\n git checkout svn-2\n git svn rebase\n git checkout svn-1\n git svn rebase svn-2\n [ more cherries ]\n git svn dcommit\n git tag -f svn-2-lastmerge svn-2\n )\n\nAt first glance, this seems to work. But there's the drawback that I\nhave to keep track of what have been merged manually. So there's\ncertainly room for improvement :)\n\n> More as an academic exercise than anything, I did find a way that will\n> let you do criss-cross merging of all changes on A and B. I still\n> don't *really* recommend you use it, because it's extremely error\n> prone, and there are lots of places where you could get merge\n> conflicts and then end up in trouble. (The above simple method, in\n> contrast, might get conflicts sometimes, but you can just fix them as\n> you encounter them and be done with it.)\n> \n> The script below demonstrates how to take branches remote-ab and\n> remote-ac, and auto-pick their changes (as they happen) into a new\n> (automatically managed) branch public. Then it merges public back\n> into each branch, while avoiding conflicts. The magic itself happens\n> in extract() and crossmerge().\n> \n> If nothing else, this method makes the gitk output far more sane than\n> the original method. This is because it doesn't include the history\n> of 'public' in the site-specific branches. That was the fundamental\n> flaw in the method I had identified originally. You can trick that\n> original method into working too, but it's stunningly complex. This\n> is much more sane, albeit still not really sane.\n\nI will have to play a little bit with this script to get a better\nunderstanding how it works. But from the description, I got the\nimpression that it matches my (current) work flow pretty good:\nCurrently, initial changes are done in some private repository and\npropagated to the other repositories from there. The only exception\nis that currently, there's no such thing as a \"public\" repository.\n\n> P.S. Sorry for the mess. I suppose I should have broken down and\n> written (or asked for :)) a minimal test case earlier, as it quickly\n> revealed the problem.\n\nOh, I have learned a lot in this thread. And BTW: I _have_ tried to\nwrite a minimal test case several times. But I simply was not able to\nreproduce the problems there. The problems showed up only on the real\nrepositories.\n\nThank you very much Avery!\n"},{"id":"113966","messageId":"32541b130905141457u196e1a68w8250489b88eb83c4@mail.gmail.com","threadId":"19090","inReplyTo":"20090514214120.GR15420@raven.wolf.lan","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2009-05-14T21:57:00Z","receivedAt":"2009-05-14T21:57:00Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Thu, May 14, 2009 at 5:41 PM, Josef Wolf wrote:\n> On Thu, May 14, 2009 at 02:35:18AM -0400, Avery Pennarun wrote:\n>> No criss-cross merges, no insanity, no question about whether it's correct.\n>\n> Indeed, this looks pretty simple.  But AFAICS, this works only when\n> starting out with a virgin repository.  In my situation, public is\n> currently empty and have to be constructed from scratch by picking\n> from the privates.\n\nNot exactly; you simply produce a \"public\" repository however you want\nto produce it. One easy way would be to copy one of the existing svn\nbranches, cherry pick and revert whatever commits you want, and call\nit public. The prior history of that branch doesn't matter to the\nalgorithm.\n\n> So here's my second plan:\n> 1. instead of doing the cherry-picking in a single repository, it might\n>   be helpful to do it in separate repositories: one repository for each\n>   direction.  While there are still two remote svn repositories in each\n>   svn repository, there is no need for criss-cross anymore.  The flow\n>   of the data is in one direction and it seems (at least at first glance)\n>   I can use git-svn-rebase to get a linear history.\n\nit's still criss-crossing, it's just less obvious that way. One\nrepository is exactly the same as two repositories in git; all that\nmatters is the branch histories. So if you think this will fix it,\nyou're probably missing something :)\n\n> 2. After the synchronization is done, I would merge the two repositories\n>   into a third one to create the public repository.  Since this will be\n>   a pure git environment, I hope that the problems that are caused svn's\n>   lack of merge support will vanish.\n\nI'd say that basically none of your problems have anything to do with\nsvn's lack of merge support, and everything to do with the fact that\nyou aren't doing all your changes first on a 'public' branch and then\nmerging from there into the private branches. (That's really not so\nhard to do in svn either, and would save a ton of confusion.)\n\n> At first glance, this seems to work.  But there's the drawback that I\n> have to keep track of what have been merged manually.  So there's\n> certainly room for improvement :)\n\nRight, the crossmerge() function in the script I sent is designed to avoid that.\n\n> I will have to play a little bit with this script to get a better\n> understanding how it works.  But from the description, I got the\n> impression that it matches my (current) work flow pretty good:\n> Currently, initial changes are done in some private repository and\n> propagated to the other repositories from there.  The only exception\n> is that currently, there's no such thing as a \"public\" repository.\n\nThe public repository is nothing special and doesn't require any work\nfrom you; it's simply maintained automatically from the private\nbranches. (Of course, if you start doing all your changes in the\npublic repository, life gets a little simpler.)\n\nHave fun,\n\nAvery\n"},{"id":"114005","messageId":"20090515175203.GS15420@raven.wolf.lan","threadId":"19090","inReplyTo":"32541b130905141457u196e1a68w8250489b88eb83c4@mail.gmail.com","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Josef Wolf","fromEmail":"jw@raven.inka.de","sentAt":"2009-05-15T17:52:03Z","receivedAt":"2009-05-15T17:52:03Z","isPatch":false,"sender":{"key":"jw@raven.inka.de","avatar":null},"body":"On Thu, May 14, 2009 at 05:57:00PM -0400, Avery Pennarun wrote:\n> On Thu, May 14, 2009 at 5:41 PM, Josef Wolf wrote:\n> > So here's my second plan:\n> > 1. instead of doing the cherry-picking in a single repository, it might\n> >   be helpful to do it in separate repositories: one repository for each\n> >   direction.  While there are still two remote svn repositories in each\n> >   svn repository, there is no need for criss-cross anymore.  The flow\n> >   of the data is in one direction and it seems (at least at first glance)\n> >   I can use git-svn-rebase to get a linear history.\n> \n> it's still criss-crossing, it's just less obvious that way. One\n> repository is exactly the same as two repositories in git; all that\n> matters is the branch histories.\n\nYeah, I see... But this step is here _only_ to get the existing svn\nrepositories in sync again. After cherry-picking and dcommitting, those\ncherry-pick repositories would be wiped. They have no real history. The\nsteps I outlined in my previous mail wouldn't even create any files in\nthe .git/refs subdirectory.\n\nOnce that is done, I can declare one of the existing repositories as\npublic and pull it via git-svn into a freshly created repos. The other\nrepos can then be recreated by cloning and applying patches. No svn\ninvolved anymore here.\n\n> > 2. After the synchronization is done, I would merge the two repositories\n> >   into a third one to create the public repository.  Since this will be\n> >   a pure git environment, I hope that the problems that are caused svn's\n> >   lack of merge support will vanish.\n> \n> I'd say that basically none of your problems have anything to do with\n> svn's lack of merge support, and everything to do with the fact that\n> you aren't doing all your changes first on a 'public' branch and then\n> merging from there into the private branches. (That's really not so\n> hard to do in svn either, and would save a ton of confusion.)\n\nThe problem here is that it does not match the work flow. IMHO, my work\nflow is very similar to the work flow of the kernel, so I fail to see why\nit can not work. See the analogies:\n\nkernel: Submodule maintainers are committing into private repositories\nme: People are committing into private repositories\n\nkernel: Those commits are forwarded to Linus's repository\nme: Those commits are forwarded to the public repository\n\nkernel: Maintainers receive commits for other submodules from linus\nme: Commits are distributed from public to private repositories\n\nI can't believe all changes spring into life in linus's repository.\n\nThe only differences I can see are:\n- size of the project (obviously)\n- convert from multiple svn repos instead of bitkeeper\n- private repostories have to keep local patches (but I guess maintainers\n do that also)\n"},{"id":"114018","messageId":"32541b130905151205h6ca89d85q97e72ce23bf233ee@mail.gmail.com","threadId":"19090","inReplyTo":"20090515175203.GS15420@raven.wolf.lan","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2009-05-15T19:05:14Z","receivedAt":"2009-05-15T19:05:14Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Fri, May 15, 2009 at 1:52 PM, Josef Wolf wrote:\n> On Thu, May 14, 2009 at 05:57:00PM -0400, Avery Pennarun wrote:\n>> On Thu, May 14, 2009 at 5:41 PM, Josef Wolf wrote:\n>> > So here's my second plan:\n>> > 1. instead of doing the cherry-picking in a single repository, it might\n>> >   be helpful to do it in separate repositories: one repository for each\n>> >   direction.  While there are still two remote svn repositories in each\n>> >   svn repository, there is no need for criss-cross anymore.  The flow\n>> >   of the data is in one direction and it seems (at least at first glance)\n>> >   I can use git-svn-rebase to get a linear history.\n>>\n>> it's still criss-crossing, it's just less obvious that way.  One\n>> repository is exactly the same as two repositories in git; all that\n>> matters is the branch histories.\n>\n> Yeah, I see...  But this step is here _only_ to get the existing svn\n> repositories in sync again.  After cherry-picking and dcommitting, those\n> cherry-pick repositories would be wiped.  They have no real history.  The\n> steps I outlined in my previous mail wouldn't even create any files in\n> the .git/refs subdirectory.\n\nHmm, getting them in sync the first time seems to be \"easy\"\n(relatively), in that you've already done it, right? So it's a\none-time thing, doesn't need automation, and you already figured that\npart out. So it seems like a non-issue one way or the other.\n\n> Once that is done, I can declare one of the existing repositories as\n> public and pull it via git-svn into a freshly created repos.  The other\n> repos can then be recreated by cloning and applying patches.  No svn\n> involved anymore here.\n\nYes, that works fine. Nothing stopping you from declaring one or the\nother svn repos to be identical to \"public.\"\n\n>> I'd say that basically none of your problems have anything to do with\n>> svn's lack of merge support, and everything to do with the fact that\n>> you aren't doing all your changes first on a 'public' branch and then\n>> merging from there into the private branches.  (That's really not so\n>> hard to do in svn either, and would save a ton of confusion.)\n>\n> The problem here is that it does not match the work flow.  IMHO, my work\n> flow is very similar to the work flow of the kernel, so I fail to see why\n> it can not work.  See the analogies:\n>\n> kernel: Submodule maintainers are committing into private repositories\n> me:     People are committing into private repositories\n>\n> kernel: Those commits are forwarded to Linus's repository\n> me:     Those commits are forwarded to the public repository\n>\n> kernel: Maintainers receive commits for other submodules from linus\n> me:     Commits are distributed from public to private repositories\n\nThere is one critical difference here: if someone merges from Linus\nand then Linus merges back from them, then the two resulting\nrepositories will be *identical* (at least, the trees will be; if the\nsecond merge uses --no-ff, the histories will be very slightly\ndifferent, but not importantly so).\n\nIf someone has patches that they don't want to send back to Linus, and\nthose patches are intermixed with ones they *do* want to send back,\nthen they either have to cherry pick them over to a separate branch\n(which Linus can then pull), or equivalently they email individual\npatches to Linus, or they need to rebase a lot, or they need to just\nput their \"finished\" patches onto a separate branch and keep the\nunfinished ones somewhere else that Linus won't pull.\n\nRebasing is (I think) actually the most common solution to this\nproblem, but it doesn't help if you're using svn. svn has no concept\nof rebasing. (git svn rebase uses git rebase, but it's not really for\nthe same purpose.)\n\n> The only differences I can see are:\n> - size of the project (obviously)\n> - convert from multiple svn repos instead of bitkeeper\n> - private repostories have to keep local patches (but I guess maintainers\n>  do that also)\n\nThat last one is the source of all your problems. That said, the\nscript I provided *does* let you do this, if you're brave.\n\nHave fun,\n\nAvery\n"},{"id":"114129","messageId":"20090517112449.GT15420@raven.wolf.lan","threadId":"19090","inReplyTo":"32541b130905151205h6ca89d85q97e72ce23bf233ee@mail.gmail.com","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Josef Wolf","fromEmail":"jw@raven.inka.de","sentAt":"2009-05-17T11:24:49Z","receivedAt":"2009-05-17T11:24:49Z","isPatch":false,"sender":{"key":"jw@raven.inka.de","avatar":null},"body":"On Fri, May 15, 2009 at 03:05:14PM -0400, Avery Pennarun wrote:\n> On Fri, May 15, 2009 at 1:52 PM, Josef Wolf wrote:\n> > On Thu, May 14, 2009 at 05:57:00PM -0400, Avery Pennarun wrote:\n> >> On Thu, May 14, 2009 at 5:41 PM, Josef Wolf wrote:\n> >> > So here's my second plan:\n> >> > 1. instead of doing the cherry-picking in a single repository, it might\n> >> >   be helpful to do it in separate repositories: one repository for each\n> >> >   direction.  While there are still two remote svn repositories in each\n> >> >   svn repository, there is no need for criss-cross anymore.  The flow\n> >> >   of the data is in one direction and it seems (at least at first glance)\n> >> >   I can use git-svn-rebase to get a linear history.\n> >>\n> >> it's still criss-crossing, it's just less obvious that way.  One\n> >> repository is exactly the same as two repositories in git; all that\n> >> matters is the branch histories.\n> >\n> > Yeah, I see...  But this step is here _only_ to get the existing svn\n> > repositories in sync again.  After cherry-picking and dcommitting, those\n> > cherry-pick repositories would be wiped.  They have no real history.  The\n> > steps I outlined in my previous mail wouldn't even create any files in\n> > the .git/refs subdirectory.\n> \n> Hmm, getting them in sync the first time seems to be \"easy\"\n> (relatively), in that you've already done it, right?\n\nNo. I have a perl script to do the cherry-picking and to resolve\nconflicts. Because of this, I can try out different methods to do the\nsync. And of course, I'd like to do it in a way that produces the least\nproblems in the future, since this is a one-shot thing.\n\n> >> I'd say that basically none of your problems have anything to do with\n> >> svn's lack of merge support, and everything to do with the fact that\n> >> you aren't doing all your changes first on a 'public' branch and then\n> >> merging from there into the private branches.  (That's really not so\n> >> hard to do in svn either, and would save a ton of confusion.)\n> >\n> > The problem here is that it does not match the work flow.  IMHO, my work\n> > flow is very similar to the work flow of the kernel, so I fail to see why\n> > it can not work.  See the analogies:\n> >\n> > kernel: Submodule maintainers are committing into private repositories\n> > me:     People are committing into private repositories\n> >\n> > kernel: Those commits are forwarded to Linus's repository\n> > me:     Those commits are forwarded to the public repository\n> >\n> > kernel: Maintainers receive commits for other submodules from linus\n> > me:     Commits are distributed from public to private repositories\n> \n> There is one critical difference here: if someone merges from Linus\n> and then Linus merges back from them, then the two resulting\n> repositories will be *identical* (at least, the trees will be; if the\n> second merge uses --no-ff, the histories will be very slightly\n> different, but not importantly so).\n> \n> If someone has patches that they don't want to send back to Linus, and\n> those patches are intermixed with ones they *do* want to send back,\n> then they either have to cherry pick them over to a separate branch\n> (which Linus can then pull), or equivalently they email individual\n> patches to Linus, or they need to rebase a lot, or they need to just\n> put their \"finished\" patches onto a separate branch and keep the\n> unfinished ones somewhere else that Linus won't pull.\n\nDoes any description exist how this process works in detail?\n\n> Rebasing is (I think) actually the most common solution to this\n> problem, but it doesn't help if you're using svn. svn has no concept\n> of rebasing. (git svn rebase uses git rebase, but it's not really for\n> the same purpose.)\n\nIn the long term, I am willing to get rid of svn. But I have to create\na migration path, so I need to keep git+svn in parallel for a couple of\nmonths.\n"},{"id":"114355","messageId":"20090520164014.GU15420@raven.wolf.lan","threadId":"19090","inReplyTo":"32541b130905151205h6ca89d85q97e72ce23bf233ee@mail.gmail.com","subject":"Re: Trying to sync two svn repositories with git-svn (repost)","fromName":"Josef Wolf","fromEmail":"jw@raven.inka.de","sentAt":"2009-05-20T16:40:14Z","receivedAt":"2009-05-20T16:40:14Z","isPatch":false,"sender":{"key":"jw@raven.inka.de","avatar":null},"body":"On Fri, May 15, 2009 at 03:05:14PM -0400, Avery Pennarun wrote:\n> On Fri, May 15, 2009 at 1:52 PM, Josef Wolf wrote:\n> > On Thu, May 14, 2009 at 05:57:00PM -0400, Avery Pennarun wrote:\n> >> I'd say that basically none of your problems have anything to do with\n> >> svn's lack of merge support, and everything to do with the fact that\n> >> you aren't doing all your changes first on a 'public' branch and then\n> >> merging from there into the private branches.  (That's really not so\n> >> hard to do in svn either, and would save a ton of confusion.)\n> >\n> > The problem here is that it does not match the work flow.  IMHO, my work\n> > flow is very similar to the work flow of the kernel, so I fail to see why\n> > it can not work.  See the analogies:\n> >\n> > kernel: Submodule maintainers are committing into private repositories\n> > me:     People are committing into private repositories\n> >\n> > kernel: Those commits are forwarded to Linus's repository\n> > me:     Those commits are forwarded to the public repository\n> >\n> > kernel: Maintainers receive commits for other submodules from linus\n> > me:     Commits are distributed from public to private repositories\n> \n> There is one critical difference here: if someone merges from Linus\n> and then Linus merges back from them, then the two resulting\n> repositories will be *identical* (at least, the trees will be; if the\n> second merge uses --no-ff, the histories will be very slightly\n> different, but not importantly so).\n\nHmm, maybe submodules could be of some help here? If I put the generic\ncontent into a submodule, and the localized content into (multiple)\nsuperprojects, the kernel work flow should be easy to adopt, or am I\nmissing something?\n"}]}