Stitching together private svn repo and public git repo

Short Version: 

Repo B is a completely linear repo that has been tracking public repo A for some time. A number of manual merges were done to bring new A releases into B but there are no connections between the two repos. (How) can I stitch B to A to reflect this relationship? I want to retain B's history and leave myself with an arrangement in which I can pull from A into B periodically? Many thanks for your suggestions, Greg.

-- Long Version:

I have been tracking my modifications of a public project released as tar balls with my own local svn repository. My svn repo was completely linear and I always merged in changes manually each time a new tar ball was released. This started to get painful.

Now I want to do better. I have set up a public git repository ("A") to track the tar ball releases from the public project. I have converted my svn repo with git-svn to a git repo ("B").

My goal is to end up with a (new?) git repo tracking the public repo (A) so that I can pull in any changes and merge easily. Fine, but I also want to keep the history that I have imported from my svn repository (B). So my question is, can anyone suggest some pointers for how to stitch together A and B?

Right now I have been trying to pull B into A to splice:

A $ git checkout v1.91 B $ git checkout v1.91-manualmerge B $ git pull --no-commit -s ours ../A

This looks right when I run gitk, but if I repeat this for a second merge point then the previous merge seems to disappear from history when I bring up gitk.

I haven't found any docs/wiki info that has enlightened me yet. Any suggestions, wisdom, links etc very much appreciated - I do want to try to get this right now, so I don't have to rejig everything in the future. Best wishes and many thanks,

Greg.

On Wed, Jan 02, 2008 at 07:25:41PM +0000, Gregory Jefferis wrote:

So you have something like this in B:

Av1* -- Av1_b -- Av1_c -- Av1_d -- Av2_d* -- Av2_e -- Av3_e*

Where the * are manual merges of the offical Avn releases...

And you want this:

Av1   --       --       -- Av2   --      -- Av3
  \                          \                \
  Av1_b -- Av1_c -- Av1_d -- Av2_d -- Av2_e -- Av3_e  ???

This is a "rewrite history" job as parent lists are part of each commit. (See Linus' big bold-face warning about history re-writes.)

First of all I would add a remote for the 'A' repository so that those commits are available in the 'B' repository.

Something like: git remote add repoA /path/to/A git fetch repoA

You could then do this with a 'git filter-branch --parent-filter' to rewrite the parents of the merge commits. As far as I can see, you would need to call filter-branch once per merge to rewrite everything from the merge commit forwards. At this point all later commits would have different ids, so attempting to rewrite subsequent parent-ids in the same filter-branch invocation is probably futile.

It might be possible to use an intelligent script in a single --commit-filter invocation of filter-branch, but the script of the actual filter would have to be a bit (a lot!) more sophisticated, remember the ids of the new commits as it created them with 'git commit-tree' and merging in the repoA parents at the right points.

Leaving that aside and concentrating on the multiple filter-branch invocation option... for example, the first parent-filter script could be: sed -e 's/^$/-p <commit id of Av1>/'

( This is if you want your exisiting tree to branch from Av1 in the original repo. If you wanted to replace your tree root with a root at Av1, because your current root is just a copy of Av1 then you want: sed -e 's/<commit id of repoB's Av1 copy>/<commit id of Av1 in A>/' )

Then, for whatever the Av2_d commit's new id is, you could do a parent-filter of: sed -e 's/\(<new commit id of Av1_d>\)/\1 -p <commit id of Av2>/'

This causes the Av2_d commit to be rewritten, maintaining its original parent and adding another parent which comes from repoA. It then rewrites all the descendent commits of Av2_d to take account of this new commit id and all the subsequent new commit ids.

Now you can do similar for the next merge commit: sed -e 's/\(<new commit id of Av2_e>\)/\1 -p <commit id of Av3>/'

As noted, you end up with completely new commit objects with a completely new history, so you will screw everyone who is pulling/cloning from you.

Charles.

On Jan 3, 2008 12:40 AM, Charles Bailey <charles@hashpling.org> wrote:

I believe there is a much easier way to do that using .git/info/grafts The first step is to create .git/info/grafts, which specifies commit-id and its parents for each commit that you want to change. Then you can check the result using gitk, and if you are satisfied with what you see then you run git filter-branch on it to convert 'fake' parents into real ones.

Dmitry

On Thu, Jan 03, 2008 at 01:13:54AM +0300, Dmitry Potapov wrote:

Oh yes, this is much easier. Unless I'm missing something, the documentation on grafts is fairly sparse, though. They are mentioned (almost in passing) in git help filter-branch but the file format is only documented in repository-layout.txt which seems more developer than user oriented.

Charles.

Many thanks to both for your replies. Starting from your comments and consulting:

http://www.kernel.org/pub/software/scm/git/docs/repository-layout.html

for (fairly scant) details of .git/info/grafts and
 
http://www.kernel.org/pub/software/scm/git/docs/git-filter-branch.html

I think I have now been able to do the necessary. Since I found this a trivial operation (in the true mathematical sense of quasi-impossible until someone helps you to figure it out and simple once you have) and I tripped over a couple of times, I thought I would share my final shell script below.

One point that I missed initially. The graft file format is unforgiving. You are allowed comments but each regular line is:

Commit-SHA-1 Parent1-SHA-1 Parent2-SHA-1\n
      
If you put e.g. more than a single space between your SHA-1s you will get
"bad graft data:" - I missed one in the fake merge line and ran round in
circles for a bit thinking it was something to do with the state of my
trees.

Anyway, boy did it feel great when I was able to git-pull the latest public release into my private repository:

    34 files changed, 11563 insertions, 8399 deletions

and only 2 simple conflicts to fix with git-mergetool. That was not a merge I wanted to do by hand!

Best wishes and thanks again,

Greg.

I suspect your message was linewrapped by your MUA and ...

Gregory Jefferis <jefferis@gmail.com> writes:

... your grafts file look as if you are marking five commits as root commits, which is not what you wanted ;-).

The reason I mention this is because I think with a bit more polishing and some background description your message is a good material to add to Documentation/howto/.