threads / discuss / 32255

Stitching histories of several repositories

Subject: Stitching histories of several repositories

## tl;dr

2 messages between Dec 1, 2012 and Dec 6, 2012.

replies: 1people: 2as markdown or json

Ramkumar Ramachandra· Dec 1, 2012, 09:11 UTC · lore

Hi,

I've written a tool to stitch the first-parent histories of several git repositories. To illustrate, consider that we have a toplevel git repository inside which the other repositories reside.

  /.git
  /a/.git
  /b/.git

The tool pulls in the objects from /a/.git and /b/.git into /.git by using alternates. Then, it gets the list of commits by rev-list'ing and emits a fast-import stream rewriting the paths of the files in a/ from / to a/ and the paths of files in b/ form / to b/ (using M 040000), taking care to have one unified first-parent line. It turns out that this is quite non-trivial when the individual git repositories contain lots of merges. To illustrate, this is an example of how histories from two repositories with one merge commit each would be stitched:

  *   c1f81f1 (refs/figlets/first/origin/master)
  |\
  | * 6eaf22f (refs/figlets/first-2/origin/master)
  * 15cd841

  *   1f2f408 (refs/figlets/second/origin/master)
  |\
  | * 4bd7fe3 (refs/figlets/second-2/origin/master)
  * bfa9f3c

  *   5e1d337 (refs/replay/stitch2, fig/stitch2)
  |\
  | * 23437d5 (refs/replay/1f2f40-2)
  |/
  *   4f2d70b
  |\
  | * e017fa5 (refs/replay/c1f81f-2)
  * c0251c7
  * 4036fea

I'd like to know whether the tool would be useful to a wider audience, before I polish it and consider submitting it for inclusion in contrib/. I think the tool is especially useful for running bisect and tracking bugs that occur in large projects that consist of many git repositories. Will a unified log showing commits in different submodules be useful?

Ram

Robin Rosenberg· Dec 6, 2012, 00:00 UTC · re: Ramkumar Ramachandra · lore

Re: Stitching histories of several repositories

----- Ursprungligt meddelande -----

Show 7 quoted lines

> Hi,
> 
> I've written a tool to stitch the first-parent histories of several
> git repositories.  To illustrate, consider that we have a toplevel
> git
> repository inside which the other repositories reside.
>

[...]

Show 8 quoted lines

> 
> I'd like to know whether the tool would be useful to a wider
> audience,
> before I polish it and consider submitting it for inclusion in
> contrib/.  I think the tool is especially useful for running bisect
> and tracking bugs that occur in large projects that consist of many
> git repositories.  Will a unified log showing commits in different
> submodules be useful?

I think it is useful. I did something like that creating a super-repo for performing a bisect over EGit and JGit over a period over a year. I don't think I restricted myself to the first parent, which was probably a mistake.

The solution isn't perfect, but provided that is well-known you can live with that. A problem I found was that many stiches versions weren't even compilable so one might want to have the option of matching commits using a window of some sort to identify compilable combinations and have the ability to use only those for bisect. I'm not sure it's practical though.

-- robin