Re: working with a large repository and git svn

From: Joe Corneli <holtzermann17@gmail.com>
Date: Jan 13, 2011, 00:54 UTC
Message-ID: <AANLkTi=uuBuunYmwmLYD_vUnPkDBk9YDLtATw9GtX33z@mail.gmail.com>
In-Reply-To: <201101120830.47016.wjl@icecavern.net>

>  1) Sounds like git-svn is running out of resources on your machine --
> that's probably a bug, but work around it: Don't dcommit all 20000 revisions
> at once. Maybe write a shell script that goes through and dcommits a 100
> commits at a time.

Hm, I found a related blog post here, but designed for interactive use: http://fredericiana.com/2009/12/31/partial-svn-dcommit-with-git/ Could you give me a more detailed hint about how to do what you suggested?

>  2) Do you need the full history to be in SVN? Can you rebase/squash large
> parts together and thus need to commit less revisions in the first place?

Maybe. We want a tool for managing the entire history, and Git seems like a good tool for that. At the same time, checking out the entire history can take a long time - if we could just check out just the latest files and check them back in in a sensible way, that would be good - SVN does seem suitable for that purpose. If there's a git-only way to do this I'd be happy to know about that as well!

>  3) I love git-svn for working with Subversion repositories, but you could
> consider a different tool, like tailor, if you can't make git-svn do what
> you want.

Tried it, but it didn't even get through the initiation phase. I asked for help in the relevant mailing list.

> people working on a fast-import tool for SVN, so you could git-fast-export
> and svn-fast-import in a big batch.

Not finding these.

>  4) Does 8 GB of data really belong in the same repository? Maybe it should
> really be split up and used with git submodules or SVN externals? That may
> make things easier to work with in the long term.

Probably true. if there was a nice way to give each *file* its own associated "repository", then stitch these together into packets (even "on demand"), that would be cool. I was assuming we could do fancy stuff like this as "future work" however - and it would seem that if we use a completely git-based solution we'll be there.

>  5) Do you really want to be going from Git, to Subversion? That seems like
> a big step backwards. =)

If there's a good way to just pull down the latest revision into a working copy and be able to push that back to the repo that would be nice. This doesn't seem to be the Git way, but for an 8 gig repo it's probably pretty important feature. Thoughts?

Thanks, Joe