How to use git-svn to clone from a mirror?

Hi, I seem to be too stupid to get this right.

I have used something like

git-svn clone -T trunk -b branches -t tags file:///tmp/rsync-mirror

to clone an rsync mirror of an SVN repository. Now I want to have fetch revert to pulling from the upstream repository in future. However, if I change the respective line in .git/config to svn://the.svn.link/whatever, git-rebase will fetch the right updates, but then says that it can't work with the objects in the git repository.

Changing the config back will make git-rebase -l work.

So what would be the right procedure to shift the SVN source from an rsync mirror to the original, without git-svn breaking?