threads / discuss / 14392

Errors importing Apache Synapse SVN using Git

Subject: Errors importing Apache Synapse SVN using Git

## tl;dr

6 messages between Jul 10, 2008 and Jul 14, 2008.

replies: 5people: 4as markdown or json

Asankha C. Perera· Jul 10, 2008, 12:01 UTC · lore
Hi All

I am an Apache Synapse developer, and want to import the Synapse SVN repo into Git, so that Ohloh can properly get the Synapse history (http://www.ohloh.net/topics/1326?page=1#post_6287)

However, when I try the command: "git svn clone --trunk=trunk --tags=tags --branches=branches http://svn.apache.org/repos/asf/synapse" it seems to take forever, (or at least until the next network glitch), and keeps filling up a file with just plain zeros ("0") : ./.git/svn/trunk/.rev_db.13f79535-47bb-0310-9956-ffa450edef68

Can someone try the above command on the Synapse repo and tell me what I can do to import from the SVN?

many thanks asankha

Michael J Gruber· Jul 10, 2008, 12:59 UTC · re: Asankha C. Perera · lore

Re: Errors importing Apache Synapse SVN using Git

Asankha C. Perera venit, vidit, dixit 10.07.2008 14:01:
Show 14 quoted lines
> Hi All
> 
> I am an Apache Synapse developer, and want to import the Synapse SVN 
> repo into Git, so that Ohloh can properly get the Synapse history 
> (http://www.ohloh.net/topics/1326?page=1#post_6287)
> 
> However, when I try the command: "git svn clone --trunk=trunk 
> --tags=tags --branches=branches http://svn.apache.org/repos/asf/synapse" 
> it seems to take forever, (or at least until the next network glitch), 
> and keeps filling up a file with just plain zeros ("0") : 
> ./.git/svn/trunk/.rev_db.13f79535-47bb-0310-9956-ffa450edef68
> 
> Can someone try the above command on the Synapse repo and tell me what I 
> can do to import from the SVN?

"svn log" takes forever on that repo, too. Current rev seems to be 675546, and the synapse path does not exist in early revisions. Knowing the initial revision would help, then you could save "git svn" from having to comb through (supposedly) tens of thousands of irrelevant revs.

I just checked out trunk using svn 1.4.6, "svn log ." takes forever in the root dir. So the svn repo seems to be largely unusable, at least when accessed from svn 1.4.6 clients (the server is 1.5.0, I see).

Okay, I bisected it and got r234477 as the beginning of time for synapse. "svn log -r 234477:HEAD" is still painful.

You may want to fetch 1000 revs each or so from there each time.
Michael
Björn Steinbrink· Jul 10, 2008, 22:02 UTC · re: Michael J Gruber · lore

Re: Errors importing Apache Synapse SVN using Git

On 2008.07.10 14:59:11 +0200, Michael J Gruber wrote:
Show 30 quoted lines
> Asankha C. Perera venit, vidit, dixit 10.07.2008 14:01:
>> Hi All
>>
>> I am an Apache Synapse developer, and want to import the Synapse SVN  
>> repo into Git, so that Ohloh can properly get the Synapse history  
>> (http://www.ohloh.net/topics/1326?page=1#post_6287)
>>
>> However, when I try the command: "git svn clone --trunk=trunk  
>> --tags=tags --branches=branches 
>> http://svn.apache.org/repos/asf/synapse" it seems to take forever, (or 
>> at least until the next network glitch), and keeps filling up a file 
>> with just plain zeros ("0") :  
>> ./.git/svn/trunk/.rev_db.13f79535-47bb-0310-9956-ffa450edef68
>>
>> Can someone try the above command on the Synapse repo and tell me what 
>> I can do to import from the SVN?
>
> "svn log" takes forever on that repo, too. Current rev seems to be  
> 675546, and the synapse path does not exist in early revisions. Knowing  
> the initial revision would help, then you could save "git svn" from  
> having to comb through (supposedly) tens of thousands of irrelevant revs.
>
> I just checked out trunk using svn 1.4.6, "svn log ." takes forever in  
> the root dir. So the svn repo seems to be largely unusable, at least  
> when accessed from svn 1.4.6 clients (the server is 1.5.0, I see).
>
> Okay, I bisected it and got r234477 as the beginning of time for  
> synapse. "svn log -r 234477:HEAD" is still painful.
>
> You may want to fetch 1000 revs each or so from there each time.

Also, upgrading git to 1.5.6.2 might be a good idea. It doesn't use the .revdb file format anymore, but a more efficient .revmap file. And it has quite a few performance improvements (although they won't help against the primary issue with that svn server). Don't use 1.5.6/1.5.6.1, they have a git-svn bug that can lead to corrupted clones (just in case that your distro has packages for those versions but not yet got 1.5.6.2).

Björn
Michael J Gruber· Jul 11, 2008, 11:54 UTC · re: Asankha C. Perera · lore

Re: Errors importing Apache Synapse SVN using Git

Asankha C. Perera venit, vidit, dixit 10.07.2008 14:01:
Show 11 quoted lines
> Hi All
> 
> I am an Apache Synapse developer, and want to import the Synapse SVN 
> repo into Git, so that Ohloh can properly get the Synapse history 
> (http://www.ohloh.net/topics/1326?page=1#post_6287)
> 
> However, when I try the command: "git svn clone --trunk=trunk 
> --tags=tags --branches=branches http://svn.apache.org/repos/asf/synapse" 
> it seems to take forever, (or at least until the next network glitch), 
> and keeps filling up a file with just plain zeros ("0") : 
> ./.git/svn/trunk/.rev_db.13f79535-47bb-0310-9956-ffa450edef68

Okay, besides the zero offset, there seems to be big problem with the svn repo:

svn list -r 234478 http://svn.apache.org/repos/asf/synapse branches/ site/ tags/ trunk/

svn list -r 234478 http://svn.apache.org/repos/asf|grep synapse
Right, nothing returned.

git-svn insists on using http://svn.apache.org/repos/asf as the base URL, which is basically correct, but doesn't work for this repo.

Even editing .git/config and .git/svn/metadata (after git svn init, befire the first fetch) to change the url doesn't help. git svn still uses .../asf. Is this as intended? cc'ing the git svn author.

Michael
Eric Wong· Jul 14, 2008, 08:16 UTC · re: Michael J Gruber · lore

Re: Errors importing Apache Synapse SVN using Git

Michael J Gruber <michaeljgruber+gmane@fastmail.fm> wrote:
Show 12 quoted lines
> Asankha C. Perera venit, vidit, dixit 10.07.2008 14:01:
> >Hi All
> >
> >I am an Apache Synapse developer, and want to import the Synapse SVN 
> >repo into Git, so that Ohloh can properly get the Synapse history 
> >(http://www.ohloh.net/topics/1326?page=1#post_6287)
> >
> >However, when I try the command: "git svn clone --trunk=trunk 
> >--tags=tags --branches=branches http://svn.apache.org/repos/asf/synapse" 
> >it seems to take forever, (or at least until the next network glitch), 
> >and keeps filling up a file with just plain zeros ("0") : 
> >./.git/svn/trunk/.rev_db.13f79535-47bb-0310-9956-ffa450edef68
The latest versions of git-svn should be vastly more space efficient
with .rev_map files.
Show 10 quoted lines
> Okay, besides the zero offset, there seems to be big problem with the 
> svn repo:
> 
> svn list -r 234478 http://svn.apache.org/repos/asf/synapse
> branches/
> site/
> tags/
> trunk/
> 
> svn list -r 234478 http://svn.apache.org/repos/asf|grep synapse
Ah, that's because it started off in the /incubator directory, not /synapse:
$ svn log -v -r 234478 http://svn.apache.org/repos/asf
------------------------------------------------------------------------
r234478 | dims | 2005-08-22 05:39:51 -0700 (Mon, 22 Aug 2005) | 3 lines
Changed paths:
   A /incubator/synapse/branches
   A /incubator/synapse/site
   A /incubator/synapse/tags
   A /incubator/synapse/trunk
init synapse SVN
------------------------------------------------------------------------
Show 8 quoted lines
> Right, nothing returned.
> 
> git-svn insists on using http://svn.apache.org/repos/asf as the base 
> URL, which is basically correct, but doesn't work for this repo.
> 
> Even editing .git/config and .git/svn/metadata (after git svn init, 
> befire the first fetch) to change the url doesn't help. git svn still 
> uses .../asf. Is this as intended? cc'ing the git svn author.
Can you try using the latest git-svn and also --no-follow-parent?

--no-follow-parent won't traverse copy history so it may be faster, but you'll get an incomplete history.

On the other hand, it looks like the asf repo is hopelessly slow for history retrieval.

I'm pulling the asf log (svn log -v http://svn.apache.org/repos/asf > tmp.log) but it's taking forever... git-svn uses basically the same operation as 'svn log -v' and I've yet to see an SVN server take this long to show history.

I've gotten 70,000 revisions after over one hour with the above svn log command. I'm going to restart this from a server with a better connection since I need to power off this one when I sleep.

-- 
Eric Wong
Michael J Gruber· Jul 14, 2008, 09:14 UTC · re: Eric Wong · lore

Re: Errors importing Apache Synapse SVN using Git

Eric Wong venit, vidit, dixit 14.07.2008 10:16:
Show 40 quoted lines
> Michael J Gruber <michaeljgruber+gmane@fastmail.fm> wrote:
>> Asankha C. Perera venit, vidit, dixit 10.07.2008 14:01:
>>> Hi All
>>>
>>> I am an Apache Synapse developer, and want to import the Synapse SVN 
>>> repo into Git, so that Ohloh can properly get the Synapse history 
>>> (http://www.ohloh.net/topics/1326?page=1#post_6287)
>>>
>>> However, when I try the command: "git svn clone --trunk=trunk 
>>> --tags=tags --branches=branches http://svn.apache.org/repos/asf/synapse" 
>>> it seems to take forever, (or at least until the next network glitch), 
>>> and keeps filling up a file with just plain zeros ("0") : 
>>> ./.git/svn/trunk/.rev_db.13f79535-47bb-0310-9956-ffa450edef68
> 
> The latest versions of git-svn should be vastly more space efficient
> with .rev_map files.
>  
>> Okay, besides the zero offset, there seems to be big problem with the 
>> svn repo:
>>
>> svn list -r 234478 http://svn.apache.org/repos/asf/synapse
>> branches/
>> site/
>> tags/
>> trunk/
>>
>> svn list -r 234478 http://svn.apache.org/repos/asf|grep synapse
> 
> Ah, that's because it started off in the /incubator directory, not /synapse:
> 
> $ svn log -v -r 234478 http://svn.apache.org/repos/asf
> ------------------------------------------------------------------------
> r234478 | dims | 2005-08-22 05:39:51 -0700 (Mon, 22 Aug 2005) | 3 lines
> Changed paths:
>    A /incubator/synapse/branches
>    A /incubator/synapse/site
>    A /incubator/synapse/tags
>    A /incubator/synapse/trunk
> 
> init synapse SVN

Yes, and later on there was some more moving around to "webservices/synapse/synapse", ".../syaps2", ".../synapse" (i.e. renaming "synapse/synapse" to "synapse") and then to the final destination.

Some svn client command know how to follow moves, other don't. I can't make them return results by using peg revisions, but I don't know how git-svn handles them.

Show 27 quoted lines
> ------------------------------------------------------------------------
> 
>> Right, nothing returned.
>>
>> git-svn insists on using http://svn.apache.org/repos/asf as the base 
>> URL, which is basically correct, but doesn't work for this repo.
>>
>> Even editing .git/config and .git/svn/metadata (after git svn init, 
>> befire the first fetch) to change the url doesn't help. git svn still 
>> uses .../asf. Is this as intended? cc'ing the git svn author.
> 
> Can you try using the latest git-svn and also --no-follow-parent?
> 
> --no-follow-parent won't traverse copy history so it may be faster,
> but you'll get an incomplete history.
> 
> On the other hand, it looks like the asf repo is hopelessly slow
> for history retrieval.
> 
> I'm pulling the asf log (svn log -v http://svn.apache.org/repos/asf >
> tmp.log) but it's taking forever...  git-svn uses basically the same
> operation as 'svn log -v' and I've yet to see an SVN server take this
> long to show history.
> 
> I've gotten 70,000 revisions after over one hour with the above svn log
> command.  I'm going to restart this from a server with a better
> connection since I need to power off this one when I sleep.

Note that I'm not the OP, I'm just a git-svn user who got curious about that post. I think the asf repo shows exactly those mistakes which svn drives its users into (and which I've made also, on a much smaller scale, of course). As far as I understand, the synapse repo there is historical, and they need to convert to git in order to feed it into a CMS. I don't know if OP is still interested.

Michael

← back to recent threads