threads / discuss / 18850

Again with git-svn: File was not found in commit

Subject: Again with git-svn: File was not found in commit

## tl;dr

9 messages between Apr 13, 2009 and Apr 28, 2009.

replies: 8people: 3as markdown or json

David E. Wheeler· Apr 13, 2009, 18:08 UTC · lore
Hello Gits,

I'm looking to migrate [Bricolage][] from Subversion to Git. As such, I'm running `git svn` to create a git repository for the Bricolage repository. I moved the files around in Subversion quite a while ago (the trunk, branches, and tags directories used to be at the root, but now they're under a "bricolage" subdirectory), so there's a fair amount of duplication, but I don't mind that, because I can prune those dupes.

[Bricolage]: http://bricolage.cc/
So I ran `git svn` like so:
     md brictmp
     cd brictmp
     git svn init http://svn.bricolage.cc/bricolage --no-metadata -s
     git config svn.authorsfile ~/Desktop/bricolage_committers.txt
     git svn fetch

After around 30 hours of the fan running full tilt on my MacBook Pro, creating 343 branches (yeah, we have a lot of tags), `git svn` exited with this error:

     bricolage/branches/rev_1_8/lib/Bric/App/ApacheConfig.pm was not  
found in commit
     e5145931069a511e98a087d4cb1a8bb75f43f899 (r5256)
This seemed strange to me, so I had a look at SVN:
     svn list -r5256 http://svn.bricolage.cc/bricolage/branches/rev_1_8/lib/Bric/App/ApacheConfig.pm
     ApacheConfig.pm

So the file *is* there in that revision. I had been running 1.6.1.2, so I upgraded to 1.6.2.2 and ran `git svn fetch` again to see if it would pick up where it left off, but it returned the same error (please don't tell me I have to start over!).

I see that this issue has come up [before][], earlier this year; has there been any progress on it? Is there perhaps something I can do to get this working?

[before]: http://lists-archives.org/git/677591-git-svn-file-was-not-found-in-commit.html
Many thanks,
David
Eric Wong· Apr 21, 2009, 18:02 UTC · re: David E. Wheeler · lore

Re: Again with git-svn: File was not found in commit

"David E. Wheeler" <david@kineticode.com> wrote:
Show 17 quoted lines
> After around 30 hours of the fan running full tilt on my MacBook Pro,  
> creating 343 branches (yeah, we have a lot of tags), `git svn` exited  
> with this error:
>
>     bricolage/branches/rev_1_8/lib/Bric/App/ApacheConfig.pm was not  
> found in commit
>     e5145931069a511e98a087d4cb1a8bb75f43f899 (r5256)
>
> This seemed strange to me, so I had a look at SVN:
>
>     svn list -r5256 http://svn.bricolage.cc/bricolage/branches/rev_1_8/lib/Bric/App/ApacheConfig.pm
>     ApacheConfig.pm
>
> So the file *is* there in that revision. I had been running 1.6.1.2, so I 
> upgraded to 1.6.2.2 and ran `git svn fetch` again to see if it would pick 
> up where it left off, but it returned the same error (please don't tell 
> me I have to start over!).

Interesting. I finally managed to reproduce it over the weekend. I'll try to look into it later tonight.

-- 
Eric Wong
David E. Wheeler· Apr 21, 2009, 18:16 UTC · re: Eric Wong · lore

Re: Again with git-svn: File was not found in commit

On Apr 21, 2009, at 11:02 AM, Eric Wong wrote:
Show 20 quoted lines
>>    bricolage/branches/rev_1_8/lib/Bric/App/ApacheConfig.pm was not
>> found in commit
>>    e5145931069a511e98a087d4cb1a8bb75f43f899 (r5256)
>>
>> This seemed strange to me, so I had a look at SVN:
>>
>>    svn list -r5256 http://svn.bricolage.cc/bricolage/branches/rev_1_8/lib/Bric/App/ApacheConfig.pm
>>    ApacheConfig.pm
>>
>> So the file *is* there in that revision. I had been running  
>> 1.6.1.2, so I
>> upgraded to 1.6.2.2 and ran `git svn fetch` again to see if it  
>> would pick
>> up where it left off, but it returned the same error (please don't  
>> tell
>> me I have to start over!).
>
> Interesting.  I finally managed to reproduce it over the weekend.   
> I'll
> try to look into it later tonight.

Thanks, appreciate it. I've gotten around it by going back to the old CVS repository on SourceForge and migrating that, and then migrating the SVN repo only from after we moved to SVN. Then I'll just have to stitch the CVS and SVN migrations together and I'll be done.

Best,
David
Eric Wong· Apr 26, 2009, 01:53 UTC · re: David E. Wheeler · lore

Re: Again with git-svn: File was not found in commit

"David E. Wheeler" <david@kineticode.com> wrote:
Show 27 quoted lines
> On Apr 21, 2009, at 11:02 AM, Eric Wong wrote:
>
>>>    bricolage/branches/rev_1_8/lib/Bric/App/ApacheConfig.pm was not
>>> found in commit
>>>    e5145931069a511e98a087d4cb1a8bb75f43f899 (r5256)
>>>
>>> This seemed strange to me, so I had a look at SVN:
>>>
>>>    svn list -r5256 http://svn.bricolage.cc/bricolage/branches/rev_1_8/lib/Bric/App/ApacheConfig.pm
>>>    ApacheConfig.pm
>>>
>>> So the file *is* there in that revision. I had been running 1.6.1.2, 
>>> so I
>>> upgraded to 1.6.2.2 and ran `git svn fetch` again to see if it would 
>>> pick
>>> up where it left off, but it returned the same error (please don't  
>>> tell
>>> me I have to start over!).
>>
>> Interesting.  I finally managed to reproduce it over the weekend.   
>> I'll
>> try to look into it later tonight.
>
> Thanks, appreciate it. I've gotten around it by going back to the old  
> CVS repository on SourceForge and migrating that, and then migrating the 
> SVN repo only from after we moved to SVN. Then I'll just have to stitch 
> the CVS and SVN migrations together and I'll be done.
Hopefully the workaround is working out for you.

The problem with r5256 is quite nasty and I can't think of a good automated solution right now involving --stdlayout/branches/tags

------------------------------------------------------------------------
r5256 | theory | 2004-05-09 19:17:49 -0700 (Sun, 09 May 2004) | 2 lines
Changed paths:
   A /bricolage/branches/rev_1_8 (from /bricolage/branches/rev_1_8_temp/bricolage:5255)
   D /bricolage/branches/rev_1_8_temp/bricolage
Permanent home.
------------------------------------------------------------------------

The problem is that the "bricolage" subdirectory became the top-level tree and git-svn doesn't have a way of detecting that. This is occasionally a problem with other repos, too

Overall, this repo is very nasty given the amount of times follow-parent needs to be used (refs with a "@<revno>" suffix). The --stdlayout switch is really for the common repositories with a consistent depth for branches/tags. Unfortunately, SVN is way too freeform in this regard and mixed-depth branches/tags hierarchies have always been a problem :<

It might be a bit easier to write (or script) the "fetch" lines (like the one --stdlayout generates for trunk) for each tag/branch and maybe use --no-follow-parent.

-- 
Eric Wong
David E. Wheeler· Apr 26, 2009, 02:40 UTC · re: Eric Wong · lore

Re: Again with git-svn: File was not found in commit

On Apr 25, 2009, at 6:53 PM, Eric Wong wrote:
> Hopefully the workaround is working out for you.
[Finally], yes.
[Finally]: http://justatheory.com/computers/vcs/git/bricolage-to-git.html
Show 18 quoted lines
> The problem with r5256 is quite nasty and I can't think of a good
> automated solution right now involving --stdlayout/branches/tags
>
> ------------------------------------------------------------------------
> r5256 | theory | 2004-05-09 19:17:49 -0700 (Sun, 09 May 2004) | 2  
> lines
> Changed paths:
>   A /bricolage/branches/rev_1_8 (from /bricolage/branches/ 
> rev_1_8_temp/bricolage:5255)
>   D /bricolage/branches/rev_1_8_temp/bricolage
>
> Permanent home.
>
> ------------------------------------------------------------------------
>
> The problem is that the "bricolage" subdirectory became the top-level
> tree and git-svn doesn't have a way of detecting that.  This is
> occasionally a problem with other repos, too

We moved shit around a *lot*. There were two times we moved things around: First, when Bricolage was first imported from CVS, it assumed that the root directory was the project directory, even though we had several projects. So we had

/trunk/ /branches/ /tags/

We spent several hundred commits moving things around so that we would have the more standard:

/bricolage/trunk/ /bricolage/branches/ /bricolage/tags/

Fortunately, we did all of this moving before we committed anything useful to Subversion. So for this project, I first converted the old SourceForge CVS repository to Git, then converted the Subversion repository starting from after all this moving around (r5517, well after the problematic commit), and then I joined them together, as detailed in my blog post.

Show 8 quoted lines
> Overall, this repo is very nasty given the amount of times follow- 
> parent
> needs to be used (refs with a "@<revno>" suffix).  The --stdlayout
> switch is really for the common repositories with a consistent depth  
> for
> branches/tags.  Unfortunately, SVN is way too freeform in this regard
> and mixed-depth branches/tags hierarchies have always been a  
> problem :<
Tell me about it. This is why it took 30 hours before `git-svn` failed.

The other time we moved stuff around was to reorganize tags into subdirectories. This was because we had so many tags that they were unwieldy as a directory listing in SVN. This turned out to be a PITA when migrating, but I was able to [work around it] using GitX and some pretty hacky code to generate grafts. It'd be nice if there was a way to tell git-svn how deep a hierarchy to look for for tags (and perhaps branches), although it wouldn't have helped us, because we *changed* that hierarchy.

[work around it]: http://justatheory.com/computers/vcs/git/bricolage-svn-to-git.html

I'm so glad to have it all in Git now where things are far more rationally organized.

> It might be a bit easier to write (or script) the "fetch" lines
> (like the one --stdlayout generates for trunk) for each tag/branch
> and maybe use --no-follow-parent.

--no-follow-parent was indeed key for me being able to get this done. Thanks for writing such a capable client that, although it couldn't reconstruct what we were thinking when we moved things around, it let me tell it where to stop and what stuff to ignore (ultimately, I had it ignore all tag directories) so that I could get the job done.

The new Git repository is now on [GitHub], if you're curious.
[GitHub]: http://github.com/bricoleurs/bricolage/
Thanks,
David
Jakub Narebski· Apr 26, 2009, 08:17 UTC · re: David E. Wheeler · lore

Re: Again with git-svn: File was not found in commit

"David E. Wheeler" <david@kineticode.com> writes:
Show 7 quoted lines
> On Apr 25, 2009, at 6:53 PM, Eric Wong wrote:
> 
> > Hopefully the workaround is working out for you.
> 
> [Finally], yes.
> 
> [Finally]: http://justatheory.com/computers/vcs/git/bricolage-to-git.html
Thank you for posting this detailed analysis.
-- 
Jakub Narebski
Poland
ShadeHawk on #git
David E. Wheeler· Apr 27, 2009, 00:27 UTC · re: Jakub Narebski · lore

Re: Again with git-svn: File was not found in commit

On Apr 26, 2009, at 1:17 AM, Jakub Narebski wrote:
>> [Finally]: http://justatheory.com/computers/vcs/git/bricolage-to-git.html
>
> Thank you for posting this detailed analysis.

My pleasure; hopefully someone will get something out of it to help with *their* migration. I'll likely publish some final notes and the scripts I wrote sometime this week.

Best,
David
Jakub Narebski· Apr 28, 2009, 09:40 UTC · re: David E. Wheeler · lore

Re: Again with git-svn: File was not found in commit

"David E. Wheeler" <david@kineticode.com> writes:
Show 9 quoted lines
> On Apr 26, 2009, at 1:17 AM, Jakub Narebski wrote:
> 
>>> [Finally]: http://justatheory.com/computers/vcs/git/bricolage-to-git.html
>>
>> Thank you for posting this detailed analysis.
> 
> My pleasure; hopefully someone will get something out of it to help
> with *their* migration. I'll likely publish some final notes and the
> scripts I wrote sometime this week.

BTW the above story was mentioned in "Perlbuzz news roundup 2009-04-27" http://perlbuzz.com/2009/04/perlbuzz-news-roundup-2009-04-27.html

P.S. Any relation to David A. Wheeler (http://www.dwheeler.com/blog)?
-- 
Jakub Narebski
Poland
ShadeHawk on #git
David E. Wheeler· Apr 28, 2009, 16:15 UTC · re: Jakub Narebski · lore

Re: Again with git-svn: File was not found in commit

On Apr 28, 2009, at 2:40 AM, Jakub Narebski wrote:
Show 5 quoted lines
> BTW the above story was mentioned in "Perlbuzz news roundup  
> 2009-04-27"
> http://perlbuzz.com/2009/04/perlbuzz-news-roundup-2009-04-27.html
>
> P.S. Any relation to David A. Wheeler (http://www.dwheeler.com/blog)?
No, just my doppelgänger. :-)
Best,
David

← back to recent threads