# Re: native-git-svn: A Summer of Code 2010 proposal

6 messages from 2010-03-27 to 2010-03-29. Participants: Steven Michalske, Ramkumar Ramachandra, Eric Raymond.
Thread: https://gitlist.dev/t/23198

## Steven Michalske, 2010-03-27 05:40

Subject: Re: native-git-svn: A Summer of Code 2010 proposal
Message-ID: <3d4937ff1003262240t6159d9c5sc9253f555c3aed1@mail.gmail.com>
URL: https://gitlist.dev/e/3d4937ff1003262240t6159d9c5sc9253f555c3aed1%40mail.gmail.com

```
Ramkumar Ramachandra <artagnon <at> gmail.com> writes:
---8<------
> The following resources are relevant to the project:
> 1. git_remote_helpers/git/git.py is a minimalistic remote helper
> written by Sverre. I plan to extend this as much as possible before
> rewriting it in C.

Would cython meet the needs of increasing the speed of the python code
without requiring a rewrite?

> 2. libsvn contains excellent documentation and clear examples to
> create the SVN client.
> 3. git-svn.perl has a lot functionality that I plan to re-implement in
> native-git-svn:
>    3.1 parse_svn_date: Given a date (in UTC) from Subversion, return a
> string in the format "<TZ Offset> <local date/time>" that Git will use
>    3.2 load_authors: <svn username> = real-name <email address>
> mapping based on git-svnimport

One feature that I would like to see is a way to call an application for a name
lookup author file maintenance.

i.e. if the SVN authors file is missing the lookup,  call the lookup tool.

I work at a company with a LDAP server that I can look up the svn username to
get real name and email address.  This way I don't have to manually maintain a
svn authors file.

This is really a remote helper component, not just SVN

>    3.3 do_git_init_db: Create and maintain svn-remotes
>    3.4 get_commit_entry: Parse commit messages, and encode them; SVN
> requires messages to be UTF-8 when entering the repo
>    3.5 cmd_branch: Handle branching/ tagging

I'm torn on how the current system handles this,  I like all tags to
be tags, and
that if a tag had a branch like behavior (bad SVN users!), that a branch exists
for it, with the tag pointing to its branches head.

>    3.6 cmd_create_ignore: Reads svn:ignore and puts the information
> into .gitignore
> 4. There are several existing third-party SVN exporters worth looking into [2].
-----8<------

A couple of side thoughts.

--
Support for SVN's blank folders.  Some of the old build systems I have used
need the blank folders, so I have to create to make the build work :-(
can't use
git-bisect easily.  Well it's that i have to make the bisect run
script make the
needed folders, not too hard, but annoying.  Could we track if in a particular
SVN revision we had a blank folder that was either created or removed.  Stuff a
hidden file '.git_svn_empty_folder' or a .gitignore with a * in it so
git can then
track the SVN's empty folder, and if the SVN folder gets contents the
.git_ignore
needs the ignore removed?

--
One of my SVN repositories using the current system fails to import that
repository is missing a revision in its SVN history.  In other words
the SVN repo
has corrupted history the current git-svn will fail to import the repository.

Example:  R31 of the SVN repo is this status, R32 fails to checkout
due to a SVN
error, but R33 will checkout and is valid.  I would like to see the
helper pause
and ask me what to do.  Either fail or skip that revision.  It's a
shame the history
is gone, but I now have to tell the current git-svn to do a shallow clone and
start at R33 and I loose all of commits R1 to R31, This leads to branches that
have no known roots......  My case is this happened roughly at
revision 1700, the
server's hard drive crashed and restore was done with a backup that was at a
revision around 1500 so there is a big gap..... of lost history, but
that history is
5-6 years old so the daily backups to reconstruct it are LONG gone.
Too bad we didn't have git back then, could have restored all the
history with a
push! ;-)

If you want me to test your work on a hairy repository with corrupt history and
thousands of branches, I'll do that for you.


--
In that same corrupt repository, each branch has a large PDF that NEVER
changes.  This makes me think that it might make exports faster if I could tell
the SVN client that a file is static, and to only track if it gets
removed or size
changes,  don't know if libsvn would let you do that....  You might
even be able
to detect that kind of condition, large unchanging binary like files,
 might make a great bandwidth/speed optimization.



Sorry if I didn't see these points brought up in other emails on your
proposal.
But working at a company with lots of history in SVN makes me passionate
about the SVN integration in git :-)


Good luck!
Steve

```

## Ramkumar Ramachandra, 2010-03-27 06:46

Subject: Re: native-git-svn: A Summer of Code 2010 proposal
Message-ID: <f3271551003262346g286e7e72u751e15cbc99a9c1@mail.gmail.com>
URL: https://gitlist.dev/e/f3271551003262346g286e7e72u751e15cbc99a9c1%40mail.gmail.com
In-Reply-To: <3d4937ff1003262240t6159d9c5sc9253f555c3aed1@mail.gmail.com>

```
[Please don't cull people from cc]

> Would cython meet the needs of increasing the speed of the python code
> without requiring a rewrite?

Actually, I've subsequently decided that this is unnecessary. Besides,
I'm writing in C mainly for portability to msysgit (Windows). Could
you please look at the updated version of my project proposal?

> I work at a company with a LDAP server that I can look up the svn username to
> get real name and email address.  This way I don't have to manually maintain a
> svn authors file.

> I'm torn on how the current system handles this,  I like all tags to
> be tags, and
> that if a tag had a branch like behavior (bad SVN users!), that a branch exists
> for it, with the tag pointing to its branches head.

This shouldn't be a problem to implement/ improve at the end of my
GSoC term. However, it's important that I don't lose focus and
concentrate on the core task at hand for GSoC, which is more about
getting native support for SVN than anything else. I have neither the
expertise or time (one GSoC term) to build a fantastic importer and
get native support: I will be re-using several parts of existing
importers for the purpose of the GSoC.

> Support for SVN's blank folders.  Some of the old build systems I have used
> need the blank folders, so I have to create to make the build work :-(

Okay, this should be simple enough to implement. Thank you for pointing it out.

> One of my SVN repositories using the current system fails to import that
> repository is missing a revision in its SVN history.  In other words
> the SVN repo
> has corrupted history the current git-svn will fail to import the repository.

I'll keep this in mind when designing svn-fast-import: a certain
revision's checkout can fail, and a mechanism to bail the user out of
such a situation can be helpful. Again, I can't promise that this'll
be completed by the end of the GSoC term, but I will make it easy
enough to write the functionality in later on.

> If you want me to test your work on a hairy repository with corrupt history and
> thousands of branches, I'll do that for you.

Thanks! That'll be wonderful. If my proposal gets accepted, I
certainly will contact you (and several others) for testing, once the
core task of the GSoC is complete.

> But working at a company with lots of history in SVN makes me passionate
> about the SVN integration in git :-)

Good to know. Thank you for your support :)

-- Ram

```

## Steven Michalske, 2010-03-27 08:03

Subject: Re: native-git-svn: A Summer of Code 2010 proposal
Message-ID: <9C72512D-F49A-4E94-A5D2-5F8D5BCCDFFA@gmail.com>
URL: https://gitlist.dev/e/9C72512D-F49A-4E94-A5D2-5F8D5BCCDFFA%40gmail.com
In-Reply-To: <f3271551003262346g286e7e72u751e15cbc99a9c1@mail.gmail.com>

```
On Mar 26, 2010, at 11:46 PM, Ramkumar Ramachandra  
<artagnon@gmail.com> wrote:

> [Please don't cull people from cc]

I didn't intend to,  I wasn't on the mailing list before and the  
message was not a reply to but a new message with the same subject.
>
>> Would cython meet the needs of increasing the speed of the python  
>> code
>> without requiring a rewrite?
>
> Actually, I've subsequently decided that this is unnecessary. Besides,
> I'm writing in C mainly for portability to msysgit (Windows). Could
> you please look at the updated version of my project proposal?
>> I work at a company with a LDAP server that I can look up the svn  
>> username to
>> get real name and email address.  This way I don't have to manually  
>> maintain a
>> svn authors file.
>
>> I'm torn on how the current system handles this,  I like all tags to
>> be tags, and
>> that if a tag had a branch like behavior (bad SVN users!), that a  
>> branch exists
>> for it, with the tag pointing to its branches head.
>
> This shouldn't be a problem to implement/ improve at the end of my
> GSoC term. However, it's important that I don't lose focus and
> concentrate on the core task at hand for GSoC, which is more about
> getting native support for SVN than anything else. I have neither the
> expertise or time (one GSoC term) to build a fantastic importer and
> get native support: I will be re-using several parts of existing
> importers for the purpose of the GSoC.
>
Clean documented locations for adding the calling of these kinds of  
addons is all that's needed,  then us other folks can chip in too with  
the bits like the LDAP support.

>> Support for SVN's blank folders.  Some of the old build systems I  
>> have used
>> need the blank folders, so I have to create to make the build  
>> work :-(
>
> Okay, this should be simple enough to implement. Thank you for  
> pointing it out.
>
>> One of my SVN repositories using the current system fails to import  
>> that
>> repository is missing a revision in its SVN history.  In other words
>> the SVN repo
>> has corrupted history the current git-svn will fail to import the  
>> repository.
>
> I'll keep this in mind when designing svn-fast-import: a certain
> revision's checkout can fail, and a mechanism to bail the user out of
> such a situation can be helpful. Again, I can't promise that this'll
> be completed by the end of the GSoC term, but I will make it easy
> enough to write the functionality in later on.
>
>> If you want me to test your work on a hairy repository with corrupt  
>> history and
>> thousands of branches, I'll do that for you.
>
> Thanks! That'll be wonderful. If my proposal gets accepted, I
> certainly will contact you (and several others) for testing, once the
> core task of the GSoC is complete.
>
>> But working at a company with lots of history in SVN makes me  
>> passionate
>> about the SVN integration in git :-)
>
> Good to know. Thank you for your support :)
>
> -- Ram

Happy hacking!

```

## Eric Raymond, 2010-03-27 09:19

Subject: Re: native-git-svn: A Summer of Code 2010 proposal
Message-ID: <20100327091938.GA4395@thyrsus.com>
URL: https://gitlist.dev/e/20100327091938.GA4395%40thyrsus.com
In-Reply-To: <3d4937ff1003262240t6159d9c5sc9253f555c3aed1@mail.gmail.com>

```
Steven Michalske <smichalske@gmail.com>:
> >    3.5 cmd_branch: Handle branching/ tagging
> 
> I'm torn on how the current system handles this,  I like all tags to
> be tags, and
> that if a tag had a branch like behavior (bad SVN users!), that a branch exists
> for it, with the tag pointing to its branches head.

Ah.  This sounds like s discussion, pre-dating my joining the list, of
my issue #2 about git-svn - not rendering unmodified tag directories as
git tags.  It's good that someone wants to rackle this seriously.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

```

## Eric Raymond, 2010-03-28 12:10

Subject: Re: native-git-svn: A Summer of Code 2010 proposal
Message-ID: <20100328121034.GC25402@thyrsus.com>
URL: https://gitlist.dev/e/20100328121034.GC25402%40thyrsus.com
In-Reply-To: <f3271551003280225v17af30d4s6d3d24b4d548ff7d@mail.gmail.com>

```
Ramkumar Ramachandra <artagnon@gmail.com>:
> Project Proposal: git-remote-svn | Native SVN support in Git
> Student: Ramkumar Ramachandra
> Mentor: Sverre Rabbelier

+1

I've just been through a Subversion-to-git migration, and have been
directly affected by an inadequacy in git-svn - failure to recognizer
SVN tags as tags rather than branches. See my blog post at
<http://esr.ibiblio.org/?p=1806>, "Subversion to GIT Migration: A Tale
of Two Gotchas" for discussion.

Accordingly, I support Ramkumar's proposal to rethink and rewrite the 
Subversion interface.  A concerted effort to do seamless interoperability 
would be well justified given the ubiquity of Subversion.  I think Rankumar
has chosen a goal that is useful, well defined, and realistically scoped.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

```

## Ramkumar Ramachandra, 2010-03-29 20:04

Subject: Re: native-git-svn: A Summer of Code 2010 proposal
Message-ID: <f3271551003291304u27c56d8fr648d7546f6256857@mail.gmail.com>
URL: https://gitlist.dev/e/f3271551003291304u27c56d8fr648d7546f6256857%40mail.gmail.com
In-Reply-To: <20100328121034.GC25402@thyrsus.com>

```
Hi,

Just a quick heads up: I've submitted my proposal via Google.

On Sun, Mar 28, 2010 at 5:40 PM, Eric Raymond <esr@thyrsus.com> wrote:
> Accordingly, I support Ramkumar's proposal to rethink and rewrite the
> Subversion interface.  A concerted effort to do seamless interoperability
> would be well justified given the ubiquity of Subversion.  I think Rankumar
> has chosen a goal that is useful, well defined, and realistically scoped.

Thank you :) I'll do my best.

-- Ram

```