git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: git-svn and repository hierarchy?

From
JWJosef Wolf <jw@raven.inka.de>
Date
Mar 4, 2009, 19:27 UTC
Message-ID
<20090304192752.GC11278@raven.wolf.lan>
In-Reply-To
<eaa105840903031618s5e0b6f24j64aade8d752fb11@mail.gmail.com>
On Tue, Mar 03, 2009 at 07:18:04PM -0500, Peter Harris wrote:
Show 9 quoted lines
> On Tue, Mar 3, 2009 at 5:36 PM, Josef Wolf wrote:
> >
> > I'd rather not let every clone talk to subversion for several reasons.
> > One of them is that it is very inconvenient (e.g. the password has to
> > be entered several times for every commit).
> 
> Sounds like subversion isn't properly caching your credentials, or
> your admin is paranoid and turned off the svn credential cache. I
> can't remember the last time I had to enter a password.

What credential cache are you talking about? I don't know of any worth to be mentioned. If you talk about "store-passwords=yes" in subversion's config or about the .netrc file that has to be used for git, you might be interested to read http://curl.haxx.se/docs/adv_20090303.html If you know about another method, please tell me.

BTW: The paranoid admin is myself.   ;-)
BTW1: I know there's the possibility of client certificates.  But AFAIK,
      there's no equivalent to ssh-agent, so it's pointless.
> Of course, git-svn-repo can't cache credentials, since it has to
> impersonate different users. You are impersonating different users so
> that the svn author field is correct, aren't you? But that shouldn't
> be a problem for userN working on cloneN.

Ah, that would have been one of my next questions (I don't like to start new threads as long as old threads are on-going ;-) Within svn, the author field seems to be correct (at least, it is identical to those of the commits that were done directly to svn) But when the commits come back to git, it is set to "jw <jw@041dc122-08ea-11de-a769-bddc3e64b585>" with the host-part being the uuid of the svn-repo.

Any ideas how to fix that?

Oh, and what happens if svn's rev-properties (commit log, author, date...) are changed?

Show 10 quoted lines
> >  After all, the whole point
> > for having git-svn-repos is for the clone to avoid working directly
> > against the subversion repos.  If every clone works against subversion
> > anyway, I can get rid of git-svn-repos as well.
> 
> From my perspective, the main advantage of git-svn-repos is the inital
> clone. Subversion is way too slow to clone an entire project's history
> (days, vs minutes for git). Subsequent 'git pull --rebase's are faster
> than 'git svn rebase's, too, although not by the same ratio (except
> for large subtree moves, which really are that much faster).

Of course, initial clone is an argument. But you do this exactly once per clone. And you can do that unattended over the weekend.

In contrast, working directly against an svn repository is _very_ tedious. With a dozen commits pending, you have to enter your password about 30..50 times in "git svn dcommit". While you can run the initial clone unattended over the weekend, there's currently no (sensible) way to avoid this massive interactive password hammering. And you have to do that on every dcommit.

Show 18 quoted lines
> > On Tue, Mar 03, 2009 at 02:35:28PM -0500, Peter Harris wrote:
> >> Also, this line says "rebase my changes onto those of ../clone$clone",
> >> which isn't what you want. It will end up rebasing svn commits that
> >> the client didn't have on top of the client's commits, and will break
> >> git-svn's index. Don't use --rebase here.
> >
> > Hmm, I must have misunderstood Michael, then.  Wasn't he suggesting
> > to rebase here?  Here's the citation:
> >
> > |>   (cd git-svn-repos; git pull ../clone1)  # if this line is executed,
> > |
> > |That's the problem. This creates a merge after which you 1-2-3-4 and
> > |1-2'-3'-4' plus the merge of 4 and 4'.
> > |
> > |Instead, use git pull --rebase here. You don't want merges in the branch
> > |from which you dcommit.
> 
> I think he meant to say "git pull ../cloneN && git rebase trunk".

Did you mean "git pull ../cloneN && git rebase master"? There's no trunk in git, AFAIK?

So here's my current idea about the work flow.  At the end of this mail,
I've attached the whole test-script again.  But this time, I have
extracted the main tasks
  1. work on some clones
  2. move commits from clone to subversion
  3. move commits received from subversion to clone
to make them more self-contained.  Ideally, those three tasks would
be packed into shell functions or something...
  # 1. work on some clones
  #
  cd clone$clone
  git checkout master
  git pull --rebase                     # need rebase?
  git checkout -b topic-branch
  for commit in 1 2 3; do
    echo change $clone $commit >>test
    git commit -a -m "commit $clone $commit"
  done
  # 2. move commits from clone to subversion
  #
  cd git-svn-repos
  git svn rebase
  git pull ../clone$clone topic-branch   # need rebase?
  # Check for conflict and pretend we have resolved it
  grep change test >test.resolved
  if diff test test.resolved ; then
    rm test.resolved
  else
    mv test.resolved test
    git add test
    git commit -m "merge"
  fi
  git svn dcommit
  # 3. move commits received from subversion to clone
  #
  cd clone$clone
  git checkout master
  git pull                               # need rebase?
  git branch -D topic-branch

This work flow seems to generate a proper result and a sensible history. But I am still not sure about when (and how) I have to rebase. I have marked the lines in question with "#need rebase?".

#! /bin/sh
(
  set -x
  # create test directory
  #
  TESTDIR=`mktemp --tmpdir=. git-svn-hierarchy-test-XXXXXXXX`
  rm -rf $TESTDIR
  mkdir -p $TESTDIR
  cd $TESTDIR
  SUBVERSION_REPOS=file://`pwd`/subversion-repos
  # create subversion repos with some history
  #
  svnadmin create subversion-repos
  svn -m "create standard layout" mkdir \
      $SUBVERSION_REPOS/trunk \
      $SUBVERSION_REPOS/branches \
      $SUBVERSION_REPOS/tags
  svn co $SUBVERSION_REPOS/trunk subversion-wc
  echo change1 >>subversion-wc/test
  svn add subversion-wc/test
  svn ci -m "commit 0" subversion-wc
  # create git-svn-repos
  #
  git svn init --stdlayout $SUBVERSION_REPOS git-svn-repos
  (cd git-svn-repos; git svn fetch)
  # create some clones
  #
  git clone git-svn-repos clone1
  git clone git-svn-repos clone2
  git clone git-svn-repos clone3
  for round in 1 2 3; do
    for clone in 1 2 3; do
     # work on clone
     #
     (
       cd clone$clone
       git checkout master
       git pull --rebase
       git checkout -b topic-branch
       for commit in 1 2 3; do
         echo change $round $clone $commit >>test
         git commit -a -m "commit $round $clone $commit"
       done
     )
     # move commits from clone to subversion
     #
     (
       cd git-svn-repos
       git svn rebase
       git pull ../clone$clone topic-branch
       grep change test >test.resolved
       # Check for conflict and pretend we have resolved it
       if diff test test.resolved ; then
         rm test.resolved
       else
         mv test.resolved test
         git add test
         git commit -m "merge"
       fi
       git svn dcommit
     )
     # move commits from subversion to clone
     (
       cd clone$clone
       git checkout master
       git pull
       git branch -D topic-branch
     )
    done
    # commit from svn
    #
    (
      cd subversion-wc
      svn up
      echo change $round s >>test
      svn ci -m "commit $round s"
    )
  done
)
Previous: Peter HarrisNext: Peter Harris
Message 17 of 24 in “git-svn and repository hierarchy?”
  1. Josef WolfFeb 24, 2009
  2. Michael J GruberFeb 25, 2009
  3. Josef WolfFeb 25, 2009
  4. Peter HarrisFeb 26, 2009
  5. Josef WolfFeb 27, 2009
  6. Peter HarrisFeb 27, 2009
  7. Josef WolfFeb 27, 2009
  8. Peter HarrisFeb 28, 2009
  9. Josef WolfFeb 27, 2009
  10. Michael J GruberFeb 27, 2009
  11. Josef WolfFeb 27, 2009
  12. Michael J GruberFeb 28, 2009
  13. Josef WolfMar 3, 2009
  14. Peter HarrisMar 3, 2009
  15. Josef WolfMar 3, 2009
  16. Peter HarrisMar 4, 2009
  17. Josef WolfMar 4, 2009
  18. Peter HarrisMar 4, 2009
  19. Josef WolfMar 5, 2009
  20. Peter HarrisMar 5, 2009
  21. Josef WolfMar 6, 2009
  22. Peter HarrisMar 6, 2009
  23. Josef WolfMar 6, 2009
  24. Florian MicklerMar 8, 2009

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.