{"thread":{"id":"14319","subject":"git-svn sucks when it should not","startedAt":"2008-07-07T00:00:21Z","lastAt":"2008-07-07T17:18:48Z","messageCount":5,"participants":["Johannes Schindelin","Eric Wong","Avery Pennarun"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"82405","messageId":"alpine.DEB.1.00.0807070135450.7342@eeepc-johanness","threadId":"14319","inReplyTo":null,"subject":"git-svn sucks when it should not","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-07-07T00:00:21Z","receivedAt":"2008-07-07T00:00:21Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Eric,\n\nI have the pleasure of needing to work with a subversion project where \nparts of the webserver are password restricted.\n\nIn particular, I cannot access the parent directory, and one of \nthe branches is protected, too.\n\nMaybe you remember me describing that problem on IRC a few weeks ago: yes, \nit is still persistent.\n\nNow, I thought that I know my way around Perl, at least a little bit, but \nwhile git-svn barfed on the repository, I... uhm, well, you probably get \nthe idea.\n\nThe funny part is this: when I say \"git svn clone $URL/trunk\", or the same \nwith the absolute paths to the single tags, instead of \"git svn clone -s \n$URL\", git-svn does the correct thing.  It works, importing the stuff as \n\"git-svn\".\n\nSo I tried to just edit out by hand the branches section, so that the \npassword-protected branch would not be a problem.\n\nThe result was surprising: git svn fetch exited with success, but it \ndid... absolutely nothing.\n\nAfter a lot of frustrating hours, which were not at all helped by \nbrilliant variable names such as \"r\" and \"gsv\", I now know this: the log \ncontains paths that do not have a prefix \"trunk\", but \"<dir>/trunk\", \nwhere \"<dir>\" is the last directory of the URL.\n\nChanging git-svn's URL to the parent of <dir> is a no-go, since that is -- \nas I mentioned above -- password protected.\n\nYes, in a perfect world I could just force the admin to change that, but \nno, this is not a perfect world, so do not even try to suggest that if \nyou want to help.\n\nChanging the fetch line to \"<dir>/trunk:refs/remotes/trunk\" does not work \neither, since git-svn cleverly checks $URL/<dir>/<dir>/trunk/.\n\nI then tried to hack match_globs() and match_paths() to add that extra \nprefix to the patterns, so that that extra prefix + trunk would be \nmatched and edited out.  This happened to work out alright.\n\nBut I tried for several hours to get in a proper solution which does not \nthrow up on the tags, and I have to conclude that this piece of code is \nnot hackable by anybody else but you.\n\nSo I stand defeated by your program.  Thank you.\n\nMy ugly, ugly workaround that is however easy, easy, is a shell script \nthat uses curl to find out what refs are new, and clones each ref \nindividually, then pushes all the results together into one repository.\n\nShould not have been _that_ hard,\nDscho\n"},{"id":"82434","messageId":"20080707094438.GA5964@untitled","threadId":"14319","inReplyTo":"alpine.DEB.1.00.0807070135450.7342@eeepc-johanness","subject":"Re: git-svn sucks when it should not","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2008-07-07T09:44:38Z","receivedAt":"2008-07-07T09:44:38Z","isPatch":false,"sender":{"key":"e@80x24.org","avatar":null},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:\n> Hi Eric,\n\nHi Johannes,\n\n> I have the pleasure of needing to work with a subversion project where \n> parts of the webserver are password restricted.\n> \n> In particular, I cannot access the parent directory, and one of \n> the branches is protected, too.\n> \n> Maybe you remember me describing that problem on IRC a few weeks ago: yes, \n> it is still persistent.\n> \n> Now, I thought that I know my way around Perl, at least a little bit, but \n> while git-svn barfed on the repository, I... uhm, well, you probably get \n> the idea.\n> \n> The funny part is this: when I say \"git svn clone $URL/trunk\", or the same \n> with the absolute paths to the single tags, instead of \"git svn clone -s \n> $URL\", git-svn does the correct thing.  It works, importing the stuff as \n> \"git-svn\".\n\nNoted, -s/--stdlayout doesn't work well with access-restricted\nrepositories.  I remember leaving the minimize_url() stuff out of the\nnon--stdlayout code path so git-svn could at least have a way of working\nwith restricted repositories.\n\n> So I tried to just edit out by hand the branches section, so that the \n> password-protected branch would not be a problem.\n> \n> The result was surprising: git svn fetch exited with success, but it \n> did... absolutely nothing.\n> \n> After a lot of frustrating hours, which were not at all helped by \n> brilliant variable names such as \"r\" and \"gsv\", I now know this: the log \n> contains paths that do not have a prefix \"trunk\", but \"<dir>/trunk\", \n> where \"<dir>\" is the last directory of the URL.\n\n$r should always revision number ('r' is pretty common in the svn\ncommand-line client in referring to revision numbers).  $gsv is just\na $gs vector (like 'argv') with multiple Git::SVN objects.\n\nOn the other hand, your comments about some of the other code being\ngross is completely valid...\n\n> Changing git-svn's URL to the parent of <dir> is a no-go, since that is -- \n> as I mentioned above -- password protected.\n> \n> Yes, in a perfect world I could just force the admin to change that, but \n> no, this is not a perfect world, so do not even try to suggest that if \n> you want to help.\n\nIn a perfect world, everybody would be using git and we wouldn't have to\ndeal with SVN :)\n\nI highly doubt I'd ask you to get an admin to give you access, so please\ndon't suggest that's something I would ask of you.  I always try to stay\nas under-the-radar as possible as far as dealing with admins go and\navoid ever mentioning git-svn to them.\n\nClosed repositories suck, yes, and I've had the fortune (and misfortune\nto some users such as yourself) to have never had work with anything\nless than full read access.\n\n> Changing the fetch line to \"<dir>/trunk:refs/remotes/trunk\" does not work \n> either, since git-svn cleverly checks $URL/<dir>/<dir>/trunk/.\n> \n> I then tried to hack match_globs() and match_paths() to add that extra \n> prefix to the patterns, so that that extra prefix + trunk would be \n> matched and edited out.  This happened to work out alright.\n> \n> But I tried for several hours to get in a proper solution which does not \n> throw up on the tags, and I have to conclude that this piece of code is \n> not hackable by anybody else but you.\n\nI've actually been afraid to touch the globbing/paths stuff myself.  I\ngot it working for common repository types ~18 months ago and have been\nafraid to look at it ever since, so I definitely feel your pain.\n\ngit-svn works alright for most repositories people come across, but\nyes, it has trouble with less-common, restricted ones.\n\nDoes it do some brain-damaged things?  Yes, and those things should get\nfixed.  But nowadays I'm much busier with other projects and working\nwith SVN/SVN bindings was a traumatic experience for me, and (possibly\nas a result) the git-svn code scares me, too.\n\nA first step to making the git-svn code more accessible would be\nto split out the individual packages into smaller, more manageable\nfiles.  The second would be documenting more of the internal data\nstructures that it uses (including the ones SVN returns).\n\nRewriting the paths/globbing code from scratch would probably be\nthe next step...\n\n> So I stand defeated by your program.  Thank you.\n> \n> My ugly, ugly workaround that is however easy, easy, is a shell script \n> that uses curl to find out what refs are new, and clones each ref \n> individually, then pushes all the results together into one repository.\n> \n> Should not have been _that_ hard,\n> Dscho\n\nIt might be helpful to publish your script so other people can see/use it.\n\ndcommit, multi-init/fetch (precursor to clone --stdlayout) were all\nprototyped as shell scripts using git-svn commands before they were\nintegrated into the main program.\n\n-- \nEric Wong\n"},{"id":"82437","messageId":"alpine.DEB.1.00.0807071242180.18205@racer","threadId":"14319","inReplyTo":"20080707094438.GA5964@untitled","subject":"Re: git-svn sucks when it should not","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-07-07T11:49:09Z","receivedAt":"2008-07-07T11:49:09Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 7 Jul 2008, Eric Wong wrote:\n\n> Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:\n>\n> > [...] a shell script that uses curl to find out what refs are new, and \n> > clones each ref individually, then pushes all the results together \n> > into one repository.\n> \n> It might be helpful to publish your script so other people can see/use \n> it.\n\n-- snipsnap --\n#!/bin/sh\n\n# This script looks for the config variable svn-manual.url, and if it\n# is set, will traverse the url and its subdirectories with curl, and\n# install different svn-remotes for all found refs.\n#\n# It heavily relies on curl being able to screen-scrape the directories,\n# in other words, it wants an HTTP on the other side that has directory\n# listings enabled.\n#\n# The quick and dirty heuristics to find out what makes a ref is that\n# a ref's subdirectory contains files, while a subdirectory containing\n# only subdirectories is supposed to contain refs (or subdirectories\n# of refs).\n\nlist_contains_files () {\n\twhile test $# -gt 0\n\tdo\n\t\tcase \"$1\" in\n\t\t*/) ;;\n\t\t*) echo \"$1\";;\n\t\tesac\n\t\tshift\n\tdone\n}\n\nsvn_manually_fetch_one_dir () {\n\tcontents=\"$(curl --silent -k \"$1\"/ |\n\t\tsed -n \"s/.*a href=\\\"\\([^\\\"]*\\).*/\\1/p\" |\n\t\tgrep -ve '^\\.\\./$' -e '^http:' -e '^/')\"\n\ttest -z \"$contents\" && return\n\ttest -z \"$(list_contains_files $contents)\" || {\n\t\ttest -z \"$(git config svn-remote.\"$2\".url)\" && {\n\t\t\tgit config svn-remote.\"$2\".url \"$1\" &&\n\t\t\tgit config svn-remote.\"$2\".fetch :\"$2\" ||\n\t\t\treturn\n\t\t}\n\t\tgit svn fetch -R \"$2\"\n\t\treturn\n\t}\n\tfor dir in $contents\n\tdo\n\t\tdir=${dir%%/}\n\t\tsvn_manually_fetch_one_dir \"$1/$dir\" \"$2/$dir\" || break\n\tdone\n}\n\nsvn_fetch_semi_manually () {\n\turl=\"$(git config svn-manual.url)\"\n\ttest -z \"$url\" && return 1\n\tsvn_manually_fetch_one_dir \"$url\" refs/remotes\n}\n\nsvn_fetch_semi_manually || git svn fetch\n"},{"id":"82469","messageId":"32541b130807070929q65e9a0d8g788cdb8c28ef28b9@mail.gmail.com","threadId":"14319","inReplyTo":"alpine.DEB.1.00.0807071242180.18205@racer","subject":"Re: git-svn sucks when it should not","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-07-07T16:29:43Z","receivedAt":"2008-07-07T16:29:43Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On 7/7/08, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:\n>  # It heavily relies on curl being able to screen-scrape the directories,\n>  # in other words, it wants an HTTP on the other side that has directory\n>  # listings enabled.\n\nI wrote a similar script myself, although it makes assumptions about\nthe meaning of /branches and /tags rather than using the dirs vs.\nfiles trick.\n\nRather than relying on screen scraping with curl, you might prefer to\nuse \"svn ls\" instead, since it'll work with any svn-compatible\nrepository type and (I presume) doesn't require special web server\nsettings.\n\nHave fun,\n\nAvery\n"},{"id":"82471","messageId":"alpine.DEB.1.00.0807071818250.18205@racer","threadId":"14319","inReplyTo":"32541b130807070929q65e9a0d8g788cdb8c28ef28b9@mail.gmail.com","subject":"Re: git-svn sucks when it should not","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-07-07T17:18:48Z","receivedAt":"2008-07-07T17:18:48Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 7 Jul 2008, Avery Pennarun wrote:\n\n> On 7/7/08, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:\n> >  # It heavily relies on curl being able to screen-scrape the \n> >  # directories, in other words, it wants an HTTP on the other side \n> >  # that has directory listings enabled.\n> \n> I wrote a similar script myself, although it makes assumptions about the \n> meaning of /branches and /tags rather than using the dirs vs. files \n> trick.\n> \n> Rather than relying on screen scraping with curl, you might prefer to \n> use \"svn ls\" instead, since it'll work with any svn-compatible \n> repository type and (I presume) doesn't require special web server \n> settings.\n\nGood to know!  At least if you want to use svn more often ;-)\n\nThanks,\nDscho\n"}]}