{"thread":{"id":"26889","subject":"\"git svn mkdirs\" is very slow","startedAt":"2011-03-28T15:05:14Z","lastAt":"2011-04-01T10:26:00Z","messageCount":3,"participants":["Michael Haggerty","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"164440","messageId":"4D90A3AA.3050601@alum.mit.edu","threadId":"26889","inReplyTo":null,"subject":"\"git svn mkdirs\" is very slow","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2011-03-28T15:05:14Z","receivedAt":"2011-03-28T15:05:14Z","isPatch":false,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"Hello,\n\nI've noticed that in our repository, \"git svn mkdirs\" or any other\ncommand that calls mkemptydirs() is very slow (approximately 8.5 s even\nwith a warm disk cache).  Presumably this is related to the fact that\nthe unhandled.log.gz for our Subversion trunk is almost 10 Mb.\n\nOf course it would be nice if this were faster.\n\nBut we have laid out our repository to make the presence/absence of\nempty directories irrelevant, so we would be just as happy if git-svn\nwould *not* create empty directories, like in the good old days.  So I\nwanted to implement the following feature:\n\n1. An svn.autoMkdirs / svn-remote.<name>.autoMkdirs configuration\nvariable.  The value should default to true for backwards compatibility.\n\n2. Only call mkemptydirs() if this variable is set to true.\n\n3. Make an exception for \"git svn mkdirs\", which should do its thing\nregardless of how this configuration option is set.\n\nI think it should only be about a 10-line change, plus documentation and\ntests.  Unfortunately, my perl-foo is very limited, and it will take me\na while to figure out how option parsing and handling works in git-svn.\n\nWould this feature be welcome?\n\nIs there anybody willing to make the Perl changes?  I would be willing\nto work on the documentation and test suite changes.\n\nThanks,\nMichael\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\nhttp://softwareswirl.blogspot.com/\n"},{"id":"164470","messageId":"7vhbanywb4.fsf@alter.siamese.dyndns.org","threadId":"26889","inReplyTo":"4D90A3AA.3050601@alum.mit.edu","subject":"Re: \"git svn mkdirs\" is very slow","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-03-28T17:44:15Z","receivedAt":"2011-03-28T17:44:15Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Michael Haggerty <mhagger@alum.mit.edu> writes:\n\n> So I\n> wanted to implement the following feature:\n>\n> 1. An svn.autoMkdirs / svn-remote.<name>.autoMkdirs configuration\n> variable.  The value should default to true for backwards compatibility.\n>\n> 2. Only call mkemptydirs() if this variable is set to true.\n>\n> 3. Make an exception for \"git svn mkdirs\", which should do its thing\n> regardless of how this configuration option is set.\n>\n> I think it should only be about a 10-line change, plus documentation and\n> tests.  Unfortunately, my perl-foo is very limited, and it will take me\n> a while to figure out how option parsing and handling works in git-svn.\n>\n> Would this feature be welcome?\n>\n> Is there anybody willing to make the Perl changes?  I would be willing\n> to work on the documentation and test suite changes.\n\nSounds like a sensible thing to do, but I wonder if we also want a command\nline option --automkdirs (or --auto-create-empty-directories) in %fc_opts\nto make it overridable from the command line.  Perhaps it is not worth it.\n\nA completely untested patch is here.\n\n git-svn.perl |   18 ++++++++++++++++--\n 1 files changed, 16 insertions(+), 2 deletions(-)\n\ndiff --git a/git-svn.perl b/git-svn.perl\nindex a5857c1..fe4c716 100755\n--- a/git-svn.perl\n+++ b/git-svn.perl\n@@ -781,6 +781,16 @@ sub cmd_find_rev {\n \tprint \"$result\\n\" if $result;\n }\n \n+sub auto_create_empty_directories {\n+\tmy ($gs) = @_;\n+\tmy $var = eval { command_oneline('config', '--get', '--bool',\n+\t\t\t\t\t \"svn-remote.$gs->{repo_id}.automkdirs\") };\n+\t# By default, create empty directories by consulting the unhandled log,\n+\t# but allow setting it to 'false' to skip it.  I wonder if the variable\n+\t# should be \"skip create empty directories\", though...\n+\treturn !($var && $var eq 'false');\n+}\n+\n sub cmd_rebase {\n \tcommand_noisy(qw/update-index --refresh/);\n \tmy ($url, $rev, $uuid, $gs) = working_head_info('HEAD');\n@@ -804,7 +814,9 @@ sub cmd_rebase {\n \t\t$_fetch_all ? $gs->fetch_all : $gs->fetch;\n \t}\n \tcommand_noisy(rebase_cmd(), $gs->refname);\n-\t$gs->mkemptydirs;\n+\tif (auto_create_empty_directories($gs)) {\n+\t\t$gs->mkemptydirs;\n+\t}\n }\n \n sub cmd_show_ignore {\n@@ -1242,7 +1254,9 @@ sub post_fetch_checkout {\n \tcommand_noisy(qw/read-tree -m -u -v HEAD HEAD/);\n \tprint STDERR \"Checked out HEAD:\\n  \",\n \t             $gs->full_url, \" r\", $gs->last_rev, \"\\n\";\n-\t$gs->mkemptydirs($gs->last_rev);\n+\tif (auto_create_empty_directories($gs)) {\n+\t\t$gs->mkemptydirs($gs->last_rev);\n+\t}\n }\n \n sub complete_svn_url {\n"},{"id":"164882","messageId":"1301653560-10769-1-git-send-email-mhagger@alum.mit.edu","threadId":"26889","inReplyTo":"7vhbanywb4.fsf@alter.siamese.dyndns.org","subject":"[PATCH] git-svn: add an option to skip the creation of empty directories","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2011-04-01T10:26:00Z","receivedAt":"2011-04-01T10:26:00Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"\"git svn mkdirs\" (which creates empty directories in the current\nworking copy) can be very slow and is often unnecessary.  Provide a\nconfig file option \"svn-remote.<name>.automkdirs\" that prevents empty\ndirectories from being created automatically.  (They are still created\nif \"git svn mkdirs\" is invoked explicitly.)\n\nBased-on-patch-by: Junio C Hamano <gitster@pobox.com>\nSigned-off-by: Michael Haggerty <mhagger@alum.mit.edu>\n---\nI hope I have done the attribution correctly.  The git-svn.perl file\nchange is by Junio and the doc and test changes are by me.\n\nRegarding the name of the option: I prefer \"automkdirs\" to something\nlike \"skipautomkdirs\" because the latter is effectively the inverse of\ndoing something, leading to confusing double-negatives like \"if (!\nskipautomkdirs)\".  But I am not adamant about this preference.\n\nFor completeness it might be nice to have corresponding command-line\noptions like --auto-mkdirs and --no-auto-mkdirs, but I doubt that it\nis worth the extra effort.  If you want to create the directories when\nautomkdirs==false, you can simply run \"git svn mkdirs\".  It is more\ninconvenient if you have automkdirs==true but occasionally want to\nskip the creation of the directories, but this seems an uncommon\nscenario.\n\n Documentation/git-svn.txt     |   10 ++++++++++\n git-svn.perl                  |   17 +++++++++++++++--\n t/t9146-git-svn-empty-dirs.sh |   17 +++++++++++++++++\n 3 files changed, 42 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-svn.txt b/Documentation/git-svn.txt\nindex ea8fafd..f470fbf 100644\n--- a/Documentation/git-svn.txt\n+++ b/Documentation/git-svn.txt\n@@ -343,6 +343,8 @@ Any other arguments are passed directly to 'git log'\n \tEmpty directories are automatically recreated when using\n \t\"git svn clone\" and \"git svn rebase\", so \"mkdirs\" is intended\n \tfor use after commands like \"git checkout\" or \"git reset\".\n+\t(See the svn-remote.<name>.automkdirs config file option for\n+\tmore information.)\n \n 'commit-diff'::\n \tCommits the diff of two tree-ish arguments from the\n@@ -663,6 +665,14 @@ svn.pathnameencoding::\n \tlocales to avoid corrupted file names with non-ASCII characters.\n \tValid encodings are the ones supported by Perl's Encode module.\n \n+svn-remote.<name>.automkdirs::\n+\tNormally, the \"git svn clone\" and \"git svn rebase\" commands\n+\tattempt to recreate empty directories that are in the\n+\tSubversion repository.  If this option is set to \"false\", then\n+\tempty directories will only be created if the \"git svn mkdirs\"\n+\tcommand is run explicitly.  If unset, 'git svn' assumes this\n+\toption to be \"true\".\n+\n Since the noMetadata, rewriteRoot, rewriteUUID, useSvnsyncProps and useSvmProps\n options all affect the metadata generated and used by 'git svn'; they\n *must* be set in the configuration file before any history is imported\ndiff --git a/git-svn.perl b/git-svn.perl\nindex a5857c1..9a9b094 100755\n--- a/git-svn.perl\n+++ b/git-svn.perl\n@@ -781,6 +781,15 @@ sub cmd_find_rev {\n \tprint \"$result\\n\" if $result;\n }\n \n+sub auto_create_empty_directories {\n+\tmy ($gs) = @_;\n+\tmy $var = eval { command_oneline('config', '--get', '--bool',\n+\t\t\t\t\t \"svn-remote.$gs->{repo_id}.automkdirs\") };\n+\t# By default, create empty directories by consulting the unhandled log,\n+\t# but allow setting it to 'false' to skip it.\n+\treturn !($var && $var eq 'false');\n+}\n+\n sub cmd_rebase {\n \tcommand_noisy(qw/update-index --refresh/);\n \tmy ($url, $rev, $uuid, $gs) = working_head_info('HEAD');\n@@ -804,7 +813,9 @@ sub cmd_rebase {\n \t\t$_fetch_all ? $gs->fetch_all : $gs->fetch;\n \t}\n \tcommand_noisy(rebase_cmd(), $gs->refname);\n-\t$gs->mkemptydirs;\n+\tif (auto_create_empty_directories($gs)) {\n+\t\t$gs->mkemptydirs;\n+\t}\n }\n \n sub cmd_show_ignore {\n@@ -1242,7 +1253,9 @@ sub post_fetch_checkout {\n \tcommand_noisy(qw/read-tree -m -u -v HEAD HEAD/);\n \tprint STDERR \"Checked out HEAD:\\n  \",\n \t             $gs->full_url, \" r\", $gs->last_rev, \"\\n\";\n-\t$gs->mkemptydirs($gs->last_rev);\n+\tif (auto_create_empty_directories($gs)) {\n+\t\t$gs->mkemptydirs($gs->last_rev);\n+\t}\n }\n \n sub complete_svn_url {\ndiff --git a/t/t9146-git-svn-empty-dirs.sh b/t/t9146-git-svn-empty-dirs.sh\nindex 158c8e3..6d3130e 100755\n--- a/t/t9146-git-svn-empty-dirs.sh\n+++ b/t/t9146-git-svn-empty-dirs.sh\n@@ -28,6 +28,23 @@ test_expect_success 'empty directories exist' '\n \t)\n '\n \n+test_expect_success 'option automkdirs set to false' '\n+\t(\n+\t\tgit svn init \"$svnrepo\" cloned-no-mkdirs &&\n+\t\tcd cloned-no-mkdirs &&\n+\t\tgit config svn-remote.svn.automkdirs false &&\n+\t\tgit svn fetch &&\n+\t\tfor i in a b c d d/e d/e/f \"weird file name\"\n+\t\tdo\n+\t\t\tif test -d \"$i\"\n+\t\t\tthen\n+\t\t\t\techo >&2 \"$i exists\"\n+\t\t\t\texit 1\n+\t\t\tfi\n+\t\tdone\n+\t)\n+'\n+\n test_expect_success 'more emptiness' '\n \tsvn_cmd mkdir -m \"bang bang\"  \"$svnrepo\"/\"! !\"\n '\n-- \n1.7.4.2\n"}]}