{"thread":{"id":"27749","subject":"[PATCH 1/8] vcs-svn: move url parameter from _read to _init","startedAt":"2011-07-03T17:57:49Z","lastAt":"2011-07-06T11:40:07Z","messageCount":14,"participants":["Dmitry Ivankov","Sverre Rabbelier","Ramkumar Ramachandra"],"isPatch":true,"patchVersion":1,"patchTotal":8},"messages":[{"id":"170796","messageId":"1309715877-13814-1-git-send-email-divanorama@gmail.com","threadId":"27749","inReplyTo":null,"subject":"[PATCH 0/8] vcs-svn, svn-fe: add a couple of options","fromName":"Dmitry Ivankov","fromEmail":"divanorama@gmail.com","sentAt":"2011-07-03T17:57:49Z","receivedAt":"2011-07-03T17:57:49Z","isPatch":true,"sender":{"key":"divanorama@gmail.com","avatar":"https://avatars.githubusercontent.com/u/158999?v=4"},"body":"This patch set adds a few options for svn-fe that can be used in\na svn remote helper and useful in general for vcs-svn/. The last\nthree patches add the options, others prepare the ground for them.\n\nThe patch base is svn-fe branch at git://repo.or.cz/git/jrn.git\n\nDmitry Ivankov (8):\n vcs-svn: move url parameter from _read to _init\n svn-fe: add man target to Makefile\n svn-fe: add EXTLIBS needed for parse-options\n svn-fe: add usage and unpositional arguments versions\n test-svn-fe: use parse-options\n vcs-svn: allow to specify dump destination ref\n vcs-svn: convert REPORT_FILENO to an option\n vcs-svn: allow to disable 'progress' lines\n\n contrib/svn-fe/Makefile | 18 ++++----\n contrib/svn-fe/svn-fe.c | 42 +++++++++++++++++--\n contrib/svn-fe/svn-fe.txt | 29 +++++++++++--\n t/t9010-svn-fe.sh | 99 +++++++++++++++++++++++++++++++++++++--------\n test-svn-fe.c | 50 ++++++++++++++++-------\n vcs-svn/fast_export.c | 13 +++++-\n vcs-svn/fast_export.h | 2 +-\n vcs-svn/svndump.c | 11 ++---\n vcs-svn/svndump.h | 4 +-\n 9 files changed, 205 insertions(+), 63 deletions(-)\n\n-- \n1.7.3.4\n"},{"id":"170795","messageId":"1309715877-13814-2-git-send-email-divanorama@gmail.com","threadId":"27749","inReplyTo":"1309715877-13814-1-git-send-email-divanorama@gmail.com","subject":"[PATCH 1/8] vcs-svn: move url parameter from _read to _init","fromName":"Dmitry Ivankov","fromEmail":"divanorama@gmail.com","sentAt":"2011-07-03T17:57:50Z","receivedAt":"2011-07-03T17:57:50Z","isPatch":true,"sender":{"key":"divanorama@gmail.com","avatar":"https://avatars.githubusercontent.com/u/158999?v=4"},"body":"svndump_read takes a url parameter that is used in git-svn-id: lines\ngeneration. Internally it is stored in dump_ctx which is initialized\nin svndump_init with reset_dump_ctx and then is reinitialized again\nin svndump_read.\n\nMove url parameter to svndump_init so that reset_dump_ctx is done\nonce per dump and in the same place as other resets. More parameters\nwill arise and all will go to svndump_init to setup the module for\ndumping.\n\nSigned-off-by: Dmitry Ivankov \n---\n contrib/svn-fe/svn-fe.c | 4 ++--\n test-svn-fe.c | 4 ++--\n vcs-svn/svndump.c | 7 +++----\n vcs-svn/svndump.h | 4 ++--\n 4 files changed, 9 insertions(+), 10 deletions(-)\n\ndiff --git a/contrib/svn-fe/svn-fe.c b/contrib/svn-fe/svn-fe.c\nindex 35db24f..11d7128 100644\n--- a/contrib/svn-fe/svn-fe.c\n+++ b/contrib/svn-fe/svn-fe.c\n@@ -8,9 +8,9 @@\n \n int main(int argc, char **argv)\n {\n-\tif (svndump_init(NULL))\n+\tif (svndump_init(NULL, (argc > 1) ? argv[1] : NULL))\n \t\treturn 1;\n-\tsvndump_read((argc > 1) ? argv[1] : NULL);\n+\tsvndump_read();\n \tsvndump_deinit();\n \tsvndump_reset();\n \treturn 0;\ndiff --git a/test-svn-fe.c b/test-svn-fe.c\nindex 332a5f7..3ee5559 100644\n--- a/test-svn-fe.c\n+++ b/test-svn-fe.c\n@@ -40,9 +40,9 @@ static int apply_delta(int argc, char *argv[])\n int main(int argc, char *argv[])\n {\n \tif (argc == 2) {\n-\t\tif (svndump_init(argv[1]))\n+\t\tif (svndump_init(argv[1], NULL))\n \t\t\treturn 1;\n-\t\tsvndump_read(NULL);\n+\t\tsvndump_read();\n \t\tsvndump_deinit();\n \t\tsvndump_reset();\n \t\treturn 0;\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex b1f4161..a88d392 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -313,14 +313,13 @@ static void end_revision(void)\n \t\tfast_export_end_commit(rev_ctx.revision);\n }\n \n-void svndump_read(const char *url)\n+void svndump_read(void)\n {\n \tchar *val;\n \tchar *t;\n \tuint32_t active_ctx = DUMP_CTX;\n \tuint32_t len;\n \n-\treset_dump_ctx(url);\n \twhile ((t = buffer_read_line(&input))) {\n \t\tval = strchr(t, ':');\n \t\tif (!val)\n@@ -455,7 +454,7 @@ void svndump_read(const char *url)\n \t\tend_revision();\n }\n \n-int svndump_init(const char *filename)\n+int svndump_init(const char *filename, const char *url)\n {\n \tif (buffer_init(&input, filename))\n \t\treturn error(\"cannot open %s: %s\", filename, strerror(errno));\n@@ -466,7 +465,7 @@ int svndump_init(const char *filename)\n \tstrbuf_init(&rev_ctx.author, 4096);\n \tstrbuf_init(&node_ctx.src, 4096);\n \tstrbuf_init(&node_ctx.dst, 4096);\n-\treset_dump_ctx(NULL);\n+\treset_dump_ctx(url);\n \treset_rev_ctx(0);\n \treset_node_ctx(NULL);\n \treturn 0;\ndiff --git a/vcs-svn/svndump.h b/vcs-svn/svndump.h\nindex df9ceb0..1bcaab6 100644\n--- a/vcs-svn/svndump.h\n+++ b/vcs-svn/svndump.h\n@@ -1,8 +1,8 @@\n #ifndef SVNDUMP_H_\n #define SVNDUMP_H_\n \n-int svndump_init(const char *filename);\n-void svndump_read(const char *url);\n+int svndump_init(const char *filename, const char *url);\n+void svndump_read(void);\n void svndump_deinit(void);\n void svndump_reset(void);\n \n-- \n1.7.3.4\n"},{"id":"170797","messageId":"1309715877-13814-3-git-send-email-divanorama@gmail.com","threadId":"27749","inReplyTo":"1309715877-13814-1-git-send-email-divanorama@gmail.com","subject":"[PATCH 2/8] svn-fe: add man target to Makefile","fromName":"Dmitry Ivankov","fromEmail":"divanorama@gmail.com","sentAt":"2011-07-03T17:57:51Z","receivedAt":"2011-07-03T17:57:51Z","isPatch":true,"sender":{"key":"divanorama@gmail.com","avatar":"https://avatars.githubusercontent.com/u/158999?v=4"},"body":"There already is a svn-fe.1 target. But 'man' being a standard\ntarget is easier to discover or type. It can also be reused if\nmore manpages arise here.\n\nSigned-off-by: Dmitry Ivankov \n---\n contrib/svn-fe/Makefile | 4 +++-\n 1 files changed, 3 insertions(+), 1 deletions(-)\n\ndiff --git a/contrib/svn-fe/Makefile b/contrib/svn-fe/Makefile\nindex 360d8da..bc03a3e 100644\n--- a/contrib/svn-fe/Makefile\n+++ b/contrib/svn-fe/Makefile\n@@ -33,6 +33,8 @@ ifndef V\n endif\n endif\n \n+man: svn-fe.1\n+\n svn-fe$X: svn-fe.o $(VCSSVN_LIB) $(GIT_LIB)\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ svn-fe.o \\\n \t\t$(ALL_LDFLAGS) $(LIBS)\n@@ -60,4 +62,4 @@ svn-fe.1: svn-fe.txt\n clean:\n \t$(RM) svn-fe$X svn-fe.o svn-fe.html svn-fe.xml svn-fe.1\n \n-.PHONY: all clean FORCE\n+.PHONY: all man clean FORCE\n-- \n1.7.3.4\n"},{"id":"170798","messageId":"1309715877-13814-4-git-send-email-divanorama@gmail.com","threadId":"27749","inReplyTo":"1309715877-13814-1-git-send-email-divanorama@gmail.com","subject":"[PATCH 3/8] svn-fe: add EXTLIBS needed for parse-options","fromName":"Dmitry Ivankov","fromEmail":"divanorama@gmail.com","sentAt":"2011-07-03T17:57:52Z","receivedAt":"2011-07-03T17:57:52Z","isPatch":true,"sender":{"key":"divanorama@gmail.com","avatar":"https://avatars.githubusercontent.com/u/158999?v=4"},"body":"Currently parse-options.o pull quite a big bunch of dependencies\nthat are neither pulled in by svn-fe Makefile nor included in libgit.a.\n\nUse a temporary hack: put hardcoded EXTLIBS, this may not work in all\nsetups because /Makefile logic is not repeated.\n\nFor example, one may need -lcrypto instead of -lssl or no crypto library\nif BLK_SHA1 is set, also an additional -lz or -lpcre could be required.\n\nSigned-off-by: Dmitry Ivankov \n---\n contrib/svn-fe/Makefile | 12 +++++-------\n 1 files changed, 5 insertions(+), 7 deletions(-)\n\ndiff --git a/contrib/svn-fe/Makefile b/contrib/svn-fe/Makefile\nindex bc03a3e..bf1625c 100644\n--- a/contrib/svn-fe/Makefile\n+++ b/contrib/svn-fe/Makefile\n@@ -8,11 +8,12 @@ CFLAGS = -g -O2 -Wall\n LDFLAGS =\n ALL_CFLAGS = $(CFLAGS)\n ALL_LDFLAGS = $(LDFLAGS)\n-EXTLIBS =\n+EXTLIBS = -lssl -lpthread\n \n GIT_LIB = ../../libgit.a\n VCSSVN_LIB = ../../vcs-svn/lib.a\n-LIBS = $(VCSSVN_LIB) $(GIT_LIB) $(EXTLIBS)\n+XDIFF_LIB = ../../xdiff/lib.a\n+LIBS = $(VCSSVN_LIB) $(GIT_LIB) $(XDIFF_LIB) $(EXTLIBS)\n \n QUIET_SUBDIR0 = +$(MAKE) -C # space to separate -C and subdir\n QUIET_SUBDIR1 =\n@@ -53,11 +54,8 @@ svn-fe.1: svn-fe.txt\n \t\t../contrib/svn-fe/$@\n \t$(MV) ../../Documentation/svn-fe.1 .\n \n-../../vcs-svn/lib.a: FORCE\n-\t$(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) vcs-svn/lib.a\n-\n-../../libgit.a: FORCE\n-\t$(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) libgit.a\n+$(VCSSVN_LIB) $(GIT_LIB) $(XDIFF_LIB): ../../%.a: FORCE\n+\t$(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) $*.a\n \n clean:\n \t$(RM) svn-fe$X svn-fe.o svn-fe.html svn-fe.xml svn-fe.1\n-- \n1.7.3.4\n"},{"id":"170800","messageId":"1309715877-13814-5-git-send-email-divanorama@gmail.com","threadId":"27749","inReplyTo":"1309715877-13814-1-git-send-email-divanorama@gmail.com","subject":"[PATCH 4/8] svn-fe: add usage and unpositional arguments versions","fromName":"Dmitry Ivankov","fromEmail":"divanorama@gmail.com","sentAt":"2011-07-03T17:57:53Z","receivedAt":"2011-07-03T17:57:53Z","isPatch":true,"sender":{"key":"divanorama@gmail.com","avatar":"https://avatars.githubusercontent.com/u/158999?v=4"},"body":"There will be more command line options for svn-fe so support\nunpositional version for flexibility. Also clarify the meaning\nof url parameter.\n\n$ svn-fe --git-svn-id-url=url\ndoes the same thing as\n$ svn-fe url\ni.e., url is used to generate git-svn-id: lines, if url is set.\n\nSigned-off-by: Dmitry Ivankov \n---\n contrib/svn-fe/Makefile | 2 +-\n contrib/svn-fe/svn-fe.c | 30 +++++++++++++++++++++++++++---\n contrib/svn-fe/svn-fe.txt | 17 +++++++++++++----\n 3 files changed, 41 insertions(+), 8 deletions(-)\n\ndiff --git a/contrib/svn-fe/Makefile b/contrib/svn-fe/Makefile\nindex bf1625c..3e18395 100644\n--- a/contrib/svn-fe/Makefile\n+++ b/contrib/svn-fe/Makefile\n@@ -41,7 +41,7 @@ svn-fe$X: svn-fe.o $(VCSSVN_LIB) $(GIT_LIB)\n \t\t$(ALL_LDFLAGS) $(LIBS)\n \n svn-fe.o: svn-fe.c ../../vcs-svn/svndump.h\n-\t$(QUIET_CC)$(CC) -I../../vcs-svn -o $*.o -c $(ALL_CFLAGS) $<\n+\t$(QUIET_CC)$(CC) -I../../vcs-svn -I../.. -o $*.o -c $(ALL_CFLAGS) $<\n \n svn-fe.html: svn-fe.txt\n \t$(QUIET_SUBDIR0)../../Documentation $(QUIET_SUBDIR1) \\\ndiff --git a/contrib/svn-fe/svn-fe.c b/contrib/svn-fe/svn-fe.c\nindex 11d7128..033dd5b 100644\n--- a/contrib/svn-fe/svn-fe.c\n+++ b/contrib/svn-fe/svn-fe.c\n@@ -3,12 +3,36 @@\n * You may freely use, modify, distribute, and relicense it.\n */\n \n-#include \n+#include \"git-compat-util.h\"\n+#include \"parse-options.h\"\n #include \"svndump.h\"\n \n-int main(int argc, char **argv)\n+static const char * const svn_fe_usage[] = {\n+\t\"svn-fe [options] [git-svn-id-url] < dump | fast-import-backend\",\n+\tNULL\n+};\n+\n+static const char *url;\n+\n+static struct option svn_fe_options[] = {\n+\tOPT_STRING(0, \"git-svn-id-url\", &url, \"url\",\n+\t\t\"append git-svn metadata line to commit messages\"),\n+\tOPT_END()\n+};\n+\n+int main(int argc, const char **argv)\n {\n-\tif (svndump_init(NULL, (argc > 1) ? argv[1] : NULL))\n+\targc = parse_options(argc, argv, NULL, svn_fe_options,\n+\t\t\t\t\t\tsvn_fe_usage, 0);\n+\tif (argc == 1) {\n+\t\tif (url)\n+\t\t\tusage_msg_opt(\"git-svn-id-url is set twice: as a \"\n+\t\t\t\t\t\"--parameter and as a [parameter]\",\n+\t\t\t\t\tsvn_fe_usage, svn_fe_options);\n+\t\turl = argv[0];\n+\t} else if (argc)\n+\t\tusage_with_options(svn_fe_usage, svn_fe_options);\n+\tif (svndump_init(NULL, url))\n \t\treturn 1;\n \tsvndump_read();\n \tsvndump_deinit();\ndiff --git a/contrib/svn-fe/svn-fe.txt b/contrib/svn-fe/svn-fe.txt\nindex 2dd27ce..8c6d347 100644\n--- a/contrib/svn-fe/svn-fe.txt\n+++ b/contrib/svn-fe/svn-fe.txt\n@@ -10,7 +10,7 @@ SYNOPSIS\n [verse]\n mkfifo backchannel &&\n svnadmin dump --deltas REPO |\n-\tsvn-fe [url] 3backchannel\n \n DESCRIPTION\n@@ -25,6 +25,14 @@ command.\n Note: this tool is very young. The details of its commandline\n interface may change in backward incompatible ways.\n \n+OPTIONS\n+-------\n+\n+--git-svn-id-url=::\n+\tUrl to be used in git-svn-id: lines in git-svn\n+\tmetadata lines format. See NOTES for more detailed\n+\tdescription.\n+\n INPUT FORMAT\n ------------\n Subversion's repository dump format is documented in full in\n@@ -50,9 +58,10 @@ user \n as committer, where 'user' is the value of the `svn:author` property\n and 'UUID' the repository's identifier.\n \n-To support incremental imports, 'svn-fe' puts a `git-svn-id` line at\n-the end of each commit log message if passed an url on the command\n-line. This line has the form `git-svn-id: URL@REVNO UUID`.\n+'svn-fe' can be used in preparing a repository for 'git-svn' as follows.\n+If `git-svn-id-url` is specified, 'svn-fe' will put `git-svn-id` line at\n+the end of each commit log message.\n+This line has the form `git-svn-id: URL@REVNO UUID`.\n \n The resulting repository will generally require further processing\n to put each project in its own repository and to separate the history\n-- \n1.7.3.4\n"},{"id":"170803","messageId":"1309715877-13814-6-git-send-email-divanorama@gmail.com","threadId":"27749","inReplyTo":"1309715877-13814-1-git-send-email-divanorama@gmail.com","subject":"[PATCH 5/8] test-svn-fe: use parse-options","fromName":"Dmitry Ivankov","fromEmail":"divanorama@gmail.com","sentAt":"2011-07-03T17:57:54Z","receivedAt":"2011-07-03T17:57:54Z","isPatch":true,"sender":{"key":"divanorama@gmail.com","avatar":"https://avatars.githubusercontent.com/u/158999?v=4"},"body":"There was custom options parsing. As more options arise it will\nbe easier to add and document new options with parse-options api.\n\nSigned-off-by: Dmitry Ivankov \n---\n test-svn-fe.c | 42 ++++++++++++++++++++++++++++--------------\n 1 files changed, 28 insertions(+), 14 deletions(-)\n\ndiff --git a/test-svn-fe.c b/test-svn-fe.c\nindex 3ee5559..603d3fb 100644\n--- a/test-svn-fe.c\n+++ b/test-svn-fe.c\n@@ -3,28 +3,38 @@\n */\n \n #include \"git-compat-util.h\"\n+#include \"parse-options.h\"\n #include \"vcs-svn/svndump.h\"\n #include \"vcs-svn/svndiff.h\"\n #include \"vcs-svn/sliding_window.h\"\n #include \"vcs-svn/line_buffer.h\"\n \n-static const char test_svnfe_usage[] =\n-\t\"test-svn-fe ( | [-d] )\";\n+static const char * const test_svnfe_usage[] = {\n+\t\"test-svn-fe ( | -d )\",\n+\tNULL\n+};\n \n-static int apply_delta(int argc, char *argv[])\n+static int d;\n+\n+static struct option test_svnfe_options[] = {\n+\tOPT_SET_INT('d', NULL, &d, \"test apply_delta\", 1),\n+\tOPT_END()\n+};\n+\n+static int apply_delta(int argc, const char *argv[])\n {\n \tstruct line_buffer preimage = LINE_BUFFER_INIT;\n \tstruct line_buffer delta = LINE_BUFFER_INIT;\n \tstruct sliding_view preimage_view = SLIDING_VIEW_INIT(&preimage, -1);\n \n-\tif (argc != 5)\n-\t\tusage(test_svnfe_usage);\n+\tif (argc != 3)\n+\t\tusage_with_options(test_svnfe_usage, test_svnfe_options);\n \n-\tif (buffer_init(&preimage, argv[2]))\n+\tif (buffer_init(&preimage, argv[0]))\n \t\tdie_errno(\"cannot open preimage\");\n-\tif (buffer_init(&delta, argv[3]))\n+\tif (buffer_init(&delta, argv[1]))\n \t\tdie_errno(\"cannot open delta\");\n-\tif (svndiff0_apply(&delta, (off_t) strtoull(argv[4], NULL, 0),\n+\tif (svndiff0_apply(&delta, (off_t) strtoull(argv[2], NULL, 0),\n \t\t\t\t\t&preimage_view, stdout))\n \t\treturn 1;\n \tif (buffer_deinit(&preimage))\n@@ -37,10 +47,16 @@ static int apply_delta(int argc, char *argv[])\n \treturn 0;\n }\n \n-int main(int argc, char *argv[])\n+int main(int argc, const char *argv[])\n {\n-\tif (argc == 2) {\n-\t\tif (svndump_init(argv[1], NULL))\n+\targc = parse_options(argc, argv, NULL, test_svnfe_options,\n+\t\t\t\t\t\ttest_svnfe_usage, 0);\n+\n+\tif (d)\n+\t\treturn apply_delta(argc, argv);\n+\n+\tif (argc == 1) {\n+\t\tif (svndump_init(argv[0], NULL))\n \t\t\treturn 1;\n \t\tsvndump_read();\n \t\tsvndump_deinit();\n@@ -48,7 +64,5 @@ int main(int argc, char *argv[])\n \t\treturn 0;\n \t}\n \n-\tif (argc >= 2 && !strcmp(argv[1], \"-d\"))\n-\t\treturn apply_delta(argc, argv);\n-\tusage(test_svnfe_usage);\n+\tusage_with_options(test_svnfe_usage, test_svnfe_options);\n }\n-- \n1.7.3.4\n"},{"id":"170801","messageId":"1309715877-13814-7-git-send-email-divanorama@gmail.com","threadId":"27749","inReplyTo":"1309715877-13814-1-git-send-email-divanorama@gmail.com","subject":"[PATCH 6/8] vcs-svn: allow to specify dump destination ref","fromName":"Dmitry Ivankov","fromEmail":"divanorama@gmail.com","sentAt":"2011-07-03T17:57:55Z","receivedAt":"2011-07-03T17:57:55Z","isPatch":true,"sender":{"key":"divanorama@gmail.com","avatar":"https://avatars.githubusercontent.com/u/158999?v=4"},"body":"svn-fe produces fast-import stream for a fixed refs/heads/master ref.\nIt is usually desired to write to a different ref. In a remote helper\nit would be a ref in private namespace. If svn-fe is used by someone\ndirectly it'll be more safe to remind where the commits can go. And\nin both cases it may be needed to import from two repos and hence to\ndifferent refs.\n\nAdd a destination ref parameter to vcs-svn/, a corresponding parameter\nto svn-fe and a simple test for it.\n\n$ svn-fe --ref=refs/heads/master ...\nis an explicit way to stay with the default destination.\n\nSigned-off-by: Dmitry Ivankov \n---\n contrib/svn-fe/svn-fe.c | 5 +++-\n contrib/svn-fe/svn-fe.txt | 3 ++\n t/t9010-svn-fe.sh | 49 +++++++++++++++++++++++++++++---------------\n test-svn-fe.c | 7 ++++-\n vcs-svn/fast_export.c | 8 +++++-\n vcs-svn/fast_export.h | 2 +-\n vcs-svn/svndump.c | 4 +-\n vcs-svn/svndump.h | 2 +-\n 8 files changed, 54 insertions(+), 26 deletions(-)\n\ndiff --git a/contrib/svn-fe/svn-fe.c b/contrib/svn-fe/svn-fe.c\nindex 033dd5b..78c7cd0 100644\n--- a/contrib/svn-fe/svn-fe.c\n+++ b/contrib/svn-fe/svn-fe.c\n@@ -13,10 +13,13 @@ static const char * const svn_fe_usage[] = {\n };\n \n static const char *url;\n+static const char *ref = \"refs/heads/master\";\n \n static struct option svn_fe_options[] = {\n \tOPT_STRING(0, \"git-svn-id-url\", &url, \"url\",\n \t\t\"append git-svn metadata line to commit messages\"),\n+\tOPT_STRING(0, \"ref\", &ref, \"dst_ref\",\n+\t\t\"write to dst_ref instead of refs/heads/master\"),\n \tOPT_END()\n };\n \n@@ -32,7 +35,7 @@ int main(int argc, const char **argv)\n \t\turl = argv[0];\n \t} else if (argc)\n \t\tusage_with_options(svn_fe_usage, svn_fe_options);\n-\tif (svndump_init(NULL, url))\n+\tif (svndump_init(NULL, url, ref))\n \t\treturn 1;\n \tsvndump_read();\n \tsvndump_deinit();\ndiff --git a/contrib/svn-fe/svn-fe.txt b/contrib/svn-fe/svn-fe.txt\nindex 8c6d347..20c3315 100644\n--- a/contrib/svn-fe/svn-fe.txt\n+++ b/contrib/svn-fe/svn-fe.txt\n@@ -32,6 +32,9 @@ OPTIONS\n \tUrl to be used in git-svn-id: lines in git-svn\n \tmetadata lines format. See NOTES for more detailed\n \tdescription.\n+--ref=::\n+\tRef to be written by the generated stream.\n+\tDefault is refs/heads/master.\n \n INPUT FORMAT\n ------------\ndiff --git a/t/t9010-svn-fe.sh b/t/t9010-svn-fe.sh\nindex b7eed24..52efabe 100755\n--- a/t/t9010-svn-fe.sh\n+++ b/t/t9010-svn-fe.sh\n@@ -16,18 +16,24 @@ reinit_git () {\n \tmkfifo stream backflow\n }\n \n-try_dump () {\n+try_dump_ext () {\n+\targs=$1 &&\n+\tshift &&\n \tinput=$1 &&\n \tmaybe_fail_svnfe=${2:+test_$2} &&\n \tmaybe_fail_fi=${3:+test_$3} &&\n \n \t{\n-\t\t$maybe_fail_svnfe test-svn-fe \"$input\" >stream 3stream 3backflow &&\n \twait $!\n }\n \n+try_dump () {\n+\ttry_dump_ext \"\" $@\n+}\n+\n properties () {\n \twhile test \"$#\" -ne 0\n \tdo\n@@ -54,6 +60,22 @@ text_no_props () {\n \n >empty\n \n+cat >emptyprop.dump <<-EOF &&\n+SVN-fs-dump-format-version: 3\n+\n+Revision-number: 1\n+Prop-content-length: 10\n+Content-length: 10\n+\n+PROPS-END\n+\n+Revision-number: 2\n+Prop-content-length: 10\n+Content-length: 10\n+\n+PROPS-END\n+EOF\n+\n test_expect_success 'setup: have pipes?' '\n \trm -f frob &&\n \tif mkfifo frob\n@@ -97,21 +119,6 @@ test_expect_failure PIPE 'empty revision' '\n test_expect_success PIPE 'empty properties' '\n \treinit_git &&\n \tprintf \"rev : %s\\n\" \"\" \"\" >expect &&\n-\tcat >emptyprop.dump <<-\\EOF &&\n-\tSVN-fs-dump-format-version: 3\n-\n-\tRevision-number: 1\n-\tProp-content-length: 10\n-\tContent-length: 10\n-\n-\tPROPS-END\n-\n-\tRevision-number: 2\n-\tProp-content-length: 10\n-\tContent-length: 10\n-\n-\tPROPS-END\n-\tEOF\n \ttry_dump emptyprop.dump &&\n \tgit log -p --format=\"rev <%an, %ae>: %s\" HEAD >actual &&\n \ttest_cmp expect actual\n@@ -1111,4 +1118,12 @@ test_expect_success SVNREPO,PIPE 't9135/svn.dump' '\n \t)\n '\n \n+test_expect_success PIPE 'import to notmaster ref' '\n+\treinit_git &&\n+\ttry_dump_ext \"--ref=refs/heads/notmaster\" emptyprop.dump &&\n+\n+\tgit rev-parse --verify notmaster &&\n+\ttest_must_fail git rev-parse --verify master\n+'\n+\n test_done\ndiff --git a/test-svn-fe.c b/test-svn-fe.c\nindex 603d3fb..afa551c 100644\n--- a/test-svn-fe.c\n+++ b/test-svn-fe.c\n@@ -10,14 +10,17 @@\n #include \"vcs-svn/line_buffer.h\"\n \n static const char * const test_svnfe_usage[] = {\n-\t\"test-svn-fe ( | -d )\",\n+\t\"test-svn-fe ([options] | -d )\",\n \tNULL\n };\n \n+static const char *ref = \"refs/heads/master\";\n static int d;\n \n static struct option test_svnfe_options[] = {\n \tOPT_SET_INT('d', NULL, &d, \"test apply_delta\", 1),\n+\tOPT_STRING(0, \"ref\", &ref, \"dst_ref\",\n+\t\t\"write to dst_ref instead of refs/heads/master\"),\n \tOPT_END()\n };\n \n@@ -56,7 +59,7 @@ int main(int argc, const char *argv[])\n \t\treturn apply_delta(argc, argv);\n \n \tif (argc == 1) {\n-\t\tif (svndump_init(argv[0], NULL))\n+\t\tif (svndump_init(argv[0], NULL, ref))\n \t\t\treturn 1;\n \t\tsvndump_read();\n \t\tsvndump_deinit();\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex 19d7c34..cfb0f2b 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -18,6 +18,7 @@\n static uint32_t first_commit_done;\n static struct line_buffer postimage = LINE_BUFFER_INIT;\n static struct line_buffer report_buffer = LINE_BUFFER_INIT;\n+static struct strbuf ref_name = STRBUF_INIT;\n \n /* NEEDSWORK: move to fast_export_init() */\n static int init_postimage(void)\n@@ -29,15 +30,18 @@ static int init_postimage(void)\n \treturn buffer_tmpfile_init(&postimage);\n }\n \n-void fast_export_init(int fd)\n+void fast_export_init(int fd, const char *dst_ref)\n {\n \tfirst_commit_done = 0;\n+\tstrbuf_reset(&ref_name);\n+\tstrbuf_addstr(&ref_name, dst_ref);\n \tif (buffer_fdinit(&report_buffer, fd))\n \t\tdie_errno(\"cannot read from file descriptor %d\", fd);\n }\n \n void fast_export_deinit(void)\n {\n+\tstrbuf_release(&ref_name);\n \tif (buffer_deinit(&report_buffer))\n \t\tdie_errno(\"error closing fast-import feedback stream\");\n }\n@@ -89,7 +93,7 @@ void fast_export_begin_commit(uint32_t revision, const char *author,\n \t} else {\n \t\t*gitsvnline = '\\0';\n \t}\n-\tprintf(\"commit refs/heads/master\\n\");\n+\tprintf(\"commit %s\\n\", ref_name.buf);\n \tprintf(\"mark :%\"PRIu32\"\\n\", revision);\n \tprintf(\"committer %s <%s@%s> %ld +0000\\n\",\n \t\t *author ? author : \"nobody\",\ndiff --git a/vcs-svn/fast_export.h b/vcs-svn/fast_export.h\nindex 43d05b6..d7eb7cc 100644\n--- a/vcs-svn/fast_export.h\n+++ b/vcs-svn/fast_export.h\n@@ -4,7 +4,7 @@\n struct strbuf;\n struct line_buffer;\n \n-void fast_export_init(int fd);\n+void fast_export_init(int fd, const char *dst_ref);\n void fast_export_deinit(void);\n void fast_export_reset(void);\n \ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex a88d392..f0df381 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -454,11 +454,11 @@ void svndump_read(void)\n \t\tend_revision();\n }\n \n-int svndump_init(const char *filename, const char *url)\n+int svndump_init(const char *filename, const char *url, const char *dst_ref)\n {\n \tif (buffer_init(&input, filename))\n \t\treturn error(\"cannot open %s: %s\", filename, strerror(errno));\n-\tfast_export_init(REPORT_FILENO);\n+\tfast_export_init(REPORT_FILENO, dst_ref);\n \tstrbuf_init(&dump_ctx.uuid, 4096);\n \tstrbuf_init(&dump_ctx.url, 4096);\n \tstrbuf_init(&rev_ctx.log, 4096);\ndiff --git a/vcs-svn/svndump.h b/vcs-svn/svndump.h\nindex 1bcaab6..08922fb 100644\n--- a/vcs-svn/svndump.h\n+++ b/vcs-svn/svndump.h\n@@ -1,7 +1,7 @@\n #ifndef SVNDUMP_H_\n #define SVNDUMP_H_\n \n-int svndump_init(const char *filename, const char *url);\n+int svndump_init(const char *filename, const char *url, const char *dst_ref);\n void svndump_read(void);\n void svndump_deinit(void);\n void svndump_reset(void);\n-- \n1.7.3.4\n"},{"id":"170799","messageId":"1309715877-13814-8-git-send-email-divanorama@gmail.com","threadId":"27749","inReplyTo":"1309715877-13814-1-git-send-email-divanorama@gmail.com","subject":"[PATCH 7/8] vcs-svn: convert REPORT_FILENO to an option","fromName":"Dmitry Ivankov","fromEmail":"divanorama@gmail.com","sentAt":"2011-07-03T17:57:56Z","receivedAt":"2011-07-03T17:57:56Z","isPatch":true,"sender":{"key":"divanorama@gmail.com","avatar":"https://avatars.githubusercontent.com/u/158999?v=4"},"body":"svn-fe needs to read fast-import's responses to \"ls\" and \"cat-blob\".\nThese come through a file descriptor number 3.\n\nSometimes it is easier to setup variable fd than a fixed one. It is\nthe case with pipe() call and even more fd=3 can be already taken.\nOn Windows file descriptors are not by default inherited by a child\nprocess, nor there is an option to setup descriptors other than\nstandard stdin, stdout, stderr at a process creation time.\n\nAdd an option for this file descriptor number in vcs-svn/ and svn-fe,\nadd a simple test for it.\n\nTo be used like following:\n$ svn-fe --read-blob-fd=7 ... 7\n---\n contrib/svn-fe/svn-fe.c | 5 +++-\n contrib/svn-fe/svn-fe.txt | 8 ++++-\n t/t9010-svn-fe.sh | 58 +++++++++++++++++++++++++++++++++++++++++---\n test-svn-fe.c | 5 +++-\n vcs-svn/svndump.c | 6 +---\n vcs-svn/svndump.h | 2 +-\n 6 files changed, 71 insertions(+), 13 deletions(-)\n\ndiff --git a/contrib/svn-fe/svn-fe.c b/contrib/svn-fe/svn-fe.c\nindex 78c7cd0..32755b1 100644\n--- a/contrib/svn-fe/svn-fe.c\n+++ b/contrib/svn-fe/svn-fe.c\n@@ -14,12 +14,15 @@ static const char * const svn_fe_usage[] = {\n \n static const char *url;\n static const char *ref = \"refs/heads/master\";\n+static int backflow_fd = 3;\n \n static struct option svn_fe_options[] = {\n \tOPT_STRING(0, \"git-svn-id-url\", &url, \"url\",\n \t\t\"append git-svn metadata line to commit messages\"),\n \tOPT_STRING(0, \"ref\", &ref, \"dst_ref\",\n \t\t\"write to dst_ref instead of refs/heads/master\"),\n+\tOPT_INTEGER(0, \"read-blob-fd\", &backflow_fd,\n+\t\t\"read blobs and trees from this fd instead of 3\"),\n \tOPT_END()\n };\n \n@@ -35,7 +38,7 @@ int main(int argc, const char **argv)\n \t\turl = argv[0];\n \t} else if (argc)\n \t\tusage_with_options(svn_fe_usage, svn_fe_options);\n-\tif (svndump_init(NULL, url, ref))\n+\tif (svndump_init(NULL, url, ref, backflow_fd))\n \t\treturn 1;\n \tsvndump_read();\n \tsvndump_deinit();\ndiff --git a/contrib/svn-fe/svn-fe.txt b/contrib/svn-fe/svn-fe.txt\nindex 20c3315..a7ad368 100644\n--- a/contrib/svn-fe/svn-fe.txt\n+++ b/contrib/svn-fe/svn-fe.txt\n@@ -8,9 +8,9 @@ svn-fe - convert an SVN \"dumpfile\" to a fast-import stream\n SYNOPSIS\n --------\n [verse]\n-mkfifo backchannel &&\n+mkfifo backchannel && fd=3 &&\n svnadmin dump --deltas REPO |\n-\tsvn-fe [options] [git-svn-id-url] 3backchannel\n \n DESCRIPTION\n@@ -35,6 +35,10 @@ OPTIONS\n --ref=::\n \tRef to be written by the generated stream.\n \tDefault is refs/heads/master.\n+--read-blob-fd=::\n+\tInteger number of file descriptor from which\n+\tresponses to 'ls' and 'cat-blob' requests will come.\n+\tDefault is fd=3.\n \n INPUT FORMAT\n ------------\ndiff --git a/t/t9010-svn-fe.sh b/t/t9010-svn-fe.sh\nindex 52efabe..6dcad94 100755\n--- a/t/t9010-svn-fe.sh\n+++ b/t/t9010-svn-fe.sh\n@@ -18,20 +18,21 @@ reinit_git () {\n \n try_dump_ext () {\n \targs=$1 &&\n-\tshift &&\n+\tfd=${2:-3} &&\n+\tshift 2 &&\n \tinput=$1 &&\n \tmaybe_fail_svnfe=${2:+test_$2} &&\n \tmaybe_fail_fi=${3:+test_$3} &&\n \n \t{\n-\t\t$maybe_fail_svnfe test-svn-fe $args \"$input\" >stream 3stream $fdbackflow &&\n \twait $!\n }\n \n try_dump () {\n-\ttry_dump_ext \"\" $@\n+\ttry_dump_ext \"\" \"\" $@\n }\n \n properties () {\n@@ -1120,10 +1121,59 @@ test_expect_success SVNREPO,PIPE 't9135/svn.dump' '\n \n test_expect_success PIPE 'import to notmaster ref' '\n \treinit_git &&\n-\ttry_dump_ext \"--ref=refs/heads/notmaster\" emptyprop.dump &&\n+\ttry_dump_ext \"--ref=refs/heads/notmaster\" 3 emptyprop.dump &&\n \n \tgit rev-parse --verify notmaster &&\n \ttest_must_fail git rev-parse --verify master\n '\n \n+test_expect_success PIPE 'use different backflow fd' '\n+\treinit_git &&\n+\techo hi >hi &&\n+\t{\n+\t\tproperties \\\n+\t\t\tsvn:author author@example.com \\\n+\t\t\tsvn:date \"1999-02-01T00:01:002.000000Z\" \\\n+\t\t\tsvn:log \"add directory with some files in it\" &&\n+\t\techo PROPS-END\n+\t} >props &&\n+\t{\n+\t\techo Prop-content-length: $(wc -c revprops &&\n+\t{\n+\t\tcat <<-EOF &&\n+\t\tSVN-fs-dump-format-version: 3\n+\n+\t\tRevision-number: 1\n+\t\tEOF\n+\t\tcat revprops &&\n+\t\tcat <<-EOF &&\n+\t\tNode-path: directory\n+\t\tNode-kind: dir\n+\t\tNode-action: add\n+\t\tNode-path: directory/somefile\n+\t\tNode-kind: file\n+\t\tNode-action: add\n+\t\tEOF\n+\t\ttext_no_props hi &&\n+\n+\t\techo \"Revision-number: 2\" &&\n+\t\tcat revprops &&\n+\t\tcat <<-\\EOF\n+\t\tNode-path: otherfile\n+\t\tNode-kind: file\n+\t\tNode-action: add\n+\t\tNode-copyfrom-rev: 1\n+\t\tNode-copyfrom-path: directory/somefile\n+\t\tEOF\n+\t} >directory.dump &&\n+\ttry_dump_ext \"--read-blob-fd=7\" 7 directory.dump &&\n+\n+\tgit checkout HEAD otherfile &&\n+\ttest_cmp hi otherfile\n+'\n+\n test_done\ndiff --git a/test-svn-fe.c b/test-svn-fe.c\nindex afa551c..7885eb1 100644\n--- a/test-svn-fe.c\n+++ b/test-svn-fe.c\n@@ -16,11 +16,14 @@ static const char * const test_svnfe_usage[] = {\n \n static const char *ref = \"refs/heads/master\";\n static int d;\n+static int backflow_fd = 3;\n \n static struct option test_svnfe_options[] = {\n \tOPT_SET_INT('d', NULL, &d, \"test apply_delta\", 1),\n \tOPT_STRING(0, \"ref\", &ref, \"dst_ref\",\n \t\t\"write to dst_ref instead of refs/heads/master\"),\n+\tOPT_INTEGER(0, \"read-blob-fd\", &backflow_fd,\n+\t\t\"read blobs and trees from this fd instead of 3\"),\n \tOPT_END()\n };\n \n@@ -59,7 +62,7 @@ int main(int argc, const char *argv[])\n \t\treturn apply_delta(argc, argv);\n \n \tif (argc == 1) {\n-\t\tif (svndump_init(argv[0], NULL, ref))\n+\t\tif (svndump_init(argv[0], NULL, ref, backflow_fd))\n \t\t\treturn 1;\n \t\tsvndump_read();\n \t\tsvndump_deinit();\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex f0df381..c52faf1 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -19,8 +19,6 @@\n */\n #define constcmp(s, ref) memcmp(s, ref, sizeof(ref) - 1)\n \n-#define REPORT_FILENO 3\n-\n #define NODEACT_REPLACE 4\n #define NODEACT_DELETE 3\n #define NODEACT_ADD 2\n@@ -454,11 +452,11 @@ void svndump_read(void)\n \t\tend_revision();\n }\n \n-int svndump_init(const char *filename, const char *url, const char *dst_ref)\n+int svndump_init(const char *filename, const char *url, const char *dst_ref, int report_fileno)\n {\n \tif (buffer_init(&input, filename))\n \t\treturn error(\"cannot open %s: %s\", filename, strerror(errno));\n-\tfast_export_init(REPORT_FILENO, dst_ref);\n+\tfast_export_init(report_fileno, dst_ref);\n \tstrbuf_init(&dump_ctx.uuid, 4096);\n \tstrbuf_init(&dump_ctx.url, 4096);\n \tstrbuf_init(&rev_ctx.log, 4096);\ndiff --git a/vcs-svn/svndump.h b/vcs-svn/svndump.h\nindex 08922fb..85c82c5 100644\n--- a/vcs-svn/svndump.h\n+++ b/vcs-svn/svndump.h\n@@ -1,7 +1,7 @@\n #ifndef SVNDUMP_H_\n #define SVNDUMP_H_\n \n-int svndump_init(const char *filename, const char *url, const char *dst_ref);\n+int svndump_init(const char *filename, const char *url, const char *dst_ref, int report_fileno);\n void svndump_read(void);\n void svndump_deinit(void);\n void svndump_reset(void);\n-- \n1.7.3.4\n"},{"id":"170802","messageId":"1309715877-13814-9-git-send-email-divanorama@gmail.com","threadId":"27749","inReplyTo":"1309715877-13814-1-git-send-email-divanorama@gmail.com","subject":"[PATCH 8/8] vcs-svn: allow to disable 'progress' lines","fromName":"Dmitry Ivankov","fromEmail":"divanorama@gmail.com","sentAt":"2011-07-03T17:57:57Z","receivedAt":"2011-07-03T17:57:57Z","isPatch":true,"sender":{"key":"divanorama@gmail.com","avatar":"https://avatars.githubusercontent.com/u/158999?v=4"},"body":"vcs-svn/ writes a progress line after each processed revision. It\nis too noisy for big imports. That's a stress for a terminal and\nany other output can be lost or scrolled away among these lines.\n\nFor now just add a switch to turn progress lines off:\n$ svn-fe --no-progress ...\n\nSigned-off-by: Dmitry Ivankov \n---\n contrib/svn-fe/svn-fe.c | 6 +++++-\n contrib/svn-fe/svn-fe.txt | 3 +++\n test-svn-fe.c | 2 +-\n vcs-svn/fast_export.c | 7 +++++--\n vcs-svn/fast_export.h | 2 +-\n vcs-svn/svndump.c | 4 ++--\n vcs-svn/svndump.h | 2 +-\n 7 files changed, 18 insertions(+), 8 deletions(-)\n\ndiff --git a/contrib/svn-fe/svn-fe.c b/contrib/svn-fe/svn-fe.c\nindex 32755b1..792ffad 100644\n--- a/contrib/svn-fe/svn-fe.c\n+++ b/contrib/svn-fe/svn-fe.c\n@@ -15,8 +15,12 @@ static const char * const svn_fe_usage[] = {\n static const char *url;\n static const char *ref = \"refs/heads/master\";\n static int backflow_fd = 3;\n+static int progress = 1;\n \n static struct option svn_fe_options[] = {\n+\tOPT_BIT(0, \"progress\", &progress,\n+\t\t\"write a progress line after each commit\",\n+\t\t1),\n \tOPT_STRING(0, \"git-svn-id-url\", &url, \"url\",\n \t\t\"append git-svn metadata line to commit messages\"),\n \tOPT_STRING(0, \"ref\", &ref, \"dst_ref\",\n@@ -38,7 +42,7 @@ int main(int argc, const char **argv)\n \t\turl = argv[0];\n \t} else if (argc)\n \t\tusage_with_options(svn_fe_usage, svn_fe_options);\n-\tif (svndump_init(NULL, url, ref, backflow_fd))\n+\tif (svndump_init(NULL, url, ref, backflow_fd, progress))\n \t\treturn 1;\n \tsvndump_read();\n \tsvndump_deinit();\ndiff --git a/contrib/svn-fe/svn-fe.txt b/contrib/svn-fe/svn-fe.txt\nindex a7ad368..f1a459e 100644\n--- a/contrib/svn-fe/svn-fe.txt\n+++ b/contrib/svn-fe/svn-fe.txt\n@@ -39,6 +39,9 @@ OPTIONS\n \tInteger number of file descriptor from which\n \tresponses to 'ls' and 'cat-blob' requests will come.\n \tDefault is fd=3.\n+--[no-]progress::\n+\tWrite 'progress' lines to fast-import stream. These\n+\tcan be displayed by fast-import.\n \n INPUT FORMAT\n ------------\ndiff --git a/test-svn-fe.c b/test-svn-fe.c\nindex 7885eb1..e4bfda0 100644\n--- a/test-svn-fe.c\n+++ b/test-svn-fe.c\n@@ -62,7 +62,7 @@ int main(int argc, const char *argv[])\n \t\treturn apply_delta(argc, argv);\n \n \tif (argc == 1) {\n-\t\tif (svndump_init(argv[0], NULL, ref, backflow_fd))\n+\t\tif (svndump_init(argv[0], NULL, ref, backflow_fd, 1))\n \t\t\treturn 1;\n \t\tsvndump_read();\n \t\tsvndump_deinit();\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex cfb0f2b..a8b8603 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -19,6 +19,7 @@ static uint32_t first_commit_done;\n static struct line_buffer postimage = LINE_BUFFER_INIT;\n static struct line_buffer report_buffer = LINE_BUFFER_INIT;\n static struct strbuf ref_name = STRBUF_INIT;\n+static int print_progress;\n \n /* NEEDSWORK: move to fast_export_init() */\n static int init_postimage(void)\n@@ -30,9 +31,10 @@ static int init_postimage(void)\n \treturn buffer_tmpfile_init(&postimage);\n }\n \n-void fast_export_init(int fd, const char *dst_ref)\n+void fast_export_init(int fd, const char *dst_ref, int progress)\n {\n \tfirst_commit_done = 0;\n+\tprint_progress = progress;\n \tstrbuf_reset(&ref_name);\n \tstrbuf_addstr(&ref_name, dst_ref);\n \tif (buffer_fdinit(&report_buffer, fd))\n@@ -112,7 +114,8 @@ void fast_export_begin_commit(uint32_t revision, const char *author,\n \n void fast_export_end_commit(uint32_t revision)\n {\n-\tprintf(\"progress Imported commit %\"PRIu32\".\\n\\n\", revision);\n+\tif (print_progress)\n+\t\tprintf(\"progress Imported commit %\"PRIu32\".\\n\\n\", revision);\n }\n \n static void ls_from_rev(uint32_t rev, const char *path)\ndiff --git a/vcs-svn/fast_export.h b/vcs-svn/fast_export.h\nindex d7eb7cc..7cab7b3 100644\n--- a/vcs-svn/fast_export.h\n+++ b/vcs-svn/fast_export.h\n@@ -4,7 +4,7 @@\n struct strbuf;\n struct line_buffer;\n \n-void fast_export_init(int fd, const char *dst_ref);\n+void fast_export_init(int fd, const char *dst_ref, int progress);\n void fast_export_deinit(void);\n void fast_export_reset(void);\n \ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex c52faf1..5e3a44d 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -452,11 +452,11 @@ void svndump_read(void)\n \t\tend_revision();\n }\n \n-int svndump_init(const char *filename, const char *url, const char *dst_ref, int report_fileno)\n+int svndump_init(const char *filename, const char *url, const char *dst_ref, int report_fileno, int progress)\n {\n \tif (buffer_init(&input, filename))\n \t\treturn error(\"cannot open %s: %s\", filename, strerror(errno));\n-\tfast_export_init(report_fileno, dst_ref);\n+\tfast_export_init(report_fileno, dst_ref, progress);\n \tstrbuf_init(&dump_ctx.uuid, 4096);\n \tstrbuf_init(&dump_ctx.url, 4096);\n \tstrbuf_init(&rev_ctx.log, 4096);\ndiff --git a/vcs-svn/svndump.h b/vcs-svn/svndump.h\nindex 85c82c5..becea11 100644\n--- a/vcs-svn/svndump.h\n+++ b/vcs-svn/svndump.h\n@@ -1,7 +1,7 @@\n #ifndef SVNDUMP_H_\n #define SVNDUMP_H_\n \n-int svndump_init(const char *filename, const char *url, const char *dst_ref, int report_fileno);\n+int svndump_init(const char *filename, const char *url, const char *dst_ref, int report_fileno, int progress);\n void svndump_read(void);\n void svndump_deinit(void);\n void svndump_reset(void);\n-- \n1.7.3.4\n"},{"id":"170812","messageId":"CAGdFq_gQFGy6xWYv_9sN=Txk0hmzT7h=LUfLP-Q4=EFEvS=KLA@mail.gmail.com","threadId":"27749","inReplyTo":"1309715877-13814-1-git-send-email-divanorama@gmail.com","subject":"Re: [PATCH 0/8] vcs-svn, svn-fe: add a couple of options","fromName":"Sverre Rabbelier","fromEmail":"srabbelier@gmail.com","sentAt":"2011-07-04T11:06:53Z","receivedAt":"2011-07-04T11:06:53Z","isPatch":true,"sender":{"key":"srabbelier@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3098?v=4"},"body":"Heya,\n\n[+Ram]\n\nOn Sun, Jul 3, 2011 at 19:57, Dmitry Ivankov wrote:\n> This patch set adds a few options for svn-fe that can be used in\n> a svn remote helper and useful in general for vcs-svn/. The last\n> three patches add the options, others prepare the ground for them.\n>\n> The patch base is svn-fe branch at git://repo.or.cz/git/jrn.git\n>\n> Dmitry Ivankov (8):\n>  vcs-svn: move url parameter from _read to _init\n>  svn-fe: add man target to Makefile\n>  svn-fe: add EXTLIBS needed for parse-options\n>  svn-fe: add usage and unpositional arguments versions\n>  test-svn-fe: use parse-options\n>  vcs-svn: allow to specify dump destination ref\n>  vcs-svn: convert REPORT_FILENO to an option\n>  vcs-svn: allow to disable 'progress' lines\n>\n>  contrib/svn-fe/Makefile   |   18 ++++----\n>  contrib/svn-fe/svn-fe.c   |   42 +++++++++++++++++--\n>  contrib/svn-fe/svn-fe.txt |   29 +++++++++++--\n>  t/t9010-svn-fe.sh         |   99 +++++++++++++++++++++++++++++++++++++--------\n>  test-svn-fe.c             |   50 ++++++++++++++++-------\n>  vcs-svn/fast_export.c     |   13 +++++-\n>  vcs-svn/fast_export.h     |    2 +-\n>  vcs-svn/svndump.c         |   11 ++---\n>  vcs-svn/svndump.h         |    4 +-\n>  9 files changed, 205 insertions(+), 63 deletions(-)\n>\n> --\n> 1.7.3.4\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"170878","messageId":"CALkWK0nWSVrrtzwE2ePucK=XTnSHWuojxhoayRSUbZmk1We9Fg@mail.gmail.com","threadId":"27749","inReplyTo":"1309715877-13814-9-git-send-email-divanorama@gmail.com","subject":"Re: [PATCH 8/8] vcs-svn: allow to disable 'progress' lines","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2011-07-06T08:25:17Z","receivedAt":"2011-07-06T08:25:17Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi Dmitry,\n\nDmitry Ivankov writes:\n> vcs-svn/ writes a progress line after each processed revision. It\n> is too noisy for big imports. That's a stress for a terminal and\n> any other output can be lost or scrolled away among these lines.\n>\n> For now just add a switch to turn progress lines off:\n> $ svn-fe --no-progress ...\n\nAgreed. Sounds like a good change.\n\n> diff --git a/contrib/svn-fe/svn-fe.c b/contrib/svn-fe/svn-fe.c\n> index 32755b1..792ffad 100644\n> --- a/contrib/svn-fe/svn-fe.c\n> +++ b/contrib/svn-fe/svn-fe.c\n> @@ -38,7 +42,7 @@ int main(int argc, const char **argv)\n>                url = argv[0];\n>        } else if (argc)\n>                usage_with_options(svn_fe_usage, svn_fe_options);\n> -       if (svndump_init(NULL, url, ref, backflow_fd))\n> +       if (svndump_init(NULL, url, ref, backflow_fd, progress))\n\nYou're modifying the svndump_init API for every new option that's\nadded. This'll clearly break down when you have many options -- how\nabout wrapping it up in an options structure and then passing that?\n\n> diff --git a/contrib/svn-fe/svn-fe.txt b/contrib/svn-fe/svn-fe.txt\n> index a7ad368..f1a459e 100644\n> --- a/contrib/svn-fe/svn-fe.txt\n> +++ b/contrib/svn-fe/svn-fe.txt\n> @@ -39,6 +39,9 @@ OPTIONS\n>        Integer number of file descriptor from which\n>        responses to 'ls' and 'cat-blob' requests will come.\n>        Default is fd=3.\n> +--[no-]progress::\n> +       Write 'progress' lines to fast-import stream. These\n> +       can be displayed by fast-import.\n\nHm, this will make it a little too silent for the end-user. What do\nyou feel about printing something minimalistic like a '.' for each\nimported revision? Atleast it won't look like it's hung. Also, how\ndoes this interact with the 'progress' option of fast-import protocol?\n\n>  INPUT FORMAT\n>  ------------\n> diff --git a/test-svn-fe.c b/test-svn-fe.c\n> index 7885eb1..e4bfda0 100644\n> --- a/test-svn-fe.c\n> +++ b/test-svn-fe.c\n> @@ -62,7 +62,7 @@ int main(int argc, const char *argv[])\n>                return apply_delta(argc, argv);\n>\n>        if (argc == 1) {\n> -               if (svndump_init(argv[0], NULL, ref, backflow_fd))\n> +               if (svndump_init(argv[0], NULL, ref, backflow_fd, 1))\n>                        return 1;\n>                svndump_read();\n>                svndump_deinit();\n> diff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\n> index cfb0f2b..a8b8603 100644\n> --- a/vcs-svn/fast_export.c\n> +++ b/vcs-svn/fast_export.c\n> @@ -19,6 +19,7 @@ static uint32_t first_commit_done;\n>  static struct line_buffer postimage = LINE_BUFFER_INIT;\n>  static struct line_buffer report_buffer = LINE_BUFFER_INIT;\n>  static struct strbuf ref_name = STRBUF_INIT;\n> +static int print_progress;\n\n> @@ -30,9 +31,10 @@ static int init_postimage(void)\n>        return buffer_tmpfile_init(&postimage);\n>  }\n>\n> -void fast_export_init(int fd, const char *dst_ref)\n> +void fast_export_init(int fd, const char *dst_ref, int progress)\n>  {\n>        first_commit_done = 0;\n> +       print_progress = progress;\n\nThe only reason you're modifying the API of fast_export_init is so\nthat it can set a global static variable? Also, this change seems\nmore absurd because progress reporting isn't directly related to\nfast_export_init. How about having a dedicated function for option\nparsing that sets all the global statics?\n\nI'm sorry I haven't been more involved with this project. Still, I\nhope this review helps.\n\n-- Ram\n"},{"id":"170893","messageId":"CALkWK0nO7dKkh-5kdLSwbncujOW4R-x_uRCXQcSbkhVP-qstVw@mail.gmail.com","threadId":"27749","inReplyTo":"1309715877-13814-8-git-send-email-divanorama@gmail.com","subject":"Re: [PATCH 7/8] vcs-svn: convert REPORT_FILENO to an option","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2011-07-06T10:09:47Z","receivedAt":"2011-07-06T10:09:47Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi again,\n\nDmitry Ivankov writes:\n> svn-fe needs to read fast-import's responses to \"ls\" and \"cat-blob\".\n> These come through a file descriptor number 3.\n>\n> Sometimes it is easier to setup variable fd than a fixed one. It is\n> the case with pipe() call and even more fd=3 can be already taken.\n> On Windows file descriptors are not by default inherited by a child\n> process, nor there is an option to setup descriptors other than\n> standard stdin, stdout, stderr at a process creation time.\n>\n> Add an option for this file descriptor number in vcs-svn/ and svn-fe,\n> add a simple test for it.\n>\n> To be used like following:\n> $ svn-fe --read-blob-fd=7 ... 7> diff --git a/contrib/svn-fe/svn-fe.c b/contrib/svn-fe/svn-fe.c\n[...]\n>> -       if (svndump_init(NULL, url, ref, backflow_fd))\n>> +       if (svndump_init(NULL, url, ref, backflow_fd, progress))\n>\n> You're modifying the svndump_init API for every new option that's\n> added.  This'll clearly break down when you have many options -- how\n> about wrapping it up in an options structure and then passing that?\nWell, there has to be a function to init that structure then. And the\nstructure will become a part of API.\nSo don't know if it's worthy.\n\n>> diff --git a/contrib/svn-fe/svn-fe.txt b/contrib/svn-fe/svn-fe.txt\n>> index a7ad368..f1a459e 100644\n>> --- a/contrib/svn-fe/svn-fe.txt\n>> +++ b/contrib/svn-fe/svn-fe.txt\n>> @@ -39,6 +39,9 @@ OPTIONS\n>>        Integer number of file descriptor from which\n>>        responses to 'ls' and 'cat-blob' requests will come.\n>>        Default is fd=3.\n>> +--[no-]progress::\n>> +       Write 'progress' lines to fast-import stream. These\n>> +       can be displayed by fast-import.\n>\n> Hm, this will make it a little too silent for the end-user.  What do\n> you feel about printing something minimalistic like a '.' for each\n> imported revision? Atleast it won't look like it's hung.\nFor a medium 8k commit repo it is 100 lines - still too much.\nA single line for the first revision seem harmless and will indicate\nthat the remote connection succeeded\n(helps to see that it's not a connection timeout, probably caused by\ndns lookup or a firewall).\n\n>  Also, how does this interact with the 'progress' option of fast-import protocol?\ngit fast-import --quiet prints any progress line produced by a helper.\ntransport-helper.c tries to set the option\nbut doesn't fail if it is not accepted or if helper doesn't support\noptions at all.\nFor now the helper doesn't use this protocol option.\n\nA better solution could be to use progress.o api to display progress.\nOr an ad-hoc hack with adaptive progress step, say report a progress\non each \"power of two\"-th revision.\n\nAs a starting point for tests let there bust a simple switch-off.\n\n[...]\n>> diff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\n>> index cfb0f2b..a8b8603 100644\n>> --- a/vcs-svn/fast_export.c\n>> +++ b/vcs-svn/fast_export.c\n>> @@ -19,6 +19,7 @@ static uint32_t first_commit_done;\n>>  static struct line_buffer postimage = LINE_BUFFER_INIT;\n>>  static struct line_buffer report_buffer = LINE_BUFFER_INIT;\n>>  static struct strbuf ref_name = STRBUF_INIT;\n>> +static int print_progress;\n[...]\n>> -void fast_export_init(int fd, const char *dst_ref)\n>> +void fast_export_init(int fd, const char *dst_ref, int progress)\n>>  {\n>>        first_commit_done = 0;\n>> +       print_progress = progress;\n>\n> The only reason you're modifying the API of fast_export_init is so\n> that it can set a global static variable?\nLooked once more at how these new variables are used. We can move\nprogress lines generation to svndump.c. And also move ref_name from\n_init and being global static to a parameter in fast_export_begin_commit(),\nand to be more sane s/revision/target_mark/ s/revision - 1/from_mark/ and\nadd a from_mark parameter, move first_commit_done logic to svndump.o.\nThis way fast_export.o can operate on single commits and maybe it'll be\neasier to use it to apply svn branches layout in svn-fe in one run, though I'm\nnot sure I'll use svn-fe to manage svn branches.\n\n>  Also, this change seems\n> more absurd because progress reporting isn't directly related to\n> fast_export_init.  How about having a dedicated function for option\n> parsing that sets all the global statics?\n>\n> I'm sorry I haven't been more involved with this project.  Still, I\n> hope this review helps.\nThis is very helpful, thanks.\n\n>\n> -- Ram\n>\n"},{"id":"170910","messageId":"CALkWK0mVcEvaXiODtvNT0S88b4uuc_h1AZx-CLi6KN6KO2=ESw@mail.gmail.com","threadId":"27749","inReplyTo":"CA+gfSn-LEW36+bbxs1ydnhd3iQu-+wvrsSTuTszTYKm+jmuhOw@mail.gmail.com","subject":"Re: [PATCH 8/8] vcs-svn: allow to disable 'progress' lines","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2011-07-06T11:40:07Z","receivedAt":"2011-07-06T11:40:07Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi again,\n\nDmitry Ivankov writes:\n>>> -       if (svndump_init(NULL, url, ref, backflow_fd))\n>>> +       if (svndump_init(NULL, url, ref, backflow_fd, progress))\n>>\n>> You're modifying the svndump_init API for every new option that's\n>> added.  This'll clearly break down when you have many options -- how\n>> about wrapping it up in an options structure and then passing that?\n> Well, there has to be a function to init that structure then. And the\n> structure will become a part of API.\n> So don't know if it's worthy.\n\nWell, you'll be wrapping up url, ref, backflow_fd, progress, and all\nother future command-line parameters in one variable. Think about it.\n You can use the parse-options API to fill in default arguments\nanyway.\n\n>>> diff --git a/contrib/svn-fe/svn-fe.txt b/contrib/svn-fe/svn-fe.txt\n>>> index a7ad368..f1a459e 100644\n>>> --- a/contrib/svn-fe/svn-fe.txt\n>>> +++ b/contrib/svn-fe/svn-fe.txt\n>>> @@ -39,6 +39,9 @@ OPTIONS\n>>>        Integer number of file descriptor from which\n>>>        responses to 'ls' and 'cat-blob' requests will come.\n>>>        Default is fd=3.\n>>> +--[no-]progress::\n>>> +       Write 'progress' lines to fast-import stream. These\n>>> +       can be displayed by fast-import.\n>>\n>> Hm, this will make it a little too silent for the end-user.  What do\n>> you feel about printing something minimalistic like a '.' for each\n>> imported revision? Atleast it won't look like it's hung.\n> For a medium 8k commit repo it is 100 lines - still too much.\n> A single line for the first revision seem harmless and will indicate\n> that the remote connection succeeded\n> (helps to see that it's not a connection timeout, probably caused by\n> dns lookup or a firewall).\n\nAh, right. I agree with you.\n\n>>  Also, how does this interact with the 'progress' option of fast-import protocol?\n> git fast-import --quiet prints any progress line produced by a helper.\n> transport-helper.c tries to set the option\n> but doesn't fail if it is not accepted or if helper doesn't support\n> options at all.\n> For now the helper doesn't use this protocol option.\n>\n> A better solution could be to use progress.o api to display progress.\n> Or an ad-hoc hack with adaptive progress step, say report a progress\n> on each \"power of two\"-th revision.\n> As a starting point for tests let there bust a simple switch-off.\n\nYeah, that makes sense -- would love to see that in future.\n\n>>> -void fast_export_init(int fd, const char *dst_ref)\n>>> +void fast_export_init(int fd, const char *dst_ref, int progress)\n>>>  {\n>>>        first_commit_done = 0;\n>>> +       print_progress = progress;\n>>\n>> The only reason you're modifying the API of fast_export_init is so\n>> that it can set a global static variable?\n> Looked once more at how these new variables are used. We can move\n> progress lines generation to svndump.c. And also move ref_name from\n> _init and being global static to a parameter in fast_export_begin_commit(),\n> and to be more sane s/revision/target_mark/ s/revision - 1/from_mark/ and\n> add a from_mark parameter, move first_commit_done logic to svndump.o.\n> This way fast_export.o can operate on single commits and maybe it'll be\n> easier to use it to apply svn branches layout in svn-fe in one run, though I'm\n> not sure I'll use svn-fe to manage svn branches.\n\nMakes sense.\nThanks.\n\n-- Ram\n"}]}