{"thread":{"id":"27803","subject":"[PATCH v2 00/11] vcs-svn,svn-fe add a couple of options","startedAt":"2011-07-13T12:21:02Z","lastAt":"2011-07-28T10:27:29Z","messageCount":24,"participants":["Dmitry Ivankov","Jonathan Nieder"],"isPatch":true,"patchVersion":2,"patchTotal":11},"messages":[{"id":"171233","messageId":"1310559673-5026-1-git-send-email-divanorama@gmail.com","threadId":"27803","inReplyTo":null,"subject":"[PATCH v2 00/11] vcs-svn,svn-fe add a couple of options","fromName":"Dmitry Ivankov","fromEmail":"divanorama@gmail.com","sentAt":"2011-07-13T12:21:02Z","receivedAt":"2011-07-13T12:21:02Z","isPatch":true,"sender":{"key":"divanorama@gmail.com","avatar":"https://avatars.githubusercontent.com/u/158999?v=4"},"body":"This is a second iteration of [1].\n\nThe most significant changes since [1] are:\n1) svndump_init parameters are wrapped into a struct svndump_init_args\nThis way it is a bit easier to play with various parameters.\nThanks to Ram for insisting on this.\n2) added [6/11] \"vcs-svn: move commit parameters logic to svndump.c\"\nNow fast_export is touched only in one commit of this series and there\nare other nice implications mentioned in the commit message.\n3) added [10/11] and [11/11] - two more options to be used in a\nupcoming iteration of remote-svn-alpha series\n\nThe patch base is svn-fe branch at git://repo.or.cz/git/jrn.git\n\n[1] http://thread.gmane.org/gmane.comp.version-control.git/176578\n\nDmitry Ivankov (11):\n svn-fe: add man target to Makefile\n test-svn-fe: use parse-options\n svn-fe: add EXTLIBS needed for parse-options\n svn-fe: add usage and unpositional arguments versions\n vcs-svn: move url parameter from _read to _init\n vcs-svn: move commit parameters logic to svndump.c\n vcs-svn,svn-fe: allow to specify dump destination ref\n vcs-svn,svn-fe: convert REPORT_FILENO to an option\n vcs-svn,svn-fe: allow to disable 'progress' lines\n vcs-svn,svn-fe: add --incremental option\n vcs-svn,svn-fe: add an option to write svnrev notes\n\n contrib/svn-fe/Makefile | 18 +++---\n contrib/svn-fe/svn-fe.c | 46 ++++++++++++-\n contrib/svn-fe/svn-fe.txt | 37 +++++++++--\n t/t9010-svn-fe.sh | 157 +++++++++++++++++++++++++++++++++++++++-----\n test-svn-fe.c | 59 +++++++++++++----\n vcs-svn/fast_export.c | 44 ++++---------\n vcs-svn/fast_export.h | 8 ++-\n vcs-svn/svndump.c | 82 +++++++++++++++++++----\n vcs-svn/svndump.h | 11 +++-\n 9 files changed, 361 insertions(+), 101 deletions(-)\n\n-- \n1.7.3.4\n"},{"id":"171236","messageId":"1310559673-5026-2-git-send-email-divanorama@gmail.com","threadId":"27803","inReplyTo":"1310559673-5026-1-git-send-email-divanorama@gmail.com","subject":"[PATCH v2 01/11] svn-fe: add man target to Makefile","fromName":"Dmitry Ivankov","fromEmail":"divanorama@gmail.com","sentAt":"2011-07-13T12:21:03Z","receivedAt":"2011-07-13T12:21:03Z","isPatch":true,"sender":{"key":"divanorama@gmail.com","avatar":"https://avatars.githubusercontent.com/u/158999?v=4"},"body":"There already is a svn-fe.1 target. But 'man' being a standard\ntarget is easier to discover or type. It can also be reused if\nmore manpages arise here.\n\nSigned-off-by: Dmitry Ivankov \n---\n contrib/svn-fe/Makefile | 4 +++-\n 1 files changed, 3 insertions(+), 1 deletions(-)\n\ndiff --git a/contrib/svn-fe/Makefile b/contrib/svn-fe/Makefile\nindex 360d8da..bc03a3e 100644\n--- a/contrib/svn-fe/Makefile\n+++ b/contrib/svn-fe/Makefile\n@@ -33,6 +33,8 @@ ifndef V\n endif\n endif\n \n+man: svn-fe.1\n+\n svn-fe$X: svn-fe.o $(VCSSVN_LIB) $(GIT_LIB)\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ svn-fe.o \\\n \t\t$(ALL_LDFLAGS) $(LIBS)\n@@ -60,4 +62,4 @@ svn-fe.1: svn-fe.txt\n clean:\n \t$(RM) svn-fe$X svn-fe.o svn-fe.html svn-fe.xml svn-fe.1\n \n-.PHONY: all clean FORCE\n+.PHONY: all man clean FORCE\n-- \n1.7.3.4\n"},{"id":"171234","messageId":"1310559673-5026-3-git-send-email-divanorama@gmail.com","threadId":"27803","inReplyTo":"1310559673-5026-1-git-send-email-divanorama@gmail.com","subject":"[PATCH v2 02/11] test-svn-fe: use parse-options","fromName":"Dmitry Ivankov","fromEmail":"divanorama@gmail.com","sentAt":"2011-07-13T12:21:04Z","receivedAt":"2011-07-13T12:21:04Z","isPatch":true,"sender":{"key":"divanorama@gmail.com","avatar":"https://avatars.githubusercontent.com/u/158999?v=4"},"body":"There was custom options parsing. As more options arise it will\nbe easier to add and document new options with parse-options api.\n\nSigned-off-by: Dmitry Ivankov \n---\n test-svn-fe.c | 42 ++++++++++++++++++++++++++++--------------\n 1 files changed, 28 insertions(+), 14 deletions(-)\n\ndiff --git a/test-svn-fe.c b/test-svn-fe.c\nindex 332a5f7..0aab245 100644\n--- a/test-svn-fe.c\n+++ b/test-svn-fe.c\n@@ -3,28 +3,38 @@\n */\n \n #include \"git-compat-util.h\"\n+#include \"parse-options.h\"\n #include \"vcs-svn/svndump.h\"\n #include \"vcs-svn/svndiff.h\"\n #include \"vcs-svn/sliding_window.h\"\n #include \"vcs-svn/line_buffer.h\"\n \n-static const char test_svnfe_usage[] =\n-\t\"test-svn-fe ( | [-d] )\";\n+static const char * const test_svnfe_usage[] = {\n+\t\"test-svn-fe ( | -d )\",\n+\tNULL\n+};\n \n-static int apply_delta(int argc, char *argv[])\n+static int d;\n+\n+static struct option test_svnfe_options[] = {\n+\tOPT_SET_INT('d', NULL, &d, \"test apply_delta\", 1),\n+\tOPT_END()\n+};\n+\n+static int apply_delta(int argc, const char *argv[])\n {\n \tstruct line_buffer preimage = LINE_BUFFER_INIT;\n \tstruct line_buffer delta = LINE_BUFFER_INIT;\n \tstruct sliding_view preimage_view = SLIDING_VIEW_INIT(&preimage, -1);\n \n-\tif (argc != 5)\n-\t\tusage(test_svnfe_usage);\n+\tif (argc != 3)\n+\t\tusage_with_options(test_svnfe_usage, test_svnfe_options);\n \n-\tif (buffer_init(&preimage, argv[2]))\n+\tif (buffer_init(&preimage, argv[0]))\n \t\tdie_errno(\"cannot open preimage\");\n-\tif (buffer_init(&delta, argv[3]))\n+\tif (buffer_init(&delta, argv[1]))\n \t\tdie_errno(\"cannot open delta\");\n-\tif (svndiff0_apply(&delta, (off_t) strtoull(argv[4], NULL, 0),\n+\tif (svndiff0_apply(&delta, (off_t) strtoull(argv[2], NULL, 0),\n \t\t\t\t\t&preimage_view, stdout))\n \t\treturn 1;\n \tif (buffer_deinit(&preimage))\n@@ -37,10 +47,16 @@ static int apply_delta(int argc, char *argv[])\n \treturn 0;\n }\n \n-int main(int argc, char *argv[])\n+int main(int argc, const char *argv[])\n {\n-\tif (argc == 2) {\n-\t\tif (svndump_init(argv[1]))\n+\targc = parse_options(argc, argv, NULL, test_svnfe_options,\n+\t\t\t\t\t\ttest_svnfe_usage, 0);\n+\n+\tif (d)\n+\t\treturn apply_delta(argc, argv);\n+\n+\tif (argc == 1) {\n+\t\tif (svndump_init(argv[0]))\n \t\t\treturn 1;\n \t\tsvndump_read(NULL);\n \t\tsvndump_deinit();\n@@ -48,7 +64,5 @@ int main(int argc, char *argv[])\n \t\treturn 0;\n \t}\n \n-\tif (argc >= 2 && !strcmp(argv[1], \"-d\"))\n-\t\treturn apply_delta(argc, argv);\n-\tusage(test_svnfe_usage);\n+\tusage_with_options(test_svnfe_usage, test_svnfe_options);\n }\n-- \n1.7.3.4\n"},{"id":"171235","messageId":"1310559673-5026-4-git-send-email-divanorama@gmail.com","threadId":"27803","inReplyTo":"1310559673-5026-1-git-send-email-divanorama@gmail.com","subject":"[PATCH v2 03/11] svn-fe: add EXTLIBS needed for parse-options","fromName":"Dmitry Ivankov","fromEmail":"divanorama@gmail.com","sentAt":"2011-07-13T12:21:05Z","receivedAt":"2011-07-13T12:21:05Z","isPatch":true,"sender":{"key":"divanorama@gmail.com","avatar":"https://avatars.githubusercontent.com/u/158999?v=4"},"body":"Currently parse-options.o pull quite a big bunch of dependencies\nthat are neither pulled in by svn-fe Makefile nor included in libgit.a.\n\nUse a temporary hack: put hardcoded EXTLIBS, this may not work in all\nsetups because /Makefile logic is not repeated.\n\nFor example, one may need -lcrypto instead of -lssl or no crypto library\nif BLK_SHA1 is set, also an additional -lz or -lpcre could be required.\n\nSigned-off-by: Dmitry Ivankov \n---\n contrib/svn-fe/Makefile | 12 +++++-------\n 1 files changed, 5 insertions(+), 7 deletions(-)\n\ndiff --git a/contrib/svn-fe/Makefile b/contrib/svn-fe/Makefile\nindex bc03a3e..bf1625c 100644\n--- a/contrib/svn-fe/Makefile\n+++ b/contrib/svn-fe/Makefile\n@@ -8,11 +8,12 @@ CFLAGS = -g -O2 -Wall\n LDFLAGS =\n ALL_CFLAGS = $(CFLAGS)\n ALL_LDFLAGS = $(LDFLAGS)\n-EXTLIBS =\n+EXTLIBS = -lssl -lpthread\n \n GIT_LIB = ../../libgit.a\n VCSSVN_LIB = ../../vcs-svn/lib.a\n-LIBS = $(VCSSVN_LIB) $(GIT_LIB) $(EXTLIBS)\n+XDIFF_LIB = ../../xdiff/lib.a\n+LIBS = $(VCSSVN_LIB) $(GIT_LIB) $(XDIFF_LIB) $(EXTLIBS)\n \n QUIET_SUBDIR0 = +$(MAKE) -C # space to separate -C and subdir\n QUIET_SUBDIR1 =\n@@ -53,11 +54,8 @@ svn-fe.1: svn-fe.txt\n \t\t../contrib/svn-fe/$@\n \t$(MV) ../../Documentation/svn-fe.1 .\n \n-../../vcs-svn/lib.a: FORCE\n-\t$(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) vcs-svn/lib.a\n-\n-../../libgit.a: FORCE\n-\t$(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) libgit.a\n+$(VCSSVN_LIB) $(GIT_LIB) $(XDIFF_LIB): ../../%.a: FORCE\n+\t$(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) $*.a\n \n clean:\n \t$(RM) svn-fe$X svn-fe.o svn-fe.html svn-fe.xml svn-fe.1\n-- \n1.7.3.4\n"},{"id":"171237","messageId":"1310559673-5026-5-git-send-email-divanorama@gmail.com","threadId":"27803","inReplyTo":"1310559673-5026-1-git-send-email-divanorama@gmail.com","subject":"[PATCH v2 04/11] svn-fe: add usage and unpositional arguments versions","fromName":"Dmitry Ivankov","fromEmail":"divanorama@gmail.com","sentAt":"2011-07-13T12:21:06Z","receivedAt":"2011-07-13T12:21:06Z","isPatch":true,"sender":{"key":"divanorama@gmail.com","avatar":"https://avatars.githubusercontent.com/u/158999?v=4"},"body":"There will be more command line options for svn-fe so support\nunpositional version for flexibility. Also clarify the meaning\nof url parameter.\n\n$ svn-fe --git-svn-id-url=url\ndoes the same thing as\n$ svn-fe url\ni.e., url is used to generate git-svn-id: lines, if url is set.\n\nSigned-off-by: Dmitry Ivankov \n---\n contrib/svn-fe/Makefile | 2 +-\n contrib/svn-fe/svn-fe.c | 30 +++++++++++++++++++++++++++---\n contrib/svn-fe/svn-fe.txt | 17 +++++++++++++----\n 3 files changed, 41 insertions(+), 8 deletions(-)\n\ndiff --git a/contrib/svn-fe/Makefile b/contrib/svn-fe/Makefile\nindex bf1625c..3e18395 100644\n--- a/contrib/svn-fe/Makefile\n+++ b/contrib/svn-fe/Makefile\n@@ -41,7 +41,7 @@ svn-fe$X: svn-fe.o $(VCSSVN_LIB) $(GIT_LIB)\n \t\t$(ALL_LDFLAGS) $(LIBS)\n \n svn-fe.o: svn-fe.c ../../vcs-svn/svndump.h\n-\t$(QUIET_CC)$(CC) -I../../vcs-svn -o $*.o -c $(ALL_CFLAGS) $<\n+\t$(QUIET_CC)$(CC) -I../../vcs-svn -I../.. -o $*.o -c $(ALL_CFLAGS) $<\n \n svn-fe.html: svn-fe.txt\n \t$(QUIET_SUBDIR0)../../Documentation $(QUIET_SUBDIR1) \\\ndiff --git a/contrib/svn-fe/svn-fe.c b/contrib/svn-fe/svn-fe.c\nindex 35db24f..7e829b9 100644\n--- a/contrib/svn-fe/svn-fe.c\n+++ b/contrib/svn-fe/svn-fe.c\n@@ -3,14 +3,38 @@\n * You may freely use, modify, distribute, and relicense it.\n */\n \n-#include \n+#include \"git-compat-util.h\"\n+#include \"parse-options.h\"\n #include \"svndump.h\"\n \n-int main(int argc, char **argv)\n+static const char * const svn_fe_usage[] = {\n+\t\"svn-fe [options] [git-svn-id-url] < dump | fast-import-backend\",\n+\tNULL\n+};\n+\n+static const char *url;\n+\n+static struct option svn_fe_options[] = {\n+\tOPT_STRING(0, \"git-svn-id-url\", &url, \"url\",\n+\t\t\"append git-svn metadata line to commit messages\"),\n+\tOPT_END()\n+};\n+\n+int main(int argc, const char **argv)\n {\n+\targc = parse_options(argc, argv, NULL, svn_fe_options,\n+\t\t\t\t\t\tsvn_fe_usage, 0);\n+\tif (argc == 1) {\n+\t\tif (url)\n+\t\t\tusage_msg_opt(\"git-svn-id-url is set twice: as a \"\n+\t\t\t\t\t\"--parameter and as a [parameter]\",\n+\t\t\t\t\tsvn_fe_usage, svn_fe_options);\n+\t\turl = argv[0];\n+\t} else if (argc)\n+\t\tusage_with_options(svn_fe_usage, svn_fe_options);\n \tif (svndump_init(NULL))\n \t\treturn 1;\n-\tsvndump_read((argc > 1) ? argv[1] : NULL);\n+\tsvndump_read(url);\n \tsvndump_deinit();\n \tsvndump_reset();\n \treturn 0;\ndiff --git a/contrib/svn-fe/svn-fe.txt b/contrib/svn-fe/svn-fe.txt\nindex 2dd27ce..8c6d347 100644\n--- a/contrib/svn-fe/svn-fe.txt\n+++ b/contrib/svn-fe/svn-fe.txt\n@@ -10,7 +10,7 @@ SYNOPSIS\n [verse]\n mkfifo backchannel &&\n svnadmin dump --deltas REPO |\n-\tsvn-fe [url] 3backchannel\n \n DESCRIPTION\n@@ -25,6 +25,14 @@ command.\n Note: this tool is very young. The details of its commandline\n interface may change in backward incompatible ways.\n \n+OPTIONS\n+-------\n+\n+--git-svn-id-url=::\n+\tUrl to be used in git-svn-id: lines in git-svn\n+\tmetadata lines format. See NOTES for more detailed\n+\tdescription.\n+\n INPUT FORMAT\n ------------\n Subversion's repository dump format is documented in full in\n@@ -50,9 +58,10 @@ user \n as committer, where 'user' is the value of the `svn:author` property\n and 'UUID' the repository's identifier.\n \n-To support incremental imports, 'svn-fe' puts a `git-svn-id` line at\n-the end of each commit log message if passed an url on the command\n-line. This line has the form `git-svn-id: URL@REVNO UUID`.\n+'svn-fe' can be used in preparing a repository for 'git-svn' as follows.\n+If `git-svn-id-url` is specified, 'svn-fe' will put `git-svn-id` line at\n+the end of each commit log message.\n+This line has the form `git-svn-id: URL@REVNO UUID`.\n \n The resulting repository will generally require further processing\n to put each project in its own repository and to separate the history\n-- \n1.7.3.4\n"},{"id":"171241","messageId":"1310559673-5026-6-git-send-email-divanorama@gmail.com","threadId":"27803","inReplyTo":"1310559673-5026-1-git-send-email-divanorama@gmail.com","subject":"[PATCH v2 05/11] vcs-svn: move url parameter from _read to _init","fromName":"Dmitry Ivankov","fromEmail":"divanorama@gmail.com","sentAt":"2011-07-13T12:21:07Z","receivedAt":"2011-07-13T12:21:07Z","isPatch":true,"sender":{"key":"divanorama@gmail.com","avatar":"https://avatars.githubusercontent.com/u/158999?v=4"},"body":"svndump_read takes a url parameter that is used in git-svn-id: lines\ngeneration. Internally it is stored in dump_ctx which is initialized\nin svndump_init with reset_dump_ctx and then is reinitialized again\nin svndump_read.\n\nMove url parameter to svndump_init so that reset_dump_ctx is done\nonce per dump and in the same place as other resets. Wrap all _init\nparameters to a struct svndump_args. More parameters will arise and\nall will go to this struct to setup the module for dumping. Having\na struct reduces a chance to confuse one parameter with another a\nbit, if they are filled via named assignments or common defines.\n\nSigned-off-by: Dmitry Ivankov \n---\n contrib/svn-fe/svn-fe.c | 12 ++++++------\n test-svn-fe.c | 7 +++++--\n vcs-svn/svndump.c | 12 ++++++------\n vcs-svn/svndump.h | 8 ++++++--\n 4 files changed, 23 insertions(+), 16 deletions(-)\n\ndiff --git a/contrib/svn-fe/svn-fe.c b/contrib/svn-fe/svn-fe.c\nindex 7e829b9..59141d6 100644\n--- a/contrib/svn-fe/svn-fe.c\n+++ b/contrib/svn-fe/svn-fe.c\n@@ -12,10 +12,10 @@ static const char * const svn_fe_usage[] = {\n \tNULL\n };\n \n-static const char *url;\n+static struct svndump_args args;\n \n static struct option svn_fe_options[] = {\n-\tOPT_STRING(0, \"git-svn-id-url\", &url, \"url\",\n+\tOPT_STRING(0, \"git-svn-id-url\", &args.url, \"url\",\n \t\t\"append git-svn metadata line to commit messages\"),\n \tOPT_END()\n };\n@@ -25,16 +25,16 @@ int main(int argc, const char **argv)\n \targc = parse_options(argc, argv, NULL, svn_fe_options,\n \t\t\t\t\t\tsvn_fe_usage, 0);\n \tif (argc == 1) {\n-\t\tif (url)\n+\t\tif (args.url)\n \t\t\tusage_msg_opt(\"git-svn-id-url is set twice: as a \"\n \t\t\t\t\t\"--parameter and as a [parameter]\",\n \t\t\t\t\tsvn_fe_usage, svn_fe_options);\n-\t\turl = argv[0];\n+\t\targs.url = argv[0];\n \t} else if (argc)\n \t\tusage_with_options(svn_fe_usage, svn_fe_options);\n-\tif (svndump_init(NULL))\n+\tif (svndump_init(&args))\n \t\treturn 1;\n-\tsvndump_read(url);\n+\tsvndump_read();\n \tsvndump_deinit();\n \tsvndump_reset();\n \treturn 0;\ndiff --git a/test-svn-fe.c b/test-svn-fe.c\nindex 0aab245..9e5b1a6 100644\n--- a/test-svn-fe.c\n+++ b/test-svn-fe.c\n@@ -14,6 +14,8 @@ static const char * const test_svnfe_usage[] = {\n \tNULL\n };\n \n+static struct svndump_args args;\n+\n static int d;\n \n static struct option test_svnfe_options[] = {\n@@ -56,9 +58,10 @@ int main(int argc, const char *argv[])\n \t\treturn apply_delta(argc, argv);\n \n \tif (argc == 1) {\n-\t\tif (svndump_init(argv[0]))\n+\t\targs.filename = argv[0];\n+\t\tif (svndump_init(&args))\n \t\t\treturn 1;\n-\t\tsvndump_read(NULL);\n+\t\tsvndump_read();\n \t\tsvndump_deinit();\n \t\tsvndump_reset();\n \t\treturn 0;\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex b1f4161..60cccad 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -12,6 +12,7 @@\n #include \"fast_export.h\"\n #include \"line_buffer.h\"\n #include \"strbuf.h\"\n+#include \"svndump.h\"\n \n /*\n * Compare start of string to literal of equal length;\n@@ -313,14 +314,13 @@ static void end_revision(void)\n \t\tfast_export_end_commit(rev_ctx.revision);\n }\n \n-void svndump_read(const char *url)\n+void svndump_read(void)\n {\n \tchar *val;\n \tchar *t;\n \tuint32_t active_ctx = DUMP_CTX;\n \tuint32_t len;\n \n-\treset_dump_ctx(url);\n \twhile ((t = buffer_read_line(&input))) {\n \t\tval = strchr(t, ':');\n \t\tif (!val)\n@@ -455,10 +455,10 @@ void svndump_read(const char *url)\n \t\tend_revision();\n }\n \n-int svndump_init(const char *filename)\n+int svndump_init(const struct svndump_args *args)\n {\n-\tif (buffer_init(&input, filename))\n-\t\treturn error(\"cannot open %s: %s\", filename, strerror(errno));\n+\tif (buffer_init(&input, args->filename))\n+\t\treturn error(\"cannot open %s: %s\", args->filename, strerror(errno));\n \tfast_export_init(REPORT_FILENO);\n \tstrbuf_init(&dump_ctx.uuid, 4096);\n \tstrbuf_init(&dump_ctx.url, 4096);\n@@ -466,7 +466,7 @@ int svndump_init(const char *filename)\n \tstrbuf_init(&rev_ctx.author, 4096);\n \tstrbuf_init(&node_ctx.src, 4096);\n \tstrbuf_init(&node_ctx.dst, 4096);\n-\treset_dump_ctx(NULL);\n+\treset_dump_ctx(args->url);\n \treset_rev_ctx(0);\n \treset_node_ctx(NULL);\n \treturn 0;\ndiff --git a/vcs-svn/svndump.h b/vcs-svn/svndump.h\nindex df9ceb0..b3dbf24 100644\n--- a/vcs-svn/svndump.h\n+++ b/vcs-svn/svndump.h\n@@ -1,8 +1,12 @@\n #ifndef SVNDUMP_H_\n #define SVNDUMP_H_\n \n-int svndump_init(const char *filename);\n-void svndump_read(const char *url);\n+struct svndump_args {\n+\tconst char *filename, *url;\n+};\n+\n+int svndump_init(const struct svndump_args *args);\n+void svndump_read(void);\n void svndump_deinit(void);\n void svndump_reset(void);\n \n-- \n1.7.3.4\n"},{"id":"171243","messageId":"1310559673-5026-7-git-send-email-divanorama@gmail.com","threadId":"27803","inReplyTo":"1310559673-5026-1-git-send-email-divanorama@gmail.com","subject":"[PATCH v2 06/11] vcs-svn: move commit parameters logic to svndump.c","fromName":"Dmitry Ivankov","fromEmail":"divanorama@gmail.com","sentAt":"2011-07-13T12:21:08Z","receivedAt":"2011-07-13T12:21:08Z","isPatch":true,"sender":{"key":"divanorama@gmail.com","avatar":"https://avatars.githubusercontent.com/u/158999?v=4"},"body":"fast_export.c had logic to set up commit ref, author name, email,\nparent commit, import mark and git-svn-id: line based on both it's\nown state (current import batch history) and the arguments passed.\n\nLift the decision on these parameters to the caller. This way it is\neasier to customize them. Move progress lines generation to the caller\nfor the same reason.\n\nNow fast_export doesn't have any internal state except the files set\nup in fast_export_init, so it doesn't rely on being passed commits\nsequentially and to one and the same branch. It operates just on a\ncurrent commit. Which makes it possible to generate an incremental\nstream (if stream's first commit parent is set up properly by the\ncaller) or maybe to generate a stream for multiple svn branches.\n\nAlso progress lines generation is lifted up to svndump.o. So further\nprogress indication enhancements won't need to change fast_export.o\napi.\n\nSigned-off-by: Dmitry Ivankov \n---\n vcs-svn/fast_export.c | 44 ++++++++++++++------------------------------\n vcs-svn/fast_export.h | 8 +++++---\n vcs-svn/svndump.c | 30 ++++++++++++++++++++++++++----\n 3 files changed, 45 insertions(+), 37 deletions(-)\n\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex 19d7c34..04001b8 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -13,9 +13,6 @@\n #include \"sliding_window.h\"\n #include \"line_buffer.h\"\n \n-#define MAX_GITSVN_LINE_LEN 4096\n-\n-static uint32_t first_commit_done;\n static struct line_buffer postimage = LINE_BUFFER_INIT;\n static struct line_buffer report_buffer = LINE_BUFFER_INIT;\n \n@@ -31,7 +28,6 @@ static int init_postimage(void)\n \n void fast_export_init(int fd)\n {\n-\tfirst_commit_done = 0;\n \tif (buffer_fdinit(&report_buffer, fd))\n \t\tdie_errno(\"cannot read from file descriptor %d\", fd);\n }\n@@ -73,42 +69,30 @@ void fast_export_modify(const char *path, uint32_t mode, const char *dataref)\n \tputchar('\\n');\n }\n \n-static char gitsvnline[MAX_GITSVN_LINE_LEN];\n-void fast_export_begin_commit(uint32_t revision, const char *author,\n-\t\t\tconst struct strbuf *log,\n-\t\t\tconst char *uuid, const char *url,\n-\t\t\tunsigned long timestamp)\n+void fast_export_begin_commit(uint32_t set_mark, const char *committer_name,\n+\t\t\tconst char *committer_login, const char *committer_domain,\n+\t\t\tconst struct strbuf *log, const char *gitsvnline,\n+\t\t\tunsigned long timestamp, uint32_t from_mark,\n+\t\t\tconst char *dst_ref)\n {\n-\tstatic const struct strbuf empty = STRBUF_INIT;\n-\tif (!log)\n-\t\tlog = ∅\n-\tif (*uuid && *url) {\n-\t\tsnprintf(gitsvnline, MAX_GITSVN_LINE_LEN,\n-\t\t\t\t\"\\n\\ngit-svn-id: %s@%\"PRIu32\" %s\\n\",\n-\t\t\t\t url, revision, uuid);\n-\t} else {\n-\t\t*gitsvnline = '\\0';\n-\t}\n-\tprintf(\"commit refs/heads/master\\n\");\n-\tprintf(\"mark :%\"PRIu32\"\\n\", revision);\n+\tif (!gitsvnline)\n+\t\tgitsvnline = \"\";\n+\tprintf(\"commit %s\\n\", dst_ref);\n+\tif (set_mark)\n+\t\tprintf(\"mark :%\"PRIu32\"\\n\", set_mark);\n \tprintf(\"committer %s <%s@%s> %ld +0000\\n\",\n-\t\t *author ? author : \"nobody\",\n-\t\t *author ? author : \"nobody\",\n-\t\t *uuid ? uuid : \"local\", timestamp);\n+\t\tcommitter_name, committer_login, committer_domain,\n+\t\ttimestamp);\n \tprintf(\"data %\"PRIuMAX\"\\n\",\n \t\t(uintmax_t) (log->len + strlen(gitsvnline)));\n \tfwrite(log->buf, log->len, 1, stdout);\n \tprintf(\"%s\\n\", gitsvnline);\n-\tif (!first_commit_done) {\n-\t\tif (revision > 1)\n-\t\t\tprintf(\"from :%\"PRIu32\"\\n\", revision - 1);\n-\t\tfirst_commit_done = 1;\n-\t}\n+\tif (from_mark)\n+\t\tprintf(\"from :%\"PRIu32\"\\n\", from_mark);\n }\n \n void fast_export_end_commit(uint32_t revision)\n {\n-\tprintf(\"progress Imported commit %\"PRIu32\".\\n\\n\", revision);\n }\n \n static void ls_from_rev(uint32_t rev, const char *path)\ndiff --git a/vcs-svn/fast_export.h b/vcs-svn/fast_export.h\nindex 43d05b6..6c1c2be 100644\n--- a/vcs-svn/fast_export.h\n+++ b/vcs-svn/fast_export.h\n@@ -10,9 +10,11 @@ void fast_export_reset(void);\n \n void fast_export_delete(const char *path);\n void fast_export_modify(const char *path, uint32_t mode, const char *dataref);\n-void fast_export_begin_commit(uint32_t revision, const char *author,\n-\t\t\tconst struct strbuf *log, const char *uuid,\n-\t\t\tconst char *url, unsigned long timestamp);\n+void fast_export_begin_commit(uint32_t set_mark, const char *committer_name,\n+\t\t\tconst char *committer_login, const char *committer_domain,\n+\t\t\tconst struct strbuf *log, const char *gitsvnline,\n+\t\t\tunsigned long timestamp, uint32_t from_mark,\n+\t\t\tconst char *dst_ref);\n void fast_export_end_commit(uint32_t revision);\n void fast_export_data(uint32_t mode, uint32_t len, struct line_buffer *input);\n void fast_export_blob_delta(uint32_t mode,\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex 60cccad..c58262a 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -37,6 +37,8 @@\n #define LENGTH_UNKNOWN (~0)\n #define DATE_RFC2822_LEN 31\n \n+#define MAX_GITSVN_LINE_LEN 4096\n+\n static struct line_buffer input = LINE_BUFFER_INIT;\n \n static struct {\n@@ -54,6 +56,7 @@ static struct {\n static struct {\n \tuint32_t version;\n \tstruct strbuf uuid, url;\n+\tint first_commit_done;\n } dump_ctx;\n \n static void reset_node_ctx(char *fname)\n@@ -86,6 +89,7 @@ static void reset_dump_ctx(const char *url)\n \t\tstrbuf_addstr(&dump_ctx.url, url);\n \tdump_ctx.version = 1;\n \tstrbuf_reset(&dump_ctx.uuid);\n+\tdump_ctx.first_commit_done = 0;\n }\n \n static void handle_property(const struct strbuf *key_buf,\n@@ -299,19 +303,37 @@ static void handle_node(void)\n \t\t\t\tnode_ctx.textLength, &input);\n }\n \n+static char gitsvnline[MAX_GITSVN_LINE_LEN];\n static void begin_revision(void)\n {\n+\tint from_mark;\n+\tconst char *author;\n+\tconst char *domain;\n \tif (!rev_ctx.revision)\t/* revision 0 gets no git commit. */\n \t\treturn;\n-\tfast_export_begin_commit(rev_ctx.revision, rev_ctx.author.buf,\n-\t\t&rev_ctx.log, dump_ctx.uuid.buf, dump_ctx.url.buf,\n-\t\trev_ctx.timestamp);\n+\tif (*dump_ctx.uuid.buf && *dump_ctx.url.buf) {\n+\t\tsnprintf(gitsvnline, MAX_GITSVN_LINE_LEN,\n+\t\t\t\t\"\\n\\ngit-svn-id: %s@%\"PRIu32\" %s\\n\",\n+\t\t\t\t dump_ctx.url.buf, rev_ctx.revision, dump_ctx.uuid.buf);\n+\t} else {\n+\t\t*gitsvnline = 0;\n+\t}\n+\tfrom_mark = dump_ctx.first_commit_done ? rev_ctx.revision - 1 : 0;\n+\tauthor = *rev_ctx.author.buf ? rev_ctx.author.buf : \"nobody\";\n+\tdomain = *dump_ctx.uuid.buf ? dump_ctx.uuid.buf : \"local\";\n+\n+\tfast_export_begin_commit(rev_ctx.revision, author, author, domain,\n+\t\t&rev_ctx.log, gitsvnline, rev_ctx.timestamp,\n+\t\tfrom_mark);\n }\n \n static void end_revision(void)\n {\n-\tif (rev_ctx.revision)\n+\tif (rev_ctx.revision) {\n \t\tfast_export_end_commit(rev_ctx.revision);\n+\t\tprintf(\"progress Imported commit %\"PRIu32\".\\n\\n\", rev_ctx.revision);\n+\t\tdump_ctx.first_commit_done = 1;\n+\t}\n }\n \n void svndump_read(void)\n-- \n1.7.3.4\n"},{"id":"171240","messageId":"1310559673-5026-8-git-send-email-divanorama@gmail.com","threadId":"27803","inReplyTo":"1310559673-5026-1-git-send-email-divanorama@gmail.com","subject":"[PATCH v2 07/11] vcs-svn,svn-fe: allow to specify dump destination ref","fromName":"Dmitry Ivankov","fromEmail":"divanorama@gmail.com","sentAt":"2011-07-13T12:21:09Z","receivedAt":"2011-07-13T12:21:09Z","isPatch":true,"sender":{"key":"divanorama@gmail.com","avatar":"https://avatars.githubusercontent.com/u/158999?v=4"},"body":"svn-fe produces fast-import stream for a fixed refs/heads/master ref.\nIt is usually desired to write to a different ref. In a remote helper\nit would be a ref in private namespace. If svn-fe is used by someone\ndirectly it'll be more safe to remind where the commits can go. And\nin both cases it may be needed to import from two repos and hence to\ndifferent refs.\n\nAdd a destination ref parameter to vcs-svn/, a corresponding parameter\nto svn-fe and a simple test for it.\n\n$ svn-fe --ref=refs/heads/master ...\nis an explicit way to stay with the default destination.\n\nSigned-off-by: Dmitry Ivankov \n---\n contrib/svn-fe/svn-fe.c | 3 ++\n contrib/svn-fe/svn-fe.txt | 3 ++\n t/t9010-svn-fe.sh | 49 +++++++++++++++++++++++++++++---------------\n test-svn-fe.c | 5 +++-\n vcs-svn/svndump.c | 11 ++++++---\n vcs-svn/svndump.h | 1 +\n 6 files changed, 50 insertions(+), 22 deletions(-)\n\ndiff --git a/contrib/svn-fe/svn-fe.c b/contrib/svn-fe/svn-fe.c\nindex 59141d6..11739bc 100644\n--- a/contrib/svn-fe/svn-fe.c\n+++ b/contrib/svn-fe/svn-fe.c\n@@ -17,11 +17,14 @@ static struct svndump_args args;\n static struct option svn_fe_options[] = {\n \tOPT_STRING(0, \"git-svn-id-url\", &args.url, \"url\",\n \t\t\"append git-svn metadata line to commit messages\"),\n+\tOPT_STRING(0, \"ref\", &args.ref, \"dst_ref\",\n+\t\t\"write to dst_ref instead of refs/heads/master\"),\n \tOPT_END()\n };\n \n int main(int argc, const char **argv)\n {\n+\targs.ref = \"refs/heads/master\";\n \targc = parse_options(argc, argv, NULL, svn_fe_options,\n \t\t\t\t\t\tsvn_fe_usage, 0);\n \tif (argc == 1) {\ndiff --git a/contrib/svn-fe/svn-fe.txt b/contrib/svn-fe/svn-fe.txt\nindex 8c6d347..20c3315 100644\n--- a/contrib/svn-fe/svn-fe.txt\n+++ b/contrib/svn-fe/svn-fe.txt\n@@ -32,6 +32,9 @@ OPTIONS\n \tUrl to be used in git-svn-id: lines in git-svn\n \tmetadata lines format. See NOTES for more detailed\n \tdescription.\n+--ref=::\n+\tRef to be written by the generated stream.\n+\tDefault is refs/heads/master.\n \n INPUT FORMAT\n ------------\ndiff --git a/t/t9010-svn-fe.sh b/t/t9010-svn-fe.sh\nindex b7eed24..52efabe 100755\n--- a/t/t9010-svn-fe.sh\n+++ b/t/t9010-svn-fe.sh\n@@ -16,18 +16,24 @@ reinit_git () {\n \tmkfifo stream backflow\n }\n \n-try_dump () {\n+try_dump_ext () {\n+\targs=$1 &&\n+\tshift &&\n \tinput=$1 &&\n \tmaybe_fail_svnfe=${2:+test_$2} &&\n \tmaybe_fail_fi=${3:+test_$3} &&\n \n \t{\n-\t\t$maybe_fail_svnfe test-svn-fe \"$input\" >stream 3stream 3backflow &&\n \twait $!\n }\n \n+try_dump () {\n+\ttry_dump_ext \"\" $@\n+}\n+\n properties () {\n \twhile test \"$#\" -ne 0\n \tdo\n@@ -54,6 +60,22 @@ text_no_props () {\n \n >empty\n \n+cat >emptyprop.dump <<-EOF &&\n+SVN-fs-dump-format-version: 3\n+\n+Revision-number: 1\n+Prop-content-length: 10\n+Content-length: 10\n+\n+PROPS-END\n+\n+Revision-number: 2\n+Prop-content-length: 10\n+Content-length: 10\n+\n+PROPS-END\n+EOF\n+\n test_expect_success 'setup: have pipes?' '\n \trm -f frob &&\n \tif mkfifo frob\n@@ -97,21 +119,6 @@ test_expect_failure PIPE 'empty revision' '\n test_expect_success PIPE 'empty properties' '\n \treinit_git &&\n \tprintf \"rev : %s\\n\" \"\" \"\" >expect &&\n-\tcat >emptyprop.dump <<-\\EOF &&\n-\tSVN-fs-dump-format-version: 3\n-\n-\tRevision-number: 1\n-\tProp-content-length: 10\n-\tContent-length: 10\n-\n-\tPROPS-END\n-\n-\tRevision-number: 2\n-\tProp-content-length: 10\n-\tContent-length: 10\n-\n-\tPROPS-END\n-\tEOF\n \ttry_dump emptyprop.dump &&\n \tgit log -p --format=\"rev <%an, %ae>: %s\" HEAD >actual &&\n \ttest_cmp expect actual\n@@ -1111,4 +1118,12 @@ test_expect_success SVNREPO,PIPE 't9135/svn.dump' '\n \t)\n '\n \n+test_expect_success PIPE 'import to notmaster ref' '\n+\treinit_git &&\n+\ttry_dump_ext \"--ref=refs/heads/notmaster\" emptyprop.dump &&\n+\n+\tgit rev-parse --verify notmaster &&\n+\ttest_must_fail git rev-parse --verify master\n+'\n+\n test_done\ndiff --git a/test-svn-fe.c b/test-svn-fe.c\nindex 9e5b1a6..bc437b3 100644\n--- a/test-svn-fe.c\n+++ b/test-svn-fe.c\n@@ -10,7 +10,7 @@\n #include \"vcs-svn/line_buffer.h\"\n \n static const char * const test_svnfe_usage[] = {\n-\t\"test-svn-fe ( | -d )\",\n+\t\"test-svn-fe ([options] | -d )\",\n \tNULL\n };\n \n@@ -20,6 +20,8 @@ static int d;\n \n static struct option test_svnfe_options[] = {\n \tOPT_SET_INT('d', NULL, &d, \"test apply_delta\", 1),\n+\tOPT_STRING(0, \"ref\", &args.ref, \"dst_ref\",\n+\t\t\"write to dst_ref instead of refs/heads/master\"),\n \tOPT_END()\n };\n \n@@ -51,6 +53,7 @@ static int apply_delta(int argc, const char *argv[])\n \n int main(int argc, const char *argv[])\n {\n+\targs.ref = \"refs/heads/master\";\n \targc = parse_options(argc, argv, NULL, test_svnfe_options,\n \t\t\t\t\t\ttest_svnfe_usage, 0);\n \ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex c58262a..616f9ec 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -57,6 +57,7 @@ static struct {\n \tuint32_t version;\n \tstruct strbuf uuid, url;\n \tint first_commit_done;\n+\tstruct strbuf ref_name;\n } dump_ctx;\n \n static void reset_node_ctx(char *fname)\n@@ -82,7 +83,7 @@ static void reset_rev_ctx(uint32_t revision)\n \tstrbuf_reset(&rev_ctx.author);\n }\n \n-static void reset_dump_ctx(const char *url)\n+static void reset_dump_ctx(const char *url, const char *dst_ref)\n {\n \tstrbuf_reset(&dump_ctx.url);\n \tif (url)\n@@ -90,6 +91,8 @@ static void reset_dump_ctx(const char *url)\n \tdump_ctx.version = 1;\n \tstrbuf_reset(&dump_ctx.uuid);\n \tdump_ctx.first_commit_done = 0;\n+\tstrbuf_reset(&dump_ctx.ref_name);\n+\tstrbuf_addstr(&dump_ctx.ref_name, dst_ref);\n }\n \n static void handle_property(const struct strbuf *key_buf,\n@@ -324,7 +327,7 @@ static void begin_revision(void)\n \n \tfast_export_begin_commit(rev_ctx.revision, author, author, domain,\n \t\t&rev_ctx.log, gitsvnline, rev_ctx.timestamp,\n-\t\tfrom_mark);\n+\t\tfrom_mark, dump_ctx.ref_name.buf);\n }\n \n static void end_revision(void)\n@@ -488,7 +491,7 @@ int svndump_init(const struct svndump_args *args)\n \tstrbuf_init(&rev_ctx.author, 4096);\n \tstrbuf_init(&node_ctx.src, 4096);\n \tstrbuf_init(&node_ctx.dst, 4096);\n-\treset_dump_ctx(args->url);\n+\treset_dump_ctx(args->url, args->ref);\n \treset_rev_ctx(0);\n \treset_node_ctx(NULL);\n \treturn 0;\n@@ -497,7 +500,7 @@ int svndump_init(const struct svndump_args *args)\n void svndump_deinit(void)\n {\n \tfast_export_deinit();\n-\treset_dump_ctx(NULL);\n+\treset_dump_ctx(NULL, \"\");\n \treset_rev_ctx(0);\n \treset_node_ctx(NULL);\n \tstrbuf_release(&rev_ctx.log);\ndiff --git a/vcs-svn/svndump.h b/vcs-svn/svndump.h\nindex b3dbf24..904e628 100644\n--- a/vcs-svn/svndump.h\n+++ b/vcs-svn/svndump.h\n@@ -3,6 +3,7 @@\n \n struct svndump_args {\n \tconst char *filename, *url;\n+\tconst char *ref;\n };\n \n int svndump_init(const struct svndump_args *args);\n-- \n1.7.3.4\n"},{"id":"171238","messageId":"1310559673-5026-9-git-send-email-divanorama@gmail.com","threadId":"27803","inReplyTo":"1310559673-5026-1-git-send-email-divanorama@gmail.com","subject":"[PATCH v2 08/11] vcs-svn,svn-fe: convert REPORT_FILENO to an option","fromName":"Dmitry Ivankov","fromEmail":"divanorama@gmail.com","sentAt":"2011-07-13T12:21:10Z","receivedAt":"2011-07-13T12:21:10Z","isPatch":true,"sender":{"key":"divanorama@gmail.com","avatar":"https://avatars.githubusercontent.com/u/158999?v=4"},"body":"svn-fe needs to read fast-import's responses to \"ls\" and \"cat-blob\".\nThese come through a file descriptor number 3.\n\nSometimes it is easier to setup variable fd than a fixed one. It is\nthe case with pipe() call and even more fd=3 can be already taken.\nOn Windows file descriptors are not by default inherited by a child\nprocess, nor there is an option to setup descriptors other than\nstandard stdin, stdout, stderr at a process creation time.\n\nAdd an option for this file descriptor number in vcs-svn/ and svn-fe,\nadd a simple test for it.\n\nTo be used like following:\n$ svn-fe --read-blob-fd=7 ... 7\n---\n contrib/svn-fe/svn-fe.c | 3 ++\n contrib/svn-fe/svn-fe.txt | 8 ++++-\n t/t9010-svn-fe.sh | 58 +++++++++++++++++++++++++++++++++++++++++---\n test-svn-fe.c | 3 ++\n vcs-svn/svndump.c | 4 +--\n vcs-svn/svndump.h | 1 +\n 6 files changed, 68 insertions(+), 9 deletions(-)\n\ndiff --git a/contrib/svn-fe/svn-fe.c b/contrib/svn-fe/svn-fe.c\nindex 11739bc..cd9e449 100644\n--- a/contrib/svn-fe/svn-fe.c\n+++ b/contrib/svn-fe/svn-fe.c\n@@ -19,12 +19,15 @@ static struct option svn_fe_options[] = {\n \t\t\"append git-svn metadata line to commit messages\"),\n \tOPT_STRING(0, \"ref\", &args.ref, \"dst_ref\",\n \t\t\"write to dst_ref instead of refs/heads/master\"),\n+\tOPT_INTEGER(0, \"read-blob-fd\", &args.backflow_fd,\n+\t\t\"read blobs and trees from this fd instead of 3\"),\n \tOPT_END()\n };\n \n int main(int argc, const char **argv)\n {\n \targs.ref = \"refs/heads/master\";\n+\targs.backflow_fd = 3;\n \targc = parse_options(argc, argv, NULL, svn_fe_options,\n \t\t\t\t\t\tsvn_fe_usage, 0);\n \tif (argc == 1) {\ndiff --git a/contrib/svn-fe/svn-fe.txt b/contrib/svn-fe/svn-fe.txt\nindex 20c3315..a7ad368 100644\n--- a/contrib/svn-fe/svn-fe.txt\n+++ b/contrib/svn-fe/svn-fe.txt\n@@ -8,9 +8,9 @@ svn-fe - convert an SVN \"dumpfile\" to a fast-import stream\n SYNOPSIS\n --------\n [verse]\n-mkfifo backchannel &&\n+mkfifo backchannel && fd=3 &&\n svnadmin dump --deltas REPO |\n-\tsvn-fe [options] [git-svn-id-url] 3backchannel\n \n DESCRIPTION\n@@ -35,6 +35,10 @@ OPTIONS\n --ref=::\n \tRef to be written by the generated stream.\n \tDefault is refs/heads/master.\n+--read-blob-fd=::\n+\tInteger number of file descriptor from which\n+\tresponses to 'ls' and 'cat-blob' requests will come.\n+\tDefault is fd=3.\n \n INPUT FORMAT\n ------------\ndiff --git a/t/t9010-svn-fe.sh b/t/t9010-svn-fe.sh\nindex 52efabe..6dcad94 100755\n--- a/t/t9010-svn-fe.sh\n+++ b/t/t9010-svn-fe.sh\n@@ -18,20 +18,21 @@ reinit_git () {\n \n try_dump_ext () {\n \targs=$1 &&\n-\tshift &&\n+\tfd=${2:-3} &&\n+\tshift 2 &&\n \tinput=$1 &&\n \tmaybe_fail_svnfe=${2:+test_$2} &&\n \tmaybe_fail_fi=${3:+test_$3} &&\n \n \t{\n-\t\t$maybe_fail_svnfe test-svn-fe $args \"$input\" >stream 3stream $fdbackflow &&\n \twait $!\n }\n \n try_dump () {\n-\ttry_dump_ext \"\" $@\n+\ttry_dump_ext \"\" \"\" $@\n }\n \n properties () {\n@@ -1120,10 +1121,59 @@ test_expect_success SVNREPO,PIPE 't9135/svn.dump' '\n \n test_expect_success PIPE 'import to notmaster ref' '\n \treinit_git &&\n-\ttry_dump_ext \"--ref=refs/heads/notmaster\" emptyprop.dump &&\n+\ttry_dump_ext \"--ref=refs/heads/notmaster\" 3 emptyprop.dump &&\n \n \tgit rev-parse --verify notmaster &&\n \ttest_must_fail git rev-parse --verify master\n '\n \n+test_expect_success PIPE 'use different backflow fd' '\n+\treinit_git &&\n+\techo hi >hi &&\n+\t{\n+\t\tproperties \\\n+\t\t\tsvn:author author@example.com \\\n+\t\t\tsvn:date \"1999-02-01T00:01:002.000000Z\" \\\n+\t\t\tsvn:log \"add directory with some files in it\" &&\n+\t\techo PROPS-END\n+\t} >props &&\n+\t{\n+\t\techo Prop-content-length: $(wc -c revprops &&\n+\t{\n+\t\tcat <<-EOF &&\n+\t\tSVN-fs-dump-format-version: 3\n+\n+\t\tRevision-number: 1\n+\t\tEOF\n+\t\tcat revprops &&\n+\t\tcat <<-EOF &&\n+\t\tNode-path: directory\n+\t\tNode-kind: dir\n+\t\tNode-action: add\n+\t\tNode-path: directory/somefile\n+\t\tNode-kind: file\n+\t\tNode-action: add\n+\t\tEOF\n+\t\ttext_no_props hi &&\n+\n+\t\techo \"Revision-number: 2\" &&\n+\t\tcat revprops &&\n+\t\tcat <<-\\EOF\n+\t\tNode-path: otherfile\n+\t\tNode-kind: file\n+\t\tNode-action: add\n+\t\tNode-copyfrom-rev: 1\n+\t\tNode-copyfrom-path: directory/somefile\n+\t\tEOF\n+\t} >directory.dump &&\n+\ttry_dump_ext \"--read-blob-fd=7\" 7 directory.dump &&\n+\n+\tgit checkout HEAD otherfile &&\n+\ttest_cmp hi otherfile\n+'\n+\n test_done\ndiff --git a/test-svn-fe.c b/test-svn-fe.c\nindex bc437b3..43e19b2 100644\n--- a/test-svn-fe.c\n+++ b/test-svn-fe.c\n@@ -22,6 +22,8 @@ static struct option test_svnfe_options[] = {\n \tOPT_SET_INT('d', NULL, &d, \"test apply_delta\", 1),\n \tOPT_STRING(0, \"ref\", &args.ref, \"dst_ref\",\n \t\t\"write to dst_ref instead of refs/heads/master\"),\n+\tOPT_INTEGER(0, \"read-blob-fd\", &args.backflow_fd,\n+\t\t\"read blobs and trees from this fd instead of 3\"),\n \tOPT_END()\n };\n \n@@ -54,6 +56,7 @@ static int apply_delta(int argc, const char *argv[])\n int main(int argc, const char *argv[])\n {\n \targs.ref = \"refs/heads/master\";\n+\targs.backflow_fd = 3;\n \targc = parse_options(argc, argv, NULL, test_svnfe_options,\n \t\t\t\t\t\ttest_svnfe_usage, 0);\n \ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex 616f9ec..4194052 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -20,8 +20,6 @@\n */\n #define constcmp(s, ref) memcmp(s, ref, sizeof(ref) - 1)\n \n-#define REPORT_FILENO 3\n-\n #define NODEACT_REPLACE 4\n #define NODEACT_DELETE 3\n #define NODEACT_ADD 2\n@@ -484,7 +482,7 @@ int svndump_init(const struct svndump_args *args)\n {\n \tif (buffer_init(&input, args->filename))\n \t\treturn error(\"cannot open %s: %s\", args->filename, strerror(errno));\n-\tfast_export_init(REPORT_FILENO);\n+\tfast_export_init(args->backflow_fd);\n \tstrbuf_init(&dump_ctx.uuid, 4096);\n \tstrbuf_init(&dump_ctx.url, 4096);\n \tstrbuf_init(&rev_ctx.log, 4096);\ndiff --git a/vcs-svn/svndump.h b/vcs-svn/svndump.h\nindex 904e628..d266b68 100644\n--- a/vcs-svn/svndump.h\n+++ b/vcs-svn/svndump.h\n@@ -4,6 +4,7 @@\n struct svndump_args {\n \tconst char *filename, *url;\n \tconst char *ref;\n+\tint backflow_fd;\n };\n \n int svndump_init(const struct svndump_args *args);\n-- \n1.7.3.4\n"},{"id":"171242","messageId":"1310559673-5026-10-git-send-email-divanorama@gmail.com","threadId":"27803","inReplyTo":"1310559673-5026-1-git-send-email-divanorama@gmail.com","subject":"[PATCH v2 09/11] vcs-svn,svn-fe: allow to disable 'progress' lines","fromName":"Dmitry Ivankov","fromEmail":"divanorama@gmail.com","sentAt":"2011-07-13T12:21:11Z","receivedAt":"2011-07-13T12:21:11Z","isPatch":true,"sender":{"key":"divanorama@gmail.com","avatar":"https://avatars.githubusercontent.com/u/158999?v=4"},"body":"vcs-svn/ writes a progress line after each processed revision. It\nis too noisy for big imports. That's a stress for a terminal and\nany other output can be lost or scrolled away among these lines.\nIf svn-fe is invoked by a remote helper the import stream with\nprogress lines in it will go directly to the git fast-import which\nalways prints every progress line met in the stream.\n\nFor now just add a switch to turn progress lines off:\n$ svn-fe --no-progress ...\n\nSigned-off-by: Dmitry Ivankov \n---\n contrib/svn-fe/svn-fe.c | 3 +++\n contrib/svn-fe/svn-fe.txt | 3 +++\n test-svn-fe.c | 1 +\n vcs-svn/svndump.c | 6 +++++-\n vcs-svn/svndump.h | 1 +\n 5 files changed, 13 insertions(+), 1 deletions(-)\n\ndiff --git a/contrib/svn-fe/svn-fe.c b/contrib/svn-fe/svn-fe.c\nindex cd9e449..a388750 100644\n--- a/contrib/svn-fe/svn-fe.c\n+++ b/contrib/svn-fe/svn-fe.c\n@@ -15,6 +15,9 @@ static const char * const svn_fe_usage[] = {\n static struct svndump_args args;\n \n static struct option svn_fe_options[] = {\n+\t{ OPTION_BIT, 0, \"progress\", &args.progress,\n+\t\tNULL, \"don't write a progress line after each commit\",\n+\t\tPARSE_OPT_NOARG | PARSE_OPT_NEGHELP, NULL, 1 },\n \tOPT_STRING(0, \"git-svn-id-url\", &args.url, \"url\",\n \t\t\"append git-svn metadata line to commit messages\"),\n \tOPT_STRING(0, \"ref\", &args.ref, \"dst_ref\",\ndiff --git a/contrib/svn-fe/svn-fe.txt b/contrib/svn-fe/svn-fe.txt\nindex a7ad368..f1a459e 100644\n--- a/contrib/svn-fe/svn-fe.txt\n+++ b/contrib/svn-fe/svn-fe.txt\n@@ -39,6 +39,9 @@ OPTIONS\n \tInteger number of file descriptor from which\n \tresponses to 'ls' and 'cat-blob' requests will come.\n \tDefault is fd=3.\n+--[no-]progress::\n+\tWrite 'progress' lines to fast-import stream. These\n+\tcan be displayed by fast-import.\n \n INPUT FORMAT\n ------------\ndiff --git a/test-svn-fe.c b/test-svn-fe.c\nindex 43e19b2..f2711e6 100644\n--- a/test-svn-fe.c\n+++ b/test-svn-fe.c\n@@ -57,6 +57,7 @@ int main(int argc, const char *argv[])\n {\n \targs.ref = \"refs/heads/master\";\n \targs.backflow_fd = 3;\n+\targs.progress = 1;\n \targc = parse_options(argc, argv, NULL, test_svnfe_options,\n \t\t\t\t\t\ttest_svnfe_usage, 0);\n \ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex 4194052..6ad9f63 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -37,6 +37,8 @@\n \n #define MAX_GITSVN_LINE_LEN 4096\n \n+static int print_progress;\n+\n static struct line_buffer input = LINE_BUFFER_INIT;\n \n static struct {\n@@ -332,7 +334,8 @@ static void end_revision(void)\n {\n \tif (rev_ctx.revision) {\n \t\tfast_export_end_commit(rev_ctx.revision);\n-\t\tprintf(\"progress Imported commit %\"PRIu32\".\\n\\n\", rev_ctx.revision);\n+\t\tif (print_progress)\n+\t\t\tprintf(\"progress Imported commit %\"PRIu32\".\\n\\n\", rev_ctx.revision);\n \t\tdump_ctx.first_commit_done = 1;\n \t}\n }\n@@ -482,6 +485,7 @@ int svndump_init(const struct svndump_args *args)\n {\n \tif (buffer_init(&input, args->filename))\n \t\treturn error(\"cannot open %s: %s\", args->filename, strerror(errno));\n+\tprint_progress = args->progress;\n \tfast_export_init(args->backflow_fd);\n \tstrbuf_init(&dump_ctx.uuid, 4096);\n \tstrbuf_init(&dump_ctx.url, 4096);\ndiff --git a/vcs-svn/svndump.h b/vcs-svn/svndump.h\nindex d266b68..af63edb 100644\n--- a/vcs-svn/svndump.h\n+++ b/vcs-svn/svndump.h\n@@ -5,6 +5,7 @@ struct svndump_args {\n \tconst char *filename, *url;\n \tconst char *ref;\n \tint backflow_fd;\n+\tint progress;\n };\n \n int svndump_init(const struct svndump_args *args);\n-- \n1.7.3.4\n"},{"id":"171239","messageId":"1310559673-5026-11-git-send-email-divanorama@gmail.com","threadId":"27803","inReplyTo":"1310559673-5026-1-git-send-email-divanorama@gmail.com","subject":"[PATCH v2 10/11] vcs-svn,svn-fe: add --incremental option","fromName":"Dmitry Ivankov","fromEmail":"divanorama@gmail.com","sentAt":"2011-07-13T12:21:12Z","receivedAt":"2011-07-13T12:21:12Z","isPatch":true,"sender":{"key":"divanorama@gmail.com","avatar":"https://avatars.githubusercontent.com/u/158999?v=4"},"body":"This option is to make svn-fe write commits on top of the existing ref\ninstead of overwriting it. More precise, the first commit's parent is\nset to be :(first_revision_in_current_dump - 1) mark.\n\nPrerequisite is to (re)use import marks (from previous imports). It is\nsafe to use this option on a svn dump that starts with r0/r1. The svn\ndump itself should be incremental too.\n\nSigned-off-by: Dmitry Ivankov \n---\n contrib/svn-fe/svn-fe.c | 3 +++\n contrib/svn-fe/svn-fe.txt | 5 +++++\n t/t9010-svn-fe.sh | 34 +++++++++++++++++++++++++++++-----\n test-svn-fe.c | 3 +++\n vcs-svn/svndump.c | 13 +++++++++----\n vcs-svn/svndump.h | 2 +-\n 6 files changed, 50 insertions(+), 10 deletions(-)\n\ndiff --git a/contrib/svn-fe/svn-fe.c b/contrib/svn-fe/svn-fe.c\nindex a388750..211dc4d 100644\n--- a/contrib/svn-fe/svn-fe.c\n+++ b/contrib/svn-fe/svn-fe.c\n@@ -18,6 +18,9 @@ static struct option svn_fe_options[] = {\n \t{ OPTION_BIT, 0, \"progress\", &args.progress,\n \t\tNULL, \"don't write a progress line after each commit\",\n \t\tPARSE_OPT_NOARG | PARSE_OPT_NEGHELP, NULL, 1 },\n+\tOPT_BIT(0, \"incremental\", &args.incremental,\n+\t\t\"resume export, requires marks and incremental dump\",\n+\t\t1),\n \tOPT_STRING(0, \"git-svn-id-url\", &args.url, \"url\",\n \t\t\"append git-svn metadata line to commit messages\"),\n \tOPT_STRING(0, \"ref\", &args.ref, \"dst_ref\",\ndiff --git a/contrib/svn-fe/svn-fe.txt b/contrib/svn-fe/svn-fe.txt\nindex f1a459e..0b6c29e 100644\n--- a/contrib/svn-fe/svn-fe.txt\n+++ b/contrib/svn-fe/svn-fe.txt\n@@ -42,6 +42,11 @@ OPTIONS\n --[no-]progress::\n \tWrite 'progress' lines to fast-import stream. These\n \tcan be displayed by fast-import.\n+--incremental::\n+\tIf the first revision in dump has number greater than\n+\t1, make :(revision - 1) it's parent. For this to work\n+\tfast-import must be supplied with import-marks file\n+\tand the dump must be incremental.\n \n INPUT FORMAT\n ------------\ndiff --git a/t/t9010-svn-fe.sh b/t/t9010-svn-fe.sh\nindex 6dcad94..e5b78a9 100755\n--- a/t/t9010-svn-fe.sh\n+++ b/t/t9010-svn-fe.sh\n@@ -19,7 +19,8 @@ reinit_git () {\n try_dump_ext () {\n \targs=$1 &&\n \tfd=${2:-3} &&\n-\tshift 2 &&\n+\tfi_args=${3:-} &&\n+\tshift 3 &&\n \tinput=$1 &&\n \tmaybe_fail_svnfe=${2:+test_$2} &&\n \tmaybe_fail_fi=${3:+test_$3} &&\n@@ -27,12 +28,12 @@ try_dump_ext () {\n \t{\n \t\teval \"$maybe_fail_svnfe test-svn-fe $args \"$input\" >stream $fdbackflow &&\n+\teval \"$maybe_fail_fi git fast-import $fi_args --cat-blob-fd=3 backflow\" &&\n \twait $!\n }\n \n try_dump () {\n-\ttry_dump_ext \"\" \"\" $@\n+\ttry_dump_ext \"\" \"\" \"\" $@\n }\n \n properties () {\n@@ -76,6 +77,15 @@ Content-length: 10\n \n PROPS-END\n EOF\n+cat >moreempty.dump <<-EOF &&\n+SVN-fs-dump-format-version: 3\n+\n+Revision-number: 3\n+Prop-content-length: 10\n+Content-length: 10\n+\n+PROPS-END\n+EOF\n \n test_expect_success 'setup: have pipes?' '\n \trm -f frob &&\n@@ -1121,7 +1131,7 @@ test_expect_success SVNREPO,PIPE 't9135/svn.dump' '\n \n test_expect_success PIPE 'import to notmaster ref' '\n \treinit_git &&\n-\ttry_dump_ext \"--ref=refs/heads/notmaster\" 3 emptyprop.dump &&\n+\ttry_dump_ext \"--ref=refs/heads/notmaster\" 3 \"\" emptyprop.dump &&\n \n \tgit rev-parse --verify notmaster &&\n \ttest_must_fail git rev-parse --verify master\n@@ -1170,10 +1180,24 @@ test_expect_success PIPE 'use different backflow fd' '\n \t\tNode-copyfrom-path: directory/somefile\n \t\tEOF\n \t} >directory.dump &&\n-\ttry_dump_ext \"--read-blob-fd=7\" 7 directory.dump &&\n+\ttry_dump_ext \"--read-blob-fd=7\" 7 \"\" directory.dump &&\n \n \tgit checkout HEAD otherfile &&\n \ttest_cmp hi otherfile\n '\n \n+test_expect_success PIPE 'incremental import' '\n+\treinit_git &&\n+\t>./marks &&\n+\n+\ttry_dump_ext \"--incremental\" \"\" \"--export-marks=./marks\" emptyprop.dump &&\n+\ttest_line_count = 2 ./marks &&\n+\n+\ttry_dump_ext \"--incremental\" \"\" \"--import-marks=./marks --export-marks=./marks\" moreempty.dump &&\n+\ttest_line_count = 3 ./marks &&\n+\n+\tgit log --format=oneline >history &&\n+\ttest_line_count = 3 ./history\n+'\n+\n test_done\ndiff --git a/test-svn-fe.c b/test-svn-fe.c\nindex f2711e6..8d3cc99 100644\n--- a/test-svn-fe.c\n+++ b/test-svn-fe.c\n@@ -22,6 +22,9 @@ static struct option test_svnfe_options[] = {\n \tOPT_SET_INT('d', NULL, &d, \"test apply_delta\", 1),\n \tOPT_STRING(0, \"ref\", &args.ref, \"dst_ref\",\n \t\t\"write to dst_ref instead of refs/heads/master\"),\n+\tOPT_BIT(0, \"incremental\", &args.incremental,\n+\t\t\"resume export, requires marks and incremental dump\",\n+\t\t1),\n \tOPT_INTEGER(0, \"read-blob-fd\", &args.backflow_fd,\n \t\t\"read blobs and trees from this fd instead of 3\"),\n \tOPT_END()\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex 6ad9f63..2b11f96 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -58,6 +58,7 @@ static struct {\n \tstruct strbuf uuid, url;\n \tint first_commit_done;\n \tstruct strbuf ref_name;\n+\tint incremental;\n } dump_ctx;\n \n static void reset_node_ctx(char *fname)\n@@ -83,7 +84,7 @@ static void reset_rev_ctx(uint32_t revision)\n \tstrbuf_reset(&rev_ctx.author);\n }\n \n-static void reset_dump_ctx(const char *url, const char *dst_ref)\n+static void reset_dump_ctx(const char *url, const char *dst_ref, int incremental)\n {\n \tstrbuf_reset(&dump_ctx.url);\n \tif (url)\n@@ -93,6 +94,7 @@ static void reset_dump_ctx(const char *url, const char *dst_ref)\n \tdump_ctx.first_commit_done = 0;\n \tstrbuf_reset(&dump_ctx.ref_name);\n \tstrbuf_addstr(&dump_ctx.ref_name, dst_ref);\n+\tdump_ctx.incremental = incremental;\n }\n \n static void handle_property(const struct strbuf *key_buf,\n@@ -321,7 +323,10 @@ static void begin_revision(void)\n \t} else {\n \t\t*gitsvnline = 0;\n \t}\n-\tfrom_mark = dump_ctx.first_commit_done ? rev_ctx.revision - 1 : 0;\n+\tif (dump_ctx.incremental)\n+\t\tfrom_mark = rev_ctx.revision - 1;\n+\telse\n+\t\tfrom_mark = dump_ctx.first_commit_done ? rev_ctx.revision - 1 : 0;\n \tauthor = *rev_ctx.author.buf ? rev_ctx.author.buf : \"nobody\";\n \tdomain = *dump_ctx.uuid.buf ? dump_ctx.uuid.buf : \"local\";\n \n@@ -493,7 +498,7 @@ int svndump_init(const struct svndump_args *args)\n \tstrbuf_init(&rev_ctx.author, 4096);\n \tstrbuf_init(&node_ctx.src, 4096);\n \tstrbuf_init(&node_ctx.dst, 4096);\n-\treset_dump_ctx(args->url, args->ref);\n+\treset_dump_ctx(args->url, args->ref, args->incremental);\n \treset_rev_ctx(0);\n \treset_node_ctx(NULL);\n \treturn 0;\n@@ -502,7 +507,7 @@ int svndump_init(const struct svndump_args *args)\n void svndump_deinit(void)\n {\n \tfast_export_deinit();\n-\treset_dump_ctx(NULL, \"\");\n+\treset_dump_ctx(NULL, \"\", 0);\n \treset_rev_ctx(0);\n \treset_node_ctx(NULL);\n \tstrbuf_release(&rev_ctx.log);\ndiff --git a/vcs-svn/svndump.h b/vcs-svn/svndump.h\nindex af63edb..f2bb58c 100644\n--- a/vcs-svn/svndump.h\n+++ b/vcs-svn/svndump.h\n@@ -5,7 +5,7 @@ struct svndump_args {\n \tconst char *filename, *url;\n \tconst char *ref;\n \tint backflow_fd;\n-\tint progress;\n+\tint progress, incremental;\n };\n \n int svndump_init(const struct svndump_args *args);\n-- \n1.7.3.4\n"},{"id":"171244","messageId":"1310559673-5026-12-git-send-email-divanorama@gmail.com","threadId":"27803","inReplyTo":"1310559673-5026-1-git-send-email-divanorama@gmail.com","subject":"[PATCH v2 11/11] vcs-svn,svn-fe: add an option to write svnrev notes","fromName":"Dmitry Ivankov","fromEmail":"divanorama@gmail.com","sentAt":"2011-07-13T12:21:13Z","receivedAt":"2011-07-13T12:21:13Z","isPatch":true,"sender":{"key":"divanorama@gmail.com","avatar":"https://avatars.githubusercontent.com/u/158999?v=4"},"body":"There are already a few options to determine svn revision from which\na git commit imported with svn-fe came from. One is to make svn-fe\nwrite a git-svn-id: line to commit messages. Another one is to calc\ndistance to the root commit. The former includes a \"url\" and is for\ngit-svn compatibility, the latter is obviously slow and a bit fragile.\n\n$ svn-fe --notes_ref=notes_tree --ref=branch...\nwill write annotations for branch commits to the notes_tree, each\nannotation is a simple \"rN\" string. Then these annotations can be\nviewed manually or used in incremental import to detect the last\nimported revision or to (re)create the import marks for further\nimports.\n\nSigned-off-by: Dmitry Ivankov \n---\n contrib/svn-fe/svn-fe.c | 2 ++\n contrib/svn-fe/svn-fe.txt | 3 +++\n t/t9010-svn-fe.sh | 32 ++++++++++++++++++++++++++++++++\n test-svn-fe.c | 2 ++\n vcs-svn/svndump.c | 28 ++++++++++++++++++++++++----\n vcs-svn/svndump.h | 2 +-\n 6 files changed, 64 insertions(+), 5 deletions(-)\n\ndiff --git a/contrib/svn-fe/svn-fe.c b/contrib/svn-fe/svn-fe.c\nindex 211dc4d..8410221 100644\n--- a/contrib/svn-fe/svn-fe.c\n+++ b/contrib/svn-fe/svn-fe.c\n@@ -25,6 +25,8 @@ static struct option svn_fe_options[] = {\n \t\t\"append git-svn metadata line to commit messages\"),\n \tOPT_STRING(0, \"ref\", &args.ref, \"dst_ref\",\n \t\t\"write to dst_ref instead of refs/heads/master\"),\n+\tOPT_STRING(0, \"notes-ref\", &args.notes_ref, \"notes\",\n+\t\t\"write \\\"rN\\\" notes to the tree\"),\n \tOPT_INTEGER(0, \"read-blob-fd\", &args.backflow_fd,\n \t\t\"read blobs and trees from this fd instead of 3\"),\n \tOPT_END()\ndiff --git a/contrib/svn-fe/svn-fe.txt b/contrib/svn-fe/svn-fe.txt\nindex 0b6c29e..ce0582d 100644\n--- a/contrib/svn-fe/svn-fe.txt\n+++ b/contrib/svn-fe/svn-fe.txt\n@@ -35,6 +35,9 @@ OPTIONS\n --ref=::\n \tRef to be written by the generated stream.\n \tDefault is refs/heads/master.\n+--notes-ref=::\n+\tWrite \"rN\" notes to the notes_ref tree for each\n+\timported commit.\n --read-blob-fd=::\n \tInteger number of file descriptor from which\n \tresponses to 'ls' and 'cat-blob' requests will come.\ndiff --git a/t/t9010-svn-fe.sh b/t/t9010-svn-fe.sh\nindex e5b78a9..2841a3e 100755\n--- a/t/t9010-svn-fe.sh\n+++ b/t/t9010-svn-fe.sh\n@@ -1200,4 +1200,36 @@ test_expect_success PIPE 'incremental import' '\n \ttest_line_count = 3 ./history\n '\n \n+test_expect_success PIPE 'write notes' '\n+\treinit_git &&\n+\tcat >expect <<-EOF &&\n+\tr2\n+\n+\tr1\n+\n+\tEOF\n+\ttry_dump_ext \"--notes=refs/notes/test\" \"\" \"\" emptyprop.dump &&\n+\tgit log --show-notes=refs/notes/test --format=%N >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success PIPE 'write notes incremental' '\n+\treinit_git &&\n+\t>./marks &&\n+\tcat >expect <<-EOF &&\n+\tr3\n+\n+\tr2\n+\n+\tr1\n+\n+\tEOF\n+\n+\ttry_dump_ext \"--notes=refs/notes/test\" \"\" \"--export-marks=marks\" emptyprop.dump &&\n+\ttry_dump_ext \"--incremental --notes=refs/notes/test\" \"\" \"--import-marks=marks\" moreempty.dump &&\n+\n+\tgit log --show-notes=refs/notes/test --format=%N >actual &&\n+\ttest_cmp expect actual\n+'\n+\n test_done\ndiff --git a/test-svn-fe.c b/test-svn-fe.c\nindex 8d3cc99..e6d9ae6 100644\n--- a/test-svn-fe.c\n+++ b/test-svn-fe.c\n@@ -27,6 +27,8 @@ static struct option test_svnfe_options[] = {\n \t\t1),\n \tOPT_INTEGER(0, \"read-blob-fd\", &args.backflow_fd,\n \t\t\"read blobs and trees from this fd instead of 3\"),\n+\tOPT_STRING(0, \"notes-ref\", &args.notes_ref, \"notes\",\n+\t\t\"write \\\"rN\\\" notes to the tree\"),\n \tOPT_END()\n };\n \ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex 2b11f96..514703f 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -57,7 +57,7 @@ static struct {\n \tuint32_t version;\n \tstruct strbuf uuid, url;\n \tint first_commit_done;\n-\tstruct strbuf ref_name;\n+\tstruct strbuf ref_name, notes_ref_name;\n \tint incremental;\n } dump_ctx;\n \n@@ -84,7 +84,7 @@ static void reset_rev_ctx(uint32_t revision)\n \tstrbuf_reset(&rev_ctx.author);\n }\n \n-static void reset_dump_ctx(const char *url, const char *dst_ref, int incremental)\n+static void reset_dump_ctx(const char *url, const char *dst_ref, int incremental, const char *dst_notes_ref)\n {\n \tstrbuf_reset(&dump_ctx.url);\n \tif (url)\n@@ -95,6 +95,9 @@ static void reset_dump_ctx(const char *url, const char *dst_ref, int incremental\n \tstrbuf_reset(&dump_ctx.ref_name);\n \tstrbuf_addstr(&dump_ctx.ref_name, dst_ref);\n \tdump_ctx.incremental = incremental;\n+\tstrbuf_reset(&dump_ctx.notes_ref_name);\n+\tif (dst_notes_ref)\n+\t\tstrbuf_addstr(&dump_ctx.notes_ref_name, dst_notes_ref);\n }\n \n static void handle_property(const struct strbuf *key_buf,\n@@ -337,8 +340,25 @@ static void begin_revision(void)\n \n static void end_revision(void)\n {\n+\tchar buf[32];\n+\tchar tmbuf[32];\n \tif (rev_ctx.revision) {\n \t\tfast_export_end_commit(rev_ctx.revision);\n+\t\tif (dump_ctx.notes_ref_name.len) {\n+\t\t\tdatestamp(tmbuf, 32);\n+\t\t\tprintf(\"commit %s\\n\", dump_ctx.notes_ref_name.buf);\n+\t\t\tprintf(\"committer %s <%s@%s> %s\\n\",\n+\t\t\t\t\t\"vcs-svn\", \"vcs-svn\", \"local\", tmbuf);\n+\t\t\tprintf(\"data < 1)\n+\t\t\t\tprintf(\"from %s^0\\n\", dump_ctx.notes_ref_name.buf);\n+\t\t\tsnprintf(buf, 32, \"r%d\", rev_ctx.revision);\n+\t\t\tprintf(\"N inline :%d\\n\", rev_ctx.revision);\n+\t\t\tprintf(\"data %ld\\n\", strlen(buf));\n+\t\t\tprintf(\"%s\\n\", buf);\n+\t\t}\n \t\tif (print_progress)\n \t\t\tprintf(\"progress Imported commit %\"PRIu32\".\\n\\n\", rev_ctx.revision);\n \t\tdump_ctx.first_commit_done = 1;\n@@ -498,7 +518,7 @@ int svndump_init(const struct svndump_args *args)\n \tstrbuf_init(&rev_ctx.author, 4096);\n \tstrbuf_init(&node_ctx.src, 4096);\n \tstrbuf_init(&node_ctx.dst, 4096);\n-\treset_dump_ctx(args->url, args->ref, args->incremental);\n+\treset_dump_ctx(args->url, args->ref, args->incremental, args->notes_ref);\n \treset_rev_ctx(0);\n \treset_node_ctx(NULL);\n \treturn 0;\n@@ -507,7 +527,7 @@ int svndump_init(const struct svndump_args *args)\n void svndump_deinit(void)\n {\n \tfast_export_deinit();\n-\treset_dump_ctx(NULL, \"\", 0);\n+\treset_dump_ctx(NULL, \"\", 0, \"\");\n \treset_rev_ctx(0);\n \treset_node_ctx(NULL);\n \tstrbuf_release(&rev_ctx.log);\ndiff --git a/vcs-svn/svndump.h b/vcs-svn/svndump.h\nindex f2bb58c..928bb0b 100644\n--- a/vcs-svn/svndump.h\n+++ b/vcs-svn/svndump.h\n@@ -3,7 +3,7 @@\n \n struct svndump_args {\n \tconst char *filename, *url;\n-\tconst char *ref;\n+\tconst char *ref, *notes_ref;\n \tint backflow_fd;\n \tint progress, incremental;\n };\n-- \n1.7.3.4\n"},{"id":"171938","messageId":"20110724125200.GA2934@elie.dc0b.debconf.org","threadId":"27803","inReplyTo":"1310559673-5026-2-git-send-email-divanorama@gmail.com","subject":"Re: [PATCH v2 01/11] svn-fe: add man target to Makefile","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-07-24T12:52:00Z","receivedAt":"2011-07-24T12:52:00Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi Dmitry,\n\nDmitry Ivankov wrote:\n\n> There already is a svn-fe.1 target. But 'man' being a standard\n> target is easier to discover or type. It can also be reused if\n> more manpages arise here.\n\nThis will make life a little easier when wanting to test-build\ngit-remote-svn.1 along with svn-fe.1 (once git-remote-svn exists).\nThanks, queued.\n"},{"id":"171940","messageId":"20110724130443.GB2934@elie.dc0b.debconf.org","threadId":"27803","inReplyTo":"1310559673-5026-3-git-send-email-divanorama@gmail.com","subject":"Re: [PATCH v2 02/11] test-svn-fe: use parse-options","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-07-24T13:04:43Z","receivedAt":"2011-07-24T13:04:43Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nSorry for the slow response.\n\nDmitry Ivankov wrote:\n\n> There was custom options parsing. As more options arise it will\n> be easier to add and document new options with parse-options api.\n\nIn particular this gives us a \"test-svn-fe -h\" command --- sounds\ngood. Might make sense to combine this with the patch that\nparsoptifies contrib/svn-fe/svn-fe.c.\n\n> --- a/test-svn-fe.c\n> +++ b/test-svn-fe.c\n> @@ -3,28 +3,38 @@\n> */\n> \n> #include \"git-compat-util.h\"\n> +#include \"parse-options.h\"\n> #include \"vcs-svn/svndump.h\"\n> #include \"vcs-svn/svndiff.h\"\n> #include \"vcs-svn/sliding_window.h\"\n> #include \"vcs-svn/line_buffer.h\"\n> \n> -static const char test_svnfe_usage[] =\n> -\t\"test-svn-fe ( | [-d] )\";\n> +static const char * const test_svnfe_usage[] = {\n> +\t\"test-svn-fe ( | -d )\",\n> +\tNULL\n> +};\n\nWith this API, we're allowed to print multiple usage strings. Might as\nwell take advantage of that for clarity:\n\n static const char * const test_svnfe_usage[] = {\n\t\"test-svn-fe \",\n\t\"test-svn-fe -d \",\n\tNULL\n };\n\n> \n> +static int d;\n> +\n\nThe variable name is not so memorable. Maybe something like\n\"apply_delta\" would do.\n\n> -static int apply_delta(int argc, char *argv[])\n> +static struct option test_svnfe_options[] = {\n> +\tOPT_SET_INT('d', NULL, &d, \"test apply_delta\", 1),\n> +\tOPT_END()\n> +};\n\nMight make sense to take the opportunity to add a mnemonic long\noption name while at it:\n\n\tOPT_SET_INT('d', \"apply-delta\", ...\n\n[...]\n> @@ -37,10 +47,16 @@ static int apply_delta(int argc, char *argv[])\n> \treturn 0;\n> }\n> \n> -int main(int argc, char *argv[])\n> +int main(int argc, const char *argv[])\n> {\n> -\tif (argc == 2) {\n> -\t\tif (svndump_init(argv[1]))\n> +\targc = parse_options(argc, argv, NULL, test_svnfe_options,\n> +\t\t\t\t\t\ttest_svnfe_usage, 0);\n> +\n> +\tif (d)\n> +\t\treturn apply_delta(argc, argv);\n> +\n> +\tif (argc == 1) {\n\nProbably easier to read with the simple and exceptional case first.\n\n\tif (apply_delta_instead)\n\t\treturn apply_delta(argc, argv);\n\tif (argc != 1)\n\t\tusage_with_options(...);\n\n\tif (svndump_init(argv[0]))\n\t\treturn 1;\n\t...\n\n\t\n> +\t\tif (svndump_init(argv[0]))\n> \t\t\treturn 1;\n> \t\tsvndump_read(NULL);\n> \t\tsvndump_deinit();\n> @@ -48,7 +64,5 @@ int main(int argc, char *argv[])\n> \t\treturn 0;\n> \t}\n> \n> -\tif (argc >= 2 && !strcmp(argv[1], \"-d\"))\n> -\t\treturn apply_delta(argc, argv);\n> -\tusage(test_svnfe_usage);\n> +\tusage_with_options(test_svnfe_usage, test_svnfe_options);\n\nExcept for the minor nits noted above (in particular, hopefully this\ncan be squashed with the corresponding svn-fe patch),\n\nReviewed-by: Jonathan Nieder \n---\n test-svn-fe.c | 29 +++++++++++++++--------------\n 1 files changed, 15 insertions(+), 14 deletions(-)\n\ndiff --git a/test-svn-fe.c b/test-svn-fe.c\nindex 0aab2450..db56b6ba 100644\n--- a/test-svn-fe.c\n+++ b/test-svn-fe.c\n@@ -3,21 +3,23 @@\n */\n \n #include \"git-compat-util.h\"\n-#include \"parse-options.h\"\n #include \"vcs-svn/svndump.h\"\n #include \"vcs-svn/svndiff.h\"\n #include \"vcs-svn/sliding_window.h\"\n #include \"vcs-svn/line_buffer.h\"\n+#include \"parse-options.h\"\n \n static const char * const test_svnfe_usage[] = {\n-\t\"test-svn-fe ( | -d )\",\n+\t\"test-svn-fe \",\n+\t\"test-svn-fe -d \",\n \tNULL\n };\n \n-static int d;\n+static int apply_delta_instead;\n \n static struct option test_svnfe_options[] = {\n-\tOPT_SET_INT('d', NULL, &d, \"test apply_delta\", 1),\n+\tOPT_SET_INT('d', \"apply-delta\",\n+\t\t&apply_delta_instead, \"apply a subversion-format delta\", 1),\n \tOPT_END()\n };\n \n@@ -52,17 +54,16 @@ int main(int argc, const char *argv[])\n \targc = parse_options(argc, argv, NULL, test_svnfe_options,\n \t\t\t\t\t\ttest_svnfe_usage, 0);\n \n-\tif (d)\n+\tif (apply_delta_instead)\n \t\treturn apply_delta(argc, argv);\n \n-\tif (argc == 1) {\n-\t\tif (svndump_init(argv[0]))\n-\t\t\treturn 1;\n-\t\tsvndump_read(NULL);\n-\t\tsvndump_deinit();\n-\t\tsvndump_reset();\n-\t\treturn 0;\n-\t}\n+\tif (argc != 1)\n+\t\tusage_with_options(test_svnfe_usage, test_svnfe_options);\n \n-\tusage_with_options(test_svnfe_usage, test_svnfe_options);\n+\tif (svndump_init(argv[0]))\n+\t\treturn 1;\n+\tsvndump_read(NULL);\n+\tsvndump_deinit();\n+\tsvndump_reset();\n+\treturn 0;\n }\n-- \n1.7.6\n"},{"id":"171941","messageId":"20110724131439.GC2934@elie.dc0b.debconf.org","threadId":"27803","inReplyTo":"1310559673-5026-4-git-send-email-divanorama@gmail.com","subject":"Re: [PATCH v2 03/11] svn-fe: add EXTLIBS needed for parse-options","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-07-24T13:14:39Z","receivedAt":"2011-07-24T13:14:39Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Dmitry Ivankov wrote:\n\n> Currently parse-options.o pull quite a big bunch of dependencies\n> that are neither pulled in by svn-fe Makefile nor included in libgit.a.\n>\n> Use a temporary hack: put hardcoded EXTLIBS\n\nIn other words, this is a workaround for the lack of\nhttp://thread.gmane.org/gmane.comp.version-control.git/176318/focus=176573\n(Reduce parse-options.o dependencies).\n\n> this may not work in all\n> setups because /Makefile logic is not repeated.\n>\n> For example, one may need -lcrypto instead of -lssl or no crypto library\n> if BLK_SHA1 is set, also an additional -lz or -lpcre could be required.\n\nBetter to pull in too many libs and let the operator remove them from\nthe Makefile than too few and make her guess. Though of course\nneither should be needed. :)\n\nWith the following applied on top locally, it works for me.\n\n-- >8 --\nSubject: squash! svn-fe: add EXTLIBS needed for parse-options\n\n-lcrypto is needed for SHA-1 routines unless NO_OPENSSL or BLK_SHA1\nis set, -lpcre is for grep if USE_LIBPCRE is set, and -lz is needed\nthroughout.\n\nIn the future, none of these should be needed, after a little\nrearranging to ensure that parse-options.o has no references to\ntranslation units that need to access the object db.\n\nSigned-off-by: Jonathan Nieder \n---\n contrib/svn-fe/Makefile | 2 +-\n 1 files changed, 1 insertions(+), 1 deletions(-)\n\ndiff --git a/contrib/svn-fe/Makefile b/contrib/svn-fe/Makefile\nindex dc6dafef..14b07b5b 100644\n--- a/contrib/svn-fe/Makefile\n+++ b/contrib/svn-fe/Makefile\n@@ -8,7 +8,7 @@ CFLAGS = -g -O2 -Wall\n LDFLAGS =\n ALL_CFLAGS = $(CFLAGS)\n ALL_LDFLAGS = $(LDFLAGS)\n-EXTLIBS = -lssl -lpthread\n+EXTLIBS = -lssl -lcrypto -lpcre -lz -lpthread\n \n GIT_LIB = ../../libgit.a\n VCSSVN_LIB = ../../vcs-svn/lib.a\n-- \n1.7.6\n"},{"id":"171942","messageId":"20110724132551.GD2934@elie.dc0b.debconf.org","threadId":"27803","inReplyTo":"1310559673-5026-5-git-send-email-divanorama@gmail.com","subject":"Re: [PATCH v2 04/11] svn-fe: add usage and unpositional arguments versions","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-07-24T13:25:51Z","receivedAt":"2011-07-24T13:25:51Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Dmitry Ivankov wrote:\n\n> $ svn-fe --git-svn-id-url=url\n> does the same thing as\n> $ svn-fe url\n> i.e., url is used to generate git-svn-id: lines, if url is set.\n\nReasonable. Let's see...\n\n> --- a/contrib/svn-fe/Makefile\n> +++ b/contrib/svn-fe/Makefile\n> @@ -41,7 +41,7 @@ svn-fe$X: svn-fe.o $(VCSSVN_LIB) $(GIT_LIB)\n> \t\t$(ALL_LDFLAGS) $(LIBS)\n> \n> svn-fe.o: svn-fe.c ../../vcs-svn/svndump.h\n> -\t$(QUIET_CC)$(CC) -I../../vcs-svn -o $*.o -c $(ALL_CFLAGS) $<\n> +\t$(QUIET_CC)$(CC) -I../../vcs-svn -I../.. -o $*.o -c $(ALL_CFLAGS) $<\n\nI'd prefer to see the -I flags as part of ALL_CFLAGS, though that's\nsomewhat orthogonal to this patch.\n\n[...]\n> --- a/contrib/svn-fe/svn-fe.c\n> +++ b/contrib/svn-fe/svn-fe.c\n> @@ -3,14 +3,38 @@\n[...]\n> +static struct option svn_fe_options[] = {\n> +\tOPT_STRING(0, \"git-svn-id-url\", &url, \"url\",\n> +\t\t\"append git-svn metadata line to commit messages\"),\n\nHmm. How about this?\n\n\t\"add git-svn-id line to log messages, imitating git-svn\"\n\n[...]\n> +\targc = parse_options(argc, argv, NULL, svn_fe_options,\n> +\t\t\t\t\t\tsvn_fe_usage, 0);\n> +\tif (argc == 1) {\n> +\t\tif (url)\n> +\t\t\tusage_msg_opt(\"git-svn-id-url is set twice: as a \"\n> +\t\t\t\t\t\"--parameter and as a [parameter]\",\n> +\t\t\t\t\tsvn_fe_usage, svn_fe_options);\n> +\t\turl = argv[0];\n> +\t} else if (argc)\n> +\t\tusage_with_options(svn_fe_usage, svn_fe_options);\n\nIMHO would be more readable with the simplest and exceptional case\nfirst:\n\n\tif (argc > 1)\n\t\tusage_with_options(...);\n\nThis way, a person reading can be comforted with the knowledge that\nargc <= 1 from then on.\n\n\tif (argc == 1) {\n\t\tif (url)\n\t\t\t...\n\t}\n\nTo sum up, the patch looks good, and the only improvements I can think\nof are tiny nits. :)\n\nWith whatever changes mentioned above seem suitable,\nReviewed-by: Jonathan Nieder \n\nThanks.\n---\n contrib/svn-fe/Makefile | 4 ++--\n contrib/svn-fe/svn-fe.c | 8 +++++---\n 2 files changed, 7 insertions(+), 5 deletions(-)\n\ndiff --git a/contrib/svn-fe/Makefile b/contrib/svn-fe/Makefile\nindex 14b07b5b..62a5c628 100644\n--- a/contrib/svn-fe/Makefile\n+++ b/contrib/svn-fe/Makefile\n@@ -6,7 +6,7 @@ MV = mv\n \n CFLAGS = -g -O2 -Wall\n LDFLAGS =\n-ALL_CFLAGS = $(CFLAGS)\n+ALL_CFLAGS = -I../.. -I../../vcs-svn $(CFLAGS)\n ALL_LDFLAGS = $(LDFLAGS)\n EXTLIBS = -lssl -lcrypto -lpcre -lz -lpthread\n \n@@ -39,7 +39,7 @@ svn-fe$X: svn-fe.o $(VCSSVN_LIB) $(GIT_LIB)\n \t\t$(ALL_LDFLAGS) $(LIBS)\n \n svn-fe.o: svn-fe.c ../../vcs-svn/svndump.h\n-\t$(QUIET_CC)$(CC) -I../../vcs-svn -I../.. -o $*.o -c $(ALL_CFLAGS) $<\n+\t$(QUIET_CC)$(CC) -o $@ -c $(ALL_CFLAGS) $<\n \n svn-fe.html: svn-fe.txt\n \t$(QUIET_SUBDIR0)../../Documentation $(QUIET_SUBDIR1) \\\ndiff --git a/contrib/svn-fe/svn-fe.c b/contrib/svn-fe/svn-fe.c\nindex 7e829b91..a95e72f4 100644\n--- a/contrib/svn-fe/svn-fe.c\n+++ b/contrib/svn-fe/svn-fe.c\n@@ -16,7 +16,7 @@ static const char *url;\n \n static struct option svn_fe_options[] = {\n \tOPT_STRING(0, \"git-svn-id-url\", &url, \"url\",\n-\t\t\"append git-svn metadata line to commit messages\"),\n+\t\t\"add git-svn-id line to log messages, imitating git-svn\"),\n \tOPT_END()\n };\n \n@@ -24,14 +24,16 @@ int main(int argc, const char **argv)\n {\n \targc = parse_options(argc, argv, NULL, svn_fe_options,\n \t\t\t\t\t\tsvn_fe_usage, 0);\n+\tif (argc > 1)\n+\t\tusage_with_options(svn_fe_usage, svn_fe_options);\n+\n \tif (argc == 1) {\n \t\tif (url)\n \t\t\tusage_msg_opt(\"git-svn-id-url is set twice: as a \"\n \t\t\t\t\t\"--parameter and as a [parameter]\",\n \t\t\t\t\tsvn_fe_usage, svn_fe_options);\n \t\turl = argv[0];\n-\t} else if (argc)\n-\t\tusage_with_options(svn_fe_usage, svn_fe_options);\n+\t}\n \tif (svndump_init(NULL))\n \t\treturn 1;\n \tsvndump_read(url);\n-- \n1.7.6\n"},{"id":"171944","messageId":"20110724134009.GE2934@elie.dc0b.debconf.org","threadId":"27803","inReplyTo":"1310559673-5026-6-git-send-email-divanorama@gmail.com","subject":"Re: [PATCH v2 05/11] vcs-svn: move url parameter from _read to _init","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-07-24T13:40:09Z","receivedAt":"2011-07-24T13:40:09Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Dmitry Ivankov wrote:\n\n> svndump_read takes a url parameter that is used in git-svn-id: lines\n[...]\n> Move url parameter to svndump_init so that reset_dump_ctx is done\n> once per dump and in the same place as other resets. Wrap all _init\n[...]\n\nI'm getting lost seeing the forest for the trees, so let me try to\nsummarize.\n\nBefore:\n\n\tif (svndump_init(url))\n\t\tdie(\"svndump_init failed\");\n\tsvndump_read(dumpfile);\n\nAfter:\n\n\tstruct svndump_args opts;\n\n\tmemset(&opts, 0, sizeof(opts));\n\topts.url = url;\n\topts.filename = dumpfile;\n\n\tif (svndump_init(&opts))\n\t\tdie(\"svndump_init failed\");\n\tsvndump_read();\n\nUsing an options struct instead of a list of arguments means each\noption is optional and has a descriptive name mentioned at the call\nsite, and means it is easy to add new arguments in the future.\n\nThe patch still keeps the init/read distinction even though we don't\nneed it anywhere (i.e., all call sites look the same) to minimize its\ninvasiveness.\n\nDo I understand correctly?\n\nIf so, it sounds like a good idea, and I have only minor nitpicks:\n\n - It's tempting to call the struct svndump_options, by analogy\n with struct merge_options from merge-recursive.h.\n\n - Now that we're making the name of the \"url\" argument part of the\n public API, maybe we should emphasize that the url is only for\n show and git will never try to contact it. Maybe something like\n \"metadata_url\"? (Sorry, I'm not so great at coming up with names.)\n\n - Likewise, the \"filename\" argument could be made more\n self-explanatory. Maybe \"dumpfile\"?\n\n - Now that the filename argument is passed at init time instead of\n read time, there is some uncertainty about when the file is going\n to be opened. A comment could help, or merging the two functions\n could help. :)\n\nThanks, and hope that helps.\nJonathan\n---\n contrib/svn-fe/svn-fe.c | 4 ++--\n test-svn-fe.c | 4 ++--\n vcs-svn/svndump.c | 10 ++++++----\n vcs-svn/svndump.h | 7 ++++---\n 4 files changed, 14 insertions(+), 11 deletions(-)\n\ndiff --git a/contrib/svn-fe/svn-fe.c b/contrib/svn-fe/svn-fe.c\nindex 22d4cc14..88fca6a3 100644\n--- a/contrib/svn-fe/svn-fe.c\n+++ b/contrib/svn-fe/svn-fe.c\n@@ -9,10 +9,10 @@\n \n int main(int argc, char **argv)\n {\n-\tstruct svndump_args args;\n+\tstruct svndump_options args;\n \n \tmemset(&args, 0, sizeof(args));\n-\targs.url = argc > 1 ? argv[1] : NULL;\n+\targs.metadata_url = argc > 1 ? argv[1] : NULL;\n \tif (svndump_init(&args))\n \t\treturn 1;\n \tsvndump_read();\ndiff --git a/test-svn-fe.c b/test-svn-fe.c\nindex d80734fd..12a5c3ab 100644\n--- a/test-svn-fe.c\n+++ b/test-svn-fe.c\n@@ -39,11 +39,11 @@ static int apply_delta(int argc, char *argv[])\n \n int main(int argc, char *argv[])\n {\n-\tstruct svndump_args args;\n+\tstruct svndump_options args;\n \n \tmemset(&args, 0, sizeof(args));\n \tif (argc == 2) {\n-\t\targs.filename = argv[1];\n+\t\targs.dumpfile = argv[1];\n \t\tif (svndump_init(&args))\n \t\t\treturn 1;\n \t\tsvndump_read();\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex 60cccadc..805c94b6 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -455,10 +455,12 @@ void svndump_read(void)\n \t\tend_revision();\n }\n \n-int svndump_init(const struct svndump_args *args)\n+int svndump_init(const struct svndump_options *opts)\n {\n-\tif (buffer_init(&input, args->filename))\n-\t\treturn error(\"cannot open %s: %s\", args->filename, strerror(errno));\n+\tconst char *filename = opts->dumpfile;\n+\n+\tif (buffer_init(&input, filename))\n+\t\treturn error(\"cannot open %s: %s\", filename, strerror(errno));\n \tfast_export_init(REPORT_FILENO);\n \tstrbuf_init(&dump_ctx.uuid, 4096);\n \tstrbuf_init(&dump_ctx.url, 4096);\n@@ -466,7 +468,7 @@ int svndump_init(const struct svndump_args *args)\n \tstrbuf_init(&rev_ctx.author, 4096);\n \tstrbuf_init(&node_ctx.src, 4096);\n \tstrbuf_init(&node_ctx.dst, 4096);\n-\treset_dump_ctx(args->url);\n+\treset_dump_ctx(opts->metadata_url);\n \treset_rev_ctx(0);\n \treset_node_ctx(NULL);\n \treturn 0;\ndiff --git a/vcs-svn/svndump.h b/vcs-svn/svndump.h\nindex b3dbf24e..b8f52172 100644\n--- a/vcs-svn/svndump.h\n+++ b/vcs-svn/svndump.h\n@@ -1,11 +1,12 @@\n #ifndef SVNDUMP_H_\n #define SVNDUMP_H_\n \n-struct svndump_args {\n-\tconst char *filename, *url;\n+struct svndump_options {\n+\tconst char *dumpfile;\n+\tconst char *metadata_url;\n };\n \n-int svndump_init(const struct svndump_args *args);\n+int svndump_init(const struct svndump_options *o);\n void svndump_read(void);\n void svndump_deinit(void);\n void svndump_reset(void);\n-- \n1.7.6\n"},{"id":"171946","messageId":"20110724135811.GG2934@elie.dc0b.debconf.org","threadId":"27803","inReplyTo":"1310559673-5026-7-git-send-email-divanorama@gmail.com","subject":"Re: [PATCH v2 06/11] vcs-svn: move commit parameters logic to svndump.c","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-07-24T13:58:11Z","receivedAt":"2011-07-24T13:58:11Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Dmitry Ivankov wrote:\n\n> fast_export.c had logic to set up commit ref, author name, email,\n> parent commit, import mark and git-svn-id: line based on both it's\n> own state (current import batch history) and the arguments passed.\n>\n> Lift the decision on these parameters to the caller.\n\nAgain I find myself getting lost. I think this is another internal\nAPI change, with the intent being to make the fast_export lib more\nintuitive by making it focus on communicating with fast-import and the\ndelta applier instead of taking care of so much svn-fe-specific logic.\nIn other words, the idea would be to avoid a few layering violations.\nIs that right?\n\nIf so:\n\n> --- a/vcs-svn/fast_export.c\n> +++ b/vcs-svn/fast_export.c\n> @@ -13,9 +13,6 @@\n[...]\n> -static uint32_t first_commit_done;\n\nThis is a good change.\n\n[...]\n> @@ -73,42 +69,30 @@ void fast_export_modify(const char *path, uint32_t mode, const char *dataref)\n> \tputchar('\\n');\n> }\n> \n> -static char gitsvnline[MAX_GITSVN_LINE_LEN];\n> -void fast_export_begin_commit(uint32_t revision, const char *author,\n> -\t\t\tconst struct strbuf *log,\n> -\t\t\tconst char *uuid, const char *url,\n> -\t\t\tunsigned long timestamp)\n> +void fast_export_begin_commit(uint32_t set_mark, const char *committer_name,\n> +\t\t\tconst char *committer_login, const char *committer_domain,\n> +\t\t\tconst struct strbuf *log, const char *gitsvnline,\n> +\t\t\tunsigned long timestamp, uint32_t from_mark,\n> +\t\t\tconst char *dst_ref)\n\nThe argument list is getting scary.\n\n[...]\n> void fast_export_end_commit(uint32_t revision)\n> {\n> -\tprintf(\"progress Imported commit %\"PRIu32\".\\n\\n\", revision);\n> }\n\nThis change leaves fast_export_end_commit empty. Why not remove\nit? (Later patches that want to insert code there could reintroduce\nthe function.)\n\n[...]\n> --- a/vcs-svn/svndump.c\n> +++ b/vcs-svn/svndump.c\n> @@ -37,6 +37,8 @@\n> #define LENGTH_UNKNOWN (~0)\n> #define DATE_RFC2822_LEN 31\n> \n> +#define MAX_GITSVN_LINE_LEN 4096\n> +\n> static struct line_buffer input = LINE_BUFFER_INIT;\n> \n> static struct {\n> @@ -54,6 +56,7 @@ static struct {\n> static struct {\n> \tuint32_t version;\n> \tstruct strbuf uuid, url;\n> +\tint first_commit_done;\n\nSneaky: changing the type from uint32_t to int. Good. :)\n\n[...]\n> @@ -299,19 +303,37 @@ static void handle_node(void)\n[...]\n> +\tfast_export_begin_commit(rev_ctx.revision, author, author, domain,\n> +\t\t&rev_ctx.log, gitsvnline, rev_ctx.timestamp,\n> +\t\tfrom_mark);\n\nThis doesn't compile for me (missing \"ref\" argument).\n\n> }\n> \n> static void end_revision(void)\n> {\n> -\tif (rev_ctx.revision)\n> +\tif (rev_ctx.revision) {\n> \t\tfast_export_end_commit(rev_ctx.revision);\n> +\t\tprintf(\"progress Imported commit %\"PRIu32\".\\n\\n\", rev_ctx.revision);\n\nUntil now, svndump.c did not have to know about the fast-import\nformat (e.g., the existence of a \"progress\" command). Is that\nworth changing?\n\nQuick sketch with suggestions. What do you think?\n\n-- >8 --\nSubject: squash! vcs-svn: move commit parameters logic to svndump.c\n\nThe previous commit doesn't build because we forgot to pass the new\nref name argument to the fast_export API. While fixing that, simplify\nfast_export_begin_commit to be more intuitive by using a set of\nparameters closer to what gets written to fast-import.\n\nThe actual impact of this patch would be to run a little slower, since\nwe needlessly copy the author name into a temporary buffer for an\nemail address. That is a small per-commit rather than per-path cost\nso the loss in speed might be worth the gain in readability.\n\nSigned-off-by: Jonathan Nieder \n---\n vcs-svn/fast_export.c | 35 ++++++++++++++++-------------------\n vcs-svn/fast_export.h | 10 ++++------\n vcs-svn/svndump.c | 41 ++++++++++++++++++++++++-----------------\n 3 files changed, 44 insertions(+), 42 deletions(-)\n\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex 04001b83..f61113b4 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -69,30 +69,27 @@ void fast_export_modify(const char *path, uint32_t mode, const char *dataref)\n \tputchar('\\n');\n }\n \n-void fast_export_begin_commit(uint32_t set_mark, const char *committer_name,\n-\t\t\tconst char *committer_login, const char *committer_domain,\n-\t\t\tconst struct strbuf *log, const char *gitsvnline,\n-\t\t\tunsigned long timestamp, uint32_t from_mark,\n-\t\t\tconst char *dst_ref)\n+void fast_export_begin_commit(const char *ref, uint32_t mark, uint32_t prev_mark,\n+\t\t\tconst char *author_name, const char *author_email,\n+\t\t\tconst struct strbuf *log, unsigned long timestamp)\n {\n-\tif (!gitsvnline)\n-\t\tgitsvnline = \"\";\n-\tprintf(\"commit %s\\n\", dst_ref);\n-\tif (set_mark)\n-\t\tprintf(\"mark :%\"PRIu32\"\\n\", set_mark);\n-\tprintf(\"committer %s <%s@%s> %ld +0000\\n\",\n-\t\tcommitter_name, committer_login, committer_domain,\n-\t\ttimestamp);\n-\tprintf(\"data %\"PRIuMAX\"\\n\",\n-\t\t(uintmax_t) (log->len + strlen(gitsvnline)));\n+\tif (!ref)\n+\t\tref = \"refs/heads/master\";\n+\tprintf(\"commit %s\\n\", ref);\n+\tif (mark)\n+\t\tprintf(\"mark :%\"PRIu32\"\\n\", mark);\n+\tprintf(\"committer %s <%s> %ld +0000\\n\",\n+\t\tauthor_name, author_email, timestamp);\n+\tprintf(\"data %\"PRIuMAX\"\\n\", (uintmax_t) log->len);\n \tfwrite(log->buf, log->len, 1, stdout);\n-\tprintf(\"%s\\n\", gitsvnline);\n-\tif (from_mark)\n-\t\tprintf(\"from :%\"PRIu32\"\\n\", from_mark);\n+\tputchar('\\n');\n+\tif (prev_mark)\n+\t\tprintf(\"from :%\"PRIu32\"\\n\", prev_mark);\n }\n \n-void fast_export_end_commit(uint32_t revision)\n+void fast_export_progress(uint32_t revision)\n {\n+\tprintf(\"progress Imported commit %\"PRIu32\".\\n\\n\", revision);\n }\n \n static void ls_from_rev(uint32_t rev, const char *path)\ndiff --git a/vcs-svn/fast_export.h b/vcs-svn/fast_export.h\nindex 6c1c2be9..f2baf99d 100644\n--- a/vcs-svn/fast_export.h\n+++ b/vcs-svn/fast_export.h\n@@ -10,12 +10,10 @@ void fast_export_reset(void);\n \n void fast_export_delete(const char *path);\n void fast_export_modify(const char *path, uint32_t mode, const char *dataref);\n-void fast_export_begin_commit(uint32_t set_mark, const char *committer_name,\n-\t\t\tconst char *committer_login, const char *committer_domain,\n-\t\t\tconst struct strbuf *log, const char *gitsvnline,\n-\t\t\tunsigned long timestamp, uint32_t from_mark,\n-\t\t\tconst char *dst_ref);\n-void fast_export_end_commit(uint32_t revision);\n+void fast_export_begin_commit(const char *ref, uint32_t mark, uint32_t prev_mark,\n+\t\t\tconst char *author_name, const char *author_email,\n+\t\t\tconst struct strbuf *log, unsigned long timestamp);\n+void fast_export_progress(uint32_t revision);\n void fast_export_data(uint32_t mode, uint32_t len, struct line_buffer *input);\n void fast_export_blob_delta(uint32_t mode,\n \t\t\tuint32_t old_mode, const char *old_data,\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex 97d5fdb7..c562cdaa 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -302,35 +302,42 @@ static void handle_node(void)\n \t\t\t\tnode_ctx.textLength, &input);\n }\n \n-static char gitsvnline[MAX_GITSVN_LINE_LEN];\n+static void add_metadata_trailer(struct strbuf *buf)\n+{\n+\tif (*dump_ctx.uuid.buf && *dump_ctx.url.buf)\n+\t\tstrbuf_addf(buf, \"\\n\\ngit-svn-id: %s@%\"PRIu32\" %s\\n\",\n+\t\t\t dump_ctx.url.buf, rev_ctx.revision, dump_ctx.uuid.buf);\n+}\n+\n static void begin_revision(void)\n {\n-\tint from_mark;\n+\tstatic struct strbuf email;\n \tconst char *author;\n-\tconst char *domain;\n+\tuint32_t prev;\n+\n \tif (!rev_ctx.revision)\t/* revision 0 gets no git commit. */\n \t\treturn;\n-\tif (*dump_ctx.uuid.buf && *dump_ctx.url.buf) {\n-\t\tsnprintf(gitsvnline, MAX_GITSVN_LINE_LEN,\n-\t\t\t\t\"\\n\\ngit-svn-id: %s@%\"PRIu32\" %s\\n\",\n-\t\t\t\t dump_ctx.url.buf, rev_ctx.revision, dump_ctx.uuid.buf);\n-\t} else {\n-\t\t*gitsvnline = 0;\n-\t}\n-\tfrom_mark = dump_ctx.first_commit_done ? rev_ctx.revision - 1 : 0;\n+\tprev = dump_ctx.first_commit_done ? rev_ctx.revision - 1 : 0;\n \tauthor = *rev_ctx.author.buf ? rev_ctx.author.buf : \"nobody\";\n-\tdomain = *dump_ctx.uuid.buf ? dump_ctx.uuid.buf : \"local\";\n \n-\tfast_export_begin_commit(rev_ctx.revision, author, author, domain,\n-\t\t&rev_ctx.log, gitsvnline, rev_ctx.timestamp,\n-\t\tfrom_mark);\n+\tstrbuf_reset(&email);\n+\tstrbuf_addstr(&email, author);\n+\tstrbuf_addch(&email, '@');\n+\tif (*dump_ctx.uuid.buf)\n+\t\tstrbuf_addstr(&email, dump_ctx.uuid.buf);\n+\telse\n+\t\tstrbuf_addstr(&email, \"local\");\n+\n+\tadd_metadata_trailer(&rev_ctx.log);\n+\n+\tfast_export_begin_commit(NULL, rev_ctx.revision, prev,\n+\t\tauthor, email.buf, &rev_ctx.log, rev_ctx.timestamp);\n }\n \n static void end_revision(void)\n {\n \tif (rev_ctx.revision) {\n-\t\tfast_export_end_commit(rev_ctx.revision);\n-\t\tprintf(\"progress Imported commit %\"PRIu32\".\\n\\n\", rev_ctx.revision);\n+\t\tfast_export_progress(rev_ctx.revision);\n \t\tdump_ctx.first_commit_done = 1;\n \t}\n }\n-- \n1.7.6\n"},{"id":"171982","messageId":"20110725085716.GA2440@elie.dc0b.debconf.org","threadId":"27803","inReplyTo":"1310559673-5026-8-git-send-email-divanorama@gmail.com","subject":"Re: [PATCH v2 07/11] vcs-svn,svn-fe: allow to specify dump destination ref","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-07-25T08:57:16Z","receivedAt":"2011-07-25T08:57:16Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Dmitry Ivankov wrote:\n\n> svn-fe produces fast-import stream for a fixed refs/heads/master ref.\n> It is usually desired to write to a different ref. In a remote helper\n> it would be a ref in private namespace. If svn-fe is used by someone\n> directly it'll be more safe to remind where the commits can go. And\n> in both cases it may be needed to import from two repos and hence to\n> different refs.\n>\n> Add a destination ref parameter to vcs-svn/, a corresponding parameter\n> to svn-fe and a simple test for it.\n>\n> $ svn-fe --ref=refs/heads/master ...\n> is an explicit way to stay with the default destination.\n\nThis improvement is very welcome!\n\nAnother reason to avoid fetching straight to \"master\" is that changing\nHEAD from under the user's feet can be confusing. See [*] for some\nexplanation.\n\n> --- a/contrib/svn-fe/svn-fe.c\n> +++ b/contrib/svn-fe/svn-fe.c\n> @@ -17,11 +17,14 @@ static struct svndump_args args;\n> static struct option svn_fe_options[] = {\n> \tOPT_STRING(0, \"git-svn-id-url\", &args.url, \"url\",\n> \t\t\"append git-svn metadata line to commit messages\"),\n> +\tOPT_STRING(0, \"ref\", &args.ref, \"dst_ref\",\n> +\t\t\"write to dst_ref instead of refs/heads/master\"),\n\nA small nit: such an underscored identifier would be typical for a\nvariable name in code but less so for a user-visible syntactic\nplaceholder. One possibility for avoiding that (inspired \"git\nupdate-ref\") is:\n\n\tOPT_STRING(0, \"ref\", &args.ref, \"refname\",\n\t\t\"write to instead of refs/heads/master\"),\n\nwhich shows up in \"svn-fe -h\" output as\n\n --ref write to instead of refs/heads/master\n\n> \tOPT_END()\n> };\n> \n> int main(int argc, const char **argv)\n> {\n> +\targs.ref = \"refs/heads/master\";\n> \targc = parse_options(argc, argv, NULL, svn_fe_options,\n> \t\t\t\t\t\tsvn_fe_usage, 0);\n\nWould it make sense to avoid having to pre-initialize ref by\ninterpreting NULL as refs/heads/master?\n\nMy secret goal in asking that is to find some way to avoid the\ngit-specific idiom of the refs/heads/master ref in fast-import\nfrontends of the future, by coming up with some improvement to the\nsyntax some day (like \"commit default\").\n\n> --- a/contrib/svn-fe/svn-fe.txt\n> +++ b/contrib/svn-fe/svn-fe.txt\n> @@ -32,6 +32,9 @@ OPTIONS\n> \tUrl to be used in git-svn-id: lines in git-svn\n> \tmetadata lines format. See NOTES for more detailed\n> \tdescription.\n> +--ref=::\n> +\tRef to be written by the generated stream.\n> +\tDefault is refs/heads/master.\n\nStyle: usually there is a blank line between items in definition list\nmarkup. Options in manpages (unlike commands, files, and\nconfiguration items) are usually described in the imperative mood, as\na command you give to the program. Like so:\n\n\t--ref=::\n\t\tMake the ref point to the tip of the history\n\t\timported so far, instead of writing to\n\t\trefs/heads/master. This can be useful when importing\n\t\tinto a non-bare repository and the \"master\" branch is\n\t\tchecked out.\n\n[...]\n> --- a/t/t9010-svn-fe.sh\n> +++ b/t/t9010-svn-fe.sh\n> @@ -16,18 +16,24 @@ reinit_git () {\n> \tmkfifo stream backflow\n> }\n> \n> -try_dump () {\n> +try_dump_ext () {\n> +\targs=$1 &&\n> +\tshift &&\n> \tinput=$1 &&\n> \tmaybe_fail_svnfe=${2:+test_$2} &&\n> \tmaybe_fail_fi=${3:+test_$3} &&\n> \n> \t{\n> -\t\t$maybe_fail_svnfe test-svn-fe \"$input\" >stream 3 +\t\t$maybe_fail_svnfe test-svn-fe $args \"$input\" >stream 3 @@ -54,6 +60,22 @@ text_no_props () {\n> \n> >empty\n> \n> +cat >emptyprop.dump <<-EOF &&\n\nNo need for the minus sign and &&, since this is unindented and just\nsupplying test data rather than part of a chain of test assertions\nthat can fail. Putting a \\ before the EOF can be a friendly touch to\nsave reviewers the trouble of looking for shell metacharacters in the\nhere document's body.\n\n[...]\n> @@ -97,21 +119,6 @@ test_expect_failure PIPE 'empty revision' '\n> test_expect_success PIPE 'empty properties' '\n> \treinit_git &&\n> \tprintf \"rev : %s\\n\" \"\" \"\" >expect &&\n> -\tcat >emptyprop.dump <<-\\EOF &&\n\nNice.\n\n> @@ -1111,4 +1118,12 @@ test_expect_success SVNREPO,PIPE 't9135/svn.dump' '\n> \t)\n> '\n> \n> +test_expect_success PIPE 'import to notmaster ref' '\n> +\treinit_git &&\n> +\ttry_dump_ext \"--ref=refs/heads/notmaster\" emptyprop.dump &&\n> +\n> +\tgit rev-parse --verify notmaster &&\n> +\ttest_must_fail git rev-parse --verify master\n> +'\n\nNice and simple. Maybe it would be logical to put it earlier in the\nfile (for three reasons: on one hand to make the file easier to read\nstraight through; on the other hand to make other patches adding tests\nless likely to conflict with this one; and to keep the slow test that\nuses svnadmin at the end of the test script, so results from simpler\nones like this come sooner).\n\n> --- a/test-svn-fe.c\n> +++ b/test-svn-fe.c\n[...]\n> @@ -20,6 +20,8 @@ static int d;\n> \n> static struct option test_svnfe_options[] = {\n> \tOPT_SET_INT('d', NULL, &d, \"test apply_delta\", 1),\n> +\tOPT_STRING(0, \"ref\", &args.ref, \"dst_ref\",\n> +\t\t\"write to dst_ref instead of refs/heads/master\"),\n[...]\n> @@ -51,6 +53,7 @@ static int apply_delta(int argc, const char *argv[])\n> \n> int main(int argc, const char *argv[])\n> {\n> +\targs.ref = \"refs/heads/master\";\n\nSee corresponding comments for contrib/svn-fe.\n\n[...]\n> --- a/vcs-svn/svndump.c\n> +++ b/vcs-svn/svndump.c\n> @@ -57,6 +57,7 @@ static struct {\n> \tuint32_t version;\n> \tstruct strbuf uuid, url;\n> \tint first_commit_done;\n> +\tstruct strbuf ref_name;\n\nThis means the svndump module keeps its own copy of the refname\nstring, saving the caller the trouble of keeping it alive. Probably\nsensible, especially because there could be code between\nsvndump_init() and svndump_read() that actually uses it (for example,\nif the refname is taken from an envvar that getenv() clobbers).\n\n> @@ -324,7 +327,7 @@ static void begin_revision(void)\n> \n> \tfast_export_begin_commit(rev_ctx.revision, author, author, domain,\n> \t\t&rev_ctx.log, gitsvnline, rev_ctx.timestamp,\n> -\t\tfrom_mark);\n> +\t\tfrom_mark, dump_ctx.ref_name.buf);\n> }\n\nHm. fast_export_begin_commit acts on a ref in a sense. That could\nmean the argument should be the first or last one, depending on\nwhether we are imitating fprintf() or fwrite().\n\nFor reference, here's what I tested locally (as a full patch instead\nof incremental because patches it's based on changed enough that the\noriginal did not apply cleanly).\n\nThanks.\n\n[*] https://git.wiki.kernel.org/index.php/GitFaq#How_would_I_use_.22git_push.22_to_sync_out_of_a_host_that_I_cannot_pull_from.3F\n\n-- >8 --\nFrom: Dmitry Ivankov \nDate: Wed, 13 Jul 2011 18:21:09 +0600\nSubject: vcs-svn,svn-fe: allow to specify dump destination ref\n\nsvn-fe produces fast-import stream for a fixed refs/heads/master ref.\nIt is usually desired to write to a different ref. In a remote helper\nit would be a ref in private namespace. If svn-fe is used by someone\ndirectly it'll be more safe to remind where the commits can go. And\nin both cases it may be needed to import from two repos and hence to\ndifferent refs.\n\nAdd a destination ref parameter to vcs-svn/, a corresponding parameter\nto svn-fe and a simple test for it.\n\n$ svn-fe --ref=refs/heads/master ...\nis an explicit way to stay with the default destination.\n\n[jn: use NULL for default, clarify documentation, reorder\n reset_dump_ctx arguments, last fast_export module specify default,\n pass by strbuf so signature changes (API break), test simplifications]\n\nSigned-off-by: Dmitry Ivankov \nSigned-off-by: Jonathan Nieder \n---\n contrib/svn-fe/svn-fe.c | 2 ++\n contrib/svn-fe/svn-fe.txt | 7 +++++++\n t/t9010-svn-fe.sh | 42 +++++++++++++++++++++++++-----------------\n test-svn-fe.c | 6 +++---\n vcs-svn/fast_export.c | 9 +++++----\n vcs-svn/fast_export.h | 5 ++++-\n vcs-svn/svndump.c | 14 ++++++++++----\n vcs-svn/svndump.h | 1 +\n 8 files changed, 57 insertions(+), 29 deletions(-)\n\ndiff --git a/contrib/svn-fe/svn-fe.c b/contrib/svn-fe/svn-fe.c\nindex 5adb2706..c3ebe272 100644\n--- a/contrib/svn-fe/svn-fe.c\n+++ b/contrib/svn-fe/svn-fe.c\n@@ -17,6 +17,8 @@ static struct svndump_options args;\n static struct option svn_fe_options[] = {\n \tOPT_STRING(0, \"git-svn-id-url\", &args.metadata_url, \"url\",\n \t\t\"add git-svn-id line to log messages, imitating git-svn\"),\n+\tOPT_STRING(0, \"ref\", &args.ref, \"refname\",\n+\t\t\"write to instead of refs/heads/master\"),\n \tOPT_END()\n };\n \ndiff --git a/contrib/svn-fe/svn-fe.txt b/contrib/svn-fe/svn-fe.txt\nindex 8c6d3471..7d389ca1 100644\n--- a/contrib/svn-fe/svn-fe.txt\n+++ b/contrib/svn-fe/svn-fe.txt\n@@ -33,6 +33,13 @@ OPTIONS\n \tmetadata lines format. See NOTES for more detailed\n \tdescription.\n \n+--ref=::\n+\tMake the ref point to the tip of the history\n+\timported so far, instead of writing to\n+\t`refs/heads/master`. This can be useful when importing\n+\tinto a non-bare repository and the \"master\" branch is\n+\tchecked out.\n+\n INPUT FORMAT\n ------------\n Subversion's repository dump format is documented in full in\ndiff --git a/t/t9010-svn-fe.sh b/t/t9010-svn-fe.sh\nindex b7eed248..5df1a9bd 100755\n--- a/t/t9010-svn-fe.sh\n+++ b/t/t9010-svn-fe.sh\n@@ -17,12 +17,12 @@ reinit_git () {\n }\n \n try_dump () {\n-\tinput=$1 &&\n+\targs=$1 &&\n \tmaybe_fail_svnfe=${2:+test_$2} &&\n \tmaybe_fail_fi=${3:+test_$3} &&\n \n \t{\n-\t\t$maybe_fail_svnfe test-svn-fe \"$input\" >stream 3stream 3backflow &&\n \twait $!\n@@ -54,6 +54,22 @@ text_no_props () {\n \n >empty\n \n+cat >emptyprop.dump <<\\EOF\n+SVN-fs-dump-format-version: 3\n+\n+Revision-number: 1\n+Prop-content-length: 10\n+Content-length: 10\n+\n+PROPS-END\n+\n+Revision-number: 2\n+Prop-content-length: 10\n+Content-length: 10\n+\n+PROPS-END\n+EOF\n+\n test_expect_success 'setup: have pipes?' '\n \trm -f frob &&\n \tif mkfifo frob\n@@ -97,26 +113,18 @@ test_expect_failure PIPE 'empty revision' '\n test_expect_success PIPE 'empty properties' '\n \treinit_git &&\n \tprintf \"rev : %s\\n\" \"\" \"\" >expect &&\n-\tcat >emptyprop.dump <<-\\EOF &&\n-\tSVN-fs-dump-format-version: 3\n-\n-\tRevision-number: 1\n-\tProp-content-length: 10\n-\tContent-length: 10\n-\n-\tPROPS-END\n-\n-\tRevision-number: 2\n-\tProp-content-length: 10\n-\tContent-length: 10\n-\n-\tPROPS-END\n-\tEOF\n \ttry_dump emptyprop.dump &&\n \tgit log -p --format=\"rev <%an, %ae>: %s\" HEAD >actual &&\n \ttest_cmp expect actual\n '\n \n+test_expect_success PIPE 'import to notmaster ref' '\n+\treinit_git &&\n+\ttry_dump \"--ref=refs/heads/notmaster emptyprop.dump\" &&\n+\tgit rev-parse --verify notmaster &&\n+\ttest_must_fail git rev-parse --verify master\n+'\n+\n test_expect_success PIPE 'author name and commit message' '\n \treinit_git &&\n \techo \"\" >expect.author &&\ndiff --git a/test-svn-fe.c b/test-svn-fe.c\nindex e114bc7c..40f9eaea 100644\n--- a/test-svn-fe.c\n+++ b/test-svn-fe.c\n@@ -15,11 +15,14 @@ static const char * const test_svnfe_usage[] = {\n \tNULL\n };\n \n+static struct svndump_options args;\n static int apply_delta_instead;\n \n static struct option test_svnfe_options[] = {\n \tOPT_SET_INT('d', \"apply-delta\",\n \t\t&apply_delta_instead, \"apply a subversion-format delta\", 1),\n+\tOPT_STRING(0, \"ref\", &args.ref, \"refname\",\n+\t\t\"write to instead of refs/heads/master\"),\n \tOPT_END()\n };\n \n@@ -51,9 +54,6 @@ static int apply_delta(int argc, const char *argv[])\n \n int main(int argc, const char *argv[])\n {\n-\tstruct svndump_options args;\n-\n-\tmemset(&args, 0, sizeof(args));\n \targc = parse_options(argc, argv, NULL, test_svnfe_options,\n \t\t\t\t\t\ttest_svnfe_usage, 0);\n \ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex f61113b4..638485f1 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -69,13 +69,14 @@ void fast_export_modify(const char *path, uint32_t mode, const char *dataref)\n \tputchar('\\n');\n }\n \n-void fast_export_begin_commit(const char *ref, uint32_t mark, uint32_t prev_mark,\n+void fast_export_begin_commit(const struct strbuf *ref,\n+\t\t\tuint32_t mark, uint32_t prev_mark,\n \t\t\tconst char *author_name, const char *author_email,\n \t\t\tconst struct strbuf *log, unsigned long timestamp)\n {\n-\tif (!ref)\n-\t\tref = \"refs/heads/master\";\n-\tprintf(\"commit %s\\n\", ref);\n+\tprintf(\"commit \");\n+\tfwrite(ref->buf, ref->len, 1, stdout);\n+\tputchar('\\n');\n \tif (mark)\n \t\tprintf(\"mark :%\"PRIu32\"\\n\", mark);\n \tprintf(\"committer %s <%s> %ld +0000\\n\",\ndiff --git a/vcs-svn/fast_export.h b/vcs-svn/fast_export.h\nindex f2baf99d..4987afbd 100644\n--- a/vcs-svn/fast_export.h\n+++ b/vcs-svn/fast_export.h\n@@ -1,6 +1,8 @@\n #ifndef FAST_EXPORT_H_\n #define FAST_EXPORT_H_\n \n+#define SVN_FE_DEFAULT_REF \"refs/heads/master\"\n+\n struct strbuf;\n struct line_buffer;\n \n@@ -10,7 +12,8 @@ void fast_export_reset(void);\n \n void fast_export_delete(const char *path);\n void fast_export_modify(const char *path, uint32_t mode, const char *dataref);\n-void fast_export_begin_commit(const char *ref, uint32_t mark, uint32_t prev_mark,\n+void fast_export_begin_commit(const struct strbuf *ref,\n+\t\t\tuint32_t mark, uint32_t prev_mark,\n \t\t\tconst char *author_name, const char *author_email,\n \t\t\tconst struct strbuf *log, unsigned long timestamp);\n void fast_export_progress(uint32_t revision);\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex 0136747b..55c5b657 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -57,6 +57,7 @@ static struct {\n \tuint32_t version;\n \tstruct strbuf uuid, url;\n \tint first_commit_done;\n+\tstruct strbuf ref;\n } dump_ctx;\n \n static void reset_node_ctx(char *fname)\n@@ -82,7 +83,7 @@ static void reset_rev_ctx(uint32_t revision)\n \tstrbuf_reset(&rev_ctx.author);\n }\n \n-static void reset_dump_ctx(const char *url)\n+static void reset_dump_ctx(const char *ref, const char *url)\n {\n \tstrbuf_reset(&dump_ctx.url);\n \tif (url)\n@@ -90,6 +91,11 @@ static void reset_dump_ctx(const char *url)\n \tdump_ctx.version = 1;\n \tstrbuf_reset(&dump_ctx.uuid);\n \tdump_ctx.first_commit_done = 0;\n+\tstrbuf_reset(&dump_ctx.ref);\n+\tif (ref)\n+\t\tstrbuf_addstr(&dump_ctx.ref, ref);\n+\telse\n+\t\tstrbuf_addstr(&dump_ctx.ref, SVN_FE_DEFAULT_REF);\n }\n \n static void handle_property(const struct strbuf *key_buf,\n@@ -331,7 +337,7 @@ static void begin_revision(void)\n \n \tadd_metadata_trailer(&rev_ctx.log);\n \n-\tfast_export_begin_commit(NULL, rev_ctx.revision, prev,\n+\tfast_export_begin_commit(&dump_ctx.ref, rev_ctx.revision, prev,\n \t\tauthor, email.buf, &rev_ctx.log, rev_ctx.timestamp);\n }\n \n@@ -497,7 +503,7 @@ int svndump_init(const struct svndump_options *opts)\n \tstrbuf_init(&rev_ctx.author, 4096);\n \tstrbuf_init(&node_ctx.src, 4096);\n \tstrbuf_init(&node_ctx.dst, 4096);\n-\treset_dump_ctx(opts->metadata_url);\n+\treset_dump_ctx(opts->ref, opts->metadata_url);\n \treset_rev_ctx(0);\n \treset_node_ctx(NULL);\n \treturn 0;\n@@ -506,7 +512,7 @@ int svndump_init(const struct svndump_options *opts)\n void svndump_deinit(void)\n {\n \tfast_export_deinit();\n-\treset_dump_ctx(NULL);\n+\treset_dump_ctx(NULL, NULL);\n \treset_rev_ctx(0);\n \treset_node_ctx(NULL);\n \tstrbuf_release(&rev_ctx.log);\ndiff --git a/vcs-svn/svndump.h b/vcs-svn/svndump.h\nindex b8f52172..4de98700 100644\n--- a/vcs-svn/svndump.h\n+++ b/vcs-svn/svndump.h\n@@ -3,6 +3,7 @@\n \n struct svndump_options {\n \tconst char *dumpfile;\n+\tconst char *ref;\n \tconst char *metadata_url;\n };\n \n-- \n1.7.6\n"},{"id":"172013","messageId":"20110725212626.GA8708@elie.dc0b.debconf.org","threadId":"27803","inReplyTo":"1310559673-5026-9-git-send-email-divanorama@gmail.com","subject":"Re: [PATCH v2 08/11] vcs-svn,svn-fe: convert REPORT_FILENO to an option","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-07-25T21:26:26Z","receivedAt":"2011-07-25T21:26:26Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Dmitry Ivankov wrote:\n\n> svn-fe needs to read fast-import's responses to \"ls\" and \"cat-blob\".\n> These come through a file descriptor number 3.\n>\n> Sometimes it is easier to setup variable fd than a fixed one. It is\n> the case with pipe() call and even more fd=3 can be already taken.\n> On Windows file descriptors are not by default inherited by a child\n> process, nor there is an option to setup descriptors other than\n> standard stdin, stdout, stderr at a process creation time.\n>\n> Add an option for this file descriptor number in vcs-svn/ and svn-fe,\n> add a simple test for it.\n>\n> To be used like following:\n> $ svn-fe --read-blob-fd=7 ... 7 --- a/contrib/svn-fe/svn-fe.c\n> +++ b/contrib/svn-fe/svn-fe.c\n> @@ -19,12 +19,15 @@ static struct option svn_fe_options[] = {\n> \t\t\"append git-svn metadata line to commit messages\"),\n> \tOPT_STRING(0, \"ref\", &args.ref, \"dst_ref\",\n> \t\t\"write to dst_ref instead of refs/heads/master\"),\n> +\tOPT_INTEGER(0, \"read-blob-fd\", &args.backflow_fd,\n> +\t\t\"read blobs and trees from this fd instead of 3\"),\n\n>From the operator's point of view, I think this is just the other end\nof the pipe that fast-import --cat-blob-fd writes to. Maybe\n\n\t\"read fast-import replies from file descriptor (default: 3)\"\n\n[...]\n> \tOPT_END()\n> };\n> \n> int main(int argc, const char **argv)\n> {\n> \targs.ref = \"refs/heads/master\";\n> +\targs.backflow_fd = 3;\n\nLike in the last patch, it is tempting to push this default to the\nlibrary so others can conveniently use \"-1\".\n\n[...]\n> --- a/contrib/svn-fe/svn-fe.txt\n> +++ b/contrib/svn-fe/svn-fe.txt\n[...]\n> @@ -35,6 +35,10 @@ OPTIONS\n> --ref=::\n> \tRef to be written by the generated stream.\n> \tDefault is refs/heads/master.\n> +--read-blob-fd=::\n> +\tInteger number of file descriptor from which\n> +\tresponses to 'ls' and 'cat-blob' requests will come.\n> +\tDefault is fd=3.\n\nNit: more usual to use a verb phrase.\n\n> --- a/t/t9010-svn-fe.sh\n> +++ b/t/t9010-svn-fe.sh\n> @@ -18,20 +18,21 @@ reinit_git () {\n> \n> try_dump_ext () {\n\nIf try_dump_ext from the previous patch gets removed, it would ripple\nthrough here, too. Demonstration of one possible approach below.\n\n[...]\n> +test_expect_success PIPE 'use different backflow fd' '\n> +\treinit_git &&\n> +\techo hi >hi &&\n> +\t{\n> +\t\tproperties \\\n> +\t\t\tsvn:author author@example.com \\\n> +\t\t\tsvn:date \"1999-02-01T00:01:002.000000Z\" \\\n> +\t\t\tsvn:log \"add directory with some files in it\" &&\n\nIs this dump copy/pasted from another test? Would it be possible to\nsimplify or share the dumpfile?\n\nSome but not all of the suggestions above implemented below (this is\njust an example; if something looks crazy, please feel free to drop or\nfix it, of course).\n\nSorry to take so long to look this over. In broad strokes your\npatches carry out very good changes.\n\n-- >8 --\nFrom: Dmitry Ivankov \n\nsvn-fe needs to read fast-import's responses to \"ls\" and \"cat-blob\".\nThese come through a file descriptor number 3.\n\nSometimes it is easier to setup variable fd than a fixed one. It is\nthe case with pipe() call and even more fd=3 can be already taken.\nOn Windows file descriptors are not by default inherited by a child\nprocess, nor there is an option to setup descriptors other than\nstandard stdin, stdout, stderr at a process creation time.\n\nAdd an option for this file descriptor number in vcs-svn/ and svn-fe,\nadd a simple test for it.\n\nTo be used like following:\n$ svn-fe --read-blob-fd=7 ... 7\nSigned-off-by: Jonathan Nieder \n---\n contrib/svn-fe/svn-fe.c | 4 ++-\n contrib/svn-fe/svn-fe.txt | 8 +++++-\n t/t9010-svn-fe.sh | 54 +++++++++++++++++++++++++++++++++++++++++++-\n test-svn-fe.c | 6 ++--\n vcs-svn/fast_export.c | 2 +\n vcs-svn/svndump.c | 4 +--\n vcs-svn/svndump.h | 3 ++\n 7 files changed, 70 insertions(+), 11 deletions(-)\n\ndiff --git a/contrib/svn-fe/svn-fe.c b/contrib/svn-fe/svn-fe.c\nindex 5adb2706..eaab2c79 100644\n--- a/contrib/svn-fe/svn-fe.c\n+++ b/contrib/svn-fe/svn-fe.c\n@@ -12,11 +12,13 @@ static const char * const svn_fe_usage[] = {\n \tNULL\n };\n \n-static struct svndump_options args;\n+static struct svndump_options args = SVNDUMP_OPTIONS_INIT;\n \n static struct option svn_fe_options[] = {\n \tOPT_STRING(0, \"git-svn-id-url\", &args.metadata_url, \"url\",\n \t\t\"add git-svn-id line to log messages, imitating git-svn\"),\n+\tOPT_INTEGER(0, \"read-blob-fd\", &args.backflow_fd,\n+\t\t\"read fast-import replies from file descriptor (default: 3)\"),\n \tOPT_END()\n };\n \ndiff --git a/contrib/svn-fe/svn-fe.txt b/contrib/svn-fe/svn-fe.txt\nindex 8c6d3471..13ab81b3 100644\n--- a/contrib/svn-fe/svn-fe.txt\n+++ b/contrib/svn-fe/svn-fe.txt\n@@ -8,9 +8,9 @@ svn-fe - convert an SVN \"dumpfile\" to a fast-import stream\n SYNOPSIS\n --------\n [verse]\n-mkfifo backchannel &&\n+mkfifo backchannel && fd=3 &&\n svnadmin dump --deltas REPO |\n-\tsvn-fe [options] [git-svn-id-url] 3backchannel\n \n DESCRIPTION\n@@ -33,6 +33,10 @@ OPTIONS\n \tmetadata lines format. See NOTES for more detailed\n \tdescription.\n \n+--read-blob-fd=::\n+\tRead responses to 'ls' and 'cat-blob' requests from\n+\tthis file descriptor. The default is 3.\n+\n INPUT FORMAT\n ------------\n Subversion's repository dump format is documented in full in\ndiff --git a/t/t9010-svn-fe.sh b/t/t9010-svn-fe.sh\nindex b7eed248..e8585260 100755\n--- a/t/t9010-svn-fe.sh\n+++ b/t/t9010-svn-fe.sh\n@@ -17,12 +17,13 @@ reinit_git () {\n }\n \n try_dump () {\n-\tinput=$1 &&\n+\targs=$1 &&\n \tmaybe_fail_svnfe=${2:+test_$2} &&\n \tmaybe_fail_fi=${3:+test_$3} &&\n+\tfd=${4:-3}\n \n \t{\n-\t\t$maybe_fail_svnfe test-svn-fe \"$input\" >stream 3stream $fdbackflow &&\n \twait $!\n@@ -739,6 +740,55 @@ test_expect_success PIPE 'deltas supported' '\n \ttry_dump delta.dump\n '\n \n+test_expect_success PIPE 'use different backflow fd' '\n+\treinit_git &&\n+\techo hi >hi &&\n+\t{\n+\t\tproperties \\\n+\t\t\tsvn:author author@example.com \\\n+\t\t\tsvn:date \"1999-02-01T00:01:002.000000Z\" \\\n+\t\t\tsvn:log \"add directory with some files in it\" &&\n+\t\techo PROPS-END\n+\t} >props &&\n+\t{\n+\t\techo Prop-content-length: $(wc -c revprops &&\n+\t{\n+\t\tcat <<-EOF &&\n+\t\tSVN-fs-dump-format-version: 3\n+\n+\t\tRevision-number: 1\n+\t\tEOF\n+\t\tcat revprops &&\n+\t\tcat <<-EOF &&\n+\t\tNode-path: directory\n+\t\tNode-kind: dir\n+\t\tNode-action: add\n+\t\tNode-path: directory/somefile\n+\t\tNode-kind: file\n+\t\tNode-action: add\n+\t\tEOF\n+\t\ttext_no_props hi &&\n+\n+\t\techo \"Revision-number: 2\" &&\n+\t\tcat revprops &&\n+\t\tcat <<-\\EOF\n+\t\tNode-path: otherfile\n+\t\tNode-kind: file\n+\t\tNode-action: add\n+\t\tNode-copyfrom-rev: 1\n+\t\tNode-copyfrom-path: directory/somefile\n+\t\tEOF\n+\t} >directory.dump &&\n+\ttry_dump \"--read-blob-fd=7 directory.dump\" \"\" \"\" 7 &&\n+\n+\tgit checkout HEAD otherfile &&\n+\ttest_cmp hi otherfile\n+'\n+\n test_expect_success PIPE 'property deltas supported' '\n \treinit_git &&\n \tcat >expect <<-\\EOF &&\ndiff --git a/test-svn-fe.c b/test-svn-fe.c\nindex e114bc7c..a399e183 100644\n--- a/test-svn-fe.c\n+++ b/test-svn-fe.c\n@@ -15,11 +15,14 @@ static const char * const test_svnfe_usage[] = {\n \tNULL\n };\n \n+static struct svndump_options args = SVNDUMP_OPTIONS_INIT;\n static int apply_delta_instead;\n \n static struct option test_svnfe_options[] = {\n \tOPT_SET_INT('d', \"apply-delta\",\n \t\t&apply_delta_instead, \"apply a subversion-format delta\", 1),\n+\tOPT_INTEGER(0, \"read-blob-fd\", &args.backflow_fd,\n+\t\t\"read fast-import replies from file descriptor (default: 3)\"),\n \tOPT_END()\n };\n \n@@ -51,9 +54,6 @@ static int apply_delta(int argc, const char *argv[])\n \n int main(int argc, const char *argv[])\n {\n-\tstruct svndump_options args;\n-\n-\tmemset(&args, 0, sizeof(args));\n \targc = parse_options(argc, argv, NULL, test_svnfe_options,\n \t\t\t\t\t\ttest_svnfe_usage, 0);\n \ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex f61113b4..ecb56e4b 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -28,6 +28,8 @@ static int init_postimage(void)\n \n void fast_export_init(int fd)\n {\n+\tif (fd < 0)\n+\t\tfd = 3;\n \tif (buffer_fdinit(&report_buffer, fd))\n \t\tdie_errno(\"cannot read from file descriptor %d\", fd);\n }\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex 0136747b..a996fbda 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -20,8 +20,6 @@\n */\n #define constcmp(s, ref) memcmp(s, ref, sizeof(ref) - 1)\n \n-#define REPORT_FILENO 3\n-\n #define NODEACT_REPLACE 4\n #define NODEACT_DELETE 3\n #define NODEACT_ADD 2\n@@ -490,7 +488,7 @@ int svndump_init(const struct svndump_options *opts)\n \n \tif (buffer_init(&input, filename))\n \t\treturn error(\"cannot open %s: %s\", filename, strerror(errno));\n-\tfast_export_init(REPORT_FILENO);\n+\tfast_export_init(opts->backflow_fd);\n \tstrbuf_init(&dump_ctx.uuid, 4096);\n \tstrbuf_init(&dump_ctx.url, 4096);\n \tstrbuf_init(&rev_ctx.log, 4096);\ndiff --git a/vcs-svn/svndump.h b/vcs-svn/svndump.h\nindex b8f52172..92ad9ba8 100644\n--- a/vcs-svn/svndump.h\n+++ b/vcs-svn/svndump.h\n@@ -4,8 +4,11 @@\n struct svndump_options {\n \tconst char *dumpfile;\n \tconst char *metadata_url;\n+\tint backflow_fd;\n };\n \n+#define SVNDUMP_OPTIONS_INIT { NULL, NULL, -1 }\n+\n int svndump_init(const struct svndump_options *o);\n void svndump_read(void);\n void svndump_deinit(void);\n-- \n1.7.6\n"},{"id":"172014","messageId":"20110725213526.GB8708@elie.dc0b.debconf.org","threadId":"27803","inReplyTo":"1310559673-5026-11-git-send-email-divanorama@gmail.com","subject":"Re: [PATCH v2 10/11] vcs-svn,svn-fe: add --incremental option","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-07-25T21:35:26Z","receivedAt":"2011-07-25T21:35:26Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Dmitry Ivankov wrote:\n\n> This option is to make svn-fe write commits on top of the existing ref\n> instead of overwriting it. More precise, the first commit's parent is\n> set to be :(first_revision_in_current_dump - 1) mark.\n>\n> Prerequisite is to (re)use import marks (from previous imports). It is\n> safe to use this option on a svn dump that starts with r0/r1. The svn\n> dump itself should be incremental too.\n\nIn other words, this allows running svn-fe to resume a partial import or\nto resume an import after the remote repository has added more history.\nHoorah!\n\n[...]\n> +\ttry_dump_ext \"--incremental\" \"\" \"--export-marks=./marks\" emptyprop.dump &&\n> +\ttest_line_count = 2 ./marks &&\n> +\n> +\ttry_dump_ext \"--incremental\" \"\" \"--import-marks=./marks --export-marks=./marks\" moreempty.dump &&\n> +\ttest_line_count = 3 ./marks &&\n\nIt should be possible to avoid this extra argument by making \"reinit_git\"\nremove the marks file and using something like\n\n\tgit fast-import --import-marks-if-exists=marks --export-marks=marks\n\n> --- a/test-svn-fe.c\n> +++ b/test-svn-fe.c\n> @@ -22,6 +22,9 @@ static struct option test_svnfe_options[] = {\n> \tOPT_SET_INT('d', NULL, &d, \"test apply_delta\", 1),\n> \tOPT_STRING(0, \"ref\", &args.ref, \"dst_ref\",\n> \t\t\"write to dst_ref instead of refs/heads/master\"),\n> +\tOPT_BIT(0, \"incremental\", &args.incremental,\n> +\t\t\"resume export, requires marks and incremental dump\",\n> +\t\t1),\n\nWhy OPT_BIT?\n\nHope that helps,\nJonathan\n"},{"id":"172016","messageId":"20110725213923.GC8708@elie.dc0b.debconf.org","threadId":"27803","inReplyTo":"1310559673-5026-12-git-send-email-divanorama@gmail.com","subject":"Re: [PATCH v2 11/11] vcs-svn,svn-fe: add an option to write svnrev notes","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-07-25T21:39:23Z","receivedAt":"2011-07-25T21:39:23Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nDmitry Ivankov wrote:\n\n> There are already a few options to determine svn revision from which\n> a git commit imported with svn-fe came from. One is to make svn-fe\n> write a git-svn-id: line to commit messages. Another one is to calc\n> distance to the root commit. The former includes a \"url\" and is for\n> git-svn compatibility, the latter is obviously slow and a bit fragile.\n>\n> $ svn-fe --notes_ref=notes_tree --ref=branch...\n> will write annotations for branch commits to the notes_tree, each\n> annotation is a simple \"rN\" string. Then these annotations can be\n> viewed manually or used in incremental import to detect the last\n> imported revision or to (re)create the import marks for further\n> imports.\n\nWouldn't another way be to look at the mark numbers?\n\nI am not sure I like this. svn-fe is supposed to be a generally\nuseful tool, and this patch hard-codes the particular note format rN.\nIf it is needed, maybe it would be possible to do something like\n\n\t--notes-ref=refs/notes/svn-rev --note='project foo, r%N'\n\nAs a bonus, that would allow including more information using\ndifferent flag characters in the note in the future.\n"},{"id":"172234","messageId":"CA+gfSn9MGiBJ-vhj25wMa5Jwu0hWo0Uyv2wiYst09+M6VX1SQA@mail.gmail.com","threadId":"27803","inReplyTo":"20110725213923.GC8708@elie.dc0b.debconf.org","subject":"Re: [PATCH v2 11/11] vcs-svn,svn-fe: add an option to write svnrev notes","fromName":"Dmitry Ivankov","fromEmail":"divanorama@gmail.com","sentAt":"2011-07-28T06:03:42Z","receivedAt":"2011-07-28T06:03:42Z","isPatch":true,"sender":{"key":"divanorama@gmail.com","avatar":"https://avatars.githubusercontent.com/u/158999?v=4"},"body":"Sorry for a way too slow response. This patch is probably the most\nunbaked one, so I'll start here.\n\nOn Tue, Jul 26, 2011 at 3:39 AM, Jonathan Nieder wrote:\n> Hi,\n>\n> Dmitry Ivankov wrote:\n>\n>> There are already a few options to determine svn revision from which\n>> a git commit imported with svn-fe came from. One is to make svn-fe\n>> write a git-svn-id: line to commit messages. Another one is to calc\n>> distance to the root commit. The former includes a \"url\" and is for\n>> git-svn compatibility, the latter is obviously slow and a bit fragile.\n>>\n>> $ svn-fe --notes_ref=notes_tree --ref=branch...\n>> will write annotations for branch commits to the notes_tree, each\n>> annotation is a simple \"rN\" string. Then these annotations can be\n>> viewed manually or used in incremental import to detect the last\n>> imported revision or to (re)create the import marks for further\n>> imports.\n>\n> Wouldn't another way be to look at the mark numbers?\nIf marks file is absent (after clone for example), we'll need to look\nat notes anyway.\nIf it is present and has a mark for latest revision we can hope it is\nvalid and not\nregenerate it.\nIf for some reason notes are lost and marks are present, it's possible\nto recreate\nnotes from marks though, at least \"rN\" ones.\n\n> I am not sure I like this.  svn-fe is supposed to be a generally\n> useful tool, and this patch hard-codes the particular note format rN.\n> If it is needed, maybe it would be possible to do something like\n>\n>        --notes-ref=refs/notes/svn-rev --note='project foo, r%N'\n>\n> As a bonus, that would allow including more information using\n> different flag characters in the note in the future.\nFormat string looks nice. While the whole notes thing may need\nmore thinking.\nThe main reason I wrote these in svn-fe is that it's more atomic\nto write note just after writing a commit. Also \"checkpoint\" will\ncreate notes for current status (and will write marks too, but weren't\nwe going to consider notes as a primary data copy? it can be cloned\nfor example).\n\nOne more consideration is that copy-from information most likely\nwill be written by svn-fe (nothing else knows it anyway) to some\nnotes, so we'll need some notes writing in svn-fe.\n"},{"id":"172247","messageId":"20110728102729.GC6991@elie","threadId":"27803","inReplyTo":"CA+gfSn9MGiBJ-vhj25wMa5Jwu0hWo0Uyv2wiYst09+M6VX1SQA@mail.gmail.com","subject":"Re: [PATCH v2 11/11] vcs-svn,svn-fe: add an option to write svnrev notes","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-07-28T10:27:29Z","receivedAt":"2011-07-28T10:27:29Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Dmitry Ivankov wrote:\n> On Tue, Jul 26, 2011 at 3:39 AM, Jonathan Nieder wrote:\n\n>>        --notes-ref=refs/notes/svn-rev --note='project foo, r%N'\n>>\n>> As a bonus, that would allow including more information using\n>> different flag characters in the note in the future.\n>\n> Format string looks nice.\n\nYes, it sounds pleasant to work with to me. Open questions:\n\n - how fine-grained should the notes commits be? E.g., should they\n be written one at a time, in batches of 10, or something else?\n\n - is there a way to make some commits get no note at all? Is this\n template-based approach the right way to go? (I guess yes, it is.)\n\n - what if someone wants multiple notes refs (e.g., revision numbers\n and revprops in separate notes refs)? Will this support that? If\n not, is it extensible enough to sensibly support that in the\n future?\n\n[...]\n> One more consideration is that copy-from information most likely\n> will be written by svn-fe (nothing else knows it anyway) to some\n> notes, so we'll need some notes writing in svn-fe.\n\nYep, copyfrom info has to get downstream somehow. Which means\n\n a. in log messages\n b. in notes\n c. in comments or \"progress\" lines in the stream, or\nor d. in a second output stream, with file descriptor number specified\n by the caller.\n\nI was leaning towards (a) or (c), but (b) certainly also seems\nsensible.\n"}]}