{"thread":{"id":"26286","subject":"[PATCH 2/3] vcs-svn: Start working on the dumpfile producer","startedAt":"2011-01-15T06:51:09Z","lastAt":"2011-01-15T08:11:30Z","messageCount":8,"participants":["Ramkumar Ramachandra","Jonathan Nieder","Peter Baumann"],"isPatch":true,"patchVersion":1,"patchTotal":3},"messages":[{"id":"159522","messageId":"1295074272-19559-1-git-send-email-artagnon@gmail.com","threadId":"26286","inReplyTo":null,"subject":"[RFC PATCH 0/3] Towards a Git-to-SVN bridge","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2011-01-15T06:51:09Z","receivedAt":"2011-01-15T06:51:09Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi,\n\nOver the last couple of days, I've been working on a parser that\nconverts a fast-import stream into a SVN dumpfile. So far, it's very\nrough and works minimally for some common fast-import\ncommands. However, the major roadblock is persisting blobs: in this\nimplementation, they're persisted as an array of strbufs. This is very\nmemory-intensive and not scalable at all. With some valuable insight\nfrom Jonathan on IRC, I've decided to try re-implementing fast-export\nto eliminate blob marks and produce them inline instead [1].\n\nComments are much appreciated.\n\n[1]: http://colabti.org/irclogger/irclogger_log/git-devel?date=2011-01-14\n\nRamkumar Ramachandra (3):\n  date: Expose the time_to_tm function\n  vcs-svn: Start working on the dumpfile producer\n  Build an svn-fi target in contrib/svn-fe\n\n Makefile                  |    2 +-\n cache.h                   |    1 +\n contrib/svn-fe/Makefile   |   23 ++++-\n contrib/svn-fe/svn-fi.c   |   16 +++\n contrib/svn-fe/svn-fi.txt |   28 +++++\n date.c                    |    2 +-\n vcs-svn/dump_export.c     |   73 +++++++++++\n vcs-svn/svnload.c         |  294 +++++++++++++++++++++++++++++++++++++++++++++\n 8 files changed, 435 insertions(+), 4 deletions(-)\n create mode 100644 contrib/svn-fe/svn-fi.c\n create mode 100644 contrib/svn-fe/svn-fi.txt\n create mode 100644 vcs-svn/dump_export.c\n create mode 100644 vcs-svn/svnload.c\n\n-- \n1.7.4.rc1.7.g2cf08.dirty\n"},{"id":"159523","messageId":"1295074272-19559-2-git-send-email-artagnon@gmail.com","threadId":"26286","inReplyTo":"1295074272-19559-1-git-send-email-artagnon@gmail.com","subject":"[PATCH 1/3] date: Expose the time_to_tm function","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2011-01-15T06:51:10Z","receivedAt":"2011-01-15T06:51:10Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Signed-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\n---\n cache.h |    1 +\n date.c  |    2 +-\n 2 files changed, 2 insertions(+), 1 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex d83d68c..95fea31 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -816,6 +816,7 @@ enum date_mode {\n \tDATE_RAW\n };\n \n+struct tm *time_to_tm(unsigned long time, int tz);\n const char *show_date(unsigned long time, int timezone, enum date_mode mode);\n const char *show_date_relative(unsigned long time, int tz,\n \t\t\t       const struct timeval *now,\ndiff --git a/date.c b/date.c\nindex 00f9eb5..e601a50 100644\n--- a/date.c\n+++ b/date.c\n@@ -54,7 +54,7 @@ static time_t gm_time_t(unsigned long time, int tz)\n  * thing, which means that tz -0100 is passed in as the integer -100,\n  * even though it means \"sixty minutes off\"\n  */\n-static struct tm *time_to_tm(unsigned long time, int tz)\n+struct tm *time_to_tm(unsigned long time, int tz)\n {\n \ttime_t t = gm_time_t(time, tz);\n \treturn gmtime(&t);\n-- \n1.7.4.rc1.7.g2cf08.dirty\n"},{"id":"159520","messageId":"1295074272-19559-3-git-send-email-artagnon@gmail.com","threadId":"26286","inReplyTo":"1295074272-19559-1-git-send-email-artagnon@gmail.com","subject":"[PATCH 2/3] vcs-svn: Start working on the dumpfile producer","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2011-01-15T06:51:11Z","receivedAt":"2011-01-15T06:51:11Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Start off with some broad design sketches. Compile succeeds, but\nparser is incorrect. Include a Makefile rule to build it into\nvcs-svn/lib.a.\n\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\n---\n Makefile              |    2 +-\n vcs-svn/dump_export.c |   73 ++++++++++++\n vcs-svn/svnload.c     |  294 +++++++++++++++++++++++++++++++++++++++++++++++++\n 3 files changed, 368 insertions(+), 1 deletions(-)\n create mode 100644 vcs-svn/dump_export.c\n create mode 100644 vcs-svn/svnload.c\n\ndiff --git a/Makefile b/Makefile\nindex 1345c38..40f6691 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1834,7 +1834,7 @@ ifndef NO_CURL\n endif\n XDIFF_OBJS = xdiff/xdiffi.o xdiff/xprepare.o xdiff/xutils.o xdiff/xemit.o \\\n \txdiff/xmerge.o xdiff/xpatience.o\n-VCSSVN_OBJS = vcs-svn/line_buffer.o \\\n+VCSSVN_OBJS = vcs-svn/line_buffer.o vcs-svn/svnload.o vcs-svn/dump_export.o \\\n \tvcs-svn/repo_tree.o vcs-svn/fast_export.o vcs-svn/sliding_window.o \\\n \tvcs-svn/svndiff.o vcs-svn/svndump.o\n VCSSVN_TEST_OBJS = test-obj-pool.o \\\ndiff --git a/vcs-svn/dump_export.c b/vcs-svn/dump_export.c\nnew file mode 100644\nindex 0000000..04ede06\n--- /dev/null\n+++ b/vcs-svn/dump_export.c\n@@ -0,0 +1,73 @@\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"git-compat-util.h\"\n+#include \"strbuf.h\"\n+#include \"line_buffer.h\"\n+#include \"dump_export.h\"\n+\n+void dump_export_begin_rev(int revision, const char *revprops,\n+\t\t\tint prop_len) {\n+\tprintf(\"Revision-number: %d\\n\", revision);\n+\tprintf(\"Prop-content-length: %d\\n\", prop_len);\n+\tprintf(\"Content-length: %d\\n\\n\", prop_len);\n+\tprintf(\"%s\\n\", revprops);\n+}\n+\n+void dump_export_node(const char *path, enum node_kind kind,\n+\t\tenum node_action action, unsigned long text_len,\n+\t\tunsigned long copyfrom_rev, const char *copyfrom_path) {\n+\tprintf(\"Node-path: %s\\n\", path);\n+\tprintf(\"Node-kind: \");\n+\tswitch (action) {\n+\tcase NODE_KIND_NORMAL:\n+\t\tprintf(\"file\\n\");\n+\t\tbreak;\n+\tcase NODE_KIND_EXECUTABLE:\n+\t\tprintf(\"file\\n\");\n+\t\tbreak;\n+\tcase NODE_KIND_SYMLINK:\n+\t\tprintf(\"file\\n\");\n+\t\tbreak;\n+\tcase NODE_KIND_GITLINK:\n+\t\tprintf(\"file\\n\");\n+\t\tbreak;\n+\tcase NODE_KIND_SUBDIR:\n+\t\tdie(\"Unsupported: subdirectory\");\n+\tdefault:\n+\t\tbreak;\n+\t}\n+\tprintf(\"Node-action: \");\n+\tswitch (action) {\n+\tcase NODE_ACTION_CHANGE:\n+\t\tprintf(\"change\\n\");\n+\t\tbreak;\n+\tcase NODE_ACTION_ADD:\n+\t\tprintf(\"add\\n\");\n+\t\tbreak;\n+\tcase NODE_ACTION_REPLACE:\n+\t\tprintf(\"replace\\n\");\n+\t\tbreak;\n+\tcase NODE_ACTION_DELETE:\n+\t\tprintf(\"delete\\n\");\n+\t\tbreak;\n+\tdefault:\n+\t\tbreak;\n+\t}\n+\tif (copyfrom_rev != SVN_INVALID_REV) {\n+\t\tprintf(\"Node-copyfrom-rev: %lu\\n\", copyfrom_rev);\n+\t\tprintf(\"Node-copyfrom-path: %s\\n\", copyfrom_path);\n+\t}\n+\tprintf(\"Prop-delta: false\\n\");\n+\tprintf(\"Prop-content-length: 10\\n\"); /* Constant 10 for \"PROPS-END\" */\n+\tprintf(\"Text-delta: false\\n\");\n+\tprintf(\"Text-content-length: %lu\\n\", text_len);\n+\tprintf(\"Content-length: %lu\\n\\n\", text_len + 10);\n+\tprintf(\"PROPS-END\\n\\n\");\n+}\n+\n+void dump_export_text(struct line_buffer *data, off_t len) {\n+\tbuffer_copy_bytes(data, len);\n+}\ndiff --git a/vcs-svn/svnload.c b/vcs-svn/svnload.c\nnew file mode 100644\nindex 0000000..7043ae7\n--- /dev/null\n+++ b/vcs-svn/svnload.c\n@@ -0,0 +1,294 @@\n+/*\n+ * Produce a dumpfile v3 from a fast-import stream.\n+ * Load the dump into the SVN repository with:\n+ * svnrdump load <URL> <dumpfile\n+ *\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"cache.h\"\n+#include \"git-compat-util.h\"\n+#include \"line_buffer.h\"\n+#include \"dump_export.h\"\n+#include \"strbuf.h\"\n+\n+#define SVN_DATE_FORMAT \"%Y-%m-%dT%H:%M:%S.000000Z\"\n+#define SVN_DATE_LEN 28\n+#define LENGTH_UNKNOWN (~0)\n+\n+static struct line_buffer input = LINE_BUFFER_INIT;\n+static struct strbuf blobs[100];\n+\t\n+static struct {\n+\tunsigned long prop_len, text_len, copyfrom_rev, mark;\n+\tint text_delta, prop_delta; /* Boolean */\n+\tenum node_action action;\n+\tenum node_kind kind;\n+\tstruct strbuf copyfrom_path, path;\n+} node_ctx;\n+\n+static struct {\n+\tint rev, text_len;\n+\tstruct strbuf props, log;\n+\tstruct strbuf svn_author, author, committer;\n+\tstruct strbuf author_date, committer_date;\n+\tstruct strbuf author_email, committer_email;\n+} rev_ctx;\n+\n+static enum {\n+\tUNKNOWN_CTX,\n+\tCOMMIT_CTX,\n+\tBLOB_CTX\n+} active_ctx;\n+\n+static void reset_rev_ctx(int revision)\n+{\n+\trev_ctx.rev = revision;\n+\tstrbuf_reset(&rev_ctx.props);\n+\tstrbuf_reset(&rev_ctx.log);\n+\tstrbuf_reset(&rev_ctx.svn_author);\n+\tstrbuf_reset(&rev_ctx.author);\n+\tstrbuf_reset(&rev_ctx.committer);\n+\tstrbuf_reset(&rev_ctx.author_date);\n+\tstrbuf_reset(&rev_ctx.committer_date);\n+\tstrbuf_reset(&rev_ctx.author_email);\n+\tstrbuf_reset(&rev_ctx.committer_email);\n+}\n+\n+static void reset_node_ctx(void)\n+{\n+\tnode_ctx.prop_len = LENGTH_UNKNOWN;\n+\tnode_ctx.text_len = LENGTH_UNKNOWN;\n+\tnode_ctx.mark = 0;\n+\tnode_ctx.copyfrom_rev = 0;\n+\tnode_ctx.text_delta = -1;\n+\tnode_ctx.prop_delta = -1;\n+\tstrbuf_reset(&node_ctx.copyfrom_path);\n+\tstrbuf_reset(&node_ctx.path);\n+}\n+\n+static void populate_props(struct strbuf *props, const char *author,\n+\t\t\tconst char *log, const char *date) {\n+\tstrbuf_reset(props);\t\n+\tstrbuf_addf(props, \"K\\nsvn:author\\nV\\n%s\\n\", author);\n+\tstrbuf_addf(props, \"K\\nsvn:log\\nV\\n%s\", log);\n+\tstrbuf_addf(props, \"K\\nsvn:date\\nV\\n%s\\n\", date);\n+\tstrbuf_add(props, \"PROPS-END\\n\", 10);\n+}\n+\n+static void parse_author_line(char *val, struct strbuf *name,\n+\t\t\tstruct strbuf *email, struct strbuf *date) {\n+\tchar *t, *tz_off;\n+\tchar time_buf[SVN_DATE_LEN];\n+\tconst struct tm *tm_time;\n+\n+\t/* Simon Hausmann <shausman@trolltech.com> 1170199019 +0100 */\n+\tstrbuf_reset(name);\n+\tstrbuf_reset(email);\n+\tstrbuf_reset(date);\n+\ttz_off = strrchr(val, ' ');\n+\t*tz_off++ = '\\0';\n+\tt = strrchr(val, ' ');\n+\t*(t - 1) = '\\0'; /* Ignore '>' from email */\n+\tt ++;\n+\ttm_time = time_to_tm(strtoul(t, NULL, 10), atoi(tz_off));\n+\tstrftime(time_buf, SVN_DATE_LEN, SVN_DATE_FORMAT, tm_time);\n+\tstrbuf_add(date, time_buf, SVN_DATE_LEN);\n+\tt = strchr(val, '<');\n+\t*(t - 1) = '\\0'; /* Ignore ' <' from email */\n+\tt ++;\n+\tstrbuf_add(email, t, strlen(t));\n+\tstrbuf_add(name, val, strlen(val));\n+}\n+\n+void svnload_read(void) {\n+\tchar *t, *val;\n+\tint mode_incr;\n+\tstruct strbuf *to_dump;\n+\n+\twhile ((t = buffer_read_line(&input))) {\n+\t\tval = strchr(t, ' ');\n+\t\tif (!val) {\n+\t\t\tif (!memcmp(t, \"blob\", 4))\n+\t\t\t\tactive_ctx = BLOB_CTX;\n+\t\t\telse if (!memcmp(t, \"deleteall\", 9))\n+\t\t\t\t;\n+\t\t\tcontinue;\n+\t\t}\n+\t\t*val++ = '\\0';\n+\n+\t\t/* strlen(key) */\n+\t\tswitch (val - t - 1) {\n+\t\tcase 1:\n+\t\t\tif (!memcmp(t, \"D\", 1)) {\n+\t\t\t\tnode_ctx.action = NODE_ACTION_DELETE;\n+\t\t\t}\n+\t\t\telse if (!memcmp(t, \"C\", 1)) {\n+\t\t\t\tnode_ctx.action = NODE_ACTION_ADD;\n+\t\t\t}\n+\t\t\telse if (!memcmp(t, \"R\", 1)) {\n+\t\t\t\tnode_ctx.action = NODE_ACTION_REPLACE;\n+\t\t\t}\n+\t\t\telse if (!memcmp(t, \"M\", 1)) {\n+\t\t\t\tnode_ctx.action = NODE_ACTION_CHANGE;\n+\t\t\t\tmode_incr = 7;\n+\t\t\t\tif (!memcmp(val, \"100644\", 6))\n+\t\t\t\t\tnode_ctx.kind = NODE_KIND_NORMAL;\n+\t\t\t\telse if (!memcmp(val, \"100755\", 6))\n+\t\t\t\t\tnode_ctx.kind = NODE_KIND_EXECUTABLE;\n+\t\t\t\telse if (!memcmp(val, \"120000\", 6))\n+\t\t\t\t\tnode_ctx.kind = NODE_KIND_SYMLINK;\n+\t\t\t\telse if (!memcmp(val, \"160000\", 6))\n+\t\t\t\t\tnode_ctx.kind = NODE_KIND_GITLINK;\n+\t\t\t\telse if (!memcmp(val, \"040000\", 6))\n+\t\t\t\t\tnode_ctx.kind = NODE_KIND_SUBDIR;\n+\t\t\t\telse {\n+\t\t\t\t\tif (!memcmp(val, \"755\", 3))\n+\t\t\t\t\t\tnode_ctx.kind = NODE_KIND_EXECUTABLE;\n+\t\t\t\t\telse if(!memcmp(val, \"644\", 3))\n+\t\t\t\t\t\tnode_ctx.kind = NODE_KIND_NORMAL;\n+\t\t\t\t\telse\n+\t\t\t\t\t\tdie(\"Unrecognized mode: %s\", val);\n+\t\t\t\t\tmode_incr = 4;\n+\t\t\t\t}\n+\t\t\t\tval += mode_incr;\n+\t\t\t\tt = strchr(val, ' ');\n+\t\t\t\t*t++ = '\\0';\n+\t\t\t\tstrbuf_reset(&node_ctx.path);\n+\t\t\t\tstrbuf_add(&node_ctx.path, t, strlen(t));\n+\t\t\t\tif (!memcmp(val + 1, \"inline\", 6))\n+\t\t\t\t\tdie(\"Unsupported dataref: inline\");\n+\t\t\t\telse if (*val == ':')\n+\t\t\t\t\tto_dump = &blobs[strtoul(val + 1, NULL, 10)];\n+\t\t\t\telse\n+\t\t\t\t\tdie(\"Unsupported dataref: sha1\");\n+\t\t\t\tdump_export_node(node_ctx.path.buf, node_ctx.kind,\n+\t\t\t\t\t\tnode_ctx.action, to_dump->len,\n+\t\t\t\t\t\t0, NULL);\n+\t\t\t\tprintf(\"%s\", to_dump->buf);\n+\t\t\t}\n+\t\t\tbreak;\n+\t\tcase 3:\n+\t\t\tif (!memcmp(t, \"tag\", 3))\n+\t\t\t\tcontinue;\n+\t\t\tbreak;\n+\t\tcase 4:\n+\t\t\tif (!memcmp(t, \"mark\", 4))\n+\t\t\t\tswitch(active_ctx) {\n+\t\t\t\tcase COMMIT_CTX:\n+\t\t\t\t\t/* What do we do with commit marks? */\n+\t\t\t\t\tcontinue;\n+\t\t\t\tcase BLOB_CTX:\n+\t\t\t\t\tnode_ctx.mark = strtoul(val + 1, NULL, 10);\n+\t\t\t\t\tbreak;\n+\t\t\t\tdefault:\n+\t\t\t\t\tbreak;\n+\t\t\t\t}\n+\t\t\telse if (!memcmp(t, \"from\", 4))\n+\t\t\t\tcontinue;\n+\t\t\telse if (!memcmp(t, \"data\", 4)) {\n+\t\t\t\tswitch (active_ctx) {\n+\t\t\t\tcase COMMIT_CTX:\n+\t\t\t\t\tstrbuf_reset(&rev_ctx.log);\n+\t\t\t\t\tbuffer_read_binary(&input,\n+\t\t\t\t\t\t\t&rev_ctx.log,\n+\t\t\t\t\t\t\tstrtoul(val, NULL, 10));\n+\t\t\t\t\tpopulate_props(&rev_ctx.props,\n+\t\t\t\t\t\trev_ctx.svn_author.buf,\n+\t\t\t\t\t\trev_ctx.log.buf,\n+\t\t\t\t\t\trev_ctx.author_date.buf);\n+\t\t\t\t\tdump_export_begin_rev(rev_ctx.rev,\n+\t\t\t\t\t\t\trev_ctx.props.buf,\n+\t\t\t\t\t\t\trev_ctx.props.len);\n+\t\t\t\t\tbreak;\n+\t\t\t\tcase BLOB_CTX:\n+\t\t\t\t\tnode_ctx.text_len = strtoul(val, NULL, 10);\n+\t\t\t\t\tbuffer_read_binary(&input,\n+\t\t\t\t\t\t\t&blobs[node_ctx.mark],\n+\t\t\t\t\t\t\tnode_ctx.text_len);\n+\t\t\t\t\tbreak;\n+\t\t\t\tdefault:\n+\t\t\t\t\tbreak;\n+\t\t\t\t}\n+\t\t\t}\n+\t\t\tbreak;\n+\t\tcase 5:\n+\t\t\tif (!memcmp(t, \"reset\", 5))\n+\t\t\t\tcontinue;\n+\t\t\tif (!memcmp(t, \"merge\", 5))\n+\t\t\t\tcontinue;\n+\t\t\tbreak;\n+\t\tcase 6:\n+\t\t\tif (!memcmp(t, \"author\", 6)) {\n+\t\t\t\tparse_author_line(val, &rev_ctx.author,\n+\t\t\t\t\t\t&rev_ctx.author_email,\n+\t\t\t\t\t\t&rev_ctx.author_date);\n+\t\t\t\t/* Build svn_author */\n+\t\t\t\tt = strchr(rev_ctx.author_email.buf, '@');\n+\t\t\t\tstrbuf_reset(&rev_ctx.svn_author);\n+\t\t\t\tstrbuf_add(&rev_ctx.svn_author,\n+\t\t\t\t\trev_ctx.author_email.buf,\n+\t\t\t\t\tt - rev_ctx.author_email.buf);\n+\n+\t\t\t}\n+\t\t\telse if (!memcmp(t, \"commit\", 6)) {\n+\t\t\t\trev_ctx.rev ++;\n+\t\t\t\tactive_ctx = COMMIT_CTX;\n+\t\t\t}\n+\t\t\tbreak;\n+\t\tcase 9:\n+\t\t\tif (!memcmp(t, \"committer\", 9))\n+\t\t\t\tparse_author_line(val, &rev_ctx.committer,\n+\t\t\t\t\t\t&rev_ctx.committer_email,\n+\t\t\t\t\t\t&rev_ctx.committer_date);\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\tbreak;\n+\t\t}\n+\t}\n+}\n+\n+int svnload_init(const char *filename)\n+{\n+\tint i;\n+\tif (buffer_init(&input, filename))\n+\t\treturn error(\"cannot open %s: %s\", filename, strerror(errno));\n+\tactive_ctx = UNKNOWN_CTX;\n+\tstrbuf_init(&rev_ctx.props, MAX_GITSVN_LINE_LEN);\n+\tstrbuf_init(&rev_ctx.log, MAX_GITSVN_LINE_LEN);\n+\tstrbuf_init(&rev_ctx.author, MAX_GITSVN_LINE_LEN);\n+\tstrbuf_init(&rev_ctx.committer, MAX_GITSVN_LINE_LEN);\n+\tstrbuf_init(&rev_ctx.author_date, MAX_GITSVN_LINE_LEN);\n+\tstrbuf_init(&rev_ctx.committer_date, MAX_GITSVN_LINE_LEN);\n+\tstrbuf_init(&rev_ctx.author_email, MAX_GITSVN_LINE_LEN);\n+\tstrbuf_init(&rev_ctx.committer_email, MAX_GITSVN_LINE_LEN);\n+\tstrbuf_init(&node_ctx.path, MAX_GITSVN_LINE_LEN);\n+\tstrbuf_init(&node_ctx.copyfrom_path, MAX_GITSVN_LINE_LEN);\n+\tfor (i = 0; i < 100; i ++)\n+\t\tstrbuf_init(&blobs[i], 10000);\n+\treturn 0;\n+}\n+\n+void svnload_deinit(void)\n+{\n+\tint i;\n+\treset_rev_ctx(0);\n+\treset_node_ctx();\n+\tstrbuf_release(&rev_ctx.props);\n+\tstrbuf_release(&rev_ctx.log);\n+\tstrbuf_release(&rev_ctx.author);\n+\tstrbuf_release(&rev_ctx.committer);\n+\tstrbuf_release(&rev_ctx.author_date);\n+\tstrbuf_release(&rev_ctx.committer_date);\n+\tstrbuf_release(&rev_ctx.author_email);\n+\tstrbuf_release(&rev_ctx.committer_email);\n+\tstrbuf_release(&node_ctx.path);\n+\tstrbuf_release(&node_ctx.copyfrom_path);\n+\tfor (i = 0; i < 100; i ++)\n+\t\tstrbuf_release(&blobs[i]);\n+\tif (buffer_deinit(&input))\n+\t\tfprintf(stderr, \"Input error\\n\");\n+\tif (ferror(stdout))\n+\t\tfprintf(stderr, \"Output error\\n\");\n+}\n-- \n1.7.4.rc1.7.g2cf08.dirty\n"},{"id":"159521","messageId":"1295074272-19559-4-git-send-email-artagnon@gmail.com","threadId":"26286","inReplyTo":"1295074272-19559-1-git-send-email-artagnon@gmail.com","subject":"[PATCH 3/3] Build an svn-fi target in contrib/svn-fe","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2011-01-15T06:51:12Z","receivedAt":"2011-01-15T06:51:12Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Build an svn-fi target for testing the dumpfile producer in vcs-svn/.\n\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\n---\n contrib/svn-fe/Makefile   |   23 +++++++++++++++++++++--\n contrib/svn-fe/svn-fi.c   |   16 ++++++++++++++++\n contrib/svn-fe/svn-fi.txt |   28 ++++++++++++++++++++++++++++\n 3 files changed, 65 insertions(+), 2 deletions(-)\n create mode 100644 contrib/svn-fe/svn-fi.c\n create mode 100644 contrib/svn-fe/svn-fi.txt\n\ndiff --git a/contrib/svn-fe/Makefile b/contrib/svn-fe/Makefile\nindex 360d8da..555a8ff 100644\n--- a/contrib/svn-fe/Makefile\n+++ b/contrib/svn-fe/Makefile\n@@ -37,7 +37,7 @@ svn-fe$X: svn-fe.o $(VCSSVN_LIB) $(GIT_LIB)\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ svn-fe.o \\\n \t\t$(ALL_LDFLAGS) $(LIBS)\n \n-svn-fe.o: svn-fe.c ../../vcs-svn/svndump.h\n+svn-fe.o: svn-fe.c ../../vcs-svn/svnload.h\n \t$(QUIET_CC)$(CC) -I../../vcs-svn -o $*.o -c $(ALL_CFLAGS) $<\n \n svn-fe.html: svn-fe.txt\n@@ -51,6 +51,24 @@ svn-fe.1: svn-fe.txt\n \t\t../contrib/svn-fe/$@\n \t$(MV) ../../Documentation/svn-fe.1 .\n \n+svn-fi$X: svn-fi.o $(VCSSVN_LIB) $(GIT_LIB)\n+\t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ svn-fi.o \\\n+\t\t$(ALL_LDFLAGS) $(LIBS)\n+\n+svn-fi.o: svn-fi.c ../../vcs-svn/svnload.h\n+\t$(QUIET_CC)$(CC) -I../../vcs-svn -o $*.o -c $(ALL_CFLAGS) $<\n+\n+svn-fi.html: svn-fi.txt\n+\t$(QUIET_SUBDIR0)../../Documentation $(QUIET_SUBDIR1) \\\n+\t\tMAN_TXT=../contrib/svn-fe/svn-fi.txt \\\n+\t\t../contrib/svn-fe/$@\n+\n+svn-fi.1: svn-fi.txt\n+\t$(QUIET_SUBDIR0)../../Documentation $(QUIET_SUBDIR1) \\\n+\t\tMAN_TXT=../contrib/svn-fe/svn-fi.txt \\\n+\t\t../contrib/svn-fe/$@\n+\t$(MV) ../../Documentation/svn-fi.1 .\n+\n ../../vcs-svn/lib.a: FORCE\n \t$(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) vcs-svn/lib.a\n \n@@ -58,6 +76,7 @@ svn-fe.1: svn-fe.txt\n \t$(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) libgit.a\n \n clean:\n-\t$(RM) svn-fe$X svn-fe.o svn-fe.html svn-fe.xml svn-fe.1\n+\t$(RM) svn-fe$X svn-fe.o svn-fe.html svn-fe.xml svn-fe.1 \\\n+\tsvn-fi$X svn-fi.o svn-fi.html svn-fi.xml svn-fi.1\n \n .PHONY: all clean FORCE\ndiff --git a/contrib/svn-fe/svn-fi.c b/contrib/svn-fe/svn-fi.c\nnew file mode 100644\nindex 0000000..81347b0\n--- /dev/null\n+++ b/contrib/svn-fe/svn-fi.c\n@@ -0,0 +1,16 @@\n+/*\n+ * This file is in the public domain.\n+ * You may freely use, modify, distribute, and relicense it.\n+ */\n+\n+#include <stdlib.h>\n+#include \"svnload.h\"\n+\n+int main(int argc, char **argv)\n+{\n+\tif (svnload_init(NULL))\n+\t\treturn 1;\n+\tsvnload_read();\n+\tsvnload_deinit();\n+\treturn 0;\n+}\ndiff --git a/contrib/svn-fe/svn-fi.txt b/contrib/svn-fe/svn-fi.txt\nnew file mode 100644\nindex 0000000..996a175\n--- /dev/null\n+++ b/contrib/svn-fe/svn-fi.txt\n@@ -0,0 +1,28 @@\n+svn-fe(1)\n+=========\n+\n+NAME\n+----\n+svn-fi - convert fast-import stream to an SVN \"dumpfile\"\n+\n+SYNOPSIS\n+--------\n+[verse]\n+svn-fi\n+\n+DESCRIPTION\n+-----------\n+\n+Converts a git-fast-import(1) stream into a Subversion dumpfile.\n+\n+INPUT FORMAT\n+-------------\n+The fast-import format is documented by the git-fast-import(1)\n+manual page.\n+\n+OUTPUT FORMAT\n+------------\n+Subversion's repository dump format is documented in full in\n+`notes/dump-load-format.txt` from the Subversion source tree.\n+Files in this format can be generated using the 'svnadmin dump' or\n+'svk admin dump' command.\n-- \n1.7.4.rc1.7.g2cf08.dirty\n"},{"id":"159524","messageId":"20110115072211.GB25253@burratino","threadId":"26286","inReplyTo":"1295074272-19559-1-git-send-email-artagnon@gmail.com","subject":"Re: [RFC PATCH 0/3] Towards a Git-to-SVN bridge","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-01-15T07:22:11Z","receivedAt":"2011-01-15T07:22:11Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi Ram,\n\nRamkumar Ramachandra wrote:\n\n> Over the last couple of days, I've been working on a parser that\n> converts a fast-import stream into a SVN dumpfile. So far, it's very\n> rough and works minimally for some common fast-import\n> commands.\n\nSome early questions:\n\n - what are the design goals?  Is this meant to be super fast?\n   Robust?  Simple?  Why should I be excited about it?[1]\n\n - what subset of fast-import commands is supported?  Is it well\n   enough defined to make a manpage?\n\n - does this produce v2 or v3 dumpfiles?\n\n - why would I use this instead of git2svn?  Does git2svn do anything\n   this will not eventually be able to do?  (Not a trick question ---\n   I don't have enough experience with git2svn to tell its strengths\n   and weaknesses.)\n\n> I've decided to try re-implementing fast-export\n> to eliminate blob marks\n\nHopefully \"re-implement\" means \"patch\" here. :)\n\nI can comment on the code but it's probably better if I have a sense\nof the design first (in any event, thanks for sending it).\n\nRegards,\nJonathan\n\n[1] I found the original svn-fe design interesting because\n (1) it reused code from an existing svndump parser, at least in\n     spirit,\n (2) the repo_tree data structure was well fitted to the design\n     constraints,\n (3) the line_buffer input abstraction was oddly satisfying, even\n     though it does not buy anything obvious out of the box over\n     direct use of strbuf and stdio;\n (4) speed; and, most importantly\n (5) the command-line interface was easy to debug, very flexible,\n     and dead simple.\n\nI find the current svn-fe satisfying in a different way --- a sort of\n\"line by line\" translation between dump formats is becoming possible.\n"},{"id":"159526","messageId":"20110115073925.GA21744@m62s10.vlinux.de","threadId":"26286","inReplyTo":"1295074272-19559-3-git-send-email-artagnon@gmail.com","subject":"Re: [PATCH 2/3] vcs-svn: Start working on the dumpfile producer","fromName":"Peter Baumann","fromEmail":"waste.manager@gmx.de","sentAt":"2011-01-15T07:39:25Z","receivedAt":"2011-01-15T07:39:25Z","isPatch":true,"sender":{"key":"waste.manager@gmx.de","avatar":null},"body":"On Sat, Jan 15, 2011 at 12:21:11PM +0530, Ramkumar Ramachandra wrote:\n> Start off with some broad design sketches. Compile succeeds, but\n> parser is incorrect. Include a Makefile rule to build it into\n> vcs-svn/lib.a.\n> \n> Signed-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\n> ---\n>  Makefile              |    2 +-\n>  vcs-svn/dump_export.c |   73 ++++++++++++\n>  vcs-svn/svnload.c     |  294 +++++++++++++++++++++++++++++++++++++++++++++++++\n>  3 files changed, 368 insertions(+), 1 deletions(-)\n>  create mode 100644 vcs-svn/dump_export.c\n>  create mode 100644 vcs-svn/svnload.c\n> \n...\n> diff --git a/vcs-svn/svnload.c b/vcs-svn/svnload.c\n> new file mode 100644\n> index 0000000..7043ae7\n> --- /dev/null\n> +++ b/vcs-svn/svnload.c\n> @@ -0,0 +1,294 @@\n> +/*\n> + * Produce a dumpfile v3 from a fast-import stream.\n> + * Load the dump into the SVN repository with:\n> + * svnrdump load <URL> <dumpfile\n> + *\n> + * Licensed under a two-clause BSD-style license.\n> + * See LICENSE for details.\n> + */\n> +\n> +#include \"cache.h\"\n> +#include \"git-compat-util.h\"\n> +#include \"line_buffer.h\"\n> +#include \"dump_export.h\"\n> +#include \"strbuf.h\"\n> +\n> +#define SVN_DATE_FORMAT \"%Y-%m-%dT%H:%M:%S.000000Z\"\n> +#define SVN_DATE_LEN 28\n> +#define LENGTH_UNKNOWN (~0)\n> +\n> +static struct line_buffer input = LINE_BUFFER_INIT;\n> +static struct strbuf blobs[100];\n> +\t\n> +static struct {\n> +\tunsigned long prop_len, text_len, copyfrom_rev, mark;\n> +\tint text_delta, prop_delta; /* Boolean */\n> +\tenum node_action action;\n> +\tenum node_kind kind;\n> +\tstruct strbuf copyfrom_path, path;\n> +} node_ctx;\n> +\n> +static struct {\n> +\tint rev, text_len;\n> +\tstruct strbuf props, log;\n> +\tstruct strbuf svn_author, author, committer;\n> +\tstruct strbuf author_date, committer_date;\n> +\tstruct strbuf author_email, committer_email;\n> +} rev_ctx;\n> +\n> +static enum {\n> +\tUNKNOWN_CTX,\n> +\tCOMMIT_CTX,\n> +\tBLOB_CTX\n> +} active_ctx;\n> +\n> +static void reset_rev_ctx(int revision)\n> +{\n> +\trev_ctx.rev = revision;\n> +\tstrbuf_reset(&rev_ctx.props);\n> +\tstrbuf_reset(&rev_ctx.log);\n> +\tstrbuf_reset(&rev_ctx.svn_author);\n> +\tstrbuf_reset(&rev_ctx.author);\n> +\tstrbuf_reset(&rev_ctx.committer);\n> +\tstrbuf_reset(&rev_ctx.author_date);\n> +\tstrbuf_reset(&rev_ctx.committer_date);\n> +\tstrbuf_reset(&rev_ctx.author_email);\n> +\tstrbuf_reset(&rev_ctx.committer_email);\n> +}\n> +\n> +static void reset_node_ctx(void)\n> +{\n> +\tnode_ctx.prop_len = LENGTH_UNKNOWN;\n> +\tnode_ctx.text_len = LENGTH_UNKNOWN;\n> +\tnode_ctx.mark = 0;\n> +\tnode_ctx.copyfrom_rev = 0;\n> +\tnode_ctx.text_delta = -1;\n> +\tnode_ctx.prop_delta = -1;\n> +\tstrbuf_reset(&node_ctx.copyfrom_path);\n> +\tstrbuf_reset(&node_ctx.path);\n> +}\n> +\n> +static void populate_props(struct strbuf *props, const char *author,\n> +\t\t\tconst char *log, const char *date) {\n> +\tstrbuf_reset(props);\t\n> +\tstrbuf_addf(props, \"K\\nsvn:author\\nV\\n%s\\n\", author);\n> +\tstrbuf_addf(props, \"K\\nsvn:log\\nV\\n%s\", log);\n> +\tstrbuf_addf(props, \"K\\nsvn:date\\nV\\n%s\\n\", date);\n> +\tstrbuf_add(props, \"PROPS-END\\n\", 10);\n> +}\n> +\n> +static void parse_author_line(char *val, struct strbuf *name,\n> +\t\t\tstruct strbuf *email, struct strbuf *date) {\n> +\tchar *t, *tz_off;\n> +\tchar time_buf[SVN_DATE_LEN];\n> +\tconst struct tm *tm_time;\n> +\n> +\t/* Simon Hausmann <shausman@trolltech.com> 1170199019 +0100 */\n> +\tstrbuf_reset(name);\n> +\tstrbuf_reset(email);\n> +\tstrbuf_reset(date);\n> +\ttz_off = strrchr(val, ' ');\n> +\t*tz_off++ = '\\0';\n> +\tt = strrchr(val, ' ');\n> +\t*(t - 1) = '\\0'; /* Ignore '>' from email */\n> +\tt ++;\n> +\ttm_time = time_to_tm(strtoul(t, NULL, 10), atoi(tz_off));\n> +\tstrftime(time_buf, SVN_DATE_LEN, SVN_DATE_FORMAT, tm_time);\n> +\tstrbuf_add(date, time_buf, SVN_DATE_LEN);\n> +\tt = strchr(val, '<');\n> +\t*(t - 1) = '\\0'; /* Ignore ' <' from email */\n> +\tt ++;\n> +\tstrbuf_add(email, t, strlen(t));\n> +\tstrbuf_add(name, val, strlen(val));\n> +}\n> +\n> +void svnload_read(void) {\n> +\tchar *t, *val;\n> +\tint mode_incr;\n> +\tstruct strbuf *to_dump;\n> +\n> +\twhile ((t = buffer_read_line(&input))) {\n> +\t\tval = strchr(t, ' ');\n> +\t\tif (!val) {\n> +\t\t\tif (!memcmp(t, \"blob\", 4))\n> +\t\t\t\tactive_ctx = BLOB_CTX;\n> +\t\t\telse if (!memcmp(t, \"deleteall\", 9))\n> +\t\t\t\t;\n> +\t\t\tcontinue;\n\nHaving actually no idea what the input you are reading from might look like, but\nseeing those two memcmp compares above makes me wonder if 't' might ever be smaller\nthan 4 (or 9 for the else part). Which obviously would lead to a SEGFAULT. \nIn the code below there are also memcmp class which might step out of the\nbuffer.\n\n> +\t\t}\n> +\t\t*val++ = '\\0';\n> +\n> +\t\t/* strlen(key) */\n> +\t\tswitch (val - t - 1) {\n> +\t\tcase 1:\n> +\t\t\tif (!memcmp(t, \"D\", 1)) {\n> +\t\t\t\tnode_ctx.action = NODE_ACTION_DELETE;\n> +\t\t\t}\n> +\t\t\telse if (!memcmp(t, \"C\", 1)) {\n> +\t\t\t\tnode_ctx.action = NODE_ACTION_ADD;\n> +\t\t\t}\n> +\t\t\telse if (!memcmp(t, \"R\", 1)) {\n> +\t\t\t\tnode_ctx.action = NODE_ACTION_REPLACE;\n> +\t\t\t}\n> +\t\t\telse if (!memcmp(t, \"M\", 1)) {\n> +\t\t\t\tnode_ctx.action = NODE_ACTION_CHANGE;\n> +\t\t\t\tmode_incr = 7;\n> +\t\t\t\tif (!memcmp(val, \"100644\", 6))\n> +\t\t\t\t\tnode_ctx.kind = NODE_KIND_NORMAL;\n> +\t\t\t\telse if (!memcmp(val, \"100755\", 6))\n> +\t\t\t\t\tnode_ctx.kind = NODE_KIND_EXECUTABLE;\n> +\t\t\t\telse if (!memcmp(val, \"120000\", 6))\n> +\t\t\t\t\tnode_ctx.kind = NODE_KIND_SYMLINK;\n> +\t\t\t\telse if (!memcmp(val, \"160000\", 6))\n> +\t\t\t\t\tnode_ctx.kind = NODE_KIND_GITLINK;\n> +\t\t\t\telse if (!memcmp(val, \"040000\", 6))\n> +\t\t\t\t\tnode_ctx.kind = NODE_KIND_SUBDIR;\n> +\t\t\t\telse {\n> +\t\t\t\t\tif (!memcmp(val, \"755\", 3))\n> +\t\t\t\t\t\tnode_ctx.kind = NODE_KIND_EXECUTABLE;\n> +\t\t\t\t\telse if(!memcmp(val, \"644\", 3))\n> +\t\t\t\t\t\tnode_ctx.kind = NODE_KIND_NORMAL;\n> +\t\t\t\t\telse\n> +\t\t\t\t\t\tdie(\"Unrecognized mode: %s\", val);\n> +\t\t\t\t\tmode_incr = 4;\n> +\t\t\t\t}\n> +\t\t\t\tval += mode_incr;\n> +\t\t\t\tt = strchr(val, ' ');\n> +\t\t\t\t*t++ = '\\0';\n> +\t\t\t\tstrbuf_reset(&node_ctx.path);\n> +\t\t\t\tstrbuf_add(&node_ctx.path, t, strlen(t));\n> +\t\t\t\tif (!memcmp(val + 1, \"inline\", 6))\n> +\t\t\t\t\tdie(\"Unsupported dataref: inline\");\n> +\t\t\t\telse if (*val == ':')\n> +\t\t\t\t\tto_dump = &blobs[strtoul(val + 1, NULL, 10)];\n> +\t\t\t\telse\n> +\t\t\t\t\tdie(\"Unsupported dataref: sha1\");\n> +\t\t\t\tdump_export_node(node_ctx.path.buf, node_ctx.kind,\n> +\t\t\t\t\t\tnode_ctx.action, to_dump->len,\n> +\t\t\t\t\t\t0, NULL);\n> +\t\t\t\tprintf(\"%s\", to_dump->buf);\n> +\t\t\t}\n> +\t\t\tbreak;\n> +\t\tcase 3:\n> +\t\t\tif (!memcmp(t, \"tag\", 3))\n> +\t\t\t\tcontinue;\n> +\t\t\tbreak;\n> +\t\tcase 4:\n> +\t\t\tif (!memcmp(t, \"mark\", 4))\n> +\t\t\t\tswitch(active_ctx) {\n> +\t\t\t\tcase COMMIT_CTX:\n> +\t\t\t\t\t/* What do we do with commit marks? */\n> +\t\t\t\t\tcontinue;\n> +\t\t\t\tcase BLOB_CTX:\n> +\t\t\t\t\tnode_ctx.mark = strtoul(val + 1, NULL, 10);\n> +\t\t\t\t\tbreak;\n> +\t\t\t\tdefault:\n> +\t\t\t\t\tbreak;\n> +\t\t\t\t}\n> +\t\t\telse if (!memcmp(t, \"from\", 4))\n> +\t\t\t\tcontinue;\n> +\t\t\telse if (!memcmp(t, \"data\", 4)) {\n> +\t\t\t\tswitch (active_ctx) {\n> +\t\t\t\tcase COMMIT_CTX:\n> +\t\t\t\t\tstrbuf_reset(&rev_ctx.log);\n> +\t\t\t\t\tbuffer_read_binary(&input,\n> +\t\t\t\t\t\t\t&rev_ctx.log,\n> +\t\t\t\t\t\t\tstrtoul(val, NULL, 10));\n> +\t\t\t\t\tpopulate_props(&rev_ctx.props,\n> +\t\t\t\t\t\trev_ctx.svn_author.buf,\n> +\t\t\t\t\t\trev_ctx.log.buf,\n> +\t\t\t\t\t\trev_ctx.author_date.buf);\n> +\t\t\t\t\tdump_export_begin_rev(rev_ctx.rev,\n> +\t\t\t\t\t\t\trev_ctx.props.buf,\n> +\t\t\t\t\t\t\trev_ctx.props.len);\n> +\t\t\t\t\tbreak;\n> +\t\t\t\tcase BLOB_CTX:\n> +\t\t\t\t\tnode_ctx.text_len = strtoul(val, NULL, 10);\n> +\t\t\t\t\tbuffer_read_binary(&input,\n> +\t\t\t\t\t\t\t&blobs[node_ctx.mark],\n> +\t\t\t\t\t\t\tnode_ctx.text_len);\n> +\t\t\t\t\tbreak;\n> +\t\t\t\tdefault:\n> +\t\t\t\t\tbreak;\n> +\t\t\t\t}\n> +\t\t\t}\n> +\t\t\tbreak;\n> +\t\tcase 5:\n> +\t\t\tif (!memcmp(t, \"reset\", 5))\n> +\t\t\t\tcontinue;\n> +\t\t\tif (!memcmp(t, \"merge\", 5))\n> +\t\t\t\tcontinue;\n> +\t\t\tbreak;\n> +\t\tcase 6:\n> +\t\t\tif (!memcmp(t, \"author\", 6)) {\n> +\t\t\t\tparse_author_line(val, &rev_ctx.author,\n> +\t\t\t\t\t\t&rev_ctx.author_email,\n> +\t\t\t\t\t\t&rev_ctx.author_date);\n> +\t\t\t\t/* Build svn_author */\n> +\t\t\t\tt = strchr(rev_ctx.author_email.buf, '@');\n> +\t\t\t\tstrbuf_reset(&rev_ctx.svn_author);\n> +\t\t\t\tstrbuf_add(&rev_ctx.svn_author,\n> +\t\t\t\t\trev_ctx.author_email.buf,\n> +\t\t\t\t\tt - rev_ctx.author_email.buf);\n> +\n> +\t\t\t}\n> +\t\t\telse if (!memcmp(t, \"commit\", 6)) {\n> +\t\t\t\trev_ctx.rev ++;\n> +\t\t\t\tactive_ctx = COMMIT_CTX;\n> +\t\t\t}\n> +\t\t\tbreak;\n> +\t\tcase 9:\n> +\t\t\tif (!memcmp(t, \"committer\", 9))\n> +\t\t\t\tparse_author_line(val, &rev_ctx.committer,\n> +\t\t\t\t\t\t&rev_ctx.committer_email,\n> +\t\t\t\t\t\t&rev_ctx.committer_date);\n> +\t\t\tbreak;\n> +\t\tdefault:\n> +\t\t\tbreak;\n> +\t\t}\n> +\t}\n> +}\n> +\n> +int svnload_init(const char *filename)\n> +{\n> +\tint i;\n> +\tif (buffer_init(&input, filename))\n> +\t\treturn error(\"cannot open %s: %s\", filename, strerror(errno));\n> +\tactive_ctx = UNKNOWN_CTX;\n> +\tstrbuf_init(&rev_ctx.props, MAX_GITSVN_LINE_LEN);\n> +\tstrbuf_init(&rev_ctx.log, MAX_GITSVN_LINE_LEN);\n> +\tstrbuf_init(&rev_ctx.author, MAX_GITSVN_LINE_LEN);\n> +\tstrbuf_init(&rev_ctx.committer, MAX_GITSVN_LINE_LEN);\n> +\tstrbuf_init(&rev_ctx.author_date, MAX_GITSVN_LINE_LEN);\n> +\tstrbuf_init(&rev_ctx.committer_date, MAX_GITSVN_LINE_LEN);\n> +\tstrbuf_init(&rev_ctx.author_email, MAX_GITSVN_LINE_LEN);\n> +\tstrbuf_init(&rev_ctx.committer_email, MAX_GITSVN_LINE_LEN);\n> +\tstrbuf_init(&node_ctx.path, MAX_GITSVN_LINE_LEN);\n> +\tstrbuf_init(&node_ctx.copyfrom_path, MAX_GITSVN_LINE_LEN);\n> +\tfor (i = 0; i < 100; i ++)\n> +\t\tstrbuf_init(&blobs[i], 10000);\n> +\treturn 0;\n> +}\n> +\n> +void svnload_deinit(void)\n> +{\n> +\tint i;\n> +\treset_rev_ctx(0);\n> +\treset_node_ctx();\n> +\tstrbuf_release(&rev_ctx.props);\n> +\tstrbuf_release(&rev_ctx.log);\n> +\tstrbuf_release(&rev_ctx.author);\n> +\tstrbuf_release(&rev_ctx.committer);\n> +\tstrbuf_release(&rev_ctx.author_date);\n> +\tstrbuf_release(&rev_ctx.committer_date);\n> +\tstrbuf_release(&rev_ctx.author_email);\n> +\tstrbuf_release(&rev_ctx.committer_email);\n> +\tstrbuf_release(&node_ctx.path);\n> +\tstrbuf_release(&node_ctx.copyfrom_path);\n> +\tfor (i = 0; i < 100; i ++)\n> +\t\tstrbuf_release(&blobs[i]);\n> +\tif (buffer_deinit(&input))\n> +\t\tfprintf(stderr, \"Input error\\n\");\n> +\tif (ferror(stdout))\n> +\t\tfprintf(stderr, \"Output error\\n\");\n> +}\n> -- \n> 1.7.4.rc1.7.g2cf08.dirty\n> \n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at  http://vger.kernel.org/majordomo-info.html\n> \n"},{"id":"159527","messageId":"20110115074327.GA9277@kytes","threadId":"26286","inReplyTo":"20110115072211.GB25253@burratino","subject":"Re: [RFC PATCH 0/3] Towards a Git-to-SVN bridge","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2011-01-15T07:43:29Z","receivedAt":"2011-01-15T07:43:29Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi Jonathan,\n\nJonathan Nieder writes:\n> Ramkumar Ramachandra wrote:\n> \n> > Over the last couple of days, I've been working on a parser that\n> > converts a fast-import stream into a SVN dumpfile. So far, it's very\n> > rough and works minimally for some common fast-import\n> > commands.\n> \n> Some early questions:\n\nThanks for raising these questions. People interested in the project\nshould find this useful.\n\n>  - what are the design goals?  Is this meant to be super fast?\n>    Robust?  Simple?  Why should I be excited about it?[1]\n\nI want it to be a lot like current svn-fe: as you can see, I've\nre-used many parsing ideas from it. It has to be atleast as fast as\nsvnrdump, because I don't want it to bottleneck in the remote helper\npipeline. It has to be simple because it'll give rise to other simple\nremote helpers- all the complexity has to be offloaded onto the lower\nlayers like fast-import/ fast-export, and not onto the developer of\nthe remote helper.\n\n>  - what subset of fast-import commands is supported?  Is it well\n>    enough defined to make a manpage?\n\nCurrently, it supports just \"commit\", \"blob\", \"author\", \"committer\"\nand \"mark\" that appear after a blob. It should support more commands\nsoon enough- this implementation is just a proof of concept. Also,\nInstead of giving it the ability to parse /any/ valid fast-import\nstream, I want to simply focus on parsing the stream produced by git\nfast-export. That should explain why I'm trying to patch git\nfast-export primarily.\n\n>  - does this produce v2 or v3 dumpfiles?\n\nThis is one issue I haven't thought about fully yet. I'm currently\nthinking of generating a non-deltified dumpfile v3 -- something that\nsvnrdump will accept. Generating deltas might be an unnecessary\noverhead- but as you pointed out yesterday, that clearly needs more\nthought.\n\n>  - why would I use this instead of git2svn?  Does git2svn do anything\n>    this will not eventually be able to do?  (Not a trick question ---\n>    I don't have enough experience with git2svn to tell its strengths\n>    and weaknesses.)\n\ngit2svn persists blobs in-memory. It's written in Perl and it's\nslow. I thought we needed something nicer to be used with a remote\nhelper, and started writing svn-fi.\n\n> > I've decided to try re-implementing fast-export\n> > to eliminate blob marks\n> \n> Hopefully \"re-implement\" means \"patch\" here. :)\n\nYep. Just a big one :)\n\n> I can comment on the code but it's probably better if I have a sense\n> of the design first (in any event, thanks for sending it).\n\nI haven't had time to clean up the code. Note that it \"just works\" at\nthe moment- yes, it's already very very fast :) However, I'm going to\nstall the branch and work on fast-export-inline for now.\n\n-- Ram\n"},{"id":"159528","messageId":"20110115081128.GB9277@kytes","threadId":"26286","inReplyTo":"20110115073925.GA21744@m62s10.vlinux.de","subject":"Re: [PATCH 2/3] vcs-svn: Start working on the dumpfile producer","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2011-01-15T08:11:30Z","receivedAt":"2011-01-15T08:11:30Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi Peter,\n\nPeter Baumann writes:\n> > +\twhile ((t = buffer_read_line(&input))) {\n> > +\t\tval = strchr(t, ' ');\n> > +\t\tif (!val) {\n> > +\t\t\tif (!memcmp(t, \"blob\", 4))\n> > +\t\t\t\tactive_ctx = BLOB_CTX;\n> > +\t\t\telse if (!memcmp(t, \"deleteall\", 9))\n> > +\t\t\t\t;\n> > +\t\t\tcontinue;\n> \n> Having actually no idea what the input you are reading from might look like, but\n> seeing those two memcmp compares above makes me wonder if 't' might ever be smaller\n> than 4 (or 9 for the else part). Which obviously would lead to a SEGFAULT. \n> In the code below there are also memcmp class which might step out of the\n> buffer.\n\nRight. Silly mistake on my part. Thanks for pointing it out.\n\nThere are probably many more trivial mistakes- I was in a hurry to get\n/something/ working, and didn't have a chance to clean up the code.\n\n-- Ram\n"}]}