{"thread":{"id":"31372","subject":"[RFC v2 0/4] GSOC: prepare svndump for branch detection","startedAt":"2012-08-28T09:15:00Z","lastAt":"2012-08-28T09:15:04Z","messageCount":5,"participants":["Florian Achleitner"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"197981","messageId":"1346145304-27149-1-git-send-email-florian.achleitner.2.6.31@gmail.com","threadId":"31372","inReplyTo":null,"subject":"[RFC v2 0/4] GSOC: prepare svndump for branch detection","fromName":"Florian Achleitner","fromEmail":"florian.achleitner.2.6.31@gmail.com","sentAt":"2012-08-28T09:15:00Z","receivedAt":"2012-08-28T09:15:00Z","isPatch":false,"sender":{"key":"florian.achleitner.2.6.31@gmail.com","avatar":"https://avatars.githubusercontent.com/u/880777?v=4"},"body":"Includes a small fixup by Ramsey:\nImproves integer type compatibility on different platforms.\nsvndump.c:\n- replace %lu by PRIuMAX in printf format string.\n- cast lhs of cast operator to uintmax_t\n\n[RFC v2 1/4] svndump: move struct definitions to .h\n[RFC v2 2/4] vcs-svn/svndump: restructure node_ctx, rev_ctx handling\n[RFC v2 3/4] vcs-svn/svndump: rewrite handle_node(),\n[RFC v2 4/4] vcs-svn: remove repo_tree\n"},{"id":"197982","messageId":"1346145304-27149-2-git-send-email-florian.achleitner.2.6.31@gmail.com","threadId":"31372","inReplyTo":"1346145304-27149-1-git-send-email-florian.achleitner.2.6.31@gmail.com","subject":"[RFC v2 1/4] svndump: move struct definitions to .h","fromName":"Florian Achleitner","fromEmail":"florian.achleitner.2.6.31@gmail.com","sentAt":"2012-08-28T09:15:01Z","receivedAt":"2012-08-28T09:15:01Z","isPatch":false,"sender":{"key":"florian.achleitner.2.6.31@gmail.com","avatar":"https://avatars.githubusercontent.com/u/880777?v=4"},"body":"This is a preparation for adding branch detection code in separate\n.c files.\n\nExternal branch detection functions will need to use these.\n\nSigned-off-by: Florian Achleitner <florian.achleitner.2.6.31@gmail.com>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\n vcs-svn/svndump.c |   21 ++++-----------------\n vcs-svn/svndump.h |   19 +++++++++++++++++++\n 2 files changed, 23 insertions(+), 17 deletions(-)\n\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex 31d1d83..6ca94de 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -38,23 +38,10 @@\n \n static struct line_buffer input = LINE_BUFFER_INIT;\n \n-static struct {\n-\tuint32_t action, srcRev, type;\n-\toff_t prop_length, text_length;\n-\tstruct strbuf src, dst;\n-\tuint32_t text_delta, prop_delta;\n-} node_ctx;\n-\n-static struct {\n-\tuint32_t revision;\n-\tunsigned long timestamp;\n-\tstruct strbuf log, author, note;\n-} rev_ctx;\n-\n-static struct {\n-\tuint32_t version;\n-\tstruct strbuf uuid, url;\n-} dump_ctx;\n+static struct node_ctx_t node_ctx;\n+static struct rev_ctx_t rev_ctx;\n+static struct dump_ctx_t dump_ctx;\n+\n \n static void reset_node_ctx(char *fname)\n {\ndiff --git a/vcs-svn/svndump.h b/vcs-svn/svndump.h\nindex b8eb129..d545453 100644\n--- a/vcs-svn/svndump.h\n+++ b/vcs-svn/svndump.h\n@@ -1,5 +1,6 @@\n #ifndef SVNDUMP_H_\n #define SVNDUMP_H_\n+#include \"cache.h\"\n \n int svndump_init(const char *filename);\n int svndump_init_fd(int in_fd, int back_fd);\n@@ -7,4 +8,22 @@ void svndump_read(const char *url, const char *local_ref, const char *notes_ref)\n void svndump_deinit(void);\n void svndump_reset(void);\n \n+struct node_ctx_t {\n+\tuint32_t action, srcRev, type;\n+\toff_t prop_length, text_length;\n+\tstruct strbuf src, dst;\n+\tuint32_t text_delta, prop_delta;\n+};\n+\n+struct rev_ctx_t {\n+\tuint32_t revision;\n+\tunsigned long timestamp;\n+\tstruct strbuf log, author, note;\n+};\n+\n+struct dump_ctx_t {\n+\tuint32_t version;\n+\tstruct strbuf uuid, url;\n+};\n+\n #endif\n-- \n1.7.9.5\n"},{"id":"197984","messageId":"1346145304-27149-3-git-send-email-florian.achleitner.2.6.31@gmail.com","threadId":"31372","inReplyTo":"1346145304-27149-2-git-send-email-florian.achleitner.2.6.31@gmail.com","subject":"[RFC v2 2/4] vcs-svn/svndump: restructure node_ctx, rev_ctx handling","fromName":"Florian Achleitner","fromEmail":"florian.achleitner.2.6.31@gmail.com","sentAt":"2012-08-28T09:15:02Z","receivedAt":"2012-08-28T09:15:02Z","isPatch":false,"sender":{"key":"florian.achleitner.2.6.31@gmail.com","avatar":"https://avatars.githubusercontent.com/u/880777?v=4"},"body":"As a preparation for handling branches in svndumps, make rev_ctx and\nnode_ctx more flexible.\n\nAdd the object to work on to the arguments of reset_*_ctx() and to\nhandle_node() to allow for multiple *_ctx objects.\n\nConvert the static global node_ctx to a linked list of such objects to\nbe able to accumulate all Node data of a revision in memory before\nprocessing it.\n\nSigned-off-by: Florian Achleitner <florian.achleitner.2.6.31@gmail.com>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\n vcs-svn/svndump.c |  207 +++++++++++++++++++++++++++++++----------------------\n vcs-svn/svndump.h |    2 +\n 2 files changed, 124 insertions(+), 85 deletions(-)\n\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex 6ca94de..385523a 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -38,42 +38,81 @@\n \n static struct line_buffer input = LINE_BUFFER_INIT;\n \n-static struct node_ctx_t node_ctx;\n+static struct node_ctx_t *node_ctx;\n static struct rev_ctx_t rev_ctx;\n static struct dump_ctx_t dump_ctx;\n+static const char *current_ref;\n \n+static struct node_ctx_t *node_list, *node_list_tail;\n \n-static void reset_node_ctx(char *fname)\n+static struct node_ctx_t *new_node_ctx(char *fname)\n {\n-\tnode_ctx.type = 0;\n-\tnode_ctx.action = NODEACT_UNKNOWN;\n-\tnode_ctx.prop_length = -1;\n-\tnode_ctx.text_length = -1;\n-\tstrbuf_reset(&node_ctx.src);\n-\tnode_ctx.srcRev = 0;\n-\tstrbuf_reset(&node_ctx.dst);\n+\tstruct node_ctx_t *node = xmalloc(sizeof(struct node_ctx_t));\n+\ttrace_printf(\"new_node_ctx %p\\n\", node);\n+\tnode->type = 0;\n+\tnode->action = NODEACT_UNKNOWN;\n+\tnode->prop_length = -1;\n+\tnode->text_length = -1;\n+\tstrbuf_init(&node->src, 4096);\n+\tnode->srcRev = 0;\n+\tstrbuf_init(&node->dst, 4096);\n \tif (fname)\n-\t\tstrbuf_addstr(&node_ctx.dst, fname);\n-\tnode_ctx.text_delta = 0;\n-\tnode_ctx.prop_delta = 0;\n+\t\tstrbuf_addstr(&node->dst, fname);\n+\tnode->text_delta = 0;\n+\tnode->prop_delta = 0;\n+\tnode->dataref = NULL;\n+\tnode->next = NULL;\n+\treturn node;\n }\n \n-static void reset_rev_ctx(uint32_t revision)\n+static void free_node_ctx(struct node_ctx_t *node)\n {\n-\trev_ctx.revision = revision;\n-\trev_ctx.timestamp = 0;\n-\tstrbuf_reset(&rev_ctx.log);\n-\tstrbuf_reset(&rev_ctx.author);\n-\tstrbuf_reset(&rev_ctx.note);\n+\ttrace_printf(\"free_node_ctx %p\\n\", node);\n+\tstrbuf_release(&node->src);\n+\tstrbuf_release(&node->dst);\n+\tfree((char*)node->dataref);\n+\tfree(node);\n }\n \n-static void reset_dump_ctx(const char *url)\n+static void free_node_list(void)\n {\n-\tstrbuf_reset(&dump_ctx.url);\n+\tstruct node_ctx_t *p = node_list, *n;\n+\ttrace_printf(\"free_node_list head %p tail %p\\n\", node_list, node_list_tail);\n+\twhile (p) {\n+\t\tn = p->next;\n+\t\tfree_node_ctx(p);\n+\t\tp = n;\n+\t}\n+\tnode_list = node_list_tail = NULL;\n+}\n+\n+static void append_node_list(struct node_ctx_t *n)\n+{\n+\ttrace_printf(\"append_node_list %p head %p tail %p\\n\", n, node_list, node_list_tail);\n+\tif (!node_list)\n+\t\tnode_list = node_list_tail = n;\n+\telse {\n+\t\tnode_list_tail->next = n;\n+\t\tnode_list_tail = n;\n+\t}\n+}\n+\n+static void reset_rev_ctx(struct rev_ctx_t *rev, uint32_t revision)\n+{\n+\trev->revision = revision;\n+\trev->timestamp = 0;\n+\tstrbuf_reset(&rev->log);\n+\tstrbuf_reset(&rev->author);\n+\tstrbuf_reset(&rev->note);\n+}\n+\n+static void reset_dump_ctx(struct dump_ctx_t *dump, const char *url)\n+{\n+\tstrbuf_reset(&dump->url);\n \tif (url)\n-\t\tstrbuf_addstr(&dump_ctx.url, url);\n-\tdump_ctx.version = 1;\n-\tstrbuf_reset(&dump_ctx.uuid);\n+\t\tstrbuf_addstr(&dump->url, url);\n+\tdump->version = 1;\n+\tstrbuf_reset(&dump->uuid);\n }\n \n static void handle_property(const struct strbuf *key_buf,\n@@ -121,11 +160,11 @@ static void handle_property(const struct strbuf *key_buf,\n \t\t\tdie(\"invalid dump: sets type twice\");\n \t\t}\n \t\tif (!val) {\n-\t\t\tnode_ctx.type = REPO_MODE_BLB;\n+\t\t\tnode_ctx->type = REPO_MODE_BLB;\n \t\t\treturn;\n \t\t}\n \t\t*type_set = 1;\n-\t\tnode_ctx.type = keylen == strlen(\"svn:executable\") ?\n+\t\tnode_ctx->type = keylen == strlen(\"svn:executable\") ?\n \t\t\t\tREPO_MODE_EXE :\n \t\t\t\tREPO_MODE_LNK;\n \t}\n@@ -193,11 +232,11 @@ static void read_props(void)\n \t}\n }\n \n-static void handle_node(void)\n+static void handle_node(struct node_ctx_t *node)\n {\n-\tconst uint32_t type = node_ctx.type;\n-\tconst int have_props = node_ctx.prop_length != -1;\n-\tconst int have_text = node_ctx.text_length != -1;\n+\tconst uint32_t type = node->type;\n+\tconst int have_props = node->prop_length != -1;\n+\tconst int have_text = node->text_length != -1;\n \t/*\n \t * Old text for this node:\n \t *  NULL\t- directory or bug\n@@ -208,21 +247,21 @@ static void handle_node(void)\n \tconst char *old_data = NULL;\n \tuint32_t old_mode = REPO_MODE_BLB;\n \n-\tif (node_ctx.action == NODEACT_DELETE) {\n-\t\tif (have_text || have_props || node_ctx.srcRev)\n+\tif (node->action == NODEACT_DELETE) {\n+\t\tif (have_text || have_props || node->srcRev)\n \t\t\tdie(\"invalid dump: deletion node has \"\n \t\t\t\t\"copyfrom info, text, or properties\");\n-\t\trepo_delete(node_ctx.dst.buf);\n+\t\trepo_delete(node->dst.buf);\n \t\treturn;\n \t}\n-\tif (node_ctx.action == NODEACT_REPLACE) {\n-\t\trepo_delete(node_ctx.dst.buf);\n-\t\tnode_ctx.action = NODEACT_ADD;\n+\tif (node->action == NODEACT_REPLACE) {\n+\t\trepo_delete(node->dst.buf);\n+\t\tnode->action = NODEACT_ADD;\n \t}\n-\tif (node_ctx.srcRev) {\n-\t\trepo_copy(node_ctx.srcRev, node_ctx.src.buf, node_ctx.dst.buf);\n-\t\tif (node_ctx.action == NODEACT_ADD)\n-\t\t\tnode_ctx.action = NODEACT_CHANGE;\n+\tif (node->srcRev) {\n+\t\trepo_copy(node->srcRev, node->src.buf, node->dst.buf);\n+\t\tif (node->action == NODEACT_ADD)\n+\t\t\tnode->action = NODEACT_CHANGE;\n \t}\n \tif (have_text && type == REPO_MODE_DIR)\n \t\tdie(\"invalid dump: directories cannot have text attached\");\n@@ -230,20 +269,20 @@ static void handle_node(void)\n \t/*\n \t * Find old content (old_data) and decide on the new mode.\n \t */\n-\tif (node_ctx.action == NODEACT_CHANGE && !*node_ctx.dst.buf) {\n+\tif (node->action == NODEACT_CHANGE && !*node->dst.buf) {\n \t\tif (type != REPO_MODE_DIR)\n \t\t\tdie(\"invalid dump: root of tree is not a regular file\");\n \t\told_data = NULL;\n-\t} else if (node_ctx.action == NODEACT_CHANGE) {\n+\t} else if (node->action == NODEACT_CHANGE) {\n \t\tuint32_t mode;\n-\t\told_data = repo_read_path(node_ctx.dst.buf, &mode);\n+\t\told_data = repo_read_path(node->dst.buf, &mode); /* malloced buffer */\n \t\tif (mode == REPO_MODE_DIR && type != REPO_MODE_DIR)\n \t\t\tdie(\"invalid dump: cannot modify a directory into a file\");\n \t\tif (mode != REPO_MODE_DIR && type == REPO_MODE_DIR)\n \t\t\tdie(\"invalid dump: cannot modify a file into a directory\");\n-\t\tnode_ctx.type = mode;\n+\t\tnode->type = mode;\n \t\told_mode = mode;\n-\t} else if (node_ctx.action == NODEACT_ADD) {\n+\t} else if (node->action == NODEACT_ADD) {\n \t\tif (type == REPO_MODE_DIR)\n \t\t\told_data = NULL;\n \t\telse if (have_text)\n@@ -258,9 +297,9 @@ static void handle_node(void)\n \t * Adjust mode to reflect properties.\n \t */\n \tif (have_props) {\n-\t\tif (!node_ctx.prop_delta)\n-\t\t\tnode_ctx.type = type;\n-\t\tif (node_ctx.prop_length)\n+\t\tif (!node->prop_delta)\n+\t\t\tnode->type = type;\n+\t\tif (node->prop_length)\n \t\t\tread_props();\n \t}\n \n@@ -274,17 +313,17 @@ static void handle_node(void)\n \t\t/* For the fast_export_* functions, NULL means empty. */\n \t\told_data = NULL;\n \tif (!have_text) {\n-\t\tfast_export_modify(node_ctx.dst.buf, node_ctx.type, old_data);\n+\t\tfast_export_modify(node->dst.buf, node->type, old_data);\n \t\treturn;\n \t}\n-\tif (!node_ctx.text_delta) {\n-\t\tfast_export_modify(node_ctx.dst.buf, node_ctx.type, \"inline\");\n-\t\tfast_export_data(node_ctx.type, node_ctx.text_length, &input);\n+\tif (!node->text_delta) {\n+\t\tfast_export_modify(node->dst.buf, node->type, \"inline\");\n+\t\tfast_export_data(node->type, node->text_length, &input);\n \t\treturn;\n \t}\n-\tfast_export_modify(node_ctx.dst.buf, node_ctx.type, \"inline\");\n-\tfast_export_blob_delta(node_ctx.type, old_mode, old_data,\n-\t\t\t\tnode_ctx.text_length, &input);\n+\tfast_export_modify(node->dst.buf, node->type, \"inline\");\n+\tfast_export_blob_delta(node->type, old_mode, old_data,\n+\t\t\t\tnode->text_length, &input);\n }\n \n static void begin_revision(const char *remote_ref)\n@@ -316,7 +355,7 @@ void svndump_read(const char *url, const char *local_ref, const char *notes_ref)\n \tuint32_t active_ctx = DUMP_CTX;\n \tuint32_t len;\n \n-\treset_dump_ctx(url);\n+\treset_dump_ctx(&dump_ctx, url);\n \twhile ((t = buffer_read_line(&input))) {\n \t\tval = strchr(t, ':');\n \t\tif (!val)\n@@ -346,13 +385,13 @@ void svndump_read(const char *url, const char *local_ref, const char *notes_ref)\n \t\t\tif (constcmp(t, \"Revision-number\"))\n \t\t\t\tcontinue;\n \t\t\tif (active_ctx == NODE_CTX)\n-\t\t\t\thandle_node();\n+\t\t\t\thandle_node(node_ctx);\n \t\t\tif (active_ctx == REV_CTX)\n \t\t\t\tbegin_revision(local_ref);\n \t\t\tif (active_ctx != DUMP_CTX)\n \t\t\t\tend_revision(notes_ref);\n \t\t\tactive_ctx = REV_CTX;\n-\t\t\treset_rev_ctx(atoi(val));\n+\t\t\treset_rev_ctx(&rev_ctx, atoi(val));\n \t\t\tstrbuf_addf(&rev_ctx.note, \"%s\\n\", t);\n \t\t\tbreak;\n \t\tcase sizeof(\"Node-path\"):\n@@ -360,11 +399,11 @@ void svndump_read(const char *url, const char *local_ref, const char *notes_ref)\n \t\t\t\tcontinue;\n \t\t\tif (!constcmp(t + strlen(\"Node-\"), \"path\")) {\n \t\t\t\tif (active_ctx == NODE_CTX)\n-\t\t\t\t\thandle_node();\n+\t\t\t\t\thandle_node(node_ctx);\n \t\t\t\tif (active_ctx == REV_CTX)\n \t\t\t\t\tbegin_revision(local_ref);\n \t\t\t\tactive_ctx = NODE_CTX;\n-\t\t\t\treset_node_ctx(val);\n+\t\t\t\tnode_ctx = new_node_ctx(val);\n \t\t\t\tstrbuf_addf(&rev_ctx.note, \"%s\\n\", t);\n \t\t\t\tbreak;\n \t\t\t}\n@@ -372,9 +411,9 @@ void svndump_read(const char *url, const char *local_ref, const char *notes_ref)\n \t\t\t\tcontinue;\n \t\t\tstrbuf_addf(&rev_ctx.note, \"%s\\n\", t);\n \t\t\tif (!strcmp(val, \"dir\"))\n-\t\t\t\tnode_ctx.type = REPO_MODE_DIR;\n+\t\t\t\tnode_ctx->type = REPO_MODE_DIR;\n \t\t\telse if (!strcmp(val, \"file\"))\n-\t\t\t\tnode_ctx.type = REPO_MODE_BLB;\n+\t\t\t\tnode_ctx->type = REPO_MODE_BLB;\n \t\t\telse\n \t\t\t\tfprintf(stderr, \"Unknown node-kind: %s\\n\", val);\n \t\t\tbreak;\n@@ -383,29 +422,29 @@ void svndump_read(const char *url, const char *local_ref, const char *notes_ref)\n \t\t\t\tcontinue;\n \t\t\tstrbuf_addf(&rev_ctx.note, \"%s\\n\", t);\n \t\t\tif (!strcmp(val, \"delete\")) {\n-\t\t\t\tnode_ctx.action = NODEACT_DELETE;\n+\t\t\t\tnode_ctx->action = NODEACT_DELETE;\n \t\t\t} else if (!strcmp(val, \"add\")) {\n-\t\t\t\tnode_ctx.action = NODEACT_ADD;\n+\t\t\t\tnode_ctx->action = NODEACT_ADD;\n \t\t\t} else if (!strcmp(val, \"change\")) {\n-\t\t\t\tnode_ctx.action = NODEACT_CHANGE;\n+\t\t\t\tnode_ctx->action = NODEACT_CHANGE;\n \t\t\t} else if (!strcmp(val, \"replace\")) {\n-\t\t\t\tnode_ctx.action = NODEACT_REPLACE;\n+\t\t\t\tnode_ctx->action = NODEACT_REPLACE;\n \t\t\t} else {\n \t\t\t\tfprintf(stderr, \"Unknown node-action: %s\\n\", val);\n-\t\t\t\tnode_ctx.action = NODEACT_UNKNOWN;\n+\t\t\t\tnode_ctx->action = NODEACT_UNKNOWN;\n \t\t\t}\n \t\t\tbreak;\n \t\tcase sizeof(\"Node-copyfrom-path\"):\n \t\t\tif (constcmp(t, \"Node-copyfrom-path\"))\n \t\t\t\tcontinue;\n-\t\t\tstrbuf_reset(&node_ctx.src);\n-\t\t\tstrbuf_addstr(&node_ctx.src, val);\n+\t\t\tstrbuf_reset(&node_ctx->src);\n+\t\t\tstrbuf_addstr(&node_ctx->src, val);\n \t\t\tstrbuf_addf(&rev_ctx.note, \"%s\\n\", t);\n \t\t\tbreak;\n \t\tcase sizeof(\"Node-copyfrom-rev\"):\n \t\t\tif (constcmp(t, \"Node-copyfrom-rev\"))\n \t\t\t\tcontinue;\n-\t\t\tnode_ctx.srcRev = atoi(val);\n+\t\t\tnode_ctx->srcRev = atoi(val);\n \t\t\tstrbuf_addf(&rev_ctx.note, \"%s\\n\", t);\n \t\t\tbreak;\n \t\tcase sizeof(\"Text-content-length\"):\n@@ -424,19 +463,19 @@ void svndump_read(const char *url, const char *local_ref, const char *notes_ref)\n \t\t\t\t\tdie(\"unrepresentable length in dump: %s\", val);\n \n \t\t\t\tif (*t == 'T')\n-\t\t\t\t\tnode_ctx.text_length = (off_t) len;\n+\t\t\t\t\tnode_ctx->text_length = (off_t) len;\n \t\t\t\telse\n-\t\t\t\t\tnode_ctx.prop_length = (off_t) len;\n+\t\t\t\t\tnode_ctx->prop_length = (off_t) len;\n \t\t\t\tbreak;\n \t\t\t}\n \t\tcase sizeof(\"Text-delta\"):\n \t\t\tif (!constcmp(t, \"Text-delta\")) {\n-\t\t\t\tnode_ctx.text_delta = !strcmp(val, \"true\");\n+\t\t\t\tnode_ctx->text_delta = !strcmp(val, \"true\");\n \t\t\t\tbreak;\n \t\t\t}\n \t\t\tif (constcmp(t, \"Prop-delta\"))\n \t\t\t\tcontinue;\n-\t\t\tnode_ctx.prop_delta = !strcmp(val, \"true\");\n+\t\t\tnode_ctx->prop_delta = !strcmp(val, \"true\");\n \t\t\tbreak;\n \t\tcase sizeof(\"Content-length\"):\n \t\t\tif (constcmp(t, \"Content-length\"))\n@@ -450,7 +489,7 @@ void svndump_read(const char *url, const char *local_ref, const char *notes_ref)\n \t\t\tif (active_ctx == REV_CTX) {\n \t\t\t\tread_props();\n \t\t\t} else if (active_ctx == NODE_CTX) {\n-\t\t\t\thandle_node();\n+\t\t\t\thandle_node(node_ctx);\n \t\t\t\tactive_ctx = INTERNODE_CTX;\n \t\t\t} else {\n \t\t\t\tfprintf(stderr, \"Unexpected content length header: %\"PRIu32\"\\n\", len);\n@@ -462,7 +501,7 @@ void svndump_read(const char *url, const char *local_ref, const char *notes_ref)\n \tif (buffer_ferror(&input))\n \t\tdie_short_read();\n \tif (active_ctx == NODE_CTX)\n-\t\thandle_node();\n+\t\thandle_node(node_ctx);\n \tif (active_ctx == REV_CTX)\n \t\tbegin_revision(local_ref);\n \tif (active_ctx != DUMP_CTX)\n@@ -477,11 +516,10 @@ static void init(int report_fd)\n \tstrbuf_init(&rev_ctx.log, 4096);\n \tstrbuf_init(&rev_ctx.author, 4096);\n \tstrbuf_init(&rev_ctx.note, 4096);\n-\tstrbuf_init(&node_ctx.src, 4096);\n-\tstrbuf_init(&node_ctx.dst, 4096);\n-\treset_dump_ctx(NULL);\n-\treset_rev_ctx(0);\n-\treset_node_ctx(NULL);\n+\treset_dump_ctx(&dump_ctx, NULL);\n+\treset_rev_ctx(&rev_ctx, 0);\n+\tnode_ctx = new_node_ctx(NULL);\n+\tnode_list = node_list_tail = NULL;\n \treturn;\n }\n \n@@ -504,14 +542,12 @@ int svndump_init_fd(int in_fd, int back_fd)\n void svndump_deinit(void)\n {\n \tfast_export_deinit();\n-\treset_dump_ctx(NULL);\n-\treset_rev_ctx(0);\n-\treset_node_ctx(NULL);\n+\treset_dump_ctx(&dump_ctx, NULL);\n+\treset_rev_ctx(&rev_ctx, 0);\n \tstrbuf_release(&rev_ctx.log);\n \tstrbuf_release(&rev_ctx.author);\n \tstrbuf_release(&rev_ctx.note);\n-\tstrbuf_release(&node_ctx.src);\n-\tstrbuf_release(&node_ctx.dst);\n+\tfree_node_list();\n \tif (buffer_deinit(&input))\n \t\tfprintf(stderr, \"Input error\\n\");\n \tif (ferror(stdout))\n@@ -524,4 +560,5 @@ void svndump_reset(void)\n \tstrbuf_release(&dump_ctx.url);\n \tstrbuf_release(&rev_ctx.log);\n \tstrbuf_release(&rev_ctx.author);\n+\tfree_node_list();\n }\ndiff --git a/vcs-svn/svndump.h b/vcs-svn/svndump.h\nindex d545453..29794df 100644\n--- a/vcs-svn/svndump.h\n+++ b/vcs-svn/svndump.h\n@@ -13,6 +13,8 @@ struct node_ctx_t {\n \toff_t prop_length, text_length;\n \tstruct strbuf src, dst;\n \tuint32_t text_delta, prop_delta;\n+\tconst char *dataref;\n+\tstruct node_ctx_t *next;\n };\n \n struct rev_ctx_t {\n-- \n1.7.9.5\n"},{"id":"197983","messageId":"1346145304-27149-4-git-send-email-florian.achleitner.2.6.31@gmail.com","threadId":"31372","inReplyTo":"1346145304-27149-3-git-send-email-florian.achleitner.2.6.31@gmail.com","subject":"[RFC v2 3/4] vcs-svn/svndump: rewrite handle_node(), begin|end_revision()","fromName":"Florian Achleitner","fromEmail":"florian.achleitner.2.6.31@gmail.com","sentAt":"2012-08-28T09:15:03Z","receivedAt":"2012-08-28T09:15:03Z","isPatch":false,"sender":{"key":"florian.achleitner.2.6.31@gmail.com","avatar":"https://avatars.githubusercontent.com/u/880777?v=4"},"body":"Split the decision of what to do and actually doing it in\nhandle_node() to allow for detection of branches from svn nodes.\nSplit it into handle_node() and apply_node().\n\nsvn dumps are structured in revisions, which contain multiple nodes.\nNodes represent operations on data. Currently the function\nhandle_node() strongly mixes the interpretation of the node data with\nthe output of processed data to fast-import.\n\nIn a fast-import stream a commit object requires a branch name to\nwhich the new commit is added at its beginning.\n\nWe want to detect branches in svn. This can only be done by analyzing\nnode operations, like copyfrom. This conflicts with the current\nimplementation, where at the beginning of each new revision in the svn\ndump, a new commit on a hard-coded git branch is created, before even\nreading the first node.\n\nTo allow analyzing the nodes before deciding on which branch the\ncommit will be placed, store the node metadata of one complete\nrevision, and create a commit from it, when it ends.\n\nEach node can have file data appended. It's desirable to not store the\nactual file data, as it is unbounded.  fast-import has a 'blob'\ncommand that allows writing blobs, independent of commits. Use this\nfeature instead of sending data inline and send the actual file data\nimmediately when it is read in.\n\nUse marks to reference a blob later. fast-import's marks are currently\nused for marking commits, where the mark number corresponds to exactly\none svn revision.\nStore the marks for blobs in the upper half of the marks number space\nwhere the MSB is 1.\n\nChange handle_node() to interpret the node data, store it in a\nnode_ctx, send blobs to fast-import, and append the new node_ctx to\nthe list of node_ctx.  Do this until the end of a revision.\n\nJust clear the list of note_ctx in begin_revision().\n\nAt end_revision() all node metadata is available in the node_ctx list.\nFuture's branch detectors can decide what branches are to be changed.\nThen, call apply_node() for each of them to actually create a commit\nand change/add/delete files according to the node_ctx using the\nalready added blobs.\n\nThis can also be used to create commits if the node metadata does not\ncome from a svndump, but is stored in e.g. notes, for later branch\ndetection.\n\nSigned-off-by: Florian Achleitner <florian.achleitner.2.6.31@gmail.com>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\n vcs-svn/svndump.c |  167 ++++++++++++++++++++++++++++++++++-------------------\n 1 file changed, 109 insertions(+), 58 deletions(-)\n\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex 385523a..eb97e8e 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -48,7 +48,6 @@ static struct node_ctx_t *node_list, *node_list_tail;\n static struct node_ctx_t *new_node_ctx(char *fname)\n {\n \tstruct node_ctx_t *node = xmalloc(sizeof(struct node_ctx_t));\n-\ttrace_printf(\"new_node_ctx %p\\n\", node);\n \tnode->type = 0;\n \tnode->action = NODEACT_UNKNOWN;\n \tnode->prop_length = -1;\n@@ -67,7 +66,6 @@ static struct node_ctx_t *new_node_ctx(char *fname)\n \n static void free_node_ctx(struct node_ctx_t *node)\n {\n-\ttrace_printf(\"free_node_ctx %p\\n\", node);\n \tstrbuf_release(&node->src);\n \tstrbuf_release(&node->dst);\n \tfree((char*)node->dataref);\n@@ -77,7 +75,6 @@ static void free_node_ctx(struct node_ctx_t *node)\n static void free_node_list(void)\n {\n \tstruct node_ctx_t *p = node_list, *n;\n-\ttrace_printf(\"free_node_list head %p tail %p\\n\", node_list, node_list_tail);\n \twhile (p) {\n \t\tn = p->next;\n \t\tfree_node_ctx(p);\n@@ -88,7 +85,6 @@ static void free_node_list(void)\n \n static void append_node_list(struct node_ctx_t *n)\n {\n-\ttrace_printf(\"append_node_list %p head %p tail %p\\n\", n, node_list, node_list_tail);\n \tif (!node_list)\n \t\tnode_list = node_list_tail = n;\n \telse {\n@@ -246,23 +242,10 @@ static void handle_node(struct node_ctx_t *node)\n \tstatic const char *const empty_blob = \"::empty::\";\n \tconst char *old_data = NULL;\n \tuint32_t old_mode = REPO_MODE_BLB;\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tstatic uintmax_t blobmark = (uintmax_t) 1UL << (bitsizeof(uintmax_t) - 1);\n+\n \n-\tif (node->action == NODEACT_DELETE) {\n-\t\tif (have_text || have_props || node->srcRev)\n-\t\t\tdie(\"invalid dump: deletion node has \"\n-\t\t\t\t\"copyfrom info, text, or properties\");\n-\t\trepo_delete(node->dst.buf);\n-\t\treturn;\n-\t}\n-\tif (node->action == NODEACT_REPLACE) {\n-\t\trepo_delete(node->dst.buf);\n-\t\tnode->action = NODEACT_ADD;\n-\t}\n-\tif (node->srcRev) {\n-\t\trepo_copy(node->srcRev, node->src.buf, node->dst.buf);\n-\t\tif (node->action == NODEACT_ADD)\n-\t\t\tnode->action = NODEACT_CHANGE;\n-\t}\n \tif (have_text && type == REPO_MODE_DIR)\n \t\tdie(\"invalid dump: directories cannot have text attached\");\n \n@@ -270,28 +253,61 @@ static void handle_node(struct node_ctx_t *node)\n \t * Find old content (old_data) and decide on the new mode.\n \t */\n \tif (node->action == NODEACT_CHANGE && !*node->dst.buf) {\n+\t\t/*\n+\t\t * changes the root of the tree (empty dst), e.g. adding properties.\n+\t\t * see 9e8c5321\n+\t\t */\n \t\tif (type != REPO_MODE_DIR)\n-\t\t\tdie(\"invalid dump: root of tree is not a regular file\");\n+\t\t\tdie(\"invalid dump: root of tree is not a directory\");\n \t\told_data = NULL;\n \t} else if (node->action == NODEACT_CHANGE) {\n \t\tuint32_t mode;\n-\t\told_data = repo_read_path(node->dst.buf, &mode); /* malloced buffer */\n+\t\tif (fast_export_ls_rev(rev_ctx.revision - 1, node->dst.buf, &mode, &sb)) {\n+\t\t\tif (errno != ENOENT)\n+\t\t\t\tdie_errno(\"BUG: unexpected fast_export_ls error\");\n+\t\t\t/* Treat missing paths as directories. */\n+\t\t\tmode = REPO_MODE_DIR;\n+\t\t\told_data = NULL;\n+\t\t} else\n+\t\t\told_data = strbuf_detach(&sb, NULL);\n \t\tif (mode == REPO_MODE_DIR && type != REPO_MODE_DIR)\n-\t\t\tdie(\"invalid dump: cannot modify a directory into a file\");\n+\t\t\tdie(\"invalid dump: cannot modify a directory into a file: \"\n+\t\t\t\t\t\"%s. old_data %s\", node->dst.buf, old_data);\n \t\tif (mode != REPO_MODE_DIR && type == REPO_MODE_DIR)\n-\t\t\tdie(\"invalid dump: cannot modify a file into a directory\");\n+\t\t\tdie(\"invalid dump: cannot modify a file into a directory: %s\",\n+\t\t\t\t\tnode->dst.buf);\n \t\tnode->type = mode;\n \t\told_mode = mode;\n-\t} else if (node->action == NODEACT_ADD) {\n-\t\tif (type == REPO_MODE_DIR)\n+\t} else if (node->action == NODEACT_ADD || node->action == NODEACT_REPLACE) {\n+\t\tif (node->srcRev) {\t/* was copied */\n+\t\t\t/* read dataref and mode from src blob */\n+\t\t\tstrbuf_reset(&sb);\n+\t\t\tif (fast_export_ls_rev(node->srcRev, node->src.buf, &node->type, &sb)) {\n+\t\t\t\tif (errno != ENOENT)\n+\t\t\t\t\tdie_errno(\"BUG: unexpected fast_export_ls_rev error\");\n+\t\t\t\t/*\n+\t\t\t\t * if the dataref is not available, it may be a copy of an empty\n+\t\t\t\t * dir. We delete the target, and write it when the first file\n+\t\t\t\t * is added.\n+\t\t\t\t */\n+\t\t\t\tnode->action = NODEACT_DELETE;\n+\t\t\t\tnode->dataref = NULL;\n+\t\t\t} else\n+\t\t\t\tnode->dataref = strbuf_detach(&sb, NULL);\n+\t\t\told_data = NULL;\n+\t\t} else if (type == REPO_MODE_DIR)\n \t\t\told_data = NULL;\n \t\telse if (have_text)\n \t\t\told_data = empty_blob;\n \t\telse\n \t\t\tdie(\"invalid dump: adds node without text\");\n-\t} else {\n+\t} else if (node->action == NODEACT_DELETE) {\n+\t\told_data = empty_blob;\n+\t\tif (have_text || have_props || node->srcRev)\n+\t\t\tdie(\"invalid dump: deletion node has \"\n+\t\t\t\t\"copyfrom info, text, or properties\");\n+\t} else\n \t\tdie(\"invalid dump: Node-path block lacks Node-action\");\n-\t}\n \n \t/*\n \t * Adjust mode to reflect properties.\n@@ -304,48 +320,83 @@ static void handle_node(struct node_ctx_t *node)\n \t}\n \n \t/*\n-\t * Save the result.\n+\t * Send the data and save the node_ctx.\n \t */\n-\tif (type == REPO_MODE_DIR)\t/* directories are not tracked. */\n-\t\treturn;\n-\tassert(old_data);\n-\tif (old_data == empty_blob)\n-\t\t/* For the fast_export_* functions, NULL means empty. */\n-\t\told_data = NULL;\n-\tif (!have_text) {\n-\t\tfast_export_modify(node->dst.buf, node->type, old_data);\n-\t\treturn;\n+\tif (type != REPO_MODE_DIR) {\t/* directories are not tracked. */\n+\t\tassert(old_data);\n+\t\tif (old_data == empty_blob)\n+\t\t\t/* For the fast_export_* functions, NULL means empty. */\n+\t\t\told_data = NULL;\n+\t\tif (!have_text)\n+\t\t\tnode->dataref = old_data;\n+\t\telse {\n+\t\t\tif (!node->text_delta) {\n+\t\t\t\tprintf(\"blob\\n\"\n+\t\t\t\t\t\t\"mark :%\"PRIuMAX\"\\n\", ++blobmark);\n+\t\t\t\tfast_export_data(node->type, node->text_length, &input);\n+\t\t\t}\n+\t\t\telse {\n+\t\t\t\tprintf(\"blob\\n\"\n+\t\t\t\t\t\t\"mark :%\"PRIuMAX\"\\n\", ++blobmark);\n+\t\t\t\tfast_export_blob_delta(node->type, old_mode, old_data,\n+\t\t\t\t\t\tnode->text_length, &input);\n+\t\t\t}\n+\n+\t\t\tstrbuf_addf(&sb, \":%\"PRIuMAX, blobmark);\n+\t\t\tnode->dataref = sb.buf;\n+\t\t}\n \t}\n-\tif (!node->text_delta) {\n-\t\tfast_export_modify(node->dst.buf, node->type, \"inline\");\n-\t\tfast_export_data(node->type, node->text_length, &input);\n+\tappend_node_list(node);\n+}\n+\n+static void apply_node(struct node_ctx_t *node)\n+{\n+\tif (node->action == NODEACT_DELETE) {\n+\t\tfast_export_delete(node->dst.buf);\n \t\treturn;\n \t}\n-\tfast_export_modify(node->dst.buf, node->type, \"inline\");\n-\tfast_export_blob_delta(node->type, old_mode, old_data,\n-\t\t\t\tnode->text_length, &input);\n+\tif (node->action == NODEACT_REPLACE)\n+\t\tfast_export_delete(node->dst.buf);\n+\t/*\n+\t * apply the previously sent node-data to a commit\n+\t */\n+\tif (node->dataref)\n+\t\tfast_export_modify(node->dst.buf, node->type, node->dataref);\n }\n \n-static void begin_revision(const char *remote_ref)\n+\n+static void apply_node_list(void)\n {\n-\tif (!rev_ctx.revision)\t/* revision 0 gets no git commit. */\n-\t\treturn;\n-\tfast_export_begin_commit(rev_ctx.revision, rev_ctx.author.buf,\n-\t\t&rev_ctx.log, dump_ctx.uuid.buf, dump_ctx.url.buf,\n-\t\trev_ctx.timestamp, remote_ref);\n+\tstruct node_ctx_t *p = node_list, *n;\n+\twhile (p) {\n+\t\tn = p->next;\n+\t\tapply_node(p);\n+\t\tp = n;\n+\t}\n+}\n+\n+static void begin_revision(const char *remote_ref_)\n+{\n+\tcurrent_ref = remote_ref_;\n+\tfree_node_list();\n }\n \n static void end_revision(const char *note_ref)\n {\n \tstruct strbuf mark = STRBUF_INIT;\n-\tif (rev_ctx.revision) {\n-\t\tfast_export_end_commit(rev_ctx.revision);\n-\t\tfast_export_begin_note(rev_ctx.revision, \"remote-svn\",\n-\t\t\t\t\"Note created by remote-svn.\", rev_ctx.timestamp, note_ref);\n-\t\tstrbuf_addf(&mark, \":%\"PRIu32, rev_ctx.revision);\n-\t\tfast_export_note(mark.buf, \"inline\");\n-\t\tfast_export_buf_to_data(&rev_ctx.note);\n-\t}\n+\tif (!rev_ctx.revision)\t/* revision 0 gets no git commit. */\n+\t\treturn;\n+\tfast_export_begin_commit(rev_ctx.revision, rev_ctx.author.buf,\n+\t\t&rev_ctx.log, dump_ctx.uuid.buf, dump_ctx.url.buf,\n+\t\trev_ctx.timestamp, current_ref);\n+\tapply_node_list();\n+\tfast_export_end_commit(rev_ctx.revision);\n+\n+\tfast_export_begin_note(rev_ctx.revision, \"remote-svn\",\n+\t\t\t\"Note created by remote-svn.\", rev_ctx.timestamp, note_ref);\n+\tstrbuf_addf(&mark, \":%\"PRIu32, rev_ctx.revision);\n+\tfast_export_note(mark.buf, \"inline\");\n+\tfast_export_buf_to_data(&rev_ctx.note);\n }\n \n void svndump_read(const char *url, const char *local_ref, const char *notes_ref)\n-- \n1.7.9.5\n"},{"id":"197985","messageId":"1346145304-27149-5-git-send-email-florian.achleitner.2.6.31@gmail.com","threadId":"31372","inReplyTo":"1346145304-27149-4-git-send-email-florian.achleitner.2.6.31@gmail.com","subject":"[RFC v2 4/4] vcs-svn: remove repo_tree","fromName":"Florian Achleitner","fromEmail":"florian.achleitner.2.6.31@gmail.com","sentAt":"2012-08-28T09:15:04Z","receivedAt":"2012-08-28T09:15:04Z","isPatch":false,"sender":{"key":"florian.achleitner.2.6.31@gmail.com","avatar":"https://avatars.githubusercontent.com/u/880777?v=4"},"body":"Rewritten svndump.c left only very little functionality in repo_tree.c\nwhich could easily be inlined. Let's remove it.\n\nSigned-off-by: Florian Achleitner <florian.achleitner.2.6.31@gmail.com>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\n Makefile            |    1 -\n vcs-svn/repo_tree.c |   48 ------------------------------------------------\n vcs-svn/repo_tree.h |   14 --------------\n 3 files changed, 63 deletions(-)\n delete mode 100644 vcs-svn/repo_tree.c\n\ndiff --git a/Makefile b/Makefile\nindex 1b09454..5f72a65 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -2188,7 +2188,6 @@ XDIFF_OBJS += xdiff/xhistogram.o\n \n VCSSVN_OBJS += vcs-svn/line_buffer.o\n VCSSVN_OBJS += vcs-svn/sliding_window.o\n-VCSSVN_OBJS += vcs-svn/repo_tree.o\n VCSSVN_OBJS += vcs-svn/fast_export.o\n VCSSVN_OBJS += vcs-svn/svndiff.o\n VCSSVN_OBJS += vcs-svn/svndump.o\ndiff --git a/vcs-svn/repo_tree.c b/vcs-svn/repo_tree.c\ndeleted file mode 100644\nindex 67d27f0..0000000\n--- a/vcs-svn/repo_tree.c\n+++ /dev/null\n@@ -1,48 +0,0 @@\n-/*\n- * Licensed under a two-clause BSD-style license.\n- * See LICENSE for details.\n- */\n-\n-#include \"git-compat-util.h\"\n-#include \"strbuf.h\"\n-#include \"repo_tree.h\"\n-#include \"fast_export.h\"\n-\n-const char *repo_read_path(const char *path, uint32_t *mode_out)\n-{\n-\tint err;\n-\tstatic struct strbuf buf = STRBUF_INIT;\n-\n-\tstrbuf_reset(&buf);\n-\terr = fast_export_ls(path, mode_out, &buf);\n-\tif (err) {\n-\t\tif (errno != ENOENT)\n-\t\t\tdie_errno(\"BUG: unexpected fast_export_ls error\");\n-\t\t/* Treat missing paths as directories. */\n-\t\t*mode_out = REPO_MODE_DIR;\n-\t\treturn NULL;\n-\t}\n-\treturn buf.buf;\n-}\n-\n-void repo_copy(uint32_t revision, const char *src, const char *dst)\n-{\n-\tint err;\n-\tuint32_t mode;\n-\tstatic struct strbuf data = STRBUF_INIT;\n-\n-\tstrbuf_reset(&data);\n-\terr = fast_export_ls_rev(revision, src, &mode, &data);\n-\tif (err) {\n-\t\tif (errno != ENOENT)\n-\t\t\tdie_errno(\"BUG: unexpected fast_export_ls_rev error\");\n-\t\tfast_export_delete(dst);\n-\t\treturn;\n-\t}\n-\tfast_export_modify(dst, mode, data.buf);\n-}\n-\n-void repo_delete(const char *path)\n-{\n-\tfast_export_delete(path);\n-}\ndiff --git a/vcs-svn/repo_tree.h b/vcs-svn/repo_tree.h\nindex 889c6a3..6d0f51e 100644\n--- a/vcs-svn/repo_tree.h\n+++ b/vcs-svn/repo_tree.h\n@@ -1,23 +1,9 @@\n #ifndef REPO_TREE_H_\n #define REPO_TREE_H_\n \n-struct strbuf;\n-\n #define REPO_MODE_DIR 0040000\n #define REPO_MODE_BLB 0100644\n #define REPO_MODE_EXE 0100755\n #define REPO_MODE_LNK 0120000\n \n-uint32_t next_blob_mark(void);\n-void repo_copy(uint32_t revision, const char *src, const char *dst);\n-void repo_add(const char *path, uint32_t mode, uint32_t blob_mark);\n-const char *repo_read_path(const char *path, uint32_t *mode_out);\n-void repo_delete(const char *path);\n-void repo_commit(uint32_t revision, const char *author,\n-\t\tconst struct strbuf *log, const char *uuid, const char *url,\n-\t\tlong unsigned timestamp);\n-void repo_diff(uint32_t r1, uint32_t r2);\n-void repo_init(void);\n-void repo_reset(void);\n-\n #endif\n-- \n1.7.9.5\n"}]}