{"thread":{"id":"24069","subject":"[PATCH 0/6] Another attempt to get the SVN exporter merged","startedAt":"2010-06-10T13:09:43Z","lastAt":"2010-06-14T14:45:42Z","messageCount":19,"participants":["Ramkumar Ramachandra","Junio C Hamano","Jonathan Nieder","Andreas Ericsson","David Michael Barr"],"isPatch":true,"patchVersion":1,"patchTotal":6},"messages":[{"id":"143427","messageId":"1276175389-6185-1-git-send-email-artagnon@gmail.com","threadId":"24069","inReplyTo":null,"subject":"[PATCH 0/6] Another attempt to get the SVN exporter merged","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-10T13:09:43Z","receivedAt":"2010-06-10T13:09:43Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi,\n\nIt certainly looks like I'll never give up trying to get this series\nmerged- this is my third attempt :) Much thanks to David for all the\nawesome code, and to Jonathan for all the detailed timely\nreviews. Hopefully, this time the series will have fewer mistakes and\nwill actually be merged. Sadly, I still haven't been able to get rid\nof the compiler warnings about unused functions, and it looks like\nthis series won't graduate to `master` before that happens- I'd\nappreciate pointers on how to do this.\n\nPlease feel free to nitpick every little detail, but please don't use\nthat as an excuse to hold up the series. I'd appreciate some acks from\neveryone who has been involved with this to make it easier for Junio\nto decide.\n\nMajor change since last time: Removed dependency on mmap for\nportability reasons.\n\nThanks!\n\n-- Ram\n\nDavid Barr (5):\n  Add memory pool library\n  Add library for string-specific memory pool\n  Add stream helper library\n  Add infrastructure to write revisions in fast-export format\n  Add SVN dump parser\n\nJason Evans (1):\n  Add cpp macro implementation of treaps\n\n vcs-svn/fast_export.c |   74 +++++++++++\n vcs-svn/fast_export.h |   14 ++\n vcs-svn/line_buffer.c |  134 ++++++++++++++++++++\n vcs-svn/line_buffer.h |   14 ++\n vcs-svn/obj_pool.h    |   90 +++++++++++++\n vcs-svn/repo_tree.c   |  335 +++++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/repo_tree.h   |   26 ++++\n vcs-svn/string_pool.c |  116 +++++++++++++++++\n vcs-svn/string_pool.h |   15 +++\n vcs-svn/svndump.c     |  298 +++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/svndump.h     |    7 +\n vcs-svn/trp.h         |  201 +++++++++++++++++++++++++++++\n vcs-svn/trp.txt       |   61 +++++++++\n 13 files changed, 1385 insertions(+), 0 deletions(-)\n create mode 100644 vcs-svn/fast_export.c\n create mode 100644 vcs-svn/fast_export.h\n create mode 100644 vcs-svn/line_buffer.c\n create mode 100644 vcs-svn/line_buffer.h\n create mode 100644 vcs-svn/obj_pool.h\n create mode 100644 vcs-svn/repo_tree.c\n create mode 100644 vcs-svn/repo_tree.h\n create mode 100644 vcs-svn/string_pool.c\n create mode 100644 vcs-svn/string_pool.h\n create mode 100644 vcs-svn/svndump.c\n create mode 100644 vcs-svn/svndump.h\n create mode 100644 vcs-svn/trp.h\n create mode 100644 vcs-svn/trp.txt\n"},{"id":"143428","messageId":"1276175389-6185-2-git-send-email-artagnon@gmail.com","threadId":"24069","inReplyTo":"1276175389-6185-1-git-send-email-artagnon@gmail.com","subject":"[PATCH 1/6] Add memory pool library","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-10T13:09:44Z","receivedAt":"2010-06-10T13:09:44Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"From: David Barr <david.barr@cordelta.com>\n\nAdd a memory pool library implemented using cpp macros. The library\nprovides macros that can be used to create a type-specific memory pool\nAPI.\n\nThe memory pool library is distinguished from the existing specialized\nallocators in alloc.c by using a contiguous block for all allocations.\nThis means that on one hand, long-lived pointers have to be written as\noffsets, since the base address changes as the pool grows, but on the\nother hand, the entire pool can be easily written to the file system.\nThis allows the memory pool to persist between runs of an application.\n\nFor svn-fe, such a facility is useful because each svn revision can\ncopy trees and files from any previous revision.  Therefore the\nrelevant information for all revisions has to persist somehow to\nsupport incremental runs.\n\nSigned-off-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\n---\n vcs-svn/obj_pool.h |   90 ++++++++++++++++++++++++++++++++++++++++++++++++++++\n 1 files changed, 90 insertions(+), 0 deletions(-)\n create mode 100644 vcs-svn/obj_pool.h\n\ndiff --git a/vcs-svn/obj_pool.h b/vcs-svn/obj_pool.h\nnew file mode 100644\nindex 0000000..29417b3\n--- /dev/null\n+++ b/vcs-svn/obj_pool.h\n@@ -0,0 +1,90 @@\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#ifndef OBJ_POOL_H_\n+#define OBJ_POOL_H_\n+\n+#include \"git-compat-util.h\"\n+\n+/*\n+ * The obj_pool_gen() macro generates a type-specific memory pool\n+ * implementation.\n+ *\n+ * Arguments:\n+ *\n+ *   pre              : Prefix for generated functions (ex: string_).\n+ *   obj_t            : Type for treap data structure (ex: char).\n+ *   intial_capacity  : The initial size of the memory pool (ex: 4096).\n+ *\n+ */\n+#define obj_pool_gen(pre, obj_t, initial_capacity) \\\n+static struct { \\\n+\tuint32_t committed; \\\n+\tuint32_t size; \\\n+\tuint32_t capacity; \\\n+\tobj_t *base; \\\n+\tFILE *file; \\\n+} pre##_pool = { 0, 0, 0, NULL, NULL}; \\\n+static void pre##_init(void) \\\n+{ \\\n+\tstruct stat st; \\\n+\tpre##_pool.file = fopen(#pre \".bin\", \"a+\"); \\\n+\trewind(pre##_pool.file); \\\n+\tfstat(fileno(pre##_pool.file), &st); \\\n+\tpre##_pool.size = st.st_size / sizeof(obj_t); \\\n+\tpre##_pool.committed = pre##_pool.size; \\\n+\tpre##_pool.capacity = pre##_pool.size * 2; \\\n+\tif (pre##_pool.capacity < initial_capacity) \\\n+\t\tpre##_pool.capacity = initial_capacity; \\\n+\tpre##_pool.base = malloc(pre##_pool.capacity * sizeof(obj_t)); \\\n+\tfread(pre##_pool.base, sizeof(obj_t), pre##_pool.size, pre##_pool.file); \\\n+} \\\n+static uint32_t pre##_alloc(uint32_t count) \\\n+{ \\\n+\tuint32_t offset; \\\n+\tif (pre##_pool.size + count > pre##_pool.capacity) { \\\n+\t\twhile (pre##_pool.size + count > pre##_pool.capacity) \\\n+\t\t\tif (pre##_pool.capacity) \\\n+\t\t\t\tpre##_pool.capacity *= 2; \\\n+\t\t\telse \\\n+\t\t\t\tpre##_pool.capacity = initial_capacity; \\\n+\t\tpre##_pool.base = realloc(pre##_pool.base, \\\n+\t\t\t\t\tpre##_pool.capacity * sizeof(obj_t)); \\\n+\t} \\\n+\toffset = pre##_pool.size; \\\n+\tpre##_pool.size += count; \\\n+\treturn offset; \\\n+} \\\n+static void pre##_free(uint32_t count) \\\n+{ \\\n+\tpre##_pool.size -= count; \\\n+} \\\n+static uint32_t pre##_offset(obj_t *obj) \\\n+{ \\\n+\treturn obj == NULL ? ~0 : obj - pre##_pool.base; \\\n+} \\\n+static obj_t *pre##_pointer(uint32_t offset) \\\n+{ \\\n+\treturn offset >= pre##_pool.size ? NULL : &pre##_pool.base[offset]; \\\n+} \\\n+static void pre##_commit(void) \\\n+{ \\\n+\tpre##_pool.committed += fwrite(pre##_pool.base + pre##_pool.committed, \\\n+\t\tsizeof(obj_t), pre##_pool.size - pre##_pool.committed, \\\n+\t\tpre##_pool.file); \\\n+} \\\n+static void pre##_reset(void) \\\n+{ \\\n+\tif (pre##_pool.base) { \\\n+\t\tfree(pre##_pool.base); \\\n+\t\tfclose(pre##_pool.file); \\\n+\t} \\\n+\tpre##_pool.base = NULL; \\\n+\tpre##_pool.size = 0; \\\n+\tpre##_pool.capacity = 0; \\\n+\tpre##_pool.file = NULL; \\\n+}\n+\n+#endif\n-- \n1.7.1\n"},{"id":"143429","messageId":"1276175389-6185-3-git-send-email-artagnon@gmail.com","threadId":"24069","inReplyTo":"1276175389-6185-1-git-send-email-artagnon@gmail.com","subject":"[PATCH 2/6] Add cpp macro implementation of treaps","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-10T13:09:45Z","receivedAt":"2010-06-10T13:09:45Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"From: Jason Evans <jasone@canonware.com>\n\nThe implementation exposes an API to generate type-specific treap\nimplmentation and various functions to operate on it. It uses\nobj_pool.h to store memory nodes in a treap.\n\nTreaps provide a memory-efficient binary search tree structure.\nInsertion/deletion/search are about as about as fast in the average\ncase as red-black trees and the chances of worst-case behavior are\nvanishingly small, thanks to (pseudo-)randomness.  That is a small\nprice to pay, given that treaps are much simpler to implement.\n\n[db: Altered to reference nodes by offset from a common base pointer]\n[db: Bob Jenkins' hashing implementation dropped for Knuth's]\n[db: Methods unnecessary for search and insert dropped]\n\nSigned-off-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\n---\n vcs-svn/trp.h   |  201 +++++++++++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/trp.txt |   61 +++++++++++++++++\n 2 files changed, 262 insertions(+), 0 deletions(-)\n create mode 100644 vcs-svn/trp.h\n create mode 100644 vcs-svn/trp.txt\n\ndiff --git a/vcs-svn/trp.h b/vcs-svn/trp.h\nnew file mode 100644\nindex 0000000..50fde39\n--- /dev/null\n+++ b/vcs-svn/trp.h\n@@ -0,0 +1,201 @@\n+/*\n+ * cpp macro implementation of treaps.\n+ *\n+ * Usage:\n+ *   #include <stdint.h>\n+ *   #include <trp.h>\n+ *   trp_gen(...)\n+ *\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#ifndef TRP_H_\n+#define TRP_H_\n+\n+/* Node structure. */\n+struct trp_node {\n+\tuint32_t trpn_left;\n+\tuint32_t trpn_right;\n+};\n+\n+/* Root structure. */\n+struct trp_root {\n+\tuint32_t trp_root;\n+};\n+\n+/* Pointer/Offset conversion */\n+#define trpn_pointer(a_base, a_offset) (a_base##_pointer(a_offset))\n+#define trpn_offset(a_base, a_pointer) (a_base##_offset(a_pointer))\n+#define trpn_modify(a_base, a_offset) \\\n+\tdo { \\\n+\t\tif ((a_offset) < a_base##_pool.committed) { \\\n+\t\t\tuint32_t old_offset = (a_offset);\\\n+\t\t\t(a_offset) = a_base##_alloc(1); \\\n+\t\t\t*trpn_pointer(a_base, a_offset) = \\\n+\t\t\t\t*trpn_pointer(a_base, old_offset); \\\n+\t\t} \\\n+\t} while (0);\n+\n+/* Left accessors. */\n+#define trp_left_get(a_base, a_field, a_node) \\\n+\t(trpn_pointer(a_base, a_node)->a_field.trpn_left)\n+#define trp_left_set(a_base, a_field, a_node, a_left) \\\n+\tdo { trpn_modify(a_base, a_node); \\\n+\ttrp_left_get(a_base, a_field, a_node) = (a_left); } while(0)\n+\n+/* Right accessors. */\n+#define trp_right_get(a_base, a_field, a_node) \\\n+\t(trpn_pointer(a_base, a_node)->a_field.trpn_right)\n+#define trp_right_set(a_base, a_field, a_node, a_right) \\\n+\tdo { trpn_modify(a_base, a_node); \\\n+\ttrp_right_get(a_base, a_field, a_node) = (a_right); } while(0)\n+\n+/* Priority accessors. */\n+#define KNUTH_GOLDEN_RATIO_32BIT 2654435761u\n+#define trp_prio_get(a_node) \\\n+\t(KNUTH_GOLDEN_RATIO_32BIT*(a_node))\n+\n+/* Node initializer. */\n+#define trp_node_new(a_base, a_field, a_node) \\\n+\ttrp_left_set(a_base, a_field, (a_node), ~0); \\\n+\ttrp_right_set(a_base, a_field, (a_node), ~0)\n+\n+/* Internal utility macros. */\n+#define trpn_first(a_base, a_field, a_root, r_node) \\\n+\tdo { \\\n+\t\t(r_node) = (a_root); \\\n+\t\tif ((r_node) == ~0) \\\n+\t\t\treturn NULL; \\\n+\t\twhile (~trp_left_get(a_base, a_field, (r_node))) \\\n+\t\t\t(r_node) = trp_left_get(a_base, a_field, (r_node)); \\\n+\t} while (0)\n+\n+#define trpn_rotate_left(a_base, a_field, a_node, r_node) \\\n+\tdo { (r_node) = trp_right_get(a_base, a_field, (a_node)); \\\n+\ttrp_right_set(a_base, a_field, (a_node), \\\n+\t\ttrp_left_get(a_base, a_field, (r_node))); \\\n+\ttrp_left_set(a_base, a_field, (r_node), (a_node)); } while(0)\n+\n+#define trpn_rotate_right(a_base, a_field, a_node, r_node) \\\n+\tdo { (r_node) = trp_left_get(a_base, a_field, (a_node)); \\\n+\ttrp_left_set(a_base, a_field, (a_node), \\\n+\t\ttrp_right_get(a_base, a_field, (r_node))); \\\n+\ttrp_right_set(a_base, a_field, (r_node), (a_node)); } while(0)\n+\n+#define trp_gen(a_attr, a_pre, a_type, a_field, a_base, a_cmp) \\\n+a_attr a_type *a_pre##first(struct trp_root *treap) \\\n+{ \\\n+\tuint32_t ret; \\\n+\ttrpn_first(a_base, a_field, treap->trp_root, ret); \\\n+\treturn trpn_pointer(a_base, ret); \\\n+} \\\n+a_attr a_type *a_pre##next(struct trp_root *treap, a_type *node) { \\\n+\tuint32_t ret; \\\n+\tuint32_t offset = trpn_offset(a_base, node); \\\n+\tif (~trp_right_get(a_base, a_field, offset)) { \\\n+\t\ttrpn_first(a_base, a_field, \\\n+\t\t\ttrp_right_get(a_base, a_field, offset), ret); \\\n+\t} else { \\\n+\t\tuint32_t tnode = treap->trp_root; \\\n+\t\tret = ~0; \\\n+\t\twhile (1) { \\\n+\t\t\tint cmp = (a_cmp)(trpn_pointer(a_base, offset), \\\n+\t\t\t\ttrpn_pointer(a_base, tnode)); \\\n+\t\t\tif (cmp < 0) { \\\n+\t\t\t\tret = tnode; \\\n+\t\t\t\ttnode = trp_left_get(a_base, a_field, tnode); \\\n+\t\t\t} else if (cmp > 0) { \\\n+\t\t\t\ttnode = trp_right_get(a_base, a_field, tnode); \\\n+\t\t\t} else { \\\n+\t\t\t\tbreak; \\\n+\t\t\t} \\\n+\t\t} \\\n+\t} \\\n+\treturn trpn_pointer(a_base, ret); \\\n+} \\\n+a_attr a_type *a_pre##search(struct trp_root *treap, a_type *key) \\\n+{ \\\n+\tint cmp; \\\n+\tuint32_t ret = treap->trp_root; \\\n+\twhile (~ret && (cmp = (a_cmp)(key, trpn_pointer(a_base,ret)))) \\\n+\t\tif (cmp < 0) \\\n+\t\t\tret = trp_left_get(a_base, a_field, ret); \\\n+\t\telse \\\n+\t\t\tret = trp_right_get(a_base, a_field, ret); \\\n+\treturn trpn_pointer(a_base, ret); \\\n+} \\\n+a_attr uint32_t a_pre##insert_recurse(uint32_t cur_node, uint32_t ins_node) \\\n+{ \\\n+\tif (cur_node == ~0) \\\n+\t\treturn (ins_node); \\\n+\telse { \\\n+\t\tuint32_t ret; \\\n+\t\tint cmp = (a_cmp)(trpn_pointer(a_base, ins_node), \\\n+\t\t\t\t\ttrpn_pointer(a_base, cur_node)); \\\n+\t\tif (cmp < 0) { \\\n+\t\t\tuint32_t left = a_pre##insert_recurse( \\\n+\t\t\t\ttrp_left_get(a_base, a_field, cur_node), ins_node); \\\n+\t\t\ttrp_left_set(a_base, a_field, cur_node, left); \\\n+\t\t\tif (trp_prio_get(left) < trp_prio_get(cur_node)) \\\n+\t\t\t\ttrpn_rotate_right(a_base, a_field, cur_node, ret); \\\n+\t\t\telse \\\n+\t\t\t\tret = cur_node; \\\n+\t\t} else { \\\n+\t\t\tuint32_t right = a_pre##insert_recurse( \\\n+\t\t\t\ttrp_right_get(a_base, a_field, cur_node), ins_node); \\\n+\t\t\ttrp_right_set(a_base, a_field, cur_node, right); \\\n+\t\t\tif (trp_prio_get(right) < trp_prio_get(cur_node)) \\\n+\t\t\t\ttrpn_rotate_left(a_base, a_field, cur_node, ret); \\\n+\t\t\telse \\\n+\t\t\t\tret = cur_node; \\\n+\t\t} \\\n+\t\treturn (ret); \\\n+\t} \\\n+} \\\n+a_attr void a_pre##insert(struct trp_root *treap, a_type *node) \\\n+{ \\\n+\tuint32_t offset = trpn_offset(a_base, node); \\\n+\ttrp_node_new(a_base, a_field, offset); \\\n+\ttreap->trp_root = a_pre##insert_recurse( treap->trp_root, offset); \\\n+} \\\n+a_attr uint32_t a_pre##remove_recurse(uint32_t cur_node, uint32_t rem_node) \\\n+{ \\\n+\tint cmp = a_cmp(trpn_pointer(a_base, rem_node), \\\n+\t\t\ttrpn_pointer(a_base, cur_node)); \\\n+\tif (cmp == 0) { \\\n+\t\tuint32_t ret; \\\n+\t\tuint32_t left = trp_left_get(a_base, a_field, cur_node); \\\n+\t\tuint32_t right = trp_right_get(a_base, a_field, cur_node); \\\n+\t\tif (left == ~0) { \\\n+\t\t\tif (right == ~0) \\\n+\t\t\t\treturn (~0); \\\n+\t\t} else if (right == ~0 || trp_prio_get(left) < trp_prio_get(right)) { \\\n+\t\t\ttrpn_rotate_right(a_base, a_field, cur_node, ret); \\\n+\t\t\tright = a_pre##remove_recurse(cur_node, rem_node); \\\n+\t\t\ttrp_right_set(a_base, a_field, ret, right); \\\n+\t\t\treturn (ret); \\\n+\t\t} \\\n+\t\ttrpn_rotate_left(a_base, a_field, cur_node, ret); \\\n+\t\tleft = a_pre##remove_recurse(cur_node, rem_node); \\\n+\t\ttrp_left_set(a_base, a_field, ret, left); \\\n+\t\treturn (ret); \\\n+\t} else if (cmp < 0) { \\\n+\t\tuint32_t left = a_pre##remove_recurse( \\\n+\t\t\ttrp_left_get(a_base, a_field, cur_node), rem_node); \\\n+\t\ttrp_left_set(a_base, a_field, cur_node, left); \\\n+\t\treturn (cur_node); \\\n+\t} else { \\\n+\t\tuint32_t right = a_pre##remove_recurse( \\\n+\t\t\ttrp_right_get(a_base, a_field, cur_node), rem_node); \\\n+\t\ttrp_right_set(a_base, a_field, cur_node, right); \\\n+\t\treturn (cur_node); \\\n+\t} \\\n+} \\\n+a_attr void a_pre##remove(struct trp_root *treap, a_type *node) \\\n+{ \\\n+\ttreap->trp_root = a_pre##remove_recurse(treap->trp_root, \\\n+\t\ttrpn_offset(a_base, node)); \\\n+} \\\n+\n+#endif\ndiff --git a/vcs-svn/trp.txt b/vcs-svn/trp.txt\nnew file mode 100644\nindex 0000000..b7e2d18\n--- /dev/null\n+++ b/vcs-svn/trp.txt\n@@ -0,0 +1,61 @@\n+TODO: Update this documentation to match the changes to trp.h\n+\n+The trp_gen() macro generates a type-specific treap implementation,\n+based on the above cpp macros.\n+\n+Arguments:\n+\n+  a_attr     : Function attribute for generated functions (ex: static).\n+  a_pre      : Prefix for generated functions (ex: treap_).\n+  a_t_type   : Type for treap data structure (ex: treap_t).\n+  a_type     : Type for treap node data structure (ex: treap_node_t).\n+  a_field    : Name of treap node linkage (ex: treap_link).\n+  a_base     : Expression for the base pointer from which nodes are offset.\n+  a_cmp      : Node comparison function name, with the following prototype:\n+                 int (a_cmp *)(a_type *a_node, a_type *a_other);\n+                                       ^^^^^^\n+                                    or a_key\n+               Interpretation of comparision function return values:\n+                 -1 : a_node <  a_other\n+                  0 : a_node == a_other\n+                  1 : a_node >  a_other\n+               In all cases, the a_node or a_key macro argument is the first\n+               argument to the comparison function, which makes it possible\n+               to write comparison functions that treat the first argument\n+               specially.\n+\n+Assuming the following setup:\n+\n+  typedef struct ex_node_s ex_node_t;\n+  struct ex_node_s {\n+      trp_node(ex_node_t) ex_link;\n+  };\n+  typedef trp(ex_node_t) ex_t;\n+  static ex_node_t ex_base[MAX_NODES];\n+  trp_gen(static, ex_, ex_t, ex_node_t, ex_link, ex_base, ex_cmp)\n+\n+The following API is generated:\n+\n+  static void\n+  ex_new(ex_t *treap);\n+      Description: Initialize a treap structure.\n+      Args:\n+        treap: Pointer to an uninitialized treap object.\n+\n+  static ex_node_t *\n+  ex_psearch(ex_t *treap, ex_node_t *key);\n+      Description: Search for node that matches key.  If no match is found,\n+                   return what would be key's successor/predecessor, were\n+                   key in treap.\n+      Args:\n+        treap: Pointer to a initialized treap object.\n+        key  : Search key.\n+      Ret: Node in treap that matches key, or if no match, hypothetical\n+           node's successor/predecessor (NULL if no successor/predecessor).\n+\n+  static void\n+  ex_insert(ex_t *treap, ex_node_t *node);\n+      Description: Insert node into treap.\n+      Args:\n+        treap: Pointer to a initialized treap object.\n+        node : Node to be inserted into treap.\n-- \n1.7.1\n"},{"id":"143430","messageId":"1276175389-6185-4-git-send-email-artagnon@gmail.com","threadId":"24069","inReplyTo":"1276175389-6185-1-git-send-email-artagnon@gmail.com","subject":"[PATCH 3/6] Add library for string-specific memory pool","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-10T13:09:46Z","receivedAt":"2010-06-10T13:09:46Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"From: David Barr <david.barr@cordelta.com>\n\nThis library uses the macros in the obj_pool.h and trp.h to create a\nmemory pool for strings and expose an API for handling them.\n\nSigned-off-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\n---\n vcs-svn/string_pool.c |  116 +++++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/string_pool.h |   15 ++++++\n 2 files changed, 131 insertions(+), 0 deletions(-)\n create mode 100644 vcs-svn/string_pool.c\n create mode 100644 vcs-svn/string_pool.h\n\ndiff --git a/vcs-svn/string_pool.c b/vcs-svn/string_pool.c\nnew file mode 100644\nindex 0000000..523a6cc\n--- /dev/null\n+++ b/vcs-svn/string_pool.c\n@@ -0,0 +1,116 @@\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"git-compat-util.h\"\n+\n+#include \"trp.h\"\n+#include \"obj_pool.h\"\n+#include \"string_pool.h\"\n+\n+typedef struct node_s node_t;\n+static struct trp_root tree = { ~0 };\n+\n+struct node_s {\n+\tuint32_t offset;\n+\tstruct trp_node children;\n+};\n+\n+/* Create two memory pools: one for node_t, and another for strings */\n+obj_pool_gen(node, node_t, 4096);\n+obj_pool_gen(string, char, 4096);\n+\n+static char *node_value(node_t *node)\n+{\n+\treturn node ? string_pointer(node->offset) : NULL;\n+}\n+\n+static int node_cmp(node_t *a, node_t *b)\n+{\n+\treturn strcmp(node_value(a), node_value(b));\n+}\n+\n+/* Build a Treap from the node_s structure (a trp_node w/ offset) */\n+trp_gen(static, tree_, node_t, children, node, node_cmp);\n+\n+char *pool_fetch(uint32_t entry)\n+{\n+\treturn node_value(node_pointer(entry));\n+}\n+\n+uint32_t pool_intern(char *key)\n+{\n+\t/* Canonicalize key */\n+\tnode_t *match = NULL;\n+\tuint32_t key_len;\n+\tif (key == NULL)\n+\t\treturn ~0;\n+\tkey_len = strlen(key) + 1;\n+\tnode_t *node = node_pointer(node_alloc(1));\n+\tnode->offset = string_alloc(key_len);\n+\tstrcpy(node_value(node), key);\n+\tmatch = tree_search(&tree, node);\n+\tif (!match) {\n+\t\ttree_insert(&tree, node);\n+\t} else {\n+\t\tnode_free(1);\n+\t\tstring_free(key_len);\n+\t\tnode = match;\n+\t}\n+\treturn node_offset(node);\n+}\n+\n+uint32_t pool_tok_r(char *str, const char *delim, char **saveptr)\n+{\n+\tchar *token = strtok_r(str, delim, saveptr);\n+\treturn token ? pool_intern(token) : ~0;\n+}\n+\n+void pool_print_seq(uint32_t len, uint32_t *seq, char delim, FILE *stream)\n+{\n+\tuint32_t i;\n+\tfor (i = 0; i < len && ~seq[i]; i++) {\n+\t\tfputs(pool_fetch(seq[i]), stream);\n+\t\tif (i < len - 1 && ~seq[i + 1])\n+\t\t\tfputc(delim, stream);\n+\t}\n+}\n+\n+uint32_t pool_tok_seq(uint32_t max, uint32_t *seq, char *delim, char *str)\n+{\n+\tchar *context = NULL;\n+\tuint32_t length = 0, token = str ? pool_tok_r(str, delim, &context) : ~0;\n+\twhile (length < max) {\n+\t\tseq[length++] = token;\n+\t\tif (token == ~0)\n+\t\t\tbreak;\n+\t\ttoken = pool_tok_r(NULL, delim, &context);\n+\t}\n+\tseq[length ? length - 1 : 0] = ~0;\n+\treturn length;\n+}\n+\n+void pool_init(void)\n+{\n+\tuint32_t node;\n+\tuint32_t string = 0;\n+\tstring_init();\n+\twhile (string < string_pool.size) {\n+\t\tnode = node_alloc(1);\n+\t\tnode_pointer(node)->offset = string;\n+\t\ttree_insert(&tree, node_pointer(node));\n+\t\tstring += strlen(string_pointer(string)) + 1;\n+\t}\n+}\n+\n+void pool_commit(void)\n+{\n+\tstring_commit();\n+}\n+\n+void pool_reset(void)\n+{\n+\tnode_reset();\n+\tstring_reset();\n+}\ndiff --git a/vcs-svn/string_pool.h b/vcs-svn/string_pool.h\nnew file mode 100644\nindex 0000000..a69d165\n--- /dev/null\n+++ b/vcs-svn/string_pool.h\n@@ -0,1 +1,15 @@\n+#ifndef STRING_POOL_H_\n+#define STRING_POOL_H_\n+\n+#include \"git-compat-util.h\"\n+\n+uint32_t pool_intern(char *key);\n+char *pool_fetch(uint32_t entry);\n+uint32_t pool_tok_r(char *str, const char *delim, char **saveptr);\n+void pool_print_seq(uint32_t len, uint32_t *seq, char delim, FILE *stream);\n+uint32_t pool_tok_seq(uint32_t max, uint32_t *seq, char *delim, char *str);\n+void pool_init(void);\n+void pool_commit(void);\n+void pool_reset(void);\n+\n+#endif\n-- \n1.7.1\n"},{"id":"143431","messageId":"1276175389-6185-5-git-send-email-artagnon@gmail.com","threadId":"24069","inReplyTo":"1276175389-6185-1-git-send-email-artagnon@gmail.com","subject":"[PATCH 4/6] Add stream helper library","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-10T13:09:47Z","receivedAt":"2010-06-10T13:09:47Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"From: David Barr <david.barr@cordelta.com>\n\nThis library provides facilities to read streams into buffers. It\nmaintains a couple of static buffers and provides an API to use them.\n\nSigned-off-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\n---\n vcs-svn/line_buffer.c |  134 +++++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/line_buffer.h |   14 +++++\n 2 files changed, 148 insertions(+), 0 deletions(-)\n create mode 100644 vcs-svn/line_buffer.c\n create mode 100644 vcs-svn/line_buffer.h\n\ndiff --git a/vcs-svn/line_buffer.c b/vcs-svn/line_buffer.c\nnew file mode 100644\nindex 0000000..8ff1fb6\n--- /dev/null\n+++ b/vcs-svn/line_buffer.c\n@@ -0,0 +1,134 @@\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"git-compat-util.h\"\n+\n+#include \"line_buffer.h\"\n+#include \"obj_pool.h\"\n+\n+#define LINE_BUFFER_LEN 10000\n+#define COPY_BUFFER_LEN 4096\n+\n+/* Create memory pool for char sequence of known length */\n+obj_pool_gen(blob, char, 4096);\n+\n+static char line_buffer[LINE_BUFFER_LEN];\n+static char byte_buffer[COPY_BUFFER_LEN];\n+static uint32_t line_buffer_len = 0;\n+static uint32_t line_len = 0;\n+static FILE *infile;\n+\n+int buffer_init(char *filename)\n+{\n+\tinfile = fopen(filename, \"r\");\n+\tif(!infile)\n+\t\treturn 1;\n+\treturn 0;\n+}\n+\n+int buffer_deinit()\n+{\n+\tfclose(infile);\n+\treturn 0;\n+}\n+\n+char *buffer_read_line(void)\n+{\n+\tchar *end;\n+\tuint32_t n_read;\n+\n+\tif (line_len) {\n+\t\tmemmove(line_buffer, &line_buffer[line_len],\n+\t\t\tline_buffer_len - line_len);\n+\t\tline_buffer_len -= line_len;\n+\t\tline_len = 0;\n+\t}\n+\n+\tend = memchr(line_buffer, '\\n', line_buffer_len);\n+\twhile (line_buffer_len < LINE_BUFFER_LEN - 1 &&\n+\t       !feof(infile) && ferror(infile) && NULL == end) {\n+\t\tn_read = fread(&line_buffer[line_buffer_len], 1,\n+\t\t\t       LINE_BUFFER_LEN - 1 - line_buffer_len,\n+\t\t\t       infile);\n+\t\tend = memchr(&line_buffer[line_buffer_len], '\\n', n_read);\n+\t\tline_buffer_len += n_read;\n+\t}\n+\n+\tif (ferror(infile))\n+\t\treturn NULL;\n+\n+\tif (end != NULL) {\n+\t\tline_len = end - line_buffer;\n+\t\tline_buffer[line_len++] = '\\0';\n+\t} else {\n+\t\tline_len = line_buffer_len;\n+\t\tline_buffer[line_buffer_len] = '\\0';\n+\t}\n+\n+\tif (line_len == 0)\n+\t\treturn NULL;\n+\n+\treturn line_buffer;\n+}\n+\n+char *buffer_read_string(uint32_t len)\n+{\n+\tchar *s;\n+\tblob_free(blob_pool.size);\n+\ts = blob_pointer(blob_alloc(len + 1));\n+\tuint32_t offset = 0;\n+\tif (line_buffer_len > line_len) {\n+\t\toffset = line_buffer_len - line_len;\n+\t\tif (offset > len)\n+\t\t\toffset = len;\n+\t\tmemcpy(s, &line_buffer[line_len], offset);\n+\t\tline_len += offset;\n+\t}\n+\tif (offset < len)\n+\t\toffset += fread(&s[offset], 1, len - offset, infile);\n+\ts[offset] = '\\0';\n+\treturn s;\n+}\n+\n+void buffer_copy_bytes(uint32_t len)\n+{\n+\tuint32_t in;\n+\tif (line_buffer_len > line_len) {\n+\t\tin = line_buffer_len - line_len;\n+\t\tif (in > len)\n+\t\t\tin = len;\n+\t\tfwrite(&line_buffer[line_len], 1, in, stdout);\n+\t\tlen -= in;\n+\t\tline_len += in;\n+\t}\n+\twhile (len > 0 && !feof(infile)) {\n+\t\tin = len < COPY_BUFFER_LEN ? len : COPY_BUFFER_LEN;\n+\t\tin = fread(byte_buffer, 1, in, infile);\n+\t\tlen -= in;\n+\t\tfwrite(byte_buffer, 1, in, stdout);\n+\t}\n+}\n+\n+void buffer_skip_bytes(uint32_t len)\n+{\n+\tuint32_t in;\n+\tif (line_buffer_len > line_len) {\n+\t\tin = line_buffer_len - line_len;\n+\t\tif (in > len)\n+\t\t\tin = len;\n+\t\tline_len += in;\n+\t\tlen -= in;\n+\t}\n+\twhile (len > 0 && !feof(infile) && !ferror(infile)) {\n+\t\tin = len < COPY_BUFFER_LEN ? len : COPY_BUFFER_LEN;\n+\t\tin = fread(byte_buffer, 1, in, infile);\n+\t\tlen -= in;\n+\t}\n+}\n+\n+void buffer_reset(void)\n+{\n+\tblob_reset();\n+}\ndiff --git a/vcs-svn/line_buffer.h b/vcs-svn/line_buffer.h\nnew file mode 100644\nindex 0000000..a6c42d7\n--- /dev/null\n+++ b/vcs-svn/line_buffer.h\n@@ -0,1 +1,14 @@\n+#ifndef LINE_BUFFER_H_\n+#define LINE_BUFFER_H_\n+\n+#include \"git-compat-util.h\"\n+\n+int buffer_init(char *filename);\n+int buffer_deinit(void);\n+char *buffer_read_line(void);\n+char *buffer_read_string(uint32_t len);\n+void buffer_copy_bytes(uint32_t len);\n+void buffer_skip_bytes(uint32_t len);\n+void buffer_reset(void);\n+\n+#endif\n-- \n1.7.1\n"},{"id":"143433","messageId":"1276175389-6185-6-git-send-email-artagnon@gmail.com","threadId":"24069","inReplyTo":"1276175389-6185-1-git-send-email-artagnon@gmail.com","subject":"[PATCH 5/6] Add infrastructure to write revisions in fast-export format","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-10T13:09:48Z","receivedAt":"2010-06-10T13:09:48Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"From: David Barr <david.barr@cordelta.com>\n\nrepo_tree maintains the exporter's state and provides a facility to\nto call fast_export, which then writes objects to stdout suitable for\nconsumption by git-fast-import.\n\nSigned-off-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\n---\n vcs-svn/fast_export.c |   74 +++++++++++\n vcs-svn/fast_export.h |   14 ++\n vcs-svn/repo_tree.c   |  335 +++++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/repo_tree.h   |   26 ++++\n 4 files changed, 449 insertions(+), 0 deletions(-)\n create mode 100644 vcs-svn/fast_export.c\n create mode 100644 vcs-svn/fast_export.h\n create mode 100644 vcs-svn/repo_tree.c\n create mode 100644 vcs-svn/repo_tree.h\n\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nnew file mode 100644\nindex 0000000..0fe6672\n--- /dev/null\n+++ b/vcs-svn/fast_export.c\n@@ -0,0 +1,74 @@\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"git-compat-util.h\"\n+\n+#include \"fast_export.h\"\n+#include \"line_buffer.h\"\n+#include \"repo_tree.h\"\n+#include \"string_pool.h\"\n+\n+#define MAX_GITSVN_LINE_LEN 4096\n+\n+static uint32_t first_commit_done;\n+\n+void fast_export_delete(uint32_t depth, uint32_t *path)\n+{\n+\tputchar('D');\n+\tputchar(' ');\n+\tpool_print_seq(depth, path, '/', stdout);\n+\tputchar('\\n');\n+}\n+\n+void fast_export_modify(uint32_t depth, uint32_t *path, uint32_t mode,\n+\t\t\t\t\t\tuint32_t mark)\n+{\n+\tprintf(\"M %06o :%d \", mode, mark);\n+\tpool_print_seq(depth, path, '/', stdout);\n+\tputchar('\\n');\n+}\n+\n+static char gitsvnline[MAX_GITSVN_LINE_LEN];\n+void fast_export_commit(uint32_t revision, uint32_t author, char *log,\n+\t\t\tuint32_t uuid, uint32_t url,\n+\t\t\tunsigned long timestamp)\n+{\n+\tif (!log)\n+\t\tlog = \"\";\n+\tif (~uuid && ~url) {\n+\t\tsnprintf(gitsvnline, MAX_GITSVN_LINE_LEN, \"\\n\\ngit-svn-id: %s@%d %s\\n\",\n+\t\t\t\t pool_fetch(url), revision, pool_fetch(uuid));\n+\t} else {\n+\t\t*gitsvnline = '\\0';\n+\t}\n+\tprintf(\"commit refs/heads/master\\n\");\n+\tprintf(\"committer %s <%s@%s> %ld +0000\\n\",\n+\t\t   ~author ? pool_fetch(author) : \"nobody\",\n+\t\t   ~author ? pool_fetch(author) : \"nobody\",\n+\t\t   ~uuid ? pool_fetch(uuid) : \"local\", timestamp);\n+\tprintf(\"data %zd\\n%s%s\\n\",\n+\t\t   strlen(log) + strlen(gitsvnline), log, gitsvnline);\n+\tif (!first_commit_done) {\n+\t\tif (revision > 1)\n+\t\t\tprintf(\"from refs/heads/master^0\\n\");\n+\t\tfirst_commit_done = 1;\n+\t}\n+\trepo_diff(revision - 1, revision);\n+\tfputc('\\n', stdout);\n+\n+\tprintf(\"progress Imported commit %d.\\n\\n\", revision);\n+}\n+\n+void fast_export_blob(uint32_t mode, uint32_t mark, uint32_t len)\n+{\n+\tif (mode == REPO_MODE_LNK) {\n+\t\t/* svn symlink blobs start with \"link \" */\n+\t\tbuffer_skip_bytes(5);\n+\t\tlen -= 5;\n+\t}\n+\tprintf(\"blob\\nmark :%d\\ndata %d\\n\", mark, len);\n+\tbuffer_copy_bytes(len);\n+\tfputc('\\n', stdout);\n+}\ndiff --git a/vcs-svn/fast_export.h b/vcs-svn/fast_export.h\nnew file mode 100644\nindex 0000000..47e8f56\n--- /dev/null\n+++ b/vcs-svn/fast_export.h\n@@ -0,0 +1,14 @@\n+#ifndef FAST_EXPORT_H_\n+#define FAST_EXPORT_H_\n+\n+#include <stdint.h>\n+#include <time.h>\n+\n+void fast_export_delete(uint32_t depth, uint32_t *path);\n+void fast_export_modify(uint32_t depth, uint32_t *path, uint32_t mode,\n+\t\t\tuint32_t mark);\n+void fast_export_commit(uint32_t revision, uint32_t author, char *log,\n+\t\t\tuint32_t uuid, uint32_t url, unsigned long timestamp);\n+void fast_export_blob(uint32_t mode, uint32_t mark, uint32_t len);\n+\n+#endif\ndiff --git a/vcs-svn/repo_tree.c b/vcs-svn/repo_tree.c\nnew file mode 100644\nindex 0000000..6565779\n--- /dev/null\n+++ b/vcs-svn/repo_tree.c\n@@ -0,0 +1,335 @@\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"git-compat-util.h\"\n+\n+#include \"string_pool.h\"\n+#include \"repo_tree.h\"\n+#include \"obj_pool.h\"\n+#include \"fast_export.h\"\n+\n+#include \"trp.h\"\n+\n+struct repo_dirent {\n+\tuint32_t name_offset;\n+\tstruct trp_node children;\n+\tuint32_t mode;\n+\tuint32_t content_offset;\n+};\n+\n+struct repo_dir {\n+\tstruct trp_root entries;\n+};\n+\n+struct repo_commit {\n+\tuint32_t root_dir_offset;\n+};\n+\n+/* Generate memory pools for commit, dir and dirent */\n+obj_pool_gen(commit, struct repo_commit, 4096);\n+obj_pool_gen(dir, struct repo_dir, 4096);\n+obj_pool_gen(dirent, struct repo_dirent, 4096);\n+\n+static int repo_dirent_name_cmp(const void *a, const void *b);\n+\n+/* Build a Treap from the node_s structure (a trp_node w/ offset) */\n+trp_gen(static, dirent_, struct repo_dirent, children, dirent, repo_dirent_name_cmp);\n+\n+static uint32_t active_commit;\n+static uint32_t _mark;\n+\n+uint32_t next_blob_mark(void)\n+{\n+\treturn _mark++;\n+}\n+\n+static struct repo_dir *repo_commit_root_dir(struct repo_commit *commit)\n+{\n+\treturn dir_pointer(commit->root_dir_offset);\n+}\n+\n+static struct repo_dirent *repo_first_dirent(struct repo_dir *dir)\n+{\n+\treturn dirent_first(&dir->entries);\n+}\n+\n+static int repo_dirent_name_cmp(const void *a, const void *b)\n+{\n+\tconst struct repo_dirent *dirent1 = a, *dirent2 = b;\n+\tuint32_t a_offset = dirent1->name_offset;\n+\tuint32_t b_offset = dirent2->name_offset;\n+\treturn (a_offset > b_offset) - (a_offset < b_offset);\n+}\n+\n+static int repo_dirent_is_dir(struct repo_dirent *dirent)\n+{\n+\treturn dirent != NULL && dirent->mode == REPO_MODE_DIR;\n+}\n+\n+static struct repo_dir *repo_dir_from_dirent(struct repo_dirent *dirent)\n+{\n+\tif (!repo_dirent_is_dir(dirent))\n+\t\treturn NULL;\n+\treturn dir_pointer(dirent->content_offset);\n+}\n+\n+static struct repo_dir *repo_clone_dir(struct repo_dir *orig_dir)\n+{\n+\tuint32_t orig_o, new_o;\n+\torig_o = dir_offset(orig_dir);\n+\tif (orig_o >= dir_pool.committed)\n+\t\treturn orig_dir;\n+\tnew_o = dir_alloc(1);\n+\torig_dir = dir_pointer(orig_o);\n+\t*dir_pointer(new_o) = *orig_dir;\n+\treturn dir_pointer(new_o);\n+}\n+\n+static struct repo_dirent *repo_read_dirent(uint32_t revision, uint32_t *path)\n+{\n+\tuint32_t name = 0;\n+\tstruct repo_dirent *key = dirent_pointer(dirent_alloc(1));\n+\tstruct repo_dir *dir = NULL;\n+\tstruct repo_dirent *dirent = NULL;\n+\tdir = repo_commit_root_dir(commit_pointer(revision));\n+\twhile (~(name = *path++)) {\n+\t\tkey->name_offset = name;\n+\t\tdirent = dirent_search(&dir->entries, key);\n+\t\tif (dirent == NULL || !repo_dirent_is_dir(dirent))\n+\t\t\tbreak;\n+\t\tdir = repo_dir_from_dirent(dirent);\n+\t}\n+\tdirent_free(1);\n+\treturn dirent;\n+}\n+\n+static void\n+repo_write_dirent(uint32_t *path, uint32_t mode, uint32_t content_offset,\n+\t\t  uint32_t del)\n+{\n+\tuint32_t name, revision, dir_o = ~0, parent_dir_o = ~0;\n+\tstruct repo_dir *dir;\n+\tstruct repo_dirent *key;\n+\tstruct repo_dirent *dirent = NULL;\n+\trevision = active_commit;\n+\tdir = repo_commit_root_dir(commit_pointer(revision));\n+\tdir = repo_clone_dir(dir);\n+\tcommit_pointer(revision)->root_dir_offset = dir_offset(dir);\n+\twhile (~(name = *path++)) {\n+\t\tparent_dir_o = dir_offset(dir);\n+\n+\t\tkey = dirent_pointer(dirent_alloc(1));\n+\t\tkey->name_offset = name;\n+\n+\t\tdirent = dirent_search(&dir->entries, key);\n+\t\tif (dirent == NULL)\n+\t\t\tdirent = key;\n+\t\telse\n+\t\t\tdirent_free(1);\n+\n+\t\tif (dirent == key) {\n+\t\t\tdirent->mode = REPO_MODE_DIR;\n+\t\t\tdirent->content_offset = 0;\n+\t\t\tdirent_insert(&dir->entries, dirent);\n+\t\t}\n+\n+\n+\t\tif (dirent_offset(dirent) < dirent_pool.committed) {\n+\t\t\tdir_o = repo_dirent_is_dir(dirent) ? dirent->content_offset : ~0;\n+\t\t\tdirent_remove(&dir->entries, dirent);\n+\t\t\tdirent = dirent_pointer(dirent_alloc(1));\n+\t\t\tdirent->name_offset = name;\n+\t\t\tdirent->mode = REPO_MODE_DIR;\n+\t\t\tdirent->content_offset = dir_o;\n+\t\t\tdirent_insert(&dir->entries, dirent);\n+\t\t}\n+\n+\t\tdir = repo_dir_from_dirent(dirent);\n+\t\tdir = repo_clone_dir(dir);\n+\t\tdirent->content_offset = dir_offset(dir);\n+\t}\n+\tif (dirent == NULL)\n+\t\treturn;\n+\tdirent->mode = mode;\n+\tdirent->content_offset = content_offset;\n+\tif (del && ~parent_dir_o)\n+\t\tdirent_remove(&dir_pointer(parent_dir_o)->entries, dirent);\n+}\n+\n+uint32_t repo_copy(uint32_t revision, uint32_t *src, uint32_t *dst)\n+{\n+\tuint32_t mode = 0, content_offset = 0;\n+\tstruct repo_dirent *src_dirent;\n+\tsrc_dirent = repo_read_dirent(revision, src);\n+\tif (src_dirent != NULL) {\n+\t\tmode = src_dirent->mode;\n+\t\tcontent_offset = src_dirent->content_offset;\n+\t\trepo_write_dirent(dst, mode, content_offset, 0);\n+\t}\n+\treturn mode;\n+}\n+\n+void repo_add(uint32_t *path, uint32_t mode, uint32_t blob_mark)\n+{\n+\trepo_write_dirent(path, mode, blob_mark, 0);\n+}\n+\n+uint32_t repo_replace(uint32_t *path, uint32_t blob_mark)\n+{\n+\tuint32_t mode = 0;\n+\tstruct repo_dirent *src_dirent;\n+\tsrc_dirent = repo_read_dirent(active_commit, path);\n+\tif (src_dirent != NULL) {\n+\t\tmode = src_dirent->mode;\n+\t\trepo_write_dirent(path, mode, blob_mark, 0);\n+\t}\n+\treturn mode;\n+}\n+\n+void repo_modify(uint32_t *path, uint32_t mode, uint32_t blob_mark)\n+{\n+\tstruct repo_dirent *src_dirent;\n+\tsrc_dirent = repo_read_dirent(active_commit, path);\n+\tif (src_dirent != NULL && blob_mark == 0) {\n+\t\tblob_mark = src_dirent->content_offset;\n+\t}\n+\trepo_write_dirent(path, mode, blob_mark, 0);\n+}\n+\n+void repo_delete(uint32_t *path)\n+{\n+\trepo_write_dirent(path, 0, 0, 1);\n+}\n+\n+static void repo_git_add_r(uint32_t depth, uint32_t *path, struct repo_dir *dir);\n+\n+static void repo_git_add(uint32_t depth, uint32_t *path, struct repo_dirent *dirent)\n+{\n+\tif (repo_dirent_is_dir(dirent)) {\n+\t\trepo_git_add_r(depth, path, repo_dir_from_dirent(dirent));\n+\t} else {\n+\t\tfast_export_modify(depth, path, dirent->mode, dirent->content_offset);\n+\t}\n+}\n+\n+static void repo_git_add_r(uint32_t depth, uint32_t *path, struct repo_dir *dir)\n+{\n+\tstruct repo_dirent *de = repo_first_dirent(dir);\n+\twhile (de) {\n+\t\tpath[depth] = de->name_offset;\n+\t\trepo_git_add(depth + 1, path, de);\n+\t\tde = dirent_next(&dir->entries, de);\n+\t}\n+}\n+\n+static void repo_diff_r(uint32_t depth, uint32_t *path, struct repo_dir *dir1,\n+\t\t\tstruct repo_dir *dir2)\n+{\n+\tstruct repo_dirent *de1, *de2;\n+\tde1 = repo_first_dirent(dir1);\n+\tde2 = repo_first_dirent(dir2);\n+\n+\twhile (de1 && de2) {\n+\t\tif (de1->name_offset < de2->name_offset) {\n+\t\t\tpath[depth] = de1->name_offset;\n+\t\t\tfast_export_delete(depth + 1, path);\n+\t\t\tde1 = dirent_next(&dir1->entries, de1);\n+\t\t\tcontinue;\n+\t\t} else if (de1->name_offset > de2->name_offset) {\n+\t\t\tpath[depth] = de2->name_offset;\n+\t\t\trepo_git_add(depth + 1, path, de2);\n+\t\t\tde2 = dirent_next(&dir2->entries, de2);\n+\t\t\tcontinue;\n+\t\t}\n+\t\tpath[depth] = de1->name_offset;\n+\t\tif (de1->mode != de2->mode ||\n+\t\t    de1->content_offset != de2->content_offset) {\n+\t\t\tif (repo_dirent_is_dir(de1) && repo_dirent_is_dir(de2)) {\n+\t\t\t\trepo_diff_r(depth + 1, path,\n+\t\t\t\t\t    repo_dir_from_dirent(de1),\n+\t\t\t\t\t    repo_dir_from_dirent(de2));\n+\t\t\t} else {\n+\t\t\t\tif (repo_dirent_is_dir(de1) != repo_dirent_is_dir(de2)) {\n+\t\t\t\t\tfast_export_delete(depth + 1, path);\n+\t\t\t\t}\n+\t\t\t\trepo_git_add(depth + 1, path, de2);\n+\t\t\t}\n+\t\t}\n+\t\tde1 = dirent_next(&dir1->entries, de1);\n+\t\tde2 = dirent_next(&dir2->entries, de2);\n+\t}\n+\twhile (de1) {\n+\t\tpath[depth] = de1->name_offset;\n+\t\tfast_export_delete(depth + 1, path);\n+\t\tde1 = dirent_next(&dir1->entries, de1);\n+\t}\n+\twhile (de2) {\n+\t\tpath[depth] = de2->name_offset;\n+\t\trepo_git_add(depth + 1, path, de2);\n+\t\tde2 = dirent_next(&dir2->entries, de2);\n+\t}\n+}\n+\n+static uint32_t path_stack[REPO_MAX_PATH_DEPTH];\n+\n+void repo_diff(uint32_t r1, uint32_t r2)\n+{\n+\trepo_diff_r(0,\n+\t\t    path_stack,\n+\t\t    repo_commit_root_dir(commit_pointer(r1)),\n+\t\t    repo_commit_root_dir(commit_pointer(r2)));\n+}\n+\n+void repo_commit(uint32_t revision, uint32_t author, char *log, uint32_t uuid,\n+\t\t uint32_t url, unsigned long timestamp)\n+{\n+\tfast_export_commit(revision, author, log, uuid, url, timestamp);\n+\tpool_commit();\n+\tdirent_commit();\n+\tdir_commit();\n+\tcommit_commit();\n+\tactive_commit = commit_alloc(1);\n+\tcommit_pointer(active_commit)->root_dir_offset =\n+\t\tcommit_pointer(active_commit - 1)->root_dir_offset;\n+}\n+\n+static void mark_init(void)\n+{\n+\tuint32_t i;\n+\t_mark = 0;\n+\tfor (i = 0; i < dirent_pool.size; i++)\n+\t\tif (!repo_dirent_is_dir(dirent_pointer(i)) &&\n+\t\t    dirent_pointer(i)->content_offset > _mark)\n+\t\t\t_mark = dirent_pointer(i)->content_offset;\n+\t_mark++;\n+}\n+\n+void repo_init() {\n+\tpool_init();\n+\tcommit_init();\n+\tdir_init();\n+\tdirent_init();\n+\tmark_init();\n+\tif (commit_pool.size == 0) {\n+\t\t/* Create empty tree for commit 0. */\n+\t\tcommit_alloc(1);\n+\t\tcommit_pointer(0)->root_dir_offset = dir_alloc(1);\n+\t\tdir_pointer(0)->entries.trp_root = ~0;\n+\t\tdir_commit();\n+\t\tcommit_commit();\n+\t}\n+\t/* Preallocate next commit, ready for changes. */\n+\tactive_commit = commit_alloc(1);\n+\tcommit_pointer(active_commit)->root_dir_offset =\n+\t\tcommit_pointer(active_commit - 1)->root_dir_offset;\n+}\n+\n+void repo_reset(void)\n+{\n+\tpool_reset();\n+\tcommit_reset();\n+\tdir_reset();\n+\tdirent_reset();\n+}\ndiff --git a/vcs-svn/repo_tree.h b/vcs-svn/repo_tree.h\nnew file mode 100644\nindex 0000000..5476175\n--- /dev/null\n+++ b/vcs-svn/repo_tree.h\n@@ -0,0 +1,26 @@\n+#ifndef REPO_TREE_H_\n+#define REPO_TREE_H_\n+\n+#include \"git-compat-util.h\"\n+\n+#define REPO_MODE_DIR 0040000\n+#define REPO_MODE_BLB 0100644\n+#define REPO_MODE_EXE 0100755\n+#define REPO_MODE_LNK 0120000\n+\n+#define REPO_MAX_PATH_LEN 4096\n+#define REPO_MAX_PATH_DEPTH 1000\n+\n+uint32_t next_blob_mark(void);\n+uint32_t repo_copy(uint32_t revision, uint32_t *src, uint32_t *dst);\n+void repo_add(uint32_t *path, uint32_t mode, uint32_t blob_mark);\n+uint32_t repo_replace(uint32_t *path, uint32_t blob_mark);\n+void repo_modify(uint32_t *path, uint32_t mode, uint32_t blob_mark);\n+void repo_delete(uint32_t *path);\n+void repo_commit(uint32_t revision, uint32_t author, char *log, uint32_t uuid,\n+\t\t uint32_t url, long unsigned timestamp);\n+void repo_diff(uint32_t r1, uint32_t r2);\n+void repo_init(void);\n+void repo_reset(void);\n+\n+#endif\n-- \n1.7.1\n"},{"id":"143432","messageId":"1276175389-6185-7-git-send-email-artagnon@gmail.com","threadId":"24069","inReplyTo":"1276175389-6185-1-git-send-email-artagnon@gmail.com","subject":"[PATCH 6/6] Add SVN dump parser","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-10T13:09:49Z","receivedAt":"2010-06-10T13:09:49Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"From: David Barr <david.barr@cordelta.com>\n\nsvndump parses data that is in SVN dumpfile format produced by\n`svnadmin dump` with the help of line_buffer, and uses repo_tree and\nfast_export to emit a git fast-import stream.\n\nSigned-off-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\n---\n vcs-svn/svndump.c |  298 +++++++++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/svndump.h |    7 ++\n 2 files changed, 305 insertions(+), 0 deletions(-)\n create mode 100644 vcs-svn/svndump.c\n create mode 100644 vcs-svn/svndump.h\n\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nnew file mode 100644\nindex 0000000..92c15d7\n--- /dev/null\n+++ b/vcs-svn/svndump.c\n@@ -0,0 +1,298 @@\n+/*\n+ * Parse and rearrange a svnadmin dump.\n+ * Create the dump with:\n+ * svnadmin dump --incremental -r<startrev>:<endrev> <repository> >outfile\n+ *\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"cache.h\"\n+#include \"git-compat-util.h\"\n+\n+#include \"repo_tree.h\"\n+#include \"fast_export.h\"\n+#include \"line_buffer.h\"\n+#include \"obj_pool.h\"\n+#include \"string_pool.h\"\n+\n+#define NODEACT_REPLACE 4\n+#define NODEACT_DELETE 3\n+#define NODEACT_ADD 2\n+#define NODEACT_CHANGE 1\n+#define NODEACT_UNKNOWN 0\n+\n+#define DUMP_CTX 0\n+#define REV_CTX  1\n+#define NODE_CTX 2\n+\n+#define LENGTH_UNKNOWN (~0)\n+#define DATE_RFC2822_LEN 31\n+\n+/* Create memory pool for log messages */\n+obj_pool_gen(log, char, 4096);\n+\n+static char* log_copy(uint32_t length, char *log)\n+{\n+\tchar *buffer;\n+\tlog_free(log_pool.size);\n+\tbuffer = log_pointer(log_alloc(length));\n+\tstrncpy(buffer, log, length);\n+\treturn buffer;\n+}\n+\n+static struct {\n+\tuint32_t action, propLength, textLength, srcRev, srcMode, mark, type;\n+\tuint32_t src[REPO_MAX_PATH_DEPTH], dst[REPO_MAX_PATH_DEPTH];\n+} node_ctx;\n+\n+static struct {\n+\tuint32_t revision, author;\n+\tunsigned long timestamp;\n+\tchar *log;\n+} rev_ctx;\n+\n+static struct {\n+\tuint32_t uuid, url;\n+} dump_ctx;\n+\n+static struct {\n+\tuint32_t svn_log, svn_author, svn_date, svn_executable, svn_special, uuid,\n+\t\trevision_number, node_path, node_kind, node_action,\n+\t\tnode_copyfrom_path, node_copyfrom_rev, text_content_length,\n+\t\tprop_content_length, content_length;\n+} keys;\n+\n+static void reset_node_ctx(char *fname)\n+{\n+\tnode_ctx.type = 0;\n+\tnode_ctx.action = NODEACT_UNKNOWN;\n+\tnode_ctx.propLength = LENGTH_UNKNOWN;\n+\tnode_ctx.textLength = LENGTH_UNKNOWN;\n+\tnode_ctx.src[0] = ~0;\n+\tnode_ctx.srcRev = 0;\n+\tnode_ctx.srcMode = 0;\n+\tpool_tok_seq(REPO_MAX_PATH_DEPTH, node_ctx.dst, \"/\", fname);\n+\tnode_ctx.mark = 0;\n+}\n+\n+static void reset_rev_ctx(uint32_t revision)\n+{\n+\trev_ctx.revision = revision;\n+\trev_ctx.timestamp = \"\";\n+\trev_ctx.log = NULL;\n+\trev_ctx.author = ~0;\n+}\n+\n+static void reset_dump_ctx(uint32_t url)\n+{\n+\tdump_ctx.url = url;\n+\tdump_ctx.uuid = ~0;\n+}\n+\n+static void init_keys(void)\n+{\n+\tkeys.svn_log = pool_intern(\"svn:log\");\n+\tkeys.svn_author = pool_intern(\"svn:author\");\n+\tkeys.svn_date = pool_intern(\"svn:date\");\n+\tkeys.svn_executable = pool_intern(\"svn:executable\");\n+\tkeys.svn_special = pool_intern(\"svn:special\");\n+\tkeys.uuid = pool_intern(\"UUID\");\n+\tkeys.revision_number = pool_intern(\"Revision-number\");\n+\tkeys.node_path = pool_intern(\"Node-path\");\n+\tkeys.node_kind = pool_intern(\"Node-kind\");\n+\tkeys.node_action = pool_intern(\"Node-action\");\n+\tkeys.node_copyfrom_path = pool_intern(\"Node-copyfrom-path\");\n+\tkeys.node_copyfrom_rev = pool_intern(\"Node-copyfrom-rev\");\n+\tkeys.text_content_length = pool_intern(\"Text-content-length\");\n+\tkeys.prop_content_length = pool_intern(\"Prop-content-length\");\n+\tkeys.content_length = pool_intern(\"Content-length\");\n+}\n+\n+static void read_props(void)\n+{\n+\tuint32_t len;\n+\tuint32_t key = ~0;\n+\tchar buffer[27];\n+\tchar *val = NULL;\n+\tchar *t;\n+\twhile ((t = buffer_read_line()) && strcmp(t, \"PROPS-END\")) {\n+\t\tif (!strncmp(t, \"K \", 2)) {\n+\t\t\tlen = atoi(&t[2]);\n+\t\t\tkey = pool_intern(buffer_read_string(len));\n+\t\t\tbuffer_read_line();\n+\t\t} else if (!strncmp(t, \"V \", 2)) {\n+\t\t\tlen = atoi(&t[2]);\n+\t\t\tval = buffer_read_string(len);\n+\t\t\tif (key == keys.svn_log) {\n+\t\t\t\t/* Value length excludes terminating nul. */\n+\t\t\t\trev_ctx.log = log_copy(len + 1, val);\n+\t\t\t} else if (key == keys.svn_author) {\n+\t\t\t\trev_ctx.author = pool_intern(val);\n+\t\t\t} else if (key == keys.svn_date) {\n+\t\t\t\tif (parse_date(val, buffer, sizeof(buffer)) > 0)\n+\t\t\t\t\trev_ctx.timestamp = strtoul(buffer, NULL, 0);\n+\t\t\t\telse\n+\t\t\t\t\tfprintf(stderr, \"Invalid timestamp: %s\", val);\n+\t\t\t} else if (key == keys.svn_executable) {\n+\t\t\t\tnode_ctx.type = REPO_MODE_EXE;\n+\t\t\t} else if (key == keys.svn_special) {\n+\t\t\t\tnode_ctx.type = REPO_MODE_LNK;\n+\t\t\t}\n+\t\t\tkey = ~0;\n+\t\t\tbuffer_read_line();\n+\t\t}\n+\t}\n+}\n+\n+static void handle_node(void)\n+{\n+\tif (node_ctx.propLength != LENGTH_UNKNOWN && node_ctx.propLength) {\n+\t\tread_props();\n+\t}\n+\n+\tif (node_ctx.srcRev) {\n+\t\tnode_ctx.srcMode = repo_copy(node_ctx.srcRev, node_ctx.src, node_ctx.dst);\n+\t}\n+\n+\tif (node_ctx.textLength != LENGTH_UNKNOWN &&\n+\t\tnode_ctx.type != REPO_MODE_DIR) {\n+\t\tnode_ctx.mark = next_blob_mark();\n+\t}\n+\n+\tif (node_ctx.action == NODEACT_DELETE) {\n+\t\trepo_delete(node_ctx.dst);\n+\t} else if (node_ctx.action == NODEACT_CHANGE ||\n+\t\t\t   node_ctx.action == NODEACT_REPLACE) {\n+\t\tif (node_ctx.action == NODEACT_REPLACE &&\n+\t\t\tnode_ctx.type == REPO_MODE_DIR) {\n+\t\t\trepo_replace(node_ctx.dst, node_ctx.mark);\n+\t\t} else if (node_ctx.propLength != LENGTH_UNKNOWN ) {\n+\t\t\trepo_modify(node_ctx.dst, node_ctx.type, node_ctx.mark);\n+\t\t} else if (node_ctx.textLength != LENGTH_UNKNOWN) {\n+\t\t\tnode_ctx.srcMode = repo_replace(node_ctx.dst, node_ctx.mark);\n+\t\t}\n+\t} else if (node_ctx.action == NODEACT_ADD) {\n+\t\tif (node_ctx.srcRev &&\n+\t\t\tnode_ctx.propLength != LENGTH_UNKNOWN) {\n+\t\t\trepo_modify(node_ctx.dst, node_ctx.type, node_ctx.mark);\n+\t\t} else if (node_ctx.srcRev &&\n+\t\t\tnode_ctx.textLength != LENGTH_UNKNOWN) {\n+\t\t\tnode_ctx.srcMode = repo_replace(node_ctx.dst, node_ctx.mark);\n+\t\t} else if ((node_ctx.type == REPO_MODE_DIR && !node_ctx.srcRev) ||\n+\t\t\t\t   node_ctx.textLength != LENGTH_UNKNOWN){\n+\t\t\trepo_add(node_ctx.dst, node_ctx.type, node_ctx.mark);\n+\t\t}\n+\t}\n+\n+\tif (node_ctx.propLength == LENGTH_UNKNOWN && node_ctx.srcMode) {\n+\t\tnode_ctx.type = node_ctx.srcMode;\n+\t}\n+\n+\tif (node_ctx.mark) {\n+\t\tfast_export_blob(node_ctx.type, node_ctx.mark, node_ctx.textLength);\n+\t} else if (node_ctx.textLength != LENGTH_UNKNOWN) {\n+\t\tbuffer_skip_bytes(node_ctx.textLength);\n+\t}\n+}\n+\n+static void handle_revision(void)\n+{\n+\tif (rev_ctx.revision)\n+\t\trepo_commit(rev_ctx.revision, rev_ctx.author, rev_ctx.log,\n+\t\t\tdump_ctx.uuid, dump_ctx.url, rev_ctx.timestamp);\n+}\n+\n+void svndump_read(uint32_t url)\n+{\n+\tchar *val;\n+\tchar *t;\n+\tuint32_t active_ctx = DUMP_CTX;\n+\tuint32_t len;\n+\tuint32_t key;\n+\n+\treset_dump_ctx(url);\n+\twhile ((t = buffer_read_line())) {\n+\t\tval = strstr(t, \": \");\n+\t\tif (!val) continue;\n+\t\t*val++ = '\\0';\n+\t\t*val++ = '\\0';\n+\t\tkey = pool_intern(t);\n+\n+\t\tif(key == keys.uuid) {\n+\t\t\tdump_ctx.uuid = pool_intern(val);\n+\t\t} else if (key == keys.revision_number) {\n+\t\t\tif (active_ctx == NODE_CTX) handle_node();\n+\t\t\tif (active_ctx != DUMP_CTX) handle_revision();\n+\t\t\tactive_ctx = REV_CTX;\n+\t\t\treset_rev_ctx(atoi(val));\n+\t\t} else if (key == keys.node_path) {\n+\t\t\tif (active_ctx == NODE_CTX)\n+\t\t\t\thandle_node();\n+\t\t\tactive_ctx = NODE_CTX;\n+\t\t\treset_node_ctx(val);\n+\t\t} else if (key == keys.node_kind) {\n+\t\t\tif (!strcmp(val, \"dir\")) {\n+\t\t\t\tnode_ctx.type = REPO_MODE_DIR;\n+\t\t\t} else if (!strcmp(val, \"file\")) {\n+\t\t\t\tnode_ctx.type = REPO_MODE_BLB;\n+\t\t\t} else {\n+\t\t\t\tfprintf(stderr, \"Unknown node-kind: %s\\n\", val);\n+\t\t\t}\n+\t\t} else if (key == keys.node_action) {\n+\t\t\tif (!strcmp(val, \"delete\")) {\n+\t\t\t\tnode_ctx.action = NODEACT_DELETE;\n+\t\t\t} else if (!strcmp(val, \"add\")) {\n+\t\t\t\tnode_ctx.action = NODEACT_ADD;\n+\t\t\t} else if (!strcmp(val, \"change\")) {\n+\t\t\t\tnode_ctx.action = NODEACT_CHANGE;\n+\t\t\t} else if (!strcmp(val, \"replace\")) {\n+\t\t\t\tnode_ctx.action = NODEACT_REPLACE;\n+\t\t\t} else {\n+\t\t\t\tfprintf(stderr, \"Unknown node-action: %s\\n\", val);\n+\t\t\t\tnode_ctx.action = NODEACT_UNKNOWN;\n+\t\t\t}\n+\t\t} else if (key == keys.node_copyfrom_path) {\n+\t\t\tpool_tok_seq(REPO_MAX_PATH_DEPTH, node_ctx.src, \"/\", val);\n+\t\t} else if (key == keys.node_copyfrom_rev) {\n+\t\t\tnode_ctx.srcRev = atoi(val);\n+\t\t} else if (key == keys.text_content_length) {\n+\t\t\tnode_ctx.textLength = atoi(val);\n+\t\t} else if (key == keys.prop_content_length) {\n+\t\t\tnode_ctx.propLength = atoi(val);\n+\t\t} else if (key == keys.content_length) {\n+\t\t\tlen = atoi(val);\n+\t\t\tbuffer_read_line();\n+\t\t\tif (active_ctx == REV_CTX) {\n+\t\t\t\tread_props();\n+\t\t\t} else if (active_ctx == NODE_CTX) {\n+\t\t\t\thandle_node();\n+\t\t\t\tactive_ctx = REV_CTX;\n+\t\t\t} else {\n+\t\t\t\tfprintf(stderr, \"Unexpected content length header: %d\\n\", len);\n+\t\t\t\tbuffer_skip_bytes(len);\n+\t\t\t}\n+\t\t}\n+\t}\n+\tif (active_ctx == NODE_CTX) handle_node();\n+\tif (active_ctx != DUMP_CTX) handle_revision();\n+}\n+\n+static void svndump_init(void)\n+{\n+\trepo_init();\n+\treset_dump_ctx(~0);\n+\treset_rev_ctx(0);\n+\treset_node_ctx(NULL);\n+\tinit_keys();\n+}\n+\n+void svndump_reset(void)\n+{\n+\tlog_reset();\n+\tbuffer_reset();\n+\trepo_reset();\n+\treset_dump_ctx(~0);\n+\treset_rev_ctx(0);\n+\treset_node_ctx(NULL);\n+}\ndiff --git a/vcs-svn/svndump.h b/vcs-svn/svndump.h\nnew file mode 100644\nindex 0000000..e205f1f\n--- /dev/null\n+++ b/vcs-svn/svndump.h\n@@ -0,0 +1,7 @@\n+#ifndef SVNDUMP_H_\n+#define SVNDUMP_H_\n+\n+void svndump_read(char *url);\n+void svndump_reset(void);\n+\n+#endif\n-- \n1.7.1\n"},{"id":"143434","messageId":"AANLkTinw7j3MHQ_4Xw7axWWZUIWubJZSKrBCgKIzc2uN@mail.gmail.com","threadId":"24069","inReplyTo":"AANLkTin3iQK7YHGgjxlAjtchu3ZpntjQHK7LkfxxJj6q@mail.gmail.com","subject":"Re: [PATCH 0/6] Another attempt to get the SVN exporter merged","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-10T13:22:45Z","receivedAt":"2010-06-10T13:22:45Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi Ævar,\n\nÆvar Arnfjörð Bjarmason wrote:\n> I'd like to test this out but funneling patches into git-am with GMail\n> is a pain. Is it available in a public Git repository somewhere?\n\nThank you for bringing this up- I'm aware that others might be\ninterested as well. I've rolled out the patches from the 'git-merge'\nbranch, but you can test the independent project from the 'master'\nbranch which includes a Makefile and small manpage (courtesy Jonathan)\nas well [1].\n\n-- Ram\n\n[1] http://github.com/artagnon/svn-dump-fast-export\n"},{"id":"143443","messageId":"AANLkTinyIvN2cMerCD4oI-1DsvsDLBfhvUZy0-e6dmpz@mail.gmail.com","threadId":"24069","inReplyTo":"1276175389-6185-7-git-send-email-artagnon@gmail.com","subject":"Re: [PATCH 6/6] Add SVN dump parser","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-10T15:24:33Z","receivedAt":"2010-06-10T15:24:33Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi,\n\n> +void svndump_reset(void)\n> +{\n> +       log_reset();\n> +       buffer_reset();\n> +       repo_reset();\n> +       reset_dump_ctx(~0);\n> +       reset_rev_ctx(0);\n> +       reset_node_ctx(NULL);\n> +}\n\nThanks to Ævar for pointing this out- kindly read the patch with this\ndiff squashed in:\n\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex 92c15d7..516a520 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -289,9 +289,6 @@ static void svndump_init(void)\n\n void svndump_reset(void)\n {\n-\tlog_reset();\n-\tbuffer_reset();\n-\trepo_reset();\n \treset_dump_ctx(~0);\n \treset_rev_ctx(0);\n \treset_node_ctx(NULL);\n\n-- Ram\n"},{"id":"143527","messageId":"7vk4q5ictl.fsf@alter.siamese.dyndns.org","threadId":"24069","inReplyTo":"1276175389-6185-4-git-send-email-artagnon@gmail.com","subject":"Re: [PATCH 3/6] Add library for string-specific memory pool","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-06-11T19:33:42Z","receivedAt":"2010-06-11T19:33:42Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ramkumar Ramachandra <artagnon@gmail.com> writes:\n\n> diff --git a/vcs-svn/string_pool.h b/vcs-svn/string_pool.h\n> new file mode 100644\n> index 0000000..a69d165\n> --- /dev/null\n> +++ b/vcs-svn/string_pool.h\n> @@ -0,1 +1,15 @@\n\nHow did you manage to get \"-0,1\" here?  This is supposed to be a new file.\nPATCH 4/6 throws the same puzzlement at me.\n"},{"id":"143551","messageId":"20100612062613.GA2549@burratino","threadId":"24069","inReplyTo":"1276175389-6185-1-git-send-email-artagnon@gmail.com","subject":"Re: [PATCH 0/6] Another attempt to get the SVN exporter merged","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-06-12T06:26:13Z","receivedAt":"2010-06-12T06:26:13Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi Ram,\n\nRamkumar Ramachandra wrote:\n\n> It certainly looks like I'll never give up trying to get this series\n> merged- this is my third attempt\n\nFirst of all, thanks for your work on this and sorry to take so long\nto respond[1].\n\n> I still haven't been able to get rid\n> of the compiler warnings about unused functions\n\nWill reply to the relevant patch.  Since it’s not so bad to fix, I don’t\nthink that should hold up queueing the series in pu.\n\n> it looks like\n> this series won't graduate to `master` before that happens- I'd\n> appreciate pointers on how to do this.\n\nInstead, I suspect the main missing ingredient is a caller.  The\nseries builds up some library infrastructure with no new feature to\nexercise it, which makes it a good time to get feedback but not a good\ntime to merge.\n\nOf course, we could easily add a feature to exercise it: David’s\nsvn-fe tool is IMO quite useful on its own (regardless of how the code\nwill be used later by remote-svn).\n\nI don’t think that is so important for the remote-svn project.  In\nother words, I hope some other reviewers show up, but if that doesn’t\nhappen, I would suggest submitting again for inclusion once the\nremote-svn command with import capability is functional.\n\nI planned to write a series adding svn-fe to contrib/ and then the\ncomputer died.  If someone else doesn’t do it first, hopefully I can\ntry again this weekend.\n\n> Major change since last time: Removed dependency on mmap for\n> portability reasons.\n\nSad but perhaps necessary.  I guess this also opens the possibility of\nlater making the save operation atomic with the fsync() + rename()\ntrick.\n\nRegards,\nJonathan\n\n[1] My excuse: laptop died.  My new setup is finally in place and\nworking well but this Monday I will be on vacation in a land of poor\nconnectivity --- agh.\n"},{"id":"143554","messageId":"20100612064250.GB2549@burratino","threadId":"24069","inReplyTo":"1276175389-6185-2-git-send-email-artagnon@gmail.com","subject":"Re: [PATCH 1/6] Add memory pool library","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-06-12T06:42:51Z","receivedAt":"2010-06-12T06:42:51Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Ramkumar Ramachandra wrote:\n\n> +#define obj_pool_gen(pre, obj_t, initial_capacity) \\\n> +static struct { \\\n> +\tuint32_t committed; \\\n> +\tuint32_t size; \\\n> +\tuint32_t capacity; \\\n> +\tobj_t *base; \\\n> +\tFILE *file; \\\n> +} pre##_pool = { 0, 0, 0, NULL, NULL}; \\\n> +static void pre##_init(void) \\\n> +{ \\\n[...]\n\nThis defines a family of functions and not all pools use them all.\nOne workaround is to annotate them, like this:\n\n #ifdef __GNUC__\n #define MAYBE_UNUSED __attribute__((__unused__))\n #else\n #define MAYBE_UNUSED\n #endif\n\n #define obj_pool_gen(pre, obj_t, initial_capacity) \\\n ... \\\n static MAYBE_UNUSED void pre##_init(void) \\\n { \\\n ...\n\nCould that work here?\n\nThe “unused” attribute was added in gcc 2.7.\n\nJonathan\n"},{"id":"143649","messageId":"AANLkTilr9Qa9RJ3TIaCTgGwgp75TDBZbqRENC41xRFlq@mail.gmail.com","threadId":"24069","inReplyTo":"7vk4q5ictl.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH 3/6] Add library for string-specific memory pool","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-14T09:26:37Z","receivedAt":"2010-06-14T09:26:37Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi Junio,\n\nJunio C Hamano wrote:\n>> diff --git a/vcs-svn/string_pool.h b/vcs-svn/string_pool.h\n>> new file mode 100644\n>> index 0000000..a69d165\n>> --- /dev/null\n>> +++ b/vcs-svn/string_pool.h\n>> @@ -0,1 +1,15 @@\n>\n> How did you manage to get \"-0,1\" here?  This is supposed to be a new file.\n> PATCH 4/6 throws the same puzzlement at me.\n\nI hand-edited the patch in Emacs and expected that diff-mode would\ntake care of all this. Apparently, I was wrong- I will investigate\nthis problem further, because editing patches by hand and making some\nlast-minute corrections (just before send-email) is very useful.\nThanks for pointing this out!\n\nAnd thanks for getting the series into `pu`! Now I can focus on\nfiguring out the SVN API.\n\n-- Ram\n"},{"id":"143652","messageId":"7v631lbutb.fsf@alter.siamese.dyndns.org","threadId":"24069","inReplyTo":"AANLkTilr9Qa9RJ3TIaCTgGwgp75TDBZbqRENC41xRFlq@mail.gmail.com","subject":"Re: [PATCH 3/6] Add library for string-specific memory pool","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-06-14T13:36:00Z","receivedAt":"2010-06-14T13:36:00Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ramkumar Ramachandra <artagnon@gmail.com> writes:\n\n> I hand-edited the patch in Emacs and expected that diff-mode would\n> take care of all this. Apparently, I was wrong- I will investigate\n> this problem further,...\n\nThe problem with Emacs diff mode I already know about.  It gets confused\nby the \"-- SP LF\" line at the end of the format-patch output when it\nrecounts diff.\n\n> And thanks for getting the series into `pu`! Now I can focus on\n> figuring out the SVN API.\n\nDon't thank me.  You did all the work with others on the list.\n\nNote that there isn't that much difference between being in 'pu' and being\nin the mailing list archive.  Depending on how further discussions go, the\nseries can be replaced with an improvement or even can be dropped as a\nwhole.\n"},{"id":"143653","messageId":"AANLkTimFRpckXjvzaefyOdokf-1f7YkrNAQ8dVr3jQgj@mail.gmail.com","threadId":"24069","inReplyTo":"7v631lbutb.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH 3/6] Add library for string-specific memory pool","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-14T13:49:27Z","receivedAt":"2010-06-14T13:49:27Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi Junio,\n\nJunio C Hamano wrote:\n> The problem with Emacs diff mode I already know about.  It gets confused\n> by the \"-- SP LF\" line at the end of the format-patch output when it\n> recounts diff.\n\nI see. I'll probably prepare a patch then.\n\n> Note that there isn't that much difference between being in 'pu' and being\n> in the mailing list archive.  Depending on how further discussions go, the\n> series can be replaced with an improvement or even can be dropped as a\n> whole.\n\nIt's an indicator of progress, if not anything else. The project is\nalready pretty mature imho- after squashing in a few bugfixes, it\nshould be ready for `next`.\n\n-- Ram\n"},{"id":"143654","messageId":"AANLkTimtya1Ph5YjlFVAVpE2p4XOHvuaFAmsS4C0J5LH@mail.gmail.com","threadId":"24069","inReplyTo":"20100612064250.GB2549@burratino","subject":"Re: [PATCH 1/6] Add memory pool library","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-14T14:25:40Z","receivedAt":"2010-06-14T14:25:40Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi Jonathan,\n\nJonathan Nieder wrote:\n>  #ifdef __GNUC__\n>  #define MAYBE_UNUSED __attribute__((__unused__))\n>  #else\n>  #define MAYBE_UNUSED\n>  #endif\n\nYou'd suggested this earlier, but I was looking more for something\nthat we could use to mark some specific functions as unused instead of\nmarking everything as unused in the generation macro. Anyway, I have a\npatch ready, and svn-fe compiles fine with -Wall -Werror.\n\n> Could that work here?\n>\n> The “unused” attribute was added in gcc 2.7.\n\nJunio? Is this a good solution, or should we think of something else?\nWhat about other compilers?\n\n-- Ram\n"},{"id":"143655","messageId":"AANLkTim2qNNrInovraveWGcgr61FnrRzQXjuW_AsmiLc@mail.gmail.com","threadId":"24069","inReplyTo":"20100612062613.GA2549@burratino","subject":"Re: [PATCH 0/6] Another attempt to get the SVN exporter merged","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-14T14:41:16Z","receivedAt":"2010-06-14T14:41:16Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi Jonathan,\n\nJonathan Nieder wrote:\n> First of all, thanks for your work on this and sorry to take so long\n> to respond[1].\n\nAh that's alright. I'm glad you're up and running again :)\n\n> I planned to write a series adding svn-fe to contrib/ and then the\n> computer died.  If someone else doesn’t do it first, hopefully I can\n> try again this weekend.\n\nÆvar has also been asking about this. I have a branch ready to roll\nout from- I think I can do it today.\n\n> Sad but perhaps necessary.  I guess this also opens the possibility of\n> later making the save operation atomic with the fsync() + rename()\n> trick.\n\nYeah, mmap looked a lot more elegant.\n\n-- Ram\n"},{"id":"143656","messageId":"4C16403F.4040400@op5.se","threadId":"24069","inReplyTo":"AANLkTimtya1Ph5YjlFVAVpE2p4XOHvuaFAmsS4C0J5LH@mail.gmail.com","subject":"Re: [PATCH 1/6] Add memory pool library","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2010-06-14T14:44:15Z","receivedAt":"2010-06-14T14:44:15Z","isPatch":true,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"On 06/14/2010 04:25 PM, Ramkumar Ramachandra wrote:\n> Hi Jonathan,\n> \n> Jonathan Nieder wrote:\n>>   #ifdef __GNUC__\n>>   #define MAYBE_UNUSED __attribute__((__unused__))\n>>   #else\n>>   #define MAYBE_UNUSED\n>>   #endif\n> \n> You'd suggested this earlier, but I was looking more for something\n> that we could use to mark some specific functions as unused instead of\n> marking everything as unused in the generation macro. Anyway, I have a\n> patch ready, and svn-fe compiles fine with -Wall -Werror.\n> \n>> Could that work here?\n>>\n>> The “unused” attribute was added in gcc 2.7.\n> \n> Junio? Is this a good solution, or should we think of something else?\n> What about other compilers?\n> \n\n__attribute__((stuff)) is gcc-specific. It's usually macro'd away with\n#ifndef __GNUC__\n#define __attribute__(x)\n#endif\nwhich is why it requires double parantheses. gcc warns for but doesn't\nfail on unknown attributes, so it should be safe to use this with\ngcc older than 2.7 as well. Or you make it conditional to __GNUC__\nbeing >= 2 and __GNUC_MINOR__ being >= 7.\n\n-- \nAndreas Ericsson                   andreas.ericsson@op5.se\nOP5 AB                             www.op5.se\nTel: +46 8-230225                  Fax: +46 8-230231\n\nConsidering the successes of the wars on alcohol, poverty, drugs and\nterror, I think we should give some serious thought to declaring war\non peace.\n"},{"id":"143657","messageId":"7C3A5AC5-78A7-42EB-A5DF-EFFA761FF1A6@cordelta.com","threadId":"24069","inReplyTo":"AANLkTimFRpckXjvzaefyOdokf-1f7YkrNAQ8dVr3jQgj@mail.gmail.com","subject":"Re: [PATCH 3/6] Add library for string-specific memory pool","fromName":"David Michael Barr","fromEmail":"david.barr@cordelta.com","sentAt":"2010-06-14T14:45:42Z","receivedAt":"2010-06-14T14:45:42Z","isPatch":true,"sender":{"key":"david.barr@cordelta.com","avatar":"https://avatars.githubusercontent.com/u/220594?v=4"},"body":"Hi Ram,\n\n>> Note that there isn't that much difference between being in 'pu' and being\n>> in the mailing list archive.  Depending on how further discussions go, the\n>> series can be replaced with an improvement or even can be dropped as a\n>> whole.\n> \n> It's an indicator of progress, if not anything else. The project is\n> already pretty mature imho- after squashing in a few bugfixes, it\n> should be ready for `next`.\n\nI have a feeling that these patches will need a bit more love before they are\nready for 'next'. The persistence component is the least mature of the lot.\nI'd really like some feedback on making the persistence robust and simple.\nNow that persistence is append-only, the file based representation no longer\nneed be identical to the in-memory representation.\n\nI've tried several times to simplify the buffer_read_line() method in line_buffer.h\nEvery time I've ended up with slightly different behaviour.\nSomeone well versed in I/O might be able to greatly simplify it.\nIt may well be reduced to a simple wrapper around strbuf methods.\n\nI'm still toying in my head about how to simplify the data structure used to\nrepresent the trees. Conceptually, it is a multiway tree with the constraint that\nthe labels of siblings share a common prefix at the parent. It is implemented as\na ternary tree with left and right links to siblings in the multiway tree and a\nmiddle link to the 'root' child in the multiway tree, from which all children are\nreachable via left/right links.\nAs the code stands, the middle link is indirected via a 'directory' node.\nI'd like to remove this redundancy and make the design of the structure clearer.\n\nThere is scope for a massive rename of methods, arguments and variables so\nthat the code is easier to read.\n\n--\nDavid Barr.\n"}]}