{"thread":{"id":"24185","subject":"[PATCH/RFC v2 0/9] Subversion dump parsing library","startedAt":"2010-06-24T10:50:04Z","lastAt":"2010-06-30T02:09:33Z","messageCount":35,"participants":["Jonathan Nieder","Ramkumar Ramachandra","Sam Vilain"],"isPatch":true,"patchVersion":2,"patchTotal":9},"messages":[{"id":"144129","messageId":"20100624105004.GA12336@burratino","threadId":"24185","inReplyTo":null,"subject":"[PATCH/RFC v2 0/9] Subversion dump parsing library","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-06-24T10:50:04Z","receivedAt":"2010-06-24T10:50:04Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi gitsters,\n\nRam last sent this series a couple of weeks ago[1], and it was\nmerged to pu then as rr/svn-export. Here’s another iteration\nof the same for discussion, now including David Barr’s program\nthat demonstrates the functionality.\n\nPatch 1 is not so closely related; it just modifies\nparse_date_toffset() to keep me a little saner while using it.\n\nPatches 2-8 are very similar to the versions Ram sent. I\nexpanded the commit messages, took the latest code from\ngit://github.com/barrbrain/svn-dump-fast-export where possible,\nand added a simple example build system so one can see the\nresult of compiling with\n\n make vcs-svn/lib.a\n\nProbably more interesting is what the patches do not do:\n\n - they do not include any tests\n\n - they do not remove the persistent object pool functionality.\n If you try this code out, be sure to remove all the .bin\n files from the current directory after each run.\n\n - they are not guaranteed to have fewer bugs than the version\n Ram sent. In fact, the opposite is more likely, since the\n code is only lightly tested\n\nYou can try it out with\n\n ; cd contrib/svn-fe\n ; wget http://github.com/barrbrain/svn-dump-fast-export/raw/master/test.dump\n ; make svn-fe\n ; ./svn-fe \n---\n cache.h | 1 +\n date.c | 14 ++++++--------\n 2 files changed, 7 insertions(+), 8 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex ff4a7c2..4566501 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -800,6 +800,7 @@ const char *show_date_relative(unsigned long time, int tz,\n \t\t\t char *timebuf,\n \t\t\t size_t timebuf_size);\n int parse_date(const char *date, char *buf, int bufsize);\n+int parse_date_basic(const char *date, unsigned long *timestamp, int *offset);\n void datestamp(char *buf, int bufsize);\n #define approxidate(s) approxidate_careful((s), NULL)\n unsigned long approxidate_careful(const char *, int *);\ndiff --git a/date.c b/date.c\nindex 68cdcaa..383706d 100644\n--- a/date.c\n+++ b/date.c\n@@ -586,7 +586,7 @@ static int date_string(unsigned long date, int offset, char *buf, int len)\n \n /* Gr. strptime is crap for this; it doesn't have a way to require RFC2822\n (i.e. English) day/month names, and it doesn't work correctly with %z. */\n-int parse_date_toffset(const char *date, unsigned long *timestamp, int *offset)\n+int parse_date_basic(const char *date, unsigned long *timestamp, int *offset)\n {\n \tstruct tm tm;\n \tint tm_gmt;\n@@ -642,17 +642,16 @@ int parse_date_toffset(const char *date, unsigned long *timestamp, int *offset)\n \n \tif (!tm_gmt)\n \t\t*timestamp -= *offset * 60;\n-\treturn 1; /* success */\n+\treturn 0; /* success */\n }\n \n int parse_date(const char *date, char *result, int maxlen)\n {\n \tunsigned long timestamp;\n \tint offset;\n-\tif (parse_date_toffset(date, ×tamp, &offset) > 0)\n-\t\treturn date_string(timestamp, offset, result, maxlen);\n-\telse\n+\tif (parse_date_basic(date, ×tamp, &offset))\n \t\treturn -1;\n+\treturn date_string(timestamp, offset, result, maxlen);\n }\n \n enum date_mode parse_date_format(const char *format)\n@@ -1004,9 +1003,8 @@ unsigned long approxidate_relative(const char *date, const struct timeval *tv)\n \tint offset;\n \tint errors = 0;\n \n-\tif (parse_date_toffset(date, ×tamp, &offset) > 0)\n+\tif (!parse_date_basic(date, ×tamp, &offset))\n \t\treturn timestamp;\n-\n \treturn approxidate_str(date, tv, &errors);\n }\n \n@@ -1019,7 +1017,7 @@ unsigned long approxidate_careful(const char *date, int *error_ret)\n \tif (!error_ret)\n \t\terror_ret = &dummy;\n \n-\tif (parse_date_toffset(date, ×tamp, &offset) > 0) {\n+\tif (!parse_date_basic(date, ×tamp, &offset)) {\n \t\t*error_ret = 0;\n \t\treturn timestamp;\n \t}\n-- \n1.7.1\n"},{"id":"144131","messageId":"20100624105243.GB12376@burratino","threadId":"24185","inReplyTo":"20100624105004.GA12336@burratino","subject":"[PATCH 2/9] Introduce vcs-svn lib","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-06-24T10:52:44Z","receivedAt":"2010-06-24T10:52:44Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Teach the build system to build a separate library for the\nupcoming subversion interop support.\n\nThe resulting vcs-svn/lib.a does not contain any code, nor is\nit built during a normal build. This is just scaffolding for\nlater changes.\n\nSigned-off-by: Jonathan Nieder \n---\n Makefile | 8 +++++++-\n 1 files changed, 7 insertions(+), 1 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 9aca8a1..6441dcb 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -468,6 +468,7 @@ export PYTHON_PATH\n \n LIB_FILE=libgit.a\n XDIFF_LIB=xdiff/lib.a\n+VCSSVN_LIB=vcs-svn/lib.a\n \n LIB_H += advice.h\n LIB_H += archive.h\n@@ -1739,7 +1740,8 @@ ifndef NO_CURL\n endif\n XDIFF_OBJS = xdiff/xdiffi.o xdiff/xprepare.o xdiff/xutils.o xdiff/xemit.o \\\n \txdiff/xmerge.o xdiff/xpatience.o\n-OBJECTS := $(GIT_OBJS) $(XDIFF_OBJS)\n+VCSSVN_OBJS =\n+OBJECTS := $(GIT_OBJS) $(XDIFF_OBJS) $(VCSSVN_OBJS)\n \n dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d)\n dep_dirs := $(addsuffix .depend,$(sort $(dir $(OBJECTS))))\n@@ -1860,6 +1862,8 @@ http.o http-walker.o http-push.o remote-curl.o: http.h\n xdiff-interface.o $(XDIFF_OBJS): \\\n \txdiff/xinclude.h xdiff/xmacros.h xdiff/xdiff.h xdiff/xtypes.h \\\n \txdiff/xutils.h xdiff/xprepare.h xdiff/xdiffi.h xdiff/xemit.h\n+\n+$(VCSSVN_OBJS):\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\n@@ -1908,6 +1912,8 @@ $(LIB_FILE): $(LIB_OBJS)\n $(XDIFF_LIB): $(XDIFF_OBJS)\n \t$(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $(XDIFF_OBJS)\n \n+$(VCSSVN_LIB): $(VCSSVN_OBJS)\n+\t$(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $(VCSSVN_OBJS)\n \n doc:\n \t$(MAKE) -C Documentation all\n-- \n1.7.1\n"},{"id":"144132","messageId":"20100624105339.GC12376@burratino","threadId":"24185","inReplyTo":"20100624105004.GA12336@burratino","subject":"[PATCH 3/9] Add memory pool library","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-06-24T10:53:39Z","receivedAt":"2010-06-24T10:53:39Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"From: David Barr \n\nAdd a memory pool library implemented using C macros. The obj_pool_gen()\nmacro creates a type-specific memory pool API.\n\nThe memory pool library is distinguished from the existing specialized\nallocators in alloc.c by using a contiguous block for all allocations.\nThis means that on one hand, long-lived pointers have to be written as\noffsets, since the base address changes as the pool grows, but on the\nother hand, the entire pool can be easily written to the file system.\nThis allows the memory pool to persist between runs of an application.\n\nFor the svn importer, such a facility is useful because each svn\nrevision can copy trees and files from any previous revision. The\nrelevant information for all revisions has to persist somehow to\nsupport incremental runs, and for now it is simplest to avoid relying\non the target VCS for that.\n\nobj_pool_gen(pre, obj_t, initial_capability)\n\n\tpre: Prefix for generated functions (example: string).\n\tobj_t: Type for treap data structure (example: char).\n\tinitial_capacity: Initial size of the memory pool (example: 4096).\n\nvoid pre_init(void);\n\n\tRead values from a previous run to initialize the pool.\n\tIf this function is not called, the pool begins valid but empty.\n\nuint32_t pre_alloc(uint32_t nmemb);\n\n\tReserve space for a few objects in the pool and return an\n\toffset to the first one.\n\nuint32_t pre_free(uint32_t nmemb);\n\n\tUnreserve the last few objects reserved.\n\nuint32_t pre_offset(obj_t *pointer);\nobj_t *pre_pointer(uint32_t offset);\n\n\tConvert between pointers into the in-memory pool and offsets\n\tfrom the beginning (or ~0 for the NULL pointer). Pointers are\n\tnot guaranteed to remain valid after a pre_alloc() operation\n\tor pre_reset() followed by pre_init(), but offsets are.\n\nvoid pre_commit(void);\n\n\tWrite the pool to file. A pre_reset() followed by pre_init()\n\t(pehaps with exit() in between) will return the pool to the\n\tlast committed state.\n\nvoid pre_reset(void);\n\n\tDeinitialize the pool, freeing any associated memory and\n\tfile handles.\n\nSigned-off-by: David Barr \nSigned-off-by: Ramkumar Ramachandra \nSigned-off-by: Junio C Hamano \nSigned-off-by: Jonathan Nieder \n---\n Makefile | 3 +-\n vcs-svn/LICENSE | 26 +++++++++++++++++\n vcs-svn/obj_pool.h | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++++\n 3 files changed, 108 insertions(+), 1 deletions(-)\n create mode 100644 vcs-svn/LICENSE\n create mode 100644 vcs-svn/obj_pool.h\n\ndiff --git a/Makefile b/Makefile\nindex 6441dcb..fc31ee0 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1863,7 +1863,8 @@ xdiff-interface.o $(XDIFF_OBJS): \\\n \txdiff/xinclude.h xdiff/xmacros.h xdiff/xdiff.h xdiff/xtypes.h \\\n \txdiff/xutils.h xdiff/xprepare.h xdiff/xdiffi.h xdiff/xemit.h\n \n-$(VCSSVN_OBJS):\n+$(VCSSVN_OBJS): \\\n+\tvcs-svn/obj_pool.h\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\ndiff --git a/vcs-svn/LICENSE b/vcs-svn/LICENSE\nnew file mode 100644\nindex 0000000..6e52372\n--- /dev/null\n+++ b/vcs-svn/LICENSE\n@@ -0,0 +1,26 @@\n+Copyright (C) 2010 David Barr .\n+All rights reserved.\n+\n+Redistribution and use in source and binary forms, with or without\n+modification, are permitted provided that the following conditions\n+are met:\n+1. Redistributions of source code must retain the above copyright\n+ notice(s), this list of conditions and the following disclaimer\n+ unmodified other than the allowable addition of one or more\n+ copyright notices.\n+2. Redistributions in binary form must reproduce the above copyright\n+ notice(s), this list of conditions and the following disclaimer in\n+ the documentation and/or other materials provided with the\n+ distribution.\n+\n+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER(S) ``AS IS'' AND ANY\n+EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE\n+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR\n+PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER(S) BE\n+LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR\n+CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF\n+SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR\n+BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,\n+WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE\n+OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,\n+EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\ndiff --git a/vcs-svn/obj_pool.h b/vcs-svn/obj_pool.h\nnew file mode 100644\nindex 0000000..f60c872\n--- /dev/null\n+++ b/vcs-svn/obj_pool.h\n@@ -0,0 +1,80 @@\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#ifndef OBJ_POOL_H_\n+#define OBJ_POOL_H_\n+\n+#include \"git-compat-util.h\"\n+\n+#define MAYBE_UNUSED __attribute__((__unused__))\n+\n+#define obj_pool_gen(pre, obj_t, initial_capacity) \\\n+static struct { \\\n+\tuint32_t committed; \\\n+\tuint32_t size; \\\n+\tuint32_t capacity; \\\n+\tobj_t *base; \\\n+\tFILE *file; \\\n+} pre##_pool = { 0, 0, 0, NULL, NULL}; \\\n+static MAYBE_UNUSED void pre##_init(void) \\\n+{ \\\n+\tstruct stat st; \\\n+\tpre##_pool.file = fopen(#pre \".bin\", \"a+\"); \\\n+\trewind(pre##_pool.file); \\\n+\tfstat(fileno(pre##_pool.file), &st); \\\n+\tpre##_pool.size = st.st_size / sizeof(obj_t); \\\n+\tpre##_pool.committed = pre##_pool.size; \\\n+\tpre##_pool.capacity = pre##_pool.size * 2; \\\n+\tif (pre##_pool.capacity < initial_capacity) \\\n+\t\tpre##_pool.capacity = initial_capacity; \\\n+\tpre##_pool.base = malloc(pre##_pool.capacity * sizeof(obj_t)); \\\n+\tfread(pre##_pool.base, sizeof(obj_t), pre##_pool.size, pre##_pool.file); \\\n+} \\\n+static MAYBE_UNUSED uint32_t pre##_alloc(uint32_t count) \\\n+{ \\\n+\tuint32_t offset; \\\n+\tif (pre##_pool.size + count > pre##_pool.capacity) { \\\n+\t\twhile (pre##_pool.size + count > pre##_pool.capacity) \\\n+\t\t\tif (pre##_pool.capacity) \\\n+\t\t\t\tpre##_pool.capacity *= 2; \\\n+\t\t\telse \\\n+\t\t\t\tpre##_pool.capacity = initial_capacity; \\\n+\t\tpre##_pool.base = realloc(pre##_pool.base, \\\n+\t\t\t\t\tpre##_pool.capacity * sizeof(obj_t)); \\\n+\t} \\\n+\toffset = pre##_pool.size; \\\n+\tpre##_pool.size += count; \\\n+\treturn offset; \\\n+} \\\n+static MAYBE_UNUSED void pre##_free(uint32_t count) \\\n+{ \\\n+\tpre##_pool.size -= count; \\\n+} \\\n+static MAYBE_UNUSED uint32_t pre##_offset(obj_t *obj) \\\n+{ \\\n+\treturn obj == NULL ? ~0 : obj - pre##_pool.base; \\\n+} \\\n+static MAYBE_UNUSED obj_t *pre##_pointer(uint32_t offset) \\\n+{ \\\n+\treturn offset >= pre##_pool.size ? NULL : &pre##_pool.base[offset]; \\\n+} \\\n+static MAYBE_UNUSED void pre##_commit(void) \\\n+{ \\\n+\tpre##_pool.committed += fwrite(pre##_pool.base + pre##_pool.committed, \\\n+\t\tsizeof(obj_t), pre##_pool.size - pre##_pool.committed, \\\n+\t\tpre##_pool.file); \\\n+} \\\n+static MAYBE_UNUSED void pre##_reset(void) \\\n+{ \\\n+\tfree(pre##_pool.base); \\\n+\tif (pre##_pool.file) \\\n+\t\tfclose(pre##_pool.file); \\\n+\tpre##_pool.base = NULL; \\\n+\tpre##_pool.size = 0; \\\n+\tpre##_pool.capacity = 0; \\\n+\tpre##_pool.file = NULL; \\\n+}\n+\n+#endif\n-- \n1.7.1\n"},{"id":"144133","messageId":"20100624105706.GD12376@burratino","threadId":"24185","inReplyTo":"20100624105004.GA12336@burratino","subject":"[PATCH 4/9] Add treap implementation","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-06-24T10:57:06Z","receivedAt":"2010-06-24T10:57:06Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"From: Jason Evans \n\nProvide macros to generate a type-specific treap implementation and\nvarious functions to operate on it. It uses obj_pool.h to store memory\nnodes in a treap. Previously committed nodes are never removed from\nthe pool; after any *_commit operation, it is assumed (correctly, in\nthe case of svn-fast-export) that someone else must care about them.\n\nTreaps provide a memory-efficient binary search tree structure.\nInsertion/deletion/search are about as about as fast in the average\ncase as red-black trees and the chances of worst-case behavior are\nvanishingly small, thanks to (pseudo-)randomness. The bad worst-case\nbehavior is a small price to pay, given that treaps are much simpler\nto implement.\n\nFrom http://www.canonware.com/download/trp/trp_hash/trp.h\n\n[db: Altered to reference nodes by offset from a common base pointer]\n[db: Bob Jenkins' hashing implementation dropped for Knuth's]\n[db: Methods unnecessary for search and insert dropped]\n\nSigned-off-by: David Barr \nSigned-off-by: Ramkumar Ramachandra \nSigned-off-by: Junio C Hamano \nSigned-off-by: Jonathan Nieder \n---\n Makefile | 2 +-\n vcs-svn/LICENSE | 3 +\n vcs-svn/trp.h | 220 +++++++++++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/trp.txt | 90 ++++++++++++++++++++++\n 4 files changed, 314 insertions(+), 1 deletions(-)\n create mode 100644 vcs-svn/trp.h\n create mode 100644 vcs-svn/trp.txt\n\ndiff --git a/Makefile b/Makefile\nindex fc31ee0..663a366 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1864,7 +1864,7 @@ xdiff-interface.o $(XDIFF_OBJS): \\\n \txdiff/xutils.h xdiff/xprepare.h xdiff/xdiffi.h xdiff/xemit.h\n \n $(VCSSVN_OBJS): \\\n-\tvcs-svn/obj_pool.h\n+\tvcs-svn/obj_pool.h vcs-svn/trp.h\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\ndiff --git a/vcs-svn/LICENSE b/vcs-svn/LICENSE\nindex 6e52372..a3d384c 100644\n--- a/vcs-svn/LICENSE\n+++ b/vcs-svn/LICENSE\n@@ -1,6 +1,9 @@\n Copyright (C) 2010 David Barr .\n All rights reserved.\n \n+Copyright (C) 2008 Jason Evans .\n+All rights reserved.\n+\n Redistribution and use in source and binary forms, with or without\n modification, are permitted provided that the following conditions\n are met:\ndiff --git a/vcs-svn/trp.h b/vcs-svn/trp.h\nnew file mode 100644\nindex 0000000..dd7d5ee\n--- /dev/null\n+++ b/vcs-svn/trp.h\n@@ -0,0 +1,220 @@\n+/*\n+ * C macro implementation of treaps.\n+ *\n+ * Usage:\n+ * #include \n+ * #include \"trp.h\"\n+ * trp_gen(...)\n+ *\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#ifndef TRP_H_\n+#define TRP_H_\n+\n+#define MAYBE_UNUSED __attribute__((__unused__))\n+\n+/* Node structure. */\n+struct trp_node {\n+\tuint32_t trpn_left;\n+\tuint32_t trpn_right;\n+};\n+\n+/* Root structure. */\n+struct trp_root {\n+\tuint32_t trp_root;\n+};\n+\n+/* Pointer/Offset conversion. */\n+#define trpn_pointer(a_base, a_offset) (a_base##_pointer(a_offset))\n+#define trpn_offset(a_base, a_pointer) (a_base##_offset(a_pointer))\n+#define trpn_modify(a_base, a_offset) \\\n+\tdo { \\\n+\t\tif ((a_offset) < a_base##_pool.committed) { \\\n+\t\t\tuint32_t old_offset = (a_offset);\\\n+\t\t\t(a_offset) = a_base##_alloc(1); \\\n+\t\t\t*trpn_pointer(a_base, a_offset) = \\\n+\t\t\t\t*trpn_pointer(a_base, old_offset); \\\n+\t\t} \\\n+\t} while (0);\n+\n+/* Left accessors. */\n+#define trp_left_get(a_base, a_field, a_node) \\\n+\t(trpn_pointer(a_base, a_node)->a_field.trpn_left)\n+#define trp_left_set(a_base, a_field, a_node, a_left) \\\n+\tdo { \\\n+\t\ttrpn_modify(a_base, a_node); \\\n+\t\ttrp_left_get(a_base, a_field, a_node) = (a_left); \\\n+\t} while(0)\n+\n+/* Right accessors. */\n+#define trp_right_get(a_base, a_field, a_node) \\\n+\t(trpn_pointer(a_base, a_node)->a_field.trpn_right)\n+#define trp_right_set(a_base, a_field, a_node, a_right) \\\n+\tdo { \\\n+\t\ttrpn_modify(a_base, a_node); \\\n+\t\ttrp_right_get(a_base, a_field, a_node) = (a_right); \\\n+\t} while(0)\n+\n+/*\n+ * Fibonacci hash function.\n+ * The multiplier is the nearest prime to (2^32 times (√5 - 1)/2).\n+ * See Knuth §6.4: volume 3, 3rd ed, p518.\n+ */\n+#define trpn_hash(a_node) (uint32_t) (2654435761u * (a_node))\n+\n+/* Priority accessors. */\n+#define trp_prio_get(a_node) trpn_hash(a_node)\n+\n+/* Node initializer. */\n+#define trp_node_new(a_base, a_field, a_node) \\\n+\tdo { \\\n+\t\ttrp_left_set(a_base, a_field, (a_node), ~0); \\\n+\t\ttrp_right_set(a_base, a_field, (a_node), ~0); \\\n+\t} while(0)\n+\n+/* Internal utility macros. */\n+#define trpn_first(a_base, a_field, a_root, r_node) \\\n+\tdo { \\\n+\t\t(r_node) = (a_root); \\\n+\t\tif ((r_node) == ~0) \\\n+\t\t\treturn NULL; \\\n+\t\twhile (~trp_left_get(a_base, a_field, (r_node))) \\\n+\t\t\t(r_node) = trp_left_get(a_base, a_field, (r_node)); \\\n+\t} while (0)\n+\n+#define trpn_rotate_left(a_base, a_field, a_node, r_node) \\\n+\tdo { \\\n+\t\t(r_node) = trp_right_get(a_base, a_field, (a_node)); \\\n+\t\ttrp_right_set(a_base, a_field, (a_node), \\\n+\t\t\ttrp_left_get(a_base, a_field, (r_node))); \\\n+\t\ttrp_left_set(a_base, a_field, (r_node), (a_node)); \\\n+\t} while(0)\n+\n+#define trpn_rotate_right(a_base, a_field, a_node, r_node) \\\n+\tdo { \\\n+\t\t(r_node) = trp_left_get(a_base, a_field, (a_node)); \\\n+\t\ttrp_left_set(a_base, a_field, (a_node), \\\n+\t\t\ttrp_right_get(a_base, a_field, (r_node))); \\\n+\t\ttrp_right_set(a_base, a_field, (r_node), (a_node)); \\\n+\t} while(0)\n+\n+#define trp_gen(a_attr, a_pre, a_type, a_field, a_base, a_cmp) \\\n+a_attr a_type MAYBE_UNUSED *a_pre##first(struct trp_root *treap) \\\n+{ \\\n+\tuint32_t ret; \\\n+\ttrpn_first(a_base, a_field, treap->trp_root, ret); \\\n+\treturn trpn_pointer(a_base, ret); \\\n+} \\\n+a_attr a_type MAYBE_UNUSED *a_pre##next(struct trp_root *treap, a_type *node) \\\n+{ \\\n+\tuint32_t ret; \\\n+\tuint32_t offset = trpn_offset(a_base, node); \\\n+\tif (~trp_right_get(a_base, a_field, offset)) { \\\n+\t\ttrpn_first(a_base, a_field, \\\n+\t\t\ttrp_right_get(a_base, a_field, offset), ret); \\\n+\t} else { \\\n+\t\tuint32_t tnode = treap->trp_root; \\\n+\t\tret = ~0; \\\n+\t\twhile (1) { \\\n+\t\t\tint cmp = (a_cmp)(trpn_pointer(a_base, offset), \\\n+\t\t\t\ttrpn_pointer(a_base, tnode)); \\\n+\t\t\tif (cmp < 0) { \\\n+\t\t\t\tret = tnode; \\\n+\t\t\t\ttnode = trp_left_get(a_base, a_field, tnode); \\\n+\t\t\t} else if (cmp > 0) { \\\n+\t\t\t\ttnode = trp_right_get(a_base, a_field, tnode); \\\n+\t\t\t} else { \\\n+\t\t\t\tbreak; \\\n+\t\t\t} \\\n+\t\t} \\\n+\t} \\\n+\treturn trpn_pointer(a_base, ret); \\\n+} \\\n+a_attr a_type MAYBE_UNUSED *a_pre##search(struct trp_root *treap, a_type *key) \\\n+{ \\\n+\tint cmp; \\\n+\tuint32_t ret = treap->trp_root; \\\n+\twhile (~ret && (cmp = (a_cmp)(key, trpn_pointer(a_base,ret)))) { \\\n+\t\tif (cmp < 0) \\\n+\t\t\tret = trp_left_get(a_base, a_field, ret); \\\n+\t\telse \\\n+\t\t\tret = trp_right_get(a_base, a_field, ret); \\\n+\t} \\\n+\treturn trpn_pointer(a_base, ret); \\\n+} \\\n+a_attr uint32_t MAYBE_UNUSED a_pre##insert_recurse(uint32_t cur_node, uint32_t ins_node) \\\n+{ \\\n+\tif (cur_node == ~0) { \\\n+\t\treturn (ins_node); \\\n+\t} else { \\\n+\t\tuint32_t ret; \\\n+\t\tint cmp = (a_cmp)(trpn_pointer(a_base, ins_node), \\\n+\t\t\t\t\ttrpn_pointer(a_base, cur_node)); \\\n+\t\tif (cmp < 0) { \\\n+\t\t\tuint32_t left = a_pre##insert_recurse( \\\n+\t\t\t\ttrp_left_get(a_base, a_field, cur_node), ins_node); \\\n+\t\t\ttrp_left_set(a_base, a_field, cur_node, left); \\\n+\t\t\tif (trp_prio_get(left) < trp_prio_get(cur_node)) \\\n+\t\t\t\ttrpn_rotate_right(a_base, a_field, cur_node, ret); \\\n+\t\t\telse \\\n+\t\t\t\tret = cur_node; \\\n+\t\t} else { \\\n+\t\t\tuint32_t right = a_pre##insert_recurse( \\\n+\t\t\t\ttrp_right_get(a_base, a_field, cur_node), ins_node); \\\n+\t\t\ttrp_right_set(a_base, a_field, cur_node, right); \\\n+\t\t\tif (trp_prio_get(right) < trp_prio_get(cur_node)) \\\n+\t\t\t\ttrpn_rotate_left(a_base, a_field, cur_node, ret); \\\n+\t\t\telse \\\n+\t\t\t\tret = cur_node; \\\n+\t\t} \\\n+\t\treturn (ret); \\\n+\t} \\\n+} \\\n+a_attr void MAYBE_UNUSED a_pre##insert(struct trp_root *treap, a_type *node) \\\n+{ \\\n+\tuint32_t offset = trpn_offset(a_base, node); \\\n+\ttrp_node_new(a_base, a_field, offset); \\\n+\ttreap->trp_root = a_pre##insert_recurse(treap->trp_root, offset); \\\n+} \\\n+a_attr uint32_t MAYBE_UNUSED a_pre##remove_recurse(uint32_t cur_node, uint32_t rem_node) \\\n+{ \\\n+\tint cmp = a_cmp(trpn_pointer(a_base, rem_node), \\\n+\t\t\ttrpn_pointer(a_base, cur_node)); \\\n+\tif (cmp == 0) { \\\n+\t\tuint32_t ret; \\\n+\t\tuint32_t left = trp_left_get(a_base, a_field, cur_node); \\\n+\t\tuint32_t right = trp_right_get(a_base, a_field, cur_node); \\\n+\t\tif (left == ~0) { \\\n+\t\t\tif (right == ~0) \\\n+\t\t\t\treturn (~0); \\\n+\t\t} else if (right == ~0 || trp_prio_get(left) < trp_prio_get(right)) { \\\n+\t\t\ttrpn_rotate_right(a_base, a_field, cur_node, ret); \\\n+\t\t\tright = a_pre##remove_recurse(cur_node, rem_node); \\\n+\t\t\ttrp_right_set(a_base, a_field, ret, right); \\\n+\t\t\treturn (ret); \\\n+\t\t} \\\n+\t\ttrpn_rotate_left(a_base, a_field, cur_node, ret); \\\n+\t\tleft = a_pre##remove_recurse(cur_node, rem_node); \\\n+\t\ttrp_left_set(a_base, a_field, ret, left); \\\n+\t\treturn (ret); \\\n+\t} else if (cmp < 0) { \\\n+\t\tuint32_t left = a_pre##remove_recurse( \\\n+\t\t\ttrp_left_get(a_base, a_field, cur_node), rem_node); \\\n+\t\ttrp_left_set(a_base, a_field, cur_node, left); \\\n+\t\treturn (cur_node); \\\n+\t} else { \\\n+\t\tuint32_t right = a_pre##remove_recurse( \\\n+\t\t\ttrp_right_get(a_base, a_field, cur_node), rem_node); \\\n+\t\ttrp_right_set(a_base, a_field, cur_node, right); \\\n+\t\treturn (cur_node); \\\n+\t} \\\n+} \\\n+a_attr void MAYBE_UNUSED a_pre##remove(struct trp_root *treap, a_type *node) \\\n+{ \\\n+\ttreap->trp_root = a_pre##remove_recurse(treap->trp_root, \\\n+\t\ttrpn_offset(a_base, node)); \\\n+} \\\n+\n+#endif\ndiff --git a/vcs-svn/trp.txt b/vcs-svn/trp.txt\nnew file mode 100644\nindex 0000000..f387aaa\n--- /dev/null\n+++ b/vcs-svn/trp.txt\n@@ -0,0 +1,90 @@\n+treap API\n+=========\n+\n+The trp API generates a data structure and functions to handle a\n+large growing set of objects stored in a pool.\n+\n+The caller:\n+\n+. Specifies parameters for the generated functions with the\n+ trp_gen(static, foo_, ...) macro.\n+\n+. Allocates and clears a `struct trp_node` variable.\n+\n+. Adds new items to the set using `foo_insert`.\n+\n+. Can find a specific item in the set using `foo_search`.\n+\n+. Can iterate over items in the set using `foo_first` and `foo_next`.\n+\n+. Can remove an item from the set using `foo_remove`.\n+\n+. The set is never freed.\n+\n+Example:\n+\n+----\n+struct ex_node {\n+\tconst char *s;\n+\tstruct trp_node ex_link;\n+};\n+static struct trp_root ex_base;\n+obj_pool_gen(ex, struct ex_node, 4096);\n+trp_gen(static, ex_, struct ex_node, ex_link, ex, strcmp)\n+struct ex_node *item;\n+\n+item = ex_pointer(ex_alloc(1));\n+item->s = \"hello\";\n+ex_insert(&ex_base, item);\n+item = ex_pointer(ex_alloc(1));\n+item->s = \"goodbye\";\n+ex_insert(&ex_base, item);\n+for (item = ex_first(&ex_base); item; item = ex_next(&ex_base, item))\n+\tprintf(\"%s\\n\", item->s);\n+----\n+\n+Functions\n+---------\n+\n+trp_gen(attr, foo_, node_type, link_field, pool, cmp)::\n+\n+\tGenerate a type-specific treap implementation.\n++\n+. The storage class for generated functions will be 'attr' (e.g., `static`).\n+. Generated function names are prefixed with 'foo_' (e.g., `treap_`).\n+. Treap nodes will be of type 'node_type' (e.g., `struct treap_node`).\n+ This type must be a struct with at least one `struct trp_node` field\n+ to point to its children.\n+. The field used to access child nodes will be 'link_field'.\n+. All treap nodes must lie in the 'pool' object pool.\n+. Treap nodes must be totally ordered by the 'cmp' relation, with the\n+ following prototype:\n++\n+int (*cmp)(node_type \\*a, node_type \\*b)\n++\n+and returning a value less than, equal to, or greater than zero\n+according to the result of comparison.\n+\n+void foo_insert(struct trp_root *treap, node_type \\*node)::\n+\n+\tInsert node into treap. If inserted multiple times,\n+\ta node will appear in the treap multiple times.\n+\n+void foo_remove(struct trp_root *treap, node_type \\*node)::\n+\n+\tRemove node from treap. Caller must ensure node is\n+\tpresent in treap before using this function.\n+\n+node_type *foo_search(struct trp_root \\*treap, node_type \\*key)::\n+\n+\tSearch for a node that matches key. If no match is found,\n+\treturn what would be key's successor, were key in treap\n+\t(NULL if no successor).\n+\n+node_type *foo_first(struct trp_root \\*treap)::\n+\n+\tFind the first item from the treap, in sorted order.\n+\n+node_type *foo_next(struct trp_root \\*treap, node_type \\*node)::\n+\n+\tFind the next item.\n-- \n1.7.1\n"},{"id":"144134","messageId":"20100624105813.GE12376@burratino","threadId":"24185","inReplyTo":"20100624105004.GA12336@burratino","subject":"[PATCH 5/9] Add string-specific memory pool","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-06-24T10:58:14Z","receivedAt":"2010-06-24T10:58:14Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"From: David Barr \n\nIntern strings so they can be compared by address and stored without\nwasting space.\n\nThis library uses the macros in the obj_pool.h and trp.h to create a\nmemory pool for strings and expose an API for handling them.\n\nSigned-off-by: David Barr \nSigned-off-by: Ramkumar Ramachandra \nSigned-off-by: Junio C Hamano \nSigned-off-by: Jonathan Nieder \n---\n Makefile | 4 +-\n vcs-svn/string_pool.c | 114 +++++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/string_pool.h | 15 ++++++\n 3 files changed, 131 insertions(+), 2 deletions(-)\n create mode 100644 vcs-svn/string_pool.c\n create mode 100644 vcs-svn/string_pool.h\n\ndiff --git a/Makefile b/Makefile\nindex 663a366..e11e588 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1740,7 +1740,7 @@ ifndef NO_CURL\n endif\n XDIFF_OBJS = xdiff/xdiffi.o xdiff/xprepare.o xdiff/xutils.o xdiff/xemit.o \\\n \txdiff/xmerge.o xdiff/xpatience.o\n-VCSSVN_OBJS =\n+VCSSVN_OBJS = vcs-svn/string_pool.o\n OBJECTS := $(GIT_OBJS) $(XDIFF_OBJS) $(VCSSVN_OBJS)\n \n dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d)\n@@ -1864,7 +1864,7 @@ xdiff-interface.o $(XDIFF_OBJS): \\\n \txdiff/xutils.h xdiff/xprepare.h xdiff/xdiffi.h xdiff/xemit.h\n \n $(VCSSVN_OBJS): \\\n-\tvcs-svn/obj_pool.h vcs-svn/trp.h\n+\tvcs-svn/obj_pool.h vcs-svn/trp.h vcs-svn/string_pool.h\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\ndiff --git a/vcs-svn/string_pool.c b/vcs-svn/string_pool.c\nnew file mode 100644\nindex 0000000..bd5a380\n--- /dev/null\n+++ b/vcs-svn/string_pool.c\n@@ -0,0 +1,114 @@\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"git-compat-util.h\"\n+#include \"trp.h\"\n+#include \"obj_pool.h\"\n+#include \"string_pool.h\"\n+\n+static struct trp_root tree = { ~0 };\n+\n+struct node {\n+\tuint32_t offset;\n+\tstruct trp_node children;\n+};\n+\n+/* Two memory pools: one for struct node, and another for strings */\n+obj_pool_gen(node, struct node, 4096);\n+obj_pool_gen(string, char, 4096);\n+\n+static char *node_value(struct node *node)\n+{\n+\treturn node ? string_pointer(node->offset) : NULL;\n+}\n+\n+static int node_cmp(struct node *a, struct node *b)\n+{\n+\treturn strcmp(node_value(a), node_value(b));\n+}\n+\n+/* Build a Treap from the node structure (a trp_node w/ offset) */\n+trp_gen(static, tree_, struct node, children, node, node_cmp);\n+\n+char *pool_fetch(uint32_t entry)\n+{\n+\treturn node_value(node_pointer(entry));\n+}\n+\n+uint32_t pool_intern(char *key)\n+{\n+\t/* Canonicalize key */\n+\tstruct node *match = NULL;\n+\tuint32_t key_len;\n+\tif (key == NULL)\n+\t\treturn ~0;\n+\tkey_len = strlen(key) + 1;\n+\tstruct node *node = node_pointer(node_alloc(1));\n+\tnode->offset = string_alloc(key_len);\n+\tstrcpy(node_value(node), key);\n+\tmatch = tree_search(&tree, node);\n+\tif (!match) {\n+\t\ttree_insert(&tree, node);\n+\t} else {\n+\t\tnode_free(1);\n+\t\tstring_free(key_len);\n+\t\tnode = match;\n+\t}\n+\treturn node_offset(node);\n+}\n+\n+uint32_t pool_tok_r(char *str, const char *delim, char **saveptr)\n+{\n+\tchar *token = strtok_r(str, delim, saveptr);\n+\treturn token ? pool_intern(token) : ~0;\n+}\n+\n+void pool_print_seq(uint32_t len, uint32_t *seq, char delim, FILE *stream)\n+{\n+\tuint32_t i;\n+\tfor (i = 0; i < len && ~seq[i]; i++) {\n+\t\tfputs(pool_fetch(seq[i]), stream);\n+\t\tif (i < len - 1 && ~seq[i + 1])\n+\t\t\tfputc(delim, stream);\n+\t}\n+}\n+\n+uint32_t pool_tok_seq(uint32_t max, uint32_t *seq, char *delim, char *str)\n+{\n+\tchar *context = NULL;\n+\tuint32_t length = 0, token = str ? pool_tok_r(str, delim, &context) : ~0;\n+\twhile (length < max) {\n+\t\tseq[length++] = token;\n+\t\tif (token == ~0)\n+\t\t\tbreak;\n+\t\ttoken = pool_tok_r(NULL, delim, &context);\n+\t}\n+\tseq[length ? length - 1 : 0] = ~0;\n+\treturn length;\n+}\n+\n+void pool_init(void)\n+{\n+\tuint32_t node;\n+\tuint32_t string = 0;\n+\tstring_init();\n+\twhile (string < string_pool.size) {\n+\t\tnode = node_alloc(1);\n+\t\tnode_pointer(node)->offset = string;\n+\t\ttree_insert(&tree, node_pointer(node));\n+\t\tstring += strlen(string_pointer(string)) + 1;\n+\t}\n+}\n+\n+void pool_commit(void)\n+{\n+\tstring_commit();\n+}\n+\n+void pool_reset(void)\n+{\n+\tnode_reset();\n+\tstring_reset();\n+}\ndiff --git a/vcs-svn/string_pool.h b/vcs-svn/string_pool.h\nnew file mode 100644\nindex 0000000..085e6d7\n--- /dev/null\n+++ b/vcs-svn/string_pool.h\n@@ -0,0 +1,15 @@\n+#ifndef STRING_POOL_H_\n+#define STRING_POOL_H_\n+\n+#include \"git-compat-util.h\"\n+\n+uint32_t pool_intern(char *key);\n+char *pool_fetch(uint32_t entry);\n+uint32_t pool_tok_r(char *str, const char *delim, char **saveptr);\n+void pool_print_seq(uint32_t len, uint32_t *seq, char delim, FILE *stream);\n+uint32_t pool_tok_seq(uint32_t max, uint32_t *seq, char *delim, char *str);\n+void pool_init(void);\n+void pool_commit(void);\n+void pool_reset(void);\n+\n+#endif\n-- \n1.7.1\n"},{"id":"144135","messageId":"20100624110112.GF12376@burratino","threadId":"24185","inReplyTo":"20100624105004.GA12336@burratino","subject":"[PATCH 6/9] Add stream helper library","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-06-24T11:01:12Z","receivedAt":"2010-06-24T11:01:12Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"From: David Barr \n\nThis library provides thread-unsafe fgets()- and fread()-like\nfunctions where the caller does not have to supply a buffer. It\nmaintains a couple of static buffers and provides an API to use\nthem.\n\nNEEDSWORK: what should buffer_copy_bytes do on error?\n\nSigned-off-by: David Barr \nSigned-off-by: Ramkumar Ramachandra \nSigned-off-by: Junio C Hamano \nSigned-off-by: Jonathan Nieder \n---\n Makefile | 5 ++-\n vcs-svn/line_buffer.c | 93 +++++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/line_buffer.h | 14 +++++++\n 3 files changed, 110 insertions(+), 2 deletions(-)\n create mode 100644 vcs-svn/line_buffer.c\n create mode 100644 vcs-svn/line_buffer.h\n\ndiff --git a/Makefile b/Makefile\nindex e11e588..8223d9b 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1740,7 +1740,7 @@ ifndef NO_CURL\n endif\n XDIFF_OBJS = xdiff/xdiffi.o xdiff/xprepare.o xdiff/xutils.o xdiff/xemit.o \\\n \txdiff/xmerge.o xdiff/xpatience.o\n-VCSSVN_OBJS = vcs-svn/string_pool.o\n+VCSSVN_OBJS = vcs-svn/string_pool.o vcs-svn/line_buffer.o\n OBJECTS := $(GIT_OBJS) $(XDIFF_OBJS) $(VCSSVN_OBJS)\n \n dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d)\n@@ -1864,7 +1864,8 @@ xdiff-interface.o $(XDIFF_OBJS): \\\n \txdiff/xutils.h xdiff/xprepare.h xdiff/xdiffi.h xdiff/xemit.h\n \n $(VCSSVN_OBJS): \\\n-\tvcs-svn/obj_pool.h vcs-svn/trp.h vcs-svn/string_pool.h\n+\tvcs-svn/obj_pool.h vcs-svn/trp.h vcs-svn/string_pool.h \\\n+\tvcs-svn/line_buffer.h\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\ndiff --git a/vcs-svn/line_buffer.c b/vcs-svn/line_buffer.c\nnew file mode 100644\nindex 0000000..0f83426\n--- /dev/null\n+++ b/vcs-svn/line_buffer.c\n@@ -0,0 +1,93 @@\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"git-compat-util.h\"\n+\n+#include \"line_buffer.h\"\n+#include \"obj_pool.h\"\n+\n+#define LINE_BUFFER_LEN 10000\n+#define COPY_BUFFER_LEN 4096\n+\n+/* Create memory pool for char sequence of known length */\n+obj_pool_gen(blob, char, 4096);\n+\n+static char line_buffer[LINE_BUFFER_LEN];\n+static char byte_buffer[COPY_BUFFER_LEN];\n+static FILE *infile;\n+\n+int buffer_init(const char *filename)\n+{\n+\tinfile = filename ? fopen(filename, \"r\") : stdin;\n+\tif (!infile)\n+\t\treturn -1;\n+\treturn 0;\n+}\n+\n+int buffer_deinit()\n+{\n+\tfclose(infile);\n+\treturn 0;\n+}\n+\n+/* Read a line without trailing newline. */\n+char *buffer_read_line(void)\n+{\n+\tchar *end;\n+\tif (!fgets(line_buffer, sizeof(line_buffer), infile))\n+\t\t/* Error or data exhausted. */\n+\t\treturn NULL;\n+\tend = line_buffer + strlen(line_buffer);\n+\tif (end[-1] == '\\n')\n+\t\tend[-1] = '\\0';\n+\telse if (feof(infile))\n+\t\t; /* No newline at end of file. That's fine. */\n+\telse\n+\t\t/*\n+\t\t * Line was too long.\n+\t\t * There is probably a saner way to deal with this,\n+\t\t * but for now let's return an error.\n+\t\t */\n+\t\treturn NULL;\n+\treturn line_buffer;\n+}\n+\n+char *buffer_read_string(uint32_t len)\n+{\n+\tchar *s;\n+\tblob_free(blob_pool.size);\n+\ts = blob_pointer(blob_alloc(len + 1));\n+\ts[fread(s, 1, len, infile)] = '\\0';\n+\treturn ferror(infile) ? NULL : s;\n+}\n+\n+void buffer_copy_bytes(uint32_t len)\n+{\n+\tuint32_t in;\n+\twhile (len > 0 && !feof(infile)) {\n+\t\tin = len < COPY_BUFFER_LEN ? len : COPY_BUFFER_LEN;\n+\t\tin = fread(byte_buffer, 1, in, infile);\n+\t\tlen -= in;\n+\t\tfwrite(byte_buffer, 1, in, stdout);\n+\t\tif (ferror(infile) || ferror(stdout))\n+\t\t\t/* NEEDSWORK: handle error. */\n+\t\t\tbreak;\n+\t}\n+}\n+\n+void buffer_skip_bytes(uint32_t len)\n+{\n+\tuint32_t in;\n+\twhile (len > 0 && !feof(infile) && !ferror(infile)) {\n+\t\tin = len < COPY_BUFFER_LEN ? len : COPY_BUFFER_LEN;\n+\t\tin = fread(byte_buffer, 1, in, infile);\n+\t\tlen -= in;\n+\t}\n+}\n+\n+void buffer_reset(void)\n+{\n+\tblob_reset();\n+}\ndiff --git a/vcs-svn/line_buffer.h b/vcs-svn/line_buffer.h\nnew file mode 100644\nindex 0000000..631d1df\n--- /dev/null\n+++ b/vcs-svn/line_buffer.h\n@@ -0,0 +1,14 @@\n+#ifndef LINE_BUFFER_H_\n+#define LINE_BUFFER_H_\n+\n+#include \"git-compat-util.h\"\n+\n+int buffer_init(const char *filename);\n+int buffer_deinit(void);\n+char *buffer_read_line(void);\n+char *buffer_read_string(uint32_t len);\n+void buffer_copy_bytes(uint32_t len);\n+void buffer_skip_bytes(uint32_t len);\n+void buffer_reset(void);\n+\n+#endif\n-- \n1.7.1\n"},{"id":"144136","messageId":"20100624110217.GG12376@burratino","threadId":"24185","inReplyTo":"20100624105004.GA12336@burratino","subject":"[PATCH 7/9] Add infrastructure to write revisions in fast-export format","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-06-24T11:02:17Z","receivedAt":"2010-06-24T11:02:17Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"From: David Barr \n\nrepo_tree maintains the exporter's state and provides a facility to to\ncall fast_export, which writes objects to stdout suitable for\nconsumption by fast-import.\n\nThe exported functions roughly correspond to Subversion FS operations.\n\n . repo_add adds a file to the current commit.\n\n . repo_modify adds a replacement for an existing file;\n it is implemented exactly the same way, but a check could be\n added later to distinguish the two cases.\n\n . repo_copy copies a blob from a previous revision to the current\n commit.\n\n . repo_replace modifies the content of a file from the current\n commit, if and only if it exists.\n\n . repo_delete removes a file or directory from the current commit.\n\n . repo_commit calls out to fast_export to write the current commit to\n the fast-import stream in stdout.\n\n . repo_diff is used by the fast_export module to write the changes\n for a commit.\n\n . repo_reset erases the exporter's state, so valgrind can be happy.\n\nSigned-off-by: David Barr \nSigned-off-by: Ramkumar Ramachandra \nSigned-off-by: Junio C Hamano \nSigned-off-by: Jonathan Nieder \n---\n Makefile | 5 +-\n vcs-svn/fast_export.c | 75 +++++++++++\n vcs-svn/fast_export.h | 14 ++\n vcs-svn/repo_tree.c | 335 +++++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/repo_tree.h | 26 ++++\n 5 files changed, 453 insertions(+), 2 deletions(-)\n create mode 100644 vcs-svn/fast_export.c\n create mode 100644 vcs-svn/fast_export.h\n create mode 100644 vcs-svn/repo_tree.c\n create mode 100644 vcs-svn/repo_tree.h\n\ndiff --git a/Makefile b/Makefile\nindex 8223d9b..7c66dcc 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1740,7 +1740,8 @@ ifndef NO_CURL\n endif\n XDIFF_OBJS = xdiff/xdiffi.o xdiff/xprepare.o xdiff/xutils.o xdiff/xemit.o \\\n \txdiff/xmerge.o xdiff/xpatience.o\n-VCSSVN_OBJS = vcs-svn/string_pool.o vcs-svn/line_buffer.o\n+VCSSVN_OBJS = vcs-svn/string_pool.o vcs-svn/line_buffer.o \\\n+\tvcs-svn/repo_tree.o vcs-svn/fast_export.o\n OBJECTS := $(GIT_OBJS) $(XDIFF_OBJS) $(VCSSVN_OBJS)\n \n dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d)\n@@ -1865,7 +1866,7 @@ xdiff-interface.o $(XDIFF_OBJS): \\\n \n $(VCSSVN_OBJS): \\\n \tvcs-svn/obj_pool.h vcs-svn/trp.h vcs-svn/string_pool.h \\\n-\tvcs-svn/line_buffer.h\n+\tvcs-svn/line_buffer.h vcs-svn/repo_tree.h vcs-svn/fast_export.h\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nnew file mode 100644\nindex 0000000..7552803\n--- /dev/null\n+++ b/vcs-svn/fast_export.c\n@@ -0,0 +1,75 @@\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"git-compat-util.h\"\n+\n+#include \"fast_export.h\"\n+#include \"line_buffer.h\"\n+#include \"repo_tree.h\"\n+#include \"string_pool.h\"\n+\n+#define MAX_GITSVN_LINE_LEN 4096\n+\n+static uint32_t first_commit_done;\n+\n+void fast_export_delete(uint32_t depth, uint32_t *path)\n+{\n+\tputchar('D');\n+\tputchar(' ');\n+\tpool_print_seq(depth, path, '/', stdout);\n+\tputchar('\\n');\n+}\n+\n+void fast_export_modify(uint32_t depth, uint32_t *path, uint32_t mode,\n+ uint32_t mark)\n+{\n+\t/* Mode must be 100644, 100755, 120000, or 160000. */\n+\tprintf(\"M %06o :%d \", mode, mark);\n+\tpool_print_seq(depth, path, '/', stdout);\n+\tputchar('\\n');\n+}\n+\n+static char gitsvnline[MAX_GITSVN_LINE_LEN];\n+void fast_export_commit(uint32_t revision, uint32_t author, char *log,\n+\t\t\tuint32_t uuid, uint32_t url,\n+\t\t\tunsigned long timestamp)\n+{\n+\tif (!log)\n+\t\tlog = \"\";\n+\tif (~uuid && ~url) {\n+\t\tsnprintf(gitsvnline, MAX_GITSVN_LINE_LEN, \"\\n\\ngit-svn-id: %s@%d %s\\n\",\n+\t\t\t\t pool_fetch(url), revision, pool_fetch(uuid));\n+\t} else {\n+\t\t*gitsvnline = '\\0';\n+\t}\n+\tprintf(\"commit refs/heads/master\\n\");\n+\tprintf(\"committer %s <%s@%s> %ld +0000\\n\",\n+\t\t ~author ? pool_fetch(author) : \"nobody\",\n+\t\t ~author ? pool_fetch(author) : \"nobody\",\n+\t\t ~uuid ? pool_fetch(uuid) : \"local\", timestamp);\n+\tprintf(\"data %zd\\n%s%s\\n\",\n+\t\t strlen(log) + strlen(gitsvnline), log, gitsvnline);\n+\tif (!first_commit_done) {\n+\t\tif (revision > 1)\n+\t\t\tprintf(\"from refs/heads/master^0\\n\");\n+\t\tfirst_commit_done = 1;\n+\t}\n+\trepo_diff(revision - 1, revision);\n+\tfputc('\\n', stdout);\n+\n+\tprintf(\"progress Imported commit %d.\\n\\n\", revision);\n+}\n+\n+void fast_export_blob(uint32_t mode, uint32_t mark, uint32_t len)\n+{\n+\tif (mode == REPO_MODE_LNK) {\n+\t\t/* svn symlink blobs start with \"link \" */\n+\t\tbuffer_skip_bytes(5);\n+\t\tlen -= 5;\n+\t}\n+\tprintf(\"blob\\nmark :%d\\ndata %d\\n\", mark, len);\n+\tbuffer_copy_bytes(len);\n+\tfputc('\\n', stdout);\n+}\ndiff --git a/vcs-svn/fast_export.h b/vcs-svn/fast_export.h\nnew file mode 100644\nindex 0000000..47e8f56\n--- /dev/null\n+++ b/vcs-svn/fast_export.h\n@@ -0,0 +1,14 @@\n+#ifndef FAST_EXPORT_H_\n+#define FAST_EXPORT_H_\n+\n+#include \n+#include \n+\n+void fast_export_delete(uint32_t depth, uint32_t *path);\n+void fast_export_modify(uint32_t depth, uint32_t *path, uint32_t mode,\n+\t\t\tuint32_t mark);\n+void fast_export_commit(uint32_t revision, uint32_t author, char *log,\n+\t\t\tuint32_t uuid, uint32_t url, unsigned long timestamp);\n+void fast_export_blob(uint32_t mode, uint32_t mark, uint32_t len);\n+\n+#endif\ndiff --git a/vcs-svn/repo_tree.c b/vcs-svn/repo_tree.c\nnew file mode 100644\nindex 0000000..59a7434\n--- /dev/null\n+++ b/vcs-svn/repo_tree.c\n@@ -0,0 +1,335 @@\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"git-compat-util.h\"\n+\n+#include \"string_pool.h\"\n+#include \"repo_tree.h\"\n+#include \"obj_pool.h\"\n+#include \"fast_export.h\"\n+\n+#include \"trp.h\"\n+\n+struct repo_dirent {\n+\tuint32_t name_offset;\n+\tstruct trp_node children;\n+\tuint32_t mode;\n+\tuint32_t content_offset;\n+};\n+\n+struct repo_dir {\n+\tstruct trp_root entries;\n+};\n+\n+struct repo_commit {\n+\tuint32_t root_dir_offset;\n+};\n+\n+/* Memory pools for commit, dir and dirent */\n+obj_pool_gen(commit, struct repo_commit, 4096);\n+obj_pool_gen(dir, struct repo_dir, 4096);\n+obj_pool_gen(dirent, struct repo_dirent, 4096);\n+\n+static uint32_t active_commit;\n+static uint32_t mark;\n+\n+static int repo_dirent_name_cmp(const void *a, const void *b);\n+\n+/* Treap for directory entries */\n+trp_gen(static, dirent_, struct repo_dirent, children, dirent, repo_dirent_name_cmp);\n+\n+uint32_t next_blob_mark(void)\n+{\n+\treturn mark++;\n+}\n+\n+static struct repo_dir *repo_commit_root_dir(struct repo_commit *commit)\n+{\n+\treturn dir_pointer(commit->root_dir_offset);\n+}\n+\n+static struct repo_dirent *repo_first_dirent(struct repo_dir *dir)\n+{\n+\treturn dirent_first(&dir->entries);\n+}\n+\n+static int repo_dirent_name_cmp(const void *a, const void *b)\n+{\n+\tconst struct repo_dirent *dirent1 = a, *dirent2 = b;\n+\tuint32_t a_offset = dirent1->name_offset;\n+\tuint32_t b_offset = dirent2->name_offset;\n+\treturn (a_offset > b_offset) - (a_offset < b_offset);\n+}\n+\n+static int repo_dirent_is_dir(struct repo_dirent *dirent)\n+{\n+\treturn dirent != NULL && dirent->mode == REPO_MODE_DIR;\n+}\n+\n+static struct repo_dir *repo_dir_from_dirent(struct repo_dirent *dirent)\n+{\n+\tif (!repo_dirent_is_dir(dirent))\n+\t\treturn NULL;\n+\treturn dir_pointer(dirent->content_offset);\n+}\n+\n+static struct repo_dir *repo_clone_dir(struct repo_dir *orig_dir)\n+{\n+\tuint32_t orig_o, new_o;\n+\torig_o = dir_offset(orig_dir);\n+\tif (orig_o >= dir_pool.committed)\n+\t\treturn orig_dir;\n+\tnew_o = dir_alloc(1);\n+\torig_dir = dir_pointer(orig_o);\n+\t*dir_pointer(new_o) = *orig_dir;\n+\treturn dir_pointer(new_o);\n+}\n+\n+static struct repo_dirent *repo_read_dirent(uint32_t revision, uint32_t *path)\n+{\n+\tuint32_t name = 0;\n+\tstruct repo_dirent *key = dirent_pointer(dirent_alloc(1));\n+\tstruct repo_dir *dir = NULL;\n+\tstruct repo_dirent *dirent = NULL;\n+\tdir = repo_commit_root_dir(commit_pointer(revision));\n+\twhile (~(name = *path++)) {\n+\t\tkey->name_offset = name;\n+\t\tdirent = dirent_search(&dir->entries, key);\n+\t\tif (dirent == NULL || !repo_dirent_is_dir(dirent))\n+\t\t\tbreak;\n+\t\tdir = repo_dir_from_dirent(dirent);\n+\t}\n+\tdirent_free(1);\n+\treturn dirent;\n+}\n+\n+static void repo_write_dirent(uint32_t *path, uint32_t mode,\n+ uint32_t content_offset, uint32_t del)\n+{\n+\tuint32_t name, revision, dir_o = ~0, parent_dir_o = ~0;\n+\tstruct repo_dir *dir;\n+\tstruct repo_dirent *key;\n+\tstruct repo_dirent *dirent = NULL;\n+\trevision = active_commit;\n+\tdir = repo_commit_root_dir(commit_pointer(revision));\n+\tdir = repo_clone_dir(dir);\n+\tcommit_pointer(revision)->root_dir_offset = dir_offset(dir);\n+\twhile (~(name = *path++)) {\n+\t\tparent_dir_o = dir_offset(dir);\n+\n+\t\tkey = dirent_pointer(dirent_alloc(1));\n+\t\tkey->name_offset = name;\n+\n+\t\tdirent = dirent_search(&dir->entries, key);\n+\t\tif (dirent == NULL)\n+\t\t\tdirent = key;\n+\t\telse\n+\t\t\tdirent_free(1);\n+\n+\t\tif (dirent == key) {\n+\t\t\tdirent->mode = REPO_MODE_DIR;\n+\t\t\tdirent->content_offset = 0;\n+\t\t\tdirent_insert(&dir->entries, dirent);\n+\t\t}\n+\n+\t\tif (dirent_offset(dirent) < dirent_pool.committed) {\n+\t\t\tdir_o = repo_dirent_is_dir(dirent) ?\n+\t\t\t\t\tdirent->content_offset : ~0;\n+\t\t\tdirent_remove(&dir->entries, dirent);\n+\t\t\tdirent = dirent_pointer(dirent_alloc(1));\n+\t\t\tdirent->name_offset = name;\n+\t\t\tdirent->mode = REPO_MODE_DIR;\n+\t\t\tdirent->content_offset = dir_o;\n+\t\t\tdirent_insert(&dir->entries, dirent);\n+\t\t}\n+\n+\t\tdir = repo_dir_from_dirent(dirent);\n+\t\tdir = repo_clone_dir(dir);\n+\t\tdirent->content_offset = dir_offset(dir);\n+\t}\n+\tif (dirent == NULL)\n+\t\treturn;\n+\tdirent->mode = mode;\n+\tdirent->content_offset = content_offset;\n+\tif (del && ~parent_dir_o)\n+\t\tdirent_remove(&dir_pointer(parent_dir_o)->entries, dirent);\n+}\n+\n+uint32_t repo_copy(uint32_t revision, uint32_t *src, uint32_t *dst)\n+{\n+\tuint32_t mode = 0, content_offset = 0;\n+\tstruct repo_dirent *src_dirent;\n+\tsrc_dirent = repo_read_dirent(revision, src);\n+\tif (src_dirent != NULL) {\n+\t\tmode = src_dirent->mode;\n+\t\tcontent_offset = src_dirent->content_offset;\n+\t\trepo_write_dirent(dst, mode, content_offset, 0);\n+\t}\n+\treturn mode;\n+}\n+\n+void repo_add(uint32_t *path, uint32_t mode, uint32_t blob_mark)\n+{\n+\trepo_write_dirent(path, mode, blob_mark, 0);\n+}\n+\n+uint32_t repo_replace(uint32_t *path, uint32_t blob_mark)\n+{\n+\tuint32_t mode = 0;\n+\tstruct repo_dirent *src_dirent;\n+\tsrc_dirent = repo_read_dirent(active_commit, path);\n+\tif (src_dirent != NULL) {\n+\t\tmode = src_dirent->mode;\n+\t\trepo_write_dirent(path, mode, blob_mark, 0);\n+\t}\n+\treturn mode;\n+}\n+\n+void repo_modify(uint32_t *path, uint32_t mode, uint32_t blob_mark)\n+{\n+\tstruct repo_dirent *src_dirent;\n+\tsrc_dirent = repo_read_dirent(active_commit, path);\n+\tif (src_dirent != NULL && blob_mark == 0)\n+\t\tblob_mark = src_dirent->content_offset;\n+\trepo_write_dirent(path, mode, blob_mark, 0);\n+}\n+\n+void repo_delete(uint32_t *path)\n+{\n+\trepo_write_dirent(path, 0, 0, 1);\n+}\n+\n+static void repo_git_add_r(uint32_t depth, uint32_t *path, struct repo_dir *dir);\n+\n+static void repo_git_add(uint32_t depth, uint32_t *path, struct repo_dirent *dirent)\n+{\n+\tif (repo_dirent_is_dir(dirent))\n+\t\trepo_git_add_r(depth, path, repo_dir_from_dirent(dirent));\n+\telse\n+\t\tfast_export_modify(depth, path,\n+\t\t dirent->mode, dirent->content_offset);\n+}\n+\n+static void repo_git_add_r(uint32_t depth, uint32_t *path, struct repo_dir *dir)\n+{\n+\tstruct repo_dirent *de = repo_first_dirent(dir);\n+\twhile (de) {\n+\t\tpath[depth] = de->name_offset;\n+\t\trepo_git_add(depth + 1, path, de);\n+\t\tde = dirent_next(&dir->entries, de);\n+\t}\n+}\n+\n+static void repo_diff_r(uint32_t depth, uint32_t *path, struct repo_dir *dir1,\n+ struct repo_dir *dir2)\n+{\n+\tstruct repo_dirent *de1, *de2;\n+\tde1 = repo_first_dirent(dir1);\n+\tde2 = repo_first_dirent(dir2);\n+\n+\twhile (de1 && de2) {\n+\t\tif (de1->name_offset < de2->name_offset) {\n+\t\t\tpath[depth] = de1->name_offset;\n+\t\t\tfast_export_delete(depth + 1, path);\n+\t\t\tde1 = dirent_next(&dir1->entries, de1);\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (de1->name_offset > de2->name_offset) {\n+\t\t\tpath[depth] = de2->name_offset;\n+\t\t\trepo_git_add(depth + 1, path, de2);\n+\t\t\tde2 = dirent_next(&dir2->entries, de2);\n+\t\t\tcontinue;\n+\t\t}\n+\t\tpath[depth] = de1->name_offset;\n+\n+\t\tif (de1->mode == de2->mode &&\n+\t\t de1->content_offset == de2->content_offset) {\n+\t\t\t; /* No change. */\n+\t\t} else if (repo_dirent_is_dir(de1) && repo_dirent_is_dir(de2)) {\n+\t\t\trepo_diff_r(depth + 1, path,\n+\t\t\t\t repo_dir_from_dirent(de1),\n+\t\t\t\t repo_dir_from_dirent(de2));\n+\t\t} else if (!repo_dirent_is_dir(de1) && !repo_dirent_is_dir(de2)) {\n+\t\t\trepo_git_add(depth + 1, path, de2);\n+\t\t} else {\n+\t\t\tfast_export_delete(depth + 1, path);\n+\t\t\trepo_git_add(depth + 1, path, de2);\n+\t\t}\n+\t\tde1 = dirent_next(&dir1->entries, de1);\n+\t\tde2 = dirent_next(&dir2->entries, de2);\n+\t}\n+\twhile (de1) {\n+\t\tpath[depth] = de1->name_offset;\n+\t\tfast_export_delete(depth + 1, path);\n+\t\tde1 = dirent_next(&dir1->entries, de1);\n+\t}\n+\twhile (de2) {\n+\t\tpath[depth] = de2->name_offset;\n+\t\trepo_git_add(depth + 1, path, de2);\n+\t\tde2 = dirent_next(&dir2->entries, de2);\n+\t}\n+}\n+\n+static uint32_t path_stack[REPO_MAX_PATH_DEPTH];\n+\n+void repo_diff(uint32_t r1, uint32_t r2)\n+{\n+\trepo_diff_r(0,\n+\t path_stack,\n+\t repo_commit_root_dir(commit_pointer(r1)),\n+\t repo_commit_root_dir(commit_pointer(r2)));\n+}\n+\n+void repo_commit(uint32_t revision, uint32_t author, char *log, uint32_t uuid,\n+ uint32_t url, unsigned long timestamp)\n+{\n+\tfast_export_commit(revision, author, log, uuid, url, timestamp);\n+\tpool_commit();\n+\tdirent_commit();\n+\tdir_commit();\n+\tcommit_commit();\n+\tactive_commit = commit_alloc(1);\n+\tcommit_pointer(active_commit)->root_dir_offset =\n+\t\tcommit_pointer(active_commit - 1)->root_dir_offset;\n+}\n+\n+static void mark_init(void)\n+{\n+\tuint32_t i;\n+\tmark = 0;\n+\tfor (i = 0; i < dirent_pool.size; i++)\n+\t\tif (!repo_dirent_is_dir(dirent_pointer(i)) &&\n+\t\t dirent_pointer(i)->content_offset > mark)\n+\t\t\tmark = dirent_pointer(i)->content_offset;\n+\tmark++;\n+}\n+\n+void repo_init() {\n+\tpool_init();\n+\tcommit_init();\n+\tdir_init();\n+\tdirent_init();\n+\tmark_init();\n+\tif (commit_pool.size == 0) {\n+\t\t/* Create empty tree for commit 0. */\n+\t\tcommit_alloc(1);\n+\t\tcommit_pointer(0)->root_dir_offset = dir_alloc(1);\n+\t\tdir_pointer(0)->entries.trp_root = ~0;\n+\t\tdir_commit();\n+\t\tcommit_commit();\n+\t}\n+\t/* Preallocate next commit, ready for changes. */\n+\tactive_commit = commit_alloc(1);\n+\tcommit_pointer(active_commit)->root_dir_offset =\n+\t\tcommit_pointer(active_commit - 1)->root_dir_offset;\n+}\n+\n+void repo_reset(void)\n+{\n+\tpool_reset();\n+\tcommit_reset();\n+\tdir_reset();\n+\tdirent_reset();\n+}\ndiff --git a/vcs-svn/repo_tree.h b/vcs-svn/repo_tree.h\nnew file mode 100644\nindex 0000000..92a7a7b\n--- /dev/null\n+++ b/vcs-svn/repo_tree.h\n@@ -0,0 +1,26 @@\n+#ifndef REPO_TREE_H_\n+#define REPO_TREE_H_\n+\n+#include \"git-compat-util.h\"\n+\n+#define REPO_MODE_DIR 0040000\n+#define REPO_MODE_BLB 0100644\n+#define REPO_MODE_EXE 0100755\n+#define REPO_MODE_LNK 0120000\n+\n+#define REPO_MAX_PATH_LEN 4096\n+#define REPO_MAX_PATH_DEPTH 1000\n+\n+uint32_t next_blob_mark(void);\n+uint32_t repo_copy(uint32_t revision, uint32_t *src, uint32_t *dst);\n+void repo_add(uint32_t *path, uint32_t mode, uint32_t blob_mark);\n+uint32_t repo_replace(uint32_t *path, uint32_t blob_mark);\n+void repo_modify(uint32_t *path, uint32_t mode, uint32_t blob_mark);\n+void repo_delete(uint32_t *path);\n+void repo_commit(uint32_t revision, uint32_t author, char *log, uint32_t uuid,\n+ uint32_t url, long unsigned timestamp);\n+void repo_diff(uint32_t r1, uint32_t r2);\n+void repo_init(void);\n+void repo_reset(void);\n+\n+#endif\n-- \n1.7.1\n"},{"id":"144137","messageId":"20100624110325.GH12376@burratino","threadId":"24185","inReplyTo":"20100624105004.GA12336@burratino","subject":"[PATCH 8/9] Add SVN dump parser","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-06-24T11:03:25Z","receivedAt":"2010-06-24T11:03:25Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"From: David Barr \n\nsvndump parses data that is in SVN dumpfile format produced by\n`svnadmin dump` with the help of line_buffer and uses repo_tree and\nfast_export to emit a git fast-import stream.\n\nBased roughly on com.hydrografix.svndump 0.92 from the SvnToCCase\nproject at , by Stefan Hegny and\nothers.\n\nSigned-off-by: David Barr \nSigned-off-by: Ramkumar Ramachandra \nSigned-off-by: Junio C Hamano \nSigned-off-by: Jonathan Nieder \n---\n Makefile | 5 +-\n vcs-svn/LICENSE | 4 +\n vcs-svn/svndump.c | 289 +++++++++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/svndump.h | 8 ++\n 4 files changed, 304 insertions(+), 2 deletions(-)\n create mode 100644 vcs-svn/svndump.c\n create mode 100644 vcs-svn/svndump.h\n\ndiff --git a/Makefile b/Makefile\nindex 7c66dcc..e7b37e0 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1741,7 +1741,7 @@ endif\n XDIFF_OBJS = xdiff/xdiffi.o xdiff/xprepare.o xdiff/xutils.o xdiff/xemit.o \\\n \txdiff/xmerge.o xdiff/xpatience.o\n VCSSVN_OBJS = vcs-svn/string_pool.o vcs-svn/line_buffer.o \\\n-\tvcs-svn/repo_tree.o vcs-svn/fast_export.o\n+\tvcs-svn/repo_tree.o vcs-svn/fast_export.o vcs-svn/svndump.o\n OBJECTS := $(GIT_OBJS) $(XDIFF_OBJS) $(VCSSVN_OBJS)\n \n dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d)\n@@ -1866,7 +1866,8 @@ xdiff-interface.o $(XDIFF_OBJS): \\\n \n $(VCSSVN_OBJS): \\\n \tvcs-svn/obj_pool.h vcs-svn/trp.h vcs-svn/string_pool.h \\\n-\tvcs-svn/line_buffer.h vcs-svn/repo_tree.h vcs-svn/fast_export.h\n+\tvcs-svn/line_buffer.h vcs-svn/repo_tree.h vcs-svn/fast_export.h \\\n+\tvcs-svn/svndump.h\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\ndiff --git a/vcs-svn/LICENSE b/vcs-svn/LICENSE\nindex a3d384c..0a5e3c4 100644\n--- a/vcs-svn/LICENSE\n+++ b/vcs-svn/LICENSE\n@@ -4,6 +4,10 @@ All rights reserved.\n Copyright (C) 2008 Jason Evans .\n All rights reserved.\n \n+Copyright (C) 2005 Stefan Hegny, hydrografix Consulting GmbH,\n+Frankfurt/Main, Germany\n+and others, see http://svn2cc.sarovar.org\n+\n Redistribution and use in source and binary forms, with or without\n modification, are permitted provided that the following conditions\n are met:\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nnew file mode 100644\nindex 0000000..86714ed\n--- /dev/null\n+++ b/vcs-svn/svndump.c\n@@ -0,0 +1,289 @@\n+/*\n+ * Parse and rearrange a svnadmin dump.\n+ * Create the dump with:\n+ * svnadmin dump --incremental -r: >outfile\n+ *\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"cache.h\"\n+#include \"repo_tree.h\"\n+#include \"fast_export.h\"\n+#include \"line_buffer.h\"\n+#include \"obj_pool.h\"\n+#include \"string_pool.h\"\n+\n+#define NODEACT_REPLACE 4\n+#define NODEACT_DELETE 3\n+#define NODEACT_ADD 2\n+#define NODEACT_CHANGE 1\n+#define NODEACT_UNKNOWN 0\n+\n+#define DUMP_CTX 0\n+#define REV_CTX 1\n+#define NODE_CTX 2\n+\n+#define LENGTH_UNKNOWN (~0)\n+#define DATE_RFC2822_LEN 31\n+\n+/* Create memory pool for log messages */\n+obj_pool_gen(log, char, 4096);\n+\n+static char* log_copy(uint32_t length, char *log)\n+{\n+\tchar *buffer;\n+\tlog_free(log_pool.size);\n+\tbuffer = log_pointer(log_alloc(length));\n+\tstrncpy(buffer, log, length);\n+\treturn buffer;\n+}\n+\n+static struct {\n+\tuint32_t action, propLength, textLength, srcRev, srcMode, mark, type;\n+\tuint32_t src[REPO_MAX_PATH_DEPTH], dst[REPO_MAX_PATH_DEPTH];\n+} node_ctx;\n+\n+static struct {\n+\tuint32_t revision, author;\n+\tunsigned long timestamp;\n+\tchar *log;\n+} rev_ctx;\n+\n+static struct {\n+\tuint32_t uuid, url;\n+} dump_ctx;\n+\n+static struct {\n+\tuint32_t svn_log, svn_author, svn_date, svn_executable, svn_special, uuid,\n+\t\trevision_number, node_path, node_kind, node_action,\n+\t\tnode_copyfrom_path, node_copyfrom_rev, text_content_length,\n+\t\tprop_content_length, content_length;\n+} keys;\n+\n+static void reset_node_ctx(char *fname)\n+{\n+\tnode_ctx.type = 0;\n+\tnode_ctx.action = NODEACT_UNKNOWN;\n+\tnode_ctx.propLength = LENGTH_UNKNOWN;\n+\tnode_ctx.textLength = LENGTH_UNKNOWN;\n+\tnode_ctx.src[0] = ~0;\n+\tnode_ctx.srcRev = 0;\n+\tnode_ctx.srcMode = 0;\n+\tpool_tok_seq(REPO_MAX_PATH_DEPTH, node_ctx.dst, \"/\", fname);\n+\tnode_ctx.mark = 0;\n+}\n+\n+static void reset_rev_ctx(uint32_t revision)\n+{\n+\trev_ctx.revision = revision;\n+\trev_ctx.timestamp = 0;\n+\trev_ctx.log = NULL;\n+\trev_ctx.author = ~0;\n+}\n+\n+static void reset_dump_ctx(uint32_t url)\n+{\n+\tdump_ctx.url = url;\n+\tdump_ctx.uuid = ~0;\n+}\n+\n+static void init_keys(void)\n+{\n+\tkeys.svn_log = pool_intern(\"svn:log\");\n+\tkeys.svn_author = pool_intern(\"svn:author\");\n+\tkeys.svn_date = pool_intern(\"svn:date\");\n+\tkeys.svn_executable = pool_intern(\"svn:executable\");\n+\tkeys.svn_special = pool_intern(\"svn:special\");\n+\tkeys.uuid = pool_intern(\"UUID\");\n+\tkeys.revision_number = pool_intern(\"Revision-number\");\n+\tkeys.node_path = pool_intern(\"Node-path\");\n+\tkeys.node_kind = pool_intern(\"Node-kind\");\n+\tkeys.node_action = pool_intern(\"Node-action\");\n+\tkeys.node_copyfrom_path = pool_intern(\"Node-copyfrom-path\");\n+\tkeys.node_copyfrom_rev = pool_intern(\"Node-copyfrom-rev\");\n+\tkeys.text_content_length = pool_intern(\"Text-content-length\");\n+\tkeys.prop_content_length = pool_intern(\"Prop-content-length\");\n+\tkeys.content_length = pool_intern(\"Content-length\");\n+}\n+\n+static void read_props(void)\n+{\n+\tuint32_t len;\n+\tuint32_t key = ~0;\n+\tchar *val = NULL;\n+\tchar *t;\n+\twhile ((t = buffer_read_line()) && strcmp(t, \"PROPS-END\")) {\n+\t\tif (!strncmp(t, \"K \", 2)) {\n+\t\t\tlen = atoi(&t[2]);\n+\t\t\tkey = pool_intern(buffer_read_string(len));\n+\t\t\tbuffer_read_line();\n+\t\t} else if (!strncmp(t, \"V \", 2)) {\n+\t\t\tlen = atoi(&t[2]);\n+\t\t\tval = buffer_read_string(len);\n+\t\t\tif (key == keys.svn_log) {\n+\t\t\t\t/* Value length excludes terminating nul. */\n+\t\t\t\trev_ctx.log = log_copy(len + 1, val);\n+\t\t\t} else if (key == keys.svn_author) {\n+\t\t\t\trev_ctx.author = pool_intern(val);\n+\t\t\t} else if (key == keys.svn_date) {\n+\t\t\t\tif (parse_date_basic(val, &rev_ctx.timestamp, NULL))\n+\t\t\t\t\tfprintf(stderr, \"Invalid timestamp: %s\\n\", val);\n+\t\t\t} else if (key == keys.svn_executable) {\n+\t\t\t\tnode_ctx.type = REPO_MODE_EXE;\n+\t\t\t} else if (key == keys.svn_special) {\n+\t\t\t\tnode_ctx.type = REPO_MODE_LNK;\n+\t\t\t}\n+\t\t\tkey = ~0;\n+\t\t\tbuffer_read_line();\n+\t\t}\n+\t}\n+}\n+\n+static void handle_node(void)\n+{\n+\tif (node_ctx.propLength != LENGTH_UNKNOWN && node_ctx.propLength)\n+\t\tread_props();\n+\n+\tif (node_ctx.srcRev)\n+\t\tnode_ctx.srcMode = repo_copy(node_ctx.srcRev, node_ctx.src, node_ctx.dst);\n+\n+\tif (node_ctx.textLength != LENGTH_UNKNOWN &&\n+\t node_ctx.type != REPO_MODE_DIR)\n+\t\tnode_ctx.mark = next_blob_mark();\n+\n+\tif (node_ctx.action == NODEACT_DELETE) {\n+\t\trepo_delete(node_ctx.dst);\n+\t} else if (node_ctx.action == NODEACT_CHANGE ||\n+\t\t\t node_ctx.action == NODEACT_REPLACE) {\n+\t\tif (node_ctx.action == NODEACT_REPLACE &&\n+\t\t node_ctx.type == REPO_MODE_DIR)\n+\t\t\trepo_replace(node_ctx.dst, node_ctx.mark);\n+\t\telse if (node_ctx.propLength != LENGTH_UNKNOWN)\n+\t\t\trepo_modify(node_ctx.dst, node_ctx.type, node_ctx.mark);\n+\t\telse if (node_ctx.textLength != LENGTH_UNKNOWN)\n+\t\t\tnode_ctx.srcMode = repo_replace(node_ctx.dst, node_ctx.mark);\n+\t} else if (node_ctx.action == NODEACT_ADD) {\n+\t\tif (node_ctx.srcRev && node_ctx.propLength != LENGTH_UNKNOWN)\n+\t\t\trepo_modify(node_ctx.dst, node_ctx.type, node_ctx.mark);\n+\t\telse if (node_ctx.srcRev && node_ctx.textLength != LENGTH_UNKNOWN)\n+\t\t\tnode_ctx.srcMode = repo_replace(node_ctx.dst, node_ctx.mark);\n+\t\telse if ((node_ctx.type == REPO_MODE_DIR && !node_ctx.srcRev) ||\n+\t\t node_ctx.textLength != LENGTH_UNKNOWN)\n+\t\t\trepo_add(node_ctx.dst, node_ctx.type, node_ctx.mark);\n+\t}\n+\n+\tif (node_ctx.propLength == LENGTH_UNKNOWN && node_ctx.srcMode)\n+\t\tnode_ctx.type = node_ctx.srcMode;\n+\n+\tif (node_ctx.mark)\n+\t\tfast_export_blob(node_ctx.type, node_ctx.mark, node_ctx.textLength);\n+\telse if (node_ctx.textLength != LENGTH_UNKNOWN)\n+\t\tbuffer_skip_bytes(node_ctx.textLength);\n+}\n+\n+static void handle_revision(void)\n+{\n+\tif (rev_ctx.revision)\n+\t\trepo_commit(rev_ctx.revision, rev_ctx.author, rev_ctx.log,\n+\t\t\tdump_ctx.uuid, dump_ctx.url, rev_ctx.timestamp);\n+}\n+\n+void svndump_read(char *url)\n+{\n+\tchar *val;\n+\tchar *t;\n+\tuint32_t active_ctx = DUMP_CTX;\n+\tuint32_t len;\n+\tuint32_t key;\n+\n+\treset_dump_ctx(pool_intern(url));\n+\twhile ((t = buffer_read_line())) {\n+\t\tval = strstr(t, \": \");\n+\t\tif (!val)\n+\t\t\tcontinue;\n+\t\t*val++ = '\\0';\n+\t\t*val++ = '\\0';\n+\t\tkey = pool_intern(t);\n+\n+\t\tif (key == keys.uuid) {\n+\t\t\tdump_ctx.uuid = pool_intern(val);\n+\t\t} else if (key == keys.revision_number) {\n+\t\t\tif (active_ctx == NODE_CTX)\n+\t\t\t\thandle_node();\n+\t\t\tif (active_ctx != DUMP_CTX)\n+\t\t\t\thandle_revision();\n+\t\t\tactive_ctx = REV_CTX;\n+\t\t\treset_rev_ctx(atoi(val));\n+\t\t} else if (key == keys.node_path) {\n+\t\t\tif (active_ctx == NODE_CTX)\n+\t\t\t\thandle_node();\n+\t\t\tactive_ctx = NODE_CTX;\n+\t\t\treset_node_ctx(val);\n+\t\t} else if (key == keys.node_kind) {\n+\t\t\tif (!strcmp(val, \"dir\"))\n+\t\t\t\tnode_ctx.type = REPO_MODE_DIR;\n+\t\t\telse if (!strcmp(val, \"file\"))\n+\t\t\t\tnode_ctx.type = REPO_MODE_BLB;\n+\t\t\telse\n+\t\t\t\tfprintf(stderr, \"Unknown node-kind: %s\\n\", val);\n+\t\t} else if (key == keys.node_action) {\n+\t\t\tif (!strcmp(val, \"delete\")) {\n+\t\t\t\tnode_ctx.action = NODEACT_DELETE;\n+\t\t\t} else if (!strcmp(val, \"add\")) {\n+\t\t\t\tnode_ctx.action = NODEACT_ADD;\n+\t\t\t} else if (!strcmp(val, \"change\")) {\n+\t\t\t\tnode_ctx.action = NODEACT_CHANGE;\n+\t\t\t} else if (!strcmp(val, \"replace\")) {\n+\t\t\t\tnode_ctx.action = NODEACT_REPLACE;\n+\t\t\t} else {\n+\t\t\t\tfprintf(stderr, \"Unknown node-action: %s\\n\", val);\n+\t\t\t\tnode_ctx.action = NODEACT_UNKNOWN;\n+\t\t\t}\n+\t\t} else if (key == keys.node_copyfrom_path) {\n+\t\t\tpool_tok_seq(REPO_MAX_PATH_DEPTH, node_ctx.src, \"/\", val);\n+\t\t} else if (key == keys.node_copyfrom_rev) {\n+\t\t\tnode_ctx.srcRev = atoi(val);\n+\t\t} else if (key == keys.text_content_length) {\n+\t\t\tnode_ctx.textLength = atoi(val);\n+\t\t} else if (key == keys.prop_content_length) {\n+\t\t\tnode_ctx.propLength = atoi(val);\n+\t\t} else if (key == keys.content_length) {\n+\t\t\tlen = atoi(val);\n+\t\t\tbuffer_read_line();\n+\t\t\tif (active_ctx == REV_CTX) {\n+\t\t\t\tread_props();\n+\t\t\t} else if (active_ctx == NODE_CTX) {\n+\t\t\t\thandle_node();\n+\t\t\t\tactive_ctx = REV_CTX;\n+\t\t\t} else {\n+\t\t\t\tfprintf(stderr, \"Unexpected content length header: %d\\n\", len);\n+\t\t\t\tbuffer_skip_bytes(len);\n+\t\t\t}\n+\t\t}\n+\t}\n+\tif (active_ctx == NODE_CTX)\n+\t\thandle_node();\n+\tif (active_ctx != DUMP_CTX)\n+\t\thandle_revision();\n+}\n+\n+void svndump_init(const char *filename)\n+{\n+\tbuffer_init(filename);\n+\trepo_init();\n+\treset_dump_ctx(~0);\n+\treset_rev_ctx(0);\n+\treset_node_ctx(NULL);\n+\tinit_keys();\n+}\n+\n+void svndump_reset(void)\n+{\n+\tlog_reset();\n+\tbuffer_reset();\n+\trepo_reset();\n+\treset_dump_ctx(~0);\n+\treset_rev_ctx(0);\n+\treset_node_ctx(NULL);\n+}\ndiff --git a/vcs-svn/svndump.h b/vcs-svn/svndump.h\nnew file mode 100644\nindex 0000000..38ad544\n--- /dev/null\n+++ b/vcs-svn/svndump.h\n@@ -0,0 +1,8 @@\n+#ifndef SVNDUMP_H_\n+#define SVNDUMP_H_\n+\n+void svndump_init(const char *filename);\n+void svndump_read(char *url);\n+void svndump_reset(void);\n+\n+#endif\n-- \n1.7.1\n"},{"id":"144138","messageId":"20100624110752.GI12376@burratino","threadId":"24185","inReplyTo":"20100624105004.GA12336@burratino","subject":"[PATCH 9/9] Add a sample user for the svndump library","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-06-24T11:07:52Z","receivedAt":"2010-06-24T11:07:52Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"The svn-fe tool takes a Subversion dump file as input and produces\na fast-import stream as output. This can be useful as a low-level\ntool in building other importers, or for debugging the vcs-svn\nlibrary.\n\n make svn-fe\n make svn-fe.1\n\nto test.\n\nNEEDSWORK: litters cwd with useless .bin files.\nBut I hope it is enough to show the idea.\n\nSigned-off-by: Jonathan Nieder \n---\nThanks for reading. Thoughts welcome.\n\n contrib/svn-fe/.gitignore | 3 ++\n contrib/svn-fe/Makefile | 63 +++++++++++++++++++++++++++++++++++++++++++++\n contrib/svn-fe/svn-fe.c | 43 ++++++++++++++++++++++++++++++\n contrib/svn-fe/svn-fe.txt | 56 ++++++++++++++++++++++++++++++++++++++++\n 4 files changed, 165 insertions(+), 0 deletions(-)\n create mode 100644 contrib/svn-fe/.gitignore\n create mode 100644 contrib/svn-fe/Makefile\n create mode 100644 contrib/svn-fe/svn-fe.c\n create mode 100644 contrib/svn-fe/svn-fe.txt\n\ndiff --git a/contrib/svn-fe/.gitignore b/contrib/svn-fe/.gitignore\nnew file mode 100644\nindex 0000000..27a33b6\n--- /dev/null\n+++ b/contrib/svn-fe/.gitignore\n@@ -0,0 +1,3 @@\n+/*.xml\n+/*.1\n+/*.html\ndiff --git a/contrib/svn-fe/Makefile b/contrib/svn-fe/Makefile\nnew file mode 100644\nindex 0000000..4cc8d15\n--- /dev/null\n+++ b/contrib/svn-fe/Makefile\n@@ -0,0 +1,63 @@\n+all:: svn-fe$X\n+\n+CC = gcc\n+RM = rm -f\n+MV = mv\n+\n+CFLAGS = -g -O2 -Wall\n+LDFLAGS =\n+ALL_CFLAGS = $(CFLAGS)\n+ALL_LDFLAGS = $(LDFLAGS)\n+EXTLIBS =\n+\n+GIT_LIB = ../../libgit.a\n+VCSSVN_LIB = ../../vcs-svn/lib.a\n+LIBS = $(VCSSVN_LIB) $(GIT_LIB) $(EXTLIBS)\n+\n+QUIET_SUBDIR0 = +$(MAKE) -C # space to separate -C and subdir\n+QUIET_SUBDIR1 =\n+\n+ifneq ($(findstring $(MAKEFLAGS),w),w)\n+PRINT_DIR = --no-print-directory\n+else # \"make -w\"\n+NO_SUBDIR = :\n+endif\n+\n+ifneq ($(findstring $(MAKEFLAGS),s),s)\n+ifndef V\n+\tQUIET_CC = @echo ' ' CC $@;\n+\tQUIET_LINK = @echo ' ' LINK $@;\n+\tQUIET_SUBDIR0 = +@subdir=\n+\tQUIET_SUBDIR1 = ;$(NO_SUBDIR) echo ' ' SUBDIR $$subdir; \\\n+\t $(MAKE) $(PRINT_DIR) -C $$subdir\n+endif\n+endif\n+\n+svn-fe$X: svn-fe.o $(VCSSVN_LIB) $(GIT_LIB)\n+\t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ svn-fe.o \\\n+\t\t$(ALL_LDFLAGS) $(LIBS)\n+\n+svn-fe.o: svn-fe.c ../../vcs-svn/svndump.h\n+\t$(QUIET_CC)$(CC) -o $*.o -c $(ALL_CFLAGS) $<\n+\n+svn-fe.html: svn-fe.txt\n+\t$(QUIET_SUBDIR0)../../Documentation $(QUIET_SUBDIR1) \\\n+\t\tMAN_TXT=../contrib/svn-fe/svn-fe.txt \\\n+\t\t../contrib/svn-fe/$@\n+\n+svn-fe.1: svn-fe.txt\n+\t$(QUIET_SUBDIR0)../../Documentation $(QUIET_SUBDIR1) \\\n+\t\tMAN_TXT=../contrib/svn-fe/svn-fe.txt \\\n+\t\t../contrib/svn-fe/$@\n+\t$(MV) ../../Documentation/svn-fe.1 .\n+\n+../../vcs-svn/lib.a: FORCE\n+\t$(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) vcs-svn/lib.a\n+\n+../../libgit.a: FORCE\n+\t$(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) libgit.a\n+\n+clean:\n+\t$(RM) svn-fe$X svn-fe.o svn-fe.html svn-fe.xml svn-fe.1\n+\n+.PHONY: all clean FORCE\ndiff --git a/contrib/svn-fe/svn-fe.c b/contrib/svn-fe/svn-fe.c\nnew file mode 100644\nindex 0000000..d84dd4f\n--- /dev/null\n+++ b/contrib/svn-fe/svn-fe.c\n@@ -0,0 +1,43 @@\n+/*\n+ * Parse and rearrange a svnadmin dump.\n+ * Create the dump with:\n+ * svnadmin dump --incremental -r: >outfile\n+ *\n+ * Copyright (C) 2010 David Barr .\n+ * All rights reserved.\n+ *\n+ * Redistribution and use in source and binary forms, with or without\n+ * modification, are permitted provided that the following conditions\n+ * are met:\n+ * 1. Redistributions of source code must retain the above copyright\n+ * notice(s), this list of conditions and the following disclaimer\n+ * unmodified other than the allowable addition of one or more\n+ * copyright notices.\n+ * 2. Redistributions in binary form must reproduce the above copyright\n+ * notice(s), this list of conditions and the following disclaimer in\n+ * the documentation and/or other materials provided with the\n+ * distribution.\n+ *\n+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER(S) ``AS IS'' AND ANY\n+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE\n+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR\n+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER(S) BE\n+ * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR\n+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF\n+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR\n+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,\n+ * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE\n+ * OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,\n+ * EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n+ */\n+\n+#include \n+#include \"../../vcs-svn/svndump.h\"\n+\n+int main(int argc, char **argv)\n+{\n+\tsvndump_init(NULL);\n+\tsvndump_read((argc > 1) ? argv[1] : NULL);\n+\tsvndump_reset();\n+\treturn 0;\n+}\ndiff --git a/contrib/svn-fe/svn-fe.txt b/contrib/svn-fe/svn-fe.txt\nnew file mode 100644\nindex 0000000..bfd3a68\n--- /dev/null\n+++ b/contrib/svn-fe/svn-fe.txt\n@@ -0,0 +1,56 @@\n+svn-fe(1)\n+=========\n+\n+NAME\n+----\n+svn-fe - convert an SVN \"dumpfile\" to a fast-import stream\n+\n+SYNOPSIS\n+--------\n+svnadmin dump --incremental REPO | svn-fe [url] | git fast-import\n+\n+DESCRIPTION\n+-----------\n+Converts a textual representation of a Subversion repository into\n+input suitable for git-fast-import(1) and similar importers.\n+\n+INPUT FORMAT\n+------------\n+Subversion's repository dump format is documented in full in\n+`notes/dump-load-format.txt` from the Subversion source tree.\n+Files in this format can be generated using the 'svnadmin dump' or\n+'svk admin dump' command.\n+\n+OUTPUT FORMAT\n+-------------\n+The fast-import format is documented by the git-fast-import(1)\n+manual page.\n+\n+NOTES\n+-----\n+Subversion dumps do not record a separate author and committer for\n+each revision, nor a separate display name and email address for\n+each author. Like git-svn(1), 'svn-fe' will use the name\n+\n+---------\n+user \n+---------\n+\n+as committer, where 'user' is the value of the `svn:author` property\n+and 'UUID' the repository's identifier.\n+\n+To support incremental imports, 'svn-fe' will put a `git-svn-id`\n+line at the end of each commit log message if passed an url on the\n+command line. This line has the form `git-svn-id: URL@REVNO UUID`.\n+\n+Empty directories and unknown properties are silently discarded.\n+\n+The resulting repository will generally require further processing\n+to put each project in its own repository and to separate the history\n+of each branch. The 'git filter-branch --subdirectory-filter' command\n+may be useful for this purpose.\n+\n+SEE ALSO\n+--------\n+git-svn(1), svn2git(1), svk(1), git-filter-branch(1), git-fast-import(1),\n+https://svn.apache.org/repos/asf/subversion/trunk/notes/dump-load-format.txt\n-- \n1.7.1\n"},{"id":"144149","messageId":"AANLkTilGNaF05EW2IFUpjOx-lP8sDoCkSrsa1-vMZo6B@mail.gmail.com","threadId":"24185","inReplyTo":"20100624105004.GA12336@burratino","subject":"Re: [PATCH/RFC v2 0/9] Subversion dump parsing library","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-24T13:06:50Z","receivedAt":"2010-06-24T13:06:50Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"(+CC: Junio)\n\nHi Jonathan,\n\nJonathan Nieder wrote:\n> Ram last sent this series a couple of weeks ago[1], and it was\n> merged to pu then as rr/svn-export.  Here’s another iteration\n> of the same for discussion, now including David Barr’s program\n> that demonstrates the functionality.\n\nThere's a lot of work happening in the exporter and the series will\nprobably need to be re-rolled again: I recommend the following.\n1. Review this series thoroughly, but don't actually merge it because\nit's going to be re-rolled soon.\n2. Split the series in two: The infrastructure part in vcs-svn/ will\nbe re-rolled later. Another part for contrib/ should be made into a\nseparate series and merged.\n3. As soon as the client is complete, I'll roll a series that puts it\nin vcs-svn/ (infrastructure again).\n4. Finally, I'll roll out a series for the remote helper itself that\nputs it in $GIT_ROOT.\n\nTo summarize, there should be four series:\n1. David's exporter in vcs-svn/ (yet to be re-rolled): Either Jonathan\nor I will handle this.\n2. David's independent svn-fe program in contrib/: Jonathan will handle this.\n3. The RA client to generate a full-text dumpfile on-the-fly: This\nisn't done yet; as soon as it's finished, I'll roll a series.\n4. The remote helper: This done, but it doesn't make sense to send\nthis in before everything else has been merged.\n\n-- Ram\n"},{"id":"144174","messageId":"20100624182424.GA1798@burratino","threadId":"24185","inReplyTo":"AANLkTilGNaF05EW2IFUpjOx-lP8sDoCkSrsa1-vMZo6B@mail.gmail.com","subject":"Re: [PATCH/RFC v2 0/9] Subversion dump parsing library","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-06-24T18:24:24Z","receivedAt":"2010-06-24T18:24:24Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Ramkumar Ramachandra wrote:\n\n> There's a lot of work happening in the exporter and the series will\n> probably need to be re-rolled again: I recommend the following.\n> 1. Review this series thoroughly, but don't actually merge it because\n> it's going to be re-rolled soon.\n\nTo be more specific: although svndump.c, fast_export.c, and\nrepo_tree.c may need to be a bit slushy now while we figure out the\nincremental import story, the rest of the series is not likely to\nchange much in broad strokes. I get the impression that facilities\nlike strbuf and string_list benefit greatly from early review, so in\nparticular let me mention the facilities this series adds:\n\n . obj_pool, an array of fixed-size records that can be written to\n disk. This is just begging to be implemented with mmap; maybe\n in the future the compat/mmap.c shim can be tweaked to support\n faking it.\n\n . treap, a multiset datastructure built on top of obj_pool. I\n suspect API cleanups would be welcome here: it’s a bit more\n unwieldly than string_list at the moment.\n\n . string_pool, a collection of interned strings built on top of\n treap.\n\n . line_buffer, a simple fread()/fgets() wrapper with a static buffer.\n\nI would find feedback on these (or patches :)) especially welcome.\n\nJonathan\n"},{"id":"144175","messageId":"AANLkTikkR3KaMPcltwJslAxna25bmFixIKLsGzf55_ZW@mail.gmail.com","threadId":"24185","inReplyTo":"20100624105146.GA12376@burratino","subject":"Re: [PATCH 1/9] Export parse_date_basic() to convert a date string to timestamp","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-24T18:32:51Z","receivedAt":"2010-06-24T18:32:51Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi Jonathan,\n\nThis specific patch is clear. I have some issues with some of the\nother parts in the series.\n\nJonathan Nieder wrote:\n> approxidate() is not appropriate for reading machine-written dates\n> because it guesses instead of erroring out on malformed dates.\n> parse_date() is less convenient since it returns its output as a\n> string.  So export the underlying function that writes a timestamp.\n\nRight. I couldn't justify exposing it in the series that's in master now.\n\n> While at it, change the return value to match the usual convention:\n> return 0 for success and -1 for failure.\n\nSince I'm to blame for this change,\nAcked-by: Ramkumar Ramachandra \n\n-- Ram\n"},{"id":"144177","messageId":"AANLkTilItX_xbbpX9mfKg0Dfx6ThzYuVsBu25C_-fJ6W@mail.gmail.com","threadId":"24185","inReplyTo":"20100624105339.GC12376@burratino","subject":"Re: [PATCH 3/9] Add memory pool library","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-24T18:43:55Z","receivedAt":"2010-06-24T18:43:55Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi Jonathan,\n\nJonathan Nieder wrote:\n> From: David Barr \n>\n> Add a memory pool library implemented using C macros. The obj_pool_gen()\n> macro creates a type-specific memory pool API.\n\nUntil some sort of mmap is implemented, I doubt we can do much better\nthan this. By marking all the generated functions MAYBE_UNUSED, we're\nactually suppressing more warnings than intended. Maybe we can avoid\nit and somehow find a way to mark only those functions that are really\nunused? Perhaps an extra parameter in obj_pool_gen?\n\n-- Ram\n"},{"id":"144178","messageId":"20100624185521.GB1848@burratino","threadId":"24185","inReplyTo":"AANLkTilItX_xbbpX9mfKg0Dfx6ThzYuVsBu25C_-fJ6W@mail.gmail.com","subject":"Re: [PATCH 3/9] Add memory pool library","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-06-24T18:55:21Z","receivedAt":"2010-06-24T18:55:21Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi Ram,\n\nRamkumar Ramachandra wrote:\n\n> By marking all the generated functions MAYBE_UNUSED, we're\n> actually suppressing more warnings than intended. Maybe we can avoid\n> it and somehow find a way to mark only those functions that are really\n> unused?\n\nIf we used templates instead of macros, a smart compiler would notice\nwhich functions are _never_ used. But sticking to C, I think it is\nfine to rely on humans checking by hand for now. (FWIW no obj_pool\nfunctions are unused at the moment.)\n\nI tried leaving out the MAYBE_UNUSED for foo_init() before I realized\nthat calling it is optional.\n\n> Perhaps an extra parameter in obj_pool_gen?\n\nFilling out such a list for each caller sounds to me like more trouble\nthan it’s worth.\n\nJonathan\n"},{"id":"144179","messageId":"AANLkTinYmKl4M10l3u5V7deleqSzr7iJDqf7X6ghtwRM@mail.gmail.com","threadId":"24185","inReplyTo":"20100624105706.GD12376@burratino","subject":"Re: [PATCH 4/9] Add treap implementation","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-24T19:08:13Z","receivedAt":"2010-06-24T19:08:13Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi again,\n\nJonathan Nieder wrote:\n> From: Jason Evans \n>\n> Provide macros to generate a type-specific treap implementation and\n> various functions to operate on it. It uses obj_pool.h to store memory\n> nodes in a treap.  Previously committed nodes are never removed from\n> the pool; after any *_commit operation, it is assumed (correctly, in\n> the case of svn-fast-export) that someone else must care about them.\n\nThis is likely to change in a few days. David is currently working on\na Java implementation of a immutable ternary treap and will\nre-implement it as C macros. See the ternary-treap branch.\n\n>  $(VCSSVN_OBJS): \\\n> -       vcs-svn/obj_pool.h\n> +       vcs-svn/obj_pool.h vcs-svn/trp.h\n\nInteresting how you've shown this in every patch :)\n\n> +/*\n> + * Fibonacci hash function.\n> + * The multiplier is the nearest prime to (2^32 times (√5 - 1)/2).\n> + * See Knuth §6.4: volume 3, 3rd ed, p518.\n> + */\n\nUm, is it alright to put non-ascii characters in a file containing\ncode? I haven't seen such a thing in any of the other files. Will some\nold compilers complain while parsing?\n\n> --- /dev/null\n> +++ b/vcs-svn/trp.txt\n> @@ -0,0 +1,90 @@\n> +treap API\n\nThe documentation is good, but I don't see it merged into the tree.\nPerhaps send a patch to David? Also, you might want to include the\ntechnical explanation for using treaps from the commit message here?\n\n-- Ram\n"},{"id":"144181","messageId":"AANLkTinqW1l3zmyrwtgmyueVaOKqYxyvC8r9SGXZ_O4E@mail.gmail.com","threadId":"24185","inReplyTo":"20100624105813.GE12376@burratino","subject":"Re: [PATCH 5/9] Add string-specific memory pool","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-24T19:19:47Z","receivedAt":"2010-06-24T19:19:47Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi,\n\nJonathan Nieder wrote:\n> Intern strings so they can be compared by address and stored without\n> wasting space.\n\nIt's unlikely that this'll change much, if at all. It should go in\nwithout any issues. I'm not sure this is appropriate since I've\nalready signed off, but anyway:\nReviewed-by: Ramkumar Ramachandra \n\n-- Ram\n"},{"id":"144182","messageId":"20100624192234.GC1848@burratino","threadId":"24185","inReplyTo":"AANLkTinYmKl4M10l3u5V7deleqSzr7iJDqf7X6ghtwRM@mail.gmail.com","subject":"Re: [PATCH 4/9] Add treap implementation","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-06-24T19:22:34Z","receivedAt":"2010-06-24T19:22:34Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Ramkumar Ramachandra wrote:\n\n> David is currently working on\n> a Java implementation of a immutable ternary treap and will\n> re-implement it as C macros. See the ternary-treap branch.\n\nI assume his blog post from a week or so ago[1] is relevant.\n\nThe trie-based version should be interesting but less generic (which\nis fine, since we only have two users and both use string keys).\n\n>>  $(VCSSVN_OBJS): \\\n>> -       vcs-svn/obj_pool.h\n>> +       vcs-svn/obj_pool.h vcs-svn/trp.h\n>\n> Interesting how you've shown this in every patch :)\n\nIt would have been less messy to use\n\n $(VCSSVN_OBJS): \\\n\tvcs-svn/obj_pool.h \\\n\tvcs-svn/trp.h \\\n\t... \\\n\nor\n\n $(VCSSVN_OBJS): vcs-svn/obj_pool.h\n $(VCSSVN_OBJS): vcs-svn/trp.h\n ...\n\nI just can’t bring myself to care much. :)\n\n> > +/*\n> > + * Fibonacci hash function.\n> > + * The multiplier is the nearest prime to (2^32 times (√5 - 1)/2).\n> > + * See Knuth §6.4: volume 3, 3rd ed, p518.\n> > + */\n> \n> Um, is it alright to put non-ascii characters in a file containing\n> code?\n\nYes, UTF-8 is sometimes used in comments for people’s names. See\nbuiltin/branch.c, for example.\n\n> The documentation is good, but I don't see it merged into the tree.\n> Perhaps send a patch to David?\n\nYes. I should send some other patches, too, to minimize the delta.\n\n> Also, you might want to include the\n> technical explanation for using treaps from the commit message here?\n\nGood idea, thanks!\n\nJonathan\n\n[1] http://barrbrain.github.com/2010/06/19/relocatable-immutable-randomised-ternary-search-tree-part-2.html\n"},{"id":"144183","messageId":"AANLkTikyjq7HQY1hUTOGF_oBsGDj_jUI-GwDBRdacCzG@mail.gmail.com","threadId":"24185","inReplyTo":"20100624110217.GG12376@burratino","subject":"Re: [PATCH 7/9] Add infrastructure to write revisions in fast-export format","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-24T19:29:29Z","receivedAt":"2010-06-24T19:29:29Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi again,\n\nJonathan Nieder wrote:\n> repo_tree maintains the exporter's state and provides a facility to to\n> call fast_export, which writes objects to stdout suitable for\n> consumption by fast-import.\n\nThese files will also change significantly in a few days- see the\nternary_treap branch.\n\n> The exported functions roughly correspond to Subversion FS operations.\n\nThis description is sufficient for the commit message.\n\n>  . repo_add adds a file to the current commit.\n>\n>  . repo_modify adds a replacement for an existing file;\n>   it is implemented exactly the same way, but a check could be\n>   added later to distinguish the two cases.\n>\n>  . repo_copy copies a blob from a previous revision to the current\n>   commit.\n>\n>  . repo_replace modifies the content of a file from the current\n>   commit, if and only if it exists.\n>\n>  . repo_delete removes a file or directory from the current commit.\n>\n>  . repo_commit calls out to fast_export to write the current commit to\n>   the fast-import stream in stdout.\n>\n>  . repo_diff is used by the fast_export module to write the changes\n>   for a commit.\n>\n>  . repo_reset erases the exporter's state, so valgrind can be happy.\n\nThis is like API documentation- should it go into the commit message?\nMaybe put this in a a dedicated repo_tree.txt like trp.h?\n\n-- Ram\n"},{"id":"144185","messageId":"20100624193629.GA2065@burratino","threadId":"24185","inReplyTo":"AANLkTikyjq7HQY1hUTOGF_oBsGDj_jUI-GwDBRdacCzG@mail.gmail.com","subject":"Re: [PATCH 7/9] Add infrastructure to write revisions in fast-export format","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-06-24T19:36:29Z","receivedAt":"2010-06-24T19:36:29Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Ramkumar Ramachandra wrote:\n\n> This is like API documentation- should it go into the commit message?\n> Maybe put this in a a dedicated repo_tree.txt like trp.h?\n\nGood question. The answer depends on whether we want to keep it\nmaintained.\n\nSince this is not meant to be a general-purpose API, I thought it\nsimplest to document it in the commit message and let future commit\nmessages describe the purpose of future changes. We are not\napproaching any limit on the length of a commit object, after all.\n\nJonathan\n"},{"id":"144186","messageId":"AANLkTikNV0m5E7982wY5wdtcMstpfKKMX4MdZvdhqhOb@mail.gmail.com","threadId":"24185","inReplyTo":"20100624185521.GB1848@burratino","subject":"Re: [PATCH 3/9] Add memory pool library","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-24T19:37:42Z","receivedAt":"2010-06-24T19:37:42Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi Jonathan,\n\nJonathan Nieder wrote:\n> If we used templates instead of macros, a smart compiler would notice\n> which functions are _never_ used.  But sticking to C, I think it is\n> fine to rely on humans checking by hand for now.  (FWIW no obj_pool\n> functions are unused at the moment.)\n\n> Filling out such a list for each caller sounds to me like more trouble\n> than it’s worth.\n\nOkay, I have an idea. We'll document it as a comment so people will be\nable to see which exact functions were unused in this import. The\nfollowing functions are unused: blob_init, blob_offset, blob_commit,\ncommit_free, commit_offset, dir_free, node_init, node_commit,\nstring_offset, tree_first, tree_next, tree_remove, log_init,\nlog_offset, and log_commit.\n\n-- Ram\n"},{"id":"144187","messageId":"20100624194927.GB2065@burratino","threadId":"24185","inReplyTo":"AANLkTikyjq7HQY1hUTOGF_oBsGDj_jUI-GwDBRdacCzG@mail.gmail.com","subject":"Re: [PATCH 7/9] Add infrastructure to write revisions in fast-export format","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-06-24T19:49:27Z","receivedAt":"2010-06-24T19:49:27Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"On Thu, Jun 24, 2010 at 09:29:29PM +0200, Ramkumar Ramachandra wrote:\n> Jonathan Nieder wrote:\n\n>>  . repo_reset erases the exporter's state, so valgrind can be happy.\n> \n> This is like API documentation- should it go into the commit message?\n\nA more terse summary, to save readers time:\n\n . repo_add, repo_modify, repo_copy, repo_replace, and repo_delete\n update the current commit, based roughly on the corresponding\n Subversion FS operation.\n\n . repo_commit calls out to fast_export to write the current commit to\n the fast-import stream in stdout.\n\n . repo_diff is used by the fast_export module to write the changes\n for a commit.\n\n . repo_reset erases the exporter's state, so valgrind can be happy.\n\nJonathan\n"},{"id":"144188","messageId":"20100624200641.GA2209@burratino","threadId":"24185","inReplyTo":"AANLkTikNV0m5E7982wY5wdtcMstpfKKMX4MdZvdhqhOb@mail.gmail.com","subject":"Re: [PATCH 3/9] Add memory pool library","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-06-24T20:06:41Z","receivedAt":"2010-06-24T20:06:41Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Ramkumar Ramachandra wrote:\n\n> The\n> following functions are unused: blob_init, blob_offset, blob_commit,\n> commit_free, commit_offset, dir_free, node_init, node_commit,\n> string_offset, tree_first, tree_next, tree_remove, log_init,\n> log_offset, and log_commit.\n\nThanks for this list. Do you think it’s worth automating its\nproduction? i.e., a masochistic person could write a script to\ncompile with the __attribute__((unused)) suppressed, parse warnings to\nfind unused functions, and then take an intersection of sets to\nconfirm that no family of functions is unused.\n\nJonathan\n"},{"id":"144192","messageId":"AANLkTinnB_TRYa0nzJHHqbHuJk9a7tfk8mAhr95Sc98E@mail.gmail.com","threadId":"24185","inReplyTo":"20100624110752.GI12376@burratino","subject":"Re: [PATCH 9/9] Add a sample user for the svndump library","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-24T20:17:09Z","receivedAt":"2010-06-24T20:17:09Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi Jonathan,\n\nJonathan Nieder wrote:\n> NEEDSWORK: litters cwd with useless .bin files.\n> But I hope it is enough to show the idea.\n\nHow do you propose we solve this? Maybe using a generic\n$TEMP_DIRECTORY like /tmp in Unix and then getting rid of the files\nafter the export is complete?\n\n> +QUIET_SUBDIR0 = +$(MAKE) -C # space to separate -C and subdir\n> +QUIET_SUBDIR1 =\n\n> +ifneq ($(findstring $(MAKEFLAGS),s),s)\n> +ifndef V\n> +       QUIET_CC      = @echo '   ' CC $@;\n> +       QUIET_LINK    = @echo '   ' LINK $@;\n> +       QUIET_SUBDIR0 = +@subdir=\n> +       QUIET_SUBDIR1 = ;$(NO_SUBDIR) echo '   ' SUBDIR $$subdir; \\\n> +                       $(MAKE) $(PRINT_DIR) -C $$subdir\n> +endif\n> +endif\n\nI saw this in the Git Makefile too, but I didn't understand the logic\nbehind it. Could you explain it to me?\nNote: I couldn't understand most of the Makefile, so I just skipped it\nwhen I found similar declarations in the Git Makefile.\n\n> diff --git a/contrib/svn-fe/svn-fe.c b/contrib/svn-fe/svn-fe.c\n> new file mode 100644\n> index 0000000..d84dd4f\n> --- /dev/null\n> +++ b/contrib/svn-fe/svn-fe.c\n> @@ -0,0 +1,43 @@\n> +/*\n> + * Parse and rearrange a svnadmin dump.\n> + * Create the dump with:\n> + * svnadmin dump --incremental -r: >outfile\n> + *\n> + * Copyright (C) 2010 David Barr .\n> + * All rights reserved.\n\nThat's a huge license header applies just to the trivial five-line\nprogram, right? Is it necessary at all?\n\n> +#include \n> +#include \"../../vcs-svn/svndump.h\"\n\nInelegant. Why not include ../../vcs-svn in the path you're searching\nfor headers?\n\n> +svnadmin dump --incremental REPO | svn-fe [url] | git fast-import\n\nIf the user doesn't have a clue about SVN, they won't know what REPO\nis here: Without knowing anything about svnadmin, I'd naively try it\nwith a remote repository. Maybe include a note about having to mirror\na complete repository locally using svnsync (or otherwise) first?\n\n> +Converts a textual representation of a Subversion repository into\n> +input suitable for git-fast-import(1) and similar importers.\n\nTo be more specific, \"Subversion dumpfile (version: 2)\" from FILE(1).\n\n-- Ram\n"},{"id":"144193","messageId":"AANLkTimeWwsodoVK-9Yx65MBOk0IhgxxVgAjGpIuHUE7@mail.gmail.com","threadId":"24185","inReplyTo":"20100624200641.GA2209@burratino","subject":"Re: [PATCH 3/9] Add memory pool library","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-24T20:20:41Z","receivedAt":"2010-06-24T20:20:41Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi again,\n\nJonathan Nieder wrote:\n> Thanks for this list.  Do you think it’s worth automating its\n> production?  i.e., a masochistic person could write a script to\n> compile with the __attribute__((unused)) suppressed, parse warnings to\n> find unused functions, and then take an intersection of sets to\n> confirm that no family of functions is unused.\n\nEr, I think that's a bit of an overkill :p\nSomeone editing the code can always suppress MAYBE_UNUSED and check\nwith our list of unused functions in the comment by hand.\n\n-- Ram\n"},{"id":"144195","messageId":"AANLkTim3ApkAjSoy4CD7Q_o7ECP0_KYJKErxt8AeXsN4@mail.gmail.com","threadId":"24185","inReplyTo":"20100624105243.GB12376@burratino","subject":"Re: [PATCH 2/9] Introduce vcs-svn lib","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-24T20:27:27Z","receivedAt":"2010-06-24T20:27:27Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi,\n\nJonathan Nieder wrote:\n> Teach the build system to build a separate library for the\n> upcoming subversion interop support.\n>\n> The resulting vcs-svn/lib.a does not contain any code, nor is\n> it built during a normal build.  This is just scaffolding for\n> later changes.\n\nThis is very elegant indeed!\nAcked-by: Ramkumar Ramachandra \n\n-- Ram\n"},{"id":"144196","messageId":"20100624203041.GA2262@burratino","threadId":"24185","inReplyTo":"AANLkTinnB_TRYa0nzJHHqbHuJk9a7tfk8mAhr95Sc98E@mail.gmail.com","subject":"Re: [PATCH 9/9] Add a sample user for the svndump library","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-06-24T20:30:41Z","receivedAt":"2010-06-24T20:30:41Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Ramkumar Ramachandra wrote:\n> Jonathan Nieder wrote:\n\n>> NEEDSWORK: litters cwd with useless .bin files.\n>> But I hope it is enough to show the idea.\n>\n> How do you propose we solve this?\n\nTurn off persistence until it is ready. At that point, we will need\nto access the target git repo anyway, so we can keep extra metadata in\nthe .git directory.\n\n> > +QUIET_SUBDIR0 = +$(MAKE) -C # space to separate -C and subdir\n> > +QUIET_SUBDIR1 =\n> \n> > +ifneq ($(findstring $(MAKEFLAGS),s),s)\n> > +ifndef V\n> > +       QUIET_CC      = @echo '   ' CC $@;\n> > +       QUIET_LINK    = @echo '   ' LINK $@;\n> > +       QUIET_SUBDIR0 = +@subdir=\n> > +       QUIET_SUBDIR1 = ;$(NO_SUBDIR) echo '   ' SUBDIR $$subdir; \\\n> > +                       $(MAKE) $(PRINT_DIR) -C $$subdir\n> > +endif\n> > +endif\n> \n> I saw this in the Git Makefile too, but I didn't understand the logic\n> behind it. Could you explain it to me?\n\nSee commit 74f2b2a.\n\nSummary: this produces the\n\n CC foo.o\n\nlines. The idea is that long command lines distract from what is more\nimportant, which is the compiler output. The behavior can be turned\noff with “make V=1” or “make -s”.\n\n>> diff --git a/contrib/svn-fe/svn-fe.c b/contrib/svn-fe/svn-fe.c\n>> new file mode 100644\n>> index 0000000..d84dd4f\n>> --- /dev/null\n>> +++ b/contrib/svn-fe/svn-fe.c\n>> @@ -0,0 +1,43 @@\n>> +/*\n>> + * Parse and rearrange a svnadmin dump.\n>> + * Create the dump with:\n>> + * svnadmin dump --incremental -r: >outfile\n>> + *\n>> + * Copyright (C) 2010 David Barr .\n>> + * All rights reserved.\n>\n> That's a huge license header applies just to the trivial five-line\n> program, right? Is it necessary at all?\n\nI dunno. I included the license header instead of refering to LICENSE\nbecause this file tends to be installed in /usr/share/doc/git/contrib\nand LICENSE does not.\n\nMaybe the file should get a simpler license? e.g.:\n\n This file is in the public domain.\n You may freely use, modify, distribute, and relicense it.\n\n>> +#include \n>> +#include \"../../vcs-svn/svndump.h\"\n>\n> Inelegant. Why not include ../../vcs-svn in the path you're searching\n> for headers?\n\nRight, this should be changed to\n\n #include \n #include \"vcs-svn/svndump.h\"\n\n>> +svnadmin dump --incremental REPO | svn-fe [url] | git fast-import\n>\n> If the user doesn't have a clue about SVN, they won't know what REPO\n> is here: Without knowing anything about svnadmin, I'd naively try it\n> with a remote repository. Maybe include a note about having to mirror\n> a complete repository locally using svnsync (or otherwise) first?\n\nSounds reasonable. Care to suggest wording?\n\n>> +Converts a textual representation of a Subversion repository into\n>> +input suitable for git-fast-import(1) and similar importers.\n>\n> To be more specific, \"Subversion dumpfile (version: 2)\" from FILE(1).\n\nDo version 3 dumpfiles fail?\n\nJonathan\n"},{"id":"144198","messageId":"AANLkTimql4TtCvkbgZg2nWhPGDhc1nkQpOJAaiRSxyS5@mail.gmail.com","threadId":"24185","inReplyTo":"20100624110325.GH12376@burratino","subject":"Re: [PATCH 8/9] Add SVN dump parser","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-24T20:33:43Z","receivedAt":"2010-06-24T20:33:43Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi,\n\nJonathan Nieder wrote:\n> svndump parses data that is in SVN dumpfile format produced by\n> `svnadmin dump` with the help of line_buffer and uses repo_tree and\n> fast_export to emit a git fast-import stream.\n\nThis hasn't changed from last time, and isn't expected to change in\nfuture. Should go in without any issues, although I'd love it if we\ncould somehow refactor the huge if-else tree. Again, I'm not sure if\nthis is appropriate since I've already signed off:\nReviewed-by: Ramkumar Ramachandra \n\n-- Ram\n"},{"id":"144199","messageId":"AANLkTinSbQbT1tmKIiWnZnX52mJEZOlf9xCV1ENUGFT_@mail.gmail.com","threadId":"24185","inReplyTo":"20100624203041.GA2262@burratino","subject":"Re: [PATCH 9/9] Add a sample user for the svndump library","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-24T20:42:26Z","receivedAt":"2010-06-24T20:42:26Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi again,\n\nJonathan Nieder wrote:\n> Turn off persistence until it is ready.  At that point, we will need\n> to access the target git repo anyway, so we can keep extra metadata in\n> the .git directory.\n\nUnfortunately, turning off persistence isn't so easy now because the\nbranch has been merged into master now, and it's difficult to sort out\njust the commits that correspond to persistence and rebase.\nYes, the remote helper can use the .git directory, but I thought we\nwanted to keep this in contrib/ even after the remote helper is\nmerged?\n\n> See commit 74f2b2a.\n>\n> Summary: this produces the\n>\n>    CC foo.o\n>\n> lines.  The idea is that long command lines distract from what is more\n> important, which is the compiler output.  The behavior can be turned\n> off with “make V=1” or “make -s”.\n\nAh. Black magic :)\n\n> Maybe the file should get a simpler license?  e.g.:\n>\n>  This file is in the public domain.\n>  You may freely use, modify, distribute, and relicense it.\n\nYes, I like this.\n\n> Sounds reasonable.  Care to suggest wording?\n\nSomething along \"REPO is a path to a Subversion repository mirrored on\nthe local disk. Remote Subversion repositories can be mirrored on\nlocal disk using the `svnsync` command.\"\n\n> Do version 3 dumpfiles fail?\n\nYes, they do. We aren't parsing the extra headers anywhere, and\ndeltified dumps aren't supported.\n\n-- Ram\n"},{"id":"144206","messageId":"20100624205256.GA2307@burratino","threadId":"24185","inReplyTo":"AANLkTinSbQbT1tmKIiWnZnX52mJEZOlf9xCV1ENUGFT_@mail.gmail.com","subject":"Re: [PATCH 9/9] Add a sample user for the svndump library","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-06-24T20:52:56Z","receivedAt":"2010-06-24T20:52:56Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Ramkumar Ramachandra wrote:\n\n> Unfortunately, turning off persistence isn't so easy now\n\nHmm, couldn’t one just leave out some pool_init() calls?\n\nOf course, that would make pool_init() and related functionality unused,\nand we may or may not want to remove it while at it.\n\n> Yes, the remote helper can use the .git directory, but I thought we\n> wanted to keep this in contrib/ even after the remote helper is\n> merged?\n\nFor incremental imports, I suspect the standalone svn-fe is going to\nneed to invoke ‘git fast-import’ itself, which implies knowledge of\nthe location of the .git directory.\n\nFor undeltified nonincremental imports, there is no problem. :)\n\n> Jonathan Nieder wrote:\n\n>> Maybe the file should get a simpler license?  e.g.:\n>>\n>>  This file is in the public domain.\n>>  You may freely use, modify, distribute, and relicense it.\n>\n> Yes, I like this.\n\nDavid, would this be okay? (for the short svn-fe.c file only)\n\n>> Do version 3 dumpfiles fail?\n>\n> Yes, they do. We aren't parsing the extra headers anywhere, and\n> deltified dumps aren't supported.\n\nWorth documenting indeed.\n\nThanks again.\n"},{"id":"144208","messageId":"AANLkTimt-nKl_ibb0Ol925luTPwCAR0QJkKFuirDUDIx@mail.gmail.com","threadId":"24185","inReplyTo":"20100624194927.GB2065@burratino","subject":"Re: [PATCH 7/9] Add infrastructure to write revisions in fast-export format","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-24T21:14:33Z","receivedAt":"2010-06-24T21:14:33Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Jonathan Nieder wrote:\n> A more terse summary, to save readers time:\n>\n>  . repo_add, repo_modify, repo_copy, repo_replace, and repo_delete\n>   update the current commit, based roughly on the corresponding\n>   Subversion FS operation.\n>\n>  . repo_commit calls out to fast_export to write the current commit to\n>   the fast-import stream in stdout.\n>\n>  . repo_diff is used by the fast_export module to write the changes\n>   for a commit.\n>\n>  . repo_reset erases the exporter's state, so valgrind can be happy.\n\nLooks good.\n\n-- Ram\n"},{"id":"144209","messageId":"AANLkTinQT6YvwkYrehAl5kGW9Ebkq1AantCap5nD8ByR@mail.gmail.com","threadId":"24185","inReplyTo":"20100624110112.GF12376@burratino","subject":"Re: [PATCH 6/9] Add stream helper library","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-06-24T21:23:58Z","receivedAt":"2010-06-24T21:23:58Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Jonathan Nieder wrote:\n> This library provides thread-unsafe fgets()- and fread()-like\n> functions where the caller does not have to supply a buffer.  It\n> maintains a couple of static buffers and provides an API to use\n> them.\n\nFew (no?) changes since last time.\nJust a quick reminder: eventually, we might be able to factor out this\nline_buffer thing completely; it's quite non-trivial though.\n\n-- Ram\n"},{"id":"144210","messageId":"20100624212602.GA2458@burratino","threadId":"24185","inReplyTo":"AANLkTilGNaF05EW2IFUpjOx-lP8sDoCkSrsa1-vMZo6B@mail.gmail.com","subject":"Re: [PATCH/RFC v2 0/9] Subversion dump parsing library","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-06-24T21:26:02Z","receivedAt":"2010-06-24T21:26:02Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Ramkumar Ramachandra wrote:\n\n> I'm in favor of the split mainly because it doesn't make sense to\n> re-roll your contrib/ patch everytime the infrastructure in vcs-svn/\n> changes\n[...]\n> I want to get the contrib/ patch in so people can test\n> the infrastructure everytime it's re-rolled and goes to `pu`.\n\nYes, I agree: ideally this should be two branches:\n\n rr/svn-export for infrastructure\n xx/svn-fe for the frontend\n\nwith the latter based on the former.\n"},{"id":"144211","messageId":"20100624212904.GB2458@burratino","threadId":"24185","inReplyTo":"AANLkTinQT6YvwkYrehAl5kGW9Ebkq1AantCap5nD8ByR@mail.gmail.com","subject":"Re: [PATCH 6/9] Add stream helper library","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-06-24T21:29:04Z","receivedAt":"2010-06-24T21:29:04Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Ramkumar Ramachandra wrote:\n> Jonathan Nieder wrote:\n\n>> This library provides thread-unsafe fgets()- and fread()-like\n>> functions where the caller does not have to supply a buffer.  It\n>> maintains a couple of static buffers and provides an API to use\n>> them.\n>\n> Few (no?) changes since last time.\n\nI simplified it a bit by getting rid of pushback.\n\n static uint32_t line_buffer_len = 0;\n static uint32_t line_len = 0;\n\nSorry, I should have mentioned so.\n\nJonathan\n"},{"id":"144490","messageId":"1277863773.23613.14.camel@wilber","threadId":"24185","inReplyTo":"20100624110752.GI12376@burratino","subject":"Re: [PATCH 9/9] Add a sample user for the svndump library","fromName":"Sam Vilain","fromEmail":"sam@vilain.net","sentAt":"2010-06-30T02:09:33Z","receivedAt":"2010-06-30T02:09:33Z","isPatch":true,"sender":{"key":"sam@vilain.net","avatar":"https://gravatar.com/avatar/8fc840ca854dbf6f7065b4335e3b934951c1dca3b11db688e95e471901f8f4a8?d=mp&s=160"},"body":"On Thu, 2010-06-24 at 06:07 -0500, Jonathan Nieder wrote:\n> +To support incremental imports, 'svn-fe' will put a `git-svn-id`\n> +line at the end of each commit log message if passed an url on the\n> +command line. This line has the form `git-svn-id: URL@REVNO UUID`.\n\nIf you are importing from an svk mirror or svnsync mirror, it will be\nrequired to rewrite this portion.\n\n> +Empty directories and unknown properties are silently discarded.\n\nYeah. These should probably be carried over in this pass. Revision\nproperties could possibly be converted to extra RFC822-style headers in\nthe commit message. Directory properties can go under $dir/.svnfe-props\n(use an empty file to mark empty directories) and $filename.svnfe-props\n- it is up to the data mining phase whether it wants to actually do\nanything with that data later.\n\nSam\n"}]}