{"thread":{"id":"24407","subject":"[PATCH 0/8] Resurrect rr/svn-export","startedAt":"2010-07-15T16:22:56Z","lastAt":"2010-10-18T17:03:31Z","messageCount":79,"participants":["Ramkumar Ramachandra","Jonathan Nieder","Junio C Hamano","Sam Vilain"],"isPatch":true,"patchVersion":1,"patchTotal":8},"messages":[{"id":"145628","messageId":"1279210984-31604-1-git-send-email-artagnon@gmail.com","threadId":"24407","inReplyTo":null,"subject":"[PATCH 0/8] Resurrect rr/svn-export","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-07-15T16:22:56Z","receivedAt":"2010-07-15T16:22:56Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi,\n\nI've decided to get this series merged now instead of waiting for the\nternary treap refactor. It's in excellent shape, thanks to Jonathan's\nconstant reviews/ fixes and David's constant refactoring. Since\nJonathan's last send, I've incoporated some suggestions from my own\nreview of Jonathan's series and split it into two series as I proposed\nearlier; I'll send the second series (rr/contrib-svn-fe) shortly after\nthis.\n\nOnce this series is merged, I estimate that the following will come in\nas incremental patches when the work is finished:\nrr/ternary-trp-refactor\nrr/zero-tree-refactor\nrr/dumpfilev3-parser\n\nThanks.\n\n-- Ram\n\nDavid Barr (5):\n  Add memory pool library\n  Add string-specific memory pool\n  Add stream helper library\n  Add infrastructure to write revisions in fast-export format\n  Add SVN dump parser\n\nJason Evans (1):\n  Add treap implementation\n\nJonathan Nieder (2):\n  Export parse_date_basic() to convert a date string to timestamp\n  Introduce vcs-svn lib\n\n Makefile              |   12 ++-\n cache.h               |    1 +\n date.c                |   14 +--\n vcs-svn/LICENSE       |   33 +++++\n vcs-svn/fast_export.c |   75 +++++++++++\n vcs-svn/fast_export.h |   14 ++\n vcs-svn/line_buffer.c |   93 ++++++++++++++\n vcs-svn/line_buffer.h |   14 ++\n vcs-svn/obj_pool.h    |   80 ++++++++++++\n vcs-svn/repo_tree.c   |  335 +++++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/repo_tree.h   |   26 ++++\n vcs-svn/string_pool.c |  114 +++++++++++++++++\n vcs-svn/string_pool.h |   15 +++\n vcs-svn/svndump.c     |  289 ++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/svndump.h     |    8 ++\n vcs-svn/trp.h         |  220 ++++++++++++++++++++++++++++++++\n vcs-svn/trp.txt       |  102 +++++++++++++++\n 17 files changed, 1436 insertions(+), 9 deletions(-)\n create mode 100644 vcs-svn/LICENSE\n create mode 100644 vcs-svn/fast_export.c\n create mode 100644 vcs-svn/fast_export.h\n create mode 100644 vcs-svn/line_buffer.c\n create mode 100644 vcs-svn/line_buffer.h\n create mode 100644 vcs-svn/obj_pool.h\n create mode 100644 vcs-svn/repo_tree.c\n create mode 100644 vcs-svn/repo_tree.h\n create mode 100644 vcs-svn/string_pool.c\n create mode 100644 vcs-svn/string_pool.h\n create mode 100644 vcs-svn/svndump.c\n create mode 100644 vcs-svn/svndump.h\n create mode 100644 vcs-svn/trp.h\n create mode 100644 vcs-svn/trp.txt\n"},{"id":"145629","messageId":"1279210984-31604-2-git-send-email-artagnon@gmail.com","threadId":"24407","inReplyTo":"1279210984-31604-1-git-send-email-artagnon@gmail.com","subject":"[PATCH 1/8] Export parse_date_basic() to convert a date string to timestamp","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-07-15T16:22:57Z","receivedAt":"2010-07-15T16:22:57Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"From: Jonathan Nieder <jrnieder@gmail.com>\n\napproxidate() is not appropriate for reading machine-written dates\nbecause it guesses instead of erroring out on malformed dates.\nparse_date() is less convenient since it returns its output as a\nstring.  So export the underlying function that writes a timestamp.\n\nWhile at it, change the return value to match the usual convention:\nreturn 0 for success and -1 for failure.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\nAcked-by: Ramkumar Ramachandra <artagnon@gmail.com>\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\n---\n cache.h |    1 +\n date.c  |   14 ++++++--------\n 2 files changed, 7 insertions(+), 8 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex c9fa3df..68258be 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -811,6 +811,7 @@ const char *show_date_relative(unsigned long time, int tz,\n \t\t\t       char *timebuf,\n \t\t\t       size_t timebuf_size);\n int parse_date(const char *date, char *buf, int bufsize);\n+int parse_date_basic(const char *date, unsigned long *timestamp, int *offset);\n void datestamp(char *buf, int bufsize);\n #define approxidate(s) approxidate_careful((s), NULL)\n unsigned long approxidate_careful(const char *, int *);\ndiff --git a/date.c b/date.c\nindex 3c981f7..00f9eb5 100644\n--- a/date.c\n+++ b/date.c\n@@ -586,7 +586,7 @@ static int date_string(unsigned long date, int offset, char *buf, int len)\n \n /* Gr. strptime is crap for this; it doesn't have a way to require RFC2822\n    (i.e. English) day/month names, and it doesn't work correctly with %z. */\n-int parse_date_toffset(const char *date, unsigned long *timestamp, int *offset)\n+int parse_date_basic(const char *date, unsigned long *timestamp, int *offset)\n {\n \tstruct tm tm;\n \tint tm_gmt;\n@@ -642,17 +642,16 @@ int parse_date_toffset(const char *date, unsigned long *timestamp, int *offset)\n \n \tif (!tm_gmt)\n \t\t*timestamp -= *offset * 60;\n-\treturn 1; /* success */\n+\treturn 0; /* success */\n }\n \n int parse_date(const char *date, char *result, int maxlen)\n {\n \tunsigned long timestamp;\n \tint offset;\n-\tif (parse_date_toffset(date, &timestamp, &offset) > 0)\n-\t\treturn date_string(timestamp, offset, result, maxlen);\n-\telse\n+\tif (parse_date_basic(date, &timestamp, &offset))\n \t\treturn -1;\n+\treturn date_string(timestamp, offset, result, maxlen);\n }\n \n enum date_mode parse_date_format(const char *format)\n@@ -1004,9 +1003,8 @@ unsigned long approxidate_relative(const char *date, const struct timeval *tv)\n \tint offset;\n \tint errors = 0;\n \n-\tif (parse_date_toffset(date, &timestamp, &offset) > 0)\n+\tif (!parse_date_basic(date, &timestamp, &offset))\n \t\treturn timestamp;\n-\n \treturn approxidate_str(date, tv, &errors);\n }\n \n@@ -1019,7 +1017,7 @@ unsigned long approxidate_careful(const char *date, int *error_ret)\n \tif (!error_ret)\n \t\terror_ret = &dummy;\n \n-\tif (parse_date_toffset(date, &timestamp, &offset) > 0) {\n+\tif (!parse_date_basic(date, &timestamp, &offset)) {\n \t\t*error_ret = 0;\n \t\treturn timestamp;\n \t}\n-- \n1.7.1\n"},{"id":"145633","messageId":"1279210984-31604-3-git-send-email-artagnon@gmail.com","threadId":"24407","inReplyTo":"1279210984-31604-1-git-send-email-artagnon@gmail.com","subject":"[PATCH 2/8] Introduce vcs-svn lib","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-07-15T16:22:58Z","receivedAt":"2010-07-15T16:22:58Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"From: Jonathan Nieder <jrnieder@gmail.com>\n\nTeach the build system to build a separate library for the\nupcoming subversion interop support.\n\nThe resulting vcs-svn/lib.a does not contain any code, nor is\nit built during a normal build.  This is just scaffolding for\nlater changes.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\nAcked-by: Ramkumar Ramachandra <artagnon@gmail.com>\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\n---\n Makefile |    8 +++++++-\n 1 files changed, 7 insertions(+), 1 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 9aca8a1..6441dcb 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -468,6 +468,7 @@ export PYTHON_PATH\n \n LIB_FILE=libgit.a\n XDIFF_LIB=xdiff/lib.a\n+VCSSVN_LIB=vcs-svn/lib.a\n \n LIB_H += advice.h\n LIB_H += archive.h\n@@ -1739,7 +1740,8 @@ ifndef NO_CURL\n endif\n XDIFF_OBJS = xdiff/xdiffi.o xdiff/xprepare.o xdiff/xutils.o xdiff/xemit.o \\\n \txdiff/xmerge.o xdiff/xpatience.o\n-OBJECTS := $(GIT_OBJS) $(XDIFF_OBJS)\n+VCSSVN_OBJS =\n+OBJECTS := $(GIT_OBJS) $(XDIFF_OBJS) $(VCSSVN_OBJS)\n \n dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d)\n dep_dirs := $(addsuffix .depend,$(sort $(dir $(OBJECTS))))\n@@ -1860,6 +1862,8 @@ http.o http-walker.o http-push.o remote-curl.o: http.h\n xdiff-interface.o $(XDIFF_OBJS): \\\n \txdiff/xinclude.h xdiff/xmacros.h xdiff/xdiff.h xdiff/xtypes.h \\\n \txdiff/xutils.h xdiff/xprepare.h xdiff/xdiffi.h xdiff/xemit.h\n+\n+$(VCSSVN_OBJS):\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\n@@ -1908,6 +1912,8 @@ $(LIB_FILE): $(LIB_OBJS)\n $(XDIFF_LIB): $(XDIFF_OBJS)\n \t$(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $(XDIFF_OBJS)\n \n+$(VCSSVN_LIB): $(VCSSVN_OBJS)\n+\t$(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $(VCSSVN_OBJS)\n \n doc:\n \t$(MAKE) -C Documentation all\n-- \n1.7.1\n"},{"id":"145631","messageId":"1279210984-31604-4-git-send-email-artagnon@gmail.com","threadId":"24407","inReplyTo":"1279210984-31604-1-git-send-email-artagnon@gmail.com","subject":"[PATCH 3/8] Add memory pool library","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-07-15T16:22:59Z","receivedAt":"2010-07-15T16:22:59Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"From: David Barr <david.barr@cordelta.com>\n\nAdd a memory pool library implemented using C macros. The obj_pool_gen()\nmacro creates a type-specific memory pool API.\n\nThe memory pool library is distinguished from the existing specialized\nallocators in alloc.c by using a contiguous block for all allocations.\nThis means that on one hand, long-lived pointers have to be written as\noffsets, since the base address changes as the pool grows, but on the\nother hand, the entire pool can be easily written to the file system.\nThis allows the memory pool to persist between runs of an application.\n\nFor the svn importer, such a facility is useful because each svn\nrevision can copy trees and files from any previous revision.  The\nrelevant information for all revisions has to persist somehow to\nsupport incremental runs, and for now it is simplest to avoid relying\non the target VCS for that.\n\nobj_pool_gen(pre, obj_t, initial_capability)\n\n\tpre: Prefix for generated functions (example: string).\n\tobj_t: Type for treap data structure (example: char).\n\tinitial_capacity: Initial size of the memory pool (example: 4096).\n\nvoid pre_init(void);\n\n\tRead values from a previous run to initialize the pool.\n\tIf this function is not called, the pool begins valid but empty.\n\nuint32_t pre_alloc(uint32_t nmemb);\n\n\tReserve space for a few objects in the pool and return an\n\toffset to the first one.\n\nuint32_t pre_free(uint32_t nmemb);\n\n\tUnreserve the last few objects reserved.\n\nuint32_t pre_offset(obj_t *pointer);\nobj_t *pre_pointer(uint32_t offset);\n\n\tConvert between pointers into the in-memory pool and offsets\n\tfrom the beginning (or ~0 for the NULL pointer).  Pointers are\n\tnot guaranteed to remain valid after a pre_alloc() operation\n\tor pre_reset() followed by pre_init(), but offsets are.\n\nvoid pre_commit(void);\n\n\tWrite the pool to file.  A pre_reset() followed by pre_init()\n\t(pehaps with exit() in between) will return the pool to the\n\tlast committed state.\n\nvoid pre_reset(void);\n\n\tDeinitialize the pool, freeing any associated memory and\n\tfile handles.\n\nSigned-off-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n Makefile           |    3 +-\n vcs-svn/LICENSE    |   26 +++++++++++++++++\n vcs-svn/obj_pool.h |   80 ++++++++++++++++++++++++++++++++++++++++++++++++++++\n 3 files changed, 108 insertions(+), 1 deletions(-)\n create mode 100644 vcs-svn/LICENSE\n create mode 100644 vcs-svn/obj_pool.h\n\ndiff --git a/Makefile b/Makefile\nindex 6441dcb..fc31ee0 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1863,7 +1863,8 @@ xdiff-interface.o $(XDIFF_OBJS): \\\n \txdiff/xinclude.h xdiff/xmacros.h xdiff/xdiff.h xdiff/xtypes.h \\\n \txdiff/xutils.h xdiff/xprepare.h xdiff/xdiffi.h xdiff/xemit.h\n \n-$(VCSSVN_OBJS):\n+$(VCSSVN_OBJS): \\\n+\tvcs-svn/obj_pool.h\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\ndiff --git a/vcs-svn/LICENSE b/vcs-svn/LICENSE\nnew file mode 100644\nindex 0000000..6e52372\n--- /dev/null\n+++ b/vcs-svn/LICENSE\n@@ -0,0 +1,26 @@\n+Copyright (C) 2010 David Barr <david.barr@cordelta.com>.\n+All rights reserved.\n+\n+Redistribution and use in source and binary forms, with or without\n+modification, are permitted provided that the following conditions\n+are met:\n+1. Redistributions of source code must retain the above copyright\n+   notice(s), this list of conditions and the following disclaimer\n+   unmodified other than the allowable addition of one or more\n+   copyright notices.\n+2. Redistributions in binary form must reproduce the above copyright\n+   notice(s), this list of conditions and the following disclaimer in\n+   the documentation and/or other materials provided with the\n+   distribution.\n+\n+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER(S) ``AS IS'' AND ANY\n+EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE\n+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR\n+PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT HOLDER(S) BE\n+LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR\n+CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF\n+SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR\n+BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,\n+WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE\n+OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,\n+EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\ndiff --git a/vcs-svn/obj_pool.h b/vcs-svn/obj_pool.h\nnew file mode 100644\nindex 0000000..f60c872\n--- /dev/null\n+++ b/vcs-svn/obj_pool.h\n@@ -0,0 +1,80 @@\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#ifndef OBJ_POOL_H_\n+#define OBJ_POOL_H_\n+\n+#include \"git-compat-util.h\"\n+\n+#define MAYBE_UNUSED __attribute__((__unused__))\n+\n+#define obj_pool_gen(pre, obj_t, initial_capacity) \\\n+static struct { \\\n+\tuint32_t committed; \\\n+\tuint32_t size; \\\n+\tuint32_t capacity; \\\n+\tobj_t *base; \\\n+\tFILE *file; \\\n+} pre##_pool = { 0, 0, 0, NULL, NULL}; \\\n+static MAYBE_UNUSED void pre##_init(void) \\\n+{ \\\n+\tstruct stat st; \\\n+\tpre##_pool.file = fopen(#pre \".bin\", \"a+\"); \\\n+\trewind(pre##_pool.file); \\\n+\tfstat(fileno(pre##_pool.file), &st); \\\n+\tpre##_pool.size = st.st_size / sizeof(obj_t); \\\n+\tpre##_pool.committed = pre##_pool.size; \\\n+\tpre##_pool.capacity = pre##_pool.size * 2; \\\n+\tif (pre##_pool.capacity < initial_capacity) \\\n+\t\tpre##_pool.capacity = initial_capacity; \\\n+\tpre##_pool.base = malloc(pre##_pool.capacity * sizeof(obj_t)); \\\n+\tfread(pre##_pool.base, sizeof(obj_t), pre##_pool.size, pre##_pool.file); \\\n+} \\\n+static MAYBE_UNUSED uint32_t pre##_alloc(uint32_t count) \\\n+{ \\\n+\tuint32_t offset; \\\n+\tif (pre##_pool.size + count > pre##_pool.capacity) { \\\n+\t\twhile (pre##_pool.size + count > pre##_pool.capacity) \\\n+\t\t\tif (pre##_pool.capacity) \\\n+\t\t\t\tpre##_pool.capacity *= 2; \\\n+\t\t\telse \\\n+\t\t\t\tpre##_pool.capacity = initial_capacity; \\\n+\t\tpre##_pool.base = realloc(pre##_pool.base, \\\n+\t\t\t\t\tpre##_pool.capacity * sizeof(obj_t)); \\\n+\t} \\\n+\toffset = pre##_pool.size; \\\n+\tpre##_pool.size += count; \\\n+\treturn offset; \\\n+} \\\n+static MAYBE_UNUSED void pre##_free(uint32_t count) \\\n+{ \\\n+\tpre##_pool.size -= count; \\\n+} \\\n+static MAYBE_UNUSED uint32_t pre##_offset(obj_t *obj) \\\n+{ \\\n+\treturn obj == NULL ? ~0 : obj - pre##_pool.base; \\\n+} \\\n+static MAYBE_UNUSED obj_t *pre##_pointer(uint32_t offset) \\\n+{ \\\n+\treturn offset >= pre##_pool.size ? NULL : &pre##_pool.base[offset]; \\\n+} \\\n+static MAYBE_UNUSED void pre##_commit(void) \\\n+{ \\\n+\tpre##_pool.committed += fwrite(pre##_pool.base + pre##_pool.committed, \\\n+\t\tsizeof(obj_t), pre##_pool.size - pre##_pool.committed, \\\n+\t\tpre##_pool.file); \\\n+} \\\n+static MAYBE_UNUSED void pre##_reset(void) \\\n+{ \\\n+\tfree(pre##_pool.base); \\\n+\tif (pre##_pool.file) \\\n+\t\tfclose(pre##_pool.file); \\\n+\tpre##_pool.base = NULL; \\\n+\tpre##_pool.size = 0; \\\n+\tpre##_pool.capacity = 0; \\\n+\tpre##_pool.file = NULL; \\\n+}\n+\n+#endif\n-- \n1.7.1\n"},{"id":"145635","messageId":"1279210984-31604-5-git-send-email-artagnon@gmail.com","threadId":"24407","inReplyTo":"1279210984-31604-1-git-send-email-artagnon@gmail.com","subject":"[PATCH 4/8] Add treap implementation","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-07-15T16:23:00Z","receivedAt":"2010-07-15T16:23:00Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"From: Jason Evans <jasone@canonware.com>\n\nProvide macros to generate a type-specific treap implementation and\nvarious functions to operate on it. It uses obj_pool.h to store memory\nnodes in a treap.  Previously committed nodes are never removed from\nthe pool; after any *_commit operation, it is assumed (correctly, in\nthe case of svn-fast-export) that someone else must care about them.\n\nTreaps provide a memory-efficient binary search tree structure.\nInsertion/deletion/search are about as about as fast in the average\ncase as red-black trees and the chances of worst-case behavior are\nvanishingly small, thanks to (pseudo-)randomness.  The bad worst-case\nbehavior is a small price to pay, given that treaps are much simpler\nto implement.\n\nFrom http://www.canonware.com/download/trp/trp_hash/trp.h\n\n[db: Altered to reference nodes by offset from a common base pointer]\n[db: Bob Jenkins' hashing implementation dropped for Knuth's]\n[db: Methods unnecessary for search and insert dropped]\n\nSigned-off-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n Makefile        |    2 +-\n vcs-svn/LICENSE |    3 +\n vcs-svn/trp.h   |  220 +++++++++++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/trp.txt |  102 +++++++++++++++++++++++++\n 4 files changed, 326 insertions(+), 1 deletions(-)\n create mode 100644 vcs-svn/trp.h\n create mode 100644 vcs-svn/trp.txt\n\ndiff --git a/Makefile b/Makefile\nindex fc31ee0..663a366 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1864,7 +1864,7 @@ xdiff-interface.o $(XDIFF_OBJS): \\\n \txdiff/xutils.h xdiff/xprepare.h xdiff/xdiffi.h xdiff/xemit.h\n \n $(VCSSVN_OBJS): \\\n-\tvcs-svn/obj_pool.h\n+\tvcs-svn/obj_pool.h vcs-svn/trp.h\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\ndiff --git a/vcs-svn/LICENSE b/vcs-svn/LICENSE\nindex 6e52372..a3d384c 100644\n--- a/vcs-svn/LICENSE\n+++ b/vcs-svn/LICENSE\n@@ -1,6 +1,9 @@\n Copyright (C) 2010 David Barr <david.barr@cordelta.com>.\n All rights reserved.\n \n+Copyright (C) 2008 Jason Evans <jasone@canonware.com>.\n+All rights reserved.\n+\n Redistribution and use in source and binary forms, with or without\n modification, are permitted provided that the following conditions\n are met:\ndiff --git a/vcs-svn/trp.h b/vcs-svn/trp.h\nnew file mode 100644\nindex 0000000..dd7d5ee\n--- /dev/null\n+++ b/vcs-svn/trp.h\n@@ -0,0 +1,220 @@\n+/*\n+ * C macro implementation of treaps.\n+ *\n+ * Usage:\n+ *   #include <stdint.h>\n+ *   #include \"trp.h\"\n+ *   trp_gen(...)\n+ *\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#ifndef TRP_H_\n+#define TRP_H_\n+\n+#define MAYBE_UNUSED __attribute__((__unused__))\n+\n+/* Node structure. */\n+struct trp_node {\n+\tuint32_t trpn_left;\n+\tuint32_t trpn_right;\n+};\n+\n+/* Root structure. */\n+struct trp_root {\n+\tuint32_t trp_root;\n+};\n+\n+/* Pointer/Offset conversion. */\n+#define trpn_pointer(a_base, a_offset) (a_base##_pointer(a_offset))\n+#define trpn_offset(a_base, a_pointer) (a_base##_offset(a_pointer))\n+#define trpn_modify(a_base, a_offset) \\\n+\tdo { \\\n+\t\tif ((a_offset) < a_base##_pool.committed) { \\\n+\t\t\tuint32_t old_offset = (a_offset);\\\n+\t\t\t(a_offset) = a_base##_alloc(1); \\\n+\t\t\t*trpn_pointer(a_base, a_offset) = \\\n+\t\t\t\t*trpn_pointer(a_base, old_offset); \\\n+\t\t} \\\n+\t} while (0);\n+\n+/* Left accessors. */\n+#define trp_left_get(a_base, a_field, a_node) \\\n+\t(trpn_pointer(a_base, a_node)->a_field.trpn_left)\n+#define trp_left_set(a_base, a_field, a_node, a_left) \\\n+\tdo { \\\n+\t\ttrpn_modify(a_base, a_node); \\\n+\t\ttrp_left_get(a_base, a_field, a_node) = (a_left); \\\n+\t} while(0)\n+\n+/* Right accessors. */\n+#define trp_right_get(a_base, a_field, a_node) \\\n+\t(trpn_pointer(a_base, a_node)->a_field.trpn_right)\n+#define trp_right_set(a_base, a_field, a_node, a_right) \\\n+\tdo { \\\n+\t\ttrpn_modify(a_base, a_node); \\\n+\t\ttrp_right_get(a_base, a_field, a_node) = (a_right); \\\n+\t} while(0)\n+\n+/*\n+ * Fibonacci hash function.\n+ * The multiplier is the nearest prime to (2^32 times (√5 - 1)/2).\n+ * See Knuth §6.4: volume 3, 3rd ed, p518.\n+ */\n+#define trpn_hash(a_node) (uint32_t) (2654435761u * (a_node))\n+\n+/* Priority accessors. */\n+#define trp_prio_get(a_node) trpn_hash(a_node)\n+\n+/* Node initializer. */\n+#define trp_node_new(a_base, a_field, a_node) \\\n+\tdo { \\\n+\t\ttrp_left_set(a_base, a_field, (a_node), ~0); \\\n+\t\ttrp_right_set(a_base, a_field, (a_node), ~0); \\\n+\t} while(0)\n+\n+/* Internal utility macros. */\n+#define trpn_first(a_base, a_field, a_root, r_node) \\\n+\tdo { \\\n+\t\t(r_node) = (a_root); \\\n+\t\tif ((r_node) == ~0) \\\n+\t\t\treturn NULL; \\\n+\t\twhile (~trp_left_get(a_base, a_field, (r_node))) \\\n+\t\t\t(r_node) = trp_left_get(a_base, a_field, (r_node)); \\\n+\t} while (0)\n+\n+#define trpn_rotate_left(a_base, a_field, a_node, r_node) \\\n+\tdo { \\\n+\t\t(r_node) = trp_right_get(a_base, a_field, (a_node)); \\\n+\t\ttrp_right_set(a_base, a_field, (a_node), \\\n+\t\t\ttrp_left_get(a_base, a_field, (r_node))); \\\n+\t\ttrp_left_set(a_base, a_field, (r_node), (a_node)); \\\n+\t} while(0)\n+\n+#define trpn_rotate_right(a_base, a_field, a_node, r_node) \\\n+\tdo { \\\n+\t\t(r_node) = trp_left_get(a_base, a_field, (a_node)); \\\n+\t\ttrp_left_set(a_base, a_field, (a_node), \\\n+\t\t\ttrp_right_get(a_base, a_field, (r_node))); \\\n+\t\ttrp_right_set(a_base, a_field, (r_node), (a_node)); \\\n+\t} while(0)\n+\n+#define trp_gen(a_attr, a_pre, a_type, a_field, a_base, a_cmp) \\\n+a_attr a_type MAYBE_UNUSED *a_pre##first(struct trp_root *treap) \\\n+{ \\\n+\tuint32_t ret; \\\n+\ttrpn_first(a_base, a_field, treap->trp_root, ret); \\\n+\treturn trpn_pointer(a_base, ret); \\\n+} \\\n+a_attr a_type MAYBE_UNUSED *a_pre##next(struct trp_root *treap, a_type *node) \\\n+{ \\\n+\tuint32_t ret; \\\n+\tuint32_t offset = trpn_offset(a_base, node); \\\n+\tif (~trp_right_get(a_base, a_field, offset)) { \\\n+\t\ttrpn_first(a_base, a_field, \\\n+\t\t\ttrp_right_get(a_base, a_field, offset), ret); \\\n+\t} else { \\\n+\t\tuint32_t tnode = treap->trp_root; \\\n+\t\tret = ~0; \\\n+\t\twhile (1) { \\\n+\t\t\tint cmp = (a_cmp)(trpn_pointer(a_base, offset), \\\n+\t\t\t\ttrpn_pointer(a_base, tnode)); \\\n+\t\t\tif (cmp < 0) { \\\n+\t\t\t\tret = tnode; \\\n+\t\t\t\ttnode = trp_left_get(a_base, a_field, tnode); \\\n+\t\t\t} else if (cmp > 0) { \\\n+\t\t\t\ttnode = trp_right_get(a_base, a_field, tnode); \\\n+\t\t\t} else { \\\n+\t\t\t\tbreak; \\\n+\t\t\t} \\\n+\t\t} \\\n+\t} \\\n+\treturn trpn_pointer(a_base, ret); \\\n+} \\\n+a_attr a_type MAYBE_UNUSED *a_pre##search(struct trp_root *treap, a_type *key) \\\n+{ \\\n+\tint cmp; \\\n+\tuint32_t ret = treap->trp_root; \\\n+\twhile (~ret && (cmp = (a_cmp)(key, trpn_pointer(a_base,ret)))) { \\\n+\t\tif (cmp < 0) \\\n+\t\t\tret = trp_left_get(a_base, a_field, ret); \\\n+\t\telse \\\n+\t\t\tret = trp_right_get(a_base, a_field, ret); \\\n+\t} \\\n+\treturn trpn_pointer(a_base, ret); \\\n+} \\\n+a_attr uint32_t MAYBE_UNUSED a_pre##insert_recurse(uint32_t cur_node, uint32_t ins_node) \\\n+{ \\\n+\tif (cur_node == ~0) { \\\n+\t\treturn (ins_node); \\\n+\t} else { \\\n+\t\tuint32_t ret; \\\n+\t\tint cmp = (a_cmp)(trpn_pointer(a_base, ins_node), \\\n+\t\t\t\t\ttrpn_pointer(a_base, cur_node)); \\\n+\t\tif (cmp < 0) { \\\n+\t\t\tuint32_t left = a_pre##insert_recurse( \\\n+\t\t\t\ttrp_left_get(a_base, a_field, cur_node), ins_node); \\\n+\t\t\ttrp_left_set(a_base, a_field, cur_node, left); \\\n+\t\t\tif (trp_prio_get(left) < trp_prio_get(cur_node)) \\\n+\t\t\t\ttrpn_rotate_right(a_base, a_field, cur_node, ret); \\\n+\t\t\telse \\\n+\t\t\t\tret = cur_node; \\\n+\t\t} else { \\\n+\t\t\tuint32_t right = a_pre##insert_recurse( \\\n+\t\t\t\ttrp_right_get(a_base, a_field, cur_node), ins_node); \\\n+\t\t\ttrp_right_set(a_base, a_field, cur_node, right); \\\n+\t\t\tif (trp_prio_get(right) < trp_prio_get(cur_node)) \\\n+\t\t\t\ttrpn_rotate_left(a_base, a_field, cur_node, ret); \\\n+\t\t\telse \\\n+\t\t\t\tret = cur_node; \\\n+\t\t} \\\n+\t\treturn (ret); \\\n+\t} \\\n+} \\\n+a_attr void MAYBE_UNUSED a_pre##insert(struct trp_root *treap, a_type *node) \\\n+{ \\\n+\tuint32_t offset = trpn_offset(a_base, node); \\\n+\ttrp_node_new(a_base, a_field, offset); \\\n+\ttreap->trp_root = a_pre##insert_recurse(treap->trp_root, offset); \\\n+} \\\n+a_attr uint32_t MAYBE_UNUSED a_pre##remove_recurse(uint32_t cur_node, uint32_t rem_node) \\\n+{ \\\n+\tint cmp = a_cmp(trpn_pointer(a_base, rem_node), \\\n+\t\t\ttrpn_pointer(a_base, cur_node)); \\\n+\tif (cmp == 0) { \\\n+\t\tuint32_t ret; \\\n+\t\tuint32_t left = trp_left_get(a_base, a_field, cur_node); \\\n+\t\tuint32_t right = trp_right_get(a_base, a_field, cur_node); \\\n+\t\tif (left == ~0) { \\\n+\t\t\tif (right == ~0) \\\n+\t\t\t\treturn (~0); \\\n+\t\t} else if (right == ~0 || trp_prio_get(left) < trp_prio_get(right)) { \\\n+\t\t\ttrpn_rotate_right(a_base, a_field, cur_node, ret); \\\n+\t\t\tright = a_pre##remove_recurse(cur_node, rem_node); \\\n+\t\t\ttrp_right_set(a_base, a_field, ret, right); \\\n+\t\t\treturn (ret); \\\n+\t\t} \\\n+\t\ttrpn_rotate_left(a_base, a_field, cur_node, ret); \\\n+\t\tleft = a_pre##remove_recurse(cur_node, rem_node); \\\n+\t\ttrp_left_set(a_base, a_field, ret, left); \\\n+\t\treturn (ret); \\\n+\t} else if (cmp < 0) { \\\n+\t\tuint32_t left = a_pre##remove_recurse( \\\n+\t\t\ttrp_left_get(a_base, a_field, cur_node), rem_node); \\\n+\t\ttrp_left_set(a_base, a_field, cur_node, left); \\\n+\t\treturn (cur_node); \\\n+\t} else { \\\n+\t\tuint32_t right = a_pre##remove_recurse( \\\n+\t\t\ttrp_right_get(a_base, a_field, cur_node), rem_node); \\\n+\t\ttrp_right_set(a_base, a_field, cur_node, right); \\\n+\t\treturn (cur_node); \\\n+\t} \\\n+} \\\n+a_attr void MAYBE_UNUSED a_pre##remove(struct trp_root *treap, a_type *node) \\\n+{ \\\n+\ttreap->trp_root = a_pre##remove_recurse(treap->trp_root, \\\n+\t\ttrpn_offset(a_base, node)); \\\n+} \\\n+\n+#endif\ndiff --git a/vcs-svn/trp.txt b/vcs-svn/trp.txt\nnew file mode 100644\nindex 0000000..943c385\n--- /dev/null\n+++ b/vcs-svn/trp.txt\n@@ -0,0 +1,102 @@\n+Motivation\n+==========\n+\n+Treaps provide a memory-efficient binary search tree structure.\n+Insertion/deletion/search are about as about as fast in the average\n+case as red-black trees and the chances of worst-case behavior are\n+vanishingly small, thanks to (pseudo-)randomness.  The bad worst-case\n+behavior is a small price to pay, given that treaps are much simpler\n+to implement.\n+    \n+From http://www.canonware.com/download/trp/trp_hash/trp.h\n+\n+API\n+===\n+\n+The trp API generates a data structure and functions to handle a\n+large growing set of objects stored in a pool.\n+\n+The caller:\n+\n+. Specifies parameters for the generated functions with the\n+  trp_gen(static, foo_, ...) macro.\n+\n+. Allocates and clears a `struct trp_node` variable.\n+\n+. Adds new items to the set using `foo_insert`.\n+\n+. Can find a specific item in the set using `foo_search`.\n+\n+. Can iterate over items in the set using `foo_first` and `foo_next`.\n+\n+. Can remove an item from the set using `foo_remove`.\n+\n+. The set is never freed.\n+\n+Example:\n+\n+----\n+struct ex_node {\n+\tconst char *s;\n+\tstruct trp_node ex_link;\n+};\n+static struct trp_root ex_base;\n+obj_pool_gen(ex, struct ex_node, 4096);\n+trp_gen(static, ex_, struct ex_node, ex_link, ex, strcmp)\n+struct ex_node *item;\n+\n+item = ex_pointer(ex_alloc(1));\n+item->s = \"hello\";\n+ex_insert(&ex_base, item);\n+item = ex_pointer(ex_alloc(1));\n+item->s = \"goodbye\";\n+ex_insert(&ex_base, item);\n+for (item = ex_first(&ex_base); item; item = ex_next(&ex_base, item))\n+\tprintf(\"%s\\n\", item->s);\n+----\n+\n+Functions\n+---------\n+\n+trp_gen(attr, foo_, node_type, link_field, pool, cmp)::\n+\n+\tGenerate a type-specific treap implementation.\n++\n+. The storage class for generated functions will be 'attr' (e.g., `static`).\n+. Generated function names are prefixed with 'foo_' (e.g., `treap_`).\n+. Treap nodes will be of type 'node_type' (e.g., `struct treap_node`).\n+  This type must be a struct with at least one `struct trp_node` field\n+  to point to its children.\n+. The field used to access child nodes will be 'link_field'.\n+. All treap nodes must lie in the 'pool' object pool.\n+. Treap nodes must be totally ordered by the 'cmp' relation, with the\n+  following prototype:\n++\n+int (*cmp)(node_type \\*a, node_type \\*b)\n++\n+and returning a value less than, equal to, or greater than zero\n+according to the result of comparison.\n+\n+void foo_insert(struct trp_root *treap, node_type \\*node)::\n+\n+\tInsert node into treap.  If inserted multiple times,\n+\ta node will appear in the treap multiple times.\n+\n+void foo_remove(struct trp_root *treap, node_type \\*node)::\n+\n+\tRemove node from treap.  Caller must ensure node is\n+\tpresent in treap before using this function.\n+\n+node_type *foo_search(struct trp_root \\*treap, node_type \\*key)::\n+\n+\tSearch for a node that matches key.  If no match is found,\n+\treturn what would be key's successor, were key in treap\n+\t(NULL if no successor).\n+\n+node_type *foo_first(struct trp_root \\*treap)::\n+\n+\tFind the first item from the treap, in sorted order.\n+\n+node_type *foo_next(struct trp_root \\*treap, node_type \\*node)::\n+\n+\tFind the next item.\n-- \n1.7.1\n"},{"id":"145630","messageId":"1279210984-31604-6-git-send-email-artagnon@gmail.com","threadId":"24407","inReplyTo":"1279210984-31604-1-git-send-email-artagnon@gmail.com","subject":"[PATCH 5/8] Add string-specific memory pool","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-07-15T16:23:01Z","receivedAt":"2010-07-15T16:23:01Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"From: David Barr <david.barr@cordelta.com>\n\nIntern strings so they can be compared by address and stored without\nwasting space.\n\nThis library uses the macros in the obj_pool.h and trp.h to create a\nmemory pool for strings and expose an API for handling them.\n\nSigned-off-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n Makefile              |    4 +-\n vcs-svn/string_pool.c |  114 +++++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/string_pool.h |   15 ++++++\n 3 files changed, 131 insertions(+), 2 deletions(-)\n create mode 100644 vcs-svn/string_pool.c\n create mode 100644 vcs-svn/string_pool.h\n\ndiff --git a/Makefile b/Makefile\nindex 663a366..e11e588 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1740,7 +1740,7 @@ ifndef NO_CURL\n endif\n XDIFF_OBJS = xdiff/xdiffi.o xdiff/xprepare.o xdiff/xutils.o xdiff/xemit.o \\\n \txdiff/xmerge.o xdiff/xpatience.o\n-VCSSVN_OBJS =\n+VCSSVN_OBJS = vcs-svn/string_pool.o\n OBJECTS := $(GIT_OBJS) $(XDIFF_OBJS) $(VCSSVN_OBJS)\n \n dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d)\n@@ -1864,7 +1864,7 @@ xdiff-interface.o $(XDIFF_OBJS): \\\n \txdiff/xutils.h xdiff/xprepare.h xdiff/xdiffi.h xdiff/xemit.h\n \n $(VCSSVN_OBJS): \\\n-\tvcs-svn/obj_pool.h vcs-svn/trp.h\n+\tvcs-svn/obj_pool.h vcs-svn/trp.h vcs-svn/string_pool.h\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\ndiff --git a/vcs-svn/string_pool.c b/vcs-svn/string_pool.c\nnew file mode 100644\nindex 0000000..bd5a380\n--- /dev/null\n+++ b/vcs-svn/string_pool.c\n@@ -0,0 +1,114 @@\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"git-compat-util.h\"\n+#include \"trp.h\"\n+#include \"obj_pool.h\"\n+#include \"string_pool.h\"\n+\n+static struct trp_root tree = { ~0 };\n+\n+struct node {\n+\tuint32_t offset;\n+\tstruct trp_node children;\n+};\n+\n+/* Two memory pools: one for struct node, and another for strings */\n+obj_pool_gen(node, struct node, 4096);\n+obj_pool_gen(string, char, 4096);\n+\n+static char *node_value(struct node *node)\n+{\n+\treturn node ? string_pointer(node->offset) : NULL;\n+}\n+\n+static int node_cmp(struct node *a, struct node *b)\n+{\n+\treturn strcmp(node_value(a), node_value(b));\n+}\n+\n+/* Build a Treap from the node structure (a trp_node w/ offset) */\n+trp_gen(static, tree_, struct node, children, node, node_cmp);\n+\n+char *pool_fetch(uint32_t entry)\n+{\n+\treturn node_value(node_pointer(entry));\n+}\n+\n+uint32_t pool_intern(char *key)\n+{\n+\t/* Canonicalize key */\n+\tstruct node *match = NULL;\n+\tuint32_t key_len;\n+\tif (key == NULL)\n+\t\treturn ~0;\n+\tkey_len = strlen(key) + 1;\n+\tstruct node *node = node_pointer(node_alloc(1));\n+\tnode->offset = string_alloc(key_len);\n+\tstrcpy(node_value(node), key);\n+\tmatch = tree_search(&tree, node);\n+\tif (!match) {\n+\t\ttree_insert(&tree, node);\n+\t} else {\n+\t\tnode_free(1);\n+\t\tstring_free(key_len);\n+\t\tnode = match;\n+\t}\n+\treturn node_offset(node);\n+}\n+\n+uint32_t pool_tok_r(char *str, const char *delim, char **saveptr)\n+{\n+\tchar *token = strtok_r(str, delim, saveptr);\n+\treturn token ? pool_intern(token) : ~0;\n+}\n+\n+void pool_print_seq(uint32_t len, uint32_t *seq, char delim, FILE *stream)\n+{\n+\tuint32_t i;\n+\tfor (i = 0; i < len && ~seq[i]; i++) {\n+\t\tfputs(pool_fetch(seq[i]), stream);\n+\t\tif (i < len - 1 && ~seq[i + 1])\n+\t\t\tfputc(delim, stream);\n+\t}\n+}\n+\n+uint32_t pool_tok_seq(uint32_t max, uint32_t *seq, char *delim, char *str)\n+{\n+\tchar *context = NULL;\n+\tuint32_t length = 0, token = str ? pool_tok_r(str, delim, &context) : ~0;\n+\twhile (length < max) {\n+\t\tseq[length++] = token;\n+\t\tif (token == ~0)\n+\t\t\tbreak;\n+\t\ttoken = pool_tok_r(NULL, delim, &context);\n+\t}\n+\tseq[length ? length - 1 : 0] = ~0;\n+\treturn length;\n+}\n+\n+void pool_init(void)\n+{\n+\tuint32_t node;\n+\tuint32_t string = 0;\n+\tstring_init();\n+\twhile (string < string_pool.size) {\n+\t\tnode = node_alloc(1);\n+\t\tnode_pointer(node)->offset = string;\n+\t\ttree_insert(&tree, node_pointer(node));\n+\t\tstring += strlen(string_pointer(string)) + 1;\n+\t}\n+}\n+\n+void pool_commit(void)\n+{\n+\tstring_commit();\n+}\n+\n+void pool_reset(void)\n+{\n+\tnode_reset();\n+\tstring_reset();\n+}\ndiff --git a/vcs-svn/string_pool.h b/vcs-svn/string_pool.h\nnew file mode 100644\nindex 0000000..085e6d7\n--- /dev/null\n+++ b/vcs-svn/string_pool.h\n@@ -0,0 +1,15 @@\n+#ifndef STRING_POOL_H_\n+#define STRING_POOL_H_\n+\n+#include \"git-compat-util.h\"\n+\n+uint32_t pool_intern(char *key);\n+char *pool_fetch(uint32_t entry);\n+uint32_t pool_tok_r(char *str, const char *delim, char **saveptr);\n+void pool_print_seq(uint32_t len, uint32_t *seq, char delim, FILE *stream);\n+uint32_t pool_tok_seq(uint32_t max, uint32_t *seq, char *delim, char *str);\n+void pool_init(void);\n+void pool_commit(void);\n+void pool_reset(void);\n+\n+#endif\n-- \n1.7.1\n"},{"id":"145632","messageId":"1279210984-31604-7-git-send-email-artagnon@gmail.com","threadId":"24407","inReplyTo":"1279210984-31604-1-git-send-email-artagnon@gmail.com","subject":"[PATCH 6/8] Add stream helper library","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-07-15T16:23:02Z","receivedAt":"2010-07-15T16:23:02Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"From: David Barr <david.barr@cordelta.com>\n\nThis library provides thread-unsafe fgets()- and fread()-like\nfunctions where the caller does not have to supply a buffer.  It\nmaintains a couple of static buffers and provides an API to use\nthem.\n\nNEEDSWORK: what should buffer_copy_bytes do on error?\n\nSigned-off-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n Makefile              |    5 ++-\n vcs-svn/line_buffer.c |   93 +++++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/line_buffer.h |   14 +++++++\n 3 files changed, 110 insertions(+), 2 deletions(-)\n create mode 100644 vcs-svn/line_buffer.c\n create mode 100644 vcs-svn/line_buffer.h\n\ndiff --git a/Makefile b/Makefile\nindex e11e588..8223d9b 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1740,7 +1740,7 @@ ifndef NO_CURL\n endif\n XDIFF_OBJS = xdiff/xdiffi.o xdiff/xprepare.o xdiff/xutils.o xdiff/xemit.o \\\n \txdiff/xmerge.o xdiff/xpatience.o\n-VCSSVN_OBJS = vcs-svn/string_pool.o\n+VCSSVN_OBJS = vcs-svn/string_pool.o vcs-svn/line_buffer.o\n OBJECTS := $(GIT_OBJS) $(XDIFF_OBJS) $(VCSSVN_OBJS)\n \n dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d)\n@@ -1864,7 +1864,8 @@ xdiff-interface.o $(XDIFF_OBJS): \\\n \txdiff/xutils.h xdiff/xprepare.h xdiff/xdiffi.h xdiff/xemit.h\n \n $(VCSSVN_OBJS): \\\n-\tvcs-svn/obj_pool.h vcs-svn/trp.h vcs-svn/string_pool.h\n+\tvcs-svn/obj_pool.h vcs-svn/trp.h vcs-svn/string_pool.h \\\n+\tvcs-svn/line_buffer.h\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\ndiff --git a/vcs-svn/line_buffer.c b/vcs-svn/line_buffer.c\nnew file mode 100644\nindex 0000000..0f83426\n--- /dev/null\n+++ b/vcs-svn/line_buffer.c\n@@ -0,0 +1,93 @@\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"git-compat-util.h\"\n+\n+#include \"line_buffer.h\"\n+#include \"obj_pool.h\"\n+\n+#define LINE_BUFFER_LEN 10000\n+#define COPY_BUFFER_LEN 4096\n+\n+/* Create memory pool for char sequence of known length */\n+obj_pool_gen(blob, char, 4096);\n+\n+static char line_buffer[LINE_BUFFER_LEN];\n+static char byte_buffer[COPY_BUFFER_LEN];\n+static FILE *infile;\n+\n+int buffer_init(const char *filename)\n+{\n+\tinfile = filename ? fopen(filename, \"r\") : stdin;\n+\tif (!infile)\n+\t\treturn -1;\n+\treturn 0;\n+}\n+\n+int buffer_deinit()\n+{\n+\tfclose(infile);\n+\treturn 0;\n+}\n+\n+/* Read a line without trailing newline. */\n+char *buffer_read_line(void)\n+{\n+\tchar *end;\n+\tif (!fgets(line_buffer, sizeof(line_buffer), infile))\n+\t\t/* Error or data exhausted. */\n+\t\treturn NULL;\n+\tend = line_buffer + strlen(line_buffer);\n+\tif (end[-1] == '\\n')\n+\t\tend[-1] = '\\0';\n+\telse if (feof(infile))\n+\t\t; /* No newline at end of file.  That's fine. */\n+\telse\n+\t\t/*\n+\t\t * Line was too long.\n+\t\t * There is probably a saner way to deal with this,\n+\t\t * but for now let's return an error.\n+\t\t */\n+\t\treturn NULL;\n+\treturn line_buffer;\n+}\n+\n+char *buffer_read_string(uint32_t len)\n+{\n+\tchar *s;\n+\tblob_free(blob_pool.size);\n+\ts = blob_pointer(blob_alloc(len + 1));\n+\ts[fread(s, 1, len, infile)] = '\\0';\n+\treturn ferror(infile) ? NULL : s;\n+}\n+\n+void buffer_copy_bytes(uint32_t len)\n+{\n+\tuint32_t in;\n+\twhile (len > 0 && !feof(infile)) {\n+\t\tin = len < COPY_BUFFER_LEN ? len : COPY_BUFFER_LEN;\n+\t\tin = fread(byte_buffer, 1, in, infile);\n+\t\tlen -= in;\n+\t\tfwrite(byte_buffer, 1, in, stdout);\n+\t\tif (ferror(infile) || ferror(stdout))\n+\t\t\t/* NEEDSWORK: handle error. */\n+\t\t\tbreak;\n+\t}\n+}\n+\n+void buffer_skip_bytes(uint32_t len)\n+{\n+\tuint32_t in;\n+\twhile (len > 0 && !feof(infile) && !ferror(infile)) {\n+\t\tin = len < COPY_BUFFER_LEN ? len : COPY_BUFFER_LEN;\n+\t\tin = fread(byte_buffer, 1, in, infile);\n+\t\tlen -= in;\n+\t}\n+}\n+\n+void buffer_reset(void)\n+{\n+\tblob_reset();\n+}\ndiff --git a/vcs-svn/line_buffer.h b/vcs-svn/line_buffer.h\nnew file mode 100644\nindex 0000000..631d1df\n--- /dev/null\n+++ b/vcs-svn/line_buffer.h\n@@ -0,0 +1,14 @@\n+#ifndef LINE_BUFFER_H_\n+#define LINE_BUFFER_H_\n+\n+#include \"git-compat-util.h\"\n+\n+int buffer_init(const char *filename);\n+int buffer_deinit(void);\n+char *buffer_read_line(void);\n+char *buffer_read_string(uint32_t len);\n+void buffer_copy_bytes(uint32_t len);\n+void buffer_skip_bytes(uint32_t len);\n+void buffer_reset(void);\n+\n+#endif\n-- \n1.7.1\n"},{"id":"145636","messageId":"1279210984-31604-8-git-send-email-artagnon@gmail.com","threadId":"24407","inReplyTo":"1279210984-31604-1-git-send-email-artagnon@gmail.com","subject":"[PATCH 7/8] Add infrastructure to write revisions in fast-export format","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-07-15T16:23:03Z","receivedAt":"2010-07-15T16:23:03Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"From: David Barr <david.barr@cordelta.com>\n\nrepo_tree maintains the exporter's state and provides a facility to to\ncall fast_export, which writes objects to stdout suitable for\nconsumption by fast-import.\n\nThe exported functions roughly correspond to Subversion FS operations.\n\n . repo_add, repo_modify, repo_copy, repo_replace, and repo_delete\n   update the current commit, based roughly on the corresponding\n   Subversion FS operation.\n\n . repo_commit calls out to fast_export to write the current commit to\n   the fast-import stream in stdout.\n\n . repo_diff is used by the fast_export module to write the changes\n   for a commit.\n\n . repo_reset erases the exporter's state, so valgrind can be happy.\n\nSigned-off-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n Makefile              |    5 +-\n vcs-svn/fast_export.c |   75 +++++++++++\n vcs-svn/fast_export.h |   14 ++\n vcs-svn/repo_tree.c   |  335 +++++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/repo_tree.h   |   26 ++++\n 5 files changed, 453 insertions(+), 2 deletions(-)\n create mode 100644 vcs-svn/fast_export.c\n create mode 100644 vcs-svn/fast_export.h\n create mode 100644 vcs-svn/repo_tree.c\n create mode 100644 vcs-svn/repo_tree.h\n\ndiff --git a/Makefile b/Makefile\nindex 8223d9b..7c66dcc 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1740,7 +1740,8 @@ ifndef NO_CURL\n endif\n XDIFF_OBJS = xdiff/xdiffi.o xdiff/xprepare.o xdiff/xutils.o xdiff/xemit.o \\\n \txdiff/xmerge.o xdiff/xpatience.o\n-VCSSVN_OBJS = vcs-svn/string_pool.o vcs-svn/line_buffer.o\n+VCSSVN_OBJS = vcs-svn/string_pool.o vcs-svn/line_buffer.o \\\n+\tvcs-svn/repo_tree.o vcs-svn/fast_export.o\n OBJECTS := $(GIT_OBJS) $(XDIFF_OBJS) $(VCSSVN_OBJS)\n \n dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d)\n@@ -1865,7 +1866,7 @@ xdiff-interface.o $(XDIFF_OBJS): \\\n \n $(VCSSVN_OBJS): \\\n \tvcs-svn/obj_pool.h vcs-svn/trp.h vcs-svn/string_pool.h \\\n-\tvcs-svn/line_buffer.h\n+\tvcs-svn/line_buffer.h vcs-svn/repo_tree.h vcs-svn/fast_export.h\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nnew file mode 100644\nindex 0000000..7552803\n--- /dev/null\n+++ b/vcs-svn/fast_export.c\n@@ -0,0 +1,75 @@\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"git-compat-util.h\"\n+\n+#include \"fast_export.h\"\n+#include \"line_buffer.h\"\n+#include \"repo_tree.h\"\n+#include \"string_pool.h\"\n+\n+#define MAX_GITSVN_LINE_LEN 4096\n+\n+static uint32_t first_commit_done;\n+\n+void fast_export_delete(uint32_t depth, uint32_t *path)\n+{\n+\tputchar('D');\n+\tputchar(' ');\n+\tpool_print_seq(depth, path, '/', stdout);\n+\tputchar('\\n');\n+}\n+\n+void fast_export_modify(uint32_t depth, uint32_t *path, uint32_t mode,\n+                        uint32_t mark)\n+{\n+\t/* Mode must be 100644, 100755, 120000, or 160000. */\n+\tprintf(\"M %06o :%d \", mode, mark);\n+\tpool_print_seq(depth, path, '/', stdout);\n+\tputchar('\\n');\n+}\n+\n+static char gitsvnline[MAX_GITSVN_LINE_LEN];\n+void fast_export_commit(uint32_t revision, uint32_t author, char *log,\n+\t\t\tuint32_t uuid, uint32_t url,\n+\t\t\tunsigned long timestamp)\n+{\n+\tif (!log)\n+\t\tlog = \"\";\n+\tif (~uuid && ~url) {\n+\t\tsnprintf(gitsvnline, MAX_GITSVN_LINE_LEN, \"\\n\\ngit-svn-id: %s@%d %s\\n\",\n+\t\t\t\t pool_fetch(url), revision, pool_fetch(uuid));\n+\t} else {\n+\t\t*gitsvnline = '\\0';\n+\t}\n+\tprintf(\"commit refs/heads/master\\n\");\n+\tprintf(\"committer %s <%s@%s> %ld +0000\\n\",\n+\t\t   ~author ? pool_fetch(author) : \"nobody\",\n+\t\t   ~author ? pool_fetch(author) : \"nobody\",\n+\t\t   ~uuid ? pool_fetch(uuid) : \"local\", timestamp);\n+\tprintf(\"data %zd\\n%s%s\\n\",\n+\t\t   strlen(log) + strlen(gitsvnline), log, gitsvnline);\n+\tif (!first_commit_done) {\n+\t\tif (revision > 1)\n+\t\t\tprintf(\"from refs/heads/master^0\\n\");\n+\t\tfirst_commit_done = 1;\n+\t}\n+\trepo_diff(revision - 1, revision);\n+\tfputc('\\n', stdout);\n+\n+\tprintf(\"progress Imported commit %d.\\n\\n\", revision);\n+}\n+\n+void fast_export_blob(uint32_t mode, uint32_t mark, uint32_t len)\n+{\n+\tif (mode == REPO_MODE_LNK) {\n+\t\t/* svn symlink blobs start with \"link \" */\n+\t\tbuffer_skip_bytes(5);\n+\t\tlen -= 5;\n+\t}\n+\tprintf(\"blob\\nmark :%d\\ndata %d\\n\", mark, len);\n+\tbuffer_copy_bytes(len);\n+\tfputc('\\n', stdout);\n+}\ndiff --git a/vcs-svn/fast_export.h b/vcs-svn/fast_export.h\nnew file mode 100644\nindex 0000000..47e8f56\n--- /dev/null\n+++ b/vcs-svn/fast_export.h\n@@ -0,0 +1,14 @@\n+#ifndef FAST_EXPORT_H_\n+#define FAST_EXPORT_H_\n+\n+#include <stdint.h>\n+#include <time.h>\n+\n+void fast_export_delete(uint32_t depth, uint32_t *path);\n+void fast_export_modify(uint32_t depth, uint32_t *path, uint32_t mode,\n+\t\t\tuint32_t mark);\n+void fast_export_commit(uint32_t revision, uint32_t author, char *log,\n+\t\t\tuint32_t uuid, uint32_t url, unsigned long timestamp);\n+void fast_export_blob(uint32_t mode, uint32_t mark, uint32_t len);\n+\n+#endif\ndiff --git a/vcs-svn/repo_tree.c b/vcs-svn/repo_tree.c\nnew file mode 100644\nindex 0000000..59a7434\n--- /dev/null\n+++ b/vcs-svn/repo_tree.c\n@@ -0,0 +1,335 @@\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"git-compat-util.h\"\n+\n+#include \"string_pool.h\"\n+#include \"repo_tree.h\"\n+#include \"obj_pool.h\"\n+#include \"fast_export.h\"\n+\n+#include \"trp.h\"\n+\n+struct repo_dirent {\n+\tuint32_t name_offset;\n+\tstruct trp_node children;\n+\tuint32_t mode;\n+\tuint32_t content_offset;\n+};\n+\n+struct repo_dir {\n+\tstruct trp_root entries;\n+};\n+\n+struct repo_commit {\n+\tuint32_t root_dir_offset;\n+};\n+\n+/* Memory pools for commit, dir and dirent */\n+obj_pool_gen(commit, struct repo_commit, 4096);\n+obj_pool_gen(dir, struct repo_dir, 4096);\n+obj_pool_gen(dirent, struct repo_dirent, 4096);\n+\n+static uint32_t active_commit;\n+static uint32_t mark;\n+\n+static int repo_dirent_name_cmp(const void *a, const void *b);\n+\n+/* Treap for directory entries */\n+trp_gen(static, dirent_, struct repo_dirent, children, dirent, repo_dirent_name_cmp);\n+\n+uint32_t next_blob_mark(void)\n+{\n+\treturn mark++;\n+}\n+\n+static struct repo_dir *repo_commit_root_dir(struct repo_commit *commit)\n+{\n+\treturn dir_pointer(commit->root_dir_offset);\n+}\n+\n+static struct repo_dirent *repo_first_dirent(struct repo_dir *dir)\n+{\n+\treturn dirent_first(&dir->entries);\n+}\n+\n+static int repo_dirent_name_cmp(const void *a, const void *b)\n+{\n+\tconst struct repo_dirent *dirent1 = a, *dirent2 = b;\n+\tuint32_t a_offset = dirent1->name_offset;\n+\tuint32_t b_offset = dirent2->name_offset;\n+\treturn (a_offset > b_offset) - (a_offset < b_offset);\n+}\n+\n+static int repo_dirent_is_dir(struct repo_dirent *dirent)\n+{\n+\treturn dirent != NULL && dirent->mode == REPO_MODE_DIR;\n+}\n+\n+static struct repo_dir *repo_dir_from_dirent(struct repo_dirent *dirent)\n+{\n+\tif (!repo_dirent_is_dir(dirent))\n+\t\treturn NULL;\n+\treturn dir_pointer(dirent->content_offset);\n+}\n+\n+static struct repo_dir *repo_clone_dir(struct repo_dir *orig_dir)\n+{\n+\tuint32_t orig_o, new_o;\n+\torig_o = dir_offset(orig_dir);\n+\tif (orig_o >= dir_pool.committed)\n+\t\treturn orig_dir;\n+\tnew_o = dir_alloc(1);\n+\torig_dir = dir_pointer(orig_o);\n+\t*dir_pointer(new_o) = *orig_dir;\n+\treturn dir_pointer(new_o);\n+}\n+\n+static struct repo_dirent *repo_read_dirent(uint32_t revision, uint32_t *path)\n+{\n+\tuint32_t name = 0;\n+\tstruct repo_dirent *key = dirent_pointer(dirent_alloc(1));\n+\tstruct repo_dir *dir = NULL;\n+\tstruct repo_dirent *dirent = NULL;\n+\tdir = repo_commit_root_dir(commit_pointer(revision));\n+\twhile (~(name = *path++)) {\n+\t\tkey->name_offset = name;\n+\t\tdirent = dirent_search(&dir->entries, key);\n+\t\tif (dirent == NULL || !repo_dirent_is_dir(dirent))\n+\t\t\tbreak;\n+\t\tdir = repo_dir_from_dirent(dirent);\n+\t}\n+\tdirent_free(1);\n+\treturn dirent;\n+}\n+\n+static void repo_write_dirent(uint32_t *path, uint32_t mode,\n+                              uint32_t content_offset, uint32_t del)\n+{\n+\tuint32_t name, revision, dir_o = ~0, parent_dir_o = ~0;\n+\tstruct repo_dir *dir;\n+\tstruct repo_dirent *key;\n+\tstruct repo_dirent *dirent = NULL;\n+\trevision = active_commit;\n+\tdir = repo_commit_root_dir(commit_pointer(revision));\n+\tdir = repo_clone_dir(dir);\n+\tcommit_pointer(revision)->root_dir_offset = dir_offset(dir);\n+\twhile (~(name = *path++)) {\n+\t\tparent_dir_o = dir_offset(dir);\n+\n+\t\tkey = dirent_pointer(dirent_alloc(1));\n+\t\tkey->name_offset = name;\n+\n+\t\tdirent = dirent_search(&dir->entries, key);\n+\t\tif (dirent == NULL)\n+\t\t\tdirent = key;\n+\t\telse\n+\t\t\tdirent_free(1);\n+\n+\t\tif (dirent == key) {\n+\t\t\tdirent->mode = REPO_MODE_DIR;\n+\t\t\tdirent->content_offset = 0;\n+\t\t\tdirent_insert(&dir->entries, dirent);\n+\t\t}\n+\n+\t\tif (dirent_offset(dirent) < dirent_pool.committed) {\n+\t\t\tdir_o = repo_dirent_is_dir(dirent) ?\n+\t\t\t\t\tdirent->content_offset : ~0;\n+\t\t\tdirent_remove(&dir->entries, dirent);\n+\t\t\tdirent = dirent_pointer(dirent_alloc(1));\n+\t\t\tdirent->name_offset = name;\n+\t\t\tdirent->mode = REPO_MODE_DIR;\n+\t\t\tdirent->content_offset = dir_o;\n+\t\t\tdirent_insert(&dir->entries, dirent);\n+\t\t}\n+\n+\t\tdir = repo_dir_from_dirent(dirent);\n+\t\tdir = repo_clone_dir(dir);\n+\t\tdirent->content_offset = dir_offset(dir);\n+\t}\n+\tif (dirent == NULL)\n+\t\treturn;\n+\tdirent->mode = mode;\n+\tdirent->content_offset = content_offset;\n+\tif (del && ~parent_dir_o)\n+\t\tdirent_remove(&dir_pointer(parent_dir_o)->entries, dirent);\n+}\n+\n+uint32_t repo_copy(uint32_t revision, uint32_t *src, uint32_t *dst)\n+{\n+\tuint32_t mode = 0, content_offset = 0;\n+\tstruct repo_dirent *src_dirent;\n+\tsrc_dirent = repo_read_dirent(revision, src);\n+\tif (src_dirent != NULL) {\n+\t\tmode = src_dirent->mode;\n+\t\tcontent_offset = src_dirent->content_offset;\n+\t\trepo_write_dirent(dst, mode, content_offset, 0);\n+\t}\n+\treturn mode;\n+}\n+\n+void repo_add(uint32_t *path, uint32_t mode, uint32_t blob_mark)\n+{\n+\trepo_write_dirent(path, mode, blob_mark, 0);\n+}\n+\n+uint32_t repo_replace(uint32_t *path, uint32_t blob_mark)\n+{\n+\tuint32_t mode = 0;\n+\tstruct repo_dirent *src_dirent;\n+\tsrc_dirent = repo_read_dirent(active_commit, path);\n+\tif (src_dirent != NULL) {\n+\t\tmode = src_dirent->mode;\n+\t\trepo_write_dirent(path, mode, blob_mark, 0);\n+\t}\n+\treturn mode;\n+}\n+\n+void repo_modify(uint32_t *path, uint32_t mode, uint32_t blob_mark)\n+{\n+\tstruct repo_dirent *src_dirent;\n+\tsrc_dirent = repo_read_dirent(active_commit, path);\n+\tif (src_dirent != NULL && blob_mark == 0)\n+\t\tblob_mark = src_dirent->content_offset;\n+\trepo_write_dirent(path, mode, blob_mark, 0);\n+}\n+\n+void repo_delete(uint32_t *path)\n+{\n+\trepo_write_dirent(path, 0, 0, 1);\n+}\n+\n+static void repo_git_add_r(uint32_t depth, uint32_t *path, struct repo_dir *dir);\n+\n+static void repo_git_add(uint32_t depth, uint32_t *path, struct repo_dirent *dirent)\n+{\n+\tif (repo_dirent_is_dir(dirent))\n+\t\trepo_git_add_r(depth, path, repo_dir_from_dirent(dirent));\n+\telse\n+\t\tfast_export_modify(depth, path,\n+\t\t                   dirent->mode, dirent->content_offset);\n+}\n+\n+static void repo_git_add_r(uint32_t depth, uint32_t *path, struct repo_dir *dir)\n+{\n+\tstruct repo_dirent *de = repo_first_dirent(dir);\n+\twhile (de) {\n+\t\tpath[depth] = de->name_offset;\n+\t\trepo_git_add(depth + 1, path, de);\n+\t\tde = dirent_next(&dir->entries, de);\n+\t}\n+}\n+\n+static void repo_diff_r(uint32_t depth, uint32_t *path, struct repo_dir *dir1,\n+                        struct repo_dir *dir2)\n+{\n+\tstruct repo_dirent *de1, *de2;\n+\tde1 = repo_first_dirent(dir1);\n+\tde2 = repo_first_dirent(dir2);\n+\n+\twhile (de1 && de2) {\n+\t\tif (de1->name_offset < de2->name_offset) {\n+\t\t\tpath[depth] = de1->name_offset;\n+\t\t\tfast_export_delete(depth + 1, path);\n+\t\t\tde1 = dirent_next(&dir1->entries, de1);\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (de1->name_offset > de2->name_offset) {\n+\t\t\tpath[depth] = de2->name_offset;\n+\t\t\trepo_git_add(depth + 1, path, de2);\n+\t\t\tde2 = dirent_next(&dir2->entries, de2);\n+\t\t\tcontinue;\n+\t\t}\n+\t\tpath[depth] = de1->name_offset;\n+\n+\t\tif (de1->mode == de2->mode &&\n+\t\t    de1->content_offset == de2->content_offset) {\n+\t\t\t; /* No change. */\n+\t\t} else if (repo_dirent_is_dir(de1) && repo_dirent_is_dir(de2)) {\n+\t\t\trepo_diff_r(depth + 1, path,\n+\t\t\t\t    repo_dir_from_dirent(de1),\n+\t\t\t\t    repo_dir_from_dirent(de2));\n+\t\t} else if (!repo_dirent_is_dir(de1) && !repo_dirent_is_dir(de2)) {\n+\t\t\trepo_git_add(depth + 1, path, de2);\n+\t\t} else {\n+\t\t\tfast_export_delete(depth + 1, path);\n+\t\t\trepo_git_add(depth + 1, path, de2);\n+\t\t}\n+\t\tde1 = dirent_next(&dir1->entries, de1);\n+\t\tde2 = dirent_next(&dir2->entries, de2);\n+\t}\n+\twhile (de1) {\n+\t\tpath[depth] = de1->name_offset;\n+\t\tfast_export_delete(depth + 1, path);\n+\t\tde1 = dirent_next(&dir1->entries, de1);\n+\t}\n+\twhile (de2) {\n+\t\tpath[depth] = de2->name_offset;\n+\t\trepo_git_add(depth + 1, path, de2);\n+\t\tde2 = dirent_next(&dir2->entries, de2);\n+\t}\n+}\n+\n+static uint32_t path_stack[REPO_MAX_PATH_DEPTH];\n+\n+void repo_diff(uint32_t r1, uint32_t r2)\n+{\n+\trepo_diff_r(0,\n+\t            path_stack,\n+\t            repo_commit_root_dir(commit_pointer(r1)),\n+\t            repo_commit_root_dir(commit_pointer(r2)));\n+}\n+\n+void repo_commit(uint32_t revision, uint32_t author, char *log, uint32_t uuid,\n+                 uint32_t url, unsigned long timestamp)\n+{\n+\tfast_export_commit(revision, author, log, uuid, url, timestamp);\n+\tpool_commit();\n+\tdirent_commit();\n+\tdir_commit();\n+\tcommit_commit();\n+\tactive_commit = commit_alloc(1);\n+\tcommit_pointer(active_commit)->root_dir_offset =\n+\t\tcommit_pointer(active_commit - 1)->root_dir_offset;\n+}\n+\n+static void mark_init(void)\n+{\n+\tuint32_t i;\n+\tmark = 0;\n+\tfor (i = 0; i < dirent_pool.size; i++)\n+\t\tif (!repo_dirent_is_dir(dirent_pointer(i)) &&\n+\t\t    dirent_pointer(i)->content_offset > mark)\n+\t\t\tmark = dirent_pointer(i)->content_offset;\n+\tmark++;\n+}\n+\n+void repo_init() {\n+\tpool_init();\n+\tcommit_init();\n+\tdir_init();\n+\tdirent_init();\n+\tmark_init();\n+\tif (commit_pool.size == 0) {\n+\t\t/* Create empty tree for commit 0. */\n+\t\tcommit_alloc(1);\n+\t\tcommit_pointer(0)->root_dir_offset = dir_alloc(1);\n+\t\tdir_pointer(0)->entries.trp_root = ~0;\n+\t\tdir_commit();\n+\t\tcommit_commit();\n+\t}\n+\t/* Preallocate next commit, ready for changes. */\n+\tactive_commit = commit_alloc(1);\n+\tcommit_pointer(active_commit)->root_dir_offset =\n+\t\tcommit_pointer(active_commit - 1)->root_dir_offset;\n+}\n+\n+void repo_reset(void)\n+{\n+\tpool_reset();\n+\tcommit_reset();\n+\tdir_reset();\n+\tdirent_reset();\n+}\ndiff --git a/vcs-svn/repo_tree.h b/vcs-svn/repo_tree.h\nnew file mode 100644\nindex 0000000..92a7a7b\n--- /dev/null\n+++ b/vcs-svn/repo_tree.h\n@@ -0,0 +1,26 @@\n+#ifndef REPO_TREE_H_\n+#define REPO_TREE_H_\n+\n+#include \"git-compat-util.h\"\n+\n+#define REPO_MODE_DIR 0040000\n+#define REPO_MODE_BLB 0100644\n+#define REPO_MODE_EXE 0100755\n+#define REPO_MODE_LNK 0120000\n+\n+#define REPO_MAX_PATH_LEN 4096\n+#define REPO_MAX_PATH_DEPTH 1000\n+\n+uint32_t next_blob_mark(void);\n+uint32_t repo_copy(uint32_t revision, uint32_t *src, uint32_t *dst);\n+void repo_add(uint32_t *path, uint32_t mode, uint32_t blob_mark);\n+uint32_t repo_replace(uint32_t *path, uint32_t blob_mark);\n+void repo_modify(uint32_t *path, uint32_t mode, uint32_t blob_mark);\n+void repo_delete(uint32_t *path);\n+void repo_commit(uint32_t revision, uint32_t author, char *log, uint32_t uuid,\n+                 uint32_t url, long unsigned timestamp);\n+void repo_diff(uint32_t r1, uint32_t r2);\n+void repo_init(void);\n+void repo_reset(void);\n+\n+#endif\n-- \n1.7.1\n"},{"id":"145634","messageId":"1279210984-31604-9-git-send-email-artagnon@gmail.com","threadId":"24407","inReplyTo":"1279210984-31604-1-git-send-email-artagnon@gmail.com","subject":"[PATCH 8/8] Add SVN dump parser","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-07-15T16:23:04Z","receivedAt":"2010-07-15T16:23:04Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"From: David Barr <david.barr@cordelta.com>\n\nsvndump parses data that is in SVN dumpfile format produced by\n`svnadmin dump` with the help of line_buffer and uses repo_tree and\nfast_export to emit a git fast-import stream.\n\nBased roughly on com.hydrografix.svndump 0.92 from the SvnToCCase\nproject at <http://svn2cc.sarovar.org/>, by Stefan Hegny and\nothers.\n\nSigned-off-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n Makefile          |    5 +-\n vcs-svn/LICENSE   |    4 +\n vcs-svn/svndump.c |  289 +++++++++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/svndump.h |    8 ++\n 4 files changed, 304 insertions(+), 2 deletions(-)\n create mode 100644 vcs-svn/svndump.c\n create mode 100644 vcs-svn/svndump.h\n\ndiff --git a/Makefile b/Makefile\nindex 7c66dcc..e7b37e0 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1741,7 +1741,7 @@ endif\n XDIFF_OBJS = xdiff/xdiffi.o xdiff/xprepare.o xdiff/xutils.o xdiff/xemit.o \\\n \txdiff/xmerge.o xdiff/xpatience.o\n VCSSVN_OBJS = vcs-svn/string_pool.o vcs-svn/line_buffer.o \\\n-\tvcs-svn/repo_tree.o vcs-svn/fast_export.o\n+\tvcs-svn/repo_tree.o vcs-svn/fast_export.o vcs-svn/svndump.o\n OBJECTS := $(GIT_OBJS) $(XDIFF_OBJS) $(VCSSVN_OBJS)\n \n dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d)\n@@ -1866,7 +1866,8 @@ xdiff-interface.o $(XDIFF_OBJS): \\\n \n $(VCSSVN_OBJS): \\\n \tvcs-svn/obj_pool.h vcs-svn/trp.h vcs-svn/string_pool.h \\\n-\tvcs-svn/line_buffer.h vcs-svn/repo_tree.h vcs-svn/fast_export.h\n+\tvcs-svn/line_buffer.h vcs-svn/repo_tree.h vcs-svn/fast_export.h \\\n+\tvcs-svn/svndump.h\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\ndiff --git a/vcs-svn/LICENSE b/vcs-svn/LICENSE\nindex a3d384c..0a5e3c4 100644\n--- a/vcs-svn/LICENSE\n+++ b/vcs-svn/LICENSE\n@@ -4,6 +4,10 @@ All rights reserved.\n Copyright (C) 2008 Jason Evans <jasone@canonware.com>.\n All rights reserved.\n \n+Copyright (C) 2005 Stefan Hegny, hydrografix Consulting GmbH,\n+Frankfurt/Main, Germany\n+and others, see http://svn2cc.sarovar.org\n+\n Redistribution and use in source and binary forms, with or without\n modification, are permitted provided that the following conditions\n are met:\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nnew file mode 100644\nindex 0000000..86714ed\n--- /dev/null\n+++ b/vcs-svn/svndump.c\n@@ -0,0 +1,289 @@\n+/*\n+ * Parse and rearrange a svnadmin dump.\n+ * Create the dump with:\n+ * svnadmin dump --incremental -r<startrev>:<endrev> <repository> >outfile\n+ *\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"cache.h\"\n+#include \"repo_tree.h\"\n+#include \"fast_export.h\"\n+#include \"line_buffer.h\"\n+#include \"obj_pool.h\"\n+#include \"string_pool.h\"\n+\n+#define NODEACT_REPLACE 4\n+#define NODEACT_DELETE 3\n+#define NODEACT_ADD 2\n+#define NODEACT_CHANGE 1\n+#define NODEACT_UNKNOWN 0\n+\n+#define DUMP_CTX 0\n+#define REV_CTX  1\n+#define NODE_CTX 2\n+\n+#define LENGTH_UNKNOWN (~0)\n+#define DATE_RFC2822_LEN 31\n+\n+/* Create memory pool for log messages */\n+obj_pool_gen(log, char, 4096);\n+\n+static char* log_copy(uint32_t length, char *log)\n+{\n+\tchar *buffer;\n+\tlog_free(log_pool.size);\n+\tbuffer = log_pointer(log_alloc(length));\n+\tstrncpy(buffer, log, length);\n+\treturn buffer;\n+}\n+\n+static struct {\n+\tuint32_t action, propLength, textLength, srcRev, srcMode, mark, type;\n+\tuint32_t src[REPO_MAX_PATH_DEPTH], dst[REPO_MAX_PATH_DEPTH];\n+} node_ctx;\n+\n+static struct {\n+\tuint32_t revision, author;\n+\tunsigned long timestamp;\n+\tchar *log;\n+} rev_ctx;\n+\n+static struct {\n+\tuint32_t uuid, url;\n+} dump_ctx;\n+\n+static struct {\n+\tuint32_t svn_log, svn_author, svn_date, svn_executable, svn_special, uuid,\n+\t\trevision_number, node_path, node_kind, node_action,\n+\t\tnode_copyfrom_path, node_copyfrom_rev, text_content_length,\n+\t\tprop_content_length, content_length;\n+} keys;\n+\n+static void reset_node_ctx(char *fname)\n+{\n+\tnode_ctx.type = 0;\n+\tnode_ctx.action = NODEACT_UNKNOWN;\n+\tnode_ctx.propLength = LENGTH_UNKNOWN;\n+\tnode_ctx.textLength = LENGTH_UNKNOWN;\n+\tnode_ctx.src[0] = ~0;\n+\tnode_ctx.srcRev = 0;\n+\tnode_ctx.srcMode = 0;\n+\tpool_tok_seq(REPO_MAX_PATH_DEPTH, node_ctx.dst, \"/\", fname);\n+\tnode_ctx.mark = 0;\n+}\n+\n+static void reset_rev_ctx(uint32_t revision)\n+{\n+\trev_ctx.revision = revision;\n+\trev_ctx.timestamp = 0;\n+\trev_ctx.log = NULL;\n+\trev_ctx.author = ~0;\n+}\n+\n+static void reset_dump_ctx(uint32_t url)\n+{\n+\tdump_ctx.url = url;\n+\tdump_ctx.uuid = ~0;\n+}\n+\n+static void init_keys(void)\n+{\n+\tkeys.svn_log = pool_intern(\"svn:log\");\n+\tkeys.svn_author = pool_intern(\"svn:author\");\n+\tkeys.svn_date = pool_intern(\"svn:date\");\n+\tkeys.svn_executable = pool_intern(\"svn:executable\");\n+\tkeys.svn_special = pool_intern(\"svn:special\");\n+\tkeys.uuid = pool_intern(\"UUID\");\n+\tkeys.revision_number = pool_intern(\"Revision-number\");\n+\tkeys.node_path = pool_intern(\"Node-path\");\n+\tkeys.node_kind = pool_intern(\"Node-kind\");\n+\tkeys.node_action = pool_intern(\"Node-action\");\n+\tkeys.node_copyfrom_path = pool_intern(\"Node-copyfrom-path\");\n+\tkeys.node_copyfrom_rev = pool_intern(\"Node-copyfrom-rev\");\n+\tkeys.text_content_length = pool_intern(\"Text-content-length\");\n+\tkeys.prop_content_length = pool_intern(\"Prop-content-length\");\n+\tkeys.content_length = pool_intern(\"Content-length\");\n+}\n+\n+static void read_props(void)\n+{\n+\tuint32_t len;\n+\tuint32_t key = ~0;\n+\tchar *val = NULL;\n+\tchar *t;\n+\twhile ((t = buffer_read_line()) && strcmp(t, \"PROPS-END\")) {\n+\t\tif (!strncmp(t, \"K \", 2)) {\n+\t\t\tlen = atoi(&t[2]);\n+\t\t\tkey = pool_intern(buffer_read_string(len));\n+\t\t\tbuffer_read_line();\n+\t\t} else if (!strncmp(t, \"V \", 2)) {\n+\t\t\tlen = atoi(&t[2]);\n+\t\t\tval = buffer_read_string(len);\n+\t\t\tif (key == keys.svn_log) {\n+\t\t\t\t/* Value length excludes terminating nul. */\n+\t\t\t\trev_ctx.log = log_copy(len + 1, val);\n+\t\t\t} else if (key == keys.svn_author) {\n+\t\t\t\trev_ctx.author = pool_intern(val);\n+\t\t\t} else if (key == keys.svn_date) {\n+\t\t\t\tif (parse_date_basic(val, &rev_ctx.timestamp, NULL))\n+\t\t\t\t\tfprintf(stderr, \"Invalid timestamp: %s\\n\", val);\n+\t\t\t} else if (key == keys.svn_executable) {\n+\t\t\t\tnode_ctx.type = REPO_MODE_EXE;\n+\t\t\t} else if (key == keys.svn_special) {\n+\t\t\t\tnode_ctx.type = REPO_MODE_LNK;\n+\t\t\t}\n+\t\t\tkey = ~0;\n+\t\t\tbuffer_read_line();\n+\t\t}\n+\t}\n+}\n+\n+static void handle_node(void)\n+{\n+\tif (node_ctx.propLength != LENGTH_UNKNOWN && node_ctx.propLength)\n+\t\tread_props();\n+\n+\tif (node_ctx.srcRev)\n+\t\tnode_ctx.srcMode = repo_copy(node_ctx.srcRev, node_ctx.src, node_ctx.dst);\n+\n+\tif (node_ctx.textLength != LENGTH_UNKNOWN &&\n+\t    node_ctx.type != REPO_MODE_DIR)\n+\t\tnode_ctx.mark = next_blob_mark();\n+\n+\tif (node_ctx.action == NODEACT_DELETE) {\n+\t\trepo_delete(node_ctx.dst);\n+\t} else if (node_ctx.action == NODEACT_CHANGE ||\n+\t\t\t   node_ctx.action == NODEACT_REPLACE) {\n+\t\tif (node_ctx.action == NODEACT_REPLACE &&\n+\t\t    node_ctx.type == REPO_MODE_DIR)\n+\t\t\trepo_replace(node_ctx.dst, node_ctx.mark);\n+\t\telse if (node_ctx.propLength != LENGTH_UNKNOWN)\n+\t\t\trepo_modify(node_ctx.dst, node_ctx.type, node_ctx.mark);\n+\t\telse if (node_ctx.textLength != LENGTH_UNKNOWN)\n+\t\t\tnode_ctx.srcMode = repo_replace(node_ctx.dst, node_ctx.mark);\n+\t} else if (node_ctx.action == NODEACT_ADD) {\n+\t\tif (node_ctx.srcRev && node_ctx.propLength != LENGTH_UNKNOWN)\n+\t\t\trepo_modify(node_ctx.dst, node_ctx.type, node_ctx.mark);\n+\t\telse if (node_ctx.srcRev && node_ctx.textLength != LENGTH_UNKNOWN)\n+\t\t\tnode_ctx.srcMode = repo_replace(node_ctx.dst, node_ctx.mark);\n+\t\telse if ((node_ctx.type == REPO_MODE_DIR && !node_ctx.srcRev) ||\n+\t\t         node_ctx.textLength != LENGTH_UNKNOWN)\n+\t\t\trepo_add(node_ctx.dst, node_ctx.type, node_ctx.mark);\n+\t}\n+\n+\tif (node_ctx.propLength == LENGTH_UNKNOWN && node_ctx.srcMode)\n+\t\tnode_ctx.type = node_ctx.srcMode;\n+\n+\tif (node_ctx.mark)\n+\t\tfast_export_blob(node_ctx.type, node_ctx.mark, node_ctx.textLength);\n+\telse if (node_ctx.textLength != LENGTH_UNKNOWN)\n+\t\tbuffer_skip_bytes(node_ctx.textLength);\n+}\n+\n+static void handle_revision(void)\n+{\n+\tif (rev_ctx.revision)\n+\t\trepo_commit(rev_ctx.revision, rev_ctx.author, rev_ctx.log,\n+\t\t\tdump_ctx.uuid, dump_ctx.url, rev_ctx.timestamp);\n+}\n+\n+void svndump_read(char *url)\n+{\n+\tchar *val;\n+\tchar *t;\n+\tuint32_t active_ctx = DUMP_CTX;\n+\tuint32_t len;\n+\tuint32_t key;\n+\n+\treset_dump_ctx(pool_intern(url));\n+\twhile ((t = buffer_read_line())) {\n+\t\tval = strstr(t, \": \");\n+\t\tif (!val)\n+\t\t\tcontinue;\n+\t\t*val++ = '\\0';\n+\t\t*val++ = '\\0';\n+\t\tkey = pool_intern(t);\n+\n+\t\tif (key == keys.uuid) {\n+\t\t\tdump_ctx.uuid = pool_intern(val);\n+\t\t} else if (key == keys.revision_number) {\n+\t\t\tif (active_ctx == NODE_CTX)\n+\t\t\t\thandle_node();\n+\t\t\tif (active_ctx != DUMP_CTX)\n+\t\t\t\thandle_revision();\n+\t\t\tactive_ctx = REV_CTX;\n+\t\t\treset_rev_ctx(atoi(val));\n+\t\t} else if (key == keys.node_path) {\n+\t\t\tif (active_ctx == NODE_CTX)\n+\t\t\t\thandle_node();\n+\t\t\tactive_ctx = NODE_CTX;\n+\t\t\treset_node_ctx(val);\n+\t\t} else if (key == keys.node_kind) {\n+\t\t\tif (!strcmp(val, \"dir\"))\n+\t\t\t\tnode_ctx.type = REPO_MODE_DIR;\n+\t\t\telse if (!strcmp(val, \"file\"))\n+\t\t\t\tnode_ctx.type = REPO_MODE_BLB;\n+\t\t\telse\n+\t\t\t\tfprintf(stderr, \"Unknown node-kind: %s\\n\", val);\n+\t\t} else if (key == keys.node_action) {\n+\t\t\tif (!strcmp(val, \"delete\")) {\n+\t\t\t\tnode_ctx.action = NODEACT_DELETE;\n+\t\t\t} else if (!strcmp(val, \"add\")) {\n+\t\t\t\tnode_ctx.action = NODEACT_ADD;\n+\t\t\t} else if (!strcmp(val, \"change\")) {\n+\t\t\t\tnode_ctx.action = NODEACT_CHANGE;\n+\t\t\t} else if (!strcmp(val, \"replace\")) {\n+\t\t\t\tnode_ctx.action = NODEACT_REPLACE;\n+\t\t\t} else {\n+\t\t\t\tfprintf(stderr, \"Unknown node-action: %s\\n\", val);\n+\t\t\t\tnode_ctx.action = NODEACT_UNKNOWN;\n+\t\t\t}\n+\t\t} else if (key == keys.node_copyfrom_path) {\n+\t\t\tpool_tok_seq(REPO_MAX_PATH_DEPTH, node_ctx.src, \"/\", val);\n+\t\t} else if (key == keys.node_copyfrom_rev) {\n+\t\t\tnode_ctx.srcRev = atoi(val);\n+\t\t} else if (key == keys.text_content_length) {\n+\t\t\tnode_ctx.textLength = atoi(val);\n+\t\t} else if (key == keys.prop_content_length) {\n+\t\t\tnode_ctx.propLength = atoi(val);\n+\t\t} else if (key == keys.content_length) {\n+\t\t\tlen = atoi(val);\n+\t\t\tbuffer_read_line();\n+\t\t\tif (active_ctx == REV_CTX) {\n+\t\t\t\tread_props();\n+\t\t\t} else if (active_ctx == NODE_CTX) {\n+\t\t\t\thandle_node();\n+\t\t\t\tactive_ctx = REV_CTX;\n+\t\t\t} else {\n+\t\t\t\tfprintf(stderr, \"Unexpected content length header: %d\\n\", len);\n+\t\t\t\tbuffer_skip_bytes(len);\n+\t\t\t}\n+\t\t}\n+\t}\n+\tif (active_ctx == NODE_CTX)\n+\t\thandle_node();\n+\tif (active_ctx != DUMP_CTX)\n+\t\thandle_revision();\n+}\n+\n+void svndump_init(const char *filename)\n+{\n+\tbuffer_init(filename);\n+\trepo_init();\n+\treset_dump_ctx(~0);\n+\treset_rev_ctx(0);\n+\treset_node_ctx(NULL);\n+\tinit_keys();\n+}\n+\n+void svndump_reset(void)\n+{\n+\tlog_reset();\n+\tbuffer_reset();\n+\trepo_reset();\n+\treset_dump_ctx(~0);\n+\treset_rev_ctx(0);\n+\treset_node_ctx(NULL);\n+}\ndiff --git a/vcs-svn/svndump.h b/vcs-svn/svndump.h\nnew file mode 100644\nindex 0000000..38ad544\n--- /dev/null\n+++ b/vcs-svn/svndump.h\n@@ -0,0 +1,8 @@\n+#ifndef SVNDUMP_H_\n+#define SVNDUMP_H_\n+\n+void svndump_init(const char *filename);\n+void svndump_read(char *url);\n+void svndump_reset(void);\n+\n+#endif\n-- \n1.7.1\n"},{"id":"145639","messageId":"20100715172506.GA23989@burratino","threadId":"24407","inReplyTo":"1279210984-31604-2-git-send-email-artagnon@gmail.com","subject":"Re: [PATCH 1/8] Export parse_date_basic() to convert a date string to timestamp","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-07-15T17:25:06Z","receivedAt":"2010-07-15T17:25:06Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Ramkumar Ramachandra wrote:\n\n> approxidate() is not appropriate for reading machine-written dates\n> because it guesses instead of erroring out on malformed dates.\n> parse_date() is less convenient since it returns its output as a\n> string.  So export the underlying function that writes a timestamp.\n> \n> While at it, change the return value to match the usual convention:\n> return 0 for success and -1 for failure.\n\nJunio: I think this should be ejected from the series as an\nindependently useful cleanup.\n\nCurrently parse_date_toffset() is exported but not declared anywhere.\nThis patch gives it a more predictable API and adds a declaration.\n\nRam: thanks for the reminder.\n"},{"id":"145642","messageId":"20100715174629.GA32323@burratino","threadId":"24407","inReplyTo":"1279210984-31604-3-git-send-email-artagnon@gmail.com","subject":"Re: [PATCH 2/8] Introduce vcs-svn lib","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-07-15T17:46:29Z","receivedAt":"2010-07-15T17:46:29Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Ramkumar Ramachandra wrote:\n\n> @@ -1908,6 +1912,8 @@ $(LIB_FILE): $(LIB_OBJS)\n>  $(XDIFF_LIB): $(XDIFF_OBJS)\n>  \t$(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $(XDIFF_OBJS)\n>  \n> +$(VCSSVN_LIB): $(VCSSVN_OBJS)\n> +\t$(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $(VCSSVN_OBJS)\n\n $ make vcs-svn/lib.a V=1\n rm -f vcs-svn/lib.a && ar rcs vcs-svn/lib.a \n ar: vcs-svn/lib.a: No such file or directory\n make: *** [vcs-svn/lib.a] Error 1\n\nThat is because the vcs-svn directory does not exist.  So\nprobably the LICENSE should be added with the same patch\n(and git should learn to track empty directories).\n\nJonathan\n"},{"id":"145649","messageId":"20100715185706.GA2774@burratino","threadId":"24407","inReplyTo":"1279210984-31604-4-git-send-email-artagnon@gmail.com","subject":"Re: [PATCH 3/8] Add memory pool library","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-07-15T18:57:06Z","receivedAt":"2010-07-15T18:57:06Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Ramkumar Ramachandra wrote:\n\n> void pre_commit(void);\n> \n> \tWrite the pool to file.\n\nExcept as a proof of concept, this is the wrong API to have.  The problem\nis that the caller cannot choose the filename, so it ends up being a .bin\nfile in the current directory, wherever that is.\n\nThe log message leaves out a subtlety: this also increases the\n‘committed’ value, and bookkeeping for that might be useful to some\ncallers.\n\nIn other words:\n\n> +static MAYBE_UNUSED void pre##_init(void) \\\n> +{ \\\n> +\tstruct stat st; \\\n> +\tpre##_pool.file = fopen(#pre \".bin\", \"a+\"); \\\n> +\trewind(pre##_pool.file); \\\n> +\tfstat(fileno(pre##_pool.file), &st); \\\n> +\tpre##_pool.size = st.st_size / sizeof(obj_t); \\\n> +\tpre##_pool.committed = pre##_pool.size; \\\n> +\tpre##_pool.capacity = pre##_pool.size * 2; \\\n> +\tif (pre##_pool.capacity < initial_capacity) \\\n> +\t\tpre##_pool.capacity = initial_capacity; \\\n> +\tpre##_pool.base = malloc(pre##_pool.capacity * sizeof(obj_t)); \\\n> +\tfread(pre##_pool.base, sizeof(obj_t), pre##_pool.size, pre##_pool.file); \\\n> +} \\\n\nIf you just want something working, I’d suggest stubbing this out:\n\n static MAYBE_UNUSED void pre##_init(void) \\\n { \\\n } \\\n\nIt even almost makes sense as API: the _init function does all\ninitialization tasks required, which is to say, none.  (The {0, ...}\ninitializer already has taken care of setting all fields to 0).\n\n> +static MAYBE_UNUSED void pre##_commit(void) \\\n> +{ \\\n> +\tpre##_pool.committed += fwrite(pre##_pool.base + pre##_pool.committed, \\\n> +\t\tsizeof(obj_t), pre##_pool.size - pre##_pool.committed, \\\n> +\t\tpre##_pool.file); \\\n> +} \\\n\nThis can be simplified\n\n static MAYBE_UNUSED void pre##_commit(void) \\\n { \\\n\tpre##_pool.committed = pre##_pool.size; \\\n } \\\n\nIn other words, maybe something like this on top?  This includes the\nvestigal _init() function which really should not be there (it is\nconfusing that some callers use it and others don’t).  I did not\nspend much time on it because in the end I suspect we might throw\nobj_pool away anyway.\n\n---\ndiff --git a/Makefile b/Makefile\nindex fc31ee0..386a586 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -409,6 +409,7 @@ TEST_PROGRAMS_NEED_X += test-delta\n TEST_PROGRAMS_NEED_X += test-dump-cache-tree\n TEST_PROGRAMS_NEED_X += test-genrandom\n TEST_PROGRAMS_NEED_X += test-match-trees\n+TEST_PROGRAMS_NEED_X += test-obj-pool\n TEST_PROGRAMS_NEED_X += test-parse-options\n TEST_PROGRAMS_NEED_X += test-path-utils\n TEST_PROGRAMS_NEED_X += test-run-command\ndiff --git a/t/t0070-fundamental.sh b/t/t0070-fundamental.sh\nindex 680d7d6..262f304 100755\n--- a/t/t0070-fundamental.sh\n+++ b/t/t0070-fundamental.sh\n@@ -12,4 +12,10 @@ test_expect_success 'character classes (isspace, isalpha etc.)' '\n \ttest-ctype\n '\n \n+test_expect_success 'allocator for svn importer' '\n+\tprintf \"%s\\n\" 0 0 0 0 0 2 2 2 2 2 -1 >expected &&\n+\ttest-obj-pool >actual &&\n+\ttest_cmp expected actual\n+'\n+\n test_done\ndiff --git a/test-obj-pool.c b/test-obj-pool.c\nnew file mode 100644\nindex 0000000..1300049\n--- /dev/null\n+++ b/test-obj-pool.c\n@@ -0,0 +1,55 @@\n+/*\n+ * test-obj-pool.c: code to exercise the svn importer's object pool\n+ */\n+\n+#include \"vcs-svn/obj_pool.h\"\n+\n+static const char usage_str[] =\n+\t\"test-obj-pool\";\n+\n+obj_pool_gen(int, int, 2)\n+obj_pool_gen(other, int, 4096)\n+\n+int main(int argc, char *argv[])\n+{\n+\tint *p;\n+\tint i;\n+\n+\tif (argc != 1) {\n+\t\tfprintf(stderr, \"Usage: %s\\n\", usage_str);\n+\t\treturn 1;\n+\t}\n+\n+\tint_init();\n+\n+\tp = int_pointer(int_alloc(10));\n+\tfor (i = 0; i < 10; i++)\n+\t\tp[i] = 0;\n+\n+\t*other_pointer(other_alloc(1)) = -1;\n+\tother_commit();\n+\n+\tp = other_pointer(other_alloc(10));\n+\tfor (i = 0; i < 10; i++)\n+\t\tp[i] = 1;\n+\n+\tint_free(5);\n+\n+\tp = int_pointer(int_alloc(10));\n+\tfor (i = 0; i < 10; i++)\n+\t\tp[i] = 2;\n+\n+\tint_free(5);\n+\tint_commit();\n+\n+\tfor (i = 0; i < int_pool.committed; i++)\n+\t\tprintf(\"%d\\n\", *int_pointer(i));\n+\n+\tfor (i = 0; i < other_pool.committed; i++)\n+\t\tprintf(\"%d\\n\", *other_pointer(i));\n+\n+\tint_reset();\n+\tint_reset();\n+\tother_reset();\n+\treturn 0;\n+}\ndiff --git a/vcs-svn/obj_pool.h b/vcs-svn/obj_pool.h\nindex f60c872..90eda15 100644\n--- a/vcs-svn/obj_pool.h\n+++ b/vcs-svn/obj_pool.h\n@@ -16,21 +16,9 @@ static struct { \\\n \tuint32_t size; \\\n \tuint32_t capacity; \\\n \tobj_t *base; \\\n-\tFILE *file; \\\n-} pre##_pool = { 0, 0, 0, NULL, NULL}; \\\n+} pre##_pool = {0, 0, 0, NULL}; \\\n static MAYBE_UNUSED void pre##_init(void) \\\n { \\\n-\tstruct stat st; \\\n-\tpre##_pool.file = fopen(#pre \".bin\", \"a+\"); \\\n-\trewind(pre##_pool.file); \\\n-\tfstat(fileno(pre##_pool.file), &st); \\\n-\tpre##_pool.size = st.st_size / sizeof(obj_t); \\\n-\tpre##_pool.committed = pre##_pool.size; \\\n-\tpre##_pool.capacity = pre##_pool.size * 2; \\\n-\tif (pre##_pool.capacity < initial_capacity) \\\n-\t\tpre##_pool.capacity = initial_capacity; \\\n-\tpre##_pool.base = malloc(pre##_pool.capacity * sizeof(obj_t)); \\\n-\tfread(pre##_pool.base, sizeof(obj_t), pre##_pool.size, pre##_pool.file); \\\n } \\\n static MAYBE_UNUSED uint32_t pre##_alloc(uint32_t count) \\\n { \\\n@@ -62,19 +50,15 @@ static MAYBE_UNUSED obj_t *pre##_pointer(uint32_t offset) \\\n } \\\n static MAYBE_UNUSED void pre##_commit(void) \\\n { \\\n-\tpre##_pool.committed += fwrite(pre##_pool.base + pre##_pool.committed, \\\n-\t\tsizeof(obj_t), pre##_pool.size - pre##_pool.committed, \\\n-\t\tpre##_pool.file); \\\n+\tpre##_pool.committed = pre##_pool.size; \\\n } \\\n static MAYBE_UNUSED void pre##_reset(void) \\\n { \\\n \tfree(pre##_pool.base); \\\n-\tif (pre##_pool.file) \\\n-\t\tfclose(pre##_pool.file); \\\n \tpre##_pool.base = NULL; \\\n \tpre##_pool.size = 0; \\\n \tpre##_pool.capacity = 0; \\\n-\tpre##_pool.file = NULL; \\\n+\tpre##_pool.committed = 0; \\\n }\n \n #endif\n-- \n"},{"id":"145652","messageId":"20100715190902.GB2774@burratino","threadId":"24407","inReplyTo":"1279210984-31604-5-git-send-email-artagnon@gmail.com","subject":"Re: [PATCH 4/8] Add treap implementation","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-07-15T19:09:02Z","receivedAt":"2010-07-15T19:09:02Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Ramkumar Ramachandra wrote:\n\n> Treaps provide a memory-efficient binary search tree structure.\n> Insertion/deletion/search are about as about as fast in the average\n> case as red-black trees and the chances of worst-case behavior are\n> vanishingly small, thanks to (pseudo-)randomness.  The bad worst-case\n> behavior is a small price to pay, given that treaps are much simpler\n> to implement.\n\nI still haven’t checked this implementation in detail, but it seemed\nto work in practice and is about to change anyway.\n\nI like the documentation updates.  What else changed from the\nprevious round?\n"},{"id":"145653","messageId":"20100715191213.GJ22574@debian","threadId":"24407","inReplyTo":"20100715185706.GA2774@burratino","subject":"Re: [PATCH 3/8] Add memory pool library","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-07-15T19:12:13Z","receivedAt":"2010-07-15T19:12:13Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi Jonathan,\n\nJonathan Nieder writes:\n> Ramkumar Ramachandra wrote:\n> \n> > void pre_commit(void);\n> > \n> > \tWrite the pool to file.\n> \n> Except as a proof of concept, this is the wrong API to have.  The problem\n> is that the caller cannot choose the filename, so it ends up being a .bin\n> file in the current directory, wherever that is.\n> \n> The log message leaves out a subtlety: this also increases the\n> ‘committed’ value, and bookkeeping for that might be useful to some\n> callers.\n> \n> In other words:\n> \n> > +static MAYBE_UNUSED void pre##_init(void) \\\n> > +{ \\\n> > +\tstruct stat st; \\\n> > +\tpre##_pool.file = fopen(#pre \".bin\", \"a+\"); \\\n> > +\trewind(pre##_pool.file); \\\n> > +\tfstat(fileno(pre##_pool.file), &st); \\\n> > +\tpre##_pool.size = st.st_size / sizeof(obj_t); \\\n> > +\tpre##_pool.committed = pre##_pool.size; \\\n> > +\tpre##_pool.capacity = pre##_pool.size * 2; \\\n> > +\tif (pre##_pool.capacity < initial_capacity) \\\n> > +\t\tpre##_pool.capacity = initial_capacity; \\\n> > +\tpre##_pool.base = malloc(pre##_pool.capacity * sizeof(obj_t)); \\\n> > +\tfread(pre##_pool.base, sizeof(obj_t), pre##_pool.size, pre##_pool.file); \\\n> > +} \\\n> \n> If you just want something working, I’d suggest stubbing this out:\n> \n>  static MAYBE_UNUSED void pre##_init(void) \\\n>  { \\\n>  } \\\n> \n> It even almost makes sense as API: the _init function does all\n> initialization tasks required, which is to say, none.  (The {0, ...}\n> initializer already has taken care of setting all fields to 0).\n> \n> > +static MAYBE_UNUSED void pre##_commit(void) \\\n> > +{ \\\n> > +\tpre##_pool.committed += fwrite(pre##_pool.base + pre##_pool.committed, \\\n> > +\t\tsizeof(obj_t), pre##_pool.size - pre##_pool.committed, \\\n> > +\t\tpre##_pool.file); \\\n> > +} \\\n> \n> This can be simplified\n> \n>  static MAYBE_UNUSED void pre##_commit(void) \\\n>  { \\\n> \tpre##_pool.committed = pre##_pool.size; \\\n>  } \\\n> \n> In other words, maybe something like this on top?  This includes the\n> vestigal _init() function which really should not be there (it is\n> confusing that some callers use it and others don’t).  I did not\n> spend much time on it because in the end I suspect we might throw\n> obj_pool away anyway.\n\nOh, right. I remember that you asked to turn off persistence for this\nmerge. We can include persistence it in a later series.\n\nJunio: Could you squash this diff into the commit?\n\ndiff --git a/vcs-svn/obj_pool.h b/vcs-svn/obj_pool.h\nindex f60c872..7a256d4 100644\n--- a/vcs-svn/obj_pool.h\n+++ b/vcs-svn/obj_pool.h\n@@ -20,17 +20,6 @@ static struct { \\\n } pre##_pool = { 0, 0, 0, NULL, NULL}; \\\n static MAYBE_UNUSED void pre##_init(void) \\\n { \\\n-\tstruct stat st; \\\n-\tpre##_pool.file = fopen(#pre \".bin\", \"a+\"); \\\n-\trewind(pre##_pool.file); \\\n-\tfstat(fileno(pre##_pool.file), &st); \\\n-\tpre##_pool.size = st.st_size / sizeof(obj_t); \\\n-\tpre##_pool.committed = pre##_pool.size; \\\n-\tpre##_pool.capacity = pre##_pool.size * 2; \\\n-\tif (pre##_pool.capacity < initial_capacity) \\\n-\t\tpre##_pool.capacity = initial_capacity; \\\n-\tpre##_pool.base = malloc(pre##_pool.capacity * sizeof(obj_t)); \\\n-\tfread(pre##_pool.base, sizeof(obj_t), pre##_pool.size, pre##_pool.file); \\\n } \\\n static MAYBE_UNUSED uint32_t pre##_alloc(uint32_t count) \\\n { \\\n@@ -62,9 +51,7 @@ static MAYBE_UNUSED obj_t *pre##_pointer(uint32_t offset) \\\n } \\\n static MAYBE_UNUSED void pre##_commit(void) \\\n { \\\n-\tpre##_pool.committed += fwrite(pre##_pool.base + pre##_pool.committed, \\\n-\t\tsizeof(obj_t), pre##_pool.size - pre##_pool.committed, \\\n-\t\tpre##_pool.file); \\\n+\tpre##_pool.committed = pre##_pool.size; \\\n } \\\n static MAYBE_UNUSED void pre##_reset(void) \\\n { \\\n"},{"id":"145654","messageId":"20100715191533.GK22574@debian","threadId":"24407","inReplyTo":"20100715174629.GA32323@burratino","subject":"Re: [PATCH 2/8] Introduce vcs-svn lib","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-07-15T19:15:33Z","receivedAt":"2010-07-15T19:15:33Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi Jonathan,\n\nJonathan Nieder writes:\n> Ramkumar Ramachandra wrote:\n> \n> > @@ -1908,6 +1912,8 @@ $(LIB_FILE): $(LIB_OBJS)\n> >  $(XDIFF_LIB): $(XDIFF_OBJS)\n> >  \t$(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $(XDIFF_OBJS)\n> >  \n> > +$(VCSSVN_LIB): $(VCSSVN_OBJS)\n> > +\t$(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $(VCSSVN_OBJS)\n> \n>  $ make vcs-svn/lib.a V=1\n>  rm -f vcs-svn/lib.a && ar rcs vcs-svn/lib.a \n>  ar: vcs-svn/lib.a: No such file or directory\n>  make: *** [vcs-svn/lib.a] Error 1\n> \n> That is because the vcs-svn directory does not exist.  So\n> probably the LICENSE should be added with the same patch\n> (and git should learn to track empty directories).\n\nOops. Sorry about not checking this: it looked alright at a\nglance. Yes, we can add LICENSE with this patch.\n\nJunio: Could you squash in this diff?\n\n--- /dev/null\n+++ b/vcs-svn/LICENSE\n@@ -0,0 +1,26 @@\n+Copyright (C) 2010 David Barr <david.barr@cordelta.com>.\n+All rights reserved.\n+\n+Redistribution and use in source and binary forms, with or without\n+modification, are permitted provided that the following conditions\n+are met:\n+1. Redistributions of source code must retain the above copyright\n+   notice(s), this list of conditions and the following disclaimer\n+   unmodified other than the allowable addition of one or more\n+   copyright notices.\n+2. Redistributions in binary form must reproduce the above copyright\n+   notice(s), this list of conditions and the following disclaimer in\n+   the documentation and/or other materials provided with the\n+   distribution.\n+\n+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER(S) ``AS IS'' AND ANY\n+EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE\n+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR\n+PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT HOLDER(S) BE\n+LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR\n+CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF\n+SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR\n+BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,\n+WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE\n+OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,\n+EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n\n-- Ram\n"},{"id":"145655","messageId":"20100715191839.GL22574@debian","threadId":"24407","inReplyTo":"20100715190902.GB2774@burratino","subject":"Re: [PATCH 4/8] Add treap implementation","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-07-15T19:18:39Z","receivedAt":"2010-07-15T19:18:39Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi again,\n\nJonathan Nieder writes:\n> I like the documentation updates.  What else changed from the\n> previous round?\n\nNothing else. I should have posted a diff from last time.\n\n-- Ram\n"},{"id":"145656","messageId":"20100715191957.GC2774@burratino","threadId":"24407","inReplyTo":"1279210984-31604-7-git-send-email-artagnon@gmail.com","subject":"Re: [PATCH 6/8] Add stream helper library","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-07-15T19:19:57Z","receivedAt":"2010-07-15T19:19:57Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Ramkumar Ramachandra wrote:\n\n> From: David Barr <david.barr@cordelta.com>\n> \n> This library provides thread-unsafe fgets()- and fread()-like\n> functions where the caller does not have to supply a buffer.  It\n> maintains a couple of static buffers and provides an API to use\n> them.\n> \n> NEEDSWORK: what should buffer_copy_bytes do on error?\n\nFor consistency with the rest of vcs-svn, it should do nothing. :)\n\nI would love to see svn-fe diagnosing and recovering somehow from\nfaulty input.  For now it follows the easier route of just ignoring\n(and skipping) confusing input.\n\nProbably this should be mentioned in the man page somewhere.\n\n[...]\n> +void buffer_copy_bytes(uint32_t len)\n> +{\n[...]\n> +\t\tif (ferror(infile) || ferror(stdout))\n> +\t\t\t/* NEEDSWORK: handle error. */\n\nThe next input/output operation will fail, causing svn-fe to quit\nearly, so it would not be easy for such an error to go unnoticed.\n"},{"id":"145658","messageId":"20100715192839.GD2774@burratino","threadId":"24407","inReplyTo":"1279210984-31604-8-git-send-email-artagnon@gmail.com","subject":"Re: [PATCH 7/8] Add infrastructure to write revisions in fast-export format","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-07-15T19:28:39Z","receivedAt":"2010-07-15T19:28:39Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Ramkumar Ramachandra wrote:\n\n> diff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\n> new file mode 100644\n> index 0000000..7552803\n> --- /dev/null\n> +++ b/vcs-svn/fast_export.c\n> @@ -0,0 +1,75 @@\n[...]\n> +void fast_export_modify(uint32_t depth, uint32_t *path, uint32_t mode,\n> +                        uint32_t mark)\n> +{\n> +\t/* Mode must be 100644, 100755, 120000, or 160000. */\n> +\tprintf(\"M %06o :%d \", mode, mark);\n\nDavid tweaked the API nicely upstream to enforce this constraint.  So\nnice things will come with the next pull from him.\n\n> diff --git a/vcs-svn/fast_export.h b/vcs-svn/fast_export.h\n> new file mode 100644\n> index 0000000..47e8f56\n> --- /dev/null\n> +++ b/vcs-svn/fast_export.h\n> @@ -0,0 +1,14 @@\n> +#ifndef FAST_EXPORT_H_\n> +#define FAST_EXPORT_H_\n> +\n> +#include <stdint.h>\n> +#include <time.h>\n\nThe usual convention within git is to rely on .c files to include\ngit-compat-util.h (indirectly through cache.h or directly).\n\n> +/* Memory pools for commit, dir and dirent */\n> +obj_pool_gen(commit, struct repo_commit, 4096);\n> +obj_pool_gen(dir, struct repo_dir, 4096);\n> +obj_pool_gen(dirent, struct repo_dirent, 4096);\n\nAre the semicolons necessary?  (A nitpick, I know).\n\nThat said, this part is looking pretty good.\n"},{"id":"145660","messageId":"20100715195206.GE2774@burratino","threadId":"24407","inReplyTo":"1279210984-31604-9-git-send-email-artagnon@gmail.com","subject":"Re: [PATCH 8/8] Add SVN dump parser","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-07-15T19:52:06Z","receivedAt":"2010-07-15T19:52:06Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Ramkumar Ramachandra wrote:\n\n> svndump parses data that is in SVN dumpfile format produced by\n> `svnadmin dump` with the help of line_buffer and uses repo_tree and\n> fast_export to emit a git fast-import stream.\n\nProbably worth mentioning the this requires a dumpfile v2 (i.e., it\ndoes not understand the svndiff0 delta format yet).\n\n> +/* Create memory pool for log messages */\n> +obj_pool_gen(log, char, 4096);\n> +\n> +static char* log_copy(uint32_t length, char *log)\n> +{\n> +\tchar *buffer;\n> +\tlog_free(log_pool.size);\n> +\tbuffer = log_pointer(log_alloc(length));\n> +\tstrncpy(buffer, log, length);\n> +\treturn buffer;\n> +}\n\nA strbuf would do just as well.  But using obj_pool looks like a fine\nway to avoid depending on another piece of git code.\n\n> +static struct {\n> +\tuint32_t svn_log, svn_author, svn_date, svn_executable, svn_special, uuid,\n> +\t\trevision_number, node_path, node_kind, node_action,\n> +\t\tnode_copyfrom_path, node_copyfrom_rev, text_content_length,\n> +\t\tprop_content_length, content_length;\n> +} keys;\n\nNeat.  This is a textbook example of where to use a perfect hash, but\ncomparing interned strings is simpler and fast enough (and the\nbottlenecks are elsewhere).\n\n> +static void read_props(void)\n[...]\n> +\t\t\tif (key == keys.svn_log) {\n> +\t\t\t\t/* Value length excludes terminating nul. */\n> +\t\t\t\trev_ctx.log = log_copy(len + 1, val);\n> +\t\t\t} else if (key == keys.svn_author) {\n> +\t\t\t\trev_ctx.author = pool_intern(val);\n> +\t\t\t} else if (key == keys.svn_date) {\n> +\t\t\t\tif (parse_date_basic(val, &rev_ctx.timestamp, NULL))\n> +\t\t\t\t\tfprintf(stderr, \"Invalid timestamp: %s\\n\", val);\n> +\t\t\t} else if (key == keys.svn_executable) {\n> +\t\t\t\tnode_ctx.type = REPO_MODE_EXE;\n> +\t\t\t} else if (key == keys.svn_special) {\n> +\t\t\t\tnode_ctx.type = REPO_MODE_LNK;\n> +\t\t\t}\n> +\t\t\tkey = ~0;\n\nUnknown properties are ignored.  Adding stream comments to allow\nrecovering them is left as an exercise for the interested reader.\n\n> +static void handle_node(void)\n> +{\n\nA simple reader does not cope well with this kind of function.  It is\nhard to know if it exhaustively deals with all cases.\n\nBut: with real-world repos (e.g. ASF) it works well enough.\n\n> +void svndump_read(char *url)\n> +{\n\nToo long.  I realize that writing a state machine can be hard in C;\nmaybe it would be easiest to package up the state in a struct and\nhave a separate function for the main loop body.\n\nThe patches I didn’t comment on all look good.  I don’t think anything I did\ncomment on should prevent this reaching a wider audience.\n\nThanks for the pleasant read,\nJonathan\n"},{"id":"145661","messageId":"20100715200437.GF2774@burratino","threadId":"24407","inReplyTo":"20100715195206.GE2774@burratino","subject":"Re: [PATCH 8/8] Add SVN dump parser","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-07-15T20:04:37Z","receivedAt":"2010-07-15T20:04:37Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Jonathan Nieder wrote:\n\n> I don’t think anything I did\n> comment on should prevent this reaching a wider audience.\n\nExcept for the obj_pool persistence bit, but you already commented on\nthat.\n\nHappy travels,\nJonathan\n"},{"id":"145669","messageId":"7v7hkwuzjd.fsf@alter.siamese.dyndns.org","threadId":"24407","inReplyTo":"20100715172506.GA23989@burratino","subject":"Re: [PATCH 1/8] Export parse_date_basic() to convert a date string to timestamp","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-07-15T22:54:14Z","receivedAt":"2010-07-15T22:54:14Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Nieder <jrnieder@gmail.com> writes:\n\n> Junio: I think this should be ejected from the series as an\n> independently useful cleanup.\n>\n> Currently parse_date_toffset() is exported but not declared anywhere.\n> This patch gives it a more predictable API and adds a declaration.\n\nYeah, that makes sense.  What this patch does seems to be what c5043cc\n(Refactor parse_date for approxidate functions, 2010-06-03) should have\ndone from the beginning.\n\nThanks.\n"},{"id":"145684","messageId":"20100716101352.GA14374@burratino","threadId":"24407","inReplyTo":"1279210984-31604-1-git-send-email-artagnon@gmail.com","subject":"Re: [PATCH 0/8] Resurrect rr/svn-export","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-07-16T10:13:52Z","receivedAt":"2010-07-16T10:13:52Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nWhile Ram travels halfway across the globe, I have rerolled at his\nrequest.  You can find the result at\n  git://repo.or.cz/git/jrn.git rr/svn-fe\n\nThe main difference from the version sent already is the addition of\ntests and a small tweak to the treap implementation to make them\npass.  I will send the two patches with new tests as replies to this\nmessage.  Thoughts welcome, as always.\n\nDavid Barr (5):\n  Add memory pool library\n  vcs-svn: Add string-specific memory pool\n  Add stream helper library\n  Add infrastructure to write revisions in fast-export format\n  Add SVN dump parser\n\nJason Evans (1):\n  Add treap implementation\n\nJonathan Nieder (3):\n  Export parse_date_basic() to convert a date string to timestamp\n  Introduce vcs-svn lib\n  Add a sample user for the svndump library\n\n .gitignore                |    2 +\n Makefile                  |   14 ++-\n cache.h                   |    1 +\n contrib/svn-fe/.gitignore |    4 +\n contrib/svn-fe/Makefile   |   64 +++++++++\n contrib/svn-fe/svn-fe.c   |   15 ++\n contrib/svn-fe/svn-fe.txt |   68 +++++++++\n date.c                    |   14 +-\n t/t0080-vcs-svn.sh        |  101 ++++++++++++++\n test-obj-pool.c           |  116 ++++++++++++++++\n test-treap.c              |   65 +++++++++\n vcs-svn/LICENSE           |   33 +++++\n vcs-svn/fast_export.c     |   75 ++++++++++\n vcs-svn/fast_export.h     |   14 ++\n vcs-svn/line_buffer.c     |   91 +++++++++++++\n vcs-svn/line_buffer.h     |   12 ++\n vcs-svn/obj_pool.h        |   61 +++++++++\n vcs-svn/repo_tree.c       |  331 +++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/repo_tree.h       |   26 ++++\n vcs-svn/string_pool.c     |  101 ++++++++++++++\n vcs-svn/string_pool.h     |   12 ++\n vcs-svn/svndump.c         |  289 +++++++++++++++++++++++++++++++++++++++\n vcs-svn/svndump.h         |    8 +\n vcs-svn/trp.h             |  223 ++++++++++++++++++++++++++++++\n vcs-svn/trp.txt           |   98 +++++++++++++\n 25 files changed, 1829 insertions(+), 9 deletions(-)\n create mode 100644 contrib/svn-fe/.gitignore\n create mode 100644 contrib/svn-fe/Makefile\n create mode 100644 contrib/svn-fe/svn-fe.c\n create mode 100644 contrib/svn-fe/svn-fe.txt\n create mode 100755 t/t0080-vcs-svn.sh\n create mode 100644 test-obj-pool.c\n create mode 100644 test-treap.c\n create mode 100644 vcs-svn/LICENSE\n create mode 100644 vcs-svn/fast_export.c\n create mode 100644 vcs-svn/fast_export.h\n create mode 100644 vcs-svn/line_buffer.c\n create mode 100644 vcs-svn/line_buffer.h\n create mode 100644 vcs-svn/obj_pool.h\n create mode 100644 vcs-svn/repo_tree.c\n create mode 100644 vcs-svn/repo_tree.h\n create mode 100644 vcs-svn/string_pool.c\n create mode 100644 vcs-svn/string_pool.h\n create mode 100644 vcs-svn/svndump.c\n create mode 100644 vcs-svn/svndump.h\n create mode 100644 vcs-svn/trp.h\n create mode 100644 vcs-svn/trp.txt\n\n-- \n1.7.2.rc2\n"},{"id":"145685","messageId":"20100716101606.GB14374@burratino","threadId":"24407","inReplyTo":"20100716101352.GA14374@burratino","subject":"[PATCH 3/9] Add memory pool library","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-07-16T10:16:06Z","receivedAt":"2010-07-16T10:16:06Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"From: David Barr <david.barr@cordelta.com>\n\nAdd a memory pool library implemented using C macros. The\nobj_pool_gen() macro creates a type-specific memory pool.\n\nThe memory pool library is distinguished from the existing specialized\nallocators in alloc.c by using a contiguous block for all allocations.\nThis means that on one hand, long-lived pointers have to be written as\noffsets, since the base address changes as the pool grows, but on the\nother hand, the entire pool can be easily written to the file system.\nThis could allow the memory pool to persist between runs of an\napplication.\n\nFor the svn importer, such a facility is useful because each svn\nrevision can copy trees and files from any previous revision.  The\nrelevant information for all revisions has to persist somehow to\nsupport incremental runs.\n\nSigned-off-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\nStripped out pool_init.  And added tests!  There is not really\nmuch an allocator can do, so it is fun to play around with.\n\n .gitignore         |    1 +\n Makefile           |    4 +-\n t/t0080-vcs-svn.sh |   79 +++++++++++++++++++++++++++++++++++\n test-obj-pool.c    |  116 ++++++++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/obj_pool.h |   61 +++++++++++++++++++++++++++\n 5 files changed, 260 insertions(+), 1 deletions(-)\n create mode 100755 t/t0080-vcs-svn.sh\n create mode 100644 test-obj-pool.c\n create mode 100644 vcs-svn/obj_pool.h\n\ndiff --git a/.gitignore b/.gitignore\nindex 14e2b6b..1e64a6a 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -167,6 +167,7 @@\n /test-genrandom\n /test-index-version\n /test-match-trees\n+/test-obj-pool\n /test-parse-options\n /test-path-utils\n /test-run-command\ndiff --git a/Makefile b/Makefile\nindex d6a779b..3b873cd 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -409,6 +409,7 @@ TEST_PROGRAMS_NEED_X += test-delta\n TEST_PROGRAMS_NEED_X += test-dump-cache-tree\n TEST_PROGRAMS_NEED_X += test-genrandom\n TEST_PROGRAMS_NEED_X += test-match-trees\n+TEST_PROGRAMS_NEED_X += test-obj-pool\n TEST_PROGRAMS_NEED_X += test-parse-options\n TEST_PROGRAMS_NEED_X += test-path-utils\n TEST_PROGRAMS_NEED_X += test-run-command\n@@ -1863,7 +1864,8 @@ xdiff-interface.o $(XDIFF_OBJS): \\\n \txdiff/xinclude.h xdiff/xmacros.h xdiff/xdiff.h xdiff/xtypes.h \\\n \txdiff/xutils.h xdiff/xprepare.h xdiff/xdiffi.h xdiff/xemit.h\n \n-$(VCSSVN_OBJS):\n+$(VCSSVN_OBJS): \\\n+\tvcs-svn/obj_pool.h\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\ndiff --git a/t/t0080-vcs-svn.sh b/t/t0080-vcs-svn.sh\nnew file mode 100755\nindex 0000000..3f29496\n--- /dev/null\n+++ b/t/t0080-vcs-svn.sh\n@@ -0,0 +1,79 @@\n+#!/bin/sh\n+\n+test_description='check infrastructure for svn importer'\n+\n+. ./test-lib.sh\n+uint32_max=4294967295\n+\n+test_expect_success 'obj pool: store data' '\n+\tcat <<-\\EOF >expected &&\n+\t0\n+\t1\n+\tEOF\n+\n+\ttest-obj-pool <<-\\EOF >actual &&\n+\talloc one 16\n+\tset one 13\n+\ttest one 13\n+\treset one\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'obj pool: NULL is offset ~0' '\n+\techo \"$uint32_max\" >expected &&\n+\techo null one | test-obj-pool >actual &&\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'obj pool: out-of-bounds access' '\n+\tcat <<-EOF >expected &&\n+\t0\n+\t0\n+\t$uint32_max\n+\t$uint32_max\n+\t16\n+\t20\n+\t$uint32_max\n+\tEOF\n+\n+\ttest-obj-pool <<-\\EOF >actual &&\n+\talloc one 16\n+\talloc two 16\n+\toffset one 20\n+\toffset two 20\n+\talloc one 5\n+\toffset one 20\n+\tfree one 1\n+\toffset one 20\n+\treset one\n+\treset two\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'obj pool: high-water mark' '\n+\tcat <<-\\EOF >expected &&\n+\t0\n+\t0\n+\t10\n+\t20\n+\t20\n+\t20\n+\tEOF\n+\n+\ttest-obj-pool <<-\\EOF >actual &&\n+\talloc one 10\n+\tcommitted one\n+\talloc one 10\n+\tcommit one\n+\tcommitted one\n+\talloc one 10\n+\tfree one 20\n+\tcommitted one\n+\treset one\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n+test_done\ndiff --git a/test-obj-pool.c b/test-obj-pool.c\nnew file mode 100644\nindex 0000000..5018863\n--- /dev/null\n+++ b/test-obj-pool.c\n@@ -0,0 +1,116 @@\n+/*\n+ * test-obj-pool.c: code to exercise the svn importer's object pool\n+ */\n+\n+#include \"cache.h\"\n+#include \"vcs-svn/obj_pool.h\"\n+\n+enum pool { POOL_ONE, POOL_TWO };\n+obj_pool_gen(one, int, 1)\n+obj_pool_gen(two, int, 4096)\n+\n+static uint32_t strtouint32(const char *s)\n+{\n+\tchar *end;\n+\tuintmax_t n = strtoumax(s, &end, 10);\n+\tif (*s == '\\0' || (*end != '\\n' && *end != '\\0'))\n+\t\tdie(\"invalid offset: %s\", s);\n+\treturn (uint32_t) n;\n+}\n+\n+static void handle_command(const char *command, enum pool pool, const char *arg)\n+{\n+\tswitch (*command) {\n+\tcase 'a':\n+\t\tif (!prefixcmp(command, \"alloc \")) {\n+\t\t\tuint32_t n = strtouint32(arg);\n+\t\t\tprintf(\"%\"PRIu32\"\\n\",\n+\t\t\t\tpool == POOL_ONE ?\n+\t\t\t\tone_alloc(n) : two_alloc(n));\n+\t\t\treturn;\n+\t\t}\n+\tcase 'c':\n+\t\tif (!prefixcmp(command, \"commit \")) {\n+\t\t\tpool == POOL_ONE ? one_commit() : two_commit();\n+\t\t\treturn;\n+\t\t}\n+\t\tif (!prefixcmp(command, \"committed \")) {\n+\t\t\tprintf(\"%\"PRIu32\"\\n\",\n+\t\t\t\tpool == POOL_ONE ?\n+\t\t\t\tone_pool.committed : two_pool.committed);\n+\t\t\treturn;\n+\t\t}\n+\tcase 'f':\n+\t\tif (!prefixcmp(command, \"free \")) {\n+\t\t\tuint32_t n = strtouint32(arg);\n+\t\t\tpool == POOL_ONE ? one_free(n) : two_free(n);\n+\t\t\treturn;\n+\t\t}\n+\tcase 'n':\n+\t\tif (!prefixcmp(command, \"null \")) {\n+\t\t\tprintf(\"%\"PRIu32\"\\n\",\n+\t\t\t\tpool == POOL_ONE ?\n+\t\t\t\tone_offset(NULL) : two_offset(NULL));\n+\t\t\treturn;\n+\t\t}\n+\tcase 'o':\n+\t\tif (!prefixcmp(command, \"offset \")) {\n+\t\t\tuint32_t n = strtouint32(arg);\n+\t\t\tprintf(\"%\"PRIu32\"\\n\",\n+\t\t\t\tpool == POOL_ONE ?\n+\t\t\t\tone_offset(one_pointer(n)) :\n+\t\t\t\ttwo_offset(two_pointer(n)));\n+\t\t\treturn;\n+\t\t}\n+\tcase 'r':\n+\t\tif (!prefixcmp(command, \"reset \")) {\n+\t\t\tpool == POOL_ONE ? one_reset() : two_reset();\n+\t\t\treturn;\n+\t\t}\n+\tcase 's':\n+\t\tif (!prefixcmp(command, \"set \")) {\n+\t\t\tuint32_t n = strtouint32(arg);\n+\t\t\tif (pool == POOL_ONE)\n+\t\t\t\t*one_pointer(n) = 1;\n+\t\t\telse\n+\t\t\t\t*two_pointer(n) = 1;\n+\t\t\treturn;\n+\t\t}\n+\tcase 't':\n+\t\tif (!prefixcmp(command, \"test \")) {\n+\t\t\tuint32_t n = strtouint32(arg);\n+\t\t\tprintf(\"%d\\n\", pool == POOL_ONE ?\n+\t\t\t\t*one_pointer(n) : *two_pointer(n));\n+\t\t\treturn;\n+\t\t}\n+\tdefault:\n+\t\tdie(\"unrecognized command: %s\", command);\n+\t}\n+}\n+\n+static void handle_line(const char *line)\n+{\n+\tconst char *arg = strchr(line, ' ');\n+\tenum pool pool;\n+\n+\tif (arg && !prefixcmp(arg + 1, \"one\"))\n+\t\tpool = POOL_ONE;\n+\telse if (arg && !prefixcmp(arg + 1, \"two\"))\n+\t\tpool = POOL_TWO;\n+\telse\n+\t\tdie(\"no pool specified: %s\", line);\n+\n+\thandle_command(line, pool, arg + strlen(\"one \"));\n+}\n+\n+int main(int argc, char *argv[])\n+{\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tif (argc != 1)\n+\t\tusage(\"test-obj-str < script\");\n+\n+\twhile (strbuf_getline(&sb, stdin, '\\n') != EOF)\n+\t\thandle_line(sb.buf);\n+\tstrbuf_release(&sb);\n+\treturn 0;\n+}\ndiff --git a/vcs-svn/obj_pool.h b/vcs-svn/obj_pool.h\nnew file mode 100644\nindex 0000000..deb6eb8\n--- /dev/null\n+++ b/vcs-svn/obj_pool.h\n@@ -0,0 +1,61 @@\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#ifndef OBJ_POOL_H_\n+#define OBJ_POOL_H_\n+\n+#include \"git-compat-util.h\"\n+\n+#define MAYBE_UNUSED __attribute__((__unused__))\n+\n+#define obj_pool_gen(pre, obj_t, initial_capacity) \\\n+static struct { \\\n+\tuint32_t committed; \\\n+\tuint32_t size; \\\n+\tuint32_t capacity; \\\n+\tobj_t *base; \\\n+} pre##_pool = {0, 0, 0, NULL}; \\\n+static MAYBE_UNUSED uint32_t pre##_alloc(uint32_t count) \\\n+{ \\\n+\tuint32_t offset; \\\n+\tif (pre##_pool.size + count > pre##_pool.capacity) { \\\n+\t\twhile (pre##_pool.size + count > pre##_pool.capacity) \\\n+\t\t\tif (pre##_pool.capacity) \\\n+\t\t\t\tpre##_pool.capacity *= 2; \\\n+\t\t\telse \\\n+\t\t\t\tpre##_pool.capacity = initial_capacity; \\\n+\t\tpre##_pool.base = realloc(pre##_pool.base, \\\n+\t\t\t\t\tpre##_pool.capacity * sizeof(obj_t)); \\\n+\t} \\\n+\toffset = pre##_pool.size; \\\n+\tpre##_pool.size += count; \\\n+\treturn offset; \\\n+} \\\n+static MAYBE_UNUSED void pre##_free(uint32_t count) \\\n+{ \\\n+\tpre##_pool.size -= count; \\\n+} \\\n+static MAYBE_UNUSED uint32_t pre##_offset(obj_t *obj) \\\n+{ \\\n+\treturn obj == NULL ? ~0 : obj - pre##_pool.base; \\\n+} \\\n+static MAYBE_UNUSED obj_t *pre##_pointer(uint32_t offset) \\\n+{ \\\n+\treturn offset >= pre##_pool.size ? NULL : &pre##_pool.base[offset]; \\\n+} \\\n+static MAYBE_UNUSED void pre##_commit(void) \\\n+{ \\\n+\tpre##_pool.committed = pre##_pool.size; \\\n+} \\\n+static MAYBE_UNUSED void pre##_reset(void) \\\n+{ \\\n+\tfree(pre##_pool.base); \\\n+\tpre##_pool.base = NULL; \\\n+\tpre##_pool.size = 0; \\\n+\tpre##_pool.capacity = 0; \\\n+\tpre##_pool.committed = 0; \\\n+}\n+\n+#endif\n-- \n1.7.2.rc2\n"},{"id":"145686","messageId":"20100716102313.GC14374@burratino","threadId":"24407","inReplyTo":"20100716101352.GA14374@burratino","subject":"[PATCH 4/9] Add treap implementation","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-07-16T10:23:13Z","receivedAt":"2010-07-16T10:23:13Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"From: Jason Evans <jasone@canonware.com>\n\nProvide macros to generate a type-specific treap implementation and\nvarious functions to operate on it. It uses obj_pool.h to store memory\nnodes in a treap.  Previously committed nodes are never removed from\nthe pool; after any *_commit operation, it is assumed (correctly, in\nthe case of svn-fast-export) that someone else must care about them.\n\nTreaps provide a memory-efficient binary search tree structure.\nInsertion/deletion/search are about as about as fast in the average\ncase as red-black trees and the chances of worst-case behavior are\nvanishingly small, thanks to (pseudo-)randomness.  The bad worst-case\nbehavior is a small price to pay, given that treaps are much simpler\nto implement.\n\nFrom http://www.canonware.com/download/trp/trp_hash/trp.h\n\n[db: Altered to reference nodes by offset from a common base pointer]\n[db: Bob Jenkins' hashing implementation dropped for Knuth's]\n[db: Methods unnecessary for search and insert dropped]\n\nSigned-off-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\nTweaked treap_search() to always return the node after a missing\nnode, like it is documented to.  For vcs-svn this doesn’t matter\nbut the predictable semantics should make debugging easier.\n\nThe rest of the patches are almost identical to the versions Ram\nsent; see the aforementioned git tree if you are interested in\ntrying them out.  Testing would be quite welcome.\n\n .gitignore         |    1 +\n Makefile           |    3 +-\n t/t0080-vcs-svn.sh |   22 +++++\n test-treap.c       |   65 +++++++++++++++\n vcs-svn/LICENSE    |    3 +\n vcs-svn/trp.h      |  223 ++++++++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/trp.txt    |   98 +++++++++++++++++++++++\n 7 files changed, 414 insertions(+), 1 deletions(-)\n create mode 100644 test-treap.c\n create mode 100644 vcs-svn/trp.h\n create mode 100644 vcs-svn/trp.txt\n\ndiff --git a/.gitignore b/.gitignore\nindex 1e64a6a..af47653 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -173,6 +173,7 @@\n /test-run-command\n /test-sha1\n /test-sigchain\n+/test-treap\n /common-cmds.h\n *.tar.gz\n *.dsc\ndiff --git a/Makefile b/Makefile\nindex 3b873cd..71d77c4 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -415,6 +415,7 @@ TEST_PROGRAMS_NEED_X += test-path-utils\n TEST_PROGRAMS_NEED_X += test-run-command\n TEST_PROGRAMS_NEED_X += test-sha1\n TEST_PROGRAMS_NEED_X += test-sigchain\n+TEST_PROGRAMS_NEED_X += test-treap\n TEST_PROGRAMS_NEED_X += test-index-version\n \n TEST_PROGRAMS = $(patsubst %,%$X,$(TEST_PROGRAMS_NEED_X))\n@@ -1865,7 +1866,7 @@ xdiff-interface.o $(XDIFF_OBJS): \\\n \txdiff/xutils.h xdiff/xprepare.h xdiff/xdiffi.h xdiff/xemit.h\n \n $(VCSSVN_OBJS): \\\n-\tvcs-svn/obj_pool.h\n+\tvcs-svn/obj_pool.h vcs-svn/trp.h\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\ndiff --git a/t/t0080-vcs-svn.sh b/t/t0080-vcs-svn.sh\nindex 3f29496..dd5bab8 100755\n--- a/t/t0080-vcs-svn.sh\n+++ b/t/t0080-vcs-svn.sh\n@@ -76,4 +76,26 @@ test_expect_success 'obj pool: high-water mark' '\n \ttest_cmp expected actual\n '\n \n+test_expect_success 'treap sort' '\n+\tcat <<-\\EOF >unsorted &&\n+\t58\n+\t2\n+\t3\n+\t3\n+\t58\n+\t3\n+\t3\n+\t11\n+\t0\n+\t1\n+\t2\n+\t3\n+\t3\n+\tEOF\n+\tsort -n unsorted >expected &&\n+\n+\ttest-treap <unsorted >actual &&\n+\ttest_cmp expected actual\n+'\n+\n test_done\ndiff --git a/test-treap.c b/test-treap.c\nnew file mode 100644\nindex 0000000..eae7324\n--- /dev/null\n+++ b/test-treap.c\n@@ -0,0 +1,65 @@\n+/*\n+ * test-treap.c: code to exercise the svn importer's treap structure\n+ */\n+\n+#include \"cache.h\"\n+#include \"vcs-svn/obj_pool.h\"\n+#include \"vcs-svn/trp.h\"\n+\n+struct int_node {\n+\tuintmax_t n;\n+\tstruct trp_node children;\n+};\n+\n+obj_pool_gen(node, struct int_node, 3)\n+\n+static int node_cmp(struct int_node *a, struct int_node *b)\n+{\n+\treturn (a->n > b->n) - (a->n < b->n);\n+}\n+\n+trp_gen(static, treap_, struct int_node, children, node, node_cmp)\n+\n+static void strtonode(struct int_node *item, const char *s)\n+{\n+\tchar *end;\n+\titem->n = strtoumax(s, &end, 10);\n+\tif (*s == '\\0' || (*end != '\\n' && *end != '\\0'))\n+\t\tdie(\"invalid integer: %s\", s);\n+}\n+\n+int main(int argc, char *argv[])\n+{\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tstruct trp_root root = { ~0 };\n+\tuint32_t item;\n+\n+\tif (argc != 1)\n+\t\tusage(\"test-treap < ints\");\n+\n+\twhile (strbuf_getline(&sb, stdin, '\\n') != EOF) {\n+\t\titem = node_alloc(1);\n+\t\tstrtonode(node_pointer(item), sb.buf);\n+\t\ttreap_insert(&root, node_pointer(item));\n+\t}\n+\n+\titem = node_offset(treap_first(&root));\n+\twhile (~item) {\n+\t\tuint32_t next;\n+\t\tstruct int_node *tmp = node_pointer(node_alloc(1));\n+\n+\t\ttmp->n = node_pointer(item)->n;\n+\t\tnext = node_offset(treap_next(&root, node_pointer(item)));\n+\n+\t\ttreap_remove(&root, node_pointer(item));\n+\t\titem = node_offset(treap_search(&root, tmp));\n+\n+\t\tif (item != next && (!~item || node_pointer(item)->n != tmp->n))\n+\t\t\tdie(\"found %\"PRIuMAX\" in place of %\"PRIuMAX\"\",\n+\t\t\t\t~item ? node_pointer(item)->n : ~(uintmax_t) 0,\n+\t\t\t\t~next ? node_pointer(next)->n : ~(uintmax_t) 0);\n+\t\tprintf(\"%\"PRIuMAX\"\\n\", tmp->n);\n+\t}\n+\tnode_reset();\n+\treturn 0;\n+}\ndiff --git a/vcs-svn/LICENSE b/vcs-svn/LICENSE\nindex 6e52372..a3d384c 100644\n--- a/vcs-svn/LICENSE\n+++ b/vcs-svn/LICENSE\n@@ -1,6 +1,9 @@\n Copyright (C) 2010 David Barr <david.barr@cordelta.com>.\n All rights reserved.\n \n+Copyright (C) 2008 Jason Evans <jasone@canonware.com>.\n+All rights reserved.\n+\n Redistribution and use in source and binary forms, with or without\n modification, are permitted provided that the following conditions\n are met:\ndiff --git a/vcs-svn/trp.h b/vcs-svn/trp.h\nnew file mode 100644\nindex 0000000..49940cf\n--- /dev/null\n+++ b/vcs-svn/trp.h\n@@ -0,0 +1,223 @@\n+/*\n+ * C macro implementation of treaps.\n+ *\n+ * Usage:\n+ *   #include <stdint.h>\n+ *   #include \"trp.h\"\n+ *   trp_gen(...)\n+ *\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#ifndef TRP_H_\n+#define TRP_H_\n+\n+#define MAYBE_UNUSED __attribute__((__unused__))\n+\n+/* Node structure. */\n+struct trp_node {\n+\tuint32_t trpn_left;\n+\tuint32_t trpn_right;\n+};\n+\n+/* Root structure. */\n+struct trp_root {\n+\tuint32_t trp_root;\n+};\n+\n+/* Pointer/Offset conversion. */\n+#define trpn_pointer(a_base, a_offset) (a_base##_pointer(a_offset))\n+#define trpn_offset(a_base, a_pointer) (a_base##_offset(a_pointer))\n+#define trpn_modify(a_base, a_offset) \\\n+\tdo { \\\n+\t\tif ((a_offset) < a_base##_pool.committed) { \\\n+\t\t\tuint32_t old_offset = (a_offset);\\\n+\t\t\t(a_offset) = a_base##_alloc(1); \\\n+\t\t\t*trpn_pointer(a_base, a_offset) = \\\n+\t\t\t\t*trpn_pointer(a_base, old_offset); \\\n+\t\t} \\\n+\t} while (0);\n+\n+/* Left accessors. */\n+#define trp_left_get(a_base, a_field, a_node) \\\n+\t(trpn_pointer(a_base, a_node)->a_field.trpn_left)\n+#define trp_left_set(a_base, a_field, a_node, a_left) \\\n+\tdo { \\\n+\t\ttrpn_modify(a_base, a_node); \\\n+\t\ttrp_left_get(a_base, a_field, a_node) = (a_left); \\\n+\t} while(0)\n+\n+/* Right accessors. */\n+#define trp_right_get(a_base, a_field, a_node) \\\n+\t(trpn_pointer(a_base, a_node)->a_field.trpn_right)\n+#define trp_right_set(a_base, a_field, a_node, a_right) \\\n+\tdo { \\\n+\t\ttrpn_modify(a_base, a_node); \\\n+\t\ttrp_right_get(a_base, a_field, a_node) = (a_right); \\\n+\t} while(0)\n+\n+/*\n+ * Fibonacci hash function.\n+ * The multiplier is the nearest prime to (2^32 times (√5 - 1)/2).\n+ * See Knuth §6.4: volume 3, 3rd ed, p518.\n+ */\n+#define trpn_hash(a_node) (uint32_t) (2654435761u * (a_node))\n+\n+/* Priority accessors. */\n+#define trp_prio_get(a_node) trpn_hash(a_node)\n+\n+/* Node initializer. */\n+#define trp_node_new(a_base, a_field, a_node) \\\n+\tdo { \\\n+\t\ttrp_left_set(a_base, a_field, (a_node), ~0); \\\n+\t\ttrp_right_set(a_base, a_field, (a_node), ~0); \\\n+\t} while(0)\n+\n+/* Internal utility macros. */\n+#define trpn_first(a_base, a_field, a_root, r_node) \\\n+\tdo { \\\n+\t\t(r_node) = (a_root); \\\n+\t\tif ((r_node) == ~0) \\\n+\t\t\treturn NULL; \\\n+\t\twhile (~trp_left_get(a_base, a_field, (r_node))) \\\n+\t\t\t(r_node) = trp_left_get(a_base, a_field, (r_node)); \\\n+\t} while (0)\n+\n+#define trpn_rotate_left(a_base, a_field, a_node, r_node) \\\n+\tdo { \\\n+\t\t(r_node) = trp_right_get(a_base, a_field, (a_node)); \\\n+\t\ttrp_right_set(a_base, a_field, (a_node), \\\n+\t\t\ttrp_left_get(a_base, a_field, (r_node))); \\\n+\t\ttrp_left_set(a_base, a_field, (r_node), (a_node)); \\\n+\t} while(0)\n+\n+#define trpn_rotate_right(a_base, a_field, a_node, r_node) \\\n+\tdo { \\\n+\t\t(r_node) = trp_left_get(a_base, a_field, (a_node)); \\\n+\t\ttrp_left_set(a_base, a_field, (a_node), \\\n+\t\t\ttrp_right_get(a_base, a_field, (r_node))); \\\n+\t\ttrp_right_set(a_base, a_field, (r_node), (a_node)); \\\n+\t} while(0)\n+\n+#define trp_gen(a_attr, a_pre, a_type, a_field, a_base, a_cmp) \\\n+a_attr a_type MAYBE_UNUSED *a_pre##first(struct trp_root *treap) \\\n+{ \\\n+\tuint32_t ret; \\\n+\ttrpn_first(a_base, a_field, treap->trp_root, ret); \\\n+\treturn trpn_pointer(a_base, ret); \\\n+} \\\n+a_attr a_type MAYBE_UNUSED *a_pre##next(struct trp_root *treap, a_type *node) \\\n+{ \\\n+\tuint32_t ret; \\\n+\tuint32_t offset = trpn_offset(a_base, node); \\\n+\tif (~trp_right_get(a_base, a_field, offset)) { \\\n+\t\ttrpn_first(a_base, a_field, \\\n+\t\t\ttrp_right_get(a_base, a_field, offset), ret); \\\n+\t} else { \\\n+\t\tuint32_t tnode = treap->trp_root; \\\n+\t\tret = ~0; \\\n+\t\twhile (1) { \\\n+\t\t\tint cmp = (a_cmp)(trpn_pointer(a_base, offset), \\\n+\t\t\t\ttrpn_pointer(a_base, tnode)); \\\n+\t\t\tif (cmp < 0) { \\\n+\t\t\t\tret = tnode; \\\n+\t\t\t\ttnode = trp_left_get(a_base, a_field, tnode); \\\n+\t\t\t} else if (cmp > 0) { \\\n+\t\t\t\ttnode = trp_right_get(a_base, a_field, tnode); \\\n+\t\t\t} else { \\\n+\t\t\t\tbreak; \\\n+\t\t\t} \\\n+\t\t} \\\n+\t} \\\n+\treturn trpn_pointer(a_base, ret); \\\n+} \\\n+a_attr a_type MAYBE_UNUSED *a_pre##search(struct trp_root *treap, a_type *key) \\\n+{ \\\n+\tint cmp; \\\n+\tuint32_t ret = treap->trp_root; \\\n+\twhile (~ret && (cmp = (a_cmp)(key, trpn_pointer(a_base,ret)))) { \\\n+\t\tif (cmp < 0) { \\\n+\t\t\tif (!~trp_left_get(a_base, a_field, ret)) \\\n+\t\t\t\tbreak; \\\n+\t\t\tret = trp_left_get(a_base, a_field, ret); \\\n+\t\t} else { \\\n+\t\t\tret = trp_right_get(a_base, a_field, ret); \\\n+\t\t} \\\n+\t} \\\n+\treturn trpn_pointer(a_base, ret); \\\n+} \\\n+a_attr uint32_t MAYBE_UNUSED a_pre##insert_recurse(uint32_t cur_node, uint32_t ins_node) \\\n+{ \\\n+\tif (cur_node == ~0) { \\\n+\t\treturn (ins_node); \\\n+\t} else { \\\n+\t\tuint32_t ret; \\\n+\t\tint cmp = (a_cmp)(trpn_pointer(a_base, ins_node), \\\n+\t\t\t\t\ttrpn_pointer(a_base, cur_node)); \\\n+\t\tif (cmp < 0) { \\\n+\t\t\tuint32_t left = a_pre##insert_recurse( \\\n+\t\t\t\ttrp_left_get(a_base, a_field, cur_node), ins_node); \\\n+\t\t\ttrp_left_set(a_base, a_field, cur_node, left); \\\n+\t\t\tif (trp_prio_get(left) < trp_prio_get(cur_node)) \\\n+\t\t\t\ttrpn_rotate_right(a_base, a_field, cur_node, ret); \\\n+\t\t\telse \\\n+\t\t\t\tret = cur_node; \\\n+\t\t} else { \\\n+\t\t\tuint32_t right = a_pre##insert_recurse( \\\n+\t\t\t\ttrp_right_get(a_base, a_field, cur_node), ins_node); \\\n+\t\t\ttrp_right_set(a_base, a_field, cur_node, right); \\\n+\t\t\tif (trp_prio_get(right) < trp_prio_get(cur_node)) \\\n+\t\t\t\ttrpn_rotate_left(a_base, a_field, cur_node, ret); \\\n+\t\t\telse \\\n+\t\t\t\tret = cur_node; \\\n+\t\t} \\\n+\t\treturn (ret); \\\n+\t} \\\n+} \\\n+a_attr void MAYBE_UNUSED a_pre##insert(struct trp_root *treap, a_type *node) \\\n+{ \\\n+\tuint32_t offset = trpn_offset(a_base, node); \\\n+\ttrp_node_new(a_base, a_field, offset); \\\n+\ttreap->trp_root = a_pre##insert_recurse(treap->trp_root, offset); \\\n+} \\\n+a_attr uint32_t MAYBE_UNUSED a_pre##remove_recurse(uint32_t cur_node, uint32_t rem_node) \\\n+{ \\\n+\tint cmp = a_cmp(trpn_pointer(a_base, rem_node), \\\n+\t\t\ttrpn_pointer(a_base, cur_node)); \\\n+\tif (cmp == 0) { \\\n+\t\tuint32_t ret; \\\n+\t\tuint32_t left = trp_left_get(a_base, a_field, cur_node); \\\n+\t\tuint32_t right = trp_right_get(a_base, a_field, cur_node); \\\n+\t\tif (left == ~0) { \\\n+\t\t\tif (right == ~0) \\\n+\t\t\t\treturn (~0); \\\n+\t\t} else if (right == ~0 || trp_prio_get(left) < trp_prio_get(right)) { \\\n+\t\t\ttrpn_rotate_right(a_base, a_field, cur_node, ret); \\\n+\t\t\tright = a_pre##remove_recurse(cur_node, rem_node); \\\n+\t\t\ttrp_right_set(a_base, a_field, ret, right); \\\n+\t\t\treturn (ret); \\\n+\t\t} \\\n+\t\ttrpn_rotate_left(a_base, a_field, cur_node, ret); \\\n+\t\tleft = a_pre##remove_recurse(cur_node, rem_node); \\\n+\t\ttrp_left_set(a_base, a_field, ret, left); \\\n+\t\treturn (ret); \\\n+\t} else if (cmp < 0) { \\\n+\t\tuint32_t left = a_pre##remove_recurse( \\\n+\t\t\ttrp_left_get(a_base, a_field, cur_node), rem_node); \\\n+\t\ttrp_left_set(a_base, a_field, cur_node, left); \\\n+\t\treturn (cur_node); \\\n+\t} else { \\\n+\t\tuint32_t right = a_pre##remove_recurse( \\\n+\t\t\ttrp_right_get(a_base, a_field, cur_node), rem_node); \\\n+\t\ttrp_right_set(a_base, a_field, cur_node, right); \\\n+\t\treturn (cur_node); \\\n+\t} \\\n+} \\\n+a_attr void MAYBE_UNUSED a_pre##remove(struct trp_root *treap, a_type *node) \\\n+{ \\\n+\ttreap->trp_root = a_pre##remove_recurse(treap->trp_root, \\\n+\t\ttrpn_offset(a_base, node)); \\\n+} \\\n+\n+#endif\ndiff --git a/vcs-svn/trp.txt b/vcs-svn/trp.txt\nnew file mode 100644\nindex 0000000..9247eba\n--- /dev/null\n+++ b/vcs-svn/trp.txt\n@@ -0,0 +1,98 @@\n+Motivation\n+==========\n+\n+Treaps provide a memory-efficient binary search tree structure.\n+Insertion/deletion/search are about as about as fast in the average\n+case as red-black trees and the chances of worst-case behavior are\n+vanishingly small, thanks to (pseudo-)randomness.  The bad worst-case\n+behavior is a small price to pay, given that treaps are much simpler\n+to implement.\n+\n+API\n+===\n+\n+The trp API generates a data structure and functions to handle a\n+large growing set of objects stored in a pool.\n+\n+The caller:\n+\n+. Specifies parameters for the generated functions with the\n+  trp_gen(static, foo_, ...) macro.\n+\n+. Allocates a `struct trp_root` variable and sets it to {~0}.\n+\n+. Adds new nodes to the set using `foo_insert`.\n+\n+. Can find a specific item in the set using `foo_search`.\n+\n+. Can iterate over items in the set using `foo_first` and `foo_next`.\n+\n+. Can remove an item from the set using `foo_remove`.\n+\n+Example:\n+\n+----\n+struct ex_node {\n+\tconst char *s;\n+\tstruct trp_node ex_link;\n+};\n+static struct trp_root ex_base = {~0};\n+obj_pool_gen(ex, struct ex_node, 4096);\n+trp_gen(static, ex_, struct ex_node, ex_link, ex, strcmp)\n+struct ex_node *item;\n+\n+item = ex_pointer(ex_alloc(1));\n+item->s = \"hello\";\n+ex_insert(&ex_base, item);\n+item = ex_pointer(ex_alloc(1));\n+item->s = \"goodbye\";\n+ex_insert(&ex_base, item);\n+for (item = ex_first(&ex_base); item; item = ex_next(&ex_base, item))\n+\tprintf(\"%s\\n\", item->s);\n+----\n+\n+Functions\n+---------\n+\n+trp_gen(attr, foo_, node_type, link_field, pool, cmp)::\n+\n+\tGenerate a type-specific treap implementation.\n++\n+. The storage class for generated functions will be 'attr' (e.g., `static`).\n+. Generated function names are prefixed with 'foo_' (e.g., `treap_`).\n+. Treap nodes will be of type 'node_type' (e.g., `struct treap_node`).\n+  This type must be a struct with at least one `struct trp_node` field\n+  to point to its children.\n+. The field used to access child nodes will be 'link_field'.\n+. All treap nodes must lie in the 'pool' object pool.\n+. Treap nodes must be totally ordered by the 'cmp' relation, with the\n+  following prototype:\n++\n+int (*cmp)(node_type \\*a, node_type \\*b)\n++\n+and returning a value less than, equal to, or greater than zero\n+according to the result of comparison.\n+\n+void foo_insert(struct trp_root *treap, node_type \\*node)::\n+\n+\tInsert node into treap.  If inserted multiple times,\n+\ta node will appear in the treap multiple times.\n+\n+void foo_remove(struct trp_root *treap, node_type \\*node)::\n+\n+\tRemove node from treap.  Caller must ensure node is\n+\tpresent in treap before using this function.\n+\n+node_type *foo_search(struct trp_root \\*treap, node_type \\*key)::\n+\n+\tSearch for a node that matches key.  If no match is found,\n+\treturn what would be key's successor, were key in treap\n+\t(NULL if no successor).\n+\n+node_type *foo_first(struct trp_root \\*treap)::\n+\n+\tFind the first item from the treap, in sorted order.\n+\n+node_type *foo_next(struct trp_root \\*treap, node_type \\*node)::\n+\n+\tFind the next item.\n-- \n1.7.2.rc2\n"},{"id":"145700","messageId":"20100716182619.GA15661@burratino","threadId":"24407","inReplyTo":"20100716102313.GC14374@burratino","subject":"Re: [PATCH 4/9] Add treap implementation","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-07-16T18:26:19Z","receivedAt":"2010-07-16T18:26:19Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Jonathan Nieder wrote:\n\n> Tweaked treap_search() to always return the node after a missing\n> node, like it is documented to.\n\nIn this case, the documentation was wrong.\n\n> For vcs-svn this doesn’t matter\n\nOr rather, it does.  Sorry about that.\n\n-- 8< --\nSubject: vcs-svn: treap_search should return NULL for missing items\n\nIn a misguided attempt to make the code match the documentation,\ncommit 4692f8e7d (Add treap implementation, 2010-07-15) changed\nthe semantics of treap_search to return the /next/ node when a\nnode is missing.\n\nThat is great in some circumstances (and the new tests even rely on\nit), but the rest of vcs-svn relies on treap_search to return\nNULL in that case instead.  The documentation only suggested\notherwise because of a typo.\n\nSo fix it: now treap_search can do what it was always supposed\nto (return NULL on failure) and Jason Evans’s treap_nsearch function\ncan be used to keep the test suite working.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n test-treap.c    |    2 +-\n vcs-svn/trp.h   |   13 +++++++++++++\n vcs-svn/trp.txt |    9 +++++++--\n 3 files changed, 21 insertions(+), 3 deletions(-)\n\ndiff --git a/test-treap.c b/test-treap.c\nindex eae7324..cdba511 100644\n--- a/test-treap.c\n+++ b/test-treap.c\n@@ -52,7 +52,7 @@ int main(int argc, char *argv[])\n \t\tnext = node_offset(treap_next(&root, node_pointer(item)));\n \n \t\ttreap_remove(&root, node_pointer(item));\n-\t\titem = node_offset(treap_search(&root, tmp));\n+\t\titem = node_offset(treap_nsearch(&root, tmp));\n \n \t\tif (item != next && (!~item || node_pointer(item)->n != tmp->n))\n \t\t\tdie(\"found %\"PRIuMAX\" in place of %\"PRIuMAX\"\",\ndiff --git a/vcs-svn/trp.h b/vcs-svn/trp.h\nindex 49940cf..1f5f51f 100644\n--- a/vcs-svn/trp.h\n+++ b/vcs-svn/trp.h\n@@ -138,6 +138,19 @@ a_attr a_type MAYBE_UNUSED *a_pre##search(struct trp_root *treap, a_type *key) \\\n \tuint32_t ret = treap->trp_root; \\\n \twhile (~ret && (cmp = (a_cmp)(key, trpn_pointer(a_base,ret)))) { \\\n \t\tif (cmp < 0) { \\\n+\t\t\tret = trp_left_get(a_base, a_field, ret); \\\n+\t\t} else { \\\n+\t\t\tret = trp_right_get(a_base, a_field, ret); \\\n+\t\t} \\\n+\t} \\\n+\treturn trpn_pointer(a_base, ret); \\\n+} \\\n+a_attr a_type MAYBE_UNUSED *a_pre##nsearch(struct trp_root *treap, a_type *key) \\\n+{ \\\n+\tint cmp; \\\n+\tuint32_t ret = treap->trp_root; \\\n+\twhile (~ret && (cmp = (a_cmp)(key, trpn_pointer(a_base,ret)))) { \\\n+\t\tif (cmp < 0) { \\\n \t\t\tif (!~trp_left_get(a_base, a_field, ret)) \\\n \t\t\t\tbreak; \\\n \t\t\tret = trp_left_get(a_base, a_field, ret); \\\ndiff --git a/vcs-svn/trp.txt b/vcs-svn/trp.txt\nindex 9247eba..eb4c191 100644\n--- a/vcs-svn/trp.txt\n+++ b/vcs-svn/trp.txt\n@@ -86,8 +86,13 @@ void foo_remove(struct trp_root *treap, node_type \\*node)::\n node_type *foo_search(struct trp_root \\*treap, node_type \\*key)::\n \n \tSearch for a node that matches key.  If no match is found,\n-\treturn what would be key's successor, were key in treap\n-\t(NULL if no successor).\n+\tresult is NULL.\n+\n+node_type *foo_nsearch(struct trp_root \\*treap, node_type \\*key)::\n+\n+\tLike `foo_search`, but if if the key is missing return what\n+\twould be key's successor, were key in treap (NULL if no\n+\tsuccessor).\n \n node_type *foo_first(struct trp_root \\*treap)::\n \n-- \n1.7.2.rc2\n"},{"id":"147563","messageId":"20100809215719.GA4203@burratino","threadId":"24407","inReplyTo":"20100716101352.GA14374@burratino","subject":"[PATCH 0/10] rr/svn-export reroll","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-09T21:57:19Z","receivedAt":"2010-08-09T21:57:19Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi!\n\nsvn-fe has some serious changes on the horizon.  As a preparation,\nlet’s round up what we have now.\n\nThe most controversial change is probably the new svn-fe test, which\ntakes about 15 seconds (for the “svnadmin load”, not the svn-fe\nstep :)).  It is in the t9* series, so hopefully that will not\ndissuade people from running the earlier tests.\n\nThe main highlight in the changes is a new\n\n\tInput error\n\nto stderr if a system call failed in reading in the dump file.\nIt still returns status 0 in this and other error situations,\nthough.\n\nBased on maint (for no good reason; that’s just where I tried it).\nIntended to replace rr/svn-export in pu (only if Ram likes it, of\ncourse).\n\nThoughts welcome.\n\nDavid Barr (5):\n  Add memory pool library\n  Add string-specific memory pool\n  Add stream helper library\n  Infrastructure to write revisions in fast-export format\n  SVN dump parser\n\nJason Evans (1):\n  Add treap implementation\n\nJonathan Nieder (4):\n  Export parse_date_basic() to convert a date string to timestamp\n  Introduce vcs-svn lib\n  Update svn-fe manual\n  svn-fe manual: Clarify warning about deltas in dumpfiles\n\n .gitignore                |    5 +\n Makefile                  |   25 +++-\n cache.h                   |    1 +\n contrib/svn-fe/svn-fe.c   |    1 +\n contrib/svn-fe/svn-fe.txt |   19 ++--\n date.c                    |   14 +-\n t/t0080-vcs-svn.sh        |  171 +++++++++++++++++++++++\n t/t9010-svn-fe.sh         |   32 +++++\n test-line-buffer.c        |   46 +++++++\n test-obj-pool.c           |  116 ++++++++++++++++\n test-string-pool.c        |   31 +++++\n test-svn-fe.c             |   18 +++\n test-treap.c              |   65 +++++++++\n vcs-svn/LICENSE           |   33 +++++\n vcs-svn/fast_export.c     |   74 ++++++++++\n vcs-svn/fast_export.h     |   11 ++\n vcs-svn/line_buffer.c     |  102 ++++++++++++++\n vcs-svn/line_buffer.h     |   12 ++\n vcs-svn/line_buffer.txt   |   62 +++++++++\n vcs-svn/obj_pool.h        |   61 +++++++++\n vcs-svn/repo_tree.c       |  328 +++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/repo_tree.h       |   26 ++++\n vcs-svn/string_pool.c     |  102 ++++++++++++++\n vcs-svn/string_pool.h     |   11 ++\n vcs-svn/string_pool.txt   |   43 ++++++\n vcs-svn/svndump.c         |  302 +++++++++++++++++++++++++++++++++++++++++\n vcs-svn/svndump.h         |    9 ++\n vcs-svn/trp.h             |  236 ++++++++++++++++++++++++++++++++\n vcs-svn/trp.txt           |  103 ++++++++++++++\n 29 files changed, 2040 insertions(+), 19 deletions(-)\n create mode 100755 t/t0080-vcs-svn.sh\n create mode 100644 t/t9010-svn-fe.sh\n create mode 100644 test-line-buffer.c\n create mode 100644 test-obj-pool.c\n create mode 100644 test-string-pool.c\n create mode 100644 test-svn-fe.c\n create mode 100644 test-treap.c\n create mode 100644 vcs-svn/LICENSE\n create mode 100644 vcs-svn/fast_export.c\n create mode 100644 vcs-svn/fast_export.h\n create mode 100644 vcs-svn/line_buffer.c\n create mode 100644 vcs-svn/line_buffer.h\n create mode 100644 vcs-svn/line_buffer.txt\n create mode 100644 vcs-svn/obj_pool.h\n create mode 100644 vcs-svn/repo_tree.c\n create mode 100644 vcs-svn/repo_tree.h\n create mode 100644 vcs-svn/string_pool.c\n create mode 100644 vcs-svn/string_pool.h\n create mode 100644 vcs-svn/string_pool.txt\n create mode 100644 vcs-svn/svndump.c\n create mode 100644 vcs-svn/svndump.h\n create mode 100644 vcs-svn/trp.h\n create mode 100644 vcs-svn/trp.txt\n\n-- \n"},{"id":"147565","messageId":"20100809220127.GB4203@burratino","threadId":"24407","inReplyTo":"20100809215719.GA4203@burratino","subject":"[PATCH 01/10] Export parse_date_basic() to convert a date string to timestamp","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-09T22:01:27Z","receivedAt":"2010-08-09T22:01:27Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"approxidate() is not appropriate for reading machine-written dates\nbecause it guesses instead of erroring out on malformed dates.\nparse_date() is less convenient since it returns its output as a\nstring.  So export the underlying function that writes a timestamp.\n\nWhile at it, change the return value to match the usual convention:\nreturn 0 for success and -1 for failure.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\nAcked-by: Ramkumar Ramachandra <artagnon@gmail.com>\n---\nAs before, I think this improves code clarity, independently of\nits use for svn-fe.  So I would not be unhappy if it is applied\nas a separate topic.\n\nNo change from last round.\n\n cache.h |    1 +\n date.c  |   14 ++++++--------\n 2 files changed, 7 insertions(+), 8 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex c9fa3df..68258be 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -811,6 +811,7 @@ const char *show_date_relative(unsigned long time, int tz,\n \t\t\t       char *timebuf,\n \t\t\t       size_t timebuf_size);\n int parse_date(const char *date, char *buf, int bufsize);\n+int parse_date_basic(const char *date, unsigned long *timestamp, int *offset);\n void datestamp(char *buf, int bufsize);\n #define approxidate(s) approxidate_careful((s), NULL)\n unsigned long approxidate_careful(const char *, int *);\ndiff --git a/date.c b/date.c\nindex 3c981f7..00f9eb5 100644\n--- a/date.c\n+++ b/date.c\n@@ -586,7 +586,7 @@ static int date_string(unsigned long date, int offset, char *buf, int len)\n \n /* Gr. strptime is crap for this; it doesn't have a way to require RFC2822\n    (i.e. English) day/month names, and it doesn't work correctly with %z. */\n-int parse_date_toffset(const char *date, unsigned long *timestamp, int *offset)\n+int parse_date_basic(const char *date, unsigned long *timestamp, int *offset)\n {\n \tstruct tm tm;\n \tint tm_gmt;\n@@ -642,17 +642,16 @@ int parse_date_toffset(const char *date, unsigned long *timestamp, int *offset)\n \n \tif (!tm_gmt)\n \t\t*timestamp -= *offset * 60;\n-\treturn 1; /* success */\n+\treturn 0; /* success */\n }\n \n int parse_date(const char *date, char *result, int maxlen)\n {\n \tunsigned long timestamp;\n \tint offset;\n-\tif (parse_date_toffset(date, &timestamp, &offset) > 0)\n-\t\treturn date_string(timestamp, offset, result, maxlen);\n-\telse\n+\tif (parse_date_basic(date, &timestamp, &offset))\n \t\treturn -1;\n+\treturn date_string(timestamp, offset, result, maxlen);\n }\n \n enum date_mode parse_date_format(const char *format)\n@@ -1004,9 +1003,8 @@ unsigned long approxidate_relative(const char *date, const struct timeval *tv)\n \tint offset;\n \tint errors = 0;\n \n-\tif (parse_date_toffset(date, &timestamp, &offset) > 0)\n+\tif (!parse_date_basic(date, &timestamp, &offset))\n \t\treturn timestamp;\n-\n \treturn approxidate_str(date, tv, &errors);\n }\n \n@@ -1019,7 +1017,7 @@ unsigned long approxidate_careful(const char *date, int *error_ret)\n \tif (!error_ret)\n \t\terror_ret = &dummy;\n \n-\tif (parse_date_toffset(date, &timestamp, &offset) > 0) {\n+\tif (!parse_date_basic(date, &timestamp, &offset)) {\n \t\t*error_ret = 0;\n \t\treturn timestamp;\n \t}\n-- \n1.7.2.1.544.ga752d.dirty\n"},{"id":"147566","messageId":"20100809220429.GC4203@burratino","threadId":"24407","inReplyTo":"20100809215719.GA4203@burratino","subject":"[PATCH 02/10] Introduce vcs-svn lib","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-09T22:04:29Z","receivedAt":"2010-08-09T22:04:29Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Teach the build system to build a separate library for the\nupcoming subversion interop support.\n\nThe resulting vcs-svn/lib.a does not contain any code, nor is\nit built during a normal build.  This is just scaffolding for\nlater changes.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\nThis is just for reference; no change from last round.\n\n Makefile        |    8 +++++++-\n vcs-svn/LICENSE |   26 ++++++++++++++++++++++++++\n 2 files changed, 33 insertions(+), 1 deletions(-)\n create mode 100644 vcs-svn/LICENSE\n\ndiff --git a/Makefile b/Makefile\nindex f33648d..71cca35 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -468,6 +468,7 @@ export PYTHON_PATH\n \n LIB_FILE=libgit.a\n XDIFF_LIB=xdiff/lib.a\n+VCSSVN_LIB=vcs-svn/lib.a\n \n LIB_H += advice.h\n LIB_H += archive.h\n@@ -1739,7 +1740,8 @@ ifndef NO_CURL\n endif\n XDIFF_OBJS = xdiff/xdiffi.o xdiff/xprepare.o xdiff/xutils.o xdiff/xemit.o \\\n \txdiff/xmerge.o xdiff/xpatience.o\n-OBJECTS := $(GIT_OBJS) $(XDIFF_OBJS)\n+VCSSVN_OBJS =\n+OBJECTS := $(GIT_OBJS) $(XDIFF_OBJS) $(VCSSVN_OBJS)\n \n dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d)\n dep_dirs := $(addsuffix .depend,$(sort $(dir $(OBJECTS))))\n@@ -1860,6 +1862,8 @@ http.o http-walker.o http-push.o remote-curl.o: http.h\n xdiff-interface.o $(XDIFF_OBJS): \\\n \txdiff/xinclude.h xdiff/xmacros.h xdiff/xdiff.h xdiff/xtypes.h \\\n \txdiff/xutils.h xdiff/xprepare.h xdiff/xdiffi.h xdiff/xemit.h\n+\n+$(VCSSVN_OBJS):\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\n@@ -1908,6 +1912,8 @@ $(LIB_FILE): $(LIB_OBJS)\n $(XDIFF_LIB): $(XDIFF_OBJS)\n \t$(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $(XDIFF_OBJS)\n \n+$(VCSSVN_LIB): $(VCSSVN_OBJS)\n+\t$(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $(VCSSVN_OBJS)\n \n doc:\n \t$(MAKE) -C Documentation all\ndiff --git a/vcs-svn/LICENSE b/vcs-svn/LICENSE\nnew file mode 100644\nindex 0000000..6e52372\n--- /dev/null\n+++ b/vcs-svn/LICENSE\n@@ -0,0 +1,26 @@\n+Copyright (C) 2010 David Barr <david.barr@cordelta.com>.\n+All rights reserved.\n+\n+Redistribution and use in source and binary forms, with or without\n+modification, are permitted provided that the following conditions\n+are met:\n+1. Redistributions of source code must retain the above copyright\n+   notice(s), this list of conditions and the following disclaimer\n+   unmodified other than the allowable addition of one or more\n+   copyright notices.\n+2. Redistributions in binary form must reproduce the above copyright\n+   notice(s), this list of conditions and the following disclaimer in\n+   the documentation and/or other materials provided with the\n+   distribution.\n+\n+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER(S) ``AS IS'' AND ANY\n+EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE\n+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR\n+PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT HOLDER(S) BE\n+LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR\n+CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF\n+SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR\n+BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,\n+WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE\n+OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,\n+EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n-- \n1.7.2.1.544.ga752d.dirty\n"},{"id":"147567","messageId":"20100809221111.GD4203@burratino","threadId":"24407","inReplyTo":"20100809215719.GA4203@burratino","subject":"[PATCH 03/10] Add memory pool library","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-09T22:11:11Z","receivedAt":"2010-08-09T22:11:11Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"From: David Barr <david.barr@cordelta.com>\n\nAdd a memory pool library implemented using C macros. The\nobj_pool_gen() macro creates a type-specific memory pool.\n\nThe memory pool library is distinguished from the existing specialized\nallocators in alloc.c by using a contiguous block for all allocations.\nThis means that on one hand, long-lived pointers have to be written as\noffsets, since the base address changes as the pool grows, but on the\nother hand, the entire pool can be easily written to the file system.\nThis could allow the memory pool to persist between runs of an\napplication.\n\nFor the svn importer, such a facility is useful because each svn\nrevision can copy trees and files from any previous revision.  The\nrelevant information for all revisions has to persist somehow to\nsupport incremental runs.\n\n[rr: minor cleanups]\n[jn: added tests; removed file system backing for now]\n\nSigned-off-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\nThe only change from last round is the notes at the end of the commit\nmessage.  Hopefully David is less likely to be blamed for bugs I\nintroduced this way. :)\n\n .gitignore         |    1 +\n Makefile           |    4 +-\n t/t0080-vcs-svn.sh |   79 +++++++++++++++++++++++++++++++++++\n test-obj-pool.c    |  116 ++++++++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/obj_pool.h |   61 +++++++++++++++++++++++++++\n 5 files changed, 260 insertions(+), 1 deletions(-)\n create mode 100755 t/t0080-vcs-svn.sh\n create mode 100644 test-obj-pool.c\n create mode 100644 vcs-svn/obj_pool.h\n\ndiff --git a/.gitignore b/.gitignore\nindex 14e2b6b..1e64a6a 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -167,6 +167,7 @@\n /test-genrandom\n /test-index-version\n /test-match-trees\n+/test-obj-pool\n /test-parse-options\n /test-path-utils\n /test-run-command\ndiff --git a/Makefile b/Makefile\nindex 71cca35..eb471e7 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -409,6 +409,7 @@ TEST_PROGRAMS_NEED_X += test-delta\n TEST_PROGRAMS_NEED_X += test-dump-cache-tree\n TEST_PROGRAMS_NEED_X += test-genrandom\n TEST_PROGRAMS_NEED_X += test-match-trees\n+TEST_PROGRAMS_NEED_X += test-obj-pool\n TEST_PROGRAMS_NEED_X += test-parse-options\n TEST_PROGRAMS_NEED_X += test-path-utils\n TEST_PROGRAMS_NEED_X += test-run-command\n@@ -1863,7 +1864,8 @@ xdiff-interface.o $(XDIFF_OBJS): \\\n \txdiff/xinclude.h xdiff/xmacros.h xdiff/xdiff.h xdiff/xtypes.h \\\n \txdiff/xutils.h xdiff/xprepare.h xdiff/xdiffi.h xdiff/xemit.h\n \n-$(VCSSVN_OBJS):\n+$(VCSSVN_OBJS): \\\n+\tvcs-svn/obj_pool.h\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\ndiff --git a/t/t0080-vcs-svn.sh b/t/t0080-vcs-svn.sh\nnew file mode 100755\nindex 0000000..3f29496\n--- /dev/null\n+++ b/t/t0080-vcs-svn.sh\n@@ -0,0 +1,79 @@\n+#!/bin/sh\n+\n+test_description='check infrastructure for svn importer'\n+\n+. ./test-lib.sh\n+uint32_max=4294967295\n+\n+test_expect_success 'obj pool: store data' '\n+\tcat <<-\\EOF >expected &&\n+\t0\n+\t1\n+\tEOF\n+\n+\ttest-obj-pool <<-\\EOF >actual &&\n+\talloc one 16\n+\tset one 13\n+\ttest one 13\n+\treset one\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'obj pool: NULL is offset ~0' '\n+\techo \"$uint32_max\" >expected &&\n+\techo null one | test-obj-pool >actual &&\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'obj pool: out-of-bounds access' '\n+\tcat <<-EOF >expected &&\n+\t0\n+\t0\n+\t$uint32_max\n+\t$uint32_max\n+\t16\n+\t20\n+\t$uint32_max\n+\tEOF\n+\n+\ttest-obj-pool <<-\\EOF >actual &&\n+\talloc one 16\n+\talloc two 16\n+\toffset one 20\n+\toffset two 20\n+\talloc one 5\n+\toffset one 20\n+\tfree one 1\n+\toffset one 20\n+\treset one\n+\treset two\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'obj pool: high-water mark' '\n+\tcat <<-\\EOF >expected &&\n+\t0\n+\t0\n+\t10\n+\t20\n+\t20\n+\t20\n+\tEOF\n+\n+\ttest-obj-pool <<-\\EOF >actual &&\n+\talloc one 10\n+\tcommitted one\n+\talloc one 10\n+\tcommit one\n+\tcommitted one\n+\talloc one 10\n+\tfree one 20\n+\tcommitted one\n+\treset one\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n+test_done\ndiff --git a/test-obj-pool.c b/test-obj-pool.c\nnew file mode 100644\nindex 0000000..5018863\n--- /dev/null\n+++ b/test-obj-pool.c\n@@ -0,0 +1,116 @@\n+/*\n+ * test-obj-pool.c: code to exercise the svn importer's object pool\n+ */\n+\n+#include \"cache.h\"\n+#include \"vcs-svn/obj_pool.h\"\n+\n+enum pool { POOL_ONE, POOL_TWO };\n+obj_pool_gen(one, int, 1)\n+obj_pool_gen(two, int, 4096)\n+\n+static uint32_t strtouint32(const char *s)\n+{\n+\tchar *end;\n+\tuintmax_t n = strtoumax(s, &end, 10);\n+\tif (*s == '\\0' || (*end != '\\n' && *end != '\\0'))\n+\t\tdie(\"invalid offset: %s\", s);\n+\treturn (uint32_t) n;\n+}\n+\n+static void handle_command(const char *command, enum pool pool, const char *arg)\n+{\n+\tswitch (*command) {\n+\tcase 'a':\n+\t\tif (!prefixcmp(command, \"alloc \")) {\n+\t\t\tuint32_t n = strtouint32(arg);\n+\t\t\tprintf(\"%\"PRIu32\"\\n\",\n+\t\t\t\tpool == POOL_ONE ?\n+\t\t\t\tone_alloc(n) : two_alloc(n));\n+\t\t\treturn;\n+\t\t}\n+\tcase 'c':\n+\t\tif (!prefixcmp(command, \"commit \")) {\n+\t\t\tpool == POOL_ONE ? one_commit() : two_commit();\n+\t\t\treturn;\n+\t\t}\n+\t\tif (!prefixcmp(command, \"committed \")) {\n+\t\t\tprintf(\"%\"PRIu32\"\\n\",\n+\t\t\t\tpool == POOL_ONE ?\n+\t\t\t\tone_pool.committed : two_pool.committed);\n+\t\t\treturn;\n+\t\t}\n+\tcase 'f':\n+\t\tif (!prefixcmp(command, \"free \")) {\n+\t\t\tuint32_t n = strtouint32(arg);\n+\t\t\tpool == POOL_ONE ? one_free(n) : two_free(n);\n+\t\t\treturn;\n+\t\t}\n+\tcase 'n':\n+\t\tif (!prefixcmp(command, \"null \")) {\n+\t\t\tprintf(\"%\"PRIu32\"\\n\",\n+\t\t\t\tpool == POOL_ONE ?\n+\t\t\t\tone_offset(NULL) : two_offset(NULL));\n+\t\t\treturn;\n+\t\t}\n+\tcase 'o':\n+\t\tif (!prefixcmp(command, \"offset \")) {\n+\t\t\tuint32_t n = strtouint32(arg);\n+\t\t\tprintf(\"%\"PRIu32\"\\n\",\n+\t\t\t\tpool == POOL_ONE ?\n+\t\t\t\tone_offset(one_pointer(n)) :\n+\t\t\t\ttwo_offset(two_pointer(n)));\n+\t\t\treturn;\n+\t\t}\n+\tcase 'r':\n+\t\tif (!prefixcmp(command, \"reset \")) {\n+\t\t\tpool == POOL_ONE ? one_reset() : two_reset();\n+\t\t\treturn;\n+\t\t}\n+\tcase 's':\n+\t\tif (!prefixcmp(command, \"set \")) {\n+\t\t\tuint32_t n = strtouint32(arg);\n+\t\t\tif (pool == POOL_ONE)\n+\t\t\t\t*one_pointer(n) = 1;\n+\t\t\telse\n+\t\t\t\t*two_pointer(n) = 1;\n+\t\t\treturn;\n+\t\t}\n+\tcase 't':\n+\t\tif (!prefixcmp(command, \"test \")) {\n+\t\t\tuint32_t n = strtouint32(arg);\n+\t\t\tprintf(\"%d\\n\", pool == POOL_ONE ?\n+\t\t\t\t*one_pointer(n) : *two_pointer(n));\n+\t\t\treturn;\n+\t\t}\n+\tdefault:\n+\t\tdie(\"unrecognized command: %s\", command);\n+\t}\n+}\n+\n+static void handle_line(const char *line)\n+{\n+\tconst char *arg = strchr(line, ' ');\n+\tenum pool pool;\n+\n+\tif (arg && !prefixcmp(arg + 1, \"one\"))\n+\t\tpool = POOL_ONE;\n+\telse if (arg && !prefixcmp(arg + 1, \"two\"))\n+\t\tpool = POOL_TWO;\n+\telse\n+\t\tdie(\"no pool specified: %s\", line);\n+\n+\thandle_command(line, pool, arg + strlen(\"one \"));\n+}\n+\n+int main(int argc, char *argv[])\n+{\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tif (argc != 1)\n+\t\tusage(\"test-obj-str < script\");\n+\n+\twhile (strbuf_getline(&sb, stdin, '\\n') != EOF)\n+\t\thandle_line(sb.buf);\n+\tstrbuf_release(&sb);\n+\treturn 0;\n+}\ndiff --git a/vcs-svn/obj_pool.h b/vcs-svn/obj_pool.h\nnew file mode 100644\nindex 0000000..deb6eb8\n--- /dev/null\n+++ b/vcs-svn/obj_pool.h\n@@ -0,0 +1,61 @@\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#ifndef OBJ_POOL_H_\n+#define OBJ_POOL_H_\n+\n+#include \"git-compat-util.h\"\n+\n+#define MAYBE_UNUSED __attribute__((__unused__))\n+\n+#define obj_pool_gen(pre, obj_t, initial_capacity) \\\n+static struct { \\\n+\tuint32_t committed; \\\n+\tuint32_t size; \\\n+\tuint32_t capacity; \\\n+\tobj_t *base; \\\n+} pre##_pool = {0, 0, 0, NULL}; \\\n+static MAYBE_UNUSED uint32_t pre##_alloc(uint32_t count) \\\n+{ \\\n+\tuint32_t offset; \\\n+\tif (pre##_pool.size + count > pre##_pool.capacity) { \\\n+\t\twhile (pre##_pool.size + count > pre##_pool.capacity) \\\n+\t\t\tif (pre##_pool.capacity) \\\n+\t\t\t\tpre##_pool.capacity *= 2; \\\n+\t\t\telse \\\n+\t\t\t\tpre##_pool.capacity = initial_capacity; \\\n+\t\tpre##_pool.base = realloc(pre##_pool.base, \\\n+\t\t\t\t\tpre##_pool.capacity * sizeof(obj_t)); \\\n+\t} \\\n+\toffset = pre##_pool.size; \\\n+\tpre##_pool.size += count; \\\n+\treturn offset; \\\n+} \\\n+static MAYBE_UNUSED void pre##_free(uint32_t count) \\\n+{ \\\n+\tpre##_pool.size -= count; \\\n+} \\\n+static MAYBE_UNUSED uint32_t pre##_offset(obj_t *obj) \\\n+{ \\\n+\treturn obj == NULL ? ~0 : obj - pre##_pool.base; \\\n+} \\\n+static MAYBE_UNUSED obj_t *pre##_pointer(uint32_t offset) \\\n+{ \\\n+\treturn offset >= pre##_pool.size ? NULL : &pre##_pool.base[offset]; \\\n+} \\\n+static MAYBE_UNUSED void pre##_commit(void) \\\n+{ \\\n+\tpre##_pool.committed = pre##_pool.size; \\\n+} \\\n+static MAYBE_UNUSED void pre##_reset(void) \\\n+{ \\\n+\tfree(pre##_pool.base); \\\n+\tpre##_pool.base = NULL; \\\n+\tpre##_pool.size = 0; \\\n+\tpre##_pool.capacity = 0; \\\n+\tpre##_pool.committed = 0; \\\n+}\n+\n+#endif\n-- \n1.7.2.1.544.ga752d.dirty\n"},{"id":"147571","messageId":"20100809221734.GE4203@burratino","threadId":"24407","inReplyTo":"20100809215719.GA4203@burratino","subject":"[PATCH 04/10] Add treap implementation","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-09T22:17:34Z","receivedAt":"2010-08-09T22:17:34Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"From: Jason Evans <jasone@canonware.com>\n\nProvide macros to generate a type-specific treap implementation and\nvarious functions to operate on it. It uses obj_pool.h to store memory\nnodes in a treap.  Previously committed nodes are never removed from\nthe pool; after any *_commit operation, it is assumed (correctly, in\nthe case of svn-fast-export) that someone else must care about them.\n\nTreaps provide a memory-efficient binary search tree structure.\nInsertion/deletion/search are about as about as fast in the average\ncase as red-black trees and the chances of worst-case behavior are\nvanishingly small, thanks to (pseudo-)randomness.  The bad worst-case\nbehavior is a small price to pay, given that treaps are much simpler\nto implement.\n\nFrom http://www.canonware.com/download/trp/trp_hash/trp.h\n\n[db: Altered to reference nodes by offset from a common base pointer]\n[db: Bob Jenkins' hashing implementation dropped for Knuth's]\n[db: Methods unnecessary for search and insert dropped]\n[rr: Squelched compiler warnings]\n[db: Added support for immutable treap nodes]\n[jn: Reintroduced treap_nsearch(); with tests]\n\nSigned-off-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\nWith the treap_nsearch() fixup from last time squashed in and some\nmore history in the log message.\n\n .gitignore         |    1 +\n Makefile           |    3 +-\n t/t0080-vcs-svn.sh |   22 +++++\n test-treap.c       |   65 ++++++++++++++\n vcs-svn/LICENSE    |    3 +\n vcs-svn/trp.h      |  236 ++++++++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/trp.txt    |  103 +++++++++++++++++++++++\n 7 files changed, 432 insertions(+), 1 deletions(-)\n create mode 100644 test-treap.c\n create mode 100644 vcs-svn/trp.h\n create mode 100644 vcs-svn/trp.txt\n\ndiff --git a/.gitignore b/.gitignore\nindex 1e64a6a..af47653 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -173,6 +173,7 @@\n /test-run-command\n /test-sha1\n /test-sigchain\n+/test-treap\n /common-cmds.h\n *.tar.gz\n *.dsc\ndiff --git a/Makefile b/Makefile\nindex eb471e7..e7c33ec 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -415,6 +415,7 @@ TEST_PROGRAMS_NEED_X += test-path-utils\n TEST_PROGRAMS_NEED_X += test-run-command\n TEST_PROGRAMS_NEED_X += test-sha1\n TEST_PROGRAMS_NEED_X += test-sigchain\n+TEST_PROGRAMS_NEED_X += test-treap\n TEST_PROGRAMS_NEED_X += test-index-version\n \n TEST_PROGRAMS = $(patsubst %,%$X,$(TEST_PROGRAMS_NEED_X))\n@@ -1865,7 +1866,7 @@ xdiff-interface.o $(XDIFF_OBJS): \\\n \txdiff/xutils.h xdiff/xprepare.h xdiff/xdiffi.h xdiff/xemit.h\n \n $(VCSSVN_OBJS): \\\n-\tvcs-svn/obj_pool.h\n+\tvcs-svn/obj_pool.h vcs-svn/trp.h\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\ndiff --git a/t/t0080-vcs-svn.sh b/t/t0080-vcs-svn.sh\nindex 3f29496..ce02c58 100755\n--- a/t/t0080-vcs-svn.sh\n+++ b/t/t0080-vcs-svn.sh\n@@ -76,4 +76,26 @@ test_expect_success 'obj pool: high-water mark' '\n \ttest_cmp expected actual\n '\n \n+test_expect_success 'treap sort' '\n+\tcat <<-\\EOF >unsorted &&\n+\t68\n+\t12\n+\t13\n+\t13\n+\t68\n+\t13\n+\t13\n+\t21\n+\t10\n+\t11\n+\t12\n+\t13\n+\t13\n+\tEOF\n+\tsort unsorted >expected &&\n+\n+\ttest-treap <unsorted >actual &&\n+\ttest_cmp expected actual\n+'\n+\n test_done\ndiff --git a/test-treap.c b/test-treap.c\nnew file mode 100644\nindex 0000000..cdba511\n--- /dev/null\n+++ b/test-treap.c\n@@ -0,0 +1,65 @@\n+/*\n+ * test-treap.c: code to exercise the svn importer's treap structure\n+ */\n+\n+#include \"cache.h\"\n+#include \"vcs-svn/obj_pool.h\"\n+#include \"vcs-svn/trp.h\"\n+\n+struct int_node {\n+\tuintmax_t n;\n+\tstruct trp_node children;\n+};\n+\n+obj_pool_gen(node, struct int_node, 3)\n+\n+static int node_cmp(struct int_node *a, struct int_node *b)\n+{\n+\treturn (a->n > b->n) - (a->n < b->n);\n+}\n+\n+trp_gen(static, treap_, struct int_node, children, node, node_cmp)\n+\n+static void strtonode(struct int_node *item, const char *s)\n+{\n+\tchar *end;\n+\titem->n = strtoumax(s, &end, 10);\n+\tif (*s == '\\0' || (*end != '\\n' && *end != '\\0'))\n+\t\tdie(\"invalid integer: %s\", s);\n+}\n+\n+int main(int argc, char *argv[])\n+{\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tstruct trp_root root = { ~0 };\n+\tuint32_t item;\n+\n+\tif (argc != 1)\n+\t\tusage(\"test-treap < ints\");\n+\n+\twhile (strbuf_getline(&sb, stdin, '\\n') != EOF) {\n+\t\titem = node_alloc(1);\n+\t\tstrtonode(node_pointer(item), sb.buf);\n+\t\ttreap_insert(&root, node_pointer(item));\n+\t}\n+\n+\titem = node_offset(treap_first(&root));\n+\twhile (~item) {\n+\t\tuint32_t next;\n+\t\tstruct int_node *tmp = node_pointer(node_alloc(1));\n+\n+\t\ttmp->n = node_pointer(item)->n;\n+\t\tnext = node_offset(treap_next(&root, node_pointer(item)));\n+\n+\t\ttreap_remove(&root, node_pointer(item));\n+\t\titem = node_offset(treap_nsearch(&root, tmp));\n+\n+\t\tif (item != next && (!~item || node_pointer(item)->n != tmp->n))\n+\t\t\tdie(\"found %\"PRIuMAX\" in place of %\"PRIuMAX\"\",\n+\t\t\t\t~item ? node_pointer(item)->n : ~(uintmax_t) 0,\n+\t\t\t\t~next ? node_pointer(next)->n : ~(uintmax_t) 0);\n+\t\tprintf(\"%\"PRIuMAX\"\\n\", tmp->n);\n+\t}\n+\tnode_reset();\n+\treturn 0;\n+}\ndiff --git a/vcs-svn/LICENSE b/vcs-svn/LICENSE\nindex 6e52372..a3d384c 100644\n--- a/vcs-svn/LICENSE\n+++ b/vcs-svn/LICENSE\n@@ -1,6 +1,9 @@\n Copyright (C) 2010 David Barr <david.barr@cordelta.com>.\n All rights reserved.\n \n+Copyright (C) 2008 Jason Evans <jasone@canonware.com>.\n+All rights reserved.\n+\n Redistribution and use in source and binary forms, with or without\n modification, are permitted provided that the following conditions\n are met:\ndiff --git a/vcs-svn/trp.h b/vcs-svn/trp.h\nnew file mode 100644\nindex 0000000..1f5f51f\n--- /dev/null\n+++ b/vcs-svn/trp.h\n@@ -0,0 +1,236 @@\n+/*\n+ * C macro implementation of treaps.\n+ *\n+ * Usage:\n+ *   #include <stdint.h>\n+ *   #include \"trp.h\"\n+ *   trp_gen(...)\n+ *\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#ifndef TRP_H_\n+#define TRP_H_\n+\n+#define MAYBE_UNUSED __attribute__((__unused__))\n+\n+/* Node structure. */\n+struct trp_node {\n+\tuint32_t trpn_left;\n+\tuint32_t trpn_right;\n+};\n+\n+/* Root structure. */\n+struct trp_root {\n+\tuint32_t trp_root;\n+};\n+\n+/* Pointer/Offset conversion. */\n+#define trpn_pointer(a_base, a_offset) (a_base##_pointer(a_offset))\n+#define trpn_offset(a_base, a_pointer) (a_base##_offset(a_pointer))\n+#define trpn_modify(a_base, a_offset) \\\n+\tdo { \\\n+\t\tif ((a_offset) < a_base##_pool.committed) { \\\n+\t\t\tuint32_t old_offset = (a_offset);\\\n+\t\t\t(a_offset) = a_base##_alloc(1); \\\n+\t\t\t*trpn_pointer(a_base, a_offset) = \\\n+\t\t\t\t*trpn_pointer(a_base, old_offset); \\\n+\t\t} \\\n+\t} while (0);\n+\n+/* Left accessors. */\n+#define trp_left_get(a_base, a_field, a_node) \\\n+\t(trpn_pointer(a_base, a_node)->a_field.trpn_left)\n+#define trp_left_set(a_base, a_field, a_node, a_left) \\\n+\tdo { \\\n+\t\ttrpn_modify(a_base, a_node); \\\n+\t\ttrp_left_get(a_base, a_field, a_node) = (a_left); \\\n+\t} while(0)\n+\n+/* Right accessors. */\n+#define trp_right_get(a_base, a_field, a_node) \\\n+\t(trpn_pointer(a_base, a_node)->a_field.trpn_right)\n+#define trp_right_set(a_base, a_field, a_node, a_right) \\\n+\tdo { \\\n+\t\ttrpn_modify(a_base, a_node); \\\n+\t\ttrp_right_get(a_base, a_field, a_node) = (a_right); \\\n+\t} while(0)\n+\n+/*\n+ * Fibonacci hash function.\n+ * The multiplier is the nearest prime to (2^32 times (√5 - 1)/2).\n+ * See Knuth §6.4: volume 3, 3rd ed, p518.\n+ */\n+#define trpn_hash(a_node) (uint32_t) (2654435761u * (a_node))\n+\n+/* Priority accessors. */\n+#define trp_prio_get(a_node) trpn_hash(a_node)\n+\n+/* Node initializer. */\n+#define trp_node_new(a_base, a_field, a_node) \\\n+\tdo { \\\n+\t\ttrp_left_set(a_base, a_field, (a_node), ~0); \\\n+\t\ttrp_right_set(a_base, a_field, (a_node), ~0); \\\n+\t} while(0)\n+\n+/* Internal utility macros. */\n+#define trpn_first(a_base, a_field, a_root, r_node) \\\n+\tdo { \\\n+\t\t(r_node) = (a_root); \\\n+\t\tif ((r_node) == ~0) \\\n+\t\t\treturn NULL; \\\n+\t\twhile (~trp_left_get(a_base, a_field, (r_node))) \\\n+\t\t\t(r_node) = trp_left_get(a_base, a_field, (r_node)); \\\n+\t} while (0)\n+\n+#define trpn_rotate_left(a_base, a_field, a_node, r_node) \\\n+\tdo { \\\n+\t\t(r_node) = trp_right_get(a_base, a_field, (a_node)); \\\n+\t\ttrp_right_set(a_base, a_field, (a_node), \\\n+\t\t\ttrp_left_get(a_base, a_field, (r_node))); \\\n+\t\ttrp_left_set(a_base, a_field, (r_node), (a_node)); \\\n+\t} while(0)\n+\n+#define trpn_rotate_right(a_base, a_field, a_node, r_node) \\\n+\tdo { \\\n+\t\t(r_node) = trp_left_get(a_base, a_field, (a_node)); \\\n+\t\ttrp_left_set(a_base, a_field, (a_node), \\\n+\t\t\ttrp_right_get(a_base, a_field, (r_node))); \\\n+\t\ttrp_right_set(a_base, a_field, (r_node), (a_node)); \\\n+\t} while(0)\n+\n+#define trp_gen(a_attr, a_pre, a_type, a_field, a_base, a_cmp) \\\n+a_attr a_type MAYBE_UNUSED *a_pre##first(struct trp_root *treap) \\\n+{ \\\n+\tuint32_t ret; \\\n+\ttrpn_first(a_base, a_field, treap->trp_root, ret); \\\n+\treturn trpn_pointer(a_base, ret); \\\n+} \\\n+a_attr a_type MAYBE_UNUSED *a_pre##next(struct trp_root *treap, a_type *node) \\\n+{ \\\n+\tuint32_t ret; \\\n+\tuint32_t offset = trpn_offset(a_base, node); \\\n+\tif (~trp_right_get(a_base, a_field, offset)) { \\\n+\t\ttrpn_first(a_base, a_field, \\\n+\t\t\ttrp_right_get(a_base, a_field, offset), ret); \\\n+\t} else { \\\n+\t\tuint32_t tnode = treap->trp_root; \\\n+\t\tret = ~0; \\\n+\t\twhile (1) { \\\n+\t\t\tint cmp = (a_cmp)(trpn_pointer(a_base, offset), \\\n+\t\t\t\ttrpn_pointer(a_base, tnode)); \\\n+\t\t\tif (cmp < 0) { \\\n+\t\t\t\tret = tnode; \\\n+\t\t\t\ttnode = trp_left_get(a_base, a_field, tnode); \\\n+\t\t\t} else if (cmp > 0) { \\\n+\t\t\t\ttnode = trp_right_get(a_base, a_field, tnode); \\\n+\t\t\t} else { \\\n+\t\t\t\tbreak; \\\n+\t\t\t} \\\n+\t\t} \\\n+\t} \\\n+\treturn trpn_pointer(a_base, ret); \\\n+} \\\n+a_attr a_type MAYBE_UNUSED *a_pre##search(struct trp_root *treap, a_type *key) \\\n+{ \\\n+\tint cmp; \\\n+\tuint32_t ret = treap->trp_root; \\\n+\twhile (~ret && (cmp = (a_cmp)(key, trpn_pointer(a_base,ret)))) { \\\n+\t\tif (cmp < 0) { \\\n+\t\t\tret = trp_left_get(a_base, a_field, ret); \\\n+\t\t} else { \\\n+\t\t\tret = trp_right_get(a_base, a_field, ret); \\\n+\t\t} \\\n+\t} \\\n+\treturn trpn_pointer(a_base, ret); \\\n+} \\\n+a_attr a_type MAYBE_UNUSED *a_pre##nsearch(struct trp_root *treap, a_type *key) \\\n+{ \\\n+\tint cmp; \\\n+\tuint32_t ret = treap->trp_root; \\\n+\twhile (~ret && (cmp = (a_cmp)(key, trpn_pointer(a_base,ret)))) { \\\n+\t\tif (cmp < 0) { \\\n+\t\t\tif (!~trp_left_get(a_base, a_field, ret)) \\\n+\t\t\t\tbreak; \\\n+\t\t\tret = trp_left_get(a_base, a_field, ret); \\\n+\t\t} else { \\\n+\t\t\tret = trp_right_get(a_base, a_field, ret); \\\n+\t\t} \\\n+\t} \\\n+\treturn trpn_pointer(a_base, ret); \\\n+} \\\n+a_attr uint32_t MAYBE_UNUSED a_pre##insert_recurse(uint32_t cur_node, uint32_t ins_node) \\\n+{ \\\n+\tif (cur_node == ~0) { \\\n+\t\treturn (ins_node); \\\n+\t} else { \\\n+\t\tuint32_t ret; \\\n+\t\tint cmp = (a_cmp)(trpn_pointer(a_base, ins_node), \\\n+\t\t\t\t\ttrpn_pointer(a_base, cur_node)); \\\n+\t\tif (cmp < 0) { \\\n+\t\t\tuint32_t left = a_pre##insert_recurse( \\\n+\t\t\t\ttrp_left_get(a_base, a_field, cur_node), ins_node); \\\n+\t\t\ttrp_left_set(a_base, a_field, cur_node, left); \\\n+\t\t\tif (trp_prio_get(left) < trp_prio_get(cur_node)) \\\n+\t\t\t\ttrpn_rotate_right(a_base, a_field, cur_node, ret); \\\n+\t\t\telse \\\n+\t\t\t\tret = cur_node; \\\n+\t\t} else { \\\n+\t\t\tuint32_t right = a_pre##insert_recurse( \\\n+\t\t\t\ttrp_right_get(a_base, a_field, cur_node), ins_node); \\\n+\t\t\ttrp_right_set(a_base, a_field, cur_node, right); \\\n+\t\t\tif (trp_prio_get(right) < trp_prio_get(cur_node)) \\\n+\t\t\t\ttrpn_rotate_left(a_base, a_field, cur_node, ret); \\\n+\t\t\telse \\\n+\t\t\t\tret = cur_node; \\\n+\t\t} \\\n+\t\treturn (ret); \\\n+\t} \\\n+} \\\n+a_attr void MAYBE_UNUSED a_pre##insert(struct trp_root *treap, a_type *node) \\\n+{ \\\n+\tuint32_t offset = trpn_offset(a_base, node); \\\n+\ttrp_node_new(a_base, a_field, offset); \\\n+\ttreap->trp_root = a_pre##insert_recurse(treap->trp_root, offset); \\\n+} \\\n+a_attr uint32_t MAYBE_UNUSED a_pre##remove_recurse(uint32_t cur_node, uint32_t rem_node) \\\n+{ \\\n+\tint cmp = a_cmp(trpn_pointer(a_base, rem_node), \\\n+\t\t\ttrpn_pointer(a_base, cur_node)); \\\n+\tif (cmp == 0) { \\\n+\t\tuint32_t ret; \\\n+\t\tuint32_t left = trp_left_get(a_base, a_field, cur_node); \\\n+\t\tuint32_t right = trp_right_get(a_base, a_field, cur_node); \\\n+\t\tif (left == ~0) { \\\n+\t\t\tif (right == ~0) \\\n+\t\t\t\treturn (~0); \\\n+\t\t} else if (right == ~0 || trp_prio_get(left) < trp_prio_get(right)) { \\\n+\t\t\ttrpn_rotate_right(a_base, a_field, cur_node, ret); \\\n+\t\t\tright = a_pre##remove_recurse(cur_node, rem_node); \\\n+\t\t\ttrp_right_set(a_base, a_field, ret, right); \\\n+\t\t\treturn (ret); \\\n+\t\t} \\\n+\t\ttrpn_rotate_left(a_base, a_field, cur_node, ret); \\\n+\t\tleft = a_pre##remove_recurse(cur_node, rem_node); \\\n+\t\ttrp_left_set(a_base, a_field, ret, left); \\\n+\t\treturn (ret); \\\n+\t} else if (cmp < 0) { \\\n+\t\tuint32_t left = a_pre##remove_recurse( \\\n+\t\t\ttrp_left_get(a_base, a_field, cur_node), rem_node); \\\n+\t\ttrp_left_set(a_base, a_field, cur_node, left); \\\n+\t\treturn (cur_node); \\\n+\t} else { \\\n+\t\tuint32_t right = a_pre##remove_recurse( \\\n+\t\t\ttrp_right_get(a_base, a_field, cur_node), rem_node); \\\n+\t\ttrp_right_set(a_base, a_field, cur_node, right); \\\n+\t\treturn (cur_node); \\\n+\t} \\\n+} \\\n+a_attr void MAYBE_UNUSED a_pre##remove(struct trp_root *treap, a_type *node) \\\n+{ \\\n+\ttreap->trp_root = a_pre##remove_recurse(treap->trp_root, \\\n+\t\ttrpn_offset(a_base, node)); \\\n+} \\\n+\n+#endif\ndiff --git a/vcs-svn/trp.txt b/vcs-svn/trp.txt\nnew file mode 100644\nindex 0000000..eb4c191\n--- /dev/null\n+++ b/vcs-svn/trp.txt\n@@ -0,0 +1,103 @@\n+Motivation\n+==========\n+\n+Treaps provide a memory-efficient binary search tree structure.\n+Insertion/deletion/search are about as about as fast in the average\n+case as red-black trees and the chances of worst-case behavior are\n+vanishingly small, thanks to (pseudo-)randomness.  The bad worst-case\n+behavior is a small price to pay, given that treaps are much simpler\n+to implement.\n+\n+API\n+===\n+\n+The trp API generates a data structure and functions to handle a\n+large growing set of objects stored in a pool.\n+\n+The caller:\n+\n+. Specifies parameters for the generated functions with the\n+  trp_gen(static, foo_, ...) macro.\n+\n+. Allocates a `struct trp_root` variable and sets it to {~0}.\n+\n+. Adds new nodes to the set using `foo_insert`.\n+\n+. Can find a specific item in the set using `foo_search`.\n+\n+. Can iterate over items in the set using `foo_first` and `foo_next`.\n+\n+. Can remove an item from the set using `foo_remove`.\n+\n+Example:\n+\n+----\n+struct ex_node {\n+\tconst char *s;\n+\tstruct trp_node ex_link;\n+};\n+static struct trp_root ex_base = {~0};\n+obj_pool_gen(ex, struct ex_node, 4096);\n+trp_gen(static, ex_, struct ex_node, ex_link, ex, strcmp)\n+struct ex_node *item;\n+\n+item = ex_pointer(ex_alloc(1));\n+item->s = \"hello\";\n+ex_insert(&ex_base, item);\n+item = ex_pointer(ex_alloc(1));\n+item->s = \"goodbye\";\n+ex_insert(&ex_base, item);\n+for (item = ex_first(&ex_base); item; item = ex_next(&ex_base, item))\n+\tprintf(\"%s\\n\", item->s);\n+----\n+\n+Functions\n+---------\n+\n+trp_gen(attr, foo_, node_type, link_field, pool, cmp)::\n+\n+\tGenerate a type-specific treap implementation.\n++\n+. The storage class for generated functions will be 'attr' (e.g., `static`).\n+. Generated function names are prefixed with 'foo_' (e.g., `treap_`).\n+. Treap nodes will be of type 'node_type' (e.g., `struct treap_node`).\n+  This type must be a struct with at least one `struct trp_node` field\n+  to point to its children.\n+. The field used to access child nodes will be 'link_field'.\n+. All treap nodes must lie in the 'pool' object pool.\n+. Treap nodes must be totally ordered by the 'cmp' relation, with the\n+  following prototype:\n++\n+int (*cmp)(node_type \\*a, node_type \\*b)\n++\n+and returning a value less than, equal to, or greater than zero\n+according to the result of comparison.\n+\n+void foo_insert(struct trp_root *treap, node_type \\*node)::\n+\n+\tInsert node into treap.  If inserted multiple times,\n+\ta node will appear in the treap multiple times.\n+\n+void foo_remove(struct trp_root *treap, node_type \\*node)::\n+\n+\tRemove node from treap.  Caller must ensure node is\n+\tpresent in treap before using this function.\n+\n+node_type *foo_search(struct trp_root \\*treap, node_type \\*key)::\n+\n+\tSearch for a node that matches key.  If no match is found,\n+\tresult is NULL.\n+\n+node_type *foo_nsearch(struct trp_root \\*treap, node_type \\*key)::\n+\n+\tLike `foo_search`, but if if the key is missing return what\n+\twould be key's successor, were key in treap (NULL if no\n+\tsuccessor).\n+\n+node_type *foo_first(struct trp_root \\*treap)::\n+\n+\tFind the first item from the treap, in sorted order.\n+\n+node_type *foo_next(struct trp_root \\*treap, node_type \\*node)::\n+\n+\tFind the next item.\n-- \n1.7.2.1.544.ga752d.dirty\n"},{"id":"147573","messageId":"20100809223442.GA4429@burratino","threadId":"24407","inReplyTo":"20100809215719.GA4203@burratino","subject":"[PATCH 05/10] Add string-specific memory pool","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-09T22:34:42Z","receivedAt":"2010-08-09T22:34:42Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"From: David Barr <david.barr@cordelta.com>\n\nIntern strings so they can be compared by address and stored without\nwasting space.\n\nThis library uses the macros in the obj_pool.h and trp.h to create a\nmemory pool for strings and expose an API for handling them.\n\n[rr: added API docs]\n[jn: with some API simplifications, new documentation and tests]\n\nSigned-off-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\nNew test.  The return value from pool_tok_seq is not checked by\nthe vcs-svn lib but trying to use it in tests revealed it was not\nso intuitive.  pool_tok_seq() was behaving strangely when passed\nan array of size 0; I think there is nothing sane to do in that\ncase --- maybe it should abort().  The API was passing around\nchar * that cannot be modified; changed to const char *.\n\nAnother set of eyes on this would be welcome.\n\n .gitignore              |    1 +\n Makefile                |    9 +++-\n t/t0080-vcs-svn.sh      |   16 +++++++\n test-string-pool.c      |   31 ++++++++++++++\n vcs-svn/string_pool.c   |  102 +++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/string_pool.h   |   11 +++++\n vcs-svn/string_pool.txt |   43 ++++++++++++++++++++\n 7 files changed, 210 insertions(+), 3 deletions(-)\n create mode 100644 test-string-pool.c\n create mode 100644 vcs-svn/string_pool.c\n create mode 100644 vcs-svn/string_pool.h\n create mode 100644 vcs-svn/string_pool.txt\n\ndiff --git a/.gitignore b/.gitignore\nindex af47653..9f109db 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -173,6 +173,7 @@\n /test-run-command\n /test-sha1\n /test-sigchain\n+/test-string-pool\n /test-treap\n /common-cmds.h\n *.tar.gz\ndiff --git a/Makefile b/Makefile\nindex e7c33ec..24103c9 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -415,6 +415,7 @@ TEST_PROGRAMS_NEED_X += test-path-utils\n TEST_PROGRAMS_NEED_X += test-run-command\n TEST_PROGRAMS_NEED_X += test-sha1\n TEST_PROGRAMS_NEED_X += test-sigchain\n+TEST_PROGRAMS_NEED_X += test-string-pool\n TEST_PROGRAMS_NEED_X += test-treap\n TEST_PROGRAMS_NEED_X += test-index-version\n \n@@ -1742,7 +1743,7 @@ ifndef NO_CURL\n endif\n XDIFF_OBJS = xdiff/xdiffi.o xdiff/xprepare.o xdiff/xutils.o xdiff/xemit.o \\\n \txdiff/xmerge.o xdiff/xpatience.o\n-VCSSVN_OBJS =\n+VCSSVN_OBJS = vcs-svn/string_pool.o\n OBJECTS := $(GIT_OBJS) $(XDIFF_OBJS) $(VCSSVN_OBJS)\n \n dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d)\n@@ -1866,7 +1867,7 @@ xdiff-interface.o $(XDIFF_OBJS): \\\n \txdiff/xutils.h xdiff/xprepare.h xdiff/xdiffi.h xdiff/xemit.h\n \n $(VCSSVN_OBJS): \\\n-\tvcs-svn/obj_pool.h vcs-svn/trp.h\n+\tvcs-svn/obj_pool.h vcs-svn/trp.h vcs-svn/string_pool.h\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\n@@ -2017,10 +2018,12 @@ test-delta$X: diff-delta.o patch-delta.o\n \n test-parse-options$X: parse-options.o\n \n+test-string-pool$X: vcs-svn/lib.a\n+\n .PRECIOUS: $(TEST_OBJS)\n \n test-%$X: test-%.o $(GITLIBS)\n-\t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)\n+\t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(filter %.a,$^) $(LIBS)\n \n check-sha1:: test-sha1$X\n \t./test-sha1.sh\ndiff --git a/t/t0080-vcs-svn.sh b/t/t0080-vcs-svn.sh\nindex ce02c58..99a314b 100755\n--- a/t/t0080-vcs-svn.sh\n+++ b/t/t0080-vcs-svn.sh\n@@ -76,6 +76,22 @@ test_expect_success 'obj pool: high-water mark' '\n \ttest_cmp expected actual\n '\n \n+test_expect_success 'string pool' '\n+\techo a does not equal b >expected.differ &&\n+\techo a equals a >expected.match &&\n+\techo equals equals equals >expected.matchmore &&\n+\n+\ttest-string-pool \"a,--b\" >actual.differ &&\n+\ttest-string-pool \"a,a\" >actual.match &&\n+\ttest-string-pool \"equals-equals\" >actual.matchmore &&\n+\ttest_must_fail test-string-pool a,a,a &&\n+\ttest_must_fail test-string-pool a &&\n+\n+\ttest_cmp expected.differ actual.differ &&\n+\ttest_cmp expected.match actual.match &&\n+\ttest_cmp expected.matchmore actual.matchmore\n+'\n+\n test_expect_success 'treap sort' '\n \tcat <<-\\EOF >unsorted &&\n \t68\ndiff --git a/test-string-pool.c b/test-string-pool.c\nnew file mode 100644\nindex 0000000..2adf84b\n--- /dev/null\n+++ b/test-string-pool.c\n@@ -0,0 +1,31 @@\n+/*\n+ * test-string-pool.c: code to exercise the svn importer's string pool\n+ */\n+\n+#include \"git-compat-util.h\"\n+#include \"vcs-svn/string_pool.h\"\n+\n+int main(int argc, char *argv[])\n+{\n+\tconst uint32_t unequal = pool_intern(\"does not equal\");\n+\tconst uint32_t equal = pool_intern(\"equals\");\n+\tuint32_t buf[3];\n+\tuint32_t n;\n+\n+\tif (argc != 2)\n+\t\tusage(\"test-string-pool <string>,<string>\");\n+\n+\tn = pool_tok_seq(3, buf, \",-\", argv[1]);\n+\tif (n >= 3)\n+\t\tdie(\"too many strings\");\n+\tif (n <= 1)\n+\t\tdie(\"too few strings\");\n+\n+\tbuf[2] = buf[1];\n+\tbuf[1] = (buf[0] == buf[2]) ? equal : unequal;\n+\tpool_print_seq(3, buf, ' ', stdout);\n+\tfputc('\\n', stdout);\n+\n+\tpool_reset();\n+\treturn 0;\n+}\ndiff --git a/vcs-svn/string_pool.c b/vcs-svn/string_pool.c\nnew file mode 100644\nindex 0000000..550f0e5\n--- /dev/null\n+++ b/vcs-svn/string_pool.c\n@@ -0,0 +1,102 @@\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"git-compat-util.h\"\n+#include \"trp.h\"\n+#include \"obj_pool.h\"\n+#include \"string_pool.h\"\n+\n+static struct trp_root tree = { ~0 };\n+\n+struct node {\n+\tuint32_t offset;\n+\tstruct trp_node children;\n+};\n+\n+/* Two memory pools: one for struct node, and another for strings */\n+obj_pool_gen(node, struct node, 4096)\n+obj_pool_gen(string, char, 4096)\n+\n+static char *node_value(struct node *node)\n+{\n+\treturn node ? string_pointer(node->offset) : NULL;\n+}\n+\n+static int node_cmp(struct node *a, struct node *b)\n+{\n+\treturn strcmp(node_value(a), node_value(b));\n+}\n+\n+/* Build a Treap from the node structure (a trp_node w/ offset) */\n+trp_gen(static, tree_, struct node, children, node, node_cmp);\n+\n+const char *pool_fetch(uint32_t entry)\n+{\n+\treturn node_value(node_pointer(entry));\n+}\n+\n+uint32_t pool_intern(const char *key)\n+{\n+\t/* Canonicalize key */\n+\tstruct node *match = NULL;\n+\tuint32_t key_len;\n+\tif (key == NULL)\n+\t\treturn ~0;\n+\tkey_len = strlen(key) + 1;\n+\tstruct node *node = node_pointer(node_alloc(1));\n+\tnode->offset = string_alloc(key_len);\n+\tstrcpy(node_value(node), key);\n+\tmatch = tree_search(&tree, node);\n+\tif (!match) {\n+\t\ttree_insert(&tree, node);\n+\t} else {\n+\t\tnode_free(1);\n+\t\tstring_free(key_len);\n+\t\tnode = match;\n+\t}\n+\treturn node_offset(node);\n+}\n+\n+uint32_t pool_tok_r(char *str, const char *delim, char **saveptr)\n+{\n+\tchar *token = strtok_r(str, delim, saveptr);\n+\treturn token ? pool_intern(token) : ~0;\n+}\n+\n+void pool_print_seq(uint32_t len, uint32_t *seq, char delim, FILE *stream)\n+{\n+\tuint32_t i;\n+\tfor (i = 0; i < len && ~seq[i]; i++) {\n+\t\tfputs(pool_fetch(seq[i]), stream);\n+\t\tif (i < len - 1 && ~seq[i + 1])\n+\t\t\tfputc(delim, stream);\n+\t}\n+}\n+\n+uint32_t pool_tok_seq(uint32_t sz, uint32_t *seq, const char *delim, char *str)\n+{\n+\tchar *context = NULL;\n+\tuint32_t token = ~0;\n+\tuint32_t length;\n+\n+\tif (sz == 0)\n+\t\treturn ~0;\n+\tif (str)\n+\t\ttoken = pool_tok_r(str, delim, &context);\n+\tfor (length = 0; length < sz; length++) {\n+\t\tseq[length] = token;\n+\t\tif (token == ~0)\n+\t\t\treturn length;\n+\t\ttoken = pool_tok_r(NULL, delim, &context);\n+\t}\n+\tseq[sz - 1] = ~0;\n+\treturn sz;\n+}\n+\n+void pool_reset(void)\n+{\n+\tnode_reset();\n+\tstring_reset();\n+}\ndiff --git a/vcs-svn/string_pool.h b/vcs-svn/string_pool.h\nnew file mode 100644\nindex 0000000..222fb66\n--- /dev/null\n+++ b/vcs-svn/string_pool.h\n@@ -0,0 +1,11 @@\n+#ifndef STRING_POOL_H_\n+#define STRING_POOL_H_\n+\n+uint32_t pool_intern(const char *key);\n+const char *pool_fetch(uint32_t entry);\n+uint32_t pool_tok_r(char *str, const char *delim, char **saveptr);\n+void pool_print_seq(uint32_t len, uint32_t *seq, char delim, FILE *stream);\n+uint32_t pool_tok_seq(uint32_t sz, uint32_t *seq, const char *delim, char *str);\n+void pool_reset(void);\n+\n+#endif\ndiff --git a/vcs-svn/string_pool.txt b/vcs-svn/string_pool.txt\nnew file mode 100644\nindex 0000000..1b41f15\n--- /dev/null\n+++ b/vcs-svn/string_pool.txt\n@@ -0,0 +1,43 @@\n+string_pool API\n+===============\n+\n+The string_pool API provides facilities for replacing strings\n+with integer keys that can be more easily compared and stored.\n+The facilities are designed so that one could teach Git without\n+too much trouble to store the information needed for these keys to\n+remain valid over multiple executions.\n+\n+Functions\n+---------\n+\n+pool_intern::\n+\tInclude a string in the string pool and get its key.\n+\tIf that string is already in the pool, retrieves its\n+\texisting key.\n+\n+pool_fetch::\n+\tRetrieve the string associated to a given key.\n+\n+pool_tok_r::\n+\tExtract the key of the next token from a string.\n+\tInterface mimics strtok_r.\n+\n+pool_print_seq::\n+\tPrint a sequence of strings named by key to a file, using the\n+\tspecified delimiter to separate them.\n+\n+\tIf NULL (key ~0) appears in the sequence, the sequence ends\n+\tearly.\n+\n+pool_tok_seq::\n+\tSplit a string into tokens, storing the keys of segments\n+\tinto a caller-provided array.\n+\n+\tUnless sz is 0, the array will always be ~0-terminated.\n+\tIf there is not enough room for all the tokens, the\n+\tarray holds as many tokens as fit in the entries before\n+\tthe terminating ~0.  Return value is the index after the\n+\tlast token, or sz if the tokens did not fit.\n+\n+pool_reset::\n+\tDeallocate storage for the string pool.\n-- \n1.7.2.1.544.ga752d.dirty\n"},{"id":"147574","messageId":"20100809223943.GB4429@burratino","threadId":"24407","inReplyTo":"20100809215719.GA4203@burratino","subject":"[PATCH 06/10] Add stream helper library","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-09T22:39:43Z","receivedAt":"2010-08-09T22:39:43Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"From: David Barr <david.barr@cordelta.com>\n\nThis library provides thread-unsafe fgets()- and fread()-like\nfunctions where the caller does not have to supply a buffer.  It\nmaintains a couple of static buffers and provides an API to use\nthem.\n\n[rr: allow input from files other than stdin]\n[jn: with tests, documentation, and error handling improvements]\n\nSigned-off-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\nNew tests and API docs.  The return value from buffer_deinit\ncan be used to check for errors now (I found this useful when\nwriting tests).\n\n .gitignore              |    1 +\n Makefile                |    8 +++-\n t/t0080-vcs-svn.sh      |   54 ++++++++++++++++++++++++++\n test-line-buffer.c      |   46 ++++++++++++++++++++++\n vcs-svn/line_buffer.c   |   97 +++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/line_buffer.h   |   12 ++++++\n vcs-svn/line_buffer.txt |   58 ++++++++++++++++++++++++++++\n 7 files changed, 274 insertions(+), 2 deletions(-)\n create mode 100644 test-line-buffer.c\n create mode 100644 vcs-svn/line_buffer.c\n create mode 100644 vcs-svn/line_buffer.h\n create mode 100644 vcs-svn/line_buffer.txt\n\ndiff --git a/.gitignore b/.gitignore\nindex 9f109db..8c0512e 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -166,6 +166,7 @@\n /test-dump-cache-tree\n /test-genrandom\n /test-index-version\n+/test-line-buffer\n /test-match-trees\n /test-obj-pool\n /test-parse-options\ndiff --git a/Makefile b/Makefile\nindex 24103c9..a76cce5 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -408,6 +408,7 @@ TEST_PROGRAMS_NEED_X += test-date\n TEST_PROGRAMS_NEED_X += test-delta\n TEST_PROGRAMS_NEED_X += test-dump-cache-tree\n TEST_PROGRAMS_NEED_X += test-genrandom\n+TEST_PROGRAMS_NEED_X += test-line-buffer\n TEST_PROGRAMS_NEED_X += test-match-trees\n TEST_PROGRAMS_NEED_X += test-obj-pool\n TEST_PROGRAMS_NEED_X += test-parse-options\n@@ -1743,7 +1744,7 @@ ifndef NO_CURL\n endif\n XDIFF_OBJS = xdiff/xdiffi.o xdiff/xprepare.o xdiff/xutils.o xdiff/xemit.o \\\n \txdiff/xmerge.o xdiff/xpatience.o\n-VCSSVN_OBJS = vcs-svn/string_pool.o\n+VCSSVN_OBJS = vcs-svn/string_pool.o vcs-svn/line_buffer.o\n OBJECTS := $(GIT_OBJS) $(XDIFF_OBJS) $(VCSSVN_OBJS)\n \n dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d)\n@@ -1867,7 +1868,8 @@ xdiff-interface.o $(XDIFF_OBJS): \\\n \txdiff/xutils.h xdiff/xprepare.h xdiff/xdiffi.h xdiff/xemit.h\n \n $(VCSSVN_OBJS): \\\n-\tvcs-svn/obj_pool.h vcs-svn/trp.h vcs-svn/string_pool.h\n+\tvcs-svn/obj_pool.h vcs-svn/trp.h vcs-svn/string_pool.h \\\n+\tvcs-svn/line_buffer.h\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\n@@ -2016,6 +2018,8 @@ test-date$X: date.o ctype.o\n \n test-delta$X: diff-delta.o patch-delta.o\n \n+test-line-buffer$X: vcs-svn/lib.a\n+\n test-parse-options$X: parse-options.o\n \n test-string-pool$X: vcs-svn/lib.a\ndiff --git a/t/t0080-vcs-svn.sh b/t/t0080-vcs-svn.sh\nindex 99a314b..d3225ad 100755\n--- a/t/t0080-vcs-svn.sh\n+++ b/t/t0080-vcs-svn.sh\n@@ -76,6 +76,60 @@ test_expect_success 'obj pool: high-water mark' '\n \ttest_cmp expected actual\n '\n \n+test_expect_success 'line buffer' '\n+\techo HELLO >expected1 &&\n+\tprintf \"%s\\n\" \"\" HELLO >expected2 &&\n+\techo >expected3 &&\n+\tprintf \"%s\\n\" \"\" Q | q_to_nul >expected4 &&\n+\tprintf \"%s\\n\" foo \"\" >expected5 &&\n+\tprintf \"%s\\n\" \"\" foo >expected6 &&\n+\n+\ttest-line-buffer <<-\\EOF >actual1 &&\n+\t5\n+\tHELLO\n+\tEOF\n+\n+\ttest-line-buffer <<-\\EOF >actual2 &&\n+\t0\n+\n+\t5\n+\tHELLO\n+\tEOF\n+\n+\tq_to_nul <<-\\EOF |\n+\t1\n+\tQ\n+\tEOF\n+\ttest-line-buffer >actual3 &&\n+\n+\tq_to_nul <<-\\EOF |\n+\t0\n+\n+\t1\n+\tQ\n+\tEOF\n+\ttest-line-buffer >actual4 &&\n+\n+\ttest-line-buffer <<-\\EOF >actual5 &&\n+\t5\n+\tfoo\n+\tEOF\n+\n+\ttest-line-buffer <<-\\EOF >actual6 &&\n+\t0\n+\n+\t5\n+\tfoo\n+\tEOF\n+\n+\ttest_cmp expected1 actual1 &&\n+\ttest_cmp expected2 actual2 &&\n+\ttest_cmp expected3 actual3 &&\n+\ttest_cmp expected4 actual4 &&\n+\ttest_cmp expected5 actual5 &&\n+\ttest_cmp expected6 actual6\n+'\n+\n test_expect_success 'string pool' '\n \techo a does not equal b >expected.differ &&\n \techo a equals a >expected.match &&\ndiff --git a/test-line-buffer.c b/test-line-buffer.c\nnew file mode 100644\nindex 0000000..c11bf7f\n--- /dev/null\n+++ b/test-line-buffer.c\n@@ -0,0 +1,46 @@\n+/*\n+ * test-line-buffer.c: code to exercise the svn importer's input helper\n+ *\n+ * Input format:\n+ *\tnumber NL\n+ *\t(number bytes) NL\n+ *\tnumber NL\n+ *\t...\n+ */\n+\n+#include \"git-compat-util.h\"\n+#include \"vcs-svn/line_buffer.h\"\n+\n+static uint32_t strtouint32(const char *s)\n+{\n+\tchar *end;\n+\tuintmax_t n = strtoumax(s, &end, 10);\n+\tif (*s == '\\0' || *end != '\\0')\n+\t\tdie(\"invalid count: %s\", s);\n+\treturn (uint32_t) n;\n+}\n+\n+int main(int argc, char *argv[])\n+{\n+\tchar *s;\n+\n+\tif (argc != 1)\n+\t\tusage(\"test-line-buffer < input.txt\");\n+\tif (buffer_init(NULL))\n+\t\tdie_errno(\"open error\");\n+\twhile ((s = buffer_read_line())) {\n+\t\ts = buffer_read_string(strtouint32(s));\n+\t\tfputs(s, stdout);\n+\t\tfputc('\\n', stdout);\n+\t\tbuffer_skip_bytes(1);\n+\t\tif (!(s = buffer_read_line()))\n+\t\t\tbreak;\n+\t\tbuffer_copy_bytes(strtouint32(s) + 1);\n+\t}\n+\tif (buffer_deinit())\n+\t\tdie(\"input error\");\n+\tif (ferror(stdout))\n+\t\tdie(\"output error\");\n+\tbuffer_reset();\n+\treturn 0;\n+}\ndiff --git a/vcs-svn/line_buffer.c b/vcs-svn/line_buffer.c\nnew file mode 100644\nindex 0000000..1543567\n--- /dev/null\n+++ b/vcs-svn/line_buffer.c\n@@ -0,0 +1,97 @@\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"git-compat-util.h\"\n+#include \"line_buffer.h\"\n+#include \"obj_pool.h\"\n+\n+#define LINE_BUFFER_LEN 10000\n+#define COPY_BUFFER_LEN 4096\n+\n+/* Create memory pool for char sequence of known length */\n+obj_pool_gen(blob, char, 4096)\n+\n+static char line_buffer[LINE_BUFFER_LEN];\n+static char byte_buffer[COPY_BUFFER_LEN];\n+static FILE *infile;\n+\n+int buffer_init(const char *filename)\n+{\n+\tinfile = filename ? fopen(filename, \"r\") : stdin;\n+\tif (!infile)\n+\t\treturn -1;\n+\treturn 0;\n+}\n+\n+int buffer_deinit(void)\n+{\n+\tint err;\n+\tif (infile == stdin)\n+\t\treturn ferror(infile);\n+\terr = ferror(infile);\n+\terr |= fclose(infile);\n+\treturn err;\n+}\n+\n+/* Read a line without trailing newline. */\n+char *buffer_read_line(void)\n+{\n+\tchar *end;\n+\tif (!fgets(line_buffer, sizeof(line_buffer), infile))\n+\t\t/* Error or data exhausted. */\n+\t\treturn NULL;\n+\tend = line_buffer + strlen(line_buffer);\n+\tif (end[-1] == '\\n')\n+\t\tend[-1] = '\\0';\n+\telse if (feof(infile))\n+\t\t; /* No newline at end of file.  That's fine. */\n+\telse\n+\t\t/*\n+\t\t * Line was too long.\n+\t\t * There is probably a saner way to deal with this,\n+\t\t * but for now let's return an error.\n+\t\t */\n+\t\treturn NULL;\n+\treturn line_buffer;\n+}\n+\n+char *buffer_read_string(uint32_t len)\n+{\n+\tchar *s;\n+\tblob_free(blob_pool.size);\n+\ts = blob_pointer(blob_alloc(len + 1));\n+\ts[fread(s, 1, len, infile)] = '\\0';\n+\treturn ferror(infile) ? NULL : s;\n+}\n+\n+void buffer_copy_bytes(uint32_t len)\n+{\n+\tuint32_t in;\n+\twhile (len > 0 && !feof(infile) && !ferror(infile)) {\n+\t\tin = len < COPY_BUFFER_LEN ? len : COPY_BUFFER_LEN;\n+\t\tin = fread(byte_buffer, 1, in, infile);\n+\t\tlen -= in;\n+\t\tfwrite(byte_buffer, 1, in, stdout);\n+\t\tif (ferror(stdout)) {\n+\t\t\tbuffer_skip_bytes(len);\n+\t\t\treturn;\n+\t\t}\n+\t}\n+}\n+\n+void buffer_skip_bytes(uint32_t len)\n+{\n+\tuint32_t in;\n+\twhile (len > 0 && !feof(infile) && !ferror(infile)) {\n+\t\tin = len < COPY_BUFFER_LEN ? len : COPY_BUFFER_LEN;\n+\t\tin = fread(byte_buffer, 1, in, infile);\n+\t\tlen -= in;\n+\t}\n+}\n+\n+void buffer_reset(void)\n+{\n+\tblob_reset();\n+}\ndiff --git a/vcs-svn/line_buffer.h b/vcs-svn/line_buffer.h\nnew file mode 100644\nindex 0000000..9c78ae1\n--- /dev/null\n+++ b/vcs-svn/line_buffer.h\n@@ -0,0 +1,12 @@\n+#ifndef LINE_BUFFER_H_\n+#define LINE_BUFFER_H_\n+\n+int buffer_init(const char *filename);\n+int buffer_deinit(void);\n+char *buffer_read_line(void);\n+char *buffer_read_string(uint32_t len);\n+void buffer_copy_bytes(uint32_t len);\n+void buffer_skip_bytes(uint32_t len);\n+void buffer_reset(void);\n+\n+#endif\ndiff --git a/vcs-svn/line_buffer.txt b/vcs-svn/line_buffer.txt\nnew file mode 100644\nindex 0000000..8906fb1\n--- /dev/null\n+++ b/vcs-svn/line_buffer.txt\n@@ -0,0 +1,58 @@\n+line_buffer API\n+===============\n+\n+The line_buffer library provides a convenient interface for\n+mostly-line-oriented input.\n+\n+Each line is not permitted to exceed 10000 bytes.  The provided\n+functions are not thread-safe or async-signal-safe, and like\n+`fgets()`, they generally do not function correctly if interrupted\n+by a signal without SA_RESTART set.\n+\n+Calling sequence\n+----------------\n+\n+The calling program:\n+\n+ - specifies a file to read with `buffer_init`\n+ - processes input with `buffer_read_line`, `buffer_read_string`,\n+   `buffer_skip_bytes`, and `buffer_copy_bytes`\n+ - closes the file with `buffer_deinit`, perhaps to start over and\n+   read another file.\n+\n+Before exiting, the caller can use `buffer_reset` to deallocate\n+resources for the benefit of profiling tools.\n+\n+Functions\n+---------\n+\n+`buffer_init`::\n+\tOpen the named file for input.  If filename is NULL,\n+\tstart reading from stdin.  On failure, returns -1 (with\n+\terrno indicating the nature of the failure).\n+\n+`buffer_deinit`::\n+\tStop reading from the current file (closing it unless\n+\tit was stdin).  Returns nonzero if `fclose` fails or\n+\tthe error indicator was set.\n+\n+`buffer_read_line`::\n+\tRead a line and strip off the trailing newline.\n+\tOn failure or end of file, returns NULL.\n+\n+`buffer_read_string`::\n+\tRead `len` characters of input or up to the end of the\n+\tfile, whichever comes first.  Returns NULL on error.\n+\tReturns whatever characters were read (possibly \"\")\n+\tfor end of file.\n+\n+`buffer_copy_bytes`::\n+\tRead `len` bytes of input and dump them to the standard output\n+\tstream.  Returns early for error or end of file.\n+\n+`buffer_skip_bytes`::\n+\tDiscards `len` bytes from the input stream (stopping early\n+\tif necessary because of an error or eof).\n+\n+`buffer_reset`::\n+\tDeallocates non-static buffers.\n-- \n1.7.2.1.544.ga752d.dirty\n"},{"id":"147576","messageId":"20100809224810.GC4429@burratino","threadId":"24407","inReplyTo":"20100809215719.GA4203@burratino","subject":"[PATCH 07/10] Infrastructure to write revisions in fast-export format","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-09T22:48:10Z","receivedAt":"2010-08-09T22:48:10Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"From: David Barr <david.barr@cordelta.com>\n\nrepo_tree maintains the exporter's state and provides a facility to to\ncall fast_export, which writes objects to stdout suitable for\nconsumption by fast-import.\n\nThe exported functions roughly correspond to Subversion FS operations.\n\n . repo_add, repo_modify, repo_copy, repo_replace, and repo_delete\n   update the current commit, based roughly on the corresponding\n   Subversion FS operation.\n\n . repo_commit calls out to fast_export to write the current commit to\n   the fast-import stream in stdout.\n\n . repo_diff is used by the fast_export module to write the changes\n   for a commit.\n\n . repo_reset erases the exporter's state, so valgrind can be happy.\n\n[rr: squelched compiler warnings]\n[jn: removed support for maintaining state on-disk, though we may\nwant to add it back later]\n\nSigned-off-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\nNo tests for this one, since the next few patches exercise it and this\nis not a general-purpose API.  Relative to last round, the\n`pool_commit` and `commit_commit` calls have been eliminated; unlike\nthe `dir_commit` et al calls, those were only meant for committing\nstate to disk, and the changing high-water mark was not being used.\n\n Makefile              |    5 +-\n vcs-svn/fast_export.c |   74 +++++++++++\n vcs-svn/fast_export.h |   11 ++\n vcs-svn/repo_tree.c   |  328 +++++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/repo_tree.h   |   26 ++++\n 5 files changed, 442 insertions(+), 2 deletions(-)\n create mode 100644 vcs-svn/fast_export.c\n create mode 100644 vcs-svn/fast_export.h\n create mode 100644 vcs-svn/repo_tree.c\n create mode 100644 vcs-svn/repo_tree.h\n\ndiff --git a/Makefile b/Makefile\nindex a76cce5..b873399 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1744,7 +1744,8 @@ ifndef NO_CURL\n endif\n XDIFF_OBJS = xdiff/xdiffi.o xdiff/xprepare.o xdiff/xutils.o xdiff/xemit.o \\\n \txdiff/xmerge.o xdiff/xpatience.o\n-VCSSVN_OBJS = vcs-svn/string_pool.o vcs-svn/line_buffer.o\n+VCSSVN_OBJS = vcs-svn/string_pool.o vcs-svn/line_buffer.o \\\n+\tvcs-svn/repo_tree.o vcs-svn/fast_export.o\n OBJECTS := $(GIT_OBJS) $(XDIFF_OBJS) $(VCSSVN_OBJS)\n \n dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d)\n@@ -1869,7 +1870,7 @@ xdiff-interface.o $(XDIFF_OBJS): \\\n \n $(VCSSVN_OBJS): \\\n \tvcs-svn/obj_pool.h vcs-svn/trp.h vcs-svn/string_pool.h \\\n-\tvcs-svn/line_buffer.h\n+\tvcs-svn/line_buffer.h vcs-svn/repo_tree.h vcs-svn/fast_export.h\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nnew file mode 100644\nindex 0000000..3a6156f\n--- /dev/null\n+++ b/vcs-svn/fast_export.c\n@@ -0,0 +1,74 @@\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"git-compat-util.h\"\n+#include \"fast_export.h\"\n+#include \"line_buffer.h\"\n+#include \"repo_tree.h\"\n+#include \"string_pool.h\"\n+\n+#define MAX_GITSVN_LINE_LEN 4096\n+\n+static uint32_t first_commit_done;\n+\n+void fast_export_delete(uint32_t depth, uint32_t *path)\n+{\n+\tputchar('D');\n+\tputchar(' ');\n+\tpool_print_seq(depth, path, '/', stdout);\n+\tputchar('\\n');\n+}\n+\n+void fast_export_modify(uint32_t depth, uint32_t *path, uint32_t mode,\n+\t\t\tuint32_t mark)\n+{\n+\t/* Mode must be 100644, 100755, 120000, or 160000. */\n+\tprintf(\"M %06o :%d \", mode, mark);\n+\tpool_print_seq(depth, path, '/', stdout);\n+\tputchar('\\n');\n+}\n+\n+static char gitsvnline[MAX_GITSVN_LINE_LEN];\n+void fast_export_commit(uint32_t revision, uint32_t author, char *log,\n+\t\t\tuint32_t uuid, uint32_t url,\n+\t\t\tunsigned long timestamp)\n+{\n+\tif (!log)\n+\t\tlog = \"\";\n+\tif (~uuid && ~url) {\n+\t\tsnprintf(gitsvnline, MAX_GITSVN_LINE_LEN, \"\\n\\ngit-svn-id: %s@%d %s\\n\",\n+\t\t\t\t pool_fetch(url), revision, pool_fetch(uuid));\n+\t} else {\n+\t\t*gitsvnline = '\\0';\n+\t}\n+\tprintf(\"commit refs/heads/master\\n\");\n+\tprintf(\"committer %s <%s@%s> %ld +0000\\n\",\n+\t\t   ~author ? pool_fetch(author) : \"nobody\",\n+\t\t   ~author ? pool_fetch(author) : \"nobody\",\n+\t\t   ~uuid ? pool_fetch(uuid) : \"local\", timestamp);\n+\tprintf(\"data %zd\\n%s%s\\n\",\n+\t\t   strlen(log) + strlen(gitsvnline), log, gitsvnline);\n+\tif (!first_commit_done) {\n+\t\tif (revision > 1)\n+\t\t\tprintf(\"from refs/heads/master^0\\n\");\n+\t\tfirst_commit_done = 1;\n+\t}\n+\trepo_diff(revision - 1, revision);\n+\tfputc('\\n', stdout);\n+\n+\tprintf(\"progress Imported commit %d.\\n\\n\", revision);\n+}\n+\n+void fast_export_blob(uint32_t mode, uint32_t mark, uint32_t len)\n+{\n+\tif (mode == REPO_MODE_LNK) {\n+\t\t/* svn symlink blobs start with \"link \" */\n+\t\tbuffer_skip_bytes(5);\n+\t\tlen -= 5;\n+\t}\n+\tprintf(\"blob\\nmark :%d\\ndata %d\\n\", mark, len);\n+\tbuffer_copy_bytes(len);\n+\tfputc('\\n', stdout);\n+}\ndiff --git a/vcs-svn/fast_export.h b/vcs-svn/fast_export.h\nnew file mode 100644\nindex 0000000..2aaaea5\n--- /dev/null\n+++ b/vcs-svn/fast_export.h\n@@ -0,0 +1,11 @@\n+#ifndef FAST_EXPORT_H_\n+#define FAST_EXPORT_H_\n+\n+void fast_export_delete(uint32_t depth, uint32_t *path);\n+void fast_export_modify(uint32_t depth, uint32_t *path, uint32_t mode,\n+\t\t\tuint32_t mark);\n+void fast_export_commit(uint32_t revision, uint32_t author, char *log,\n+\t\t\tuint32_t uuid, uint32_t url, unsigned long timestamp);\n+void fast_export_blob(uint32_t mode, uint32_t mark, uint32_t len);\n+\n+#endif\ndiff --git a/vcs-svn/repo_tree.c b/vcs-svn/repo_tree.c\nnew file mode 100644\nindex 0000000..ba31e72\n--- /dev/null\n+++ b/vcs-svn/repo_tree.c\n@@ -0,0 +1,328 @@\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"git-compat-util.h\"\n+\n+#include \"string_pool.h\"\n+#include \"repo_tree.h\"\n+#include \"obj_pool.h\"\n+#include \"fast_export.h\"\n+\n+#include \"trp.h\"\n+\n+struct repo_dirent {\n+\tuint32_t name_offset;\n+\tstruct trp_node children;\n+\tuint32_t mode;\n+\tuint32_t content_offset;\n+};\n+\n+struct repo_dir {\n+\tstruct trp_root entries;\n+};\n+\n+struct repo_commit {\n+\tuint32_t root_dir_offset;\n+};\n+\n+/* Memory pools for commit, dir and dirent */\n+obj_pool_gen(commit, struct repo_commit, 4096)\n+obj_pool_gen(dir, struct repo_dir, 4096)\n+obj_pool_gen(dirent, struct repo_dirent, 4096)\n+\n+static uint32_t active_commit;\n+static uint32_t mark;\n+\n+static int repo_dirent_name_cmp(const void *a, const void *b);\n+\n+/* Treap for directory entries */\n+trp_gen(static, dirent_, struct repo_dirent, children, dirent, repo_dirent_name_cmp);\n+\n+uint32_t next_blob_mark(void)\n+{\n+\treturn mark++;\n+}\n+\n+static struct repo_dir *repo_commit_root_dir(struct repo_commit *commit)\n+{\n+\treturn dir_pointer(commit->root_dir_offset);\n+}\n+\n+static struct repo_dirent *repo_first_dirent(struct repo_dir *dir)\n+{\n+\treturn dirent_first(&dir->entries);\n+}\n+\n+static int repo_dirent_name_cmp(const void *a, const void *b)\n+{\n+\tconst struct repo_dirent *dirent1 = a, *dirent2 = b;\n+\tuint32_t a_offset = dirent1->name_offset;\n+\tuint32_t b_offset = dirent2->name_offset;\n+\treturn (a_offset > b_offset) - (a_offset < b_offset);\n+}\n+\n+static int repo_dirent_is_dir(struct repo_dirent *dirent)\n+{\n+\treturn dirent != NULL && dirent->mode == REPO_MODE_DIR;\n+}\n+\n+static struct repo_dir *repo_dir_from_dirent(struct repo_dirent *dirent)\n+{\n+\tif (!repo_dirent_is_dir(dirent))\n+\t\treturn NULL;\n+\treturn dir_pointer(dirent->content_offset);\n+}\n+\n+static struct repo_dir *repo_clone_dir(struct repo_dir *orig_dir)\n+{\n+\tuint32_t orig_o, new_o;\n+\torig_o = dir_offset(orig_dir);\n+\tif (orig_o >= dir_pool.committed)\n+\t\treturn orig_dir;\n+\tnew_o = dir_alloc(1);\n+\torig_dir = dir_pointer(orig_o);\n+\t*dir_pointer(new_o) = *orig_dir;\n+\treturn dir_pointer(new_o);\n+}\n+\n+static struct repo_dirent *repo_read_dirent(uint32_t revision, uint32_t *path)\n+{\n+\tuint32_t name = 0;\n+\tstruct repo_dirent *key = dirent_pointer(dirent_alloc(1));\n+\tstruct repo_dir *dir = NULL;\n+\tstruct repo_dirent *dirent = NULL;\n+\tdir = repo_commit_root_dir(commit_pointer(revision));\n+\twhile (~(name = *path++)) {\n+\t\tkey->name_offset = name;\n+\t\tdirent = dirent_search(&dir->entries, key);\n+\t\tif (dirent == NULL || !repo_dirent_is_dir(dirent))\n+\t\t\tbreak;\n+\t\tdir = repo_dir_from_dirent(dirent);\n+\t}\n+\tdirent_free(1);\n+\treturn dirent;\n+}\n+\n+static void repo_write_dirent(uint32_t *path, uint32_t mode,\n+\t\t\t      uint32_t content_offset, uint32_t del)\n+{\n+\tuint32_t name, revision, dir_o = ~0, parent_dir_o = ~0;\n+\tstruct repo_dir *dir;\n+\tstruct repo_dirent *key;\n+\tstruct repo_dirent *dirent = NULL;\n+\trevision = active_commit;\n+\tdir = repo_commit_root_dir(commit_pointer(revision));\n+\tdir = repo_clone_dir(dir);\n+\tcommit_pointer(revision)->root_dir_offset = dir_offset(dir);\n+\twhile (~(name = *path++)) {\n+\t\tparent_dir_o = dir_offset(dir);\n+\n+\t\tkey = dirent_pointer(dirent_alloc(1));\n+\t\tkey->name_offset = name;\n+\n+\t\tdirent = dirent_search(&dir->entries, key);\n+\t\tif (dirent == NULL)\n+\t\t\tdirent = key;\n+\t\telse\n+\t\t\tdirent_free(1);\n+\n+\t\tif (dirent == key) {\n+\t\t\tdirent->mode = REPO_MODE_DIR;\n+\t\t\tdirent->content_offset = 0;\n+\t\t\tdirent_insert(&dir->entries, dirent);\n+\t\t}\n+\n+\t\tif (dirent_offset(dirent) < dirent_pool.committed) {\n+\t\t\tdir_o = repo_dirent_is_dir(dirent) ?\n+\t\t\t\t\tdirent->content_offset : ~0;\n+\t\t\tdirent_remove(&dir->entries, dirent);\n+\t\t\tdirent = dirent_pointer(dirent_alloc(1));\n+\t\t\tdirent->name_offset = name;\n+\t\t\tdirent->mode = REPO_MODE_DIR;\n+\t\t\tdirent->content_offset = dir_o;\n+\t\t\tdirent_insert(&dir->entries, dirent);\n+\t\t}\n+\n+\t\tdir = repo_dir_from_dirent(dirent);\n+\t\tdir = repo_clone_dir(dir);\n+\t\tdirent->content_offset = dir_offset(dir);\n+\t}\n+\tif (dirent == NULL)\n+\t\treturn;\n+\tdirent->mode = mode;\n+\tdirent->content_offset = content_offset;\n+\tif (del && ~parent_dir_o)\n+\t\tdirent_remove(&dir_pointer(parent_dir_o)->entries, dirent);\n+}\n+\n+uint32_t repo_copy(uint32_t revision, uint32_t *src, uint32_t *dst)\n+{\n+\tuint32_t mode = 0, content_offset = 0;\n+\tstruct repo_dirent *src_dirent;\n+\tsrc_dirent = repo_read_dirent(revision, src);\n+\tif (src_dirent != NULL) {\n+\t\tmode = src_dirent->mode;\n+\t\tcontent_offset = src_dirent->content_offset;\n+\t\trepo_write_dirent(dst, mode, content_offset, 0);\n+\t}\n+\treturn mode;\n+}\n+\n+void repo_add(uint32_t *path, uint32_t mode, uint32_t blob_mark)\n+{\n+\trepo_write_dirent(path, mode, blob_mark, 0);\n+}\n+\n+uint32_t repo_replace(uint32_t *path, uint32_t blob_mark)\n+{\n+\tuint32_t mode = 0;\n+\tstruct repo_dirent *src_dirent;\n+\tsrc_dirent = repo_read_dirent(active_commit, path);\n+\tif (src_dirent != NULL) {\n+\t\tmode = src_dirent->mode;\n+\t\trepo_write_dirent(path, mode, blob_mark, 0);\n+\t}\n+\treturn mode;\n+}\n+\n+void repo_modify(uint32_t *path, uint32_t mode, uint32_t blob_mark)\n+{\n+\tstruct repo_dirent *src_dirent;\n+\tsrc_dirent = repo_read_dirent(active_commit, path);\n+\tif (src_dirent != NULL && blob_mark == 0)\n+\t\tblob_mark = src_dirent->content_offset;\n+\trepo_write_dirent(path, mode, blob_mark, 0);\n+}\n+\n+void repo_delete(uint32_t *path)\n+{\n+\trepo_write_dirent(path, 0, 0, 1);\n+}\n+\n+static void repo_git_add_r(uint32_t depth, uint32_t *path, struct repo_dir *dir);\n+\n+static void repo_git_add(uint32_t depth, uint32_t *path, struct repo_dirent *dirent)\n+{\n+\tif (repo_dirent_is_dir(dirent))\n+\t\trepo_git_add_r(depth, path, repo_dir_from_dirent(dirent));\n+\telse\n+\t\tfast_export_modify(depth, path,\n+\t\t\t\t   dirent->mode, dirent->content_offset);\n+}\n+\n+static void repo_git_add_r(uint32_t depth, uint32_t *path, struct repo_dir *dir)\n+{\n+\tstruct repo_dirent *de = repo_first_dirent(dir);\n+\twhile (de) {\n+\t\tpath[depth] = de->name_offset;\n+\t\trepo_git_add(depth + 1, path, de);\n+\t\tde = dirent_next(&dir->entries, de);\n+\t}\n+}\n+\n+static void repo_diff_r(uint32_t depth, uint32_t *path, struct repo_dir *dir1,\n+\t\t\tstruct repo_dir *dir2)\n+{\n+\tstruct repo_dirent *de1, *de2;\n+\tde1 = repo_first_dirent(dir1);\n+\tde2 = repo_first_dirent(dir2);\n+\n+\twhile (de1 && de2) {\n+\t\tif (de1->name_offset < de2->name_offset) {\n+\t\t\tpath[depth] = de1->name_offset;\n+\t\t\tfast_export_delete(depth + 1, path);\n+\t\t\tde1 = dirent_next(&dir1->entries, de1);\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (de1->name_offset > de2->name_offset) {\n+\t\t\tpath[depth] = de2->name_offset;\n+\t\t\trepo_git_add(depth + 1, path, de2);\n+\t\t\tde2 = dirent_next(&dir2->entries, de2);\n+\t\t\tcontinue;\n+\t\t}\n+\t\tpath[depth] = de1->name_offset;\n+\n+\t\tif (de1->mode == de2->mode &&\n+\t\t    de1->content_offset == de2->content_offset) {\n+\t\t\t; /* No change. */\n+\t\t} else if (repo_dirent_is_dir(de1) && repo_dirent_is_dir(de2)) {\n+\t\t\trepo_diff_r(depth + 1, path,\n+\t\t\t\t    repo_dir_from_dirent(de1),\n+\t\t\t\t    repo_dir_from_dirent(de2));\n+\t\t} else if (!repo_dirent_is_dir(de1) && !repo_dirent_is_dir(de2)) {\n+\t\t\trepo_git_add(depth + 1, path, de2);\n+\t\t} else {\n+\t\t\tfast_export_delete(depth + 1, path);\n+\t\t\trepo_git_add(depth + 1, path, de2);\n+\t\t}\n+\t\tde1 = dirent_next(&dir1->entries, de1);\n+\t\tde2 = dirent_next(&dir2->entries, de2);\n+\t}\n+\twhile (de1) {\n+\t\tpath[depth] = de1->name_offset;\n+\t\tfast_export_delete(depth + 1, path);\n+\t\tde1 = dirent_next(&dir1->entries, de1);\n+\t}\n+\twhile (de2) {\n+\t\tpath[depth] = de2->name_offset;\n+\t\trepo_git_add(depth + 1, path, de2);\n+\t\tde2 = dirent_next(&dir2->entries, de2);\n+\t}\n+}\n+\n+static uint32_t path_stack[REPO_MAX_PATH_DEPTH];\n+\n+void repo_diff(uint32_t r1, uint32_t r2)\n+{\n+\trepo_diff_r(0,\n+\t\t    path_stack,\n+\t\t    repo_commit_root_dir(commit_pointer(r1)),\n+\t\t    repo_commit_root_dir(commit_pointer(r2)));\n+}\n+\n+void repo_commit(uint32_t revision, uint32_t author, char *log, uint32_t uuid,\n+\t\t uint32_t url, unsigned long timestamp)\n+{\n+\tfast_export_commit(revision, author, log, uuid, url, timestamp);\n+\tdirent_commit();\n+\tdir_commit();\n+\tactive_commit = commit_alloc(1);\n+\tcommit_pointer(active_commit)->root_dir_offset =\n+\t\tcommit_pointer(active_commit - 1)->root_dir_offset;\n+}\n+\n+static void mark_init(void)\n+{\n+\tuint32_t i;\n+\tmark = 0;\n+\tfor (i = 0; i < dirent_pool.size; i++)\n+\t\tif (!repo_dirent_is_dir(dirent_pointer(i)) &&\n+\t\t    dirent_pointer(i)->content_offset > mark)\n+\t\t\tmark = dirent_pointer(i)->content_offset;\n+\tmark++;\n+}\n+\n+void repo_init() {\n+\tmark_init();\n+\tif (commit_pool.size == 0) {\n+\t\t/* Create empty tree for commit 0. */\n+\t\tcommit_alloc(1);\n+\t\tcommit_pointer(0)->root_dir_offset = dir_alloc(1);\n+\t\tdir_pointer(0)->entries.trp_root = ~0;\n+\t\tdir_commit();\n+\t}\n+\t/* Preallocate next commit, ready for changes. */\n+\tactive_commit = commit_alloc(1);\n+\tcommit_pointer(active_commit)->root_dir_offset =\n+\t\tcommit_pointer(active_commit - 1)->root_dir_offset;\n+}\n+\n+void repo_reset(void)\n+{\n+\tpool_reset();\n+\tcommit_reset();\n+\tdir_reset();\n+\tdirent_reset();\n+}\ndiff --git a/vcs-svn/repo_tree.h b/vcs-svn/repo_tree.h\nnew file mode 100644\nindex 0000000..5476175\n--- /dev/null\n+++ b/vcs-svn/repo_tree.h\n@@ -0,0 +1,26 @@\n+#ifndef REPO_TREE_H_\n+#define REPO_TREE_H_\n+\n+#include \"git-compat-util.h\"\n+\n+#define REPO_MODE_DIR 0040000\n+#define REPO_MODE_BLB 0100644\n+#define REPO_MODE_EXE 0100755\n+#define REPO_MODE_LNK 0120000\n+\n+#define REPO_MAX_PATH_LEN 4096\n+#define REPO_MAX_PATH_DEPTH 1000\n+\n+uint32_t next_blob_mark(void);\n+uint32_t repo_copy(uint32_t revision, uint32_t *src, uint32_t *dst);\n+void repo_add(uint32_t *path, uint32_t mode, uint32_t blob_mark);\n+uint32_t repo_replace(uint32_t *path, uint32_t blob_mark);\n+void repo_modify(uint32_t *path, uint32_t mode, uint32_t blob_mark);\n+void repo_delete(uint32_t *path);\n+void repo_commit(uint32_t revision, uint32_t author, char *log, uint32_t uuid,\n+\t\t uint32_t url, long unsigned timestamp);\n+void repo_diff(uint32_t r1, uint32_t r2);\n+void repo_init(void);\n+void repo_reset(void);\n+\n+#endif\n-- \n1.7.2.1.544.ga752d.dirty\n"},{"id":"147577","messageId":"20100809225500.GD4429@burratino","threadId":"24407","inReplyTo":"20100809215719.GA4203@burratino","subject":"[PATCH 08/10] SVN dump parser","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-09T22:55:00Z","receivedAt":"2010-08-09T22:55:00Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"From: David Barr <david.barr@cordelta.com>\n\nsvndump parses data that is in SVN dumpfile format produced by\n`svnadmin dump` with the help of line_buffer and uses repo_tree and\nfast_export to emit a git fast-import stream.\n\nBased roughly on com.hydrografix.svndump 0.92 from the SvnToCCase\nproject at <http://svn2cc.sarovar.org/>, by Stefan Hegny and\nothers.\n\n[rr: allow input from files other than stdin]\n[jn: with test, more error reporting]\n\nSigned-off-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\nNew test.  It is slow; work by svn gurus to speed this up would\nbe nice.  The test program is very similar to svn-fe from contrib,\nexcept it exercises Ram’s change to read from a file other than\nstdin.\n\n .gitignore              |    1 +\n Makefile                |    8 +-\n contrib/svn-fe/svn-fe.c |    1 +\n t/t9010-svn-fe.sh       |   32 +++++\n test-svn-fe.c           |   18 +++\n vcs-svn/LICENSE         |    4 +\n vcs-svn/svndump.c       |  302 +++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/svndump.h       |    9 ++\n 8 files changed, 373 insertions(+), 2 deletions(-)\n create mode 100644 t/t9010-svn-fe.sh\n create mode 100644 test-svn-fe.c\n create mode 100644 vcs-svn/svndump.c\n create mode 100644 vcs-svn/svndump.h\n\ndiff --git a/.gitignore b/.gitignore\nindex 8c0512e..258723f 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -175,6 +175,7 @@\n /test-sha1\n /test-sigchain\n /test-string-pool\n+/test-svn-fe\n /test-treap\n /common-cmds.h\n *.tar.gz\ndiff --git a/Makefile b/Makefile\nindex b873399..6228f66 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -417,6 +417,7 @@ TEST_PROGRAMS_NEED_X += test-run-command\n TEST_PROGRAMS_NEED_X += test-sha1\n TEST_PROGRAMS_NEED_X += test-sigchain\n TEST_PROGRAMS_NEED_X += test-string-pool\n+TEST_PROGRAMS_NEED_X += test-svn-fe\n TEST_PROGRAMS_NEED_X += test-treap\n TEST_PROGRAMS_NEED_X += test-index-version\n \n@@ -1745,7 +1746,7 @@ endif\n XDIFF_OBJS = xdiff/xdiffi.o xdiff/xprepare.o xdiff/xutils.o xdiff/xemit.o \\\n \txdiff/xmerge.o xdiff/xpatience.o\n VCSSVN_OBJS = vcs-svn/string_pool.o vcs-svn/line_buffer.o \\\n-\tvcs-svn/repo_tree.o vcs-svn/fast_export.o\n+\tvcs-svn/repo_tree.o vcs-svn/fast_export.o vcs-svn/svndump.o\n OBJECTS := $(GIT_OBJS) $(XDIFF_OBJS) $(VCSSVN_OBJS)\n \n dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d)\n@@ -1870,7 +1871,8 @@ xdiff-interface.o $(XDIFF_OBJS): \\\n \n $(VCSSVN_OBJS): \\\n \tvcs-svn/obj_pool.h vcs-svn/trp.h vcs-svn/string_pool.h \\\n-\tvcs-svn/line_buffer.h vcs-svn/repo_tree.h vcs-svn/fast_export.h\n+\tvcs-svn/line_buffer.h vcs-svn/repo_tree.h vcs-svn/fast_export.h \\\n+\tvcs-svn/svndump.h\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\n@@ -2025,6 +2027,8 @@ test-parse-options$X: parse-options.o\n \n test-string-pool$X: vcs-svn/lib.a\n \n+test-svn-fe$X: vcs-svn/lib.a\n+\n .PRECIOUS: $(TEST_OBJS)\n \n test-%$X: test-%.o $(GITLIBS)\ndiff --git a/contrib/svn-fe/svn-fe.c b/contrib/svn-fe/svn-fe.c\nindex e9b9ba4..a2677b0 100644\n--- a/contrib/svn-fe/svn-fe.c\n+++ b/contrib/svn-fe/svn-fe.c\n@@ -10,6 +10,7 @@ int main(int argc, char **argv)\n {\n \tsvndump_init(NULL);\n \tsvndump_read((argc > 1) ? argv[1] : NULL);\n+\tsvndump_deinit();\n \tsvndump_reset();\n \treturn 0;\n }\ndiff --git a/t/t9010-svn-fe.sh b/t/t9010-svn-fe.sh\nnew file mode 100644\nindex 0000000..bf9bbd6\n--- /dev/null\n+++ b/t/t9010-svn-fe.sh\n@@ -0,0 +1,32 @@\n+#!/bin/sh\n+\n+test_description='check svn dumpfile importer'\n+\n+. ./lib-git-svn.sh\n+\n+test_dump() {\n+\tlabel=$1\n+\tdump=$2\n+\ttest_expect_success \"$dump\" '\n+\t\tsvnadmin create \"$label-svn\" &&\n+\t\tsvnadmin load \"$label-svn\" < \"$TEST_DIRECTORY/$dump\" &&\n+\t\tsvn_cmd export \"file://$(pwd)/$label-svn\" \"$label-svnco\" &&\n+\t\tgit init \"$label-git\" &&\n+\t\ttest-svn-fe \"$TEST_DIRECTORY/$dump\" >\"$label.fe\" &&\n+\t\t(\n+\t\t\tcd \"$label-git\" &&\n+\t\t\tgit fast-import < ../\"$label.fe\"\n+\t\t) &&\n+\t\t(\n+\t\t\tcd \"$label-svnco\" &&\n+\t\t\tgit init &&\n+\t\t\tgit add . &&\n+\t\t\tgit fetch \"../$label-git\" master &&\n+\t\t\tgit diff --exit-code FETCH_HEAD\n+\t\t)\n+\t'\n+}\n+\n+test_dump simple t9111/svnsync.dump\n+\n+test_done\ndiff --git a/test-svn-fe.c b/test-svn-fe.c\nnew file mode 100644\nindex 0000000..616a474\n--- /dev/null\n+++ b/test-svn-fe.c\n@@ -0,0 +1,18 @@\n+/*\n+ * test-svn-fe: Code to exercise the svn import lib\n+ */\n+\n+#include \"git-compat-util.h\"\n+#include \"vcs-svn/svndump.h\"\n+\n+int main(int argc, char *argv[])\n+{\n+\tif (argc != 2)\n+\t\tusage(\"test-svn-fe <file>\");\n+\tsvndump_init(argv[1]);\n+\tsvndump_read(NULL);\n+\tsvndump_deinit();\n+\tsvndump_reset();\n+\treturn 0;\n+}\n+\ndiff --git a/vcs-svn/LICENSE b/vcs-svn/LICENSE\nindex a3d384c..0a5e3c4 100644\n--- a/vcs-svn/LICENSE\n+++ b/vcs-svn/LICENSE\n@@ -4,6 +4,10 @@ All rights reserved.\n Copyright (C) 2008 Jason Evans <jasone@canonware.com>.\n All rights reserved.\n \n+Copyright (C) 2005 Stefan Hegny, hydrografix Consulting GmbH,\n+Frankfurt/Main, Germany\n+and others, see http://svn2cc.sarovar.org\n+\n Redistribution and use in source and binary forms, with or without\n modification, are permitted provided that the following conditions\n are met:\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nnew file mode 100644\nindex 0000000..630eeb5\n--- /dev/null\n+++ b/vcs-svn/svndump.c\n@@ -0,0 +1,302 @@\n+/*\n+ * Parse and rearrange a svnadmin dump.\n+ * Create the dump with:\n+ * svnadmin dump --incremental -r<startrev>:<endrev> <repository> >outfile\n+ *\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"cache.h\"\n+#include \"repo_tree.h\"\n+#include \"fast_export.h\"\n+#include \"line_buffer.h\"\n+#include \"obj_pool.h\"\n+#include \"string_pool.h\"\n+\n+#define NODEACT_REPLACE 4\n+#define NODEACT_DELETE 3\n+#define NODEACT_ADD 2\n+#define NODEACT_CHANGE 1\n+#define NODEACT_UNKNOWN 0\n+\n+#define DUMP_CTX 0\n+#define REV_CTX  1\n+#define NODE_CTX 2\n+\n+#define LENGTH_UNKNOWN (~0)\n+#define DATE_RFC2822_LEN 31\n+\n+/* Create memory pool for log messages */\n+obj_pool_gen(log, char, 4096)\n+\n+static char* log_copy(uint32_t length, char *log)\n+{\n+\tchar *buffer;\n+\tlog_free(log_pool.size);\n+\tbuffer = log_pointer(log_alloc(length));\n+\tstrncpy(buffer, log, length);\n+\treturn buffer;\n+}\n+\n+static struct {\n+\tuint32_t action, propLength, textLength, srcRev, srcMode, mark, type;\n+\tuint32_t src[REPO_MAX_PATH_DEPTH], dst[REPO_MAX_PATH_DEPTH];\n+} node_ctx;\n+\n+static struct {\n+\tuint32_t revision, author;\n+\tunsigned long timestamp;\n+\tchar *log;\n+} rev_ctx;\n+\n+static struct {\n+\tuint32_t uuid, url;\n+} dump_ctx;\n+\n+static struct {\n+\tuint32_t svn_log, svn_author, svn_date, svn_executable, svn_special, uuid,\n+\t\trevision_number, node_path, node_kind, node_action,\n+\t\tnode_copyfrom_path, node_copyfrom_rev, text_content_length,\n+\t\tprop_content_length, content_length;\n+} keys;\n+\n+static void reset_node_ctx(char *fname)\n+{\n+\tnode_ctx.type = 0;\n+\tnode_ctx.action = NODEACT_UNKNOWN;\n+\tnode_ctx.propLength = LENGTH_UNKNOWN;\n+\tnode_ctx.textLength = LENGTH_UNKNOWN;\n+\tnode_ctx.src[0] = ~0;\n+\tnode_ctx.srcRev = 0;\n+\tnode_ctx.srcMode = 0;\n+\tpool_tok_seq(REPO_MAX_PATH_DEPTH, node_ctx.dst, \"/\", fname);\n+\tnode_ctx.mark = 0;\n+}\n+\n+static void reset_rev_ctx(uint32_t revision)\n+{\n+\trev_ctx.revision = revision;\n+\trev_ctx.timestamp = 0;\n+\trev_ctx.log = NULL;\n+\trev_ctx.author = ~0;\n+}\n+\n+static void reset_dump_ctx(uint32_t url)\n+{\n+\tdump_ctx.url = url;\n+\tdump_ctx.uuid = ~0;\n+}\n+\n+static void init_keys(void)\n+{\n+\tkeys.svn_log = pool_intern(\"svn:log\");\n+\tkeys.svn_author = pool_intern(\"svn:author\");\n+\tkeys.svn_date = pool_intern(\"svn:date\");\n+\tkeys.svn_executable = pool_intern(\"svn:executable\");\n+\tkeys.svn_special = pool_intern(\"svn:special\");\n+\tkeys.uuid = pool_intern(\"UUID\");\n+\tkeys.revision_number = pool_intern(\"Revision-number\");\n+\tkeys.node_path = pool_intern(\"Node-path\");\n+\tkeys.node_kind = pool_intern(\"Node-kind\");\n+\tkeys.node_action = pool_intern(\"Node-action\");\n+\tkeys.node_copyfrom_path = pool_intern(\"Node-copyfrom-path\");\n+\tkeys.node_copyfrom_rev = pool_intern(\"Node-copyfrom-rev\");\n+\tkeys.text_content_length = pool_intern(\"Text-content-length\");\n+\tkeys.prop_content_length = pool_intern(\"Prop-content-length\");\n+\tkeys.content_length = pool_intern(\"Content-length\");\n+}\n+\n+static void read_props(void)\n+{\n+\tuint32_t len;\n+\tuint32_t key = ~0;\n+\tchar *val = NULL;\n+\tchar *t;\n+\twhile ((t = buffer_read_line()) && strcmp(t, \"PROPS-END\")) {\n+\t\tif (!strncmp(t, \"K \", 2)) {\n+\t\t\tlen = atoi(&t[2]);\n+\t\t\tkey = pool_intern(buffer_read_string(len));\n+\t\t\tbuffer_read_line();\n+\t\t} else if (!strncmp(t, \"V \", 2)) {\n+\t\t\tlen = atoi(&t[2]);\n+\t\t\tval = buffer_read_string(len);\n+\t\t\tif (key == keys.svn_log) {\n+\t\t\t\t/* Value length excludes terminating nul. */\n+\t\t\t\trev_ctx.log = log_copy(len + 1, val);\n+\t\t\t} else if (key == keys.svn_author) {\n+\t\t\t\trev_ctx.author = pool_intern(val);\n+\t\t\t} else if (key == keys.svn_date) {\n+\t\t\t\tif (parse_date_basic(val, &rev_ctx.timestamp, NULL))\n+\t\t\t\t\tfprintf(stderr, \"Invalid timestamp: %s\\n\", val);\n+\t\t\t} else if (key == keys.svn_executable) {\n+\t\t\t\tnode_ctx.type = REPO_MODE_EXE;\n+\t\t\t} else if (key == keys.svn_special) {\n+\t\t\t\tnode_ctx.type = REPO_MODE_LNK;\n+\t\t\t}\n+\t\t\tkey = ~0;\n+\t\t\tbuffer_read_line();\n+\t\t}\n+\t}\n+}\n+\n+static void handle_node(void)\n+{\n+\tif (node_ctx.propLength != LENGTH_UNKNOWN && node_ctx.propLength)\n+\t\tread_props();\n+\n+\tif (node_ctx.srcRev)\n+\t\tnode_ctx.srcMode = repo_copy(node_ctx.srcRev, node_ctx.src, node_ctx.dst);\n+\n+\tif (node_ctx.textLength != LENGTH_UNKNOWN &&\n+\t    node_ctx.type != REPO_MODE_DIR)\n+\t\tnode_ctx.mark = next_blob_mark();\n+\n+\tif (node_ctx.action == NODEACT_DELETE) {\n+\t\trepo_delete(node_ctx.dst);\n+\t} else if (node_ctx.action == NODEACT_CHANGE ||\n+\t\t\t   node_ctx.action == NODEACT_REPLACE) {\n+\t\tif (node_ctx.action == NODEACT_REPLACE &&\n+\t\t    node_ctx.type == REPO_MODE_DIR)\n+\t\t\trepo_replace(node_ctx.dst, node_ctx.mark);\n+\t\telse if (node_ctx.propLength != LENGTH_UNKNOWN)\n+\t\t\trepo_modify(node_ctx.dst, node_ctx.type, node_ctx.mark);\n+\t\telse if (node_ctx.textLength != LENGTH_UNKNOWN)\n+\t\t\tnode_ctx.srcMode = repo_replace(node_ctx.dst, node_ctx.mark);\n+\t} else if (node_ctx.action == NODEACT_ADD) {\n+\t\tif (node_ctx.srcRev && node_ctx.propLength != LENGTH_UNKNOWN)\n+\t\t\trepo_modify(node_ctx.dst, node_ctx.type, node_ctx.mark);\n+\t\telse if (node_ctx.srcRev && node_ctx.textLength != LENGTH_UNKNOWN)\n+\t\t\tnode_ctx.srcMode = repo_replace(node_ctx.dst, node_ctx.mark);\n+\t\telse if ((node_ctx.type == REPO_MODE_DIR && !node_ctx.srcRev) ||\n+\t\t\t node_ctx.textLength != LENGTH_UNKNOWN)\n+\t\t\trepo_add(node_ctx.dst, node_ctx.type, node_ctx.mark);\n+\t}\n+\n+\tif (node_ctx.propLength == LENGTH_UNKNOWN && node_ctx.srcMode)\n+\t\tnode_ctx.type = node_ctx.srcMode;\n+\n+\tif (node_ctx.mark)\n+\t\tfast_export_blob(node_ctx.type, node_ctx.mark, node_ctx.textLength);\n+\telse if (node_ctx.textLength != LENGTH_UNKNOWN)\n+\t\tbuffer_skip_bytes(node_ctx.textLength);\n+}\n+\n+static void handle_revision(void)\n+{\n+\tif (rev_ctx.revision)\n+\t\trepo_commit(rev_ctx.revision, rev_ctx.author, rev_ctx.log,\n+\t\t\tdump_ctx.uuid, dump_ctx.url, rev_ctx.timestamp);\n+}\n+\n+void svndump_read(const char *url)\n+{\n+\tchar *val;\n+\tchar *t;\n+\tuint32_t active_ctx = DUMP_CTX;\n+\tuint32_t len;\n+\tuint32_t key;\n+\n+\treset_dump_ctx(pool_intern(url));\n+\twhile ((t = buffer_read_line())) {\n+\t\tval = strstr(t, \": \");\n+\t\tif (!val)\n+\t\t\tcontinue;\n+\t\t*val++ = '\\0';\n+\t\t*val++ = '\\0';\n+\t\tkey = pool_intern(t);\n+\n+\t\tif (key == keys.uuid) {\n+\t\t\tdump_ctx.uuid = pool_intern(val);\n+\t\t} else if (key == keys.revision_number) {\n+\t\t\tif (active_ctx == NODE_CTX)\n+\t\t\t\thandle_node();\n+\t\t\tif (active_ctx != DUMP_CTX)\n+\t\t\t\thandle_revision();\n+\t\t\tactive_ctx = REV_CTX;\n+\t\t\treset_rev_ctx(atoi(val));\n+\t\t} else if (key == keys.node_path) {\n+\t\t\tif (active_ctx == NODE_CTX)\n+\t\t\t\thandle_node();\n+\t\t\tactive_ctx = NODE_CTX;\n+\t\t\treset_node_ctx(val);\n+\t\t} else if (key == keys.node_kind) {\n+\t\t\tif (!strcmp(val, \"dir\"))\n+\t\t\t\tnode_ctx.type = REPO_MODE_DIR;\n+\t\t\telse if (!strcmp(val, \"file\"))\n+\t\t\t\tnode_ctx.type = REPO_MODE_BLB;\n+\t\t\telse\n+\t\t\t\tfprintf(stderr, \"Unknown node-kind: %s\\n\", val);\n+\t\t} else if (key == keys.node_action) {\n+\t\t\tif (!strcmp(val, \"delete\")) {\n+\t\t\t\tnode_ctx.action = NODEACT_DELETE;\n+\t\t\t} else if (!strcmp(val, \"add\")) {\n+\t\t\t\tnode_ctx.action = NODEACT_ADD;\n+\t\t\t} else if (!strcmp(val, \"change\")) {\n+\t\t\t\tnode_ctx.action = NODEACT_CHANGE;\n+\t\t\t} else if (!strcmp(val, \"replace\")) {\n+\t\t\t\tnode_ctx.action = NODEACT_REPLACE;\n+\t\t\t} else {\n+\t\t\t\tfprintf(stderr, \"Unknown node-action: %s\\n\", val);\n+\t\t\t\tnode_ctx.action = NODEACT_UNKNOWN;\n+\t\t\t}\n+\t\t} else if (key == keys.node_copyfrom_path) {\n+\t\t\tpool_tok_seq(REPO_MAX_PATH_DEPTH, node_ctx.src, \"/\", val);\n+\t\t} else if (key == keys.node_copyfrom_rev) {\n+\t\t\tnode_ctx.srcRev = atoi(val);\n+\t\t} else if (key == keys.text_content_length) {\n+\t\t\tnode_ctx.textLength = atoi(val);\n+\t\t} else if (key == keys.prop_content_length) {\n+\t\t\tnode_ctx.propLength = atoi(val);\n+\t\t} else if (key == keys.content_length) {\n+\t\t\tlen = atoi(val);\n+\t\t\tbuffer_read_line();\n+\t\t\tif (active_ctx == REV_CTX) {\n+\t\t\t\tread_props();\n+\t\t\t} else if (active_ctx == NODE_CTX) {\n+\t\t\t\thandle_node();\n+\t\t\t\tactive_ctx = REV_CTX;\n+\t\t\t} else {\n+\t\t\t\tfprintf(stderr, \"Unexpected content length header: %d\\n\", len);\n+\t\t\t\tbuffer_skip_bytes(len);\n+\t\t\t}\n+\t\t}\n+\t}\n+\tif (active_ctx == NODE_CTX)\n+\t\thandle_node();\n+\tif (active_ctx != DUMP_CTX)\n+\t\thandle_revision();\n+}\n+\n+void svndump_init(const char *filename)\n+{\n+\tbuffer_init(filename);\n+\trepo_init();\n+\treset_dump_ctx(~0);\n+\treset_rev_ctx(0);\n+\treset_node_ctx(NULL);\n+\tinit_keys();\n+}\n+\n+void svndump_deinit(void)\n+{\n+\tlog_reset();\n+\trepo_reset();\n+\treset_dump_ctx(~0);\n+\treset_rev_ctx(0);\n+\treset_node_ctx(NULL);\n+\tif (buffer_deinit())\n+\t\tfprintf(stderr, \"Input error\\n\");\n+\tif (ferror(stdout))\n+\t\tfprintf(stderr, \"Output error\\n\");\n+}\n+\n+void svndump_reset(void)\n+{\n+\tlog_reset();\n+\tbuffer_reset();\n+\trepo_reset();\n+\treset_dump_ctx(~0);\n+\treset_rev_ctx(0);\n+\treset_node_ctx(NULL);\n+}\ndiff --git a/vcs-svn/svndump.h b/vcs-svn/svndump.h\nnew file mode 100644\nindex 0000000..93c412f\n--- /dev/null\n+++ b/vcs-svn/svndump.h\n@@ -0,0 +1,9 @@\n+#ifndef SVNDUMP_H_\n+#define SVNDUMP_H_\n+\n+void svndump_init(const char *filename);\n+void svndump_read(const char *url);\n+void svndump_deinit(void);\n+void svndump_reset(void);\n+\n+#endif\n-- \n1.7.2.1.544.ga752d.dirty\n"},{"id":"147578","messageId":"20100809225543.GE4429@burratino","threadId":"24407","inReplyTo":"20100809215719.GA4203@burratino","subject":"PATCH 09/10] Update svn-fe manual","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-09T22:55:43Z","receivedAt":"2010-08-09T22:55:43Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"The svn-fe example does not litter the working directory with\n.bin files any more (hoorah!).\n\nThe permissive error handling implies a known bug.  We should\nbe flagging iffy input and, even if we continue, reporting it\non exit.\n\nCc: David Barr <david.barr@cordelta.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n contrib/svn-fe/svn-fe.txt |   14 ++++++--------\n 1 files changed, 6 insertions(+), 8 deletions(-)\n\ndiff --git a/contrib/svn-fe/svn-fe.txt b/contrib/svn-fe/svn-fe.txt\nindex de30f83..fb0ee56 100644\n--- a/contrib/svn-fe/svn-fe.txt\n+++ b/contrib/svn-fe/svn-fe.txt\n@@ -43,11 +43,9 @@ user <user@UUID>\n as committer, where 'user' is the value of the `svn:author` property\n and 'UUID' the repository's identifier.\n \n-To support incremental imports, 'svn-fe' will put a `git-svn-id`\n-line at the end of each commit log message if passed an url on the\n-command line.  This line has the form `git-svn-id: URL@REVNO UUID`.\n-\n-Empty directories and unknown properties are silently discarded.\n+To support incremental imports, 'svn-fe' puts a `git-svn-id` line at\n+the end of each commit log message if passed an url on the command\n+line.  This line has the form `git-svn-id: URL@REVNO UUID`.\n \n The resulting repository will generally require further processing\n to put each project in its own repository and to separate the history\n@@ -56,9 +54,9 @@ may be useful for this purpose.\n \n BUGS\n ----\n-Litters the current working directory with .bin files for\n-persistence. Will be fixed when the svn-fe infrastructure is aware of\n-a Git working directory.\n+Empty directories and unknown properties are silently discarded.\n+\n+The exit status does not reflect whether an error was detected.\n \n SEE ALSO\n --------\n-- \n1.7.2.1.544.ga752d.dirty\n"},{"id":"147579","messageId":"20100809225836.GF4429@burratino","threadId":"24407","inReplyTo":"20100809215719.GA4203@burratino","subject":"[PATCH 10/10] svn-fe manual: Clarify warning about deltas in dump files","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-09T22:58:36Z","receivedAt":"2010-08-09T22:58:36Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Those in the know would notice that dump file format version 2\nmeans \"svnadmin dump --no-deltas\", but for the rest of us, an\nexplicit reminder is useful.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\nThat’s the end of the series.  Thanks for reading.\n\nOn the horizon, as you may have guessed, are changes to use\ndumpfile format v3.  Doing so sanely requires two-way communication\nwith fast-import, I think (as discussed).  Ram has already put\ntogether a prototype delta applier, so it seems to be mostly a matter\nof plumbing now.\n\n contrib/svn-fe/svn-fe.txt |    5 ++++-\n 1 files changed, 4 insertions(+), 1 deletions(-)\n\ndiff --git a/contrib/svn-fe/svn-fe.txt b/contrib/svn-fe/svn-fe.txt\nindex fb0ee56..35f84bd 100644\n--- a/contrib/svn-fe/svn-fe.txt\n+++ b/contrib/svn-fe/svn-fe.txt\n@@ -12,7 +12,7 @@ svnadmin dump --incremental REPO | svn-fe [url] | git fast-import\n DESCRIPTION\n -----------\n \n-Converts a Subversion dumpfile (version: 2) into input suitable for\n+Converts a Subversion dumpfile into input suitable for\n git-fast-import(1) and similar importers. REPO is a path to a\n Subversion repository mirrored on the local disk. Remote Subversion\n repositories can be mirrored on local disk using the `svnsync`\n@@ -25,6 +25,9 @@ Subversion's repository dump format is documented in full in\n Files in this format can be generated using the 'svnadmin dump' or\n 'svk admin dump' command.\n \n+Dumps produced with 'svnadmin dump --deltas' (dumpfile format v3)\n+are not supported.\n+\n OUTPUT FORMAT\n -------------\n The fast-import format is documented by the git-fast-import(1)\n-- \n1.7.2.1.544.ga752d.dirty\n"},{"id":"147631","messageId":"20100810125317.GB3921@kytes","threadId":"24407","inReplyTo":"20100809215719.GA4203@burratino","subject":"Re: [PATCH 0/10] rr/svn-export reroll","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-08-10T12:53:45Z","receivedAt":"2010-08-10T12:53:45Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi Jonathan,\n\nJonathan Nieder writes:\n> svn-fe has some serious changes on the horizon.  As a preparation,\n> let’s round up what we have now.\n> \n> The most controversial change is probably the new svn-fe test, which\n> takes about 15 seconds (for the “svnadmin load”, not the svn-fe\n> step :)).  It is in the t9* series, so hopefully that will not\n> dissuade people from running the earlier tests.\n\nI'll comment on this separately.\n\n> The main highlight in the changes is a new\n> \n> \tInput error\n> \n> to stderr if a system call failed in reading in the dump file.\n> It still returns status 0 in this and other error situations,\n> though.\n\nI'll comment on this separately.\n\n> Based on maint (for no good reason; that’s just where I tried it).\n> Intended to replace rr/svn-export in pu (only if Ram likes it, of\n> course).\n\nThanks for re-rolling (again)! You've also added a note to the commit\nmessages briefly explaining what each contributor has done. I'd\nexpected some incremental patches instead of a full re-roll, but\nwhatever works is good :)\n\n> David Barr (5):\n>   Add memory pool library\n>   Add string-specific memory pool\n>   Add stream helper library\n>   Infrastructure to write revisions in fast-export format\n>   SVN dump parser\n>\n> Jason Evans (1):\n>   Add treap implementation\n>\n> Jonathan Nieder (4):\n>   Introduce vcs-svn lib\n\nAll these are good :)\n\n>   Export parse_date_basic() to convert a date string to timestamp\n\nWasn't this ejected from this series and made a separate patch?\n\n>   Update svn-fe manual\n\nRemoved the BUG since we've turned off persistence.\n\n>   svn-fe manual: Clarify warning about deltas in dumpfiles\n\nWe have to fix this real soon- I'm waiting for the weekend so I get\nsome solid chunks of hacking time.\n\n-- Ram\n"},{"id":"147713","messageId":"20100811015303.GB4169@burratino","threadId":"24407","inReplyTo":"20100810125317.GB3921@kytes","subject":"Re: [PATCH 0/10] rr/svn-export reroll","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-11T01:53:03Z","receivedAt":"2010-08-11T01:53:03Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Ramkumar Ramachandra wrote:\n\n>                                                             I'd\n> expected some incremental patches instead of a full re-roll, but\n> whatever works is good :)\n\nYeah, I think after this series the topic has stabilized enough\nto build on. :)\n\n> All these are good :)\n\nThanks for checking.\n"},{"id":"147918","messageId":"7vmxsru4ny.fsf@alter.siamese.dyndns.org","threadId":"24407","inReplyTo":"20100809223442.GA4429@burratino","subject":"Re: [PATCH 05/10] Add string-specific memory pool","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-08-12T17:22:41Z","receivedAt":"2010-08-12T17:22:41Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Nieder <jrnieder@gmail.com> writes:\n\n> diff --git a/Makefile b/Makefile\n> index e7c33ec..24103c9 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -415,6 +415,7 @@ TEST_PROGRAMS_NEED_X += test-path-utils\n>  TEST_PROGRAMS_NEED_X += test-run-command\n>  TEST_PROGRAMS_NEED_X += test-sha1\n>  TEST_PROGRAMS_NEED_X += test-sigchain\n> +TEST_PROGRAMS_NEED_X += test-string-pool\n>  TEST_PROGRAMS_NEED_X += test-treap\n>  TEST_PROGRAMS_NEED_X += test-index-version\n\nDoes your Makefile do the right thing to vcs-svn/*.[oa] upon \"make clean\"?\n\n> diff --git a/vcs-svn/string_pool.c b/vcs-svn/string_pool.c\n> new file mode 100644\n> index 0000000..550f0e5\n> --- /dev/null\n> +++ b/vcs-svn/string_pool.c\n> @@ -0,0 +1,102 @@\n> ...\n> +uint32_t pool_intern(const char *key)\n> +{\n> +\t/* Canonicalize key */\n> +\tstruct node *match = NULL;\n> +\tuint32_t key_len;\n> +\tif (key == NULL)\n> +\t\treturn ~0;\n> +\tkey_len = strlen(key) + 1;\n> +\tstruct node *node = node_pointer(node_alloc(1));\n\nPlease fix decl-after-stmt here.\n"},{"id":"147919","messageId":"7vhbizu4nq.fsf@alter.siamese.dyndns.org","threadId":"24407","inReplyTo":"20100809221734.GE4203@burratino","subject":"Re: [PATCH 04/10] Add treap implementation","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-08-12T17:22:49Z","receivedAt":"2010-08-12T17:22:49Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Nieder <jrnieder@gmail.com> writes:\n\n> +/* Left accessors. */\n> +#define trp_left_get(a_base, a_field, a_node) \\\n> +\t(trpn_pointer(a_base, a_node)->a_field.trpn_left)\n> +#define trp_left_set(a_base, a_field, a_node, a_left) \\\n> +\tdo { \\\n> +\t\ttrpn_modify(a_base, a_node); \\\n> +\t\ttrp_left_get(a_base, a_field, a_node) = (a_left); \\\n> +\t} while(0)\n\nNeed SP after \"while\" (there are other occurrences).\n> +a_attr a_type MAYBE_UNUSED *a_pre##search(struct trp_root *treap, a_type *key) \\\n> +{ \\\n> +\tint cmp; \\\n> +\tuint32_t ret = treap->trp_root; \\\n> +\twhile (~ret && (cmp = (a_cmp)(key, trpn_pointer(a_base,ret)))) { \\\n\nSP after \",\" (same for nsearch)\n"},{"id":"147920","messageId":"7vbp97u4nh.fsf@alter.siamese.dyndns.org","threadId":"24407","inReplyTo":"20100809225500.GD4429@burratino","subject":"Re: [PATCH 08/10] SVN dump parser","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-08-12T17:22:58Z","receivedAt":"2010-08-12T17:22:58Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Nieder <jrnieder@gmail.com> writes:\n\n> diff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\n> new file mode 100644\n> index 0000000..630eeb5\n> --- /dev/null\n> +++ b/vcs-svn/svndump.c\n> @@ -0,0 +1,302 @@\n> ...\n> +static char* log_copy(uint32_t length, char *log)\n\nStyle: static char *log_copy(...)\n"},{"id":"147950","messageId":"20100812213047.GH2029@burratino","threadId":"24407","inReplyTo":"7vmxsru4ny.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH 05/10] Add string-specific memory pool","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-12T21:30:47Z","receivedAt":"2010-08-12T21:30:47Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Junio C Hamano wrote:\n> Jonathan Nieder <jrnieder@gmail.com> writes:\n\n>> diff --git a/Makefile b/Makefile\n>> index e7c33ec..24103c9 100644\n>> --- a/Makefile\n>> +++ b/Makefile\n>> @@ -415,6 +415,7 @@ TEST_PROGRAMS_NEED_X += test-path-utils\n>>  TEST_PROGRAMS_NEED_X += test-run-command\n>>  TEST_PROGRAMS_NEED_X += test-sha1\n>>  TEST_PROGRAMS_NEED_X += test-sigchain\n>> +TEST_PROGRAMS_NEED_X += test-string-pool\n>>  TEST_PROGRAMS_NEED_X += test-treap\n>>  TEST_PROGRAMS_NEED_X += test-index-version\n>\n> Does your Makefile do the right thing to vcs-svn/*.[oa] upon \"make clean\"?\n\nGood catch.  Here’s a fixup for patch 2 (“Introduce vcs-svn lib”).\n\n-- 8< --\nSubject: vcs-svn: remove build artifacts on “make clean”\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\ndiff --git a/Makefile b/Makefile\nindex 2418820..24c4b3d 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -2184,8 +2184,8 @@ distclean: clean\n \t$(RM) configure\n \n clean:\n-\t$(RM) *.o block-sha1/*.o ppc/*.o compat/*.o compat/*/*.o xdiff/*.o \\\n-\t\tbuiltin/*.o $(LIB_FILE) $(XDIFF_LIB)\n+\t$(RM) *.o block-sha1/*.o ppc/*.o compat/*.o compat/*/*.o xdiff/*.o vcs-svn/*.o \\\n+\t\tbuiltin/*.o $(LIB_FILE) $(XDIFF_LIB) $(VCSSVN_LIB)\n \t$(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) git$X\n \t$(RM) $(TEST_PROGRAMS)\n \t$(RM) -r bin-wrappers\n-- \n"},{"id":"147954","messageId":"20100812220257.GI2029@burratino","threadId":"24407","inReplyTo":"7vhbizu4nq.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH 04/10] Add treap implementation","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-12T22:02:57Z","receivedAt":"2010-08-12T22:02:57Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Junio C Hamano wrote:\n\n> Need SP after \"while\" (there are other occurrences).\n\nGood catch.  checkpatch also notices some long lines, but I think\nthat’s worth ignoring.\n\n-- 8< --\nSubject: treap: style fix\n\nMissing spaces in while (0) and trpn_pointer(a, b).\n\nRemove parentheses around return value.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n vcs-svn/trp.h |   30 +++++++++++++++---------------\n 1 files changed, 15 insertions(+), 15 deletions(-)\n\ndiff --git a/vcs-svn/trp.h b/vcs-svn/trp.h\nindex 1f5f51f..ee35c68 100644\n--- a/vcs-svn/trp.h\n+++ b/vcs-svn/trp.h\n@@ -37,7 +37,7 @@ struct trp_root {\n \t\t\t*trpn_pointer(a_base, a_offset) = \\\n \t\t\t\t*trpn_pointer(a_base, old_offset); \\\n \t\t} \\\n-\t} while (0);\n+\t} while (0)\n \n /* Left accessors. */\n #define trp_left_get(a_base, a_field, a_node) \\\n@@ -46,7 +46,7 @@ struct trp_root {\n \tdo { \\\n \t\ttrpn_modify(a_base, a_node); \\\n \t\ttrp_left_get(a_base, a_field, a_node) = (a_left); \\\n-\t} while(0)\n+\t} while (0)\n \n /* Right accessors. */\n #define trp_right_get(a_base, a_field, a_node) \\\n@@ -55,7 +55,7 @@ struct trp_root {\n \tdo { \\\n \t\ttrpn_modify(a_base, a_node); \\\n \t\ttrp_right_get(a_base, a_field, a_node) = (a_right); \\\n-\t} while(0)\n+\t} while (0)\n \n /*\n  * Fibonacci hash function.\n@@ -72,7 +72,7 @@ struct trp_root {\n \tdo { \\\n \t\ttrp_left_set(a_base, a_field, (a_node), ~0); \\\n \t\ttrp_right_set(a_base, a_field, (a_node), ~0); \\\n-\t} while(0)\n+\t} while (0)\n \n /* Internal utility macros. */\n #define trpn_first(a_base, a_field, a_root, r_node) \\\n@@ -90,7 +90,7 @@ struct trp_root {\n \t\ttrp_right_set(a_base, a_field, (a_node), \\\n \t\t\ttrp_left_get(a_base, a_field, (r_node))); \\\n \t\ttrp_left_set(a_base, a_field, (r_node), (a_node)); \\\n-\t} while(0)\n+\t} while (0)\n \n #define trpn_rotate_right(a_base, a_field, a_node, r_node) \\\n \tdo { \\\n@@ -98,7 +98,7 @@ struct trp_root {\n \t\ttrp_left_set(a_base, a_field, (a_node), \\\n \t\t\ttrp_right_get(a_base, a_field, (r_node))); \\\n \t\ttrp_right_set(a_base, a_field, (r_node), (a_node)); \\\n-\t} while(0)\n+\t} while (0)\n \n #define trp_gen(a_attr, a_pre, a_type, a_field, a_base, a_cmp) \\\n a_attr a_type MAYBE_UNUSED *a_pre##first(struct trp_root *treap) \\\n@@ -136,7 +136,7 @@ a_attr a_type MAYBE_UNUSED *a_pre##search(struct trp_root *treap, a_type *key) \\\n { \\\n \tint cmp; \\\n \tuint32_t ret = treap->trp_root; \\\n-\twhile (~ret && (cmp = (a_cmp)(key, trpn_pointer(a_base,ret)))) { \\\n+\twhile (~ret && (cmp = (a_cmp)(key, trpn_pointer(a_base, ret)))) { \\\n \t\tif (cmp < 0) { \\\n \t\t\tret = trp_left_get(a_base, a_field, ret); \\\n \t\t} else { \\\n@@ -149,7 +149,7 @@ a_attr a_type MAYBE_UNUSED *a_pre##nsearch(struct trp_root *treap, a_type *key)\n { \\\n \tint cmp; \\\n \tuint32_t ret = treap->trp_root; \\\n-\twhile (~ret && (cmp = (a_cmp)(key, trpn_pointer(a_base,ret)))) { \\\n+\twhile (~ret && (cmp = (a_cmp)(key, trpn_pointer(a_base, ret)))) { \\\n \t\tif (cmp < 0) { \\\n \t\t\tif (!~trp_left_get(a_base, a_field, ret)) \\\n \t\t\t\tbreak; \\\n@@ -163,7 +163,7 @@ a_attr a_type MAYBE_UNUSED *a_pre##nsearch(struct trp_root *treap, a_type *key)\n a_attr uint32_t MAYBE_UNUSED a_pre##insert_recurse(uint32_t cur_node, uint32_t ins_node) \\\n { \\\n \tif (cur_node == ~0) { \\\n-\t\treturn (ins_node); \\\n+\t\treturn ins_node; \\\n \t} else { \\\n \t\tuint32_t ret; \\\n \t\tint cmp = (a_cmp)(trpn_pointer(a_base, ins_node), \\\n@@ -185,7 +185,7 @@ a_attr uint32_t MAYBE_UNUSED a_pre##insert_recurse(uint32_t cur_node, uint32_t i\n \t\t\telse \\\n \t\t\t\tret = cur_node; \\\n \t\t} \\\n-\t\treturn (ret); \\\n+\t\treturn ret; \\\n \t} \\\n } \\\n a_attr void MAYBE_UNUSED a_pre##insert(struct trp_root *treap, a_type *node) \\\n@@ -204,27 +204,27 @@ a_attr uint32_t MAYBE_UNUSED a_pre##remove_recurse(uint32_t cur_node, uint32_t r\n \t\tuint32_t right = trp_right_get(a_base, a_field, cur_node); \\\n \t\tif (left == ~0) { \\\n \t\t\tif (right == ~0) \\\n-\t\t\t\treturn (~0); \\\n+\t\t\t\treturn ~0; \\\n \t\t} else if (right == ~0 || trp_prio_get(left) < trp_prio_get(right)) { \\\n \t\t\ttrpn_rotate_right(a_base, a_field, cur_node, ret); \\\n \t\t\tright = a_pre##remove_recurse(cur_node, rem_node); \\\n \t\t\ttrp_right_set(a_base, a_field, ret, right); \\\n-\t\t\treturn (ret); \\\n+\t\t\treturn ret; \\\n \t\t} \\\n \t\ttrpn_rotate_left(a_base, a_field, cur_node, ret); \\\n \t\tleft = a_pre##remove_recurse(cur_node, rem_node); \\\n \t\ttrp_left_set(a_base, a_field, ret, left); \\\n-\t\treturn (ret); \\\n+\t\treturn ret; \\\n \t} else if (cmp < 0) { \\\n \t\tuint32_t left = a_pre##remove_recurse( \\\n \t\t\ttrp_left_get(a_base, a_field, cur_node), rem_node); \\\n \t\ttrp_left_set(a_base, a_field, cur_node, left); \\\n-\t\treturn (cur_node); \\\n+\t\treturn cur_node; \\\n \t} else { \\\n \t\tuint32_t right = a_pre##remove_recurse( \\\n \t\t\ttrp_right_get(a_base, a_field, cur_node), rem_node); \\\n \t\ttrp_right_set(a_base, a_field, cur_node, right); \\\n-\t\treturn (cur_node); \\\n+\t\treturn cur_node; \\\n \t} \\\n } \\\n a_attr void MAYBE_UNUSED a_pre##remove(struct trp_root *treap, a_type *node) \\\n-- \n1.7.2.1.544.ga752d.dirty\n"},{"id":"147958","messageId":"20100812221115.GJ2029@burratino","threadId":"24407","inReplyTo":"7vhbizu4nq.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH 04/10] Add treap implementation","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-12T22:11:15Z","receivedAt":"2010-08-12T22:11:15Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Junio C Hamano wrote:\n> Jonathan Nieder <jrnieder@gmail.com> writes:\n\n>> +#define trp_left_set(a_base, a_field, a_node, a_left) \\\n>> +\tdo { \\\n>> +\t\ttrpn_modify(a_base, a_node); \\\n>> +\t\ttrp_left_get(a_base, a_field, a_node) = (a_left); \\\n>> +\t} while(0)\n>\n> Need SP after \"while\" (there are other occurrences).\n\nHere are a few more (but feel free to ignore them).\n\n-- 8< --\nSubject: Standardize do { ... } while (0) style\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n base85.c   |    6 +++---\n cache.h    |    2 +-\n diffcore.h |    8 ++++----\n http.h     |    4 ++--\n 4 files changed, 10 insertions(+), 10 deletions(-)\n\ndiff --git a/base85.c b/base85.c\nindex e459fee..781b575 100644\n--- a/base85.c\n+++ b/base85.c\n@@ -7,9 +7,9 @@\n #define say1(a,b) fprintf(stderr, a, b)\n #define say2(a,b,c) fprintf(stderr, a, b, c)\n #else\n-#define say(a) do {} while(0)\n-#define say1(a,b) do {} while(0)\n-#define say2(a,b,c) do {} while(0)\n+#define say(a) do { /* nothing */ } while (0)\n+#define say1(a,b) do { /* nothing */ } while (0)\n+#define say2(a,b,c) do { /* nothing */ } while (0)\n #endif\n \n static const char en85[] = {\ndiff --git a/cache.h b/cache.h\nindex 68258be..37ef9d8 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -449,7 +449,7 @@ extern int init_db(const char *template_dir, unsigned int flags);\n \t\t\t\talloc = alloc_nr(alloc); \\\n \t\t\tx = xrealloc((x), alloc * sizeof(*(x))); \\\n \t\t} \\\n-\t} while(0)\n+\t} while (0)\n \n /* Initialize and use the cache information */\n extern int read_index(struct index_state *);\ndiff --git a/diffcore.h b/diffcore.h\nindex fed9b15..05ebc11 100644\n--- a/diffcore.h\n+++ b/diffcore.h\n@@ -98,7 +98,7 @@ struct diff_queue_struct {\n \t\t(q)->queue = NULL; \\\n \t\t(q)->nr = (q)->alloc = 0; \\\n \t\t(q)->run = 0; \\\n-\t} while(0);\n+\t} while (0)\n \n extern struct diff_queue_struct diff_queued_diff;\n extern struct diff_filepair *diff_queue(struct diff_queue_struct *,\n@@ -118,9 +118,9 @@ void diff_debug_filespec(struct diff_filespec *, int, const char *);\n void diff_debug_filepair(const struct diff_filepair *, int);\n void diff_debug_queue(const char *, struct diff_queue_struct *);\n #else\n-#define diff_debug_filespec(a,b,c) do {} while(0)\n-#define diff_debug_filepair(a,b) do {} while(0)\n-#define diff_debug_queue(a,b) do {} while(0)\n+#define diff_debug_filespec(a,b,c) do { /* nothing */ } while (0)\n+#define diff_debug_filepair(a,b) do { /* nothing */ } while (0)\n+#define diff_debug_queue(a,b) do { /* nothing */ } while (0)\n #endif\n \n extern int diffcore_count_changes(struct diff_filespec *src,\ndiff --git a/http.h b/http.h\nindex a0b5901..173f74c 100644\n--- a/http.h\n+++ b/http.h\n@@ -23,10 +23,10 @@\n #endif\n \n #if LIBCURL_VERSION_NUM < 0x070704\n-#define curl_global_cleanup() do { /* nothing */ } while(0)\n+#define curl_global_cleanup() do { /* nothing */ } while (0)\n #endif\n #if LIBCURL_VERSION_NUM < 0x070800\n-#define curl_global_init(a) do { /* nothing */ } while(0)\n+#define curl_global_init(a) do { /* nothing */ } while (0)\n #endif\n \n #if (LIBCURL_VERSION_NUM < 0x070c04) || (LIBCURL_VERSION_NUM == 0x071000)\n-- \n1.7.2.1.544.ga752d.dirty\n"},{"id":"147966","messageId":"7v8w4bpi2j.fsf@alter.siamese.dyndns.org","threadId":"24407","inReplyTo":"20100812221115.GJ2029@burratino","subject":"Re: [PATCH 04/10] Add treap implementation","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-08-12T22:44:20Z","receivedAt":"2010-08-12T22:44:20Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Nieder <jrnieder@gmail.com> writes:\n\n> diff --git a/diffcore.h b/diffcore.h\n> index fed9b15..05ebc11 100644\n> --- a/diffcore.h\n> +++ b/diffcore.h\n> @@ -98,7 +98,7 @@ struct diff_queue_struct {\n>  \t\t(q)->queue = NULL; \\\n>  \t\t(q)->nr = (q)->alloc = 0; \\\n>  \t\t(q)->run = 0; \\\n> -\t} while(0);\n> +\t} while (0)\n\nThis is a _bad_ one.  Thanks.\n"},{"id":"153202","messageId":"20101011023435.GA706@burratino","threadId":"24407","inReplyTo":"20100810125317.GB3921@kytes","subject":"[PATCH/WIP 00/16] svn delta applier","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-11T02:34:36Z","receivedAt":"2010-10-11T02:34:36Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nThe svndiff format has proved more difficult to parse than expected.\nThis series documents the current state of things, and though it is\nnot complete, it should be ready for nitpicking by the masses.\n\nPatches 1-4 modify the line_buffer API by introducing a struct\nline_buffer to collect state that was previously held in global\nvariables.  Callers can use multiple line_buffers to manage input from\nmultiple files at a time.\n\nPatches 5-10 add various utility functions to the line_buffer API\n(wrapping strbuf_fread(), fgetc(), etc).  Putting the helpers there\ninstead of having callers work with the FILE* directly means one\ncould easily\n\n - tweak the input stream (to insert \"link: \" at the beginning\n   for symlinks?);\n - trace reads, for debugging; or\n - use read() directly in place of stdio and limit the number of bytes\n   buffered\n\nif one wants to.\n\nPatch 11 adds a data structure and function to manage a \"sliding\nwindow\" without using mmap() or fseek().  See the svndiff0 spec[1] for\nhow this would be used.\n\nPatches 12 and 13 are some basic components for reading an svndiff0\nfile: reading variable-length integers and the opening magic bytes.\n\nPatch 15 makes the svn-fe test usable on systems (like Ram's) without\nlibsvn-perl installed.  It also should make the test easier to read\nfor people unfamiliar with lib-git-svn.sh.\n\nPatch 16 is the delta parser/applier.  This patch does _not_ add it to\ncontrib/svn-fe, even though that would be useful, since the\ncommand-line interface is not set in stone yet.  If you want to try it\nout, use the test-svn-fe command:\n\n\ttest-svn-fe -d <preimage> <delta> <delta length>\n\nThe preimage or delta arg can be /dev/stdin for use in a pipeline.\nBoth are only read sequentially; they do not need to be regular files.\n\nOne of the test cases is enormous.  The svn delta lib doesn't use\nmultiple windows except when dealing with relatively big files, but\nprobably the test case should be replaced with a smaller, artificial\nexample.\n\nOne of the test cases does not pass.  I also don't know how to apply\nthe delta by hand --- it seems to have some extra bytes at the end. :(\nUnfortunately the svndiff0 spec is not as clear about when to stop\nreading as one might like\n\nThe code separately maintains nominal and actual lengths for a few\nbuffers, since truncated input is permitted (and even required) in the\ndeltas svn produces, though the svndiff0 spec does not document the\nsemantics of that.\n\nFor svn-fe changes to take advantage of this code to handle the\ndumpfilev3 format, see <git://github.com/barrbrain/git.git>[2].  So\nnow the full svnrdump | svn-fe | fast-import pipeline can be\nexperienced.  It still chokes on some deltas in the wild.\n\nThoughts, cleanups, test cases, bug reports, improvements welcome. :)\n\nEnjoy,\nJonathan Nieder (15):\n  vcs-svn: Eliminate global byte_buffer[] array\n  vcs-svn: Replace buffer_read_string()'s memory pool with a strbuf\n  vcs-svn: Collect line_buffer data in a struct\n  vcs-svn: Teach line_buffer to handle multiple input files\n  vcs-svn: Make buffer_skip_bytes() report partial reads\n  vcs-svn: Better support for reading large files\n  vcs-svn: Add binary-safe read() function\n  vcs-svn: Let callers peek ahead to find stream end\n  vcs-svn: Allow input errors to be detected early\n  vcs-svn: Allow character-oriented input\n  vcs-svn: Add code to maintain a sliding view of a file\n  vcs-svn: Learn to parse variable-length integers\n  vcs-svn: Learn to check for SVN\\0 magic\n  compat: helper for detecting unsigned overflow\n  vcs-svn: Add svn delta parser\n\nRamkumar Ramachandra (1):\n  t9010 (svn-fe): Eliminate dependency on svn perl bindings\n\n Makefile                 |    5 +-\n vcs-svn/line_buffer.txt  |    8 +-\n vcs-svn/fast_export.c    |    6 +-\n vcs-svn/fast_export.h    |    5 +-\n vcs-svn/line_buffer.c    |   99 +-\n vcs-svn/line_buffer.h    |   29 +-\n vcs-svn/sliding_window.c |   65 +\n vcs-svn/sliding_window.h |   14 +\n vcs-svn/svndiff.c        |  344 +\n vcs-svn/svndiff.h        |    9 +\n vcs-svn/svndump.c        |   29 +-\n vcs-svn/LICENSE          |    2 +\n git-compat-util.h        |    6 +\n test-line-buffer.c       |   17 +-\n test-svn-fe.c            |   37 +-\n t/t9010-svn-fe.sh        |   29 +-\n t/t9010/Xerces.cpp.diff0 |  Bin 0 -> 12185 bytes\n t/t9010/Xerces.cpp.done  |54963 +++++++++++++++++++++++++++++++++++++++++++++\n t/t9010/Xerces.cpp.src   |55052 ++++++++++++++++++++++++++++++++++++++++++++++\n t/t9010/newdata.diff0    |  Bin 0 -> 19392 bytes\n t/t9010/newdata.done     |  522 +\n t/t9010/src.diff0        |  Bin 0 -> 74 bytes\n t/t9010/src.done         |  522 +\n 23 files changed, 111677 insertions(+), 86 deletions(-)\n create mode 100644 vcs-svn/sliding_window.c\n create mode 100644 vcs-svn/sliding_window.h\n create mode 100644 vcs-svn/svndiff.c\n create mode 100644 vcs-svn/svndiff.h\n create mode 100644 t/t9010/Xerces.cpp.diff0\n create mode 100644 t/t9010/Xerces.cpp.done\n create mode 100644 t/t9010/Xerces.cpp.src\n create mode 100644 t/t9010/blank.done\n create mode 100644 t/t9010/newdata.diff0\n create mode 100644 t/t9010/newdata.done\n create mode 100644 t/t9010/src.diff0\n create mode 100644 t/t9010/src.done\n\n[1] http://svn.apache.org/repos/asf/subversion/trunk/notes/svndiff\n[2] And some design notes:\nhttp://thread.gmane.org/gmane.comp.version-control.git/150005/focus=157119\n"},{"id":"153203","messageId":"20101011023710.GA1553@burratino","threadId":"24407","inReplyTo":"20101011023435.GA706@burratino","subject":"[PATCH 01/16] vcs-svn: Eliminate global byte_buffer[] array","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-11T02:37:10Z","receivedAt":"2010-10-11T02:37:10Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"The data stored in byte_buffer[] is always either discarded or\nwritten to stdout immediately.  No need for it to persist between\nfunction calls.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n vcs-svn/line_buffer.c |    3 ++-\n 1 files changed, 2 insertions(+), 1 deletions(-)\n\ndiff --git a/vcs-svn/line_buffer.c b/vcs-svn/line_buffer.c\nindex 1543567..f22c94f 100644\n--- a/vcs-svn/line_buffer.c\n+++ b/vcs-svn/line_buffer.c\n@@ -14,7 +14,6 @@\n obj_pool_gen(blob, char, 4096)\n \n static char line_buffer[LINE_BUFFER_LEN];\n-static char byte_buffer[COPY_BUFFER_LEN];\n static FILE *infile;\n \n int buffer_init(const char *filename)\n@@ -68,6 +67,7 @@ char *buffer_read_string(uint32_t len)\n \n void buffer_copy_bytes(uint32_t len)\n {\n+\tchar byte_buffer[COPY_BUFFER_LEN];\n \tuint32_t in;\n \twhile (len > 0 && !feof(infile) && !ferror(infile)) {\n \t\tin = len < COPY_BUFFER_LEN ? len : COPY_BUFFER_LEN;\n@@ -83,6 +83,7 @@ void buffer_copy_bytes(uint32_t len)\n \n void buffer_skip_bytes(uint32_t len)\n {\n+\tchar byte_buffer[COPY_BUFFER_LEN];\n \tuint32_t in;\n \twhile (len > 0 && !feof(infile) && !ferror(infile)) {\n \t\tin = len < COPY_BUFFER_LEN ? len : COPY_BUFFER_LEN;\n-- \n1.7.2.3\n"},{"id":"153204","messageId":"20101011023921.GC1553@burratino","threadId":"24407","inReplyTo":"20101011023435.GA706@burratino","subject":"[PATCH 03/16] vcs-svn: Collect line_buffer data in a struct","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-11T02:39:21Z","receivedAt":"2010-10-11T02:39:21Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Prepare for the line_buffer lib to support input from multiple files,\nby collecting global state in a struct that can be easily passed around.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n vcs-svn/line_buffer.c |   45 ++++++++++++++++++++++-----------------------\n vcs-svn/line_buffer.h |   11 +++++++++++\n 2 files changed, 33 insertions(+), 23 deletions(-)\n\ndiff --git a/vcs-svn/line_buffer.c b/vcs-svn/line_buffer.c\nindex 6f32f28..e7bc230 100644\n--- a/vcs-svn/line_buffer.c\n+++ b/vcs-svn/line_buffer.c\n@@ -7,17 +7,16 @@\n #include \"line_buffer.h\"\n #include \"strbuf.h\"\n \n-#define LINE_BUFFER_LEN 10000\n #define COPY_BUFFER_LEN 4096\n-\n-static char line_buffer[LINE_BUFFER_LEN];\n-static struct strbuf blob_buffer = STRBUF_INIT;\n-static FILE *infile;\n+static struct line_buffer buf_ = LINE_BUFFER_INIT;\n+static struct line_buffer *buf;\n \n int buffer_init(const char *filename)\n {\n-\tinfile = filename ? fopen(filename, \"r\") : stdin;\n-\tif (!infile)\n+\tbuf = &buf_;\n+\n+\tbuf->infile = filename ? fopen(filename, \"r\") : stdin;\n+\tif (!buf->infile)\n \t\treturn -1;\n \treturn 0;\n }\n@@ -25,10 +24,10 @@ int buffer_init(const char *filename)\n int buffer_deinit(void)\n {\n \tint err;\n-\tif (infile == stdin)\n-\t\treturn ferror(infile);\n-\terr = ferror(infile);\n-\terr |= fclose(infile);\n+\tif (buf->infile == stdin)\n+\t\treturn ferror(buf->infile);\n+\terr = ferror(buf->infile);\n+\terr |= fclose(buf->infile);\n \treturn err;\n }\n \n@@ -36,13 +35,13 @@ int buffer_deinit(void)\n char *buffer_read_line(void)\n {\n \tchar *end;\n-\tif (!fgets(line_buffer, sizeof(line_buffer), infile))\n+\tif (!fgets(buf->line_buffer, sizeof(buf->line_buffer), buf->infile))\n \t\t/* Error or data exhausted. */\n \t\treturn NULL;\n-\tend = line_buffer + strlen(line_buffer);\n+\tend = buf->line_buffer + strlen(buf->line_buffer);\n \tif (end[-1] == '\\n')\n \t\tend[-1] = '\\0';\n-\telse if (feof(infile))\n+\telse if (feof(buf->infile))\n \t\t; /* No newline at end of file.  That's fine. */\n \telse\n \t\t/*\n@@ -51,23 +50,23 @@ char *buffer_read_line(void)\n \t\t * but for now let's return an error.\n \t\t */\n \t\treturn NULL;\n-\treturn line_buffer;\n+\treturn buf->line_buffer;\n }\n \n char *buffer_read_string(uint32_t len)\n {\n-\tstrbuf_reset(&blob_buffer);\n-\tstrbuf_fread(&blob_buffer, len, infile);\n-\treturn ferror(infile) ? NULL : blob_buffer.buf;\n+\tstrbuf_reset(&buf->blob_buffer);\n+\tstrbuf_fread(&buf->blob_buffer, len, buf->infile);\n+\treturn ferror(buf->infile) ? NULL : buf->blob_buffer.buf;\n }\n \n void buffer_copy_bytes(uint32_t len)\n {\n \tchar byte_buffer[COPY_BUFFER_LEN];\n \tuint32_t in;\n-\twhile (len > 0 && !feof(infile) && !ferror(infile)) {\n+\twhile (len > 0 && !feof(buf->infile) && !ferror(buf->infile)) {\n \t\tin = len < COPY_BUFFER_LEN ? len : COPY_BUFFER_LEN;\n-\t\tin = fread(byte_buffer, 1, in, infile);\n+\t\tin = fread(byte_buffer, 1, in, buf->infile);\n \t\tlen -= in;\n \t\tfwrite(byte_buffer, 1, in, stdout);\n \t\tif (ferror(stdout)) {\n@@ -81,14 +80,14 @@ void buffer_skip_bytes(uint32_t len)\n {\n \tchar byte_buffer[COPY_BUFFER_LEN];\n \tuint32_t in;\n-\twhile (len > 0 && !feof(infile) && !ferror(infile)) {\n+\twhile (len > 0 && !feof(buf->infile) && !ferror(buf->infile)) {\n \t\tin = len < COPY_BUFFER_LEN ? len : COPY_BUFFER_LEN;\n-\t\tin = fread(byte_buffer, 1, in, infile);\n+\t\tin = fread(byte_buffer, 1, in, buf->infile);\n \t\tlen -= in;\n \t}\n }\n \n void buffer_reset(void)\n {\n-\tstrbuf_release(&blob_buffer);\n+\tstrbuf_release(&buf->blob_buffer);\n }\ndiff --git a/vcs-svn/line_buffer.h b/vcs-svn/line_buffer.h\nindex 9c78ae1..4ae1133 100644\n--- a/vcs-svn/line_buffer.h\n+++ b/vcs-svn/line_buffer.h\n@@ -1,6 +1,17 @@\n #ifndef LINE_BUFFER_H_\n #define LINE_BUFFER_H_\n \n+#include \"strbuf.h\"\n+\n+#define LINE_BUFFER_LEN 10000\n+\n+struct line_buffer {\n+\tchar line_buffer[LINE_BUFFER_LEN];\n+\tstruct strbuf blob_buffer;\n+\tFILE *infile;\n+};\n+#define LINE_BUFFER_INIT {\"\", STRBUF_INIT, NULL}\n+\n int buffer_init(const char *filename);\n int buffer_deinit(void);\n char *buffer_read_line(void);\n-- \n1.7.2.3\n"},{"id":"153205","messageId":"20101011024106.GD1553@burratino","threadId":"24407","inReplyTo":"20101011023435.GA706@burratino","subject":"[PATCH 04/16] vcs-svn: Teach line_buffer to handle multiple input files","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-11T02:41:06Z","receivedAt":"2010-10-11T02:41:06Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Collect the line_buffer state in a newly public line_buffer struct.\nCallers can use multiple line_buffers to manage input from multiple\nfiles at a time.\n\nThe Subversion-format delta applier will use this to stream a delta\nand the preimage it applies to at the same time.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n vcs-svn/line_buffer.txt |    5 +++--\n vcs-svn/fast_export.c   |    6 +++---\n vcs-svn/fast_export.h   |    5 ++++-\n vcs-svn/line_buffer.c   |   20 ++++++++------------\n vcs-svn/line_buffer.h   |   14 +++++++-------\n vcs-svn/svndump.c       |   29 ++++++++++++++++-------------\n test-line-buffer.c      |   17 +++++++++--------\n 7 files changed, 50 insertions(+), 46 deletions(-)\n\ndiff --git a/vcs-svn/line_buffer.txt b/vcs-svn/line_buffer.txt\nindex 8906fb1..f8eaa4d 100644\n--- a/vcs-svn/line_buffer.txt\n+++ b/vcs-svn/line_buffer.txt\n@@ -14,14 +14,15 @@ Calling sequence\n \n The calling program:\n \n+ - initializes a `struct line_buffer` to LINE_BUFFER_INIT\n  - specifies a file to read with `buffer_init`\n  - processes input with `buffer_read_line`, `buffer_read_string`,\n    `buffer_skip_bytes`, and `buffer_copy_bytes`\n  - closes the file with `buffer_deinit`, perhaps to start over and\n    read another file.\n \n-Before exiting, the caller can use `buffer_reset` to deallocate\n-resources for the benefit of profiling tools.\n+When finished, the caller can use `buffer_reset` to deallocate\n+resources.\n \n Functions\n ---------\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex 6cfa256..260cf50 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -63,14 +63,14 @@ void fast_export_commit(uint32_t revision, uint32_t author, char *log,\n \tprintf(\"progress Imported commit %\"PRIu32\".\\n\\n\", revision);\n }\n \n-void fast_export_blob(uint32_t mode, uint32_t mark, uint32_t len)\n+void fast_export_blob(uint32_t mode, uint32_t mark, uint32_t len, struct line_buffer *input)\n {\n \tif (mode == REPO_MODE_LNK) {\n \t\t/* svn symlink blobs start with \"link \" */\n-\t\tbuffer_skip_bytes(5);\n+\t\tbuffer_skip_bytes(input, 5);\n \t\tlen -= 5;\n \t}\n \tprintf(\"blob\\nmark :%\"PRIu32\"\\ndata %\"PRIu32\"\\n\", mark, len);\n-\tbuffer_copy_bytes(len);\n+\tbuffer_copy_bytes(input, len);\n \tfputc('\\n', stdout);\n }\ndiff --git a/vcs-svn/fast_export.h b/vcs-svn/fast_export.h\nindex 2aaaea5..054e7d5 100644\n--- a/vcs-svn/fast_export.h\n+++ b/vcs-svn/fast_export.h\n@@ -1,11 +1,14 @@\n #ifndef FAST_EXPORT_H_\n #define FAST_EXPORT_H_\n \n+#include \"line_buffer.h\"\n+\n void fast_export_delete(uint32_t depth, uint32_t *path);\n void fast_export_modify(uint32_t depth, uint32_t *path, uint32_t mode,\n \t\t\tuint32_t mark);\n void fast_export_commit(uint32_t revision, uint32_t author, char *log,\n \t\t\tuint32_t uuid, uint32_t url, unsigned long timestamp);\n-void fast_export_blob(uint32_t mode, uint32_t mark, uint32_t len);\n+void fast_export_blob(uint32_t mode, uint32_t mark, uint32_t len,\n+\t\t      struct line_buffer *input);\n \n #endif\ndiff --git a/vcs-svn/line_buffer.c b/vcs-svn/line_buffer.c\nindex e7bc230..806932b 100644\n--- a/vcs-svn/line_buffer.c\n+++ b/vcs-svn/line_buffer.c\n@@ -8,20 +8,16 @@\n #include \"strbuf.h\"\n \n #define COPY_BUFFER_LEN 4096\n-static struct line_buffer buf_ = LINE_BUFFER_INIT;\n-static struct line_buffer *buf;\n \n-int buffer_init(const char *filename)\n+int buffer_init(struct line_buffer *buf, const char *filename)\n {\n-\tbuf = &buf_;\n-\n \tbuf->infile = filename ? fopen(filename, \"r\") : stdin;\n \tif (!buf->infile)\n \t\treturn -1;\n \treturn 0;\n }\n \n-int buffer_deinit(void)\n+int buffer_deinit(struct line_buffer *buf)\n {\n \tint err;\n \tif (buf->infile == stdin)\n@@ -32,7 +28,7 @@ int buffer_deinit(void)\n }\n \n /* Read a line without trailing newline. */\n-char *buffer_read_line(void)\n+char *buffer_read_line(struct line_buffer *buf)\n {\n \tchar *end;\n \tif (!fgets(buf->line_buffer, sizeof(buf->line_buffer), buf->infile))\n@@ -53,14 +49,14 @@ char *buffer_read_line(void)\n \treturn buf->line_buffer;\n }\n \n-char *buffer_read_string(uint32_t len)\n+char *buffer_read_string(struct line_buffer *buf, uint32_t len)\n {\n \tstrbuf_reset(&buf->blob_buffer);\n \tstrbuf_fread(&buf->blob_buffer, len, buf->infile);\n \treturn ferror(buf->infile) ? NULL : buf->blob_buffer.buf;\n }\n \n-void buffer_copy_bytes(uint32_t len)\n+void buffer_copy_bytes(struct line_buffer *buf, uint32_t len)\n {\n \tchar byte_buffer[COPY_BUFFER_LEN];\n \tuint32_t in;\n@@ -70,13 +66,13 @@ void buffer_copy_bytes(uint32_t len)\n \t\tlen -= in;\n \t\tfwrite(byte_buffer, 1, in, stdout);\n \t\tif (ferror(stdout)) {\n-\t\t\tbuffer_skip_bytes(len);\n+\t\t\tbuffer_skip_bytes(buf, len);\n \t\t\treturn;\n \t\t}\n \t}\n }\n \n-void buffer_skip_bytes(uint32_t len)\n+void buffer_skip_bytes(struct line_buffer *buf, uint32_t len)\n {\n \tchar byte_buffer[COPY_BUFFER_LEN];\n \tuint32_t in;\n@@ -87,7 +83,7 @@ void buffer_skip_bytes(uint32_t len)\n \t}\n }\n \n-void buffer_reset(void)\n+void buffer_reset(struct line_buffer *buf)\n {\n \tstrbuf_release(&buf->blob_buffer);\n }\ndiff --git a/vcs-svn/line_buffer.h b/vcs-svn/line_buffer.h\nindex 4ae1133..fb37390 100644\n--- a/vcs-svn/line_buffer.h\n+++ b/vcs-svn/line_buffer.h\n@@ -12,12 +12,12 @@ struct line_buffer {\n };\n #define LINE_BUFFER_INIT {\"\", STRBUF_INIT, NULL}\n \n-int buffer_init(const char *filename);\n-int buffer_deinit(void);\n-char *buffer_read_line(void);\n-char *buffer_read_string(uint32_t len);\n-void buffer_copy_bytes(uint32_t len);\n-void buffer_skip_bytes(uint32_t len);\n-void buffer_reset(void);\n+int buffer_init(struct line_buffer *buf, const char *filename);\n+int buffer_deinit(struct line_buffer *buf);\n+char *buffer_read_line(struct line_buffer *buf);\n+char *buffer_read_string(struct line_buffer *buf, uint32_t len);\n+void buffer_copy_bytes(struct line_buffer *buf, uint32_t len);\n+void buffer_skip_bytes(struct line_buffer *buf, uint32_t len);\n+void buffer_reset(struct line_buffer *buf);\n \n #endif\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex 53d0215..3bba0fe 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -30,6 +30,8 @@\n /* Create memory pool for log messages */\n obj_pool_gen(log, char, 4096)\n \n+static struct line_buffer input = LINE_BUFFER_INIT;\n+\n static char* log_copy(uint32_t length, char *log)\n {\n \tchar *buffer;\n@@ -113,14 +115,14 @@ static void read_props(void)\n \tuint32_t key = ~0;\n \tchar *val = NULL;\n \tchar *t;\n-\twhile ((t = buffer_read_line()) && strcmp(t, \"PROPS-END\")) {\n+\twhile ((t = buffer_read_line(&input)) && strcmp(t, \"PROPS-END\")) {\n \t\tif (!strncmp(t, \"K \", 2)) {\n \t\t\tlen = atoi(&t[2]);\n-\t\t\tkey = pool_intern(buffer_read_string(len));\n-\t\t\tbuffer_read_line();\n+\t\t\tkey = pool_intern(buffer_read_string(&input, len));\n+\t\t\tbuffer_read_line(&input);\n \t\t} else if (!strncmp(t, \"V \", 2)) {\n \t\t\tlen = atoi(&t[2]);\n-\t\t\tval = buffer_read_string(len);\n+\t\t\tval = buffer_read_string(&input, len);\n \t\t\tif (key == keys.svn_log) {\n \t\t\t\t/* Value length excludes terminating nul. */\n \t\t\t\trev_ctx.log = log_copy(len + 1, val);\n@@ -135,7 +137,7 @@ static void read_props(void)\n \t\t\t\tnode_ctx.type = REPO_MODE_LNK;\n \t\t\t}\n \t\t\tkey = ~0;\n-\t\t\tbuffer_read_line();\n+\t\t\tbuffer_read_line(&input);\n \t\t}\n \t}\n }\n@@ -177,9 +179,10 @@ static void handle_node(void)\n \t\tnode_ctx.type = node_ctx.srcMode;\n \n \tif (node_ctx.mark)\n-\t\tfast_export_blob(node_ctx.type, node_ctx.mark, node_ctx.textLength);\n+\t\tfast_export_blob(node_ctx.type,\n+\t\t\t\t node_ctx.mark, node_ctx.textLength, &input);\n \telse if (node_ctx.textLength != LENGTH_UNKNOWN)\n-\t\tbuffer_skip_bytes(node_ctx.textLength);\n+\t\tbuffer_skip_bytes(&input, node_ctx.textLength);\n }\n \n static void handle_revision(void)\n@@ -198,7 +201,7 @@ void svndump_read(const char *url)\n \tuint32_t key;\n \n \treset_dump_ctx(pool_intern(url));\n-\twhile ((t = buffer_read_line())) {\n+\twhile ((t = buffer_read_line(&input))) {\n \t\tval = strstr(t, \": \");\n \t\tif (!val)\n \t\t\tcontinue;\n@@ -250,7 +253,7 @@ void svndump_read(const char *url)\n \t\t\tnode_ctx.propLength = atoi(val);\n \t\t} else if (key == keys.content_length) {\n \t\t\tlen = atoi(val);\n-\t\t\tbuffer_read_line();\n+\t\t\tbuffer_read_line(&input);\n \t\t\tif (active_ctx == REV_CTX) {\n \t\t\t\tread_props();\n \t\t\t} else if (active_ctx == NODE_CTX) {\n@@ -258,7 +261,7 @@ void svndump_read(const char *url)\n \t\t\t\tactive_ctx = REV_CTX;\n \t\t\t} else {\n \t\t\t\tfprintf(stderr, \"Unexpected content length header: %\"PRIu32\"\\n\", len);\n-\t\t\t\tbuffer_skip_bytes(len);\n+\t\t\t\tbuffer_skip_bytes(&input, len);\n \t\t\t}\n \t\t}\n \t}\n@@ -270,7 +273,7 @@ void svndump_read(const char *url)\n \n void svndump_init(const char *filename)\n {\n-\tbuffer_init(filename);\n+\tbuffer_init(&input, filename);\n \trepo_init();\n \treset_dump_ctx(~0);\n \treset_rev_ctx(0);\n@@ -285,7 +288,7 @@ void svndump_deinit(void)\n \treset_dump_ctx(~0);\n \treset_rev_ctx(0);\n \treset_node_ctx(NULL);\n-\tif (buffer_deinit())\n+\tif (buffer_deinit(&input))\n \t\tfprintf(stderr, \"Input error\\n\");\n \tif (ferror(stdout))\n \t\tfprintf(stderr, \"Output error\\n\");\n@@ -294,7 +297,7 @@ void svndump_deinit(void)\n void svndump_reset(void)\n {\n \tlog_reset();\n-\tbuffer_reset();\n+\tbuffer_reset(&input);\n \trepo_reset();\n \treset_dump_ctx(~0);\n \treset_rev_ctx(0);\ndiff --git a/test-line-buffer.c b/test-line-buffer.c\nindex c11bf7f..f9af892 100644\n--- a/test-line-buffer.c\n+++ b/test-line-buffer.c\n@@ -22,25 +22,26 @@ static uint32_t strtouint32(const char *s)\n \n int main(int argc, char *argv[])\n {\n+\tstruct line_buffer buf = LINE_BUFFER_INIT;\n \tchar *s;\n \n \tif (argc != 1)\n \t\tusage(\"test-line-buffer < input.txt\");\n-\tif (buffer_init(NULL))\n+\tif (buffer_init(&buf, NULL))\n \t\tdie_errno(\"open error\");\n-\twhile ((s = buffer_read_line())) {\n-\t\ts = buffer_read_string(strtouint32(s));\n+\twhile ((s = buffer_read_line(&buf))) {\n+\t\ts = buffer_read_string(&buf, strtouint32(s));\n \t\tfputs(s, stdout);\n \t\tfputc('\\n', stdout);\n-\t\tbuffer_skip_bytes(1);\n-\t\tif (!(s = buffer_read_line()))\n+\t\tbuffer_skip_bytes(&buf, 1);\n+\t\tif (!(s = buffer_read_line(&buf)))\n \t\t\tbreak;\n-\t\tbuffer_copy_bytes(strtouint32(s) + 1);\n+\t\tbuffer_copy_bytes(&buf, strtouint32(s) + 1);\n \t}\n-\tif (buffer_deinit())\n+\tif (buffer_deinit(&buf))\n \t\tdie(\"input error\");\n \tif (ferror(stdout))\n \t\tdie(\"output error\");\n-\tbuffer_reset();\n+\tbuffer_reset(&buf);\n \treturn 0;\n }\n-- \n1.7.2.3\n"},{"id":"153206","messageId":"20101011024421.GE1553@burratino","threadId":"24407","inReplyTo":"20101011023435.GA706@burratino","subject":"[PATCH 05/16] vcs-svn: Make buffer_skip_bytes() report partial reads","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-11T02:44:21Z","receivedAt":"2010-10-11T02:44:21Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Tell the caller know how many bytes were actually skipped.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n vcs-svn/line_buffer.txt |    3 ++-\n vcs-svn/line_buffer.c   |   15 ++++++++-------\n vcs-svn/line_buffer.h   |    2 +-\n 3 files changed, 11 insertions(+), 9 deletions(-)\n\ndiff --git a/vcs-svn/line_buffer.txt b/vcs-svn/line_buffer.txt\nindex f8eaa4d..d06db24 100644\n--- a/vcs-svn/line_buffer.txt\n+++ b/vcs-svn/line_buffer.txt\n@@ -53,7 +53,8 @@ Functions\n \n `buffer_skip_bytes`::\n \tDiscards `len` bytes from the input stream (stopping early\n-\tif necessary because of an error or eof).\n+\tif necessary because of an error or eof).  Return value is\n+\tthe number of bytes successfully read.\n \n `buffer_reset`::\n \tDeallocates non-static buffers.\ndiff --git a/vcs-svn/line_buffer.c b/vcs-svn/line_buffer.c\nindex 806932b..999368b 100644\n--- a/vcs-svn/line_buffer.c\n+++ b/vcs-svn/line_buffer.c\n@@ -72,15 +72,16 @@ void buffer_copy_bytes(struct line_buffer *buf, uint32_t len)\n \t}\n }\n \n-void buffer_skip_bytes(struct line_buffer *buf, uint32_t len)\n+uint32_t buffer_skip_bytes(struct line_buffer *buf, uint32_t nbytes)\n {\n-\tchar byte_buffer[COPY_BUFFER_LEN];\n-\tuint32_t in;\n-\twhile (len > 0 && !feof(buf->infile) && !ferror(buf->infile)) {\n-\t\tin = len < COPY_BUFFER_LEN ? len : COPY_BUFFER_LEN;\n-\t\tin = fread(byte_buffer, 1, in, buf->infile);\n-\t\tlen -= in;\n+\tuint32_t done = 0;\n+\twhile (done < nbytes && !feof(buf->infile) && !ferror(buf->infile)) {\n+\t\tchar byte_buffer[COPY_BUFFER_LEN];\n+\t\tuint32_t len = nbytes - done;\n+\t\tuint32_t in = len < COPY_BUFFER_LEN ? len : COPY_BUFFER_LEN;\n+\t\tdone += fread(byte_buffer, 1, in, buf->infile);\n \t}\n+\treturn done;\n }\n \n void buffer_reset(struct line_buffer *buf)\ndiff --git a/vcs-svn/line_buffer.h b/vcs-svn/line_buffer.h\nindex fb37390..2796ba7 100644\n--- a/vcs-svn/line_buffer.h\n+++ b/vcs-svn/line_buffer.h\n@@ -17,7 +17,7 @@ int buffer_deinit(struct line_buffer *buf);\n char *buffer_read_line(struct line_buffer *buf);\n char *buffer_read_string(struct line_buffer *buf, uint32_t len);\n void buffer_copy_bytes(struct line_buffer *buf, uint32_t len);\n-void buffer_skip_bytes(struct line_buffer *buf, uint32_t len);\n+uint32_t buffer_skip_bytes(struct line_buffer *buf, uint32_t len);\n void buffer_reset(struct line_buffer *buf);\n \n #endif\n-- \n1.7.2.3\n"},{"id":"153207","messageId":"20101011024624.GF1553@burratino","threadId":"24407","inReplyTo":"20101011023435.GA706@burratino","subject":"[PATCH 06/16] vcs-svn: Improve support for reading large files","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-11T02:46:24Z","receivedAt":"2010-10-11T02:46:24Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Tweak the line_buffer API to permit seeking and cat-ing segments\nlonger than 4 GiB.  This would be particularly useful for applying\ndeltas that remove a large segment from the middle of a file.\n\nCallers would still have to be updated to take advantage of this.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\nSince off_t is a signed type, on systems with 32-bit file offsets,\nthis might make things worse.  Is that worth worrying about?\n\n vcs-svn/line_buffer.c |    8 ++++----\n vcs-svn/line_buffer.h |    4 ++--\n 2 files changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/vcs-svn/line_buffer.c b/vcs-svn/line_buffer.c\nindex 999368b..fd1d3c3 100644\n--- a/vcs-svn/line_buffer.c\n+++ b/vcs-svn/line_buffer.c\n@@ -56,7 +56,7 @@ char *buffer_read_string(struct line_buffer *buf, uint32_t len)\n \treturn ferror(buf->infile) ? NULL : buf->blob_buffer.buf;\n }\n \n-void buffer_copy_bytes(struct line_buffer *buf, uint32_t len)\n+void buffer_copy_bytes(struct line_buffer *buf, off_t len)\n {\n \tchar byte_buffer[COPY_BUFFER_LEN];\n \tuint32_t in;\n@@ -72,12 +72,12 @@ void buffer_copy_bytes(struct line_buffer *buf, uint32_t len)\n \t}\n }\n \n-uint32_t buffer_skip_bytes(struct line_buffer *buf, uint32_t nbytes)\n+off_t buffer_skip_bytes(struct line_buffer *buf, off_t nbytes)\n {\n-\tuint32_t done = 0;\n+\toff_t done = 0;\n \twhile (done < nbytes && !feof(buf->infile) && !ferror(buf->infile)) {\n \t\tchar byte_buffer[COPY_BUFFER_LEN];\n-\t\tuint32_t len = nbytes - done;\n+\t\toff_t len = nbytes - done;\n \t\tuint32_t in = len < COPY_BUFFER_LEN ? len : COPY_BUFFER_LEN;\n \t\tdone += fread(byte_buffer, 1, in, buf->infile);\n \t}\ndiff --git a/vcs-svn/line_buffer.h b/vcs-svn/line_buffer.h\nindex 2796ba7..2849faa 100644\n--- a/vcs-svn/line_buffer.h\n+++ b/vcs-svn/line_buffer.h\n@@ -16,8 +16,8 @@ int buffer_init(struct line_buffer *buf, const char *filename);\n int buffer_deinit(struct line_buffer *buf);\n char *buffer_read_line(struct line_buffer *buf);\n char *buffer_read_string(struct line_buffer *buf, uint32_t len);\n-void buffer_copy_bytes(struct line_buffer *buf, uint32_t len);\n-uint32_t buffer_skip_bytes(struct line_buffer *buf, uint32_t len);\n+void buffer_copy_bytes(struct line_buffer *buf, off_t len);\n+off_t buffer_skip_bytes(struct line_buffer *buf, off_t len);\n void buffer_reset(struct line_buffer *buf);\n \n #endif\n-- \n1.7.2.3\n"},{"id":"153208","messageId":"20101011024702.GG1553@burratino","threadId":"24407","inReplyTo":"20101011023435.GA706@burratino","subject":"[PATCH 07/16] vcs-svn: Add binary-safe read() function","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-11T02:47:02Z","receivedAt":"2010-10-11T02:47:02Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"buffer_read_binary() writes to a strbuf so the caller does not need\nto keep track of the number of bytes read.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n vcs-svn/line_buffer.c |    6 ++++++\n vcs-svn/line_buffer.h |    1 +\n 2 files changed, 7 insertions(+), 0 deletions(-)\n\ndiff --git a/vcs-svn/line_buffer.c b/vcs-svn/line_buffer.c\nindex fd1d3c3..6dd0189 100644\n--- a/vcs-svn/line_buffer.c\n+++ b/vcs-svn/line_buffer.c\n@@ -56,6 +56,12 @@ char *buffer_read_string(struct line_buffer *buf, uint32_t len)\n \treturn ferror(buf->infile) ? NULL : buf->blob_buffer.buf;\n }\n \n+void buffer_read_binary(struct strbuf *sb, uint32_t size,\n+\t\t\tstruct line_buffer *buf)\n+{\n+\tstrbuf_fread(sb, size, buf->infile);\n+}\n+\n void buffer_copy_bytes(struct line_buffer *buf, off_t len)\n {\n \tchar byte_buffer[COPY_BUFFER_LEN];\ndiff --git a/vcs-svn/line_buffer.h b/vcs-svn/line_buffer.h\nindex 2849faa..873b0e4 100644\n--- a/vcs-svn/line_buffer.h\n+++ b/vcs-svn/line_buffer.h\n@@ -16,6 +16,7 @@ int buffer_init(struct line_buffer *buf, const char *filename);\n int buffer_deinit(struct line_buffer *buf);\n char *buffer_read_line(struct line_buffer *buf);\n char *buffer_read_string(struct line_buffer *buf, uint32_t len);\n+void buffer_read_binary(struct strbuf *sb, uint32_t len, struct line_buffer *f);\n void buffer_copy_bytes(struct line_buffer *buf, off_t len);\n off_t buffer_skip_bytes(struct line_buffer *buf, off_t len);\n void buffer_reset(struct line_buffer *buf);\n-- \n1.7.2.3\n"},{"id":"153209","messageId":"20101011024754.GH1553@burratino","threadId":"24407","inReplyTo":"20101011023435.GA706@burratino","subject":"[PATCH 08/16] vcs-svn: Let callers peek ahead to find stream end","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-11T02:47:54Z","receivedAt":"2010-10-11T02:47:54Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"The buffer_at_eof() function returns 1 if and only if all input from\nthe input stream has been exhausted (because of EOF or error).  The\nimplementation calls fgetc() followed by ungetc() to force an EOF\ncondition when there is no more input remaining.\n\nLike many functions in the line_buffer API, this function is not\nthread-safe.  It could be made to be so with a mutex if needed.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n vcs-svn/line_buffer.c |   10 ++++++++++\n vcs-svn/line_buffer.h |    1 +\n 2 files changed, 11 insertions(+), 0 deletions(-)\n\ndiff --git a/vcs-svn/line_buffer.c b/vcs-svn/line_buffer.c\nindex 6dd0189..19caa21 100644\n--- a/vcs-svn/line_buffer.c\n+++ b/vcs-svn/line_buffer.c\n@@ -27,6 +27,16 @@ int buffer_deinit(struct line_buffer *buf)\n \treturn err;\n }\n \n+int buffer_at_eof(struct line_buffer *buf)\n+{\n+\tint ch;\n+\tif ((ch = fgetc(buf->infile)) == EOF)\n+\t\treturn 1;\n+\tif (ungetc(ch, buf->infile) == EOF)\n+\t\treturn error(\"cannot unget %c: %s\\n\", ch, strerror(errno));\n+\treturn 0;\n+}\n+\n /* Read a line without trailing newline. */\n char *buffer_read_line(struct line_buffer *buf)\n {\ndiff --git a/vcs-svn/line_buffer.h b/vcs-svn/line_buffer.h\nindex 873b0e4..0269aed 100644\n--- a/vcs-svn/line_buffer.h\n+++ b/vcs-svn/line_buffer.h\n@@ -14,6 +14,7 @@ struct line_buffer {\n \n int buffer_init(struct line_buffer *buf, const char *filename);\n int buffer_deinit(struct line_buffer *buf);\n+int buffer_at_eof(struct line_buffer *buf);\n char *buffer_read_line(struct line_buffer *buf);\n char *buffer_read_string(struct line_buffer *buf, uint32_t len);\n void buffer_read_binary(struct strbuf *sb, uint32_t len, struct line_buffer *f);\n-- \n1.7.2.3\n"},{"id":"153210","messageId":"20101011025121.GI1553@burratino","threadId":"24407","inReplyTo":"20101011023435.GA706@burratino","subject":"[PATCH 09/16] vcs-svn: Allow input errors to be detected early","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-11T02:51:21Z","receivedAt":"2010-10-11T02:51:21Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Add a buffer_ferror() function to read the error flag from the input\nstream, so callers can do:\n\n\tsome_error_prone_operation(f, ...);\n\tif (buffer_ferror(f))\n\t\treturn error(\"input error: %s\", strerror(errno));\n\ninstead of waiting until it is time to close the file.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n vcs-svn/line_buffer.c |    5 +++++\n vcs-svn/line_buffer.h |    1 +\n 2 files changed, 6 insertions(+), 0 deletions(-)\n\ndiff --git a/vcs-svn/line_buffer.c b/vcs-svn/line_buffer.c\nindex 19caa21..43da509 100644\n--- a/vcs-svn/line_buffer.c\n+++ b/vcs-svn/line_buffer.c\n@@ -27,6 +27,11 @@ int buffer_deinit(struct line_buffer *buf)\n \treturn err;\n }\n \n+int buffer_ferror(struct line_buffer *buf)\n+{\n+\treturn ferror(buf->infile);\n+}\n+\n int buffer_at_eof(struct line_buffer *buf)\n {\n \tint ch;\ndiff --git a/vcs-svn/line_buffer.h b/vcs-svn/line_buffer.h\nindex 0269aed..4899289 100644\n--- a/vcs-svn/line_buffer.h\n+++ b/vcs-svn/line_buffer.h\n@@ -14,6 +14,7 @@ struct line_buffer {\n \n int buffer_init(struct line_buffer *buf, const char *filename);\n int buffer_deinit(struct line_buffer *buf);\n+int buffer_ferror(struct line_buffer *buf);\n int buffer_at_eof(struct line_buffer *buf);\n char *buffer_read_line(struct line_buffer *buf);\n char *buffer_read_string(struct line_buffer *buf, uint32_t len);\n-- \n1.7.2.3\n"},{"id":"153211","messageId":"20101011025217.GJ1553@burratino","threadId":"24407","inReplyTo":"20101011023435.GA706@burratino","subject":"[PATCH 10/16] vcs-svn: Allow character-oriented input","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-11T02:52:17Z","receivedAt":"2010-10-11T02:52:17Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"buffer_read_char() can be used in place of buffer_read_string(1)\nto avoid consuming valuable static buffer space.  The delta applier\nwill use this to read variable-length integers one byte at a time.\n\nUnderneath, it is fgetc(), wrapped so the line_buffer library can\nmaintain its role as gatekeeper of input.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n vcs-svn/line_buffer.c |    5 +++++\n vcs-svn/line_buffer.h |    1 +\n 2 files changed, 6 insertions(+), 0 deletions(-)\n\ndiff --git a/vcs-svn/line_buffer.c b/vcs-svn/line_buffer.c\nindex 43da509..c54031b 100644\n--- a/vcs-svn/line_buffer.c\n+++ b/vcs-svn/line_buffer.c\n@@ -42,6 +42,11 @@ int buffer_at_eof(struct line_buffer *buf)\n \treturn 0;\n }\n \n+int buffer_read_char(struct line_buffer *buf)\n+{\n+\treturn fgetc(buf->infile);\n+}\n+\n /* Read a line without trailing newline. */\n char *buffer_read_line(struct line_buffer *buf)\n {\ndiff --git a/vcs-svn/line_buffer.h b/vcs-svn/line_buffer.h\nindex 4899289..2375ee1 100644\n--- a/vcs-svn/line_buffer.h\n+++ b/vcs-svn/line_buffer.h\n@@ -18,6 +18,7 @@ int buffer_ferror(struct line_buffer *buf);\n int buffer_at_eof(struct line_buffer *buf);\n char *buffer_read_line(struct line_buffer *buf);\n char *buffer_read_string(struct line_buffer *buf, uint32_t len);\n+int buffer_read_char(struct line_buffer *buf);\n void buffer_read_binary(struct strbuf *sb, uint32_t len, struct line_buffer *f);\n void buffer_copy_bytes(struct line_buffer *buf, off_t len);\n off_t buffer_skip_bytes(struct line_buffer *buf, off_t len);\n-- \n1.7.2.3\n"},{"id":"153212","messageId":"20101011025341.GK1553@burratino","threadId":"24407","inReplyTo":"20101011023435.GA706@burratino","subject":"[PATCH 11/16] vcs-svn: Add code to maintain a sliding view of a file","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-11T02:53:41Z","receivedAt":"2010-10-11T02:53:41Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Subversion's delta format has the convenient property that applying\neach section of the delta only requires examining (and keeping in\nmemory) a small portion of the preimage.  At any moment, this portion\nbegins at a well-defined file offset and has a well-defined length,\nand as the delta is applied, it moves from the beginning to the end\nof the file.  Add a move_window() function to keep track of such a\nwindow into a file.\n\nYou can use it like this:\n\n\tstruct line_buffer preimage = LINE_BUFFER_INIT;\n\tbuffer_init(&preimage, NULL);\n\tstruct view window = {&preimage, 0, STRBUF_INIT};\n\tmove_window(&window, 3, 7);\t/* (1) */\n\tmove_window(&window, 5, 5);\t/* (2) */\n\tmove_window(&window, 12, 2);\t/* (3) */\n\tstrbuf_release(&window.buf);\n\tbuffer_deinit(&preimage);\n\nIn this example: (1) reads 10 bytes and discards the first 3;\n(2) discards the first 2, which are not needed any more; and (3)\nskips 2 bytes and reads 2 new bytes to work with.\n\nWhenever move_window() returns, the file position indicator is at\nposition window->off + window->buf.len and the data from positions\nwindow->off to the current file position are stored in window->buf.\n\nThis function does only sequential access and never seeks, so it\ncan be safely used on pipes and sockets.\n\nOn end-of-file, move_window() just silently reads less than the\ncaller requested.  On other errors, it prints a message to stderr\nand returns -1.\n\nHelped-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n Makefile                 |    5 ++-\n vcs-svn/sliding_window.c |   65 ++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/sliding_window.h |   14 ++++++++++\n vcs-svn/LICENSE          |    2 +\n 4 files changed, 84 insertions(+), 2 deletions(-)\n create mode 100644 vcs-svn/sliding_window.c\n create mode 100644 vcs-svn/sliding_window.h\n\ndiff --git a/Makefile b/Makefile\nindex 1f1ce04..d99da33 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1765,7 +1765,8 @@ endif\n XDIFF_OBJS = xdiff/xdiffi.o xdiff/xprepare.o xdiff/xutils.o xdiff/xemit.o \\\n \txdiff/xmerge.o xdiff/xpatience.o\n VCSSVN_OBJS = vcs-svn/string_pool.o vcs-svn/line_buffer.o \\\n-\tvcs-svn/repo_tree.o vcs-svn/fast_export.o vcs-svn/svndump.o\n+\tvcs-svn/repo_tree.o vcs-svn/fast_export.o vcs-svn/svndump.o \\\n+\tvcs-svn/sliding_window.o\n OBJECTS := $(GIT_OBJS) $(XDIFF_OBJS) $(VCSSVN_OBJS)\n \n dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d)\n@@ -1892,7 +1893,7 @@ xdiff-interface.o $(XDIFF_OBJS): \\\n $(VCSSVN_OBJS): \\\n \tvcs-svn/obj_pool.h vcs-svn/trp.h vcs-svn/string_pool.h \\\n \tvcs-svn/line_buffer.h vcs-svn/repo_tree.h vcs-svn/fast_export.h \\\n-\tvcs-svn/svndump.h\n+\tvcs-svn/svndump.h vcs-svn/sliding_window.h\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\ndiff --git a/vcs-svn/sliding_window.c b/vcs-svn/sliding_window.c\nnew file mode 100644\nindex 0000000..8273970\n--- /dev/null\n+++ b/vcs-svn/sliding_window.c\n@@ -0,0 +1,65 @@\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"git-compat-util.h\"\n+#include \"sliding_window.h\"\n+#include \"line_buffer.h\"\n+#include \"strbuf.h\"\n+\n+static void strbuf_remove_from_left(struct strbuf *sb, size_t nbytes)\n+{\n+\tassert(nbytes <= sb->len);\n+\tmemmove(sb->buf, sb->buf + nbytes, sb->len - nbytes);\n+\tstrbuf_setlen(sb, sb->len - nbytes);\n+}\n+\n+static int check_overflow(off_t a, size_t b)\n+{\n+\tif ((off_t) b < 0)\n+\t\treturn error(\"Unrepresentable length: \"\n+\t\t\t\t\"%\"PRIu64\" > OFF_MAX\", (uint64_t) b);\n+\tif (signed_add_overflows(a, (off_t) b))\n+\t\treturn error(\"Unrepresentable offset: \"\n+\t\t\t\t\"%\"PRIu64\" + %\"PRIu64\" > OFF_MAX\",\n+\t\t\t\t(uint64_t) a, (uint64_t) b);\n+\treturn 0;\n+}\n+\n+int move_window(struct view *view, off_t off, size_t len)\n+{\n+\toff_t file_offset;\n+\tassert(view && view->file);\n+\tassert(!check_overflow(view->off, view->buf.len));\n+\n+\tif (check_overflow(off, len))\n+\t\treturn -1;\n+\tif (off < view->off || off + len < view->off + view->buf.len)\n+\t\treturn error(\"Invalid delta: window slides left\");\n+\n+\tfile_offset = view->off + view->buf.len;\n+\tif (off < file_offset)\n+\t\t/* Move the overlapping region into place. */\n+\t\tstrbuf_remove_from_left(&view->buf, off - view->off);\n+\telse\n+\t\tstrbuf_setlen(&view->buf, 0);\n+\tif (off > file_offset) {\n+\t\t/* Seek ahead to skip the gap. */\n+\t\tconst off_t gap = off - file_offset;\n+\t\tconst off_t nread = buffer_skip_bytes(view->file, gap);\n+\t\tif (nread != gap) {\n+\t\t\tif (!buffer_ferror(view->file))\t/* View ends early. */\n+\t\t\t\tgoto done;\n+\t\t\treturn error(\"Cannot seek forward in input: %s\",\n+\t\t\t\t     strerror(errno));\n+\t\t}\n+\t\tfile_offset += gap;\n+\t}\n+\tbuffer_read_binary(&view->buf, len - view->buf.len, view->file);\n+\tif (buffer_ferror(view->file))\n+\t\treturn error(\"Cannot read preimage: %s\", strerror(errno));\n+ done:\n+\tview->off = off;\n+\treturn 0;\n+}\ndiff --git a/vcs-svn/sliding_window.h b/vcs-svn/sliding_window.h\nnew file mode 100644\nindex 0000000..b9f0552\n--- /dev/null\n+++ b/vcs-svn/sliding_window.h\n@@ -0,0 +1,14 @@\n+#ifndef SLIDING_WINDOW_H_\n+#define SLIDING_WINDOW_H_\n+\n+#include \"strbuf.h\"\n+\n+struct view {\n+\tstruct line_buffer *file;\n+\toff_t off;\n+\tstruct strbuf buf;\n+};\n+\n+extern int move_window(struct view *view, off_t off, size_t len);\n+\n+#endif\ndiff --git a/vcs-svn/LICENSE b/vcs-svn/LICENSE\nindex 0a5e3c4..805882c 100644\n--- a/vcs-svn/LICENSE\n+++ b/vcs-svn/LICENSE\n@@ -1,6 +1,8 @@\n Copyright (C) 2010 David Barr <david.barr@cordelta.com>.\n All rights reserved.\n \n+Copyright (C) 2010 Jonathan Nieder <jrnieder@gmail.com>.\n+\n Copyright (C) 2008 Jason Evans <jasone@canonware.com>.\n All rights reserved.\n \n-- \n1.7.2.3\n"},{"id":"153213","messageId":"20101011025556.GL1553@burratino","threadId":"24407","inReplyTo":"20101011023435.GA706@burratino","subject":"[PATCH 12/16] vcs-svn: Learn to parse variable-length integers","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-11T02:55:56Z","receivedAt":"2010-10-11T02:55:56Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"The humble beginnings of the svn-format delta applier.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\nMaybe this should be squashed with patch 16.\n\nIdeas for eliminating the code duplication?\n\n vcs-svn/svndiff.c |   59 +++++++++++++++++++++++++++++++++++++++++++++++++++++\n 1 files changed, 59 insertions(+), 0 deletions(-)\n create mode 100644 vcs-svn/svndiff.c\n\ndiff --git a/vcs-svn/svndiff.c b/vcs-svn/svndiff.c\nnew file mode 100644\nindex 0000000..4d122a5\n--- /dev/null\n+++ b/vcs-svn/svndiff.c\n@@ -0,0 +1,59 @@\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"git-compat-util.h\"\n+#include \"line_buffer.h\"\n+\n+/*\n+ * svndiff0 applier\n+ *\n+ * See http://svn.apache.org/repos/asf/subversion/trunk/notes/svndiff.\n+ *\n+ * int ::= highdigit* lowdigit;\n+ * highdigit ::= # binary 1000 0000 OR-ed with 7 bit value;\n+ * lowdigit ::= # 7 bit value;\n+ */\n+\n+#define VLI_CONTINUE\t0x80\n+#define VLI_DIGIT_MASK\t0x7f\n+#define VLI_BITS_PER_DIGIT 7\n+\n+static int read_int(struct line_buffer *in, uintmax_t *result, off_t *len)\n+{\n+\toff_t sz = *len;\n+\tuintmax_t rv = 0;\n+\twhile (sz) {\n+\t\tint ch = buffer_read_char(in);\n+\t\tif (ch == EOF)\n+\t\t\tbreak;\n+\t\tsz--;\n+\t\trv <<= VLI_BITS_PER_DIGIT;\n+\t\trv += (ch & VLI_DIGIT_MASK);\n+\t\tif (!(ch & VLI_CONTINUE)) {\n+\t\t\t*result = rv;\n+\t\t\t*len = sz;\n+\t\t\treturn 0;\n+\t\t}\n+\t}\n+\treturn error(\"Invalid delta: incomplete integer %\"PRIuMAX, rv);\n+}\n+\n+static int parse_int(const char **buf, size_t *result, const char *end)\n+{\n+\tconst char *pos;\n+\tsize_t rv = 0;\n+\tfor (pos = *buf; pos != end; pos++) {\n+\t\tunsigned char ch = *pos;\n+\t\trv <<= VLI_BITS_PER_DIGIT;\n+\t\trv += (ch & VLI_DIGIT_MASK);\n+\t\tif (!(ch & VLI_CONTINUE)) {\n+\t\t\t*result = rv;\n+\t\t\t*buf = pos + 1;\n+\t\t\treturn 0;\n+\t\t}\n+\t}\n+\treturn error(\"Invalid instruction: incomplete integer %\"PRIu64,\n+\t\t     (uint64_t) rv);\n+}\n-- \n1.7.2.3\n"},{"id":"153214","messageId":"20101011025800.GM1553@burratino","threadId":"24407","inReplyTo":"20101011023435.GA706@burratino","subject":"[PATCH 13/16] vcs-svn: Learn to check for SVN\\0 magic","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-11T02:58:00Z","receivedAt":"2010-10-11T02:58:00Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"The magic number of svn deltas is SVN followed by a null byte.\nAn alternative format (with compressed text) uses magic number SVN\\1,\nbut that is deprecated in favor of compressing the deltas as a whole\nas far as I can tell.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n vcs-svn/svndiff.c |   18 ++++++++++++++++++\n 1 files changed, 18 insertions(+), 0 deletions(-)\n\ndiff --git a/vcs-svn/svndiff.c b/vcs-svn/svndiff.c\nindex 4d122a5..df0b1a2 100644\n--- a/vcs-svn/svndiff.c\n+++ b/vcs-svn/svndiff.c\n@@ -11,6 +11,7 @@\n  *\n  * See http://svn.apache.org/repos/asf/subversion/trunk/notes/svndiff.\n  *\n+ * svndiff0 ::= 'SVN\\0' window window*;\n  * int ::= highdigit* lowdigit;\n  * highdigit ::= # binary 1000 0000 OR-ed with 7 bit value;\n  * lowdigit ::= # 7 bit value;\n@@ -20,6 +21,23 @@\n #define VLI_DIGIT_MASK\t0x7f\n #define VLI_BITS_PER_DIGIT 7\n \n+static int read_magic(struct line_buffer *in, off_t *len)\n+{\n+\tstatic const char magic[] = {'S', 'V', 'N', '\\0'};\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tif (*len < sizeof(magic))\n+\t\treturn error(\"Invalid delta: no file type header\");\n+\tbuffer_read_binary(&sb, sizeof(magic), in);\n+\tif (sb.len != sizeof(magic))\n+\t\treturn error(\"Invalid delta: no file type header\");\n+\tif (memcmp(sb.buf, magic, sizeof(magic)))\n+\t\treturn error(\"Unrecognized file type %.*s\",\n+\t\t\t     (int) sizeof(magic), sb.buf);\n+\t*len -= sizeof(magic);\n+\tstrbuf_release(&sb);\n+\treturn 0;\n+}\n+\n static int read_int(struct line_buffer *in, uintmax_t *result, off_t *len)\n {\n \toff_t sz = *len;\n-- \n1.7.2.3\n"},{"id":"153215","messageId":"20101011025926.GN1553@burratino","threadId":"24407","inReplyTo":"20101011023435.GA706@burratino","subject":"[PATCH 14/16] compat: helper for detecting unsigned overflow","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-11T02:59:26Z","receivedAt":"2010-10-11T02:59:26Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"The idiom (a + b < a) works fine for detecting that an unsigned\ninteger has overflowed, but the more explicit\n\n\tunsigned_add_overflows(a, b)\n\nmight be easier to read.\n\nDefine such a macro, expanding roughly to ((a) < UINT_MAX - (b)).\nBecause the expansion uses each argument only once outside of sizeof()\nexpressions, it is safe to use this macro with arguments that have\nside-effects.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n git-compat-util.h |    6 ++++++\n 1 files changed, 6 insertions(+), 0 deletions(-)\n\ndiff --git a/git-compat-util.h b/git-compat-util.h\nindex 2af8d3e..817f045 100644\n--- a/git-compat-util.h\n+++ b/git-compat-util.h\n@@ -31,6 +31,9 @@\n #define maximum_signed_value_of_type(a) \\\n     (INTMAX_MAX >> (bitsizeof(intmax_t) - bitsizeof(a)))\n \n+#define maximum_unsigned_value_of_type(a) \\\n+    (UINTMAX_MAX >> (bitsizeof(uintmax_t) - bitsizeof(a)))\n+\n /*\n  * Signed integer overflow is undefined in C, so here's a helper macro\n  * to detect if the sum of two integers will overflow.\n@@ -40,6 +43,9 @@\n #define signed_add_overflows(a, b) \\\n     ((b) > maximum_signed_value_of_type(a) - (a))\n \n+#define unsigned_add_overflows(a, b) \\\n+    ((b) > maximum_unsigned_value_of_type(a) - (a))\n+\n #ifdef __GNUC__\n #define TYPEOF(x) (__typeof__(x))\n #else\n-- \n1.7.2.3\n"},{"id":"153216","messageId":"20101011030040.GO1553@burratino","threadId":"24407","inReplyTo":"20101011023435.GA706@burratino","subject":"[PATCH 15/16] t9010 (svn-fe): Eliminate dependency on svn perl bindings","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-11T03:00:40Z","receivedAt":"2010-10-11T03:00:40Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"From: Ramkumar Ramachandra <artagnon@gmail.com>\n\nThe svn-fe test script only requires git and the svn command-line\ntools.  Make these tests easier to read and run by not using the perl\nlibsvn bindings and instead duplicating only the relevant code from\nlib-git-svn.sh.\n\nSigned-off-by: Ramkumar Ramachandra <artagnon@gmail.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n t/t9010-svn-fe.sh |   14 ++++++++++++--\n 1 files changed, 12 insertions(+), 2 deletions(-)\n\ndiff --git a/t/t9010-svn-fe.sh b/t/t9010-svn-fe.sh\nindex a713dfc..fd851a4 100755\n--- a/t/t9010-svn-fe.sh\n+++ b/t/t9010-svn-fe.sh\n@@ -2,9 +2,19 @@\n \n test_description='check svn dumpfile importer'\n \n-. ./lib-git-svn.sh\n+. ./test-lib.sh\n \n-test_dump() {\n+svnconf=$PWD/svnconf\n+export svnconf\n+\n+svn_cmd () {\n+\tsubcommand=$1 &&\n+\tshift &&\n+\tmkdir -p \"$svnconf\" &&\n+\tsvn \"$subcommand\" --config-dir \"$svnconf\" \"$@\"\n+}\n+\n+test_dump () {\n \tlabel=$1\n \tdump=$2\n \ttest_expect_success \"$dump\" '\n-- \n1.7.2.3\n"},{"id":"153218","messageId":"20101011031142.GQ1553@burratino","threadId":"24407","inReplyTo":"20101011023435.GA706@burratino","subject":"[PATCH 02/16] vcs-svn: Replace buffer_read_string() memory pool with a strbuf","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-11T03:11:42Z","receivedAt":"2010-10-11T03:11:42Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"The buffer_read_string() function returns a temporary string of\nsize specified by the caller.  It currently uses an obj_pool to\nstore the return value, but that is overkill: all we need is a\nbuffer that can grow between requests to accomodate larger\nstrings.\n\nUse a strbuf instead.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n[Resent after messing up the message header; sorry for the noise.]\n\n vcs-svn/line_buffer.c |   16 ++++++----------\n 1 files changed, 6 insertions(+), 10 deletions(-)\n\ndiff --git a/vcs-svn/line_buffer.c b/vcs-svn/line_buffer.c\nindex f22c94f..6f32f28 100644\n--- a/vcs-svn/line_buffer.c\n+++ b/vcs-svn/line_buffer.c\n@@ -5,15 +5,13 @@\n \n #include \"git-compat-util.h\"\n #include \"line_buffer.h\"\n-#include \"obj_pool.h\"\n+#include \"strbuf.h\"\n \n #define LINE_BUFFER_LEN 10000\n #define COPY_BUFFER_LEN 4096\n \n-/* Create memory pool for char sequence of known length */\n-obj_pool_gen(blob, char, 4096)\n-\n static char line_buffer[LINE_BUFFER_LEN];\n+static struct strbuf blob_buffer = STRBUF_INIT;\n static FILE *infile;\n \n int buffer_init(const char *filename)\n@@ -58,11 +56,9 @@ char *buffer_read_line(void)\n \n char *buffer_read_string(uint32_t len)\n {\n-\tchar *s;\n-\tblob_free(blob_pool.size);\n-\ts = blob_pointer(blob_alloc(len + 1));\n-\ts[fread(s, 1, len, infile)] = '\\0';\n-\treturn ferror(infile) ? NULL : s;\n+\tstrbuf_reset(&blob_buffer);\n+\tstrbuf_fread(&blob_buffer, len, infile);\n+\treturn ferror(infile) ? NULL : blob_buffer.buf;\n }\n \n void buffer_copy_bytes(uint32_t len)\n@@ -94,5 +90,5 @@ void buffer_skip_bytes(uint32_t len)\n \n void buffer_reset(void)\n {\n-\tblob_reset();\n+\tstrbuf_release(&blob_buffer);\n }\n-- \n1.7.2.3\n"},{"id":"153219","messageId":"20101011040140.GC5649@burratino","threadId":"24407","inReplyTo":"20101011023435.GA706@burratino","subject":"[PATCH/RFC 16'/16] vcs-svn: Add svn delta parser","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-11T04:01:40Z","receivedAt":"2010-10-11T04:01:40Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Implement an svndiff 0 interpreter, for use by the dumpfilev3\nimporter.  It is slower than it needs to be (e.g., it does not use\nfseek() on input) for simplicity.\n\nThis is based only on the spec and not Subversion's implementation of\nthe svndiff0 spec.\n\nThe tests come from various deltas encountered in importing the\nApache SVN repo.\n\nThe svndiff0 semantics are not completely documented, meaning that\nsome of this work had to be done by guesswork.  It is not complete.\n\nThis version of the patch omits the enormous Xerces.cpp test (which\nis not so interesting because it passes, anyway).\n\nHelped-by: Ramkumar Ramachandra <artagnon@gmail.com>\nHelped-by: Thomas Rast <trast@student.ethz.ch>\nHelped-by: David Barr <david.barr@cordelta.com>\nNot-signed-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n[alternate ending, for those who do not like reading 3-MiB messages -\nsorry about that]\n\nStill not signed off because I haven't checked the copyright of the\ntests.  I would prefer tiny deltas or deltas of some public-domain\nwork (e.g., drafts of foundation documents of some country).\n\nThis patch owes a great deal to David and Ram, probably more than\nit owes to me.  David made lots and lots of fixes.  Ram introduced\nthe tests and helped with design.  Thomas provided an early sanity\ncheck for code clarity.\n\nThanks for reading.\n\n Makefile              |    4 +-\n t/t9010-svn-fe.sh     |   14 ++\n t/t9010/newdata.diff0 |  Bin 0 -> 19392 bytes\n t/t9010/newdata.done  |  522 +++++++++++++++++++++++++++++++++++++++++++++++++\n t/t9010/src.diff0     |  Bin 0 -> 74 bytes\n t/t9010/src.done      |  522 +++++++++++++++++++++++++++++++++++++++++++++++++\n test-svn-fe.c         |   37 +++-\n vcs-svn/svndiff.c     |  265 +++++++++++++++++++++++++\n vcs-svn/svndiff.h     |    9 +\n 9 files changed, 1364 insertions(+), 9 deletions(-)\n create mode 100644 t/t9010/blank.done\n create mode 100644 t/t9010/newdata.diff0\n create mode 100644 t/t9010/newdata.done\n create mode 100644 t/t9010/src.diff0\n create mode 100644 t/t9010/src.done\n create mode 100644 vcs-svn/svndiff.h\n\ndiff --git a/Makefile b/Makefile\nindex d99da33..966f5c7 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1766,7 +1766,7 @@ XDIFF_OBJS = xdiff/xdiffi.o xdiff/xprepare.o xdiff/xutils.o xdiff/xemit.o \\\n \txdiff/xmerge.o xdiff/xpatience.o\n VCSSVN_OBJS = vcs-svn/string_pool.o vcs-svn/line_buffer.o \\\n \tvcs-svn/repo_tree.o vcs-svn/fast_export.o vcs-svn/svndump.o \\\n-\tvcs-svn/sliding_window.o\n+\tvcs-svn/sliding_window.o vcs-svn/svndiff.o\n OBJECTS := $(GIT_OBJS) $(XDIFF_OBJS) $(VCSSVN_OBJS)\n \n dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d)\n@@ -1893,7 +1893,7 @@ xdiff-interface.o $(XDIFF_OBJS): \\\n $(VCSSVN_OBJS): \\\n \tvcs-svn/obj_pool.h vcs-svn/trp.h vcs-svn/string_pool.h \\\n \tvcs-svn/line_buffer.h vcs-svn/repo_tree.h vcs-svn/fast_export.h \\\n-\tvcs-svn/svndump.h vcs-svn/sliding_window.h\n+\tvcs-svn/sliding_window.h vcs-svn/svndump.h vcs-svn/svndiff.h\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\ndiff --git a/t/t9010-svn-fe.sh b/t/t9010-svn-fe.sh\nindex fd851a4..9e61608 100755\n--- a/t/t9010-svn-fe.sh\n+++ b/t/t9010-svn-fe.sh\n@@ -14,6 +14,18 @@ svn_cmd () {\n \tsvn \"$subcommand\" --config-dir \"$svnconf\" \"$@\"\n }\n \n+test_delta () {\n+\tdelta=\"$TEST_DIRECTORY/$1\"\n+\tpreimage=\"$TEST_DIRECTORY/$2\"\n+\texpect=\"$TEST_DIRECTORY/$3\"\n+\texpectation=${4:-success}\n+\ttest_expect_$expectation \"$delta\" '\n+\t\tdelta_len=$(wc -c <\"$delta\") &&\n+\t\ttest-svn-fe -d \"$preimage\" \"$delta\" $delta_len >actual &&\n+\t\ttest_cmp \"$expect\" actual\n+\t'\n+}\n+\n test_dump () {\n \tlabel=$1\n \tdump=$2\n@@ -38,5 +50,7 @@ test_dump () {\n }\n \n test_dump simple t9135/svn.dump\n+test_delta t9010/newdata.diff0 t9010/blank.done t9010/newdata.done\n+test_delta t9010/src.diff0 t9010/newdata.done t9010/src.done failure\n \n test_done\ndiff --git a/t/t9010/blank.done b/t/t9010/blank.done\nnew file mode 100644\nindex 0000000..e69de29\ndiff --git a/t/t9010/newdata.diff0 b/t/t9010/newdata.diff0\nnew file mode 100644\nindex 0000000000000000000000000000000000000000..57813032b74ff90fae130841a2a32ea909d489cc\nGIT binary patch\nliteral 19392\nzcmdU%+j87ic82F7SA2>cdMctkY*N;EY>%XgEK&4Wj%2DtjZ_s^F3{a*HcT|o1E5KF\nzu9Bz7>*o8{J^+27WKZQLEL-dY_QKlhy#Bqw%lAM2?svcY@BjJ!pZNd3`2X3PXJ_A^\nzyR(beuP=VQbZ_2X{BVBu?%?v>vv+^HeB?%-kBVxVS6x0Gb@?wH@7=4c?cAcC7PBIs\nzx~y|Q{r*MP<-h-rgM;VU<oc?qSJm{<f#aukFW;WI$vms7ymX&s%WN{wN0WN7sH@i1\nzl`ERbNxjUgWmEs0PrCM`$~#->T-Ri?Sut@%>&p7-id%KHd-AupuO7|2Zu#i+^s-(x\nzlYI2=b(No7G*_pAIwVj3HPC2l*UjOuo9Av)vx};8^~@E#YO+aJ+~jUrw3Aia7Gj87\nzWj9$-X5%uCT+}YV$(uDdIXHOZxrEeWQFPE~-BHFqX71sG2M<SIJQzLv`hMJb*rk8#\nzT$VL;bv&uF=6G6HSvg%E|C~2@b=_oD_n*hJf`6{^qFEeYwOw9R$3GW~<F;#x>%0lF\nz3=~Ig(dGXaWRIhiGF05O$5k<@%j`O@4i4VtWuCR+E)|=1>%4pH+O8-|H_c~7Ra94Q\nzQJCP|qtX2Xt!eA>20guO+g08^ItXl^1$9fhZke%XQ$mxkS_v9yPIH$Gz`dL2ZT~*7\nzxX9LSoV!&E`SIG-D6~myqyTC7+2wb~ZoKMT!GEo*>ds|heUtxvRW$j6d!IzUxRHyz\nz-D+Rlpv<bPRd$t+va70Qx07^r(0%I-ie0$FqH^s7>pgUOYf_Ozuyfgdxbw)J|588+\nzaiPB@qO7`}$g!&%cU!MYwv``if3sC7Ltm`Qu2?2BbZN7md*?>(<qUZynr>R>e$SYa\nziR0+q^B4EsRh^Y)qR-6v;*O*4@w0i3A0~<6CT)Q^u#>?Ce_rA9<9w3Ap)Q_d_EjCW\nzh@YE2=Jg^!aWAXz_O@=O?XlO)#=Vnr+_KCj(bJ$ZL;l>SK?qme#%p!Y4i{NBnGXmj\nzK7u;$`CSLyNmg&8v6aQR$(kZ>)0&^0ceqFHHh1mZbdmqE#3#4|X5_8CBS5$r%95c4\nzyH6w>jzI*Jbv<_ZFWCacwbIxPlAJ;p^Jw!1eL=LzuORP;ma-Bt2iF^RF|EyZrN;?w\nzSe0u<teVx)^1FIDDv3*FVj=jCw|{tCV-GG`Lb8D%pJdeUYcou<{#jj?^{wsiknL2{\nzteLuCG&labs}=hxiz;sqBlkbaYX}^<j26Y=mc}V&Fj<YW4v9_ciNjs<xaHbC!`aO1\nz6$vHT*8RjPiPn60n#*C3oCLmUxBn4#oHdQ2_f3)6vUE3ms6zPiuzbLS>Bl}?yGd3B\nzBCu{2I3)`-q@O70Z80Se4qwRYVgi7gwBK~D%5QsjY~5|qVN87pn&uq|OVZh3?IZ-e\nzt<?+fa4uWQjPhyZcx|iSygoBqva1$({wo|Fogci(tEmDJVcP<<%&(UD>BH61SSF^l\nz(8CwU>oU7-lhFqYi_!DEB6w@v+k9EBM{ElR(aye=aHP_buovCzXt|ht3w;#-IdS=O\nzMLNtStX!<BD@b|c<(p<>ZH&zbFmGTLyC4PFWsB_Rx+%K#WA2Qrk{1_}SlRnGd>J3M\nzjoWz<;>I+e6w3J_kabFLSTd1tkxldaLE3-#;!()kD3-*HDf`Ure5Q682@^Bqn?**b\nz8)s{r=Ge79p#~}=s91Urf#%o^6F<gdIi638MHKEk<XB~k+}#Tnch4BL@H=%aaTbrl\nz=Eke0@>)ak;dVb{*L-7{0l6Za6t_c~3>S<7>6lqg!um@e0ttp>YVm*wvzxk@B7<sH\nz7F20wd)IlsG=(p7g80+0`Yc`JvszJ2$`$$j2MR0OQcEhxNs>fGj3J8#vYZ!X-PZh?\nzWcqk%)nFhhJ;av}9+@uOQE_i!28H27eU12XL$*m@NywqNGKB<^(ofWweMHqdQ!?=T\nzCxg}QmMZe8b01VpT3$PqXp|h<NmCF<aP2^DxoVbmn*=gqd0md{U&7bU{Q)jL8}8`k\nzJtcnkhcB#diTOo>UYU6+h?tP`P(&mu^4<6oLgYJtqo`2tPM<e<R+j6}af54giJR;t\nzq3t9w8p~AbZLGz`^vRzh+kt@JSx>ItSGt2l8+71~BnM1KrBX2?QK$|?w1kV0{rda&\nzpCwW~b!u)mCO4M#K=W=zX_^@GS$@5DvFfDiUT4YCY|v!9%$xFbxh7;+?vUnYMG`se\nz8ERL=7I)O%5^JCKACL30vlf6-kYqQq_bL+Ds=etiT}-alM2f`XOkGk#LT@N1%E&r`\nz?yb=sQTAdTP;>WuxAR@yT;+ND_f>u`lBR;dppuj^-6rQMN+9gBo70V5&1KChq30Pn\nzbEmY7W@NvOFRPHm+AOCO3@KXH*=z0)Bc0^SZYZVx>a@*vE`_twUc@vuBI>PXvtB<Z\nzfAlfZJ&M~Jxu-E2UUqBRgy<X#8Z_k{3+o%D_62PUmJ_1h@Va9kBo$TYS|@IdQ(Tjr\nzL#eKfOYS~OZ$yv~7>+II^hL8)B$1anA<@?WR_Cb7%Wu>R#m#u{AxESgOJ~*+nqN4A\nzdAW{Phl}3TOACg>jchR-qZw^TnU9G<6-`8+l!oo2INp2HjdT?C_TU6@SWXK97_b>M\nzQMC6{LhQH2GNsD!LHPA1Z$`1Rg6fX`?&NKoosIjX&HHS7yjOiOxf)Xgc2b1d<VT4q\nzQ^}vvf4QS%C%$x<U^ZRMvtpX8OC4it=t(F~R5wk~LyI1KNdi3-;|sJvH|iOy_}Zl0\nzZ|fsgF?w+F=RQ&C?)p)Lje^)`z5zh6J5YGLYPA@9Hic@qo2(?U`+Cqv2P=Zc7BxE6\nzgubc^9LncT+y{!2oGOgqJu9wO@^3ac56#0_DF5CButeosSUx2cc~7Kdn1{Aqq9S~%\nzwzEEUs}M%HVYR;jsZ~xM8zw6b2}NB7jU6O`noukU0=%o$OfBfCEcI9kL*Y-ODrU%&\nz(r7A=!!m?Skt;mJ;EH9DPX_H{{El)#Ib*OUG*<=@3_gBgtwxRTanIGHll&Ah$iNC~\nz?B#~csHSMJ=nS}xFIz>z!N<>61;|0W1~FP_Wy@ysYO;1mL|jU*{*!2P+RTqCU91cX\nzia+A*iMu^@KU1X*uY48Y5O>p_xbIdKkeMpr7?@uKKT*&nAunmkijwx$a~}A#{#Fga\nz=$7s=F;BcruZKpQDzy+^!$pahVoB0}qM+y<-C&!-z5r*A^HI_vW8~#*G?`;`mCLTK\nz=-n+D?!E6O!~PC-VeS43$hRuVs#N0`+S&U{H9s{#9_<ThTDdK43Q*BPNB)bI&e04c\nzWJyuzpVv{~;%)&akp$L$lO(_xipg-J>9x{G6!xbm$GSc7dZ+$+@&jG&CEo+FuQ6aO\nz5~$HCa~xd$8#>i{Z3Jql+%IUo4FnKKBOMqfavG4F(;Mps4+8vyqKGoHrd}AmtfKA%\nz>m}Zaya!6;j&fTMeNm7BS)iUr2S@aA!f&klqFwO#k(`&d4aI8ROsT?XfkM-7=Y&3s\nz$72BWTY5;8F5<*t;Mm7O+C!?h`90zWfQv_BMQcpnosI+-2z5FT?A~x6pjjn~+J(k0\nz*pZL+0x!4`?MZKc+NZ!@0?4cZj|)L6=!ce*^8lrVc0s^o9~O1TENEjBC~w?qm^X^5\nzXgsO50AU`DIcPvQG5o0BS3--1!`tB4@e^($QR6JR6Ci^2eX8}yGF3*mJ~kvRZ1MiE\nzehbBy8apG?X2tZe3H_>;Mif3hfRzMjHrB0|Xx7qvpM(N{>(Ls*AXP%DlA<|2jjnLi\nz<EnwQg0h-7nJ?_j^KyBL9v2u9Di%y`n~CCe?)&vpb_YJ4mDG_{hiB9CsO>lwgf@~o\nzfTCAjO%52;*^-$$uW;&++&INdUqgIMF))iRf@8X)r3d|h;cNp^gs;@IE~?T(;jr4p\nztTSK|XwZkzGttNyLO2k|3!$6!(!3QbT;z1!*U7+kBtRWgM^vw`<%6YcajZH5aFmuP\nzAxg*{>$w8T0Zz#SAdd!UoD&T>QToXy>U|70=Qs&WKzMk5n_5G^WpC%+LdmSK*n64a\nzBB~}GQpkXB=Tz`~!a-NFVW+~D13<K2E1tn!6d6q#<*j3YGoMC|ga31o*QBI;$U&P-\nz>X2d%;3qHJ{MI@e)n_o74p<D)ZgL&jc{EOT8zrT+y{Qf}b1<5Cp>w@cQ?Za?!Dfj&\nzpnLvzMm<9von0$vQhT=qShFsFYvOHBDyfDSAc#f1Y9pMBvS_v`T%V0e0qn&hJD4^H\nzN>p00IvrBrH3@o7_{*nT>X3)HBa0hBGlbBBtD=S5S9X<so>)|sv*COn)2I9H;?)cH\nz{QT9$2lxKs-T9?^`Qy6__w4z_`}3!OUniWj0L4=>D%MCjhmx*x(4iqzgHSqhFTz(L\nzz#tCFMj41q&_IFA48hgo8%a)X&B0c4-y4GArfDBK`^iewF<m31U;{<Bsq(GxA3Z^N\nz(Q~^fuI2{999ro=h!qbu3&RbO=}<I6vdJ$bYoFwByZ*QhYaO_oJj}Na5xFPu6hs-=\nzA?6~%-qi(JfHCFP(Me1|HJ0c+u&vB2iN{Q8XgsxHp8y@mo=&d_DqQ4y6#@%n2c4A<\nz89);;;H02((o5r0sWb^8=2R0#G-We6<w#u0xbek^ti<UQU6`&vdsGV+_S4@yk~s?M\nzYqNDr8YVfK!4<PZo#%4q6}O;x9(G~m_mEP`_H@kmv@{6XcvB?w>ad*dLF78(36UXw\nz`)?FJ{SDP`6TNMhzMv9pc3ZUb^@!*^M*y6vCPXGs+T590*ScP*v>KQ7#Ny5bT-thE\nzPyBK615ZNP#Ph;J{DFeZi^rme0+%AF4k%PoXaZrIfXqUbD=@pyD#>IBcCiNaW#*X=\nz-dhy#RU4iV%$8mosyR4g4_3Gp%u>wiTlxs`O{|JBECVDfuwra%-QLR~9jp>pnwEk@\nzh%yoQ*_*e6wEl+rmgDsnol7)I>Ld`X5d#e<_S9#74J7y;jI2QO-J5{MLQ0s!h^2I_\nzg)=w|kKNyfih{USP0M^r;eT{~a&<z?=u?F^e2?+=2D;;DX_vckJ0|3z&ryM;*c6wZ\nzy}GzOckjM`c}d8|-4A81%d91-9O5)$T}3OsNX;oR_$)TylaARj)kL5)ip4wL62mox\nz|6;WOb~exny8=<z>?1cKa^z`Py&Z@8um7%Z%0GSQa7;KDt&mz_VF=J9(u%2#pDLMd\nzFcwS6$PB1=@k_2#G~$ek!&_2o2y=<rmQF%GMwsaq1KAOZ*XqV_urX(;az)C#a2ALe\nz0^a&FFP#Sy&s5!$SS8+c!{IIiFfF_1Y<r~|)vWOmcegPUl+uDSbFb#6;8a)%!_F(Y\nzJht<`O5LuA`qtVV37A?yeCp=p%9372*&49fVu`*BGf1h7P275y^#4XPYkb1ROB9My\nzjM)kZDW43|ZX%1+qeVKf!LFXF__)o-Hj$KeA3i{NT%=13DlB<n!yPtRlxPlF^{=%{\nz9ahj#1IUrJg)HU1E(z+D+}n6yxgp7DMSDz!V)N6R;=q)8goEoxRHWD9LQOx0Wk6#^\nzY-%|CCFD?w=2c6nBz8|uwo>+vUa9?J4WvU^e~*FPC1uO<MUqglS4InrNg{phUQ&g@\nz9SH{nS2c>SkPW0%Ai#pV0<dmZ!q{!!UeReX?o=-4S#+Mznm|*-4GSj1_R`S)eVcm-\nz59%6HZyeOb3Bkt7y~dTUK(qpLlou$|ZT^HllTuzYWlDtUO6@2N0cOkQB(v$F6Aryj\nzdm1P6Zn3%du;OQbK0aI}UqYXEu2yac8r`w4yD_!IZZukQGsrQPb;qVY&0v)4_U7(h\nz3A_xjlAkU(`KwNcY7evo7>TPpNKQb0hw>$y>7&V2aT)|U0)9As>m&Qi3KfHCqsI0r\nzKKqw1PX2uI@Z^hs!Ch4PjAS8y(IDyRO*K7HaD>`uku}$=Ws=uG-03p~{OCX=!4R<F\nzU0|0x94oXeZKjjW9GVA#PEnibq#;omx<ZP8?LDh}Log*vghG@sSu`Hi|4nv0&GD#-\nzWdxODUgbxE)?GY+9N+}WlK0r<-O3x>V^eVh*+o;O7YQ}HiVfz{LW)xwIP4wfaIch-\nz#KKNgNok_ZLPHe4r2nba8n;Ch0q2QimG1^Y*&R`GPmxot36`F>6cQczHNxMbv5}F0\nz)lV;1<FJfYE2}2LTl;?U9}~7GlyAAnK+<XN{|~J6;0y&8mbl-9z9CKOQmX@++oU3@\nzRZl^+2{KL7t`yU97-C(NiYS{%JoACmXW}4*RIAFqtTj$p8MJ6mZRAj2hSdheZ<d5x\nzh4m1zOgUs<iVdJ{1%kV>NiQW672eyoI|TnCI4Hl7=G5W?U<^ve4=NEukm1V(J04Vh\nz`qnA|zS3CD;8flhBa#1r@NDFW-O7YDLqwHfpu*;lG$Yq0d-guSH%tbxE1e-FyYueU\nz$3?(IooI5c@GVu(;h>QSpwTx?rLKS_`u4{ImO+00d;)UUmAYGcXD_5SNyxdsYW$M&\nzCsQR|rg?O6BF_*E;j3~iO46UNnr8hpqGSWkhb6B+)v!a_E-S75_K-dyL8lFhWB#ai\nzK)`Qarw~HDAB$Jt2dYj3)}V#8<_bGd+yalGhPEzXmq<7=>2(w?88P~$K0TO(nG*`|\nzM<0xh*Tj3;&2V+1%gwB?FqfhiwPSB_U}N9`R!YOXLv45|CWO$?Z`el!G>tLy2J)Nh\nzBn~hAH&$t@a-ln>sWSYbnc4iEYDNhnH80!7Br^V#Lp%8AJ&N;YJBZ$C+ZfaCxR?lf\nzD+`{h@g{sn`V!_)Of+&q`t?17oyJq6So<)b?8&5mL!9#Cwiu&)10Y?4TFZg8Mnxpq\nzx2I7jGOPaMs1ud^3<Yo2ugLR8z9;pgHK!+6tmpRVj83u4A~c7JuOuc1BY(-V((DXi\nzt~`W)Y{;GxEy5lH0zIjHuOeha*8u1et#IF!o1YEQFLy|fe0YDzwX&HkDIu==be)(?\nzA_i6E5r-sSJ4epSk)8`$wo$%ufPHvB#f=y__ua}{&jv@^NnnQ?#2Gj_+N<b8E=Y5J\nzlFpEcWT-wK3&H?>v78bOE!}WavgsYp!-z2IozPt^>DFk>0weOP;r9PAagXb|Bfe+L\nzxJ|Qbtcfyc?dQtWPF*o&L$l}y3U4<Xx_~GhCQ|l5OBRQ!<p^<J)9}bNY`?L2JWo#n\nz;2LpOby!x1XBup7=n!`8*!oj~AJG4(aME?r2A@71Wb63h#7pR?&8dJ1D>k*1JYSfN\nza!Zw-0ZLSpGQ$g1!0c1Q<K)21%?MJ19BYRK6_aA~p?3F`i{7AzrXKF8bOX)ds9a1v\nzL`&5LvLl6x!Q*Su?-SowR4GkXej5ZZ=;I!di}X;J`DC)mp4K+Vy)A+GUK~goihTB3\nz)zEO)AI$@)l(Vv9qex17NG~A&t2VMJlwY%vC?*^ffrk^uu%qE#hLOPsp^uh#DZr;K\nzq-Vk$h_(eu32oo521~*)5DpEz!+ErOck$-s*^kd&pW9-pFV!V41(8vo+kxzGwL1*A\nzIy5z>#lgAb+=_5$C}Gut2e)R#ZR#u>a)9aCBPo8!;V$t6%?=n*m6ybo;;Iiv;dFVn\nzvROERUpl(+Be9za*bSS-A$2mxaG-<Tq<uei8nUNO7=w)2Q-r?2eo1wk-wudGqvm%w\nzgTe(lI!a{(lXG-JIsoHQF8>6az)w;^=cB8{r?VV82HTpEOgFL<3k^(qK$Z$ykLEn%\nzP#}&YzxeSI2i^SH^<g7u|GeIqsP*fjQQxs{lemA!L%JN3Px3tmnV8>Rr^kAJ!z}=N\nze#_-|-u5TQ%q9;K?<J?2-{uFl(%dHRewLT$9oVOGLwD+`{K-&!TJ1wn@K7TGZrk1X\nzX|I&GgZaL=HY|vY9I05;@awETc6m1$dQ-f{|BgJ$()h$io}%Spo(F2PU+0?ar4=<E\nzp6PL8_goKNbD;6Woa5Ub$ny_K{~I~%jHxmAM91|zU%$}<S2jC9H~D0#cRSboz;kNI\nz`#jAQxhI2swsZM&W}_<ifnieW%4Nnu+T_|jQ8eAT^al=#f5@&1rcka6s1}QzOa3Z=\nz?lq6<@&Grx{h>K=PwdI8-7A^gbeDRrnk(%2>Tk<+lL<5SqPQfDvDr(1wCA3L2)04U\nz1nWaUk{ppnCC}MBE1FG396P!Lo7#P@!93@m<YQjTD~JSR4^{hjKUvhZV-KFP)$mO{\nz=4om~?oW<A32qj1v~UgUYE=lpL2wo0LrqkApfM}stf7o#;f-WoEFTAKfVGu@Ap$gF\nzw53nBN^P*j=`j#{;@@ToIGWS0JRE4<3q{@J+N80N?;M<^1FEeZyPy2YP8y61Lzr6h\nzvbw><rv?pqKz3`f2!kZhVZ`Ci(=g{~WBB1|NgXFkEyiP@aPJ)-gfnLGM)hs*lwvlQ\nzBU>^kjd@$>q4XeDXJG<3+vKZKM<oJKHqatEno(2FItDiFY|~m*WS>sm`!jJ?8U<AA\nzdw-p=#DX)?;Nd!*sn~Iu*#Az?2pN!ka_`_k{A;~OFTl`se_aOOb;<KjxPT=W4Ws!}\nzW_A^JPt^GE{rTJTXO}i8;k-dXAdaI87}6Lj^(9*%&EqN}KE!slb8g1dF5%8SgH{Hn\nzd2{k+cyKM)<zOq|LE#ZO-3V%Wj<lU)&H<0aqbvFqAro<Rv?T-pLZiP05$Pud83UWj\nziL2G7k6K_LRF-5Np@om@3$YDVW>q&c_B`_E<xPbE*yx$kb|kUoiVej-8~}kIFFqVI\nz1c30ryLfwk%uttm`OdvPzr1+${`^H4Vfb~sKf)nFV!VO`&M`CR1JL2FWrJ~!V_Y!l\nz^p>8ajwDe=y9y6PJok_M?r1la(eeP<2AJupVY9?McBr4N{YVhIENQJsBu?V$kGb9+\nz%_cSW>pf6(r#3g#xZ`<|%-s&?M}vR$*1=x&Ogw@-e6JU8*dVB@26Z>3l1^Q=CF2cS\nzr@>}ZWOm$JxUS#_L$|{{y7Xrls7uw4;=l;sCjiBSpv^rVA8ZHnC3$%@#;Z5ERG<u@\nzzV=-&^*^U}8UPG2(<YQ75%~!R)xnD+%?+$Bgco}{h701#O?q+VE;qkXAIvh^@Ti^!\nz!sZL+DyEumvW7s%gkJoSi(sEN2e@3;ihCB6gM_`yh4b~8Qv<J&#4gKyBo3n9sL|rO\nz@Leh}j!Zk-rQxtN+giC(k!^jyuLRq=WTW)j`fj)8+P)x_T3cUimRKop><^FM{q3zn\nz=5|;~V%e)NayV61uP*sP?}_^zF*2zXL-M!a>eY;eGLXl!VV3JH;V5=-B4a0GKHA6S\nzzslWMEtWQ-Y$Py9rT)-{hvmBYu^g<p@`Js4(}llcqt~2D@jP8W97%2X_Z1KH5qhj>\nzus<&2i2;nSIEJH4W-eqPrGXnoMt((LkCwk{<97!5E^9UEx0|+!PB6zw4KKo9oUj5*\nzM^t@i<DoY4H}S+{?!YGL=>X{j=uyKjEYab1F)3&*6#9ISi96e7*B^1xeSdcFnR~?%\nzPU+tB=#K_yD4RZ0BS^Qu@%OLzlzK%^vUT%E?n~bKuSa~!9D^;Wm?-y`wRW^a$YGf1\nzOZS!g#(nY7eeo54Ey_Lo(mnjf{qdpu<5%v>2ky%+-Iw3EKRtAR`pSLvz<tHl-?*<I\nSy02OE;6CKe^(VDHJNREnOC7xc\n\nliteral 0\nHcmV?d00001\n\ndiff --git a/t/t9010/newdata.done b/t/t9010/newdata.done\nnew file mode 100644\nindex 0000000..f53c3bf\n--- /dev/null\n+++ b/t/t9010/newdata.done\n@@ -0,0 +1,522 @@\n+APACHE COMMONS PROJECT\n+STATUS: -*-indented-text-*- Last modified at [$Date$]\n+\n+Background:\n+    o IRC channel #apache-commons on irc.openprojects.net\n+      traffic is logged to <URL:http://Source-Zone.Org/apache-irc/>\n+      so that the content of interactive discussions is available\n+      to everyone\n+\n+Project committers (as of 2002-10-27):\n+    o commons:\n+      aaron,coar,donaldp,jerenkrantz,fitz,geirm,gstein,jim,striker\n+    o commons-site:\n+      aaron,coar,donaldp,jerenkrantz,fitz,geirm,gstein,jim,striker,\n+      sanders,nicolaken\n+\n+Release:\n+    none yet; still defining mission :-)\n+\n+\n+Resolved Issues:\n+\n+    o Commons is a parent of reusable code projects. These projects\n+      may be used by other projects of the ASF, but it is not a\n+      requirement.\n+\n+    o The Commons will be language-agnostic.\n+\n+    o Projects that are \"in scope\" are defined as:\n+    \n+      - Existing components that are, or would be, useful to multiple\n+        projects\n+\n+      - If a component does not fit the (TBD) goals of Apache Commons,\n+        then it is not considered \"in scope\" just because it has no\n+        other home. In other words, the Apache Commons is not a place\n+        of last refuge if the component does not match the Apache\n+        Commons' goals.\n+\n+      - Reusable libraries\n+        [ gstein: we should expand this definition for the mission\n+          statement; examples provided were serf and regexp ]\n+\n+      - Components that do not fit cleanly into any other top-level\n+        project, but they do fit the goals of Commons.\n+\n+    o Voting will follow the \"standard Apache voting guidelines\"\n+\n+      [ be nice to refer to an Incubator doc here ]\n+\n+    o All code donations [to the ASF, destined for Apache Commons]\n+      arrive via the Incubator, unless the Incubator states they can\n+      be placed directly into Commons.\n+\n+    o Existing Commons committers can start new components without a\n+      detour to the Incubator. These new components must be approved\n+      by the PMC and must meet the (TBD) goals of Apache Commons.\n+\n+\n+Pending issues:\n+    o Coming up with a set of bylaws for the project\n+\n+    o Enabling Reply-to on the @commons lists\n+      (pmc@ will *not* use reply-to munging, but user lists\n+      will be determined by user majority; this item applies\n+      to lists for which the decision has not yet been made)\n+      +1: aaron, coar, donaldp, geirm, acoliver, mas, bayard, sanders\n+      -1: fitz, gstein, jerenkrantz, striker, jim\n+\n+    o The name 'Commons' has caused some heartburn with the\n+      Jakarta community because of the Jakarta-Commons project.\n+      Should we rename to avoid conflicts and keep the peace?\n+      Conflicts would include Java namespace as well as\n+      philosophical aspects.\n+      +1: \n+      +0: coar (i'm willing)\n+      -0: jerenkrantz, donaldp, striker, gstein, fitz\n+      -1: sanders\n+\n+    o If we rename, to what?  What words/names describe our\n+      purpose?\n+      - toolbox\n+        +0: gstein (I'd be +1 but for the confusion with the existing\n+                    Apache Toolbox project, but *really* like this\n+                    name)\n+      - toolchest\n+        +0.5: gstein\n+      - tools\n+        +0: gstein\n+        -1: donaldp (tools are different to components)\n+      - components\n+        +0: gstein (a bit long)\n+      - util\n+      - library\n+        +0: gstein (doesn't fit well with perl/python \"modules\")\n+      - suite (sweet?)\n+      - belt (as in bat-belt or tool-belt)\n+      - mcgyver\n+      - foundry or mill\n+        +1: sanders (maybe too 'SourceForgeesque')\n+        -0: donaldp (If reorg goes through we may have multiple\n+                     foundaries or federations for different \"concepts\")\n+      - federation\n+      - share or shared\n+      - stuff\n+        +.3: fitz :)\n+      - ?\n+\n+    o Style for the mailing lists:\n+    \n+      One community mailing list, with specific breakouts:\n+        +1: fitz, jerenkrantz, sanders, coar,\n+            donaldp (lets start here and evolve)\n+      +0.5: mas\n+        -0: aaron (too early)\n+      \n+      Topical mailing lists:\n+        +1: gstein, scolebourne, acoliver, striker\n+        -0: aaron (too early), jerenkrantz\n+      -0.1: mas, sanders (too early for this), donaldp\n+        -1: coar\n+      \n+      Per-language mailing lists:\n+        -0: aaron (too early)\n+      -0.1: mas\n+        -1: gstein, sanders, fitz, jerenkrantz, striker, coar\n+\n+      Per-component mailing lists as a default (breakouts will create\n+          these as a matter of course, this is about the default)\n+      +0.7: mas\n+        -0: aaron (too early)\n+      -0.9: sanders\n+        -1: gstein, fitz, jerenkrantz, striker\n+\n+    o A number of very valid issues have been brought up on the\n+      list. We need to figure out how the Commons Project will\n+      deal with each of these, in terms of new components and\n+      how those components will contain code projects. This list\n+      is only meant to keep record of all the issues:\n+\n+        - Releasable pieces\n+        - Release rules\n+        - Voting scope\n+        - Directory structure and naming conventions\n+        - Coding style\n+        - Build system consistency (or inconsistency)\n+        - Namespace issues (esp. w/ java)\n+        - Language vs. Functional\n+\n+    o Default commit privileges\n+    \n+      - Commons-wide\n+        +1:\n+        -1: gstein, striker, donaldp\n+      \n+      - Per-component\n+        +1: gstein, striker, donaldp, jerenkrantz\n+        -1:\n+      \n+      - Per-component with self-chosen aggregation\n+        +1: gstein, donaldp\n+        -1:\n+\n+    o Granularity of CVS repositories for components (this excludes\n+      commons-site)\n+    \n+      - Commons-wide\n+        +1: gstein, donaldp, jerenkrantz\n+        -1:\n+      \n+      - Per-topic\n+        +1:\n+        -0: gstein, donaldp, jerenkrantz\n+        -1: \n+      \n+      - Per-component\n+        +1:\n+        -1: gstein, donaldp, jerenkrantz\n+\n+\n+Project Mission:\n+\n+What is the project's mission?  Our statement of goals/mission/vision\n+should arise from the answers to the following and other questions:\n+(jim notes that defining something after the fact seems very backwards\n+ and broken; gstein notes that we're refining the board-provided\n+ charter)\n+\n+    o Should commons have an sandbox component to ease infrastructure\n+      burden on smaller code bases?\n+      +1: coar, donaldp, jerenkrantz, gstein, sanders (non-binding)\n+      +0: fitz\n+      -0: striker\n+      -1: jim (the PMC is about reusability, not sandbox),\n+          aaron (what jim said; and go see incubator)\n+\n+    o What types of components would be appropriate for this project? \n+      (\"in scope\")\n+\n+      - Tools that help/promote reusability?\n+        Hypothetical: ant, jlibtool, ASF-based autoconf\n+        +1: jerenkrantz, gstein, striker, fitz, sanders (non-binding)\n+        -0: donaldp (prefer a tools PMC for that)\n+        -1: aaron (too broad, don't belong here)\n+\n+      - Development frameworks?\n+        Hypothetical: avalon\n+        +1: fitz\n+        -0: donaldp (how do we determine this given we would prolly\n+                     accept it if it was new?)\n+        -1: gstein (the avalon components, but not the whole bugger),\n+            striker, sanders (non-binding)\n+\n+     - Components that fit the (TBD) goals of Commons, have a more\n+       \"logical\" home elsewhere in the ASF, but were rejected by that\n+       home?\n+        +1: gstein, donaldp\n+         0: striker (on a case by case basis, taking reasons for rejection\n+                     seriously into account. Abstain from vote until\n+                     rephrased),\n+            fitz (what striker said), aaron (what fitz said)\n+        -1: jerenkrantz, sanders (non-binding)\n+\n+      FOLD BELOW VOTES INTO ABOVE? (i.e. eliminate the \"donation\" wording)\n+      - Donations that could fit but have a more obvious (proper) home which\n+        has already rejected it?\n+        +1: coar, donaldp, gstein (note the \"might fit\" term)\n+        -0:\n+        -1: jerenkrantz, jim, aaron, striker, fitz\n+\n+      - Existing ASF components whose committers believe that they\n+        are a better fit under commons and the commons PMC agrees?\n+        (If this component were brought up as new, we would accept it.)\n+        +1: coar, donaldp, jerenkrantz, striker, gstein, fitz\n+        -1: jim (by this definition httpd could be in commons)\n+                (gstein says: see the \"if\" part; we wouldn't accept httpd)\n+                (jim says: until we better define what the PMC would or\n+                 would not accept, then this seems too wishy-washy to me)\n+             (gstein says: jim, you're blocking closure on this;\n+              how would you refine the phrasing here; the intent\n+              here is to accept components from the other Commons\n+              projects or projects with reusable component),\n+             aaron (we need to differenciate ourselves from other\n+                    libraries first, namely APR)\n+\n+      - Packages being worked on by Apache developers, with a clear\n+        affiliation, that can't or won't be bundled?  (E.g., an\n+        httpd module)\n+        +1: coar, donaldp\n+        -1: jerenkrantz, striker, gstein, fitz, jim, aaron\n+       CLOSE THIS? (as \"not passed\"; what is a good way to phrase this?)\n+\n+      - Should we have a minimum bar of entry for components?\n+        +1:\n+        -0: donaldp, gstein\n+        -1:\n+       \n+      - Should we have a minimum set of requirements before components\n+        are released?\n+        +1: donaldp, gstein (mixed, see below), striker\n+        -1: jerenkrantz (what is released?)\n+\n+      - If yes to above then which things should be part of minimum\n+        requirements?\n+\n+        documentation: require basic overview and user docs\n+        +1: donaldp\n+        -0: gstein (recommend highly, but let the committers determine\n+                    what is right for the component),\n+            striker, jerenkrantz\n+        -1:\n+\n+        uptodate website: require website be updated to latest release\n+                          but may still host previous release docs.\n+        +1: donaldp, gstein, striker\n+        -0: jerenkrantz\n+        -1:\n+\n+        unit tests: (okay so this will never get consensus but ...)\n+        +1: donaldp\n+        -1: gstein (unit tests should be recommended, but not\n+                    mandated; I also find it unreasonable for initial\n+                    development/pre-alpha releases, but it can make\n+                    sense for \"final\" types of releases),\n+            striker, jerenkrantz\n+\n+        versioning standard: derived from\n+            http://apr.apache.org/versioning.html\n+            http://jakarta.apache.org/commons/versioning.html\n+        +1: donaldp, gstein, striker, jerenkrantz\n+        -1:\n+\n+        release process: derived from\n+          http://jakarta.apache.org/commons/releases.html\n+          http://jakarta.apache.org/turbine/maven/development/release-process.html\n+          http://cvs.apache.org/viewcvs.cgi/jakarta-ant/ReleaseInstructions?rev=1.9.2.1&content-type=text/vnd.viewcvs-markup\n+        +1: donaldp\n+        -1: gstein (we should provide \"best practices\" but allow each\n+                    components' committers to define their rules),\n+            striker, jerenkrantz\n+\n+        deprecation process: (java specific?)\n+          http://jakarta.apache.org/turbine/maven/development/deprecation.html\n+        +1: donaldp, gstein (I see this as part of the \"versioning\"\n+                             process, and we can provide best\n+                             practices here)\n+        -0: jerenkrantz (kinda sorta versioning, but not quite)\n+        -1:\n+\n+        CVS/Subversion branching:\n+          http://jakarta.apache.org/turbine/maven/development/branches.html\n+        +1: donaldp\n+        -1: gstein (we should provide \"best practices\" but allow each\n+                    components' committers to define their rules),\n+            striker, jerenkrantz\n+\n+\n+Candidate Projects:\n+\n+    o APR's serf project has voted itself to move into Commons.\n+    \n+      - Should the PMC accept it as fitting the Commons goal?\n+        +1: gstein, fitz, jerenkrantz, striker, donaldp\n+        -1: aaron (no such thing as \"the Commons goal\", how can it fit it?)\n+\n+      - When should it move?\n+\n+        Whenever it likes:\n+          +1: gstein, sanders, jerenkrantz, striker\n+          +0: donaldp (+1 if we use subversion, but if using CVS \n+              we should hold off until structure is decided upon)\n+          -1: aaron (after we know why it fits)\n+\n+        Give us a while:\n+          +1: fitz (what's the hurry?), aaron\n+          -0: gstein (we're only talking about a small seed of a\n+                codebase; it won't get in our way as we complete the\n+                charter), striker\n+\n+      - Where should the CVS code be located?\n+      \n+        commons/serf   (each component under top-level)\n+            +1: sanders (works well at jakarta-commons)\n+                fitz (Please don't mix interface and implementation \n+                  of commons!), aaron\n+            +0: jerenkrantz\n+          -0.5: gstein\n+            -1: donaldp (makes it difficult to update all related \n+                         projects with a single sweep)\n+\n+        commons/components/serf   (all components under this dir,\n+            leaving the top open for other non-code items)\n+          +0: gstein, striker, donaldp (is this just dev with a \n+                                        different name?\n+                                        (gstein says \"yes\"))\n+          -1: fitz, aaron, jerenkrantz\n+        \n+        commons/clients/serf   (topical-groups under top-level)\n+          +1: gstein, jerenkrantz\n+          -1: fitz, aaron, donaldp\n+        \n+        commons/dev/serf  (all components under \"dev\")\n+          +1: gstein, donaldp (if we are having a single \n+                               monolithic repo for all commons)\n+          -1: fitz, aaron, jerenkrantz\n+        \n+        commons/bootstrap/serf  (serf is very early stage, so maybe we\n+            have a \"bootstrap\" area; this is different from Incubator\n+            since the existing committers do not need \"training\")\n+          +1: gstein, donaldp\n+          -1: fitz, aaron, jerenkrantz\n+\n+        commons/???\n+\n+        commons/c/serf (separate out component based on language\n+                        and then have a flat structure underneath)\n+          +1: donaldp\n+          -1: jerenkrantz\n+\n+      - What mailing list should it use for dev discussions?\n+      \n+        general@commons.apache.org:  (one group for all discussion;\n+                                      dev and non-dev alike)\n+          -0.5: gstein\n+            -1: striker, aaron, jerenkrantz\n+        \n+        dev@commons.apache.org:  (one group for dev discussion;\n+                                  general@ remains for non-dev)\n+          +1: gstein, fitz, sanders, jerenkrantz, striker, donaldp\n+          \n+        clients-dev@commons.apache.org:\n+           (this is really TOPICNAME-dev@ where I preselected\n+            \"clients\" for TOPICNAME; this question is whether this\n+            style would be appropriate)\n+          +1: gstein, striker\n+          -0: sanders, donaldp (maybe in the future but too early),\n+              jerenkrantz\n+          -1: aaron (what is \"clients\"? I'd probably be +1 if I knew\n+                     what that was)\n+\n+      - Note: serf has no web site, so there isn't a need to figure\n+        that out right now.\n+\n+\n+Assets:\n+    DNS:                commons.apache.org\n+    \n+    Mailing lists:      general@commons.apache.org\n+                        announce@commons.apache.org\n+                        pmc@commons.apache.org\n+                        cvs@commons.apache.org\n+                        \n+                        [ core-cvs@commons.apache.org in case we\n+                          create a commons-core CVS module ]\n+\n+    Web site:           http://commons.apache.org/\n+    \n+    Repositories:       commons        (code, info, etc)\n+                        commons-site   (the web site)\n+\n+\n+PMC Members:\n+\n+    Aaron Bannert <aaron@apache.org>\n+    Ken Coar <coar@apache.org>\n+    Peter Donald <peter@apache.org>\n+    Justin Erenkrantz <jerenkrantz@apache.org>\n+    Brian W. Fitzpatrick <fitz@apache.org>\n+    Jim Jagielski <jim@apache.org>\n+    Geir Magnusson Jr. <geirm@apache.org>\n+    Greg Stein <gstein@lyra.org>\n+    Sander Striker <striker@apache.org>\n+\n+    Note: Ken Coar is the Chair\n+\n+\n+PMC Members, pending Board approval:\n+\n+    none yet\n+\n+    [ this may become obsolete; the Board is discussing a way for the\n+      Chair to directly alter the PMC membership; until then, however,\n+      we need PMC members ratified by the board, and this tracks them ]\n+\n+\n+Committers:\n+\n+    none yet [still defining mission]\n+\n+\n+Invited Committers:\n+\n+    none yet\n+\n+\n+Current mission/charter as approved by the board:\n+\n+    'The Apache Commons PMC hereby is responsible for the creation\n+    and maintenance of software related to reusable libraries and\n+    components, based on software licensed to the Foundation.'\n+\n+The complete text of the resolution that was passed is:\n+\n+       WHEREAS, the Board of Directors deems it to be in the best\n+       interests of the Foundation and consistent with the\n+       Foundation's purpose to establish a Project Management\n+       Committee charged with the creation and maintenance of\n+       open-source software related to reusable libraries and\n+       components, for distribution at no charge to the public.\n+\n+       NOW, THEREFORE, BE IT RESOLVED, that a Project Management\n+       Committee (PMC), to be known as the \"Apache Commons PMC\", be\n+       and hereby is established pursuant to Bylaws of the Foundation;\n+       and be it further\n+\n+       RESOLVED, that the Apache Commons PMC be and hereby is\n+       responsible for the creation and maintenance of software\n+       related to reusable libraries and components, based on software\n+       licensed to the Foundation; and be it further\n+\n+       RESOLVED, that the office of \"Vice President, Apache Commons\"\n+       be and hereby is created, the person holding such office to\n+       serve at the direction of the Board of Directors as the chair\n+       of the Apache Commons PMC, and to have primary responsibility\n+       for management of the projects within the scope of\n+       responsibility of the Apache Commons PMC; and be it further\n+\n+       RESOLVED, that the persons listed immediately below be and\n+       hereby are appointed to serve as the initial members of the\n+       Apache Commons PMC:\n+\n+              Aaron Bannert\n+              Ken Coar (chair)\n+              Peter Donald\n+              Justin Erenkrantz\n+              Brian W. Fitzpatrick\n+              Jim Jagielski\n+              Geir Magnusson Jr.\n+              Greg Stein\n+              Sander Striker\n+\n+       NOW, THEREFORE, BE IT FURTHER RESOLVED, that Ken Coar be and\n+       hereby is appointed to the office of Vice President, Apache\n+       Commons, to serve in accordance with and subject to the\n+       direction of the Board of Directors and the Bylaws of the\n+       Foundation until death, resignation, retirement, removal or\n+       disqualification, or until a successor is appointed; and be it\n+       further\n+\n+       RESOLVED, that the initial Apache Commons PMC be and hereby is\n+       tasked with the creation of a set of bylaws intended to\n+       encourage open development and increased participation in the\n+       Apache Commons Project.\n+\n+#\n+# Local Variables:\n+# mode: indented-text\n+# tab-width: 4\n+# indent-tabs-mode: nil\n+# tab-stop-list: (4 6 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80)\n+# End:\n+#\ndiff --git a/t/t9010/src.diff0 b/t/t9010/src.diff0\nnew file mode 100644\nindex 0000000000000000000000000000000000000000..50a49fc1065a313b8592645614e3779261c981fe\nGIT binary patch\nliteral 74\nzcmWFz^J8FWobJ&$y_(yQVFD||Mux_A*9puDK#;ExT$)stT2!2wpQlisnUkZCl&Vl#\nWoSLGLmS3a*lSt0bD=Es);Q|10>loAk\n\nliteral 0\nHcmV?d00001\n\ndiff --git a/t/t9010/src.done b/t/t9010/src.done\nnew file mode 100644\nindex 0000000..56b2572\n--- /dev/null\n+++ b/t/t9010/src.done\n@@ -0,0 +1,522 @@\n+APACHE COMMONS PROJECT\n+STATUS: -*-indented-text-*- Last modified at [$Date$]\n+\n+Background:\n+    o IRC channel #apache-commons on irc.openprojects.net\n+      traffic is logged to <URL:http://Source-Zone.Org/apache-irc/>\n+      so that the content of interactive discussions is available\n+      to everyone\n+\n+Project committers (as of 2002-10-27):\n+    o commons:\n+      aaron,coar,donaldp,jerenkrantz,fitz,geirm,gstein,jim,striker\n+    o commons-site:\n+      aaron,coar,donaldp,jerenkrantz,fitz,geirm,gstein,jim,striker,\n+      sanders,nicolaken\n+\n+Release:\n+    none yet; still defining mission :-)\n+\n+\n+Resolved Issues:\n+\n+    o Commons is a parent of reusable code projects. These projects\n+      may be used by other projects of the ASF, but it is not a\n+      requirement.\n+\n+    o The Commons will be language-agnostic.\n+\n+    o Projects that are \"in scope\" are defined as:\n+    \n+      - Existing components that are, or would be, useful to multiple\n+        projects\n+\n+      - If a component does not fit the (TBD) goals of Apache Commons,\n+        then it is not considered \"in scope\" just because it has no\n+        other home. In other words, the Apache Commons is not a place\n+        of last refuge if the component does not match the Apache\n+        Commons' goals.\n+\n+      - Reusable libraries\n+        [ gstein: we should expand this definition for the mission\n+          statement; examples provided were serf and regexp ]\n+\n+      - Components that do not fit cleanly into any other top-level\n+        project, but they do fit the goals of Commons.\n+\n+    o Voting will follow the \"standard Apache voting guidelines\"\n+\n+      [ be nice to refer to an Incubator doc here ]\n+\n+    o All code donations [to the ASF, destined for Apache Commons]\n+      arrive via the Incubator, unless the Incubator states they can\n+      be placed directly into Commons.\n+\n+    o Existing Commons committers can start new components without a\n+      detour to the Incubator. These new components must be approved\n+      by the PMC and must meet the (TBD) goals of Apache Commons.\n+\n+\n+Pending issues:\n+    o Co    o Subversion will be used for version controlComing up with a set of bylaws for the project\n+\n+    o Enabling Reply-to on the @commons lists\n+      (pmc@ will *not* use reply-to munging, but user lists\n+      will be determined by user majority; this item applies\n+      to lists for which the decision has not yet been made)\n+      +1: aaron, coar, donaldp, geirm, acoliver, mas, bayard, sanders\n+      -1: fitz, gstein, jerenkrantz, striker, jim\n+\n+    o The name 'Commons' has caused some heartburn with the\n+      Jakarta community because of the Jakarta-Commons project.\n+      Should we rename to avoid conflicts and keep the peace?\n+      Conflicts would include Java namespace as well as\n+      philosophical aspects.\n+      +1: \n+      +0: coar (i'm willing)\n+      -0: jerenkrantz, donaldp, striker, gstein, fitz\n+      -1: sanders\n+\n+    o If we rename, to what?  What words/names describe our\n+      purpose?\n+      - toolbox\n+        +0: gstein (I'd be +1 but for the confusion with the existing\n+                    Apache Toolbox project, but *really* like this\n+                    name)\n+      - toolchest\n+        +0.5: gstein\n+      - tools\n+        +0: gstein\n+        -1: donaldp (tools are different to components)\n+      - components\n+        +0: gstein (a bit long)\n+      - util\n+      - library\n+        +0: gstein (doesn't fit well with perl/python \"modules\")\n+      - suite (sweet?)\n+      - belt (as in bat-belt or tool-belt)\n+      - mcgyver\n+      - foundry or mill\n+        +1: sanders (maybe too 'SourceForgeesque')\n+        -0: donaldp (If reorg goes through we may have multiple\n+                     foundaries or federations for different \"concepts\")\n+      - federation\n+      - share or shared\n+      - stuff\n+        +.3: fitz :)\n+      - ?\n+\n+    o Style for the mailing lists:\n+    \n+      One community mailing list, with specific breakouts:\n+        +1: fitz, jerenkrantz, sanders, coar,\n+            donaldp (lets start here and evolve)\n+      +0.5: mas\n+        -0: aaron (too early)\n+      \n+      Topical mailing lists:\n+        +1: gstein, scolebourne, acoliver, striker\n+        -0: aaron (too early), jerenkrantz\n+      -0.1: mas, sanders (too early for this), donaldp\n+        -1: coar\n+      \n+      Per-language mailing lists:\n+        -0: aaron (too early)\n+      -0.1: mas\n+        -1: gstein, sanders, fitz, jerenkrantz, striker, coar\n+\n+      Per-component mailing lists as a default (breakouts will create\n+          these as a matter of course, this is about the default)\n+      +0.7: mas\n+        -0: aaron (too early)\n+      -0.9: sanders\n+        -1: gstein, fitz, jerenkrantz, striker\n+\n+    o A number of very valid issues have been brought up on the\n+      list. We need to figure out how the Commons Project will\n+      deal with each of these, in terms of new components and\n+      how those components will contain code projects. This list\n+      is only meant to keep record of all the issues:\n+\n+        - Releasable pieces\n+        - Release rules\n+        - Voting scope\n+        - Directory structure and naming conventions\n+        - Coding style\n+        - Build system consistency (or inconsistency)\n+        - Namespace issues (esp. w/ java)\n+        - Language vs. Functional\n+\n+    o Default commit privileges\n+    \n+      - Commons-wide\n+        +1:\n+        -1: gstein, striker, donaldp\n+      \n+      - Per-component\n+        +1: gstein, striker, donaldp, jerenkrantz\n+        -1:\n+      \n+      - Per-component with self-chosen aggregation\n+        +1: gstein, donaldp\n+        -1:\n+\n+    o Granularity of CVS repositories for components (this excludes\n+      commons-site)\n+    \n+      - Commons-wide\n+        +1: gstein, donaldp, jerenkrantz\n+        -1:\n+      \n+      - Per-topic\n+        +1:\n+        -0: gstein, donaldp, jerenkrantz\n+        -1: \n+      \n+      - Per-component\n+        +1:\n+        -1: gstein, donaldp, jerenkrantz\n+\n+\n+Project Mission:\n+\n+What is the project's mission?  Our statement of goals/mission/vision\n+should arise from the answers to the following and other questions:\n+(jim notes that defining something after the fact seems very backwards\n+ and broken; gstein notes that we're refining the board-provided\n+ charter)\n+\n+    o Should commons have an sandbox component to ease infrastructure\n+      burden on smaller code bases?\n+      +1: coar, donaldp, jerenkrantz, gstein, sanders (non-binding)\n+      +0: fitz\n+      -0: striker\n+      -1: jim (the PMC is about reusability, not sandbox),\n+          aaron (what jim said; and go see incubator)\n+\n+    o What types of components would be appropriate for this project? \n+      (\"in scope\")\n+\n+      - Tools that help/promote reusability?\n+        Hypothetical: ant, jlibtool, ASF-based autoconf\n+        +1: jerenkrantz, gstein, striker, fitz, sanders (non-binding)\n+        -0: donaldp (prefer a tools PMC for that)\n+        -1: aaron (too broad, don't belong here)\n+\n+      - Development frameworks?\n+        Hypothetical: avalon\n+        +1: fitz\n+        -0: donaldp (how do we determine this given we would prolly\n+                     accept it if it was new?)\n+        -1: gstein (the avalon components, but not the whole bugger),\n+            striker, sanders (non-binding)\n+\n+     - Components that fit the (TBD) goals of Commons, have a more\n+       \"logical\" home elsewhere in the ASF, but were rejected by that\n+       home?\n+        +1: gstein, donaldp\n+         0: striker (on a case by case basis, taking reasons for rejection\n+                     seriously into account. Abstain from vote until\n+                     rephrased),\n+            fitz (what striker said), aaron (what fitz said)\n+        -1: jerenkrantz, sanders (non-binding)\n+\n+      FOLD BELOW VOTES INTO ABOVE? (i.e. eliminate the \"donation\" wording)\n+      - Donations that could fit but have a more obvious (proper) home which\n+        has already rejected it?\n+        +1: coar, donaldp, gstein (note the \"might fit\" term)\n+        -0:\n+        -1: jerenkrantz, jim, aaron, striker, fitz\n+\n+      - Existing ASF components whose committers believe that they\n+        are a better fit under commons and the commons PMC agrees?\n+        (If this component were brought up as new, we would accept it.)\n+        +1: coar, donaldp, jerenkrantz, striker, gstein, fitz\n+        -1: jim (by this definition httpd could be in commons)\n+                (gstein says: see the \"if\" part; we wouldn't accept httpd)\n+                (jim says: until we better define what the PMC would or\n+                 would not accept, then this seems too wishy-washy to me)\n+             (gstein says: jim, you're blocking closure on this;\n+              how would you refine the phrasing here; the intent\n+              here is to accept components from the other Commons\n+              projects or projects with reusable component),\n+             aaron (we need to differenciate ourselves from other\n+                    libraries first, namely APR)\n+\n+      - Packages being worked on by Apache developers, with a clear\n+        affiliation, that can't or won't be bundled?  (E.g., an\n+        httpd module)\n+        +1: coar, donaldp\n+        -1: jerenkrantz, striker, gstein, fitz, jim, aaron\n+       CLOSE THIS? (as \"not passed\"; what is a good way to phrase this?)\n+\n+      - Should we have a minimum bar of entry for components?\n+        +1:\n+        -0: donaldp, gstein\n+        -1:\n+       \n+      - Should we have a minimum set of requirements before components\n+        are released?\n+        +1: donaldp, gstein (mixed, see below), striker\n+        -1: jerenkrantz (what is released?)\n+\n+      - If yes to above then which things should be part of minimum\n+        requirements?\n+\n+        documentation: require basic overview and user docs\n+        +1: donaldp\n+        -0: gstein (recommend highly, but let the committers determine\n+                    what is right for the component),\n+            striker, jerenkrantz\n+        -1:\n+\n+        uptodate website: require website be updated to latest release\n+                          but may still host previous release docs.\n+        +1: donaldp, gstein, striker\n+        -0: jerenkrantz\n+        -1:\n+\n+        unit tests: (okay so this will never get consensus but ...)\n+        +1: donaldp\n+        -1: gstein (unit tests should be recommended, but not\n+                    mandated; I also find it unreasonable for initial\n+                    development/pre-alpha releases, but it can make\n+                    sense for \"final\" types of releases),\n+            striker, jerenkrantz\n+\n+        versioning standard: derived from\n+            http://apr.apache.org/versioning.html\n+            http://jakarta.apache.org/commons/versioning.html\n+        +1: donaldp, gstein, striker, jerenkrantz\n+        -1:\n+\n+        release process: derived from\n+          http://jakarta.apache.org/commons/releases.html\n+          http://jakarta.apache.org/turbine/maven/development/release-process.html\n+          http://cvs.apache.org/viewcvs.cgi/jakarta-ant/ReleaseInstructions?rev=1.9.2.1&content-type=text/vnd.viewcvs-markup\n+        +1: donaldp\n+        -1: gstein (we should provide \"best practices\" but allow each\n+                    components' committers to define their rules),\n+            striker, jerenkrantz\n+\n+        deprecation process: (java specific?)\n+          http://jakarta.apache.org/turbine/maven/development/deprecation.html\n+        +1: donaldp, gstein (I see this as part of the \"versioning\"\n+                             process, and we can provide best\n+                             practices here)\n+        -0: jerenkrantz (kinda sorta versioning, but not quite)\n+        -1:\n+\n+        CVS/Subversion branching:\n+          http://jakarta.apache.org/turbine/maven/development/branches.html\n+        +1: donaldp\n+        -1: gstein (we should provide \"best practices\" but allow each\n+                    components' committers to define their rules),\n+            striker, jerenkrantz\n+\n+\n+Candidate Projects:\n+\n+    o APR's serf project has voted itself to move into Commons.\n+    \n+      - Should the PMC accept it as fitting the Commons goal?\n+        +1: gstein, fitz, jerenkrantz, striker, donaldp\n+        -1: aaron (no such thing as \"the Commons goal\", how can it fit it?)\n+\n+      - When should it move?\n+\n+        Whenever it likes:\n+          +1: gstein, sanders, jerenkrantz, striker\n+          +0: donaldp (+1 if we use subversion, but if using CVS \n+              we should hold off until structure is decided upon)\n+          -1: aaron (after we know why it fits)\n+\n+        Give us a while:\n+          +1: fitz (what's the hurry?), aaron\n+          -0: gstein (we're only talking about a small seed of a\n+                codebase; it won't get in our way as we complete the\n+                charter), striker\n+\n+      - Where should the CVS code be located?\n+      \n+        commons/serf   (each component under top-level)\n+            +1: sanders (works well at jakarta-commons)\n+                fitz (Please don't mix interface and implementation \n+                  of commons!), aaron\n+            +0: jerenkrantz\n+          -0.5: gstein\n+            -1: donaldp (makes it difficult to update all related \n+                         projects with a single sweep)\n+\n+        commons/components/serf   (all components under this dir,\n+            leaving the top open for other non-code items)\n+          +0: gstein, striker, donaldp (is this just dev with a \n+                                        different name?\n+                                        (gstein says \"yes\"))\n+          -1: fitz, aaron, jerenkrantz\n+        \n+        commons/clients/serf   (topical-groups under top-level)\n+          +1: gstein, jerenkrantz\n+          -1: fitz, aaron, donaldp\n+        \n+        commons/dev/serf  (all components under \"dev\")\n+          +1: gstein, donaldp (if we are having a single \n+                               monolithic repo for all commons)\n+          -1: fitz, aaron, jerenkrantz\n+        \n+        commons/bootstrap/serf  (serf is very early stage, so maybe we\n+            have a \"bootstrap\" area; this is different from Incubator\n+            since the existing committers do not need \"training\")\n+          +1: gstein, donaldp\n+          -1: fitz, aaron, jerenkrantz\n+\n+        commons/???\n+\n+        commons/c/serf (separate out component based on language\n+                        and then have a flat structure underneath)\n+          +1: donaldp\n+          -1: jerenkrantz\n+\n+      - What mailing list should it use for dev discussions?\n+      \n+        general@commons.apache.org:  (one group for all discussion;\n+                                      dev and non-dev alike)\n+          -0.5: gstein\n+            -1: striker, aaron, jerenkrantz\n+        \n+        dev@commons.apache.org:  (one group for dev discussion;\n+                                  general@ remains for non-dev)\n+          +1: gstein, fitz, sanders, jerenkrantz, striker, donaldp\n+          \n+        clients-dev@commons.apache.org:\n+           (this is really TOPICNAME-dev@ where I preselected\n+            \"clients\" for TOPICNAME; this question is whether this\n+            style would be appropriate)\n+          +1: gstein, striker\n+          -0: sanders, donaldp (maybe in the future but too early),\n+              jerenkrantz\n+          -1: aaron (what is \"clients\"? I'd probably be +1 if I knew\n+                     what that was)\n+\n+      - Note: serf has no web site, so there isn't a need to figure\n+        that out right now.\n+\n+\n+Assets:\n+    DNS:                commons.apache.org\n+    \n+    Mailing lists:      general@commons.apache.org\n+                        announce@commons.apache.org\n+                        pmc@commons.apache.org\n+                        cvs@commons.apache.org\n+                        \n+                        [ core-cvs@commons.apache.org in case we\n+                          create a commons-core CVS module ]\n+\n+    Web site:           http://commons.apache.org/\n+    \n+    Repositories:       commons        (code, info, etc)\n+                        commons-site   (the web site)\n+\n+\n+PMC Members:\n+\n+    Aaron Bannert <aaron@apache.org>\n+    Ken Coar <coar@apache.org>\n+    Peter Donald <peter@apache.org>\n+    Justin Erenkrantz <jerenkrantz@apache.org>\n+    Brian W. Fitzpatrick <fitz@apache.org>\n+    Jim Jagielski <jim@apache.org>\n+    Geir Magnusson Jr. <geirm@apache.org>\n+    Greg Stein <gstein@lyra.org>\n+    Sander Striker <striker@apache.org>\n+\n+    Note: Ken Coar is the Chair\n+\n+\n+PMC Members, pending Board approval:\n+\n+    none yet\n+\n+    [ this may become obsolete; the Board is discussing a way for the\n+      Chair to directly alter the PMC membership; until then, however,\n+      we need PMC members ratified by the board, and this tracks them ]\n+\n+\n+Committers:\n+\n+    none yet [still defining mission]\n+\n+\n+Invited Committers:\n+\n+    none yet\n+\n+\n+Current mission/charter as approved by the board:\n+\n+    'The Apache Commons PMC hereby is responsible for the creation\n+    and maintenance of software related to reusable libraries and\n+    components, based on software licensed to the Foundation.'\n+\n+The complete text of the resolution that was passed is:\n+\n+       WHEREAS, the Board of Directors deems it to be in the best\n+       interests of the Foundation and consistent with the\n+       Foundation's purpose to establish a Project Management\n+       Committee charged with the creation and maintenance of\n+       open-source software related to reusable libraries and\n+       components, for distribution at no charge to the public.\n+\n+       NOW, THEREFORE, BE IT RESOLVED, that a Project Management\n+       Committee (PMC), to be known as the \"Apache Commons PMC\", be\n+       and hereby is established pursuant to Bylaws of the Foundation;\n+       and be it further\n+\n+       RESOLVED, that the Apache Commons PMC be and hereby is\n+       responsible for the creation and maintenance of software\n+       related to reusable libraries and components, based on software\n+       licensed to the Foundation; and be it further\n+\n+       RESOLVED, that the office of \"Vice President, Apache Commons\"\n+       be and hereby is created, the person holding such office to\n+       serve at the direction of the Board of Directors as the chair\n+       of the Apache Commons PMC, and to have primary responsibility\n+       for management of the projects within the scope of\n+       responsibility of the Apache Commons PMC; and be it further\n+\n+       RESOLVED, that the persons listed immediately below be and\n+       hereby are appointed to serve as the initial members of the\n+       Apache Commons PMC:\n+\n+              Aaron Bannert\n+              Ken Coar (chair)\n+              Peter Donald\n+              Justin Erenkrantz\n+              Brian W. Fitzpatrick\n+              Jim Jagielski\n+              Geir Magnusson Jr.\n+              Greg Stein\n+              Sander Striker\n+\n+       NOW, THEREFORE, BE IT FURTHER RESOLVED, that Ken Coar be and\n+       hereby is appointed to the office of Vice President, Apache\n+       Commons, to serve in accordance with and subject to the\n+       direction of the Board of Directors and the Bylaws of the\n+       Foundation until death, resignation, retirement, removal or\n+       disqualification, or until a successor is appointed; and be it\n+       further\n+\n+       RESOLVED, that the initial Apache Commons PMC be and hereby is\n+       tasked with the creation of a set of bylaws intended to\n+       encourage open development and increased participation in the\n+       Apache Commons Project.\n+\n+#\n+# Local Variables:\n+# mode: indented-text\n+# tab-width: 4\n+# indent-tabs-mode: nil\n+# tab-stop-list: (4 6 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80)\n+# End:\n+#\ndiff --git a/test-svn-fe.c b/test-svn-fe.c\nindex 77cf78a..658c2a7 100644\n--- a/test-svn-fe.c\n+++ b/test-svn-fe.c\n@@ -4,14 +4,37 @@\n \n #include \"git-compat-util.h\"\n #include \"vcs-svn/svndump.h\"\n+#include \"vcs-svn/svndiff.h\"\n+#include \"vcs-svn/line_buffer.h\"\n \n int main(int argc, char *argv[])\n {\n-\tif (argc != 2)\n-\t\tusage(\"test-svn-fe <file>\");\n-\tsvndump_init(argv[1]);\n-\tsvndump_read(NULL);\n-\tsvndump_deinit();\n-\tsvndump_reset();\n-\treturn 0;\n+\tstatic const char test_svnfe_usage[] =\n+\t\t\"test-svn-fe (<dumpfile> | [-d] <preimage> <delta> <len>)\";\n+\tif (argc < 2)\n+\t\tusage(test_svnfe_usage);\n+\tif (argc == 2) {\n+\t\tsvndump_init(argv[1]);\n+\t\tsvndump_read(NULL);\n+\t\tsvndump_deinit();\n+\t\tsvndump_reset();\n+\t\treturn 0;\n+\t}\n+\tif (argc == 5 && !strcmp(argv[1], \"-d\")) {\n+\t\tstruct line_buffer preimage = LINE_BUFFER_INIT;\n+\t\tstruct line_buffer delta = LINE_BUFFER_INIT;\n+\t\tbuffer_init(&preimage, argv[2]);\n+\t\tbuffer_init(&delta, argv[3]);\n+\t\tif (svndiff0_apply(&delta, (off_t) strtoull(argv[4], NULL, 0),\n+\t\t\t\t   &preimage, stdout))\n+\t\t\treturn 1;\n+\t\tif (buffer_deinit(&preimage))\n+\t\t\tdie_errno(\"cannot close preimage\");\n+\t\tif (buffer_deinit(&delta))\n+\t\t\tdie_errno(\"cannot close delta\");\n+\t\tbuffer_reset(&preimage);\n+\t\tbuffer_reset(&delta);\n+\t\treturn 0;\n+\t}\n+\tusage(test_svnfe_usage);\n }\ndiff --git a/vcs-svn/svndiff.c b/vcs-svn/svndiff.c\nindex df0b1a2..1668bf7 100644\n--- a/vcs-svn/svndiff.c\n+++ b/vcs-svn/svndiff.c\n@@ -4,6 +4,7 @@\n  */\n \n #include \"git-compat-util.h\"\n+#include \"sliding_window.h\"\n #include \"line_buffer.h\"\n \n /*\n@@ -12,15 +13,53 @@\n  * See http://svn.apache.org/repos/asf/subversion/trunk/notes/svndiff.\n  *\n  * svndiff0 ::= 'SVN\\0' window window*;\n+ * window ::= int int int int int instructions inline_data;\n+ * instructions ::= instruction*;\n+ * instruction ::= view_selector int int\n+ *   | copyfrom_data int\n+ *   | packed_view_selector int\n+ *   | packed_copyfrom_data\n+ *   ;\n+ * view_selector ::= copyfrom_source\n+ *   | copyfrom_target\n+ *   ;\n+ * copyfrom_source ::= # binary 00 000000;\n+ * copyfrom_target ::= # binary 01 000000;\n+ * copyfrom_data ::= # binary 10 000000;\n+ * packed_view_selector ::= # view_selector OR-ed with 6 bit value;\n+ * packed_copyfrom_data ::= # copyfrom_data OR-ed with 6 bit value;\n  * int ::= highdigit* lowdigit;\n  * highdigit ::= # binary 1000 0000 OR-ed with 7 bit value;\n  * lowdigit ::= # 7 bit value;\n  */\n \n+#define INSN_MASK\t0xc0\n+#define INSN_COPYFROM_SOURCE\t0000\n+#define INSN_COPYFROM_TARGET\t0x40\n+#define INSN_COPYFROM_DATA\t0x80\n+#define OPERAND_MASK\t0x3f\n+\n #define VLI_CONTINUE\t0x80\n #define VLI_DIGIT_MASK\t0x7f\n #define VLI_BITS_PER_DIGIT 7\n \n+struct window {\n+\tstruct {\n+\t\tstruct view *window;\n+\t\tsize_t declared_len;\n+\t} in;\n+\tstruct {\n+\t\toff_t off;\n+\t\tstruct strbuf buf;\n+\t\tsize_t declared_len;\n+\t} out;\n+\tstruct strbuf instructions;\n+\tstruct {\n+\t\tstruct strbuf buf;\n+\t\tsize_t declared_len;\n+\t} data;\n+};\n+\n static int read_magic(struct line_buffer *in, off_t *len)\n {\n \tstatic const char magic[] = {'S', 'V', 'N', '\\0'};\n@@ -75,3 +114,229 @@ static int parse_int(const char **buf, size_t *result, const char *end)\n \treturn error(\"Invalid instruction: incomplete integer %\"PRIu64,\n \t\t     (uint64_t) rv);\n }\n+\n+static int read_offset(struct line_buffer *in, off_t *result, off_t *len)\n+{\n+\tuintmax_t val;\n+\tif (read_int(in, &val, len))\n+\t\treturn -1;\n+\tif (val > maximum_signed_value_of_type(off_t))\n+\t\treturn error(\"Unrepresentable offset: %\"PRIuMAX, val);\n+\t*result = val;\n+\treturn 0;\n+}\n+\n+static int read_length(struct line_buffer *in, size_t *result, off_t *len)\n+{\n+\tuintmax_t val;\n+\tif (read_int(in, &val, len))\n+\t\treturn -1;\n+\tif (val > SIZE_MAX)\n+\t\treturn error(\"Unrepresentable length: %\"PRIuMAX, val);\n+\t*result = val;\n+\treturn 0;\n+}\n+\n+static int read_chunk(struct line_buffer *delta, off_t *delta_len,\n+\t\t      struct strbuf *buf, size_t len)\n+{\n+\tint truncated = 0;\n+\tstrbuf_reset(buf);\n+\t/* Need to truncate? */\n+\tif (len > maximum_signed_value_of_type(off_t)) {\n+\t\tlen = (size_t) maximum_signed_value_of_type(off_t);\n+\t\ttruncated = 1;\n+\t}\n+\tif ((off_t) len > *delta_len) {\n+\t\tlen = *delta_len;\n+\t\ttruncated = 1;\n+\t}\n+\tbuffer_read_binary(buf, len, delta);\n+\t*delta_len -= buf->len;\n+\tif (buf->len < len)\n+\t\ttruncated = 1;\n+\treturn truncated;\n+}\n+\n+\n+static int write_strbuf(struct strbuf *sb, FILE *out)\n+{\n+\tif (fwrite(sb->buf, 1, sb->len, out) == sb->len)\t/* Success. */\n+\t\treturn 0;\n+\treturn error(\"Cannot write: %s\\n\", strerror(errno));\n+}\n+\n+static int copyfrom_source(struct window *ctx, const char **instructions,\n+\t\t\t   size_t nbytes, const char *insns_end)\n+{\n+\tsize_t offset;\n+\tif (parse_int(instructions, &offset, insns_end))\n+\t\treturn -1;\n+\tif (unsigned_add_overflows(offset, nbytes) ||\n+\t    offset + nbytes > ctx->in.declared_len)\n+\t\treturn error(\"Invalid delta: copies unallocated source data.\");\n+\tif (offset + nbytes > ctx->in.window->buf.len)\t/* Input exhausted. */\n+\t\tnbytes = ctx->in.window->buf.len - offset;\n+\tstrbuf_add(&ctx->out.buf, ctx->in.window->buf.buf + offset, nbytes);\n+\treturn 0;\n+}\n+\n+static int copyfrom_target(struct window *ctx, const char **instructions,\n+\t\t\t   size_t nbytes, const char *insns_end)\n+{\n+\tconst size_t out_pos = ctx->out.buf.len;\n+\tsize_t offset;\n+\tif (parse_int(instructions, &offset, insns_end))\n+\t\treturn -1;\n+\tif (offset > out_pos)\n+\t\treturn error(\"Invalid delta: copies from the future.\");\n+\tif (unsigned_add_overflows(offset, nbytes) ||\n+\t    offset + nbytes > ctx->out.declared_len)\n+\t\treturn error(\"Invalid delta: copies unallocated target data.\");\n+\twhile (nbytes) {\n+\t\tstrbuf_addch(&ctx->out.buf, ctx->out.buf.buf[offset++]);\n+\t\tnbytes--;\n+\t}\n+\treturn 0;\n+}\n+\n+static int copyfrom_data(struct window *ctx, size_t *data_pos, size_t nbytes)\n+{\n+\tconst size_t pos = *data_pos;\n+\tif (unsigned_add_overflows(pos, nbytes) ||\n+\t    pos + nbytes > ctx->data.declared_len)\n+\t\treturn error(\"Invalid delta: copies unallocated inline data.\");\n+\tif (pos + nbytes > ctx->data.buf.len)\t/* Data exhausted. */\n+\t\tnbytes = ctx->data.buf.len - pos;\n+\tstrbuf_add(&ctx->out.buf, ctx->data.buf.buf + pos, nbytes);\n+\t*data_pos += nbytes;\n+\treturn 0;\n+}\n+\n+static int parse_first_operand(const char **buf, size_t *out, const char *end)\n+{\n+\tsize_t result = (unsigned char) *(*buf)++ & OPERAND_MASK;\n+\tif (result) {\n+\t\t*out = result;\n+\t\treturn 0;\n+\t}\n+\treturn parse_int(buf, out, end);\n+}\n+\n+static int step(struct window *ctx, const char **instructions, size_t *data_pos)\n+{\n+\tunsigned char instruction;\n+\tconst char *insns_end = ctx->instructions.buf + ctx->instructions.len;\n+\tconst size_t out_pos = ctx->out.buf.len;\n+\tsize_t nbytes;\n+\tassert(ctx);\n+\tassert(instructions && *instructions);\n+\n+\tinstruction = (unsigned char) **instructions;\n+\tif (parse_first_operand(instructions, &nbytes, insns_end))\n+\t\treturn -1;\n+\tif (unsigned_add_overflows(out_pos, nbytes) ||\n+\t    out_pos + nbytes > ctx->out.declared_len)\n+\t\treturn error(\"Invalid delta: output overflows buffer.\");\n+\n+\tswitch (instruction & INSN_MASK) {\n+\tcase INSN_COPYFROM_SOURCE:\n+\t\treturn copyfrom_source(ctx, instructions, nbytes, insns_end);\n+\tcase INSN_COPYFROM_TARGET:\n+\t\treturn copyfrom_target(ctx, instructions, nbytes, insns_end);\n+\tcase INSN_COPYFROM_DATA:\n+\t\treturn copyfrom_data(ctx, data_pos, nbytes);\n+\tdefault:\n+\t\treturn error(\"Invalid instruction %x\",\n+\t\t\t     (unsigned int) instruction);\n+\t}\n+}\n+\n+static int apply_window_in_core(struct window *ctx)\n+{\n+\tconst char *insn = ctx->instructions.buf;\n+\tsize_t data_pos = 0;\n+\n+\t/*\n+\t * Advance p while copying data from the source, target,\n+\t * and inline data views.\n+\t */\n+\twhile (insn && insn != ctx->instructions.buf + ctx->instructions.len)\n+\t\tif (step(ctx, &insn, &data_pos))\n+\t\t\treturn -1;\n+\treturn 0;\n+}\n+\n+static int apply_one_window(struct line_buffer *delta, off_t *delta_len,\n+\t\t\t    struct view *preimage, off_t preimage_len,\n+\t\t\t    off_t *out_offset, FILE *out)\n+{\n+\tstruct window ctx = {\n+\t\t{preimage, preimage_len},\t/* preimage */\n+\t\t{*out_offset, STRBUF_INIT, 0},\t/* postimage */\n+\t\tSTRBUF_INIT,\t\t\t/* instructions */\n+\t\t{STRBUF_INIT, 0}\t\t/* inline data */\n+\t};\n+\tsize_t instructions_len;\n+\tint rv = 0;\n+\tassert(out_offset);\n+\tassert(delta_len);\n+\n+\t/* \"source view\" offset and length already handled; */\n+\tif (read_length(delta, &ctx.out.declared_len, delta_len) ||\n+\t    read_length(delta, &instructions_len, delta_len) ||\n+\t    read_length(delta, &ctx.data.declared_len, delta_len))\n+\t\treturn -1;\n+\tif (read_chunk(delta, delta_len, &ctx.instructions, instructions_len))\n+\t\twarning(\"Invalid delta: \"\n+\t\t\t\"incomplete instructions (%\"PRIu64\"/%\"PRIu64\")\",\n+\t\t\t(uint64_t) ctx.instructions.len,\n+\t\t\t(uint64_t) instructions_len);\n+\tif (read_chunk(delta, delta_len, &ctx.data.buf, ctx.data.declared_len))\n+\t\t; /* inline data is truncated.  okay. */\n+\tif (buffer_ferror(delta)) {\n+\t\trv = error(\"Cannot read delta: %s\", strerror(errno));\n+\t\tgoto done;\n+\t}\n+\tif (apply_window_in_core(&ctx) || write_strbuf(&ctx.out.buf, out)) {\n+\t\trv = -1;\n+\t\tgoto done;\n+\t}\n+ done:\n+\tstrbuf_release(&ctx.out.buf);\n+\tstrbuf_release(&ctx.instructions);\n+\tstrbuf_release(&ctx.data.buf);\n+\treturn rv;\n+}\n+\n+int svndiff0_apply(struct line_buffer *delta, off_t delta_len,\n+\t\t   struct line_buffer *preimage, FILE *postimage)\n+{\n+\tstruct view preimage_view = {preimage, 0, STRBUF_INIT};\n+\toff_t out_offset = 0;\n+\tassert(delta && preimage && postimage);\n+\n+\tif (read_magic(delta, &delta_len))\n+\t\tgoto fail;\n+\twhile (delta_len > 0) {\t/* For each window: */\n+\t\toff_t pre_off = pre_off;\n+\t\tsize_t pre_len;\n+\t\tif (read_offset(delta, &pre_off, &delta_len) ||\n+\t\t    read_length(delta, &pre_len, &delta_len) ||\n+\t\t    move_window(&preimage_view, pre_off, pre_len) ||\n+\t\t    apply_one_window(delta, &delta_len,\n+\t\t\t\t     &preimage_view, pre_len,\n+\t\t\t\t     &out_offset, postimage))\n+\t\t\tgoto fail;\n+\t\tif (delta_len && buffer_at_eof(delta)) {\n+\t\t\terror(\"Delta ends early! (%\"PRIu64\" bytes remaining)\",\n+\t\t\t      (uint64_t) delta_len);\n+\t\t\tgoto fail;\n+\t\t}\n+\t}\n+\tstrbuf_release(&preimage_view.buf);\n+\treturn 0;\n+ fail:\n+\tstrbuf_release(&preimage_view.buf);\n+\treturn -1;\n+}\ndiff --git a/vcs-svn/svndiff.h b/vcs-svn/svndiff.h\nnew file mode 100644\nindex 0000000..a986099\n--- /dev/null\n+++ b/vcs-svn/svndiff.h\n@@ -0,0 +1,9 @@\n+#ifndef SVNDIFF_H_\n+#define SVNDIFF_H_\n+\n+#include \"line_buffer.h\"\n+\n+extern int svndiff0_apply(struct line_buffer *delta, off_t delta_len,\n+\t\t\t  struct line_buffer *preimage, FILE *postimage);\n+\n+#endif\n-- \n1.7.2.3\n"},{"id":"153380","messageId":"20101013091714.GA32608@burratino","threadId":"24407","inReplyTo":"20101011040140.GC5649@burratino","subject":"[PATCH/RFC 0/11] Building up the delta parser","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-13T09:17:14Z","receivedAt":"2010-10-13T09:17:14Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Jonathan Nieder wrote:\n\n> Implement an svndiff 0 interpreter\n\nI hear that was nigh unreadable, so here's a reroll.  Less\ncargo-cult support for broken deltas, more readability and tests.\nPatches apply on top of \"[PATCH 15/16] t9010 (svn-fe): Eliminate\ndependency on svn perl bindings\".  As before, the end result\nincludes a 'test-svn-fe -d' command that can apply svndiff0-format\ndeltas, meaning less binary garbage to worry about as you puzzle\nover that confusing \"svnrdump dump\" output in debugging sessions.\n\nQuestions?  Improvements?  Bugs?\n\nPatch 1 is a fixup to the variable-length integer parsing code, to\nreport unexpected EOF (i.e., declared content length too long)\ncorrectly when it occurs in the middle of such an integer.\n\nPatch 2 is the svndiff0 interpreter in broad strokes: read window,\nread window, read window, ....  The patch doesn't encode any\nknowledge about what actually goes _in_ a window aside from the\nheader, so it will error out for nonempty windows.\n\nPatch 3 teaches the nacent interpreter to keep the appropriate\npiece of the preimage in memory.  This is probably earlier in the\nseries than it ought to be, but I wanted to try out the sliding\nwindow code.\n\nWith patches 4 and 5, the interpreter learns to read the \"data\"\nand \"instructions\" section of a window.  The effect is observable\nbecause it finds the beginning of the next window correctly.\n\nPatch 6 is an example instruction (copyfrom_data).\n\nPatches 7-8 introduce some sanity checks.\n\nPatches 9 and 10 are another instruction (copyfrom_target) and\nanother sanity check.\n\nPatch 11 is the last instruction (copyfrom_source).  That's it.\nYou can apply deltas now!\n\nIf anything seems unclear, please don't spend time puzzling it\nout --- just yell at me, so the code or documentation can be\ncleaned up.  Happy reading.\n"},{"id":"153381","messageId":"20101013091954.GB32608@burratino","threadId":"24407","inReplyTo":"20101013091714.GA32608@burratino","subject":"[PATCH 01/11] fixup! vcs-svn: Learn to parse variable-length integers","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-13T09:19:54Z","receivedAt":"2010-10-13T09:19:54Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Report EOF correctly in integer parsing code.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\nThis patch is meant for squashing.\n\n vcs-svn/svndiff.c |    3 ++-\n 1 files changed, 2 insertions(+), 1 deletions(-)\n\ndiff --git a/vcs-svn/svndiff.c b/vcs-svn/svndiff.c\nindex df0b1a2..36d2b30 100644\n--- a/vcs-svn/svndiff.c\n+++ b/vcs-svn/svndiff.c\n@@ -45,7 +45,8 @@ static int read_int(struct line_buffer *in, uintmax_t *result, off_t *len)\n \twhile (sz) {\n \t\tint ch = buffer_read_char(in);\n \t\tif (ch == EOF)\n-\t\t\tbreak;\n+\t\t\treturn error(\"Delta ends early (%\"PRIu64\" bytes remaining)\",\n+\t\t\t\t     (uint64_t) sz);\n \t\tsz--;\n \t\trv <<= VLI_BITS_PER_DIGIT;\n \t\trv += (ch & VLI_DIGIT_MASK);\n-- \n1.7.2.3\n"},{"id":"153382","messageId":"20101013092143.GC32608@burratino","threadId":"24407","inReplyTo":"20101013091714.GA32608@burratino","subject":"[PATCH 02/11] vcs-svn: Skeleton of an svn delta parser","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-13T09:21:43Z","receivedAt":"2010-10-13T09:21:43Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"A delta in the subversion delta (svndiff0) format consists of the\nmagic bytes SVN\\0 followed by a sequence of windows, each beginning\nwith a window header consisting of five integers (with variable-length\nrepresentation):\n\n\tsource view offset\n\tsource view length\n\toutput length\n\tinstructions length\n\tauxiliary data length\n\nAdd an svndiff0_apply() function and test-svn-fe -d commandline tool\nto parse such a delta in the special case of not including any\ninstructions or auxiliary data.\n\nLater patches will add features to turn this into a fully functional\ndelta applier, for use by svn-fe in parsing the streams produced by\n\"svnrdump dump\" and \"svnadmin dump --deltas\".\n\nHelped-by: Ramkumar Ramachandra <artagnon@gmail.com>\nHelped-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n Makefile          |    4 +-\n t/t9011-svn-da.sh |   82 +++++++++++++++++++++++++++++++++++++++++++++++++++++\n test-svn-fe.c     |   39 ++++++++++++++++++++----\n vcs-svn/svndiff.c |   64 +++++++++++++++++++++++++++++++++++++++++\n vcs-svn/svndiff.h |    9 ++++++\n 5 files changed, 189 insertions(+), 9 deletions(-)\n create mode 100755 t/t9011-svn-da.sh\n create mode 100644 vcs-svn/svndiff.h\n\ndiff --git a/Makefile b/Makefile\nindex d99da33..966f5c7 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1766,7 +1766,7 @@ XDIFF_OBJS = xdiff/xdiffi.o xdiff/xprepare.o xdiff/xutils.o xdiff/xemit.o \\\n \txdiff/xmerge.o xdiff/xpatience.o\n VCSSVN_OBJS = vcs-svn/string_pool.o vcs-svn/line_buffer.o \\\n \tvcs-svn/repo_tree.o vcs-svn/fast_export.o vcs-svn/svndump.o \\\n-\tvcs-svn/sliding_window.o\n+\tvcs-svn/sliding_window.o vcs-svn/svndiff.o\n OBJECTS := $(GIT_OBJS) $(XDIFF_OBJS) $(VCSSVN_OBJS)\n \n dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d)\n@@ -1893,7 +1893,7 @@ xdiff-interface.o $(XDIFF_OBJS): \\\n $(VCSSVN_OBJS): \\\n \tvcs-svn/obj_pool.h vcs-svn/trp.h vcs-svn/string_pool.h \\\n \tvcs-svn/line_buffer.h vcs-svn/repo_tree.h vcs-svn/fast_export.h \\\n-\tvcs-svn/svndump.h vcs-svn/sliding_window.h\n+\tvcs-svn/sliding_window.h vcs-svn/svndump.h vcs-svn/svndiff.h\n endif\n \n exec_cmd.s exec_cmd.o: EXTRA_CPPFLAGS = \\\ndiff --git a/t/t9011-svn-da.sh b/t/t9011-svn-da.sh\nnew file mode 100755\nindex 0000000..8dccd16\n--- /dev/null\n+++ b/t/t9011-svn-da.sh\n@@ -0,0 +1,82 @@\n+#!/bin/sh\n+\n+test_description='test handling of deltas by dumpfile importer'\n+\n+. ./test-lib.sh\n+\n+test_expect_success 'setup' '\n+\t>empty &&\n+\tprintf foo >preimage\n+'\n+\n+test_expect_success 'reject empty delta' '\n+\ttest_must_fail test-svn-fe -d preimage empty 0\n+'\n+\n+test_expect_success 'delta can empty file' '\n+\tprintf \"SVNQ\" | q_to_nul >clear.delta &&\n+\ttest-svn-fe -d preimage clear.delta 4 >actual &&\n+\ttest_cmp empty actual\n+'\n+\n+test_expect_success 'one-window empty delta' '\n+\tprintf \"SVNQ%s\" \"QQQQQ\" | q_to_nul >clear.onewindow &&\n+\ttest-svn-fe -d preimage clear.onewindow 9 >actual &&\n+\ttest_cmp empty actual\n+'\n+\n+test_expect_success 'incomplete window header' '\n+\tprintf \"SVNQ%s\" \"QQQQQ\" | q_to_nul >clear.onewindow &&\n+\tprintf \"SVNQ%s\" \"QQ\" | q_to_nul >clear.partialwindow &&\n+\ttest_must_fail test-svn-fe -d preimage clear.onewindow 6 &&\n+\ttest_must_fail test-svn-fe -d preimage clear.partialwindow 6\n+'\n+\n+test_expect_success 'declared delta longer than actual delta' '\n+\tprintf \"SVNQ%s\" \"QQQQQ\" | q_to_nul >clear.onewindow &&\n+\tprintf \"SVNQ%s\" \"QQ\" | q_to_nul >clear.partialwindow &&\n+\ttest_must_fail test-svn-fe -d preimage clear.onewindow 14 &&\n+\ttest_must_fail test-svn-fe -d preimage clear.partialwindow 9\n+'\n+\n+test_expect_success 'two-window empty delta' '\n+\tprintf \"SVNQ%s%s\" \"QQQQQ\" \"QQQQQ\" | q_to_nul >clear.twowindow &&\n+\ttest-svn-fe -d preimage clear.twowindow 14 >actual &&\n+\ttest_must_fail test-svn-fe -d preimage clear.twowindow 13 &&\n+\ttest_cmp empty actual\n+'\n+\n+test_expect_success 'noisy zeroes' '\n+\tprintf \"SVNQ%s\" \\\n+\t\t\"RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRQQQQQ\" |\n+\t\ttr R \"\\200\" |\n+\t\tq_to_nul >clear.noisy &&\n+\tlen=$(wc -c <clear.noisy) &&\n+\ttest-svn-fe -d preimage clear.noisy $len &&\n+\ttest_cmp empty actual\n+'\n+\n+test_expect_success 'reject variable-length int in magic' '\n+\tprintf \"SVNRQ\" | tr R \"\\200\" | q_to_nul >clear.badmagic &&\n+\ttest_must_fail test-svn-fe -d preimage clear.badmagic 5\n+'\n+\n+test_expect_success 'truncated integer' '\n+\tprintf \"SVNQ%s%s\" \"QQQQQ\" \"QQQQRRQ\" |\n+\t\ttr R \"\\200\" |\n+\t\tq_to_nul >clear.fullint &&\n+\tprintf \"SVNQ%s%s\" \"QQQQQ\" \"QQQQRR\" |\n+\t\ttr RT \"\\201\" |\n+\t\tq_to_nul >clear.partialint &&\n+\ttest_must_fail test-svn-fe -d preimage clear.fullint 15 &&\n+\ttest-svn-fe -d preimage clear.fullint 16 &&\n+\ttest_must_fail test-svn-fe -d preimage clear.partialint 15\n+'\n+\n+test_expect_success 'nonempty (but unused) preimage view' '\n+\tprintf \"SVNQ%b\" \"Q\\003QQQ\" | q_to_nul >clear.readpreimage &&\n+\ttest-svn-fe -d preimage clear.readpreimage 9 >actual &&\n+\ttest_cmp empty actual\n+'\n+\n+test_done\ndiff --git a/test-svn-fe.c b/test-svn-fe.c\nindex 77cf78a..197a2c3 100644\n--- a/test-svn-fe.c\n+++ b/test-svn-fe.c\n@@ -4,14 +4,39 @@\n \n #include \"git-compat-util.h\"\n #include \"vcs-svn/svndump.h\"\n+#include \"vcs-svn/svndiff.h\"\n+#include \"vcs-svn/line_buffer.h\"\n \n int main(int argc, char *argv[])\n {\n-\tif (argc != 2)\n-\t\tusage(\"test-svn-fe <file>\");\n-\tsvndump_init(argv[1]);\n-\tsvndump_read(NULL);\n-\tsvndump_deinit();\n-\tsvndump_reset();\n-\treturn 0;\n+\tstatic const char test_svnfe_usage[] =\n+\t\t\"test-svn-fe (<dumpfile> | [-d] <preimage> <delta> <len>)\";\n+\tif (argc < 2)\n+\t\tusage(test_svnfe_usage);\n+\tif (argc == 2) {\n+\t\tsvndump_init(argv[1]);\n+\t\tsvndump_read(NULL);\n+\t\tsvndump_deinit();\n+\t\tsvndump_reset();\n+\t\treturn 0;\n+\t}\n+\tif (argc == 5 && !strcmp(argv[1], \"-d\")) {\n+\t\tstruct line_buffer preimage = LINE_BUFFER_INIT;\n+\t\tstruct line_buffer delta = LINE_BUFFER_INIT;\n+\t\tif (buffer_init(&preimage, argv[2]))\n+\t\t\tdie_errno(\"cannot open preimage\");\n+\t\tif (buffer_init(&delta, argv[3]))\n+\t\t\tdie_errno(\"cannot open delta\");\n+\t\tif (svndiff0_apply(&delta, (off_t) strtoull(argv[4], NULL, 0),\n+\t\t\t\t   &preimage, stdout))\n+\t\t\treturn 1;\n+\t\tif (buffer_deinit(&preimage))\n+\t\t\tdie_errno(\"cannot close preimage\");\n+\t\tif (buffer_deinit(&delta))\n+\t\t\tdie_errno(\"cannot close delta\");\n+\t\tbuffer_reset(&preimage);\n+\t\tbuffer_reset(&delta);\n+\t\treturn 0;\n+\t}\n+\tusage(test_svnfe_usage);\n }\ndiff --git a/vcs-svn/svndiff.c b/vcs-svn/svndiff.c\nindex 36d2b30..e572a93 100644\n--- a/vcs-svn/svndiff.c\n+++ b/vcs-svn/svndiff.c\n@@ -12,6 +12,7 @@\n  * See http://svn.apache.org/repos/asf/subversion/trunk/notes/svndiff.\n  *\n  * svndiff0 ::= 'SVN\\0' window window*;\n+ * window ::= int int int int int instructions inline_data;\n  * int ::= highdigit* lowdigit;\n  * highdigit ::= # binary 1000 0000 OR-ed with 7 bit value;\n  * lowdigit ::= # 7 bit value;\n@@ -76,3 +77,66 @@ static int parse_int(const char **buf, size_t *result, const char *end)\n \treturn error(\"Invalid instruction: incomplete integer %\"PRIu64,\n \t\t     (uint64_t) rv);\n }\n+\n+static int read_offset(struct line_buffer *in, off_t *result, off_t *len)\n+{\n+\tuintmax_t val;\n+\tif (read_int(in, &val, len))\n+\t\treturn -1;\n+\tif (val > maximum_signed_value_of_type(off_t))\n+\t\treturn error(\"Unrepresentable offset: %\"PRIuMAX, val);\n+\t*result = val;\n+\treturn 0;\n+}\n+\n+static int read_length(struct line_buffer *in, size_t *result, off_t *len)\n+{\n+\tuintmax_t val;\n+\tif (read_int(in, &val, len))\n+\t\treturn -1;\n+\tif (val > SIZE_MAX)\n+\t\treturn error(\"Unrepresentable length: %\"PRIuMAX, val);\n+\t*result = val;\n+\treturn 0;\n+}\n+\n+static int apply_one_window(struct line_buffer *delta, off_t *delta_len)\n+{\n+\tsize_t out_len;\n+\tsize_t instructions_len;\n+\tsize_t data_len;\n+\tassert(delta_len);\n+\n+\t/* \"source view\" offset and length already handled; */\n+\tif (read_length(delta, &out_len, delta_len) ||\n+\t    read_length(delta, &instructions_len, delta_len) ||\n+\t    read_length(delta, &data_len, delta_len))\n+\t\treturn -1;\n+\tif (instructions_len > 0)\n+\t\treturn error(\"What do you think I am?  A delta applier?\");\n+\tif (data_len > 0)\n+\t\treturn error(\"No support for inline data yet\");\n+\treturn 0;\n+}\n+\n+int svndiff0_apply(struct line_buffer *delta, off_t delta_len,\n+\t\t   struct line_buffer *preimage, FILE *postimage)\n+{\n+\tassert(delta && preimage && postimage);\n+\n+\tif (read_magic(delta, &delta_len))\n+\t\treturn -1;\n+\twhile (delta_len > 0) {\t/* For each window: */\n+\t\toff_t pre_off;\n+\t\tsize_t pre_len;\n+\t\tif (read_offset(delta, &pre_off, &delta_len) ||\n+\t\t    read_length(delta, &pre_len, &delta_len) ||\n+\t\t    apply_one_window(delta, &delta_len))\n+\t\t\treturn -1;\n+\t\tif (delta_len && buffer_at_eof(delta))\n+\t\t\treturn error(\"Delta ends early! \"\n+\t\t\t\t     \"(%\"PRIu64\" bytes remaining)\",\n+\t\t\t\t     (uint64_t) delta_len);\n+\t}\n+\treturn 0;\n+}\ndiff --git a/vcs-svn/svndiff.h b/vcs-svn/svndiff.h\nnew file mode 100644\nindex 0000000..a986099\n--- /dev/null\n+++ b/vcs-svn/svndiff.h\n@@ -0,0 +1,9 @@\n+#ifndef SVNDIFF_H_\n+#define SVNDIFF_H_\n+\n+#include \"line_buffer.h\"\n+\n+extern int svndiff0_apply(struct line_buffer *delta, off_t delta_len,\n+\t\t\t  struct line_buffer *preimage, FILE *postimage);\n+\n+#endif\n-- \n1.7.2.3\n"},{"id":"153383","messageId":"20101013093037.GD32608@burratino","threadId":"24407","inReplyTo":"20101013091714.GA32608@burratino","subject":"[PATCH 03/11] vcs-svn: Read the preimage while applying deltas","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-13T09:30:37Z","receivedAt":"2010-10-13T09:30:37Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"The source view offset heading each svndiff0 window represents a\nnumber of bytes past the beginning of the preimage.  Together with the\nsource view length, it instructs the delta applier about what portion\nof the preimage instructions will refer to.  Read in that data right\naway using the sliding window code.\n\nMaybe some day we will mmap() to prepare to read data more lazily.\n\nFor compatibility with Subversion's implementation, tolerate source\nview offsets pointing past the end of the preimage file (a later\npatch will remove this flexibility).  For simplicity, also permit\nsource views that start within the preimage and end outside of it,\neven though Subversion does not.\n\nThis does not teach the delta applier to read instructions or copy\ndata from the source view yet.  Deltas that would produce nonempty\noutput are still rejected.\n\nHelped-by: Ramkumar Ramachandra <artagnon@gmail.com>\nHelped-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\nIt occurs to me that Sam Vilain may well have something valuable to\nsay about this series, having implemented something similar[1].\n\nSam, this series adds an svndiff0 parser for git to use in parsing\nv3 dumps (which are way easier to produce with remote access to an\nsvn repository than v2 dumps).  The beginning of the series is at [2],\nthough that cover letter is out of date: now, modulo any new bugs\nI've introduced with this reroll, it is known to successfully apply\nall the deltas involved in a complete dump of the ASF repo.\n\nI am interested in improvements and complaints of all kinds.\n\n[1] http://search.cpan.org/~samv/Parse-SVNDiff-0.03/lib/Parse/SVNDiff.pm\n[2] http://thread.gmane.org/gmane.comp.version-control.git/151086/focus=158731\n\n t/t9011-svn-da.sh |   38 ++++++++++++++++++++++++++++++++++++++\n vcs-svn/svndiff.c |   22 +++++++++++++++-------\n 2 files changed, 53 insertions(+), 7 deletions(-)\n\ndiff --git a/t/t9011-svn-da.sh b/t/t9011-svn-da.sh\nindex 8dccd16..b9aad70 100755\n--- a/t/t9011-svn-da.sh\n+++ b/t/t9011-svn-da.sh\n@@ -79,4 +79,42 @@ test_expect_success 'nonempty (but unused) preimage view' '\n \ttest_cmp empty actual\n '\n \n+test_expect_success 'preimage view: right endpoint cannot backtrack' '\n+\tprintf \"SVNQ%b%b\" \"Q\\003QQQ\" \"Q\\002QQQ\" |\n+\t\tq_to_nul >clear.backtrack &&\n+\ttest_must_fail test-svn-fe -d preimage clear.backtrack 14\n+'\n+\n+test_expect_success 'preimage view: left endpoint can advance' '\n+\tprintf \"SVNQ%b%b\" \"Q\\003QQQ\" \"\\001\\002QQQ\" |\n+\t\tq_to_nul >clear.preshrink &&\n+\tprintf \"SVNQ%b%b\" \"Q\\003QQQ\" \"\\001\\001QQQ\" |\n+\t\tq_to_nul >clear.shrinkbacktrack &&\n+\ttest-svn-fe -d preimage clear.preshrink 14 >actual &&\n+\ttest_must_fail test-svn-fe -d preimage clear.shrinkbacktrack 14 &&\n+\ttest_cmp empty actual\n+'\n+\n+test_expect_success 'preimage view: offsets compared by value' '\n+\tprintf \"SVNQ%b%b\" \"\\001\\001QQQ\" \"\\0200Q\\003QQQ\" |\n+\t\tq_to_nul >clear.noisybacktrack &&\n+\tprintf \"SVNQ%b%b\" \"\\001\\001QQQ\" \"\\0200\\001\\002QQQ\" |\n+\t\tq_to_nul >clear.noisyadvance &&\n+\ttest_must_fail test-svn-fe -d preimage clear.noisybacktrack 15\n+\ttest-svn-fe -d preimage clear.noisyadvance 15 &&\n+\ttest_cmp empty actual\n+'\n+\n+test_expect_success 'preimage view: accept truncated preimage' '\n+\tprintf \"SVNQ%b\" \"\\010QQQQ\" | q_to_nul >clear.lateemptyread &&\n+\tprintf \"SVNQ%b\" \"\\010\\001QQQ\" | q_to_nul >clear.latenonemptyread &&\n+\tprintf \"SVNQ%b\" \"\\001\\010QQQ\" | q_to_nul >clear.longread &&\n+\ttest-svn-fe -d preimage clear.lateemptyread 9 >actual.emptyread &&\n+\ttest-svn-fe -d preimage clear.latenonemptyread 9 >actual.nonemptyread &&\n+\ttest-svn-fe -d preimage clear.longread 9 >actual.longread &&\n+\ttest_cmp empty actual.emptyread &&\n+\ttest_cmp empty actual.nonemptyread &&\n+\ttest_cmp empty actual.longread\n+'\n+\n test_done\ndiff --git a/vcs-svn/svndiff.c b/vcs-svn/svndiff.c\nindex e572a93..f2876b3 100644\n--- a/vcs-svn/svndiff.c\n+++ b/vcs-svn/svndiff.c\n@@ -4,6 +4,7 @@\n  */\n \n #include \"git-compat-util.h\"\n+#include \"sliding_window.h\"\n #include \"line_buffer.h\"\n \n /*\n@@ -122,21 +123,28 @@ static int apply_one_window(struct line_buffer *delta, off_t *delta_len)\n int svndiff0_apply(struct line_buffer *delta, off_t delta_len,\n \t\t   struct line_buffer *preimage, FILE *postimage)\n {\n+\tstruct view preimage_view = {preimage, 0, STRBUF_INIT};\n \tassert(delta && preimage && postimage);\n \n \tif (read_magic(delta, &delta_len))\n-\t\treturn -1;\n+\t\tgoto fail;\n \twhile (delta_len > 0) {\t/* For each window: */\n-\t\toff_t pre_off;\n+\t\toff_t pre_off = pre_off;\n \t\tsize_t pre_len;\n \t\tif (read_offset(delta, &pre_off, &delta_len) ||\n \t\t    read_length(delta, &pre_len, &delta_len) ||\n+\t\t    move_window(&preimage_view, pre_off, pre_len) ||\n \t\t    apply_one_window(delta, &delta_len))\n-\t\t\treturn -1;\n-\t\tif (delta_len && buffer_at_eof(delta))\n-\t\t\treturn error(\"Delta ends early! \"\n-\t\t\t\t     \"(%\"PRIu64\" bytes remaining)\",\n-\t\t\t\t     (uint64_t) delta_len);\n+\t\t\tgoto fail;\n+\t\tif (delta_len && buffer_at_eof(delta)) {\n+\t\t\terror(\"Delta ends early! (%\"PRIu64\" bytes remaining)\",\n+\t\t\t      (uint64_t) delta_len);\n+\t\t\tgoto fail;\n+\t\t}\n \t}\n+\tstrbuf_release(&preimage_view.buf);\n \treturn 0;\n+ fail:\n+\tstrbuf_release(&preimage_view.buf);\n+\treturn -1;\n }\n-- \n1.7.2.3\n"},{"id":"153384","messageId":"20101013093559.GE32608@burratino","threadId":"24407","inReplyTo":"20101013091714.GA32608@burratino","subject":"[PATCH 04/11] vcs-svn: Read inline data from deltas","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-13T09:35:59Z","receivedAt":"2010-10-13T09:35:59Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Each window of an svndiff0-format delta includes a section for new\ndata that will be copied into the preimage (in the order it appears in\nthe window, possibly interspersed with other data).\n\nRead this data when encountering it.  It is not actually necessary to\ndo so --- it would be just as easy to copy straight from the delta\nto output when interpreting the relevant instructions --- but this\nway, the code that interprets svndiff0 instructions can proceed more\nquickly because it does not require any I/O.\n\nSubversion's implementation rejects deltas that do not consume all\nthe auxiliary data that is available.  Do not check that for now,\nbecause it would make it impossible to test the function of this\npatch until the instructions to consume data are implemented.\n\nDo check for truncated data sections.  Since Subversion's applier\nrejects deltas that end before the new-data section is declared to\nend, it should be safe for this applier to reject such deltas, too.\n\nHelped-by: Ramkumar Ramachandra <artagnon@gmail.com>\nHelped-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n t/t9011-svn-da.sh |   12 ++++++++++++\n vcs-svn/svndiff.c |   27 ++++++++++++++++++++++++---\n 2 files changed, 36 insertions(+), 3 deletions(-)\n\ndiff --git a/t/t9011-svn-da.sh b/t/t9011-svn-da.sh\nindex b9aad70..44832b0 100755\n--- a/t/t9011-svn-da.sh\n+++ b/t/t9011-svn-da.sh\n@@ -117,4 +117,16 @@ test_expect_success 'preimage view: accept truncated preimage' '\n \ttest_cmp empty actual.longread\n '\n \n+test_expect_success 'inline data' '\n+\tprintf \"SVNQ%b%s%b%s\" \"QQQQ\\003\" \"bar\" \"QQQQ\\001\" \"x\" |\n+\t\tq_to_nul >inline.clear &&\n+\ttest-svn-fe -d preimage inline.clear 18 >actual &&\n+\ttest_cmp empty actual\n+'\n+\n+test_expect_success 'truncated inline data' '\n+\tprintf \"SVNQ%b%s\" \"QQQQ\\003\" \"b\" | q_to_nul >inline.trunc &&\n+\ttest_must_fail test-svn-fe -d preimage inline.trunc 10\n+'\n+\n test_done\ndiff --git a/vcs-svn/svndiff.c b/vcs-svn/svndiff.c\nindex f2876b3..c60d732 100644\n--- a/vcs-svn/svndiff.c\n+++ b/vcs-svn/svndiff.c\n@@ -23,6 +23,10 @@\n #define VLI_DIGIT_MASK\t0x7f\n #define VLI_BITS_PER_DIGIT 7\n \n+struct window {\n+\tstruct strbuf data;\n+};\n+\n static int read_magic(struct line_buffer *in, off_t *len)\n {\n \tstatic const char magic[] = {'S', 'V', 'N', '\\0'};\n@@ -101,11 +105,25 @@ static int read_length(struct line_buffer *in, size_t *result, off_t *len)\n \treturn 0;\n }\n \n+static int read_chunk(struct line_buffer *delta, off_t *delta_len,\n+\t\t      struct strbuf *buf, size_t len)\n+{\n+\tif (len > maximum_signed_value_of_type(off_t) ||\n+\t    (off_t) len > *delta_len)\n+\t\treturn -1;\n+\tstrbuf_reset(buf);\n+\tbuffer_read_binary(buf, len, delta);\n+\t*delta_len -= buf->len;\n+\treturn 0;\n+}\n+\n static int apply_one_window(struct line_buffer *delta, off_t *delta_len)\n {\n+\tstruct window ctx = {STRBUF_INIT};\n \tsize_t out_len;\n \tsize_t instructions_len;\n \tsize_t data_len;\n+\tint rv = 0;\n \tassert(delta_len);\n \n \t/* \"source view\" offset and length already handled; */\n@@ -115,9 +133,12 @@ static int apply_one_window(struct line_buffer *delta, off_t *delta_len)\n \t\treturn -1;\n \tif (instructions_len > 0)\n \t\treturn error(\"What do you think I am?  A delta applier?\");\n-\tif (data_len > 0)\n-\t\treturn error(\"No support for inline data yet\");\n-\treturn 0;\n+\tif (read_chunk(delta, delta_len, &ctx.data, data_len))\n+\t\treturn error(\"Invalid delta: incomplete data section\");\n+\tif (buffer_ferror(delta))\n+\t\trv = error(\"Cannot read delta: %s\", strerror(errno));\n+\tstrbuf_release(&ctx.data);\n+\treturn rv;\n }\n \n int svndiff0_apply(struct line_buffer *delta, off_t delta_len,\n-- \n1.7.2.3\n"},{"id":"153385","messageId":"20101013093801.GF32608@burratino","threadId":"24407","inReplyTo":"20101013091714.GA32608@burratino","subject":"[PATCH 05/11] vcs-svn: Read instructions from deltas","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-13T09:38:01Z","receivedAt":"2010-10-13T09:38:01Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Buffer the instruction section upon encountering it for later\ninterpretation.\n\nAn alternative design would involve parsing the instructions\nat this point and buffering them in some processed form.  Using\nthe unprocessed form is simpler.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n t/t9011-svn-da.sh |    5 +++++\n vcs-svn/svndiff.c |   23 ++++++++++++++++++-----\n 2 files changed, 23 insertions(+), 5 deletions(-)\n\ndiff --git a/t/t9011-svn-da.sh b/t/t9011-svn-da.sh\nindex 44832b0..1383263 100755\n--- a/t/t9011-svn-da.sh\n+++ b/t/t9011-svn-da.sh\n@@ -129,4 +129,9 @@ test_expect_success 'truncated inline data' '\n \ttest_must_fail test-svn-fe -d preimage inline.trunc 10\n '\n \n+test_expect_success 'truncated inline data (after instruction section)' '\n+\tprintf \"SVNQ%b%b%s\" \"QQ\\001\\001\\003\" \"\\0201\" \"b\" | q_to_nul >insn.trunc &&\n+\ttest_must_fail test-svn-fe -d preimage insn.trunc 11\n+'\n+\n test_done\ndiff --git a/vcs-svn/svndiff.c b/vcs-svn/svndiff.c\nindex c60d732..72fe716 100644\n--- a/vcs-svn/svndiff.c\n+++ b/vcs-svn/svndiff.c\n@@ -24,6 +24,7 @@\n #define VLI_BITS_PER_DIGIT 7\n \n struct window {\n+\tstruct strbuf instructions;\n \tstruct strbuf data;\n };\n \n@@ -119,7 +120,7 @@ static int read_chunk(struct line_buffer *delta, off_t *delta_len,\n \n static int apply_one_window(struct line_buffer *delta, off_t *delta_len)\n {\n-\tstruct window ctx = {STRBUF_INIT};\n+\tstruct window ctx = {STRBUF_INIT, STRBUF_INIT};\n \tsize_t out_len;\n \tsize_t instructions_len;\n \tsize_t data_len;\n@@ -131,13 +132,25 @@ static int apply_one_window(struct line_buffer *delta, off_t *delta_len)\n \t    read_length(delta, &instructions_len, delta_len) ||\n \t    read_length(delta, &data_len, delta_len))\n \t\treturn -1;\n+\tif (read_chunk(delta, delta_len, &ctx.instructions, instructions_len))\n+\t\treturn error(\"Invalid delta: incomplete instructions section\");\n+\tif (buffer_ferror(delta)) {\n+\t\trv = error(\"Cannot read delta: %s\", strerror(errno));\n+\t\tgoto done;\n+\t}\n+\tif (read_chunk(delta, delta_len, &ctx.data, data_len)) {\n+\t\trv = error(\"Invalid delta: incomplete data section\");\n+\t\tgoto done;\n+\t}\n+\tif (buffer_ferror(delta)) {\n+\t\trv = error(\"Cannot read delta: %s\", strerror(errno));\n+\t\tgoto done;\n+\t}\n \tif (instructions_len > 0)\n \t\treturn error(\"What do you think I am?  A delta applier?\");\n-\tif (read_chunk(delta, delta_len, &ctx.data, data_len))\n-\t\treturn error(\"Invalid delta: incomplete data section\");\n-\tif (buffer_ferror(delta))\n-\t\trv = error(\"Cannot read delta: %s\", strerror(errno));\n+ done:\n \tstrbuf_release(&ctx.data);\n+\tstrbuf_release(&ctx.instructions);\n \treturn rv;\n }\n \n-- \n1.7.2.3\n"},{"id":"153386","messageId":"20101013093944.GG32608@burratino","threadId":"24407","inReplyTo":"20101013091714.GA32608@burratino","subject":"[PATCH 06/11] vcs-svn: Implement copyfrom_data delta instruction","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-13T09:39:44Z","receivedAt":"2010-10-13T09:39:44Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"The copyfrom_data instruction copies a few bytes verbatim from the\nauxiliary data section of a window to the postimage.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n t/t9011-svn-da.sh |   31 +++++++++++++++++++\n vcs-svn/svndiff.c |   86 +++++++++++++++++++++++++++++++++++++++++++++++++---\n 2 files changed, 112 insertions(+), 5 deletions(-)\n\ndiff --git a/t/t9011-svn-da.sh b/t/t9011-svn-da.sh\nindex 1383263..9279924 100755\n--- a/t/t9011-svn-da.sh\n+++ b/t/t9011-svn-da.sh\n@@ -134,4 +134,35 @@ test_expect_success 'truncated inline data (after instruction section)' '\n \ttest_must_fail test-svn-fe -d preimage insn.trunc 11\n '\n \n+test_expect_success 'copyfrom_data' '\n+\techo hi >expect &&\n+\tprintf \"SVNQ%b%b%b\" \"QQ\\003\\001\\003\" \"\\0203\" \"hi\\n\" | q_to_nul >copydat &&\n+\ttest-svn-fe -d preimage copydat 13 >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'multiple copyfrom_data' '\n+\techo hi >expect &&\n+\tprintf \"SVNQ%b%b%b%b%b\" \"QQ\\003\\002\\003\" \"\\0201\\0202\" \"hi\\n\" \\\n+\t\t\"QQQ\\002Q\" \"\\0200Q\" | q_to_nul >copy.multi &&\n+\tlen=$(wc -c <copy.multi) &&\n+\ttest-svn-fe -d preimage copy.multi $len >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'incomplete multiple insn' '\n+\tprintf \"SVNQ%b%b%b\" \"QQ\\003\\002\\003\" \"\\0203\\0200\" \"hi\\n\" |\n+\t\tq_to_nul >copy.partial &&\n+\tlen=$(wc -c <copy.partial) &&\n+\ttest_must_fail test-svn-fe -d preimage copy.partial $len\n+'\n+\n+test_expect_success 'catch attempt to copy missing data' '\n+\tprintf \"SVNQ%b%b%s%b%s\" \"QQ\\002\\002\\001\" \"\\0201\\0201\" \"X\" \\\n+\t\t\t\"QQQQ\\002\" \"YZ\" |\n+\t\tq_to_nul >copy.incomplete &&\n+\tlen=$(wc -c <copy.incomplete) &&\n+\ttest_must_fail test-svn-fe -d preimage copy.incomplete $len\n+'\n+\n test_done\ndiff --git a/vcs-svn/svndiff.c b/vcs-svn/svndiff.c\nindex 72fe716..ac776e0 100644\n--- a/vcs-svn/svndiff.c\n+++ b/vcs-svn/svndiff.c\n@@ -14,20 +14,40 @@\n  *\n  * svndiff0 ::= 'SVN\\0' window window*;\n  * window ::= int int int int int instructions inline_data;\n+ * instructions ::= instruction*;\n+ * instruction ::= view_selector int int\n+ *   | copyfrom_data int\n+ *   | packed_view_selector int\n+ *   | packed_copyfrom_data\n+ *   ;\n+ * copyfrom_data ::= # binary 10 000000;\n+ * packed_copyfrom_data ::= # copyfrom_data OR-ed with 6 bit value;\n  * int ::= highdigit* lowdigit;\n  * highdigit ::= # binary 1000 0000 OR-ed with 7 bit value;\n  * lowdigit ::= # 7 bit value;\n  */\n \n+#define INSN_MASK\t0xc0\n+#define INSN_COPYFROM_DATA\t0x80\n+#define OPERAND_MASK\t0x3f\n+\n #define VLI_CONTINUE\t0x80\n #define VLI_DIGIT_MASK\t0x7f\n #define VLI_BITS_PER_DIGIT 7\n \n struct window {\n+\tstruct strbuf out;\n \tstruct strbuf instructions;\n \tstruct strbuf data;\n };\n \n+static int write_strbuf(struct strbuf *sb, FILE *out)\n+{\n+\tif (fwrite(sb->buf, 1, sb->len, out) == sb->len)\t/* Success. */\n+\t\treturn 0;\n+\treturn error(\"Cannot write: %s\\n\", strerror(errno));\n+}\n+\n static int read_magic(struct line_buffer *in, off_t *len)\n {\n \tstatic const char magic[] = {'S', 'V', 'N', '\\0'};\n@@ -118,9 +138,63 @@ static int read_chunk(struct line_buffer *delta, off_t *delta_len,\n \treturn 0;\n }\n \n-static int apply_one_window(struct line_buffer *delta, off_t *delta_len)\n+static int copyfrom_data(struct window *ctx, size_t *data_pos, size_t nbytes)\n {\n-\tstruct window ctx = {STRBUF_INIT, STRBUF_INIT};\n+\tconst size_t pos = *data_pos;\n+\tif (unsigned_add_overflows(pos, nbytes) ||\n+\t    pos + nbytes > ctx->data.len)\n+\t\treturn error(\"Invalid delta: copies unavailable inline data.\");\n+\tstrbuf_add(&ctx->out, ctx->data.buf + pos, nbytes);\n+\t*data_pos += nbytes;\n+\treturn 0;\n+}\n+\n+static int parse_first_operand(const char **buf, size_t *out, const char *end)\n+{\n+\tsize_t result = (unsigned char) *(*buf)++ & OPERAND_MASK;\n+\tif (result) {\n+\t\t*out = result;\n+\t\treturn 0;\n+\t}\n+\treturn parse_int(buf, out, end);\n+}\n+\n+static int step(struct window *ctx, const char **instructions, size_t *data_pos)\n+{\n+\tunsigned int instruction;\n+\tconst char *insns_end = ctx->instructions.buf + ctx->instructions.len;\n+\tsize_t nbytes;\n+\tassert(ctx);\n+\tassert(instructions && *instructions);\n+\tassert(data_pos);\n+\n+\tinstruction = (unsigned char) **instructions;\n+\tif (parse_first_operand(instructions, &nbytes, insns_end))\n+\t\treturn -1;\n+\tif ((instruction & INSN_MASK) != INSN_COPYFROM_DATA)\n+\t\treturn error(\"Unknown instruction %x\", instruction);\n+\treturn copyfrom_data(ctx, data_pos, nbytes);\n+}\n+\n+static int apply_window_in_core(struct window *ctx)\n+{\n+\tconst char *insn = ctx->instructions.buf;\n+\tsize_t data_pos = 0;\n+\n+\t/*\n+\t * Populate ctx->out.buf using data from the source, target,\n+\t * and inline data views.\n+\t */\n+\twhile (insn != ctx->instructions.buf + ctx->instructions.len)\n+\t\tif (step(ctx, &insn, &data_pos))\n+\t\t\treturn -1;\n+\treturn 0;\n+}\n+\n+static int apply_one_window(struct line_buffer *delta, off_t *delta_len,\n+\t\t\t    FILE *out)\n+{\n+\tstruct window ctx = {STRBUF_INIT, STRBUF_INIT, STRBUF_INIT};\n \tsize_t out_len;\n \tsize_t instructions_len;\n \tsize_t data_len;\n@@ -146,8 +220,10 @@ static int apply_one_window(struct line_buffer *delta, off_t *delta_len)\n \t\trv = error(\"Cannot read delta: %s\", strerror(errno));\n \t\tgoto done;\n \t}\n-\tif (instructions_len > 0)\n-\t\treturn error(\"What do you think I am?  A delta applier?\");\n+\tif (apply_window_in_core(&ctx) || write_strbuf(&ctx.out, out)) {\n+\t\trv = -1;\n+\t\tgoto done;\n+\t}\n  done:\n \tstrbuf_release(&ctx.data);\n \tstrbuf_release(&ctx.instructions);\n@@ -168,7 +244,7 @@ int svndiff0_apply(struct line_buffer *delta, off_t delta_len,\n \t\tif (read_offset(delta, &pre_off, &delta_len) ||\n \t\t    read_length(delta, &pre_len, &delta_len) ||\n \t\t    move_window(&preimage_view, pre_off, pre_len) ||\n-\t\t    apply_one_window(delta, &delta_len))\n+\t\t    apply_one_window(delta, &delta_len, postimage))\n \t\t\tgoto fail;\n \t\tif (delta_len && buffer_at_eof(delta)) {\n \t\t\terror(\"Delta ends early! (%\"PRIu64\" bytes remaining)\",\n-- \n1.7.2.3\n"},{"id":"153387","messageId":"20101013094115.GH32608@burratino","threadId":"24407","inReplyTo":"20101013091714.GA32608@burratino","subject":"[PATCH 07/11] vcs-svn: Check declared number of output bytes","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-13T09:41:15Z","receivedAt":"2010-10-13T09:41:15Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Check that the declared output size for each window is correct, and\nreserve that amount of space in the output buffer in advance.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n vcs-svn/svndiff.c |    5 +++++\n 1 files changed, 5 insertions(+), 0 deletions(-)\n\ndiff --git a/vcs-svn/svndiff.c b/vcs-svn/svndiff.c\nindex ac776e0..c03cd7e 100644\n--- a/vcs-svn/svndiff.c\n+++ b/vcs-svn/svndiff.c\n@@ -220,10 +220,15 @@ static int apply_one_window(struct line_buffer *delta, off_t *delta_len,\n \t\trv = error(\"Cannot read delta: %s\", strerror(errno));\n \t\tgoto done;\n \t}\n+\tstrbuf_grow(&ctx.out, out_len);\n \tif (apply_window_in_core(&ctx) || write_strbuf(&ctx.out, out)) {\n \t\trv = -1;\n \t\tgoto done;\n \t}\n+\tif (ctx.out.len != out_len) {\n+\t\trv = error(\"Invalid delta: incorrect postimage length\");\n+\t\tgoto done;\n+\t}\n  done:\n \tstrbuf_release(&ctx.data);\n \tstrbuf_release(&ctx.instructions);\n-- \n1.7.2.3\n"},{"id":"153388","messageId":"20101013094807.GI32608@burratino","threadId":"24407","inReplyTo":"20101013091714.GA32608@burratino","subject":"[PATCH 08/11] vcs-svn: Reject deltas that do not consume all inline data","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-13T09:48:07Z","receivedAt":"2010-10-13T09:48:07Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"The main point is to constrain the format of deltas more,\nso corruption and other breakage can be more easily detected.\n\nRequiring deltas not to provide unconsumed data also opens\nthe possibility of ignoring the declared amount of new data\nand simply streaming the data as needed to fulfill\ncopyfrom_data requests.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n t/t9011-svn-da.sh |   12 +++---------\n vcs-svn/svndiff.c |    2 ++\n 2 files changed, 5 insertions(+), 9 deletions(-)\n\ndiff --git a/t/t9011-svn-da.sh b/t/t9011-svn-da.sh\nindex 9279924..c9f4768 100755\n--- a/t/t9011-svn-da.sh\n+++ b/t/t9011-svn-da.sh\n@@ -117,20 +117,14 @@ test_expect_success 'preimage view: accept truncated preimage' '\n \ttest_cmp empty actual.longread\n '\n \n-test_expect_success 'inline data' '\n+test_expect_success 'unconsumed inline data' '\n \tprintf \"SVNQ%b%s%b%s\" \"QQQQ\\003\" \"bar\" \"QQQQ\\001\" \"x\" |\n \t\tq_to_nul >inline.clear &&\n-\ttest-svn-fe -d preimage inline.clear 18 >actual &&\n-\ttest_cmp empty actual\n+\ttest_must_fail test-svn-fe -d preimage inline.clear 18 >actual\n '\n \n test_expect_success 'truncated inline data' '\n-\tprintf \"SVNQ%b%s\" \"QQQQ\\003\" \"b\" | q_to_nul >inline.trunc &&\n-\ttest_must_fail test-svn-fe -d preimage inline.trunc 10\n-'\n-\n-test_expect_success 'truncated inline data (after instruction section)' '\n-\tprintf \"SVNQ%b%b%s\" \"QQ\\001\\001\\003\" \"\\0201\" \"b\" | q_to_nul >insn.trunc &&\n+\tprintf \"SVNQ%b%b%s\" \"QQ\\003\\001\\003\" \"\\0203\" \"b\" | q_to_nul >insn.trunc &&\n \ttest_must_fail test-svn-fe -d preimage insn.trunc 11\n '\n \ndiff --git a/vcs-svn/svndiff.c b/vcs-svn/svndiff.c\nindex c03cd7e..8755c83 100644\n--- a/vcs-svn/svndiff.c\n+++ b/vcs-svn/svndiff.c\n@@ -188,6 +188,8 @@ static int apply_window_in_core(struct window *ctx)\n \twhile (insn != ctx->instructions.buf + ctx->instructions.len)\n \t\tif (step(ctx, &insn, &data_pos))\n \t\t\treturn -1;\n+\tif (data_pos != ctx->data.len)\n+\t\treturn error(\"Invalid delta: does not copy all new data\");\n \treturn 0;\n }\n \n-- \n1.7.2.3\n"},{"id":"153389","messageId":"20101013095007.GJ32608@burratino","threadId":"24407","inReplyTo":"20101013091714.GA32608@burratino","subject":"[PATCH 09/11] vcs-svn: Let deltas use data from postimage","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-13T09:50:07Z","receivedAt":"2010-10-13T09:50:07Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"The copyfrom_target instruction copies appends data that is already\npresent in the current output view to the end of output.  (The offset\nargument is relative to the beginning of output produced by the\ncurrent window.)\n\nThe region copied is allowed to run past the end of the existing\noutput.  To support that case, copy one character at a time\nrather than using memcpy() or memmove().  This allows copyfrom_target\nto be used once to repeat a string many times.  For example:\n\n\tCOPYFROM_DATA 2\n\tCOPYFROM_OUTPUT 10, 0\n\tDATA \"ab\"\n\nwould produce the output \"ababababababababababab\".\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n t/t9011-svn-da.sh |   42 ++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/svndiff.c |   30 ++++++++++++++++++++++++++++--\n 2 files changed, 70 insertions(+), 2 deletions(-)\n\ndiff --git a/t/t9011-svn-da.sh b/t/t9011-svn-da.sh\nindex c9f4768..ccd31e9 100755\n--- a/t/t9011-svn-da.sh\n+++ b/t/t9011-svn-da.sh\n@@ -159,4 +159,46 @@ test_expect_success 'catch attempt to copy missing data' '\n \ttest_must_fail test-svn-fe -d preimage copy.incomplete $len\n '\n \n+test_expect_success 'copyfrom target to repeat data' '\n+\tprintf foofoo >expect &&\n+\tprintf \"SVNQ%b%b%s\" \"QQ\\006\\004\\003\" \"\\0203\\0100\\003Q\" \"foo\" |\n+\t\tq_to_nul >copytarget.repeat &&\n+\tlen=$(wc -c <copytarget.repeat) &&\n+\ttest-svn-fe -d preimage copytarget.repeat $len >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'copyfrom target out of order' '\n+\tprintf foooof >expect &&\n+\tprintf \"SVNQ%b%b%s\" \\\n+\t\t\"QQ\\006\\007\\003\" \"\\0203\\0101\\002\\0101\\001\\0101Q\" \"foo\" |\n+\t\tq_to_nul >copytarget.reverse &&\n+\tlen=$(wc -c <copytarget.reverse) &&\n+\ttest-svn-fe -d preimage copytarget.reverse $len >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'catch copyfrom future' '\n+\tprintf \"SVNQ%b%b%s\" \"QQ\\004\\004\\003\" \"\\0202\\0101\\002\\0201\" \"XYZ\" |\n+\t\tq_to_nul >copytarget.infuture &&\n+\tlen=$(wc -c <copytarget.infuture) &&\n+\ttest_must_fail test-svn-fe -d preimage copytarget.infuture $len\n+'\n+\n+test_expect_success 'copy to sustain' '\n+\tprintf XYXYXYXYXYXZ >expect &&\n+\tprintf \"SVNQ%b%b%s\" \"QQ\\014\\004\\003\" \"\\0202\\0111Q\\0201\" \"XYZ\" |\n+\t\tq_to_nul >copytarget.sustain &&\n+\tlen=$(wc -c <copytarget.sustain) &&\n+\ttest-svn-fe -d preimage copytarget.sustain $len >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'catch copy that overflows' '\n+\tprintf \"SVNQ%b%b%s\" \"QQ\\003\\003\\001\" \"\\0201\\0177Q\" X |\n+\t\tq_to_nul >copytarget.overflow &&\n+\tlen=$(wc -c <copytarget.overflow) &&\n+\ttest_must_fail test-svn-fe -d preimage copytarget.overflow $len\n+'\n+\n test_done\ndiff --git a/vcs-svn/svndiff.c b/vcs-svn/svndiff.c\nindex 8755c83..8f1b61e 100644\n--- a/vcs-svn/svndiff.c\n+++ b/vcs-svn/svndiff.c\n@@ -20,7 +20,12 @@\n  *   | packed_view_selector int\n  *   | packed_copyfrom_data\n  *   ;\n+ * view_selector ::= copyfrom_source\n+ *   | copyfrom_target\n+ *   ;\n+ * copyfrom_target ::= # binary 01 000000;\n  * copyfrom_data ::= # binary 10 000000;\n+ * packed_view_selector ::= # view_selector OR-ed with 6 bit value;\n  * packed_copyfrom_data ::= # copyfrom_data OR-ed with 6 bit value;\n  * int ::= highdigit* lowdigit;\n  * highdigit ::= # binary 1000 0000 OR-ed with 7 bit value;\n@@ -28,6 +33,7 @@\n  */\n \n #define INSN_MASK\t0xc0\n+#define INSN_COPYFROM_TARGET\t0x40\n #define INSN_COPYFROM_DATA\t0x80\n #define OPERAND_MASK\t0x3f\n \n@@ -138,6 +144,21 @@ static int read_chunk(struct line_buffer *delta, off_t *delta_len,\n \treturn 0;\n }\n \n+static int copyfrom_target(struct window *ctx, const char **instructions,\n+\t\t\t   size_t nbytes, const char *insns_end)\n+{\n+\tsize_t offset;\n+\tif (parse_int(instructions, &offset, insns_end))\n+\t\treturn -1;\n+\tif (offset >= ctx->out.len)\n+\t\treturn error(\"Invalid delta: copies from the future.\");\n+\twhile (nbytes) {\n+\t\tstrbuf_addch(&ctx->out, ctx->out.buf[offset++]);\n+\t\tnbytes--;\n+\t}\n+\treturn 0;\n+}\n+\n static int copyfrom_data(struct window *ctx, size_t *data_pos, size_t nbytes)\n {\n \tconst size_t pos = *data_pos;\n@@ -171,9 +192,14 @@ static int step(struct window *ctx, const char **instructions, size_t *data_pos)\n \tinstruction = (unsigned char) **instructions;\n \tif (parse_first_operand(instructions, &nbytes, insns_end))\n \t\treturn -1;\n-\tif ((instruction & INSN_MASK) != INSN_COPYFROM_DATA)\n+\tswitch (instruction & INSN_MASK) {\n+\tcase INSN_COPYFROM_TARGET:\n+\t\treturn copyfrom_target(ctx, instructions, nbytes, insns_end);\n+\tcase INSN_COPYFROM_DATA:\n+\t\treturn copyfrom_data(ctx, data_pos, nbytes);\n+\tdefault:\n \t\treturn error(\"Unknown instruction %x\", instruction);\n-\treturn copyfrom_data(ctx, data_pos, nbytes);\n+\t}\n }\n \n static int apply_window_in_core(struct window *ctx)\n-- \n1.7.2.3\n"},{"id":"153390","messageId":"20101013095318.GK32608@burratino","threadId":"24407","inReplyTo":"20101013091714.GA32608@burratino","subject":"[PATCH 10/11] vcs-svn: Reject deltas that read past end of preimage","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-13T09:53:18Z","receivedAt":"2010-10-13T09:53:18Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Some particularly strange deltas of unknown origin were found to\nrequest copies beyond the end of the preimage.  But svn 1.6 never\nproduces anything like that.\n\nAlthough Subversion accepts these perverse deltas as input, let's\nerror out if some future version of subversion starts to actually\nproduce them.\n\nWithout this change, the diff applier would have to separately\nkeep track of the number of bytes supposedly and actually written out.\n\nHelped-by: Ramkumar Ramachandra <artagnon@gmail.com>\nHelped-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n t/t9011-svn-da.sh        |   11 ++++-------\n vcs-svn/sliding_window.c |   10 ++++++----\n 2 files changed, 10 insertions(+), 11 deletions(-)\n\ndiff --git a/t/t9011-svn-da.sh b/t/t9011-svn-da.sh\nindex ccd31e9..c4bd1f3 100755\n--- a/t/t9011-svn-da.sh\n+++ b/t/t9011-svn-da.sh\n@@ -105,16 +105,13 @@ test_expect_success 'preimage view: offsets compared by value' '\n \ttest_cmp empty actual\n '\n \n-test_expect_success 'preimage view: accept truncated preimage' '\n+test_expect_success 'preimage view: reject truncated preimage' '\n \tprintf \"SVNQ%b\" \"\\010QQQQ\" | q_to_nul >clear.lateemptyread &&\n \tprintf \"SVNQ%b\" \"\\010\\001QQQ\" | q_to_nul >clear.latenonemptyread &&\n \tprintf \"SVNQ%b\" \"\\001\\010QQQ\" | q_to_nul >clear.longread &&\n-\ttest-svn-fe -d preimage clear.lateemptyread 9 >actual.emptyread &&\n-\ttest-svn-fe -d preimage clear.latenonemptyread 9 >actual.nonemptyread &&\n-\ttest-svn-fe -d preimage clear.longread 9 >actual.longread &&\n-\ttest_cmp empty actual.emptyread &&\n-\ttest_cmp empty actual.nonemptyread &&\n-\ttest_cmp empty actual.longread\n+\ttest_must_fail test-svn-fe -d preimage clear.lateemptyread 9 &&\n+\ttest_must_fail test-svn-fe -d preimage clear.latenonemptyread 9 &&\n+\ttest_must_fail test-svn-fe -d preimage clear.longread 9\n '\n \n test_expect_success 'unconsumed inline data' '\ndiff --git a/vcs-svn/sliding_window.c b/vcs-svn/sliding_window.c\nindex 8273970..5c08828 100644\n--- a/vcs-svn/sliding_window.c\n+++ b/vcs-svn/sliding_window.c\n@@ -49,17 +49,19 @@ int move_window(struct view *view, off_t off, size_t len)\n \t\tconst off_t gap = off - file_offset;\n \t\tconst off_t nread = buffer_skip_bytes(view->file, gap);\n \t\tif (nread != gap) {\n-\t\t\tif (!buffer_ferror(view->file))\t/* View ends early. */\n-\t\t\t\tgoto done;\n+\t\t\tif (!buffer_ferror(view->file))\n+\t\t\t\treturn error(\"Preimage ends early\");\n \t\t\treturn error(\"Cannot seek forward in input: %s\",\n \t\t\t\t     strerror(errno));\n \t\t}\n \t\tfile_offset += gap;\n \t}\n \tbuffer_read_binary(&view->buf, len - view->buf.len, view->file);\n-\tif (buffer_ferror(view->file))\n+\tif (view->buf.len != len) {\n+\t\tif (!buffer_ferror(view->file))\n+\t\t\treturn error(\"Preimage ends early\");\n \t\treturn error(\"Cannot read preimage: %s\", strerror(errno));\n- done:\n+\t}\n \tview->off = off;\n \treturn 0;\n }\n-- \n1.7.2.3\n"},{"id":"153391","messageId":"20101013095829.GL32608@burratino","threadId":"24407","inReplyTo":"20101013091714.GA32608@burratino","subject":"[PATCH 11/11] vcs-svn: Allow deltas to copy from preimage","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-13T09:58:30Z","receivedAt":"2010-10-13T09:58:30Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"The copyfrom_source instruction appends data from the preimage\nbuffer to the end of output.  Its arguments are a length and an\noffset relative to the beginning of the source view.\n\nHelped-by: Ramkumar Ramachandra <artagnon@gmail.com>\nHelped-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\nThat's the end of the series.  Thanks for reading.  Hopefully this\nround did not introduce too many bugs but if it did, I'd be glad to\nhear about them.\n\nGood night,\nJonathan\n\n t/t9011-svn-da.sh |   35 +++++++++++++++++++++++++++++++++++\n vcs-svn/svndiff.c |   27 +++++++++++++++++++++++----\n 2 files changed, 58 insertions(+), 4 deletions(-)\n\ndiff --git a/t/t9011-svn-da.sh b/t/t9011-svn-da.sh\nindex c4bd1f3..c8959e2 100755\n--- a/t/t9011-svn-da.sh\n+++ b/t/t9011-svn-da.sh\n@@ -198,4 +198,39 @@ test_expect_success 'catch copy that overflows' '\n \ttest_must_fail test-svn-fe -d preimage copytarget.overflow $len\n '\n \n+test_expect_success 'copyfrom source' '\n+\tprintf foo >expect &&\n+\tprintf \"SVNQ%b%b\" \"Q\\003\\003\\002Q\" \"\\003Q\" | q_to_nul >copysource.all &&\n+\ttest-svn-fe -d preimage copysource.all 11 >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'copy backwards' '\n+\tprintf oof >expect &&\n+\tprintf \"SVNQ%b%b\" \"Q\\003\\003\\006Q\" \"\\001\\002\\001\\001\\001Q\" |\n+\t\tq_to_nul >copysource.rev &&\n+\ttest-svn-fe -d preimage copysource.rev 15 >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'offsets are relative to window' '\n+\tprintf fo >expect &&\n+\tprintf \"SVNQ%b%b%b%b\" \"Q\\003\\001\\002Q\" \"\\001Q\" \\\n+\t\t\"\\002\\001\\001\\002Q\" \"\\001Q\" |\n+\t\tq_to_nul >copysource.two &&\n+\ttest-svn-fe -d preimage copysource.two 18 >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'example from notes/svndiff' '\n+\tprintf aaaaccccdddddddd >expect &&\n+\tprintf aaaabbbbcccc >source &&\n+\tprintf \"SVNQ%b%b%s\" \"Q\\014\\020\\007\\001\" \\\n+\t\t\"\\004Q\\004\\010\\0201\\0107\\010\" d |\n+\t\tq_to_nul >delta.example &&\n+\tlen=$(wc -c <delta.example) &&\n+\ttest-svn-fe -d source delta.example $len >actual &&\n+\ttest_cmp expect actual\n+'\n+\n test_done\ndiff --git a/vcs-svn/svndiff.c b/vcs-svn/svndiff.c\nindex 8f1b61e..d3d1dba 100644\n--- a/vcs-svn/svndiff.c\n+++ b/vcs-svn/svndiff.c\n@@ -23,6 +23,7 @@\n  * view_selector ::= copyfrom_source\n  *   | copyfrom_target\n  *   ;\n+ * copyfrom_source ::= # binary 00 000000;\n  * copyfrom_target ::= # binary 01 000000;\n  * copyfrom_data ::= # binary 10 000000;\n  * packed_view_selector ::= # view_selector OR-ed with 6 bit value;\n@@ -33,6 +34,7 @@\n  */\n \n #define INSN_MASK\t0xc0\n+#define INSN_COPYFROM_SOURCE\t0x00\n #define INSN_COPYFROM_TARGET\t0x40\n #define INSN_COPYFROM_DATA\t0x80\n #define OPERAND_MASK\t0x3f\n@@ -42,6 +44,7 @@\n #define VLI_BITS_PER_DIGIT 7\n \n struct window {\n+\tstruct view *in;\n \tstruct strbuf out;\n \tstruct strbuf instructions;\n \tstruct strbuf data;\n@@ -144,6 +147,19 @@ static int read_chunk(struct line_buffer *delta, off_t *delta_len,\n \treturn 0;\n }\n \n+static int copyfrom_source(struct window *ctx, const char **instructions,\n+\t\t\t   size_t nbytes, const char *insns_end)\n+{\n+\tsize_t offset;\n+\tif (parse_int(instructions, &offset, insns_end))\n+\t\treturn -1;\n+\tif (unsigned_add_overflows(offset, nbytes) ||\n+\t    offset + nbytes > ctx->in->buf.len)\n+\t\treturn error(\"Invalid delta: copies source data outside view.\");\n+\tstrbuf_add(&ctx->out, ctx->in->buf.buf + offset, nbytes);\n+\treturn 0;\n+}\n+\n static int copyfrom_target(struct window *ctx, const char **instructions,\n \t\t\t   size_t nbytes, const char *insns_end)\n {\n@@ -193,12 +209,14 @@ static int step(struct window *ctx, const char **instructions, size_t *data_pos)\n \tif (parse_first_operand(instructions, &nbytes, insns_end))\n \t\treturn -1;\n \tswitch (instruction & INSN_MASK) {\n+\tcase INSN_COPYFROM_SOURCE:\n+\t\treturn copyfrom_source(ctx, instructions, nbytes, insns_end);\n \tcase INSN_COPYFROM_TARGET:\n \t\treturn copyfrom_target(ctx, instructions, nbytes, insns_end);\n \tcase INSN_COPYFROM_DATA:\n \t\treturn copyfrom_data(ctx, data_pos, nbytes);\n \tdefault:\n-\t\treturn error(\"Unknown instruction %x\", instruction);\n+\t\treturn error(\"Invalid instruction %x\", instruction);\n \t}\n }\n \n@@ -220,9 +238,9 @@ static int apply_window_in_core(struct window *ctx)\n }\n \n static int apply_one_window(struct line_buffer *delta, off_t *delta_len,\n-\t\t\t    FILE *out)\n+\t\t\t    struct view *preimage, FILE *out)\n {\n-\tstruct window ctx = {STRBUF_INIT, STRBUF_INIT, STRBUF_INIT};\n+\tstruct window ctx = {preimage, STRBUF_INIT, STRBUF_INIT, STRBUF_INIT};\n \tsize_t out_len;\n \tsize_t instructions_len;\n \tsize_t data_len;\n@@ -277,7 +295,8 @@ int svndiff0_apply(struct line_buffer *delta, off_t delta_len,\n \t\tif (read_offset(delta, &pre_off, &delta_len) ||\n \t\t    read_length(delta, &pre_len, &delta_len) ||\n \t\t    move_window(&preimage_view, pre_off, pre_len) ||\n-\t\t    apply_one_window(delta, &delta_len, postimage))\n+\t\t    apply_one_window(delta, &delta_len,\n+\t\t\t\t     &preimage_view, postimage))\n \t\t\tgoto fail;\n \t\tif (delta_len && buffer_at_eof(delta)) {\n \t\t\terror(\"Delta ends early! (%\"PRIu64\" bytes remaining)\",\n-- \n1.7.2.3\n"},{"id":"153392","messageId":"20101013100002.GM32608@burratino","threadId":"24407","inReplyTo":"20101013091714.GA32608@burratino","subject":"[PATCH 11/11] vcs-svn: Allow deltas to copy from preimage","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-13T10:00:02Z","receivedAt":"2010-10-13T10:00:02Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"The copyfrom_source instruction appends data from the preimage\nbuffer to the end of output.  Its arguments are a length and an\noffset relative to the beginning of the source view.\n\nHelped-by: Ramkumar Ramachandra <artagnon@gmail.com>\nHelped-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n[resending with cc: samv]\n\nThat's the end of the series.  Thanks for reading.  Hopefully this\nround did not introduce too many bugs but if it did, I'd be glad to\nhear about them.\n\nGood night,\nJonathan\n\n t/t9011-svn-da.sh |   35 +++++++++++++++++++++++++++++++++++\n vcs-svn/svndiff.c |   27 +++++++++++++++++++++++----\n 2 files changed, 58 insertions(+), 4 deletions(-)\n\ndiff --git a/t/t9011-svn-da.sh b/t/t9011-svn-da.sh\nindex c4bd1f3..c8959e2 100755\n--- a/t/t9011-svn-da.sh\n+++ b/t/t9011-svn-da.sh\n@@ -198,4 +198,39 @@ test_expect_success 'catch copy that overflows' '\n \ttest_must_fail test-svn-fe -d preimage copytarget.overflow $len\n '\n \n+test_expect_success 'copyfrom source' '\n+\tprintf foo >expect &&\n+\tprintf \"SVNQ%b%b\" \"Q\\003\\003\\002Q\" \"\\003Q\" | q_to_nul >copysource.all &&\n+\ttest-svn-fe -d preimage copysource.all 11 >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'copy backwards' '\n+\tprintf oof >expect &&\n+\tprintf \"SVNQ%b%b\" \"Q\\003\\003\\006Q\" \"\\001\\002\\001\\001\\001Q\" |\n+\t\tq_to_nul >copysource.rev &&\n+\ttest-svn-fe -d preimage copysource.rev 15 >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'offsets are relative to window' '\n+\tprintf fo >expect &&\n+\tprintf \"SVNQ%b%b%b%b\" \"Q\\003\\001\\002Q\" \"\\001Q\" \\\n+\t\t\"\\002\\001\\001\\002Q\" \"\\001Q\" |\n+\t\tq_to_nul >copysource.two &&\n+\ttest-svn-fe -d preimage copysource.two 18 >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'example from notes/svndiff' '\n+\tprintf aaaaccccdddddddd >expect &&\n+\tprintf aaaabbbbcccc >source &&\n+\tprintf \"SVNQ%b%b%s\" \"Q\\014\\020\\007\\001\" \\\n+\t\t\"\\004Q\\004\\010\\0201\\0107\\010\" d |\n+\t\tq_to_nul >delta.example &&\n+\tlen=$(wc -c <delta.example) &&\n+\ttest-svn-fe -d source delta.example $len >actual &&\n+\ttest_cmp expect actual\n+'\n+\n test_done\ndiff --git a/vcs-svn/svndiff.c b/vcs-svn/svndiff.c\nindex 8f1b61e..d3d1dba 100644\n--- a/vcs-svn/svndiff.c\n+++ b/vcs-svn/svndiff.c\n@@ -23,6 +23,7 @@\n  * view_selector ::= copyfrom_source\n  *   | copyfrom_target\n  *   ;\n+ * copyfrom_source ::= # binary 00 000000;\n  * copyfrom_target ::= # binary 01 000000;\n  * copyfrom_data ::= # binary 10 000000;\n  * packed_view_selector ::= # view_selector OR-ed with 6 bit value;\n@@ -33,6 +34,7 @@\n  */\n \n #define INSN_MASK\t0xc0\n+#define INSN_COPYFROM_SOURCE\t0x00\n #define INSN_COPYFROM_TARGET\t0x40\n #define INSN_COPYFROM_DATA\t0x80\n #define OPERAND_MASK\t0x3f\n@@ -42,6 +44,7 @@\n #define VLI_BITS_PER_DIGIT 7\n \n struct window {\n+\tstruct view *in;\n \tstruct strbuf out;\n \tstruct strbuf instructions;\n \tstruct strbuf data;\n@@ -144,6 +147,19 @@ static int read_chunk(struct line_buffer *delta, off_t *delta_len,\n \treturn 0;\n }\n \n+static int copyfrom_source(struct window *ctx, const char **instructions,\n+\t\t\t   size_t nbytes, const char *insns_end)\n+{\n+\tsize_t offset;\n+\tif (parse_int(instructions, &offset, insns_end))\n+\t\treturn -1;\n+\tif (unsigned_add_overflows(offset, nbytes) ||\n+\t    offset + nbytes > ctx->in->buf.len)\n+\t\treturn error(\"Invalid delta: copies source data outside view.\");\n+\tstrbuf_add(&ctx->out, ctx->in->buf.buf + offset, nbytes);\n+\treturn 0;\n+}\n+\n static int copyfrom_target(struct window *ctx, const char **instructions,\n \t\t\t   size_t nbytes, const char *insns_end)\n {\n@@ -193,12 +209,14 @@ static int step(struct window *ctx, const char **instructions, size_t *data_pos)\n \tif (parse_first_operand(instructions, &nbytes, insns_end))\n \t\treturn -1;\n \tswitch (instruction & INSN_MASK) {\n+\tcase INSN_COPYFROM_SOURCE:\n+\t\treturn copyfrom_source(ctx, instructions, nbytes, insns_end);\n \tcase INSN_COPYFROM_TARGET:\n \t\treturn copyfrom_target(ctx, instructions, nbytes, insns_end);\n \tcase INSN_COPYFROM_DATA:\n \t\treturn copyfrom_data(ctx, data_pos, nbytes);\n \tdefault:\n-\t\treturn error(\"Unknown instruction %x\", instruction);\n+\t\treturn error(\"Invalid instruction %x\", instruction);\n \t}\n }\n \n@@ -220,9 +238,9 @@ static int apply_window_in_core(struct window *ctx)\n }\n \n static int apply_one_window(struct line_buffer *delta, off_t *delta_len,\n-\t\t\t    FILE *out)\n+\t\t\t    struct view *preimage, FILE *out)\n {\n-\tstruct window ctx = {STRBUF_INIT, STRBUF_INIT, STRBUF_INIT};\n+\tstruct window ctx = {preimage, STRBUF_INIT, STRBUF_INIT, STRBUF_INIT};\n \tsize_t out_len;\n \tsize_t instructions_len;\n \tsize_t data_len;\n@@ -277,7 +295,8 @@ int svndiff0_apply(struct line_buffer *delta, off_t delta_len,\n \t\tif (read_offset(delta, &pre_off, &delta_len) ||\n \t\t    read_length(delta, &pre_len, &delta_len) ||\n \t\t    move_window(&preimage_view, pre_off, pre_len) ||\n-\t\t    apply_one_window(delta, &delta_len, postimage))\n+\t\t    apply_one_window(delta, &delta_len,\n+\t\t\t\t     &preimage_view, postimage))\n \t\t\tgoto fail;\n \t\tif (delta_len && buffer_at_eof(delta)) {\n \t\t\terror(\"Delta ends early! (%\"PRIu64\" bytes remaining)\",\n-- \n1.7.2.3\n"},{"id":"153543","messageId":"1287092755.10477.9.camel@wilber","threadId":"24407","inReplyTo":"20101013093037.GD32608@burratino","subject":"Re: [PATCH 03/11] vcs-svn: Read the preimage while applying deltas","fromName":"Sam Vilain","fromEmail":"sam@vilain.net","sentAt":"2010-10-14T21:45:55Z","receivedAt":"2010-10-14T21:45:55Z","isPatch":true,"sender":{"key":"sam@vilain.net","avatar":"https://gravatar.com/avatar/8fc840ca854dbf6f7065b4335e3b934951c1dca3b11db688e95e471901f8f4a8?d=mp&s=160"},"body":"On Wed, 2010-10-13 at 04:30 -0500, Jonathan Nieder wrote:\n> It occurs to me that Sam Vilain may well have something valuable to\n> say about this series, having implemented something similar[1].\n> \n> Sam, this series adds an svndiff0 parser for git to use in parsing\n> v3 dumps (which are way easier to produce with remote access to an\n> svn repository than v2 dumps).  The beginning of the series is at [2],\n> though that cover letter is out of date: now, modulo any new bugs\n> I've introduced with this reroll, it is known to successfully apply\n> all the deltas involved in a complete dump of the ASF repo.\n\n> [1] http://search.cpan.org/~samv/Parse-SVNDiff-0.03/lib/Parse/SVNDiff.pm\n\nAll I did was make the module lazy - version 0.02 was by 唐鳳, any\ndetailed knowledge I had of the binary format fell out of my head pretty\nquickly :-)\n\nSam\n"},{"id":"153551","messageId":"20101014234008.GC29494@burratino","threadId":"24407","inReplyTo":"1287092755.10477.9.camel@wilber","subject":"Re: [PATCH 03/11] vcs-svn: Read the preimage while applying deltas","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-14T23:40:08Z","receivedAt":"2010-10-14T23:40:08Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Sam Vilain wrote:\n\n> All I did was make the module lazy - version 0.02 was by 唐鳳, any\n> detailed knowledge I had of the binary format fell out of my head pretty\n> quickly :-)\n\nAh, my bad.  唐鳳, as part of an attempt to natively support fetching\nfrom and pushing to svn repositories, some contributors to the git\nproject are working on an svndiff0 applier.  If you're interested in\nreliving old memories, please feel free to look it over (especially\nthe test cases).  Thoughts, simplifications, bug reports, improvements\nwelcome.\n\nhttp://thread.gmane.org/gmane.comp.version-control.git/151086/focus=158913\n\nAnyone wanting to try it can check out\n\n\tgit://repo.or.cz/git/jrn.git svn-da\n\nand use the test-svn-fe command:\n\n\tmake test-svn-fe\n\t./test-svn-fe -d <preimage> <delta> <delta length>\n\nor the tests:\n\n\tmake\n\tcd t && sh t9011-svn-da.sh -v -i\n\nThe preimage or delta argument can be /dev/stdin for use in a pipeline.\n\nThanks for your work on svk and pugs!\nJonathan\n"},{"id":"153726","messageId":"20101018170018.GJ22376@kytes","threadId":"24407","inReplyTo":"20101013091714.GA32608@burratino","subject":"Re: [PATCH/RFC 0/11] Building up the delta parser","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-10-18T17:00:28Z","receivedAt":"2010-10-18T17:00:28Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi Jonathan,\n\nJonathan Nieder writes:\n> Jonathan Nieder wrote:\n> \n> > Implement an svndiff 0 interpreter\n> \n> I hear that was nigh unreadable, so here's a reroll.  Less\n> cargo-cult support for broken deltas, more readability and tests.\n> Patches apply on top of \"[PATCH 15/16] t9010 (svn-fe): Eliminate\n> dependency on svn perl bindings\".  As before, the end result\n> includes a 'test-svn-fe -d' command that can apply svndiff0-format\n> deltas, meaning less binary garbage to worry about as you puzzle\n> over that confusing \"svnrdump dump\" output in debugging sessions.\n\nThanks for this! The code looks really good and the test suite is\nquite comprehensive :)\n\nEager to see this go through to `master`.\n\n-- Ram\n"},{"id":"153727","messageId":"20101018170331.GB6718@burratino","threadId":"24407","inReplyTo":"20101018170018.GJ22376@kytes","subject":"Re: [PATCH/RFC 0/11] Building up the delta parser","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-18T17:03:31Z","receivedAt":"2010-10-18T17:03:31Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Ramkumar Ramachandra wrote:\n> Jonathan Nieder writes:\n>> Jonathan Nieder wrote:\n\n>>> Implement an svndiff 0 interpreter\n[...]\n> Eager to see this go through to `master`.\n\nI think the strbuf use breaks the contrib/svn-fe/svn-fe build\nsomewhere.  Probably something like the following is needed.\n\n(moving the -I flags off of the compilation command line is\njusta \"while at it\" thing, for consistency with the toplevel\nMakefile)\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\ndiff --git a/contrib/svn-fe/Makefile b/contrib/svn-fe/Makefile\nindex 360d8da..9e8f174 100644\n--- a/contrib/svn-fe/Makefile\n+++ b/contrib/svn-fe/Makefile\n@@ -6,7 +6,7 @@ MV = mv\n \n CFLAGS = -g -O2 -Wall\n LDFLAGS =\n-ALL_CFLAGS = $(CFLAGS)\n+ALL_CFLAGS = $(CFLAGS) -I../../vcs-svn -I../..\n ALL_LDFLAGS = $(LDFLAGS)\n EXTLIBS =\n \n@@ -38,7 +38,7 @@ svn-fe$X: svn-fe.o $(VCSSVN_LIB) $(GIT_LIB)\n \t\t$(ALL_LDFLAGS) $(LIBS)\n \n svn-fe.o: svn-fe.c ../../vcs-svn/svndump.h\n-\t$(QUIET_CC)$(CC) -I../../vcs-svn -o $*.o -c $(ALL_CFLAGS) $<\n+\t$(QUIET_CC)$(CC) -o $*.o -c $(ALL_CFLAGS) $<\n \n svn-fe.html: svn-fe.txt\n \t$(QUIET_SUBDIR0)../../Documentation $(QUIET_SUBDIR1) \\\n"}]}