{"thread":{"id":"26014","subject":"[RFC/PATCH 00/10] vcs-svn: prepare for (implement?) incremental import","startedAt":"2010-12-10T10:20:07Z","lastAt":"2011-03-09T06:55:28Z","messageCount":37,"participants":["Jonathan Nieder","David Michael Barr","David Barr","Sverre Rabbelier","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":10},"messages":[{"id":"157801","messageId":"20101210102007.GA26298@burratino","threadId":"26014","inReplyTo":null,"subject":"[RFC/PATCH 00/10] vcs-svn: prepare for (implement?) incremental import","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-12-10T10:20:07Z","receivedAt":"2010-12-10T10:20:07Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nUsing David's \"ls\" command we can eliminate the in-memory repo_tree\nand rely on the target repository for information about old revs.\nThis means all state that needs to persist between svn-fe runs is\neither in the target repo or in the marks file, so in theory this\nshould allow incremental imports already (I haven't tried it).\n\nCaveats:\n\nNot split up as nicely as it ought to be. In particular, the last\npatch should be split up at least into one patch to modify git\nfast-import, another for the vcs-svn lib. Perhaps it could be split\nfiner than that.\n\nThere is a potential deadlock, marked with NEEDSWORK.\n\nError checking is not so great. Error states tend to result in\ndeadlock rather than a nicer error.\n\nIn particular, we have no way to distinguish between a missing\n(nonsense) path and an empty directory.\n\nThere's a little blind alley in the first few patches --- cat_mark and\nthe function to apply deltas to an old rev do not survive to the end\nof the series. I kept that because (1) I am lazy and (2) it serves as\na quick intro to use of the \"ls\" command.\n\nProbably there are all sorts of bugs and unclear code. I haven't\ntested beyond running the test suite.\n\nYou can find the series alone at\n\n\tgit://repo.or.cz/git/jrn.git refs/topics/db/vcs-svn-incremental\n\nand merged with other topics at\n\n\tgit://repo.or.cz/git/jrn.git vcs-svn-pu\n\nRequires the db/text-delta and db/fast-import-blob-access topics,\navailable from the same place.\n\nAny thoughts would be welcome.\n\nJonathan Nieder (10):\n vcs-svn: use higher mark numbers for blobs\n vcs-svn: save marks for imported commits\n vcs-svn: introduce cat_mark function to retrieve a marked blob\n vcs-svn: make apply_delta caller retrieve preimage\n vcs-svn: split off function to export result from delta application\n vcs-svn: do not rely on marks for old blobs\n vcs-svn: split off function to make 'ls' requests\n vcs-svn: prepare to eliminate repo_tree structure\n vcs-svn: simplifications for repo_modify_path et al\n vcs-svn: eliminate repo_tree structure\n\n cache.h | 2 +\n fast-import.c | 42 +++++--\n t/t9010-svn-fe.sh | 55 +++++----\n vcs-svn/fast_export.c | 168 +++++++++++++++++++------\n vcs-svn/fast_export.h | 22 +++-\n vcs-svn/repo_tree.c | 333 ++++---------------------------------------------\n vcs-svn/repo_tree.h | 6 +-\n vcs-svn/string_pool.c | 2 +-\n vcs-svn/string_pool.h | 2 +-\n vcs-svn/svndump.c | 89 +++++++++----\n 10 files changed, 301 insertions(+), 420 deletions(-)\n\n-- \n1.7.2.4\n"},{"id":"157802","messageId":"20101210102135.GA26331@burratino","threadId":"26014","inReplyTo":"20101210102007.GA26298@burratino","subject":"[PATCH 01/10] vcs-svn: use higher mark numbers for blobs","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-12-10T10:21:35Z","receivedAt":"2010-12-10T10:21:35Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Prepare to use mark :5 for the commit corresponding to r5 (and so on).\n\n1 billion seems sufficiently high for blob marks to avoid conflicting\nwith rev marks, while still leaving room for 3 billion blobs. Such\nhigh mark numbers cause trouble with ancient fast-import versions, but\nthis topic cannot support git fast-import versions before 1.7.4 (which\nintroduces the cat-blob command) anyway.\n\nSigned-off-by: Jonathan Nieder \n---\n vcs-svn/repo_tree.c | 2 +-\n 1 files changed, 1 insertions(+), 1 deletions(-)\n\ndiff --git a/vcs-svn/repo_tree.c b/vcs-svn/repo_tree.c\nindex eb55636..a4d8340 100644\n--- a/vcs-svn/repo_tree.c\n+++ b/vcs-svn/repo_tree.c\n@@ -298,7 +298,7 @@ void repo_commit(uint32_t revision, uint32_t author, char *log, uint32_t uuid,\n static void mark_init(void)\n {\n \tuint32_t i;\n-\tmark = 0;\n+\tmark = 1024 * 1024 * 1024;\n \tfor (i = 0; i < dent_pool.size; i++)\n \t\tif (!repo_dirent_is_dir(dent_pointer(i)) &&\n \t\t dent_pointer(i)->content_offset > mark)\n-- \n1.7.2.4\n"},{"id":"157803","messageId":"20101210102216.GB26331@burratino","threadId":"26014","inReplyTo":"20101210102007.GA26298@burratino","subject":"[PATCH 02/10] vcs-svn: save marks for imported commits","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-12-10T10:22:16Z","receivedAt":"2010-12-10T10:22:16Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Signed-off-by: Jonathan Nieder \n---\n vcs-svn/fast_export.c | 1 +\n 1 files changed, 1 insertions(+), 0 deletions(-)\n\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex e6ebdb8..093ce1d 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -61,6 +61,7 @@ void fast_export_commit(uint32_t revision, uint32_t author, char *log,\n \t\t*gitsvnline = '\\0';\n \t}\n \tprintf(\"commit refs/heads/master\\n\");\n+\tprintf(\"mark :%\"PRIu32\"\\n\", revision);\n \tprintf(\"committer %s <%s@%s> %ld +0000\\n\",\n \t\t ~author ? pool_fetch(author) : \"nobody\",\n \t\t ~author ? pool_fetch(author) : \"nobody\",\n-- \n1.7.2.4\n"},{"id":"157804","messageId":"20101210102303.GC26331@burratino","threadId":"26014","inReplyTo":"20101210102007.GA26298@burratino","subject":"[PATCH 03/10] vcs-svn: introduce cat_mark function to retrieve a marked blob","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-12-10T10:23:03Z","receivedAt":"2010-12-10T10:23:03Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Signed-off-by: Jonathan Nieder \n---\nA blind alley. But it demonstrates how this works.\n\n vcs-svn/fast_export.c | 23 +++++++++++++++--------\n 1 files changed, 15 insertions(+), 8 deletions(-)\n\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex 093ce1d..daac201 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -116,6 +116,19 @@ static const char *get_response_line(void)\n \treturn response_line.buf;\n }\n \n+static off_t cat_mark(uint32_t mark)\n+{\n+\tconst char *response;\n+\toff_t length = length;\n+\n+\tprintf(\"cat-blob :%\"PRIu32\"\\n\", mark);\n+\tfflush(stdout);\n+\tresponse = get_response_line();\n+\tif (parse_cat_response_line(response, &length))\n+\t\tdie(\"invalid cat-blob response: %s\", response);\n+\treturn length;\n+}\n+\n static long apply_delta(uint32_t mark, off_t len, struct line_buffer *input,\n \t\t\tuint32_t old_mark, uint32_t old_mode)\n {\n@@ -126,14 +139,8 @@ static long apply_delta(uint32_t mark, off_t len, struct line_buffer *input,\n \n \tif (init_postimage() || !(out = buffer_tmpfile_rewind(&postimage)))\n \t\tdie(\"cannot open temporary file for blob retrieval\");\n-\tif (old_mark) {\n-\t\tconst char *response;\n-\t\tprintf(\"cat-blob :%\"PRIu32\"\\n\", old_mark);\n-\t\tfflush(stdout);\n-\t\tresponse = get_response_line();\n-\t\tif (parse_cat_response_line(response, &preimage_len))\n-\t\t\tdie(\"invalid cat-blob response: %s\", response);\n-\t}\n+\tif (old_mark)\n+\t\tpreimage_len = cat_mark(old_mark);\n \tif (old_mode == REPO_MODE_LNK) {\n \t\tstrbuf_addstr(&preimage.buf, \"link \");\n \t\tpreimage_len += strlen(\"link \");\n-- \n1.7.2.4\n"},{"id":"157805","messageId":"20101210102358.GD26331@burratino","threadId":"26014","inReplyTo":"20101210102007.GA26298@burratino","subject":"[PATCH 04/10] vcs-svn: make apply_delta caller retrieve preimage","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-12-10T10:23:58Z","receivedAt":"2010-12-10T10:23:58Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"The preimage argument to apply_delta is currently a mark, but some\ncallers might want to use a preimage named by sha1 or by revision\nnumber and path instead. Let the caller take care of that.\n\nThe preimage_len argument represents the length of the preimage that\nwill appear followed by a newline in the REPORT_FD stream, or -1 to\njust use an empty preimage. apply_delta is renamed to delta_apply so\ncallers that have not been updated can be detected at compile time.\n\nSigned-off-by: Jonathan Nieder \n---\nThe special-cased behavior of -1 won't need to survive.\n\n vcs-svn/fast_export.c | 16 ++++++++--------\n 1 files changed, 8 insertions(+), 8 deletions(-)\n\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex daac201..4168184 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -129,25 +129,23 @@ static off_t cat_mark(uint32_t mark)\n \treturn length;\n }\n \n-static long apply_delta(uint32_t mark, off_t len, struct line_buffer *input,\n-\t\t\tuint32_t old_mark, uint32_t old_mode)\n+static long delta_apply(uint32_t mark, off_t len, struct line_buffer *input,\n+\t\t\toff_t preimage_len, uint32_t old_mode)\n {\n \tlong ret;\n-\toff_t preimage_len = 0;\n \tstruct view preimage = {REPORT_FILENO, 0, STRBUF_INIT};\n \tFILE *out;\n \n \tif (init_postimage() || !(out = buffer_tmpfile_rewind(&postimage)))\n \t\tdie(\"cannot open temporary file for blob retrieval\");\n-\tif (old_mark)\n-\t\tpreimage_len = cat_mark(old_mark);\n \tif (old_mode == REPO_MODE_LNK) {\n \t\tstrbuf_addstr(&preimage.buf, \"link \");\n-\t\tpreimage_len += strlen(\"link \");\n+\t\tif (preimage_len >= 0)\n+\t\t\tpreimage_len += strlen(\"link \");\n \t}\n \tif (svndiff0_apply(input, len, &preimage, out))\n \t\tdie(\"cannot apply delta\");\n-\tif (old_mark) {\n+\tif (preimage_len >= 0) {\n \t\t/* Read the remainder of preimage and trailing newline. */\n \t\tif (move_window(&preimage, preimage_len, 1))\n \t\t\tdie(\"cannot seek to end of input\");\n@@ -180,7 +178,9 @@ void fast_export_blob_delta(uint32_t mode, uint32_t mark,\n \tlong postimage_len;\n \tif (len > maximum_signed_value_of_type(off_t))\n \t\tdie(\"enormous delta\");\n-\tpostimage_len = apply_delta(mark, (off_t) len, input, old_mark, old_mode);\n+\tpostimage_len = delta_apply(mark, (off_t) len, input,\n+\t\t\t\t\t\told_mark ? cat_mark(old_mark) : -1,\n+\t\t\t\t\t\told_mode);\n \tif (mode == REPO_MODE_LNK) {\n \t\tbuffer_skip_bytes(&postimage, strlen(\"link \"));\n \t\tpostimage_len -= strlen(\"link \");\n-- \n1.7.2.4\n"},{"id":"157806","messageId":"20101210102517.GE26331@burratino","threadId":"26014","inReplyTo":"20101210102007.GA26298@burratino","subject":"[PATCH 05/10] vcs-svn: split off function to export result from delta application","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-12-10T10:25:17Z","receivedAt":"2010-12-10T10:25:17Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"In this time of heavy experimentation, we will need multiple\nfast_export_blob_delta variants. Make this easier by factoring\nout the common part.\n\nSigned-off-by: Jonathan Nieder \n---\nAlso not needed, except perhaps as code cleanup.\n\n vcs-svn/fast_export.c | 20 +++++++++++++-------\n 1 files changed, 13 insertions(+), 7 deletions(-)\n\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex 4168184..960b252 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -159,6 +159,18 @@ static long delta_apply(uint32_t mark, off_t len, struct line_buffer *input,\n \treturn ret;\n }\n \n+static void record_postimage(uint32_t mark, uint32_t mode,\n+\t\t\t\tlong postimage_len)\n+{\n+\tif (mode == REPO_MODE_LNK) {\n+\t\tbuffer_skip_bytes(&postimage, strlen(\"link \"));\n+\t\tpostimage_len -= strlen(\"link \");\n+\t}\n+\tprintf(\"blob\\nmark :%\"PRIu32\"\\ndata %ld\\n\", mark, postimage_len);\n+\tbuffer_copy_bytes(&postimage, postimage_len);\n+\tfputc('\\n', stdout);\n+}\n+\n void fast_export_blob(uint32_t mode, uint32_t mark, uint32_t len, struct line_buffer *input)\n {\n \tif (mode == REPO_MODE_LNK) {\n@@ -181,11 +193,5 @@ void fast_export_blob_delta(uint32_t mode, uint32_t mark,\n \tpostimage_len = delta_apply(mark, (off_t) len, input,\n \t\t\t\t\t\told_mark ? cat_mark(old_mark) : -1,\n \t\t\t\t\t\told_mode);\n-\tif (mode == REPO_MODE_LNK) {\n-\t\tbuffer_skip_bytes(&postimage, strlen(\"link \"));\n-\t\tpostimage_len -= strlen(\"link \");\n-\t}\n-\tprintf(\"blob\\nmark :%\"PRIu32\"\\ndata %ld\\n\", mark, postimage_len);\n-\tbuffer_copy_bytes(&postimage, postimage_len);\n-\tfputc('\\n', stdout);\n+\trecord_postimage(mark, mode, postimage_len);\n }\n-- \n1.7.2.4\n"},{"id":"157807","messageId":"20101210102619.GF26331@burratino","threadId":"26014","inReplyTo":"20101210102007.GA26298@burratino","subject":"[PATCH 06/10] vcs-svn: do not rely on marks for old blobs","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-12-10T10:26:19Z","receivedAt":"2010-12-10T10:26:19Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Retrieve old blobs by name and revision number from fast-import.\nOne step closer to bounded memory usage in svn-fe.\n\nSigned-off-by: Jonathan Nieder \n---\nSuperfluous except that it shows how to parse 'ls' responses.\nA demo.\n\n vcs-svn/fast_export.c | 53 +++++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/fast_export.h | 3 ++\n vcs-svn/string_pool.c | 2 +-\n vcs-svn/string_pool.h | 2 +-\n vcs-svn/svndump.c | 6 +++++\n 5 files changed, 64 insertions(+), 2 deletions(-)\n\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex 960b252..cca9810 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -88,6 +88,21 @@ static int ends_with(const char *s, size_t len, const char *suffix)\n \treturn !memcmp(s + len - suffixlen, suffix, suffixlen);\n }\n \n+static int parse_ls_response_line(const char *line, struct strbuf *objnam)\n+{\n+\tconst char *end = line + strlen(line);\n+\tconst char *name, *tab;\n+\n+\tif (end - line < strlen(\"100644 blob \"))\n+\t\treturn error(\"ls response too short: %s\", line);\n+\tname = line + strlen(\"100644 blob \");\n+\ttab = memchr(name, '\\t', end - name);\n+\tif (!tab)\n+\t\treturn error(\"ls response does not contain tab: %s\", line);\n+\tstrbuf_add(objnam, name, tab - name);\n+\treturn 0;\n+}\n+\n static int parse_cat_response_line(const char *header, off_t *len)\n {\n \tsize_t headerlen = strlen(header);\n@@ -129,6 +144,31 @@ static off_t cat_mark(uint32_t mark)\n \treturn length;\n }\n \n+static off_t cat_from_rev(uint32_t rev, const uint32_t *path)\n+{\n+\tconst char *response;\n+\toff_t length = length;\n+\tstruct strbuf blob_name = STRBUF_INIT;\n+\n+\t/* ls :5 \"path/to/old/file\" */\n+\tprintf(\"ls :%\"PRIu32\" \\\"\", rev);\n+\tpool_print_seq(REPO_MAX_PATH_DEPTH, path, '/', stdout);\n+\tprintf(\"\\\"\\n\");\n+\tfflush(stdout);\n+\n+\tresponse = get_response_line();\n+\tif (parse_ls_response_line(response, &blob_name))\n+\t\tdie(\"invalid ls response: %s\", response);\n+\n+\tprintf(\"cat-blob %s\\n\", blob_name.buf);\n+\tfflush(stdout);\n+\tresponse = get_response_line();\n+\tif (parse_cat_response_line(response, &length))\n+\t\tdie(\"invalid cat-blob response: %s\", response);\n+\tstrbuf_release(&blob_name);\n+\treturn length;\n+}\n+\n static long delta_apply(uint32_t mark, off_t len, struct line_buffer *input,\n \t\t\toff_t preimage_len, uint32_t old_mode)\n {\n@@ -195,3 +235,16 @@ void fast_export_blob_delta(uint32_t mode, uint32_t mark,\n \t\t\t\t\t\told_mode);\n \trecord_postimage(mark, mode, postimage_len);\n }\n+\n+void fast_export_blob_delta_rev(uint32_t mode, uint32_t mark, uint32_t old_mode,\n+\t\t\t\tuint32_t old_rev, const uint32_t *old_path,\n+\t\t\t\tuint32_t len, struct line_buffer *input)\n+{\n+\tlong postimage_len;\n+\tif (len > maximum_signed_value_of_type(off_t))\n+\t\tdie(\"enormous delta\");\n+\tpostimage_len = delta_apply(mark, (off_t) len, input,\n+\t\t\t\t\tcat_from_rev(old_rev, old_path),\n+\t\t\t\t\told_mode);\n+\trecord_postimage(mark, mode, postimage_len);\n+}\ndiff --git a/vcs-svn/fast_export.h b/vcs-svn/fast_export.h\nindex 6f77c3b..487d3d4 100644\n--- a/vcs-svn/fast_export.h\n+++ b/vcs-svn/fast_export.h\n@@ -13,5 +13,8 @@ void fast_export_blob(uint32_t mode, uint32_t mark, uint32_t len,\n void fast_export_blob_delta(uint32_t mode, uint32_t mark,\n \t\t\tuint32_t old_mode, uint32_t old_mark,\n \t\t\tuint32_t len, struct line_buffer *input);\n+void fast_export_blob_delta_rev(uint32_t mode, uint32_t mark, uint32_t old_mode,\n+\t\t\tuint32_t old_rev, const uint32_t *old_path,\n+\t\t\tuint32_t len, struct line_buffer *input);\n \n #endif\ndiff --git a/vcs-svn/string_pool.c b/vcs-svn/string_pool.c\nindex f5b1da8..c08abac 100644\n--- a/vcs-svn/string_pool.c\n+++ b/vcs-svn/string_pool.c\n@@ -65,7 +65,7 @@ uint32_t pool_tok_r(char *str, const char *delim, char **saveptr)\n \treturn token ? pool_intern(token) : ~0;\n }\n \n-void pool_print_seq(uint32_t len, uint32_t *seq, char delim, FILE *stream)\n+void pool_print_seq(uint32_t len, const uint32_t *seq, char delim, FILE *stream)\n {\n \tuint32_t i;\n \tfor (i = 0; i < len && ~seq[i]; i++) {\ndiff --git a/vcs-svn/string_pool.h b/vcs-svn/string_pool.h\nindex 222fb66..3720cf8 100644\n--- a/vcs-svn/string_pool.h\n+++ b/vcs-svn/string_pool.h\n@@ -4,7 +4,7 @@\n uint32_t pool_intern(const char *key);\n const char *pool_fetch(uint32_t entry);\n uint32_t pool_tok_r(char *str, const char *delim, char **saveptr);\n-void pool_print_seq(uint32_t len, uint32_t *seq, char delim, FILE *stream);\n+void pool_print_seq(uint32_t len, const uint32_t *seq, char delim, FILE *stream);\n uint32_t pool_tok_seq(uint32_t sz, uint32_t *seq, const char *delim, char *str);\n void pool_reset(void);\n \ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex c6d6337..da968fa 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -259,6 +259,12 @@ static void handle_node(void)\n \t\tfast_export_blob(node_ctx.type, mark, node_ctx.textLength, &input);\n \t\treturn;\n \t}\n+\tif (node_ctx.srcRev) {\n+\t\tfast_export_blob_delta_rev(node_ctx.type, mark, old_mode,\n+\t\t\t\t\tnode_ctx.srcRev, node_ctx.src,\n+\t\t\t\t\tnode_ctx.textLength, &input);\n+\t\treturn;\n+\t}\n \tfast_export_blob_delta(node_ctx.type, mark, old_mode, old_mark,\n \t\t\t\tnode_ctx.textLength, &input);\n }\n-- \n1.7.2.4\n"},{"id":"157808","messageId":"20101210102716.GG26331@burratino","threadId":"26014","inReplyTo":"20101210102007.GA26298@burratino","subject":"[PATCH 07/10] vcs-svn: split off function to make 'ls' requests","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-12-10T10:27:16Z","receivedAt":"2010-12-10T10:27:16Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Signed-off-by: Jonathan Nieder \n---\nls_from_rev will survive; cat_from_rev will not.\n\n vcs-svn/fast_export.c | 16 ++++++++++------\n 1 files changed, 10 insertions(+), 6 deletions(-)\n\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex cca9810..6a4a689 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -80,6 +80,15 @@ void fast_export_commit(uint32_t revision, uint32_t author, char *log,\n \tprintf(\"progress Imported commit %\"PRIu32\".\\n\\n\", revision);\n }\n \n+static void ls_from_rev(uint32_t rev, const uint32_t *path)\n+{\n+\t/* ls :5 \"path/to/old/file\" */\n+\tprintf(\"ls :%\"PRIu32\" \\\"\", rev);\n+\tpool_print_seq(REPO_MAX_PATH_DEPTH, path, '/', stdout);\n+\tprintf(\"\\\"\\n\");\n+\tfflush(stdout);\n+}\n+\n static int ends_with(const char *s, size_t len, const char *suffix)\n {\n \tconst size_t suffixlen = strlen(suffix);\n@@ -150,12 +159,7 @@ static off_t cat_from_rev(uint32_t rev, const uint32_t *path)\n \toff_t length = length;\n \tstruct strbuf blob_name = STRBUF_INIT;\n \n-\t/* ls :5 \"path/to/old/file\" */\n-\tprintf(\"ls :%\"PRIu32\" \\\"\", rev);\n-\tpool_print_seq(REPO_MAX_PATH_DEPTH, path, '/', stdout);\n-\tprintf(\"\\\"\\n\");\n-\tfflush(stdout);\n-\n+\tls_from_rev(rev, path);\n \tresponse = get_response_line();\n \tif (parse_ls_response_line(response, &blob_name))\n \t\tdie(\"invalid ls response: %s\", response);\n-- \n1.7.2.4\n"},{"id":"157809","messageId":"20101210102806.GH26331@burratino","threadId":"26014","inReplyTo":"20101210102007.GA26298@burratino","subject":"[PATCH 08/10] vcs-svn: prepare to eliminate repo_tree structure","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-12-10T10:28:06Z","receivedAt":"2010-12-10T10:28:06Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Currently svn-fe processes each commit in two stages: first decide\non the correct content for all paths and export the relevant blobs,\nthen export a commit with the result.\n\nBut we can keep less state and simplify svn-fe a great deal by\ndoing exporting the commit in one stage: use 'inline' blobs for\neach path and remember nothing. This way, the repo_tree structure\ncould be eliminated, and we would get support for incremental\nimports 'for free'.\n\nReorganize handle_node() along these lines. This is just a code\ncleanup; the functional change to repo_tree will come later.\n\nSigned-off-by: Jonathan Nieder \n---\n vcs-svn/svndump.c | 32 +++++++++++++++++++++++---------\n 1 files changed, 23 insertions(+), 9 deletions(-)\n\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex da968fa..649a468 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -201,11 +201,12 @@ static void handle_node(void)\n \tuint32_t mark = 0, old_mode, old_mark;\n \tconst uint32_t type = node_ctx.type;\n \tconst int have_props = node_ctx.propLength != LENGTH_UNKNOWN;\n+\tconst int have_text = node_ctx.textLength != LENGTH_UNKNOWN;\n \n-\tif (node_ctx.textLength != LENGTH_UNKNOWN)\n+\tif (have_text)\n \t\tmark = next_blob_mark();\n \tif (node_ctx.action == NODEACT_DELETE) {\n-\t\tif (mark || have_props || node_ctx.srcRev)\n+\t\tif (have_text || have_props || node_ctx.srcRev)\n \t\t\tdie(\"invalid dump: deletion node has \"\n \t\t\t\t\"copyfrom info, text, or properties\");\n \t\treturn repo_delete(node_ctx.dst);\n@@ -219,8 +220,13 @@ static void handle_node(void)\n \t\tif (node_ctx.action == NODEACT_ADD)\n \t\t\tnode_ctx.action = NODEACT_CHANGE;\n \t}\n-\tif (mark && type == REPO_MODE_DIR)\n+\tif (have_text && type == REPO_MODE_DIR)\n \t\tdie(\"invalid dump: directories cannot have text attached\");\n+\n+\t/*\n+\t * Find old content (old_mark) and decide on the new content (mark)\n+\t * and mode (node_ctx.type).\n+\t */\n \tif (node_ctx.action == NODEACT_CHANGE && !~*node_ctx.dst) {\n \t\tif (type != REPO_MODE_DIR)\n \t\t\tdie(\"invalid dump: root of tree is not a regular file\");\n@@ -228,7 +234,9 @@ static void handle_node(void)\n \t} else if (node_ctx.action == NODEACT_CHANGE) {\n \t\tuint32_t mode;\n \t\told_mark = repo_read_path(node_ctx.dst);\n-\t\tmode = repo_modify_path(node_ctx.dst, 0, mark);\n+\t\tif (!have_text)\n+\t\t\tmark = old_mark;\n+\t\tmode = repo_modify_path(node_ctx.dst, 0, 0);\n \t\tif (!mode)\n \t\t\tdie(\"invalid dump: path to be modified is missing\");\n \t\tif (mode == REPO_MODE_DIR && type != REPO_MODE_DIR)\n@@ -237,23 +245,29 @@ static void handle_node(void)\n \t\t\tdie(\"invalid dump: cannot modify a file into a directory\");\n \t\tnode_ctx.type = mode;\n \t} else if (node_ctx.action == NODEACT_ADD) {\n-\t\tif (!mark && type != REPO_MODE_DIR)\n+\t\tif (!have_text && type != REPO_MODE_DIR)\n \t\t\tdie(\"invalid dump: adds node without text\");\n-\t\trepo_add(node_ctx.dst, type, mark);\n \t\told_mark = 0;\n \t} else {\n \t\tdie(\"invalid dump: Node-path block lacks Node-action\");\n \t}\n+\n+\t/*\n+\t * Adjust mode to reflect properties.\n+\t */\n \told_mode = node_ctx.type;\n \tif (have_props) {\n \t\tif (!node_ctx.prop_delta)\n \t\t\tnode_ctx.type = type;\n \t\tif (node_ctx.propLength)\n \t\t\tread_props();\n-\t\tif (node_ctx.type != old_mode)\n-\t\t\trepo_modify_path(node_ctx.dst, node_ctx.type, mark);\n \t}\n-\tif (!mark)\n+\n+\t/*\n+\t * Save the result.\n+\t */\n+\trepo_add(node_ctx.dst, node_ctx.type, mark);\n+\tif (!have_text)\n \t\treturn;\n \tif (!node_ctx.text_delta) {\n \t\tfast_export_blob(node_ctx.type, mark, node_ctx.textLength, &input);\n-- \n1.7.2.4\n"},{"id":"157810","messageId":"20101210103023.GI26331@burratino","threadId":"26014","inReplyTo":"20101210102007.GA26298@burratino","subject":"[PATCH 09/10] vcs-svn: simplifications for repo_modify_path et al","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-12-10T10:30:23Z","receivedAt":"2010-12-10T10:30:23Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Restrict repo_modify_path API to functions that are actually\nneeded. That is:\n\n - decouple reading the mode and content of dirents from\n other operations\n - remove repo_modify_path. It was only used to read the\n mode from dirents.\n - remove the ability to use repo_read_mode on a missing\n path. The existing code only errored out in that case,\n anyway.\n\nSigned-off-by: Jonathan Nieder \n---\n vcs-svn/repo_tree.c | 25 +++++++++----------------\n vcs-svn/repo_tree.h | 4 ++--\n vcs-svn/svndump.c | 4 +---\n 3 files changed, 12 insertions(+), 21 deletions(-)\n\ndiff --git a/vcs-svn/repo_tree.c b/vcs-svn/repo_tree.c\nindex a4d8340..4d98185 100644\n--- a/vcs-svn/repo_tree.c\n+++ b/vcs-svn/repo_tree.c\n@@ -166,7 +166,15 @@ uint32_t repo_read_path(uint32_t *path)\n \treturn content_offset;\n }\n \n-uint32_t repo_copy(uint32_t revision, uint32_t *src, uint32_t *dst)\n+uint32_t repo_read_mode(const uint32_t *path)\n+{\n+\tstruct repo_dirent *dent = repo_read_dirent(active_commit, path);\n+\tif (dent == NULL)\n+\t\tdie(\"invalid dump: path to be modified is missing\");\n+\treturn dent->mode;\n+}\n+\n+void repo_copy(uint32_t revision, uint32_t *src, uint32_t *dst)\n {\n \tuint32_t mode = 0, content_offset = 0;\n \tstruct repo_dirent *src_dent;\n@@ -176,7 +184,6 @@ uint32_t repo_copy(uint32_t revision, uint32_t *src, uint32_t *dst)\n \t\tcontent_offset = src_dent->content_offset;\n \t\trepo_write_dirent(dst, mode, content_offset, 0);\n \t}\n-\treturn mode;\n }\n \n void repo_add(uint32_t *path, uint32_t mode, uint32_t blob_mark)\n@@ -184,20 +191,6 @@ void repo_add(uint32_t *path, uint32_t mode, uint32_t blob_mark)\n \trepo_write_dirent(path, mode, blob_mark, 0);\n }\n \n-uint32_t repo_modify_path(uint32_t *path, uint32_t mode, uint32_t blob_mark)\n-{\n-\tstruct repo_dirent *src_dent;\n-\tsrc_dent = repo_read_dirent(active_commit, path);\n-\tif (!src_dent)\n-\t\treturn 0;\n-\tif (!blob_mark)\n-\t\tblob_mark = src_dent->content_offset;\n-\tif (!mode)\n-\t\tmode = src_dent->mode;\n-\trepo_write_dirent(path, mode, blob_mark, 0);\n-\treturn mode;\n-}\n-\n void repo_delete(uint32_t *path)\n {\n \trepo_write_dirent(path, 0, 0, 1);\ndiff --git a/vcs-svn/repo_tree.h b/vcs-svn/repo_tree.h\nindex 7070839..0499a19 100644\n--- a/vcs-svn/repo_tree.h\n+++ b/vcs-svn/repo_tree.h\n@@ -12,10 +12,10 @@\n #define REPO_MAX_PATH_DEPTH 1000\n \n uint32_t next_blob_mark(void);\n-uint32_t repo_copy(uint32_t revision, uint32_t *src, uint32_t *dst);\n+void repo_copy(uint32_t revision, uint32_t *src, uint32_t *dst);\n void repo_add(uint32_t *path, uint32_t mode, uint32_t blob_mark);\n-uint32_t repo_modify_path(uint32_t *path, uint32_t mode, uint32_t blob_mark);\n uint32_t repo_read_path(uint32_t *path);\n+uint32_t repo_read_mode(const uint32_t *path);\n void repo_delete(uint32_t *path);\n void repo_commit(uint32_t revision, uint32_t author, char *log, uint32_t uuid,\n \t\t uint32_t url, long unsigned timestamp);\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex 649a468..31c6056 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -236,9 +236,7 @@ static void handle_node(void)\n \t\told_mark = repo_read_path(node_ctx.dst);\n \t\tif (!have_text)\n \t\t\tmark = old_mark;\n-\t\tmode = repo_modify_path(node_ctx.dst, 0, 0);\n-\t\tif (!mode)\n-\t\t\tdie(\"invalid dump: path to be modified is missing\");\n+\t\tmode = repo_read_mode(node_ctx.dst);\n \t\tif (mode == REPO_MODE_DIR && type != REPO_MODE_DIR)\n \t\t\tdie(\"invalid dump: cannot modify a directory into a file\");\n \t\tif (mode != REPO_MODE_DIR && type == REPO_MODE_DIR)\n-- \n1.7.2.4\n"},{"id":"157811","messageId":"20101210103354.GJ26331@burratino","threadId":"26014","inReplyTo":"20101210102007.GA26298@burratino","subject":"[PATCH 10/10] vcs-svn: eliminate repo_tree structure","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-12-10T10:33:54Z","receivedAt":"2010-12-10T10:33:54Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Rely on fast-import for information about previous revs.\n\nThis requires always setting up backward flow of information,\neven for v2 dumps. On the plus side, it simplifies the code\nby quite a bit and opens the door to further simplifications.\n\nSigned-off-by: Jonathan Nieder \n---\nThat's the end of the series. Sorry for its rough state; just thought\nit would be useful to get this out early so other tools can be built\non it. No doubt it is terribly buggy and ugly. Reports of both sorts\nof problem would be greatly appreciated.\n\n cache.h | 2 +\n fast-import.c | 42 +++++--\n t/t9010-svn-fe.sh | 55 +++++----\n vcs-svn/fast_export.c | 173 +++++++++++++++------------\n vcs-svn/fast_export.h | 25 +++--\n vcs-svn/repo_tree.c | 310 +++----------------------------------------------\n vcs-svn/repo_tree.h | 2 +-\n vcs-svn/svndump.c | 73 +++++++-----\n 8 files changed, 239 insertions(+), 443 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 33decd9..33e69eb 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -678,6 +678,8 @@ static inline void hashclr(unsigned char *hash)\n #define EMPTY_TREE_SHA1_BIN \\\n \t \"\\x4b\\x82\\x5d\\xc6\\x42\\xcb\\x6e\\xb9\\xa0\\x60\" \\\n \t \"\\xe5\\x4b\\xf8\\xd6\\x92\\x88\\xfb\\xee\\x49\\x04\"\n+#define EMPTY_BLOB_SHA1_HEX \\\n+\t\"e69de29bb2d1d6434b8b29ae775ad8c2e48c5391\"\n \n int git_mkstemp(char *path, size_t n, const char *template);\n \ndiff --git a/fast-import.c b/fast-import.c\nindex 670f4f5..e62f34d 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -2862,12 +2862,20 @@ static struct object_entry *parse_treeish_dataref(const char **r)\n \n static void print_ls(int mode, unsigned char *sha1, char *path)\n {\n-\tenum object_type type;\n \tstruct strbuf line = STRBUF_INIT;\n-\ttype = sha1_object_info(sha1, NULL);\n+\tconst char *type;\n+\n+\t/* See show_tree(). */\n+\tif (S_ISGITLINK(mode))\n+\t\ttype = commit_type;\n+\telse if (S_ISDIR(mode))\n+\t\ttype = tree_type;\n+\telse\n+\t\ttype = blob_type;\n+\n \t/* mode SP type SP object_name TAB path LF */\n-\tstrbuf_addf(&line, \"%o %s %s\\t%s\\n\",\n-\t\t\tmode, typename(type), sha1_to_hex(sha1), path);\n+\tstrbuf_addf(&line, \"%06o %s %s\\t%s\\n\",\n+\t\t\tmode, type, sha1_to_hex(sha1), path);\n \tcat_blob_write(line.buf, line.len);\n \tstrbuf_release(&line);\n }\n@@ -2898,17 +2906,25 @@ static void parse_ls(struct branch *b)\n \tif (*p)\n \t\tdie(\"Garbage after path: %s\", command_buf.buf);\n \ttree_content_get(root, uq.buf, &leaf);\n-\tif (!leaf.versions[1].mode)\n-\t\tdie(\"Path %s not in branch\", uq.buf);\n+\tif (!leaf.versions[1].mode) {\n+\t\t/*\n+\t\t * NEEDSWORK. Missing path? Must be an empty directory!\n+\t\t *\n+\t\t * Should find a nicer way to report this --- e.g.\n+\t\t *\tdie(\"Path %s not in branch\", uq.buf);\n+\t\t */\n+\t\tprint_ls(S_IFDIR, (unsigned char *) EMPTY_TREE_SHA1_BIN, uq.buf);\n+\t} else {\n+\t\t/*\n+\t\t * A directory in preparation would have a sha1 of zero\n+\t\t * until it is saved. Save, for simplicity.\n+\t\t */\n+\t\tif (S_ISDIR(leaf.versions[1].mode))\n+\t\t\tstore_tree(&leaf);\n \n-\t/*\n-\t * A directory in preparation would have a sha1 of zero\n-\t * until it is saved. Save, for simplicity.\n-\t */\n-\tif (S_ISDIR(leaf.versions[1].mode))\n-\t\tstore_tree(&leaf);\n+\t\tprint_ls(leaf.versions[1].mode, leaf.versions[1].sha1, uq.buf);\n+\t}\n \n-\tprint_ls(leaf.versions[1].mode, leaf.versions[1].sha1, uq.buf);\n \tstrbuf_release(&uq);\n \tif (!b || root != &b->branch_tree)\n \t\trelease_tree_entry(root);\ndiff --git a/t/t9010-svn-fe.sh b/t/t9010-svn-fe.sh\nindex 022e3e3..34bb9f6 100755\n--- a/t/t9010-svn-fe.sh\n+++ b/t/t9010-svn-fe.sh\n@@ -53,8 +53,7 @@ test_expect_success 'empty dump' '\n test_expect_success 'v4 dumps not supported' '\n \treinit_git &&\n \techo \"SVN-fs-dump-format-version: 4\" >v4.dump &&\n-\ttest_must_fail test-svn-fe v4.dump >stream &&\n-\ttest_cmp empty stream\n+\ttest_must_fail test-svn-fe v4.dump\n '\n \n test_expect_failure 'empty revision' '\n@@ -301,8 +300,9 @@ test_expect_success 'action: add node without text' '\n \ttest_must_fail test-svn-fe textless.dump\n '\n \n-test_expect_failure 'change file mode but keep old content' '\n+test_expect_failure PIPE 'change file mode but keep old content' '\n \treinit_git &&\n+\trm -f backflow &&\n \tcat >expect <<-\\EOF &&\n \tOBJID\n \t:120000 100644 OBJID OBJID T\tgreeting\n@@ -364,8 +364,9 @@ test_expect_failure 'change file mode but keep old content' '\n \n \tPROPS-END\n \tEOF\n-\ttest-svn-fe filemode.dump >stream &&\n-\tgit fast-import backflow &&\n \t{\n \t\tgit rev-list HEAD |\n \t\tgit diff-tree --root --stdin |\n@@ -378,8 +379,9 @@ test_expect_failure 'change file mode but keep old content' '\n \ttest_cmp hello actual.target\n '\n \n-test_expect_success 'change file mode and reiterate content' '\n+test_expect_success PIPE 'change file mode and reiterate content' '\n \treinit_git &&\n+\trm -f backflow &&\n \tcat >expect <<-\\EOF &&\n \tOBJID\n \t:120000 100644 OBJID OBJID T\tgreeting\n@@ -390,7 +392,7 @@ test_expect_success 'change file mode and reiterate content' '\n \tEOF\n \techo \"link hello\" >expect.blob &&\n \techo hello >hello &&\n-\tcat >filemode.dump <<-\\EOF &&\n+\tcat >filemode2.dump <<-\\EOF &&\n \tSVN-fs-dump-format-version: 3\n \n \tRevision-number: 1\n@@ -445,8 +447,9 @@ test_expect_success 'change file mode and reiterate content' '\n \tPROPS-END\n \tlink hello\n \tEOF\n-\ttest-svn-fe filemode.dump >stream &&\n-\tgit fast-import backflow &&\n \t{\n \t\tgit rev-list HEAD |\n \t\tgit diff-tree --root --stdin |\n@@ -522,12 +525,13 @@ test_expect_success PIPE 'deltas supported' '\n \t\tcat delta\n \t} >delta.dump &&\n \tmkfifo backflow &&\n-\ttest_must_fail test-svn-fe delta.dump 3backflow\n '\n \n-test_expect_success 'property deltas supported' '\n+test_expect_success PIPE 'property deltas supported' '\n \treinit_git &&\n+\trm -f backflow &&\n \tcat >expect <<-\\EOF &&\n \tOBJID\n \t:100755 100644 OBJID OBJID M\tscript.sh\n@@ -582,8 +586,9 @@ test_expect_success 'property deltas supported' '\n \t\tPROPS-END\n \t\tEOF\n \t} >propdelta.dump &&\n-\ttest-svn-fe propdelta.dump >stream &&\n-\tgit fast-import backflow &&\n \t{\n \t\tgit rev-list HEAD |\n \t\tgit diff-tree --stdin |\n@@ -592,8 +597,9 @@ test_expect_success 'property deltas supported' '\n \ttest_cmp expect actual\n '\n \n-test_expect_success 'properties on /' '\n+test_expect_success PIPE 'properties on /' '\n \treinit_git &&\n+\trm -f backflow &&\n \tcat <<-\\EOF >expect &&\n \tOBJID\n \tOBJID\n@@ -637,8 +643,9 @@ test_expect_success 'properties on /' '\n \n \tPROPS-END\n \tEOF\n-\ttest-svn-fe changeroot.dump >stream &&\n-\tgit fast-import backflow &&\n \t{\n \t\tgit rev-list HEAD |\n \t\tgit diff-tree --root --always --stdin |\n@@ -647,8 +654,9 @@ test_expect_success 'properties on /' '\n \ttest_cmp expect actual\n '\n \n-test_expect_success 'deltas for typechange' '\n+test_expect_success PIPE 'deltas for typechange' '\n \treinit_git &&\n+\trm -f backflow &&\n \tcat >expect <<-\\EOF &&\n \tOBJID\n \t:120000 100644 OBJID OBJID T\ttest-file\n@@ -723,8 +731,9 @@ test_expect_success 'deltas for typechange' '\n \tPROPS-END\n \tlink testing 321\n \tEOF\n-\ttest-svn-fe deleteprop.dump >stream &&\n-\tgit fast-import backflow &&\n \t{\n \t\tgit rev-list HEAD |\n \t\tgit diff-tree --root --stdin |\n@@ -841,15 +850,17 @@ test_expect_success PIPE 'deltas need not consume the whole preimage' '\n \ttest_cmp expect.3 actual.3\n '\n \n-test_expect_success 't9135/svn.dump' '\n+test_expect_success PIPE 't9135/svn.dump' '\n+\trm -f backflow &&\n \tsvnadmin create simple-svn &&\n \tsvnadmin load simple-svn <\"$TEST_DIRECTORY/t9135/svn.dump\" &&\n \tsvn_cmd export \"file://$PWD/simple-svn\" simple-svnco &&\n \tgit init simple-git &&\n-\ttest-svn-fe \"$TEST_DIRECTORY/t9135/svn.dump\" >simple.fe &&\n+\tmkfifo backflow &&\n+\ttest-svn-fe \"$TEST_DIRECTORY/t9135/svn.dump\" 3../backflow\n \t) &&\n \t(\n \t\tcd simple-svnco &&\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex 6a4a689..96f6023 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -37,17 +37,17 @@ void fast_export_delete(uint32_t depth, uint32_t *path)\n \tputchar('\\n');\n }\n \n-void fast_export_modify(uint32_t depth, uint32_t *path, uint32_t mode,\n-\t\t\tuint32_t mark)\n+void fast_export_modify(uint32_t depth, const uint32_t *path, uint32_t mode,\n+\t\t\tconst char *dataref)\n {\n \t/* Mode must be 100644, 100755, 120000, or 160000. */\n-\tprintf(\"M %06\"PRIo32\" :%\"PRIu32\" \", mode, mark);\n+\tprintf(\"M %06\"PRIo32\" %s \", mode, dataref);\n \tpool_print_seq(depth, path, '/', stdout);\n \tputchar('\\n');\n }\n \n static char gitsvnline[MAX_GITSVN_LINE_LEN];\n-void fast_export_commit(uint32_t revision, uint32_t author, char *log,\n+void fast_export_begin_commit(uint32_t revision, uint32_t author, char *log,\n \t\t\tuint32_t uuid, uint32_t url,\n \t\t\tunsigned long timestamp)\n {\n@@ -74,10 +74,20 @@ void fast_export_commit(uint32_t revision, uint32_t author, char *log,\n \t\t\tprintf(\"from refs/heads/master^0\\n\");\n \t\tfirst_commit_done = 1;\n \t}\n-\trepo_diff(revision - 1, revision);\n-\tfputc('\\n', stdout);\n+}\n \n-\tprintf(\"progress Imported commit %\"PRIu32\".\\n\\n\", revision);\n+void fast_export_end_commit(uint32_t revision)\n+{\n+\tprintf(\"\\nprogress Imported commit %\"PRIu32\".\\n\\n\", revision);\n+}\n+\n+static struct strbuf response_line = STRBUF_INIT;\n+static const char *get_response_line(void)\n+{\n+\tstrbuf_reset(&response_line);\n+\tif (fd_read_line(&response_line, REPORT_FILENO))\n+\t\treturn NULL;\n+\treturn response_line.buf;\n }\n \n static void ls_from_rev(uint32_t rev, const uint32_t *path)\n@@ -89,6 +99,56 @@ static void ls_from_rev(uint32_t rev, const uint32_t *path)\n \tfflush(stdout);\n }\n \n+static void parse_ls_response(const char *response, uint32_t *mode,\n+\t\t\t\t\tstruct strbuf *dataref)\n+{\n+\tconst char *tab;\n+\tconst char *response_end;\n+\n+\tif (!response)\n+\t\tdie(\"cannot read ls response\");\n+\tresponse_end = response + strlen(response);\n+\n+\t/* Mode. */\n+\tif (response_end - response < strlen(\"100644\") ||\n+\t response[strlen(\"100644\")] != ' ')\n+\t\tdie(\"invalid ls response: missing mode: %s\", response);\n+\t*mode = 0;\n+\tfor (; *response != ' '; response++) {\n+\t\t*mode *= 8;\n+\t\t*mode += (*response - '0');\n+\t}\n+\n+\t/* ' blob ' or ' tree ' */\n+\tif (response_end - response < strlen(\" blob \"))\n+\t\tdie(\"invalid ls response: missing type: %s\", response);\n+\tresponse += strlen(\" blob \");\n+\n+\t/* Dataref. */\n+\ttab = memchr(response, '\\t', response_end - response);\n+\tif (!tab)\n+\t\tdie(\"invalid ls response: missing tab: %s\", response);\n+\tstrbuf_add(dataref, response, tab - response);\n+}\n+\n+void fast_export_ls_rev(uint32_t rev, const uint32_t *path,\n+\t\t\t\tuint32_t *mode, struct strbuf *dataref)\n+{\n+\tls_from_rev(rev, path);\n+\tparse_ls_response(get_response_line(), mode, dataref);\n+}\n+\n+/* Read directory entry from the active commit. */\n+void fast_export_ls(const uint32_t *path,\n+\t\t\t\tuint32_t *mode, struct strbuf *dataref)\n+{\n+\tprintf(\"ls \\\"\");\n+\tpool_print_seq(REPO_MAX_PATH_DEPTH, path, '/', stdout);\n+\tprintf(\"\\\"\\n\");\n+\tfflush(stdout);\n+\tparse_ls_response(get_response_line(), mode, dataref);\n+}\n+\n static int ends_with(const char *s, size_t len, const char *suffix)\n {\n \tconst size_t suffixlen = strlen(suffix);\n@@ -97,27 +157,15 @@ static int ends_with(const char *s, size_t len, const char *suffix)\n \treturn !memcmp(s + len - suffixlen, suffix, suffixlen);\n }\n \n-static int parse_ls_response_line(const char *line, struct strbuf *objnam)\n-{\n-\tconst char *end = line + strlen(line);\n-\tconst char *name, *tab;\n-\n-\tif (end - line < strlen(\"100644 blob \"))\n-\t\treturn error(\"ls response too short: %s\", line);\n-\tname = line + strlen(\"100644 blob \");\n-\ttab = memchr(name, '\\t', end - name);\n-\tif (!tab)\n-\t\treturn error(\"ls response does not contain tab: %s\", line);\n-\tstrbuf_add(objnam, name, tab - name);\n-\treturn 0;\n-}\n-\n static int parse_cat_response_line(const char *header, off_t *len)\n {\n-\tsize_t headerlen = strlen(header);\n+\tsize_t headerlen;\n \tconst char *type;\n \tconst char *end;\n \n+\tif (!header)\n+\t\treturn error(\"missing cat-blob response\");\n+\theaderlen = strlen(header);\n \tif (ends_with(header, headerlen, \" missing\"))\n \t\treturn error(\"cat-blob reports missing blob: %s\", header);\n \ttype = memmem(header, headerlen, \" blob \", strlen(\" blob \"));\n@@ -131,49 +179,22 @@ static int parse_cat_response_line(const char *header, off_t *len)\n \treturn 0;\n }\n \n-static struct strbuf response_line = STRBUF_INIT;\n-static const char *get_response_line(void)\n+static off_t cat_dataref(const char *dataref)\n {\n-\tstrbuf_reset(&response_line);\n-\tif (fd_read_line(&response_line, REPORT_FILENO))\n-\t\treturn NULL;\n-\treturn response_line.buf;\n-}\n-\n-static off_t cat_mark(uint32_t mark)\n-{\n-\tconst char *response;\n \toff_t length = length;\n-\n-\tprintf(\"cat-blob :%\"PRIu32\"\\n\", mark);\n-\tfflush(stdout);\n-\tresponse = get_response_line();\n-\tif (parse_cat_response_line(response, &length))\n-\t\tdie(\"invalid cat-blob response: %s\", response);\n-\treturn length;\n-}\n-\n-static off_t cat_from_rev(uint32_t rev, const uint32_t *path)\n-{\n \tconst char *response;\n-\toff_t length = length;\n-\tstruct strbuf blob_name = STRBUF_INIT;\n-\n-\tls_from_rev(rev, path);\n-\tresponse = get_response_line();\n-\tif (parse_ls_response_line(response, &blob_name))\n-\t\tdie(\"invalid ls response: %s\", response);\n \n-\tprintf(\"cat-blob %s\\n\", blob_name.buf);\n+\tif (!dataref)\n+\t\tdie(\"BUG: null data reference\");\n+\tprintf(\"cat-blob %s\\n\", dataref);\n \tfflush(stdout);\n \tresponse = get_response_line();\n \tif (parse_cat_response_line(response, &length))\n-\t\tdie(\"invalid cat-blob response: %s\", response);\n-\tstrbuf_release(&blob_name);\n+\t\tdie(\"invalid cat-blob response\");\n \treturn length;\n }\n \n-static long delta_apply(uint32_t mark, off_t len, struct line_buffer *input,\n+static long delta_apply(off_t len, struct line_buffer *input,\n \t\t\toff_t preimage_len, uint32_t old_mode)\n {\n \tlong ret;\n@@ -203,52 +224,50 @@ static long delta_apply(uint32_t mark, off_t len, struct line_buffer *input,\n \treturn ret;\n }\n \n-static void record_postimage(uint32_t mark, uint32_t mode,\n-\t\t\t\tlong postimage_len)\n+static void record_postimage(uint32_t mode, long postimage_len)\n {\n \tif (mode == REPO_MODE_LNK) {\n \t\tbuffer_skip_bytes(&postimage, strlen(\"link \"));\n \t\tpostimage_len -= strlen(\"link \");\n \t}\n-\tprintf(\"blob\\nmark :%\"PRIu32\"\\ndata %ld\\n\", mark, postimage_len);\n+\tprintf(\"data %ld\\n\", postimage_len);\n \tbuffer_copy_bytes(&postimage, postimage_len);\n \tfputc('\\n', stdout);\n }\n \n-void fast_export_blob(uint32_t mode, uint32_t mark, uint32_t len, struct line_buffer *input)\n+void fast_export_data(uint32_t mode, uint32_t len, struct line_buffer *input)\n {\n \tif (mode == REPO_MODE_LNK) {\n \t\t/* svn symlink blobs start with \"link \" */\n \t\tbuffer_skip_bytes(input, 5);\n \t\tlen -= 5;\n \t}\n-\tprintf(\"blob\\nmark :%\"PRIu32\"\\ndata %\"PRIu32\"\\n\", mark, len);\n+\tprintf(\"data %\"PRIu32\"\\n\", len);\n \tbuffer_copy_bytes(input, len);\n \tfputc('\\n', stdout);\n }\n \n-void fast_export_blob_delta(uint32_t mode, uint32_t mark,\n-\t\t\t\tuint32_t old_mode, uint32_t old_mark,\n-\t\t\t\tuint32_t len, struct line_buffer *input)\n+void fast_export_empty_blob(void)\n {\n-\tlong postimage_len;\n-\tif (len > maximum_signed_value_of_type(off_t))\n-\t\tdie(\"enormous delta\");\n-\tpostimage_len = delta_apply(mark, (off_t) len, input,\n-\t\t\t\t\t\told_mark ? cat_mark(old_mark) : -1,\n-\t\t\t\t\t\told_mode);\n-\trecord_postimage(mark, mode, postimage_len);\n+\tprintf(\"blob\\ndata 0\\n\\n\");\n }\n \n-void fast_export_blob_delta_rev(uint32_t mode, uint32_t mark, uint32_t old_mode,\n-\t\t\t\tuint32_t old_rev, const uint32_t *old_path,\n+void fast_export_delta(const uint32_t *path, uint32_t mode, uint32_t old_mode,\n+\t\t\t\tconst char *dataref,\n \t\t\t\tuint32_t len, struct line_buffer *input)\n {\n+\toff_t preimage_len;\n \tlong postimage_len;\n \tif (len > maximum_signed_value_of_type(off_t))\n \t\tdie(\"enormous delta\");\n-\tpostimage_len = delta_apply(mark, (off_t) len, input,\n-\t\t\t\t\tcat_from_rev(old_rev, old_path),\n-\t\t\t\t\told_mode);\n-\trecord_postimage(mark, mode, postimage_len);\n+\tpreimage_len = cat_dataref(dataref);\n+\n+\t/*\n+\t * NEEDSWORK: Will deadlock with very long paths.\n+\t */\n+\tfast_export_modify(REPO_MAX_PATH_DEPTH, path, mode, \"inline\");\n+\n+\tpostimage_len = delta_apply((off_t) len, input,\n+\t\t\t\t\t\tpreimage_len, old_mode);\n+\trecord_postimage(mode, postimage_len);\n }\ndiff --git a/vcs-svn/fast_export.h b/vcs-svn/fast_export.h\nindex 487d3d4..8415d2c 100644\n--- a/vcs-svn/fast_export.h\n+++ b/vcs-svn/fast_export.h\n@@ -3,18 +3,23 @@\n \n #include \"line_buffer.h\"\n \n+/* Output routines */\n void fast_export_delete(uint32_t depth, uint32_t *path);\n-void fast_export_modify(uint32_t depth, uint32_t *path, uint32_t mode,\n-\t\t\tuint32_t mark);\n-void fast_export_commit(uint32_t revision, uint32_t author, char *log,\n+void fast_export_modify(uint32_t depth, const uint32_t *path, uint32_t mode,\n+\t\t\tconst char *dataref);\n+void fast_export_begin_commit(uint32_t revision, uint32_t author, char *log,\n \t\t\tuint32_t uuid, uint32_t url, unsigned long timestamp);\n-void fast_export_blob(uint32_t mode, uint32_t mark, uint32_t len,\n-\t\t struct line_buffer *input);\n-void fast_export_blob_delta(uint32_t mode, uint32_t mark,\n-\t\t\tuint32_t old_mode, uint32_t old_mark,\n-\t\t\tuint32_t len, struct line_buffer *input);\n-void fast_export_blob_delta_rev(uint32_t mode, uint32_t mark, uint32_t old_mode,\n-\t\t\tuint32_t old_rev, const uint32_t *old_path,\n+void fast_export_end_commit(uint32_t revision);\n+void fast_export_empty_blob(void);\n+void fast_export_data(uint32_t mode, uint32_t len, struct line_buffer *input);\n+void fast_export_delta(const uint32_t *path, uint32_t mode, uint32_t old_mode,\n+\t\t\tconst char *dataref,\n \t\t\tuint32_t len, struct line_buffer *input);\n \n+/* Input routines */\n+void fast_export_ls_rev(uint32_t rev, const uint32_t *path,\n+\t\t\tuint32_t *mode_out, struct strbuf *dataref_out);\n+void fast_export_ls(const uint32_t *path,\n+\t\t\tuint32_t *mode_out, struct strbuf *dataref_out);\n+\n #endif\ndiff --git a/vcs-svn/repo_tree.c b/vcs-svn/repo_tree.c\nindex 4d98185..1283089 100644\n--- a/vcs-svn/repo_tree.c\n+++ b/vcs-svn/repo_tree.c\n@@ -4,321 +4,49 @@\n */\n \n #include \"git-compat-util.h\"\n-\n-#include \"string_pool.h\"\n+#include \"strbuf.h\"\n #include \"repo_tree.h\"\n-#include \"obj_pool.h\"\n #include \"fast_export.h\"\n \n-#include \"trp.h\"\n-\n-struct repo_dirent {\n-\tuint32_t name_offset;\n-\tstruct trp_node children;\n-\tuint32_t mode;\n-\tuint32_t content_offset;\n-};\n-\n-struct repo_dir {\n-\tstruct trp_root entries;\n-};\n-\n-struct repo_commit {\n-\tuint32_t root_dir_offset;\n-};\n-\n-/* Memory pools for commit, dir and dirent */\n-obj_pool_gen(commit, struct repo_commit, 4096)\n-obj_pool_gen(dir, struct repo_dir, 4096)\n-obj_pool_gen(dent, struct repo_dirent, 4096)\n-\n-static uint32_t active_commit;\n-static uint32_t mark;\n-\n-static int repo_dirent_name_cmp(const void *a, const void *b);\n-\n-/* Treap for directory entries */\n-trp_gen(static, dent_, struct repo_dirent, children, dent, repo_dirent_name_cmp);\n-\n-uint32_t next_blob_mark(void)\n-{\n-\treturn mark++;\n-}\n-\n-static struct repo_dir *repo_commit_root_dir(struct repo_commit *commit)\n-{\n-\treturn dir_pointer(commit->root_dir_offset);\n-}\n-\n-static struct repo_dirent *repo_first_dirent(struct repo_dir *dir)\n-{\n-\treturn dent_first(&dir->entries);\n-}\n-\n-static int repo_dirent_name_cmp(const void *a, const void *b)\n-{\n-\tconst struct repo_dirent *dent1 = a, *dent2 = b;\n-\tuint32_t a_offset = dent1->name_offset;\n-\tuint32_t b_offset = dent2->name_offset;\n-\treturn (a_offset > b_offset) - (a_offset < b_offset);\n-}\n-\n-static int repo_dirent_is_dir(struct repo_dirent *dent)\n-{\n-\treturn dent != NULL && dent->mode == REPO_MODE_DIR;\n-}\n-\n-static struct repo_dir *repo_dir_from_dirent(struct repo_dirent *dent)\n-{\n-\tif (!repo_dirent_is_dir(dent))\n-\t\treturn NULL;\n-\treturn dir_pointer(dent->content_offset);\n-}\n-\n-static struct repo_dir *repo_clone_dir(struct repo_dir *orig_dir)\n-{\n-\tuint32_t orig_o, new_o;\n-\torig_o = dir_offset(orig_dir);\n-\tif (orig_o >= dir_pool.committed)\n-\t\treturn orig_dir;\n-\tnew_o = dir_alloc(1);\n-\torig_dir = dir_pointer(orig_o);\n-\t*dir_pointer(new_o) = *orig_dir;\n-\treturn dir_pointer(new_o);\n-}\n-\n-static struct repo_dirent *repo_read_dirent(uint32_t revision, uint32_t *path)\n+static struct strbuf dataref_buf = STRBUF_INIT;\n+const char *repo_read_path(uint32_t *path)\n {\n-\tuint32_t name = 0;\n-\tstruct repo_dirent *key = dent_pointer(dent_alloc(1));\n-\tstruct repo_dir *dir = NULL;\n-\tstruct repo_dirent *dent = NULL;\n-\tdir = repo_commit_root_dir(commit_pointer(revision));\n-\twhile (~(name = *path++)) {\n-\t\tkey->name_offset = name;\n-\t\tdent = dent_search(&dir->entries, key);\n-\t\tif (dent == NULL || !repo_dirent_is_dir(dent))\n-\t\t\tbreak;\n-\t\tdir = repo_dir_from_dirent(dent);\n-\t}\n-\tdent_free(1);\n-\treturn dent;\n-}\n+\tuint32_t unused;\n \n-static void repo_write_dirent(uint32_t *path, uint32_t mode,\n-\t\t\t uint32_t content_offset, uint32_t del)\n-{\n-\tuint32_t name, revision, dir_o = ~0, parent_dir_o = ~0;\n-\tstruct repo_dir *dir;\n-\tstruct repo_dirent *key;\n-\tstruct repo_dirent *dent = NULL;\n-\trevision = active_commit;\n-\tdir = repo_commit_root_dir(commit_pointer(revision));\n-\tdir = repo_clone_dir(dir);\n-\tcommit_pointer(revision)->root_dir_offset = dir_offset(dir);\n-\twhile (~(name = *path++)) {\n-\t\tparent_dir_o = dir_offset(dir);\n-\n-\t\tkey = dent_pointer(dent_alloc(1));\n-\t\tkey->name_offset = name;\n-\n-\t\tdent = dent_search(&dir->entries, key);\n-\t\tif (dent == NULL)\n-\t\t\tdent = key;\n-\t\telse\n-\t\t\tdent_free(1);\n-\n-\t\tif (dent == key) {\n-\t\t\tdent->mode = REPO_MODE_DIR;\n-\t\t\tdent->content_offset = 0;\n-\t\t\tdent_insert(&dir->entries, dent);\n-\t\t}\n-\n-\t\tif (dent_offset(dent) < dent_pool.committed) {\n-\t\t\tdir_o = repo_dirent_is_dir(dent) ?\n-\t\t\t\t\tdent->content_offset : ~0;\n-\t\t\tdent_remove(&dir->entries, dent);\n-\t\t\tdent = dent_pointer(dent_alloc(1));\n-\t\t\tdent->name_offset = name;\n-\t\t\tdent->mode = REPO_MODE_DIR;\n-\t\t\tdent->content_offset = dir_o;\n-\t\t\tdent_insert(&dir->entries, dent);\n-\t\t}\n-\n-\t\tdir = repo_dir_from_dirent(dent);\n-\t\tdir = repo_clone_dir(dir);\n-\t\tdent->content_offset = dir_offset(dir);\n-\t}\n-\tif (dent == NULL)\n-\t\treturn;\n-\tdent->mode = mode;\n-\tdent->content_offset = content_offset;\n-\tif (del && ~parent_dir_o)\n-\t\tdent_remove(&dir_pointer(parent_dir_o)->entries, dent);\n-}\n-\n-uint32_t repo_read_path(uint32_t *path)\n-{\n-\tuint32_t content_offset = 0;\n-\tstruct repo_dirent *dent = repo_read_dirent(active_commit, path);\n-\tif (dent != NULL)\n-\t\tcontent_offset = dent->content_offset;\n-\treturn content_offset;\n+\tstrbuf_reset(&dataref_buf);\n+\tfast_export_ls(path, &unused, &dataref_buf);\n+\treturn dataref_buf.buf;\n }\n \n uint32_t repo_read_mode(const uint32_t *path)\n {\n-\tstruct repo_dirent *dent = repo_read_dirent(active_commit, path);\n-\tif (dent == NULL)\n-\t\tdie(\"invalid dump: path to be modified is missing\");\n-\treturn dent->mode;\n+\tuint32_t result;\n+\tstruct strbuf unused = STRBUF_INIT;\n+\n+\tfast_export_ls(path, &result, &unused);\n+\tstrbuf_release(&unused);\n+\treturn result;\n }\n \n void repo_copy(uint32_t revision, uint32_t *src, uint32_t *dst)\n {\n-\tuint32_t mode = 0, content_offset = 0;\n-\tstruct repo_dirent *src_dent;\n-\tsrc_dent = repo_read_dirent(revision, src);\n-\tif (src_dent != NULL) {\n-\t\tmode = src_dent->mode;\n-\t\tcontent_offset = src_dent->content_offset;\n-\t\trepo_write_dirent(dst, mode, content_offset, 0);\n-\t}\n-}\n+\tuint32_t mode;\n+\tstruct strbuf data = STRBUF_INIT;\n \n-void repo_add(uint32_t *path, uint32_t mode, uint32_t blob_mark)\n-{\n-\trepo_write_dirent(path, mode, blob_mark, 0);\n+\tfast_export_ls_rev(revision, src, &mode, &data);\n+\tfast_export_modify(REPO_MAX_PATH_DEPTH, dst, mode, data.buf);\n+\tstrbuf_release(&data);\n }\n \n void repo_delete(uint32_t *path)\n {\n-\trepo_write_dirent(path, 0, 0, 1);\n-}\n-\n-static void repo_git_add_r(uint32_t depth, uint32_t *path, struct repo_dir *dir);\n-\n-static void repo_git_add(uint32_t depth, uint32_t *path, struct repo_dirent *dent)\n-{\n-\tif (repo_dirent_is_dir(dent))\n-\t\trepo_git_add_r(depth, path, repo_dir_from_dirent(dent));\n-\telse\n-\t\tfast_export_modify(depth, path,\n-\t\t\t\t dent->mode, dent->content_offset);\n-}\n-\n-static void repo_git_add_r(uint32_t depth, uint32_t *path, struct repo_dir *dir)\n-{\n-\tstruct repo_dirent *de = repo_first_dirent(dir);\n-\twhile (de) {\n-\t\tpath[depth] = de->name_offset;\n-\t\trepo_git_add(depth + 1, path, de);\n-\t\tde = dent_next(&dir->entries, de);\n-\t}\n-}\n-\n-static void repo_diff_r(uint32_t depth, uint32_t *path, struct repo_dir *dir1,\n-\t\t\tstruct repo_dir *dir2)\n-{\n-\tstruct repo_dirent *de1, *de2;\n-\tde1 = repo_first_dirent(dir1);\n-\tde2 = repo_first_dirent(dir2);\n-\n-\twhile (de1 && de2) {\n-\t\tif (de1->name_offset < de2->name_offset) {\n-\t\t\tpath[depth] = de1->name_offset;\n-\t\t\tfast_export_delete(depth + 1, path);\n-\t\t\tde1 = dent_next(&dir1->entries, de1);\n-\t\t\tcontinue;\n-\t\t}\n-\t\tif (de1->name_offset > de2->name_offset) {\n-\t\t\tpath[depth] = de2->name_offset;\n-\t\t\trepo_git_add(depth + 1, path, de2);\n-\t\t\tde2 = dent_next(&dir2->entries, de2);\n-\t\t\tcontinue;\n-\t\t}\n-\t\tpath[depth] = de1->name_offset;\n-\n-\t\tif (de1->mode == de2->mode &&\n-\t\t de1->content_offset == de2->content_offset) {\n-\t\t\t; /* No change. */\n-\t\t} else if (repo_dirent_is_dir(de1) && repo_dirent_is_dir(de2)) {\n-\t\t\trepo_diff_r(depth + 1, path,\n-\t\t\t\t repo_dir_from_dirent(de1),\n-\t\t\t\t repo_dir_from_dirent(de2));\n-\t\t} else if (!repo_dirent_is_dir(de1) && !repo_dirent_is_dir(de2)) {\n-\t\t\trepo_git_add(depth + 1, path, de2);\n-\t\t} else {\n-\t\t\tfast_export_delete(depth + 1, path);\n-\t\t\trepo_git_add(depth + 1, path, de2);\n-\t\t}\n-\t\tde1 = dent_next(&dir1->entries, de1);\n-\t\tde2 = dent_next(&dir2->entries, de2);\n-\t}\n-\twhile (de1) {\n-\t\tpath[depth] = de1->name_offset;\n-\t\tfast_export_delete(depth + 1, path);\n-\t\tde1 = dent_next(&dir1->entries, de1);\n-\t}\n-\twhile (de2) {\n-\t\tpath[depth] = de2->name_offset;\n-\t\trepo_git_add(depth + 1, path, de2);\n-\t\tde2 = dent_next(&dir2->entries, de2);\n-\t}\n-}\n-\n-static uint32_t path_stack[REPO_MAX_PATH_DEPTH];\n-\n-void repo_diff(uint32_t r1, uint32_t r2)\n-{\n-\trepo_diff_r(0,\n-\t\t path_stack,\n-\t\t repo_commit_root_dir(commit_pointer(r1)),\n-\t\t repo_commit_root_dir(commit_pointer(r2)));\n-}\n-\n-void repo_commit(uint32_t revision, uint32_t author, char *log, uint32_t uuid,\n-\t\t uint32_t url, unsigned long timestamp)\n-{\n-\tfast_export_commit(revision, author, log, uuid, url, timestamp);\n-\tdent_commit();\n-\tdir_commit();\n-\tactive_commit = commit_alloc(1);\n-\tcommit_pointer(active_commit)->root_dir_offset =\n-\t\tcommit_pointer(active_commit - 1)->root_dir_offset;\n-}\n-\n-static void mark_init(void)\n-{\n-\tuint32_t i;\n-\tmark = 1024 * 1024 * 1024;\n-\tfor (i = 0; i < dent_pool.size; i++)\n-\t\tif (!repo_dirent_is_dir(dent_pointer(i)) &&\n-\t\t dent_pointer(i)->content_offset > mark)\n-\t\t\tmark = dent_pointer(i)->content_offset;\n-\tmark++;\n+\tfast_export_delete(REPO_MAX_PATH_DEPTH, path);\n }\n \n void repo_init(void)\n {\n-\tmark_init();\n-\tif (commit_pool.size == 0) {\n-\t\t/* Create empty tree for commit 0. */\n-\t\tcommit_alloc(1);\n-\t\tcommit_pointer(0)->root_dir_offset = dir_alloc(1);\n-\t\tdir_pointer(0)->entries.trp_root = ~0;\n-\t\tdir_commit();\n-\t}\n-\t/* Preallocate next commit, ready for changes. */\n-\tactive_commit = commit_alloc(1);\n-\tcommit_pointer(active_commit)->root_dir_offset =\n-\t\tcommit_pointer(active_commit - 1)->root_dir_offset;\n }\n \n void repo_reset(void)\n {\n-\tpool_reset();\n-\tcommit_reset();\n-\tdir_reset();\n-\tdent_reset();\n }\ndiff --git a/vcs-svn/repo_tree.h b/vcs-svn/repo_tree.h\nindex 0499a19..559f99f 100644\n--- a/vcs-svn/repo_tree.h\n+++ b/vcs-svn/repo_tree.h\n@@ -14,7 +14,7 @@\n uint32_t next_blob_mark(void);\n void repo_copy(uint32_t revision, uint32_t *src, uint32_t *dst);\n void repo_add(uint32_t *path, uint32_t mode, uint32_t blob_mark);\n-uint32_t repo_read_path(uint32_t *path);\n+const char *repo_read_path(uint32_t *path);\n uint32_t repo_read_mode(const uint32_t *path);\n void repo_delete(uint32_t *path);\n void repo_commit(uint32_t revision, uint32_t author, char *log, uint32_t uuid,\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex 31c6056..68a8435 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -20,9 +20,11 @@\n #define NODEACT_CHANGE 1\n #define NODEACT_UNKNOWN 0\n \n-#define DUMP_CTX 0\n-#define REV_CTX 1\n-#define NODE_CTX 2\n+/* States: */\n+#define DUMP_CTX 0\t/* dump metadata */\n+#define REV_CTX 1\t/* revision metadata */\n+#define NODE_CTX 2\t/* node metadata */\n+#define INTERNODE_CTX 3\t/* between nodes */\n \n #define LENGTH_UNKNOWN (~0)\n #define DATE_RFC2822_LEN 31\n@@ -198,13 +200,12 @@ static void read_props(void)\n \n static void handle_node(void)\n {\n-\tuint32_t mark = 0, old_mode, old_mark;\n+\tuint32_t old_mode;\n+\tconst char *old_data;\n \tconst uint32_t type = node_ctx.type;\n \tconst int have_props = node_ctx.propLength != LENGTH_UNKNOWN;\n \tconst int have_text = node_ctx.textLength != LENGTH_UNKNOWN;\n \n-\tif (have_text)\n-\t\tmark = next_blob_mark();\n \tif (node_ctx.action == NODEACT_DELETE) {\n \t\tif (have_text || have_props || node_ctx.srcRev)\n \t\t\tdie(\"invalid dump: deletion node has \"\n@@ -224,18 +225,15 @@ static void handle_node(void)\n \t\tdie(\"invalid dump: directories cannot have text attached\");\n \n \t/*\n-\t * Find old content (old_mark) and decide on the new content (mark)\n-\t * and mode (node_ctx.type).\n+\t * Find old content (old_data) and decide on the new mode.\n \t */\n \tif (node_ctx.action == NODEACT_CHANGE && !~*node_ctx.dst) {\n \t\tif (type != REPO_MODE_DIR)\n \t\t\tdie(\"invalid dump: root of tree is not a regular file\");\n-\t\told_mark = 0;\n+\t\told_data = NULL;\n \t} else if (node_ctx.action == NODEACT_CHANGE) {\n \t\tuint32_t mode;\n-\t\told_mark = repo_read_path(node_ctx.dst);\n-\t\tif (!have_text)\n-\t\t\tmark = old_mark;\n+\t\told_data = repo_read_path(node_ctx.dst);\n \t\tmode = repo_read_mode(node_ctx.dst);\n \t\tif (mode == REPO_MODE_DIR && type != REPO_MODE_DIR)\n \t\t\tdie(\"invalid dump: cannot modify a directory into a file\");\n@@ -243,9 +241,12 @@ static void handle_node(void)\n \t\t\tdie(\"invalid dump: cannot modify a file into a directory\");\n \t\tnode_ctx.type = mode;\n \t} else if (node_ctx.action == NODEACT_ADD) {\n-\t\tif (!have_text && type != REPO_MODE_DIR)\n+\t\tif (type == REPO_MODE_DIR)\n+\t\t\told_data = NULL;\n+\t\telse if (have_text)\n+\t\t\told_data = EMPTY_BLOB_SHA1_HEX;\n+\t\telse\n \t\t\tdie(\"invalid dump: adds node without text\");\n-\t\told_mark = 0;\n \t} else {\n \t\tdie(\"invalid dump: Node-path block lacks Node-action\");\n \t}\n@@ -264,28 +265,35 @@ static void handle_node(void)\n \t/*\n \t * Save the result.\n \t */\n-\trepo_add(node_ctx.dst, node_ctx.type, mark);\n-\tif (!have_text)\n+\tif (type == REPO_MODE_DIR)\t/* directories are not tracked. */\n \t\treturn;\n+\tif (!have_text) {\n+\t\tfast_export_modify(REPO_MAX_PATH_DEPTH, node_ctx.dst, node_ctx.type,\n+\t\t\t\t\told_data);\n+\t\treturn;\n+\t}\n \tif (!node_ctx.text_delta) {\n-\t\tfast_export_blob(node_ctx.type, mark, node_ctx.textLength, &input);\n+\t\tfast_export_modify(REPO_MAX_PATH_DEPTH, node_ctx.dst, node_ctx.type,\n+\t\t\t\t\t\"inline\");\n+\t\tfast_export_data(node_ctx.type, node_ctx.textLength, &input);\n \t\treturn;\n \t}\n-\tif (node_ctx.srcRev) {\n-\t\tfast_export_blob_delta_rev(node_ctx.type, mark, old_mode,\n-\t\t\t\t\tnode_ctx.srcRev, node_ctx.src,\n+\tfast_export_delta(node_ctx.dst, node_ctx.type, old_mode, old_data,\n \t\t\t\t\tnode_ctx.textLength, &input);\n+}\n+\n+static void begin_revision(void)\n+{\n+\tif (!rev_ctx.revision)\t/* revision 0 gets no git commit. */\n \t\treturn;\n-\t}\n-\tfast_export_blob_delta(node_ctx.type, mark, old_mode, old_mark,\n-\t\t\t\tnode_ctx.textLength, &input);\n+\tfast_export_begin_commit(rev_ctx.revision, rev_ctx.author, rev_ctx.log,\n+\t\tdump_ctx.uuid, dump_ctx.url, rev_ctx.timestamp);\n }\n \n-static void handle_revision(void)\n+static void end_revision(void)\n {\n \tif (rev_ctx.revision)\n-\t\trepo_commit(rev_ctx.revision, rev_ctx.author, rev_ctx.log,\n-\t\t\tdump_ctx.uuid, dump_ctx.url, rev_ctx.timestamp);\n+\t\tfast_export_end_commit(rev_ctx.revision);\n }\n \n void svndump_read(const char *url)\n@@ -315,13 +323,17 @@ void svndump_read(const char *url)\n \t\t} else if (key == keys.revision_number) {\n \t\t\tif (active_ctx == NODE_CTX)\n \t\t\t\thandle_node();\n+\t\t\tif (active_ctx == REV_CTX)\n+\t\t\t\tbegin_revision();\n \t\t\tif (active_ctx != DUMP_CTX)\n-\t\t\t\thandle_revision();\n+\t\t\t\tend_revision();\n \t\t\tactive_ctx = REV_CTX;\n \t\t\treset_rev_ctx(atoi(val));\n \t\t} else if (key == keys.node_path) {\n \t\t\tif (active_ctx == NODE_CTX)\n \t\t\t\thandle_node();\n+\t\t\tif (active_ctx == REV_CTX)\n+\t\t\t\tbegin_revision();\n \t\t\tactive_ctx = NODE_CTX;\n \t\t\treset_node_ctx(val);\n \t\t} else if (key == keys.node_kind) {\n@@ -363,7 +375,7 @@ void svndump_read(const char *url)\n \t\t\t\tread_props();\n \t\t\t} else if (active_ctx == NODE_CTX) {\n \t\t\t\thandle_node();\n-\t\t\t\tactive_ctx = REV_CTX;\n+\t\t\t\tactive_ctx = INTERNODE_CTX;\n \t\t\t} else {\n \t\t\t\tfprintf(stderr, \"Unexpected content length header: %\"PRIu32\"\\n\", len);\n \t\t\t\tbuffer_skip_bytes(&input, len);\n@@ -372,8 +384,10 @@ void svndump_read(const char *url)\n \t}\n \tif (active_ctx == NODE_CTX)\n \t\thandle_node();\n+\tif (active_ctx == REV_CTX)\n+\t\tbegin_revision();\n \tif (active_ctx != DUMP_CTX)\n-\t\thandle_revision();\n+\t\tend_revision();\n }\n \n int svndump_init(const char *filename)\n@@ -385,6 +399,7 @@ int svndump_init(const char *filename)\n \treset_rev_ctx(0);\n \treset_node_ctx(NULL);\n \tinit_keys();\n+\tfast_export_empty_blob();\n \treturn 0;\n }\n \n-- \n1.7.2.4\n"},{"id":"157867","messageId":"20101211224732.GA18822@burratino","threadId":"26014","inReplyTo":"20101211184654.GA17464@burratino","subject":"[RFC/PATCH] fast-import: treat filemodify with empty tree as delete","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-12-11T22:47:32Z","receivedAt":"2010-12-11T22:47:32Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Jonathan Nieder wrote:\n\n> Maybe fast-import should make the check itself, to avoid\n> writing trees with empty directories that would be hard to\n> re-create with \"git write-tree\".\n\nLike this, maybe?\n\n-- 8< --\nSubject: fast-import: treat filemodify with empty tree as delete\n\nTraditionally, git trees do not contain entries for empty\nsubdirectories. Generally speaking, subtrees are not created or\ndestroyed explicitly; instead, they automatically appear when needed\nto hold regular files, symlinks, and submodules.\n\nv1.7.3-rc0~75^2 (Teach fast-import to import subtrees named by tree\nid, 2010-06-30) changed that, by allowing an empty subtree to be\nincluded in a fast-import stream explicitly:\n\n\tM 040000 4b825dc642cb6eb9a060e54bf8d69288fbee4904 subdir\n\nThat was unintentional. Better and more closely analogous to \"git\nread-tree --prefix\" to treat such an input line as a request to delete\n(\"to empty\") subdir.\n\nNoticed-by: David Barr \nSigned-off-by: Jonathan Nieder \n---\nTests use the \"ls\" command from vcs-svn-pu. The actual change\nwould apply cleanly to master or maint, though.\n\n fast-import.c | 10 ++++\n t/t9300-fast-import.sh | 107 ++++++++++++++++++++++++++++++++++++++++++++----\n 2 files changed, 109 insertions(+), 8 deletions(-)\n\ndiff --git a/fast-import.c b/fast-import.c\nindex e62f34d..c774893 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -2196,6 +2196,16 @@ static void file_change_m(struct branch *b)\n \t\tp = uq.buf;\n \t}\n \n+\t/*\n+\t * Git does not track empty, non-toplevel directories.\n+\t */\n+\tif (S_ISDIR(mode) &&\n+\t !memcmp(sha1, (unsigned char *) EMPTY_TREE_SHA1_BIN, 20) &&\n+\t *p) {\n+\t\ttree_content_remove(&b->branch_tree, p, NULL);\n+\t\treturn;\n+\t}\n+\n \tif (S_ISGITLINK(mode)) {\n \t\tif (inline_data)\n \t\t\tdie(\"Git links cannot be specified 'inline': %s\",\ndiff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh\nindex b0e3bda..c17f704 100755\n--- a/t/t9300-fast-import.sh\n+++ b/t/t9300-fast-import.sh\n@@ -25,6 +25,14 @@ echo \"$@\"'\n \n >empty\n \n+test_expect_success 'setup: have pipes?' '\n+\trm -f frob &&\n+\tif mkfifo frob\n+\tthen\n+\t\ttest_set_prereq PIPE\n+\tfi\n+'\n+\n ###\n ### series A\n ###\n@@ -881,6 +889,97 @@ test_expect_success \\\n \t git diff-tree -C --find-copies-harder -r N4^ N4 >actual &&\n \t compare_diff_raw expect actual'\n \n+test_expect_success PIPE 'N: read and copy directory' '\n+\tcat >expect <<-\\EOF\n+\t:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100\tfile2/newf\tfile3/newf\n+\t:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 C100\tfile2/oldf\tfile3/oldf\n+\tEOF\n+\tgit update-ref -d refs/heads/N4 &&\n+\trm -f backflow &&\n+\tmkfifo backflow &&\n+\t(\n+\t\texec $GIT_COMMITTER_DATE\n+\t\tdata <backflow &&\n+\tgit diff-tree -C --find-copies-harder -r N4^ N4 >actual &&\n+\tcompare_diff_raw expect actual\n+'\n+\n+test_expect_success PIPE 'N: read and copy \"empty\" directory' '\n+\tcat <<-\\EOF >expect &&\n+\tOBJNAME\n+\t:000000 100644 OBJNAME OBJNAME A\tgreeting\n+\tOBJNAME\n+\t:100644 000000 OBJNAME OBJNAME D\tunrelated\n+\tOBJNAME\n+\t:000000 100644 OBJNAME OBJNAME A\tunrelated\n+\tEOF\n+\tgit update-ref -d refs/heads/copy-empty &&\n+\trm -f backflow &&\n+\tmkfifo backflow &&\n+\t(\n+\t\texec $GIT_COMMITTER_DATE\n+\t\tdata < $GIT_COMMITTER_DATE\n+\t\tdata < $GIT_COMMITTER_DATE\n+\t\tdata <backflow &&\n+\tgit rev-list copy-empty |\n+\tgit diff-tree -r --root --stdin |\n+\tsed \"s/$_x40/OBJNAME/g\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n test_expect_success \\\n \t'N: copy root directory by tree hash' \\\n \t'cat >expect <<-\\EOF &&\n@@ -1773,14 +1872,6 @@ test_expect_success 'R: print two blobs to stdout' '\n \ttest_cmp expect actual\n '\n \n-test_expect_success 'setup: have pipes?' '\n-\trm -f frob &&\n-\tif mkfifo frob\n-\tthen\n-\t\ttest_set_prereq PIPE\n-\tfi\n-'\n-\n test_expect_success PIPE 'R: copy using cat-file' '\n \texpect_id=$(git hash-object big) &&\n \texpect_len=$(wc -c I am not totally happy with the result, since it adds another\n> git-specific detail to svn-fe (the first was hardcoding the\n> empty blob sha1 instead of using\n> \n> \tblob\n> \tmark :0\n> \tdata 0\n> \n> ).\n\nMaybe this would help?\n\n-- 8< --\nSubject: vcs-svn: avoid git-isms in fast-import stream\n\nCurrent svn-fe is not likely to work without change with other\nfast-import backends, but don't let that stop us from trying:\n\n - instead of suppressing copies of empty trees, let the backend\n decide what to do with them;\n\n - use a mark instead of hard-coding git's name for the empty blob.\n\nHowever, we do not include commands in the stream for new empty\ndirectories, since no syntax is documented for that yet.\n\nSigned-off-by: Jonathan Nieder \n---\n vcs-svn/fast_export.c | 8 +-------\n vcs-svn/svndump.c | 2 +-\n 2 files changed, 2 insertions(+), 8 deletions(-)\n\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex 75d674e..85166a6 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -40,12 +40,6 @@ void fast_export_delete(uint32_t depth, const uint32_t *path)\n void fast_export_modify(uint32_t depth, const uint32_t *path, uint32_t mode,\n \t\t\tconst char *dataref)\n {\n-\t/* Git does not track empty directories. */\n-\tif (S_ISDIR(mode) && !strcmp(dataref, EMPTY_TREE_SHA1_HEX)) {\n-\t\tfast_export_delete(depth, path);\n-\t\treturn;\n-\t}\n-\n \t/* Mode must be 100644, 100755, 120000, or 160000. */\n \tprintf(\"M %06\"PRIo32\" %s \", mode, dataref);\n \tpool_print_seq(depth, path, '/', stdout);\n@@ -255,7 +249,7 @@ void fast_export_data(uint32_t mode, uint32_t len, struct line_buffer *input)\n \n void fast_export_empty_blob(void)\n {\n-\tprintf(\"blob\\ndata 0\\n\\n\");\n+\tprintf(\"blob\\nmark :0\\ndata 0\\n\\n\");\n }\n \n void fast_export_delta(const uint32_t *path, uint32_t mode, uint32_t old_mode,\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex 68a8435..e28e762 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -244,7 +244,7 @@ static void handle_node(void)\n \t\tif (type == REPO_MODE_DIR)\n \t\t\told_data = NULL;\n \t\telse if (have_text)\n-\t\t\told_data = EMPTY_BLOB_SHA1_HEX;\n+\t\t\told_data = \":0\";\n \t\telse\n \t\t\tdie(\"invalid dump: adds node without text\");\n \t} else {\n-- \n1.7.2.4\n"},{"id":"157869","messageId":"DD24C01C-19FD-424B-B602-E9BB1A930805@cordelta.com","threadId":"26014","inReplyTo":"20101210102007.GA26298@burratino","subject":"[PATCH 12/10] vcs-svn: quote paths correctly for ls command","fromName":"David Michael Barr","fromEmail":"david.barr@cordelta.com","sentAt":"2010-12-11T23:04:39Z","receivedAt":"2010-12-11T23:04:39Z","isPatch":true,"sender":{"key":"david.barr@cordelta.com","avatar":"https://avatars.githubusercontent.com/u/220594?v=4"},"body":"From: David Barr \nDate: Sun, 12 Dec 2010 03:59:31 +1100\nSubject: [PATCH] vcs-svn: quote paths correctly for ls command\n\nThis bug was found while importing rev 601865 of ASF.\n\nSigned-off-by: David Barr \n---\n vcs-svn/fast_export.c | 4 ++--\n vcs-svn/string_pool.c | 11 +++++++++++\n vcs-svn/string_pool.h | 1 +\n 3 files changed, 14 insertions(+), 2 deletions(-)\n\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex c798f6d..d2397d8 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -100,7 +100,7 @@ static void ls_from_rev(uint32_t rev, const uint32_t *path)\n {\n \t/* ls :5 \"path/to/old/file\" */\n \tprintf(\"ls :%\"PRIu32\" \\\"\", rev);\n-\tpool_print_seq(REPO_MAX_PATH_DEPTH, path, '/', stdout);\n+\tpool_print_seq_q(REPO_MAX_PATH_DEPTH, path, '/', stdout);\n \tprintf(\"\\\"\\n\");\n \tfflush(stdout);\n }\n@@ -149,7 +149,7 @@ void fast_export_ls(const uint32_t *path,\n \t\t\t\tuint32_t *mode, struct strbuf *dataref)\n {\n \tprintf(\"ls \\\"\");\n-\tpool_print_seq(REPO_MAX_PATH_DEPTH, path, '/', stdout);\n+\tpool_print_seq_q(REPO_MAX_PATH_DEPTH, path, '/', stdout);\n \tprintf(\"\\\"\\n\");\n \tfflush(stdout);\n \tparse_ls_response(get_response_line(), mode, dataref);\ndiff --git a/vcs-svn/string_pool.c b/vcs-svn/string_pool.c\nindex c08abac..a03f5a4 100644\n--- a/vcs-svn/string_pool.c\n+++ b/vcs-svn/string_pool.c\n@@ -4,6 +4,8 @@\n */\n \n #include \"git-compat-util.h\"\n+#include \"strbuf.h\"\n+#include \"quote.h\"\n #include \"trp.h\"\n #include \"obj_pool.h\"\n #include \"string_pool.h\"\n@@ -75,6 +77,15 @@ void pool_print_seq(uint32_t len, const uint32_t *seq, char delim, FILE *stream)\n \t}\n }\n \n+void pool_print_seq_q(uint32_t len, const uint32_t *seq, char delim, FILE *stream)\n+{\n+\tuint32_t i;\n+\tfor (i = 0; i < len && ~seq[i]; i++) {\n+\t\tquote_c_style(pool_fetch(seq[i]), NULL, stream, 1);\n+\t\tif (i < len - 1 && ~seq[i + 1])\n+\t\t\tfputc(delim, stream);\n+\t}\n+}\n uint32_t pool_tok_seq(uint32_t sz, uint32_t *seq, const char *delim, char *str)\n {\n \tchar *context = NULL;\ndiff --git a/vcs-svn/string_pool.h b/vcs-svn/string_pool.h\nindex 3720cf8..96e501d 100644\n--- a/vcs-svn/string_pool.h\n+++ b/vcs-svn/string_pool.h\n@@ -5,6 +5,7 @@ uint32_t pool_intern(const char *key);\n const char *pool_fetch(uint32_t entry);\n uint32_t pool_tok_r(char *str, const char *delim, char **saveptr);\n void pool_print_seq(uint32_t len, const uint32_t *seq, char delim, FILE *stream);\n+void pool_print_seq_q(uint32_t len, const uint32_t *seq, char delim, FILE *stream);\n uint32_t pool_tok_seq(uint32_t sz, uint32_t *seq, const char *delim, char *str);\n void pool_reset(void);\n \n-- \n1.7.3.2.846.gf4b062\n"},{"id":"157870","messageId":"20101211231123.GA31179@burratino","threadId":"26014","inReplyTo":"DD24C01C-19FD-424B-B602-E9BB1A930805@cordelta.com","subject":"[PATCH db/vcs-svn-incremental] vcs-svn: quote all paths passed to fast-import","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-12-11T23:11:23Z","receivedAt":"2010-12-11T23:11:23Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Filenames with linefeeds or double quotes need to be quoted if\nfast-import is not to misinterpret them.\n\nSigned-off-by: Jonathan Nieder \n---\n vcs-svn/fast_export.c | 13 ++++++-------\n 1 files changed, 6 insertions(+), 7 deletions(-)\n\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex 7856ff2..2d2a6b2 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -31,19 +31,18 @@ static int init_postimage(void)\n \n void fast_export_delete(uint32_t depth, const uint32_t *path)\n {\n-\tputchar('D');\n-\tputchar(' ');\n-\tpool_print_seq(depth, path, '/', stdout);\n-\tputchar('\\n');\n+\tprintf(\"D \\\"\");\n+\tpool_print_seq_q(depth, path, '/', stdout);\n+\tprintf(\"\\\"\\n\");\n }\n \n void fast_export_modify(uint32_t depth, const uint32_t *path, uint32_t mode,\n \t\t\tconst char *dataref)\n {\n \t/* Mode must be 100644, 100755, 120000, or 160000. */\n-\tprintf(\"M %06\"PRIo32\" %s \", mode, dataref);\n-\tpool_print_seq(depth, path, '/', stdout);\n-\tputchar('\\n');\n+\tprintf(\"M %06\"PRIo32\" %s \\\"\", mode, dataref);\n+\tpool_print_seq_q(depth, path, '/', stdout);\n+\tprintf(\"\\\"\\n\");\n }\n \n static char gitsvnline[MAX_GITSVN_LINE_LEN];\n-- \n1.7.2.4\n"},{"id":"157881","messageId":"11EEDD77-214C-44DB-AC7B-C1A086E35E4C@cordelta.com","threadId":"26014","inReplyTo":"20101210102007.GA26298@burratino","subject":"[PATCH 13/10] vcs-svn: use mark from previous import for parent commit","fromName":"David Michael Barr","fromEmail":"david.barr@cordelta.com","sentAt":"2010-12-12T09:32:57Z","receivedAt":"2010-12-12T09:32:57Z","isPatch":true,"sender":{"key":"david.barr@cordelta.com","avatar":"https://avatars.githubusercontent.com/u/220594?v=4"},"body":"From: David Barr \nDate: Sun, 12 Dec 2010 13:41:38 +1100\nSubject: [PATCH] vcs-svn: use mark from previous import for parent commit\n\nSigned-off-by: David Barr \n---\n vcs-svn/fast_export.c | 2 +-\n 1 files changed, 1 insertions(+), 1 deletions(-)\n\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex d2397d8..6abd108 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -77,7 +77,7 @@ void fast_export_begin_commit(uint32_t revision, uint32_t author, char *log,\n \t\t log, gitsvnline);\n \tif (!first_commit_done) {\n \t\tif (revision > 1)\n-\t\t\tprintf(\"from refs/heads/master^0\\n\");\n+\t\t\tprintf(\"from :%\"PRIu32\"\\n\", revision - 1);\n \t\tfirst_commit_done = 1;\n \t}\n }\n-- \n1.7.3.2.846.gf4b062\n"},{"id":"157891","messageId":"20101212170629.GA18847@burratino","threadId":"26014","inReplyTo":"11EEDD77-214C-44DB-AC7B-C1A086E35E4C@cordelta.com","subject":"Re: [PATCH 13/10] vcs-svn: use mark from previous import for parent commit","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-12-12T17:06:29Z","receivedAt":"2010-12-12T17:06:29Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"David Michael Barr wrote:\n\n> Subject: [PATCH] vcs-svn: use mark from previous import for parent commit\n[...]\n> +++ b/vcs-svn/fast_export.c\n> @@ -77,7 +77,7 @@ void fast_export_begin_commit(uint32_t revision, uint32_t\n> log, gitsvnline);\n> if (!first_commit_done) {\n> if (revision > 1)\n> - printf(\"from refs/heads/master^0\\n\");\n> + printf(\"from :%\"PRIu32\"\\n\", revision - 1);\n\nThis deals more sanely with attempts to continue an import starting at\nthe wrong revision.\n\nExample: if I try\n\n\timport () {\n\t\trm -f backflow\n\t\tmkfifo backflow\n\n\t\tsvn-fe 3backflow\n\t}\n\n\tsvnrdump -r0:100 $url | import\n\tsvnrdump -r100:200 $url | import continue\n\nthen svn-fe should correctly re-import r100 the second time, instead\nof trying to apply the same deltas twice. If I try\n\n\tsvnrdump -r0:100 $url | import\n\tsvnrdump -r102:200 $url | import continue\n\nthen the second command should error out.\n\nThanks, queued.\n"},{"id":"162841","messageId":"20110306111524.GA12761@elie","threadId":"26014","inReplyTo":"20101210102216.GB26331@burratino","subject":"Re: [PATCH 02/10] vcs-svn: save marks for imported commits","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-03-06T11:15:24Z","receivedAt":"2011-03-06T11:15:24Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"(pruned cc because reviving an old thread)\nHi,\n\nJonathan Nieder wrote:\n\n> [Subject: vcs-svn: save marks for imported commits]\n>\n> Signed-off-by: Jonathan Nieder \n\nThat's a lousy commit message. A version committed later (that\neventually found its way to David's repository) explains:\n\n\tThis way, a person can use\n\n\t\tsvnadmin dump $path |\n\t\tsvn-fe |\n\t\tgit fast-import --relative-marks --export-marks=svn-revs\n\n\tto get a list of what commit corresponds to each svn revision (plus\n\tsome irrelevant blob names) in .git/info/fast-import/svn-revs.\n\nIn other words, this is the first half of a two-way mapping between\nsvn and git commit names.\n\nIt should be possible to build a mapping the other way like so:\n\n\tawk '\n\tBEGIN {\n\t\tprint \"commit refs/notes/svn-id\";\n\t\tprintf \"committer \";\n\t\tsystem(\"git var GIT_COMMITTER_IDENT\");\n\t\tprint \"data <\n---\nHi,\n\nJonathan Nieder wrote:\n\n[...]\n> @@ -228,7 +234,9 @@ static void handle_node(void)\n> \t} else if (node_ctx.action == NODEACT_CHANGE) {\n> \t\tuint32_t mode;\n> \t\told_mark = repo_read_path(node_ctx.dst);\n> -\t\tmode = repo_modify_path(node_ctx.dst, 0, mark);\n> +\t\tif (!have_text)\n> +\t\t\tmark = old_mark;\n> +\t\tmode = repo_modify_path(node_ctx.dst, 0, 0);\n[...]\n\nBackported so we can merge it a little sooner (thanks!). It's very\nsimilar to the original patch except there is no old_mark --- the\nprevious text at a path is useless until we learn to patch it (which\nis a topic for another day).\n\nDavid, may I have your sign-off on this series (the\nvcs-svn-incremental branch)?\n\n vcs-svn/svndump.c | 33 +++++++++++++++++++++++----------\n 1 files changed, 23 insertions(+), 10 deletions(-)\n\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex ee7c0bb..f07376f 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -201,13 +201,14 @@ static void handle_node(void)\n \tuint32_t mark = 0;\n \tconst uint32_t type = node_ctx.type;\n \tconst int have_props = node_ctx.propLength != LENGTH_UNKNOWN;\n+\tconst int have_text = node_ctx.textLength != LENGTH_UNKNOWN;\n \n \tif (node_ctx.text_delta)\n \t\tdie(\"text deltas not supported\");\n-\tif (node_ctx.textLength != LENGTH_UNKNOWN)\n+\tif (have_text)\n \t\tmark = next_blob_mark();\n \tif (node_ctx.action == NODEACT_DELETE) {\n-\t\tif (mark || have_props || node_ctx.srcRev)\n+\t\tif (have_text || have_props || node_ctx.srcRev)\n \t\t\tdie(\"invalid dump: deletion node has \"\n \t\t\t\t\"copyfrom info, text, or properties\");\n \t\treturn repo_delete(node_ctx.dst);\n@@ -221,13 +222,20 @@ static void handle_node(void)\n \t\tif (node_ctx.action == NODEACT_ADD)\n \t\t\tnode_ctx.action = NODEACT_CHANGE;\n \t}\n-\tif (mark && type == REPO_MODE_DIR)\n+\tif (have_text && type == REPO_MODE_DIR)\n \t\tdie(\"invalid dump: directories cannot have text attached\");\n+\n+\t/*\n+\t * Decide on the new content (mark) and mode (node_ctx.type).\n+\t */\n \tif (node_ctx.action == NODEACT_CHANGE && !~*node_ctx.dst) {\n \t\tif (type != REPO_MODE_DIR)\n \t\t\tdie(\"invalid dump: root of tree is not a regular file\");\n \t} else if (node_ctx.action == NODEACT_CHANGE) {\n-\t\tuint32_t mode = repo_modify_path(node_ctx.dst, 0, mark);\n+\t\tuint32_t mode;\n+\t\tif (!have_text)\n+\t\t\tmark = repo_read_path(node_ctx.dst);\n+\t\tmode = repo_modify_path(node_ctx.dst, 0, 0);\n \t\tif (!mode)\n \t\t\tdie(\"invalid dump: path to be modified is missing\");\n \t\tif (mode == REPO_MODE_DIR && type != REPO_MODE_DIR)\n@@ -236,22 +244,27 @@ static void handle_node(void)\n \t\t\tdie(\"invalid dump: cannot modify a file into a directory\");\n \t\tnode_ctx.type = mode;\n \t} else if (node_ctx.action == NODEACT_ADD) {\n-\t\tif (!mark && type != REPO_MODE_DIR)\n+\t\tif (!have_text && type != REPO_MODE_DIR)\n \t\t\tdie(\"invalid dump: adds node without text\");\n-\t\trepo_add(node_ctx.dst, type, mark);\n \t} else {\n \t\tdie(\"invalid dump: Node-path block lacks Node-action\");\n \t}\n+\n+\t/*\n+\t * Adjust mode to reflect properties.\n+\t */\n \tif (have_props) {\n-\t\tconst uint32_t old_mode = node_ctx.type;\n \t\tif (!node_ctx.prop_delta)\n \t\t\tnode_ctx.type = type;\n \t\tif (node_ctx.propLength)\n \t\t\tread_props();\n-\t\tif (node_ctx.type != old_mode)\n-\t\t\trepo_modify_path(node_ctx.dst, node_ctx.type, mark);\n \t}\n-\tif (mark)\n+\n+\t/*\n+\t * Save the result.\n+\t */\n+\trepo_add(node_ctx.dst, node_ctx.type, mark);\n+\tif (have_text)\n \t\tfast_export_blob(node_ctx.type, mark,\n \t\t\t\t node_ctx.textLength, &input);\n }\n-- \n1.7.4.1.91.g15e19.dirty\n"},{"id":"162857","messageId":"AANLkTimX1+hZGZ-y01ePCdWhPmwjUFpUbJe1-iB4GQrq@mail.gmail.com","threadId":"26014","inReplyTo":"20110306125259.GA20010@elie","subject":"Re: [PATCH v2] vcs-svn: prepare to eliminate repo_tree structure","fromName":"David Barr","fromEmail":"david.barr@cordelta.com","sentAt":"2011-03-06T20:41:33Z","receivedAt":"2011-03-06T20:41:33Z","isPatch":true,"sender":{"key":"david.barr@cordelta.com","avatar":"https://avatars.githubusercontent.com/u/220594?v=4"},"body":"Hi,\n\n> David, may I have your sign-off on this series (the\n> vcs-svn-incremental branch)?\n\nAbsolutely, I've updated my branch with the correct sign-offs.\n\nPlease pull\n\n git://github.com/barrbrain/git.git vcs-svn-incremental\n\nfor the complete series of back-ported patches to support incremental\nimports for vcs-svn.\n\n--\nDavid Barr.\n"},{"id":"162873","messageId":"20110306225419.GA24327@elie","threadId":"26014","inReplyTo":"20101210102007.GA26298@burratino","subject":"[PATCH v2 00/12] vcs-svn: incremental import","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-03-06T22:54:40Z","receivedAt":"2011-03-06T22:54:40Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi again,\n\nJonathan Nieder wrote:\n\n> Using David's \"ls\" command we can eliminate the in-memory repo_tree\n> and rely on the target repository for information about old revs.\n\nHere's a reroll. Aside from the aspects already mentioned (which\navoid a dependency on the mostly orthogonal topic of support for text\ndeltas), the original patch #10 has been split into smaller, more\neasily digestible pieces.\n\nMost of the credit for this incarnation of the series belongs to\nDavid, who heroically streamlined it and untangled it from other\ntopics.\n\nPatch 1 changes the mark numbers for blobs to be ridiculously high,\nto make room for memorable commit marks (:1 for r1, :2 for r2, etc).\nPatch 2 brings those commit marks into existence, as mentioned before.\n\nPatches 3-5 simplify the repo-tree API somewhat. They are somewhat\nminimal; patches on top of this series offering further simplification\nwould be very welcome.\n\nPatch 6 is a bit sneaky. We want svn-fe's output to change from\n\n\t\n\t\n\t...\n\t\n\t\n\nto\n\n\t\n\tM 100644 inline one/path\n\t\n\tM 100644 inline another/path\n\t...\n\t\n\nsince the latter allows svn-fe to maintain much less state. But\nthat's a big change, so patch 6 introduces a stepping stone on the way\nthere:\n\n\t\n\t\n\t...\n\t\n\nThat paves the way for patches 7-11, which teach svn-fe to rely\non the fast-import backend for information about previously\nimported blobs, at long last.\n\nThe visible effects should be:\n\n - svn-fe _requires_ a backchannel from the fast-import\n backend now. You can't do\n\n\tsvn-fe stream &&\n\tfast-import backchannel\n\n for both imports.\n\nI'm not happy about the loss of usability but I'm happy about the gain\nin functionality. A good next step might be to build a simple remote\nhelper to make this comfortable to use.\n\nThoughts? Improvements? Complaints? Despite the deficiencies just\nmentioned I'm tempted to push this out soon. Feedback in either\ndirection would be welcome.\n\nDavid Barr (3):\n vcs-svn: set up channel to read fast-import cat-blob response\n vcs-svn: quote paths correctly for ls command\n vcs-svn: use mark from previous import for parent commit\n\nJonathan Nieder (9):\n vcs-svn: use higher mark numbers for blobs\n vcs-svn: save marks for imported commits\n vcs-svn: introduce repo_read_path to check the content at a path\n vcs-svn: handle_node: use repo_read_path\n vcs-svn: simplify repo_modify_path and repo_copy\n vcs-svn: add a comment before each commit\n vcs-svn: allow input errors to be detected promptly\n vcs-svn: eliminate repo_tree structure\n vcs-svn: handle filenames with dq correctly\n\n contrib/svn-fe/svn-fe.txt | 6 +-\n t/t9010-svn-fe.sh | 217 +++++++++++++++++++------\n vcs-svn/fast_export.c | 145 +++++++++++++++--\n vcs-svn/fast_export.h | 39 +++--\n vcs-svn/line_buffer.c | 5 +\n vcs-svn/line_buffer.h | 1 +\n vcs-svn/repo_tree.c | 386 ++++++++-------------------------------------\n vcs-svn/repo_tree.h | 5 +-\n vcs-svn/string_pool.c | 13 ++-\n vcs-svn/string_pool.h | 3 +-\n vcs-svn/svndump.c | 106 +++++++++----\n 11 files changed, 490 insertions(+), 436 deletions(-)\n rewrite vcs-svn/fast_export.h (75%)\n rewrite vcs-svn/repo_tree.c (96%)\n"},{"id":"162875","messageId":"20110306230348.GC24327@elie","threadId":"26014","inReplyTo":"20110306225419.GA24327@elie","subject":"[PATCH 01/12] vcs-svn: use higher mark numbers for blobs","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-03-06T23:03:48Z","receivedAt":"2011-03-06T23:03:48Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Date: Fri, 10 Dec 2010 04:21:35 -0600\n\nPrepare to use mark :5 for the commit corresponding to r5 (and so on).\n\n1 billion seems sufficiently high for blob marks to avoid conflicting\nwith rev marks, while still leaving room for 3 billion blobs. Such\nhigh mark numbers cause trouble with ancient fast-import versions, but\nthis topic cannot support git fast-import versions before 1.7.4 (which\nintroduces the cat-blob command) anyway.\n\nSigned-off-by: Jonathan Nieder \n---\nUnchanged except for a tiny tweak in the working of the change\ndescription.\n\n vcs-svn/repo_tree.c | 2 +-\n 1 files changed, 1 insertions(+), 1 deletions(-)\n\ndiff --git a/vcs-svn/repo_tree.c b/vcs-svn/repo_tree.c\nindex 491f013..093c5ff 100644\n--- a/vcs-svn/repo_tree.c\n+++ b/vcs-svn/repo_tree.c\n@@ -289,7 +289,7 @@ void repo_commit(uint32_t revision, uint32_t author, char *log, uint32_t uuid,\n static void mark_init(void)\n {\n \tuint32_t i;\n-\tmark = 0;\n+\tmark = 1024 * 1024 * 1024;\n \tfor (i = 0; i < dent_pool.size; i++)\n \t\tif (!repo_dirent_is_dir(dent_pointer(i)) &&\n \t\t dent_pointer(i)->content_offset > mark)\n-- \n1.7.4.1\n"},{"id":"162876","messageId":"20110306230423.GD24327@elie","threadId":"26014","inReplyTo":"20110306225419.GA24327@elie","subject":"[PATCH 02/12] vcs-svn: save marks for imported commits","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-03-06T23:04:23Z","receivedAt":"2011-03-06T23:04:23Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Date: Thu, 9 Dec 2010 18:57:13 -0600\n\nThis way, a person can use\n\n\tsvnadmin dump $path |\n\tsvn-fe |\n\tgit fast-import --relative-marks --export-marks=svn-revs\n\nto get a list of what commit corresponds to each svn revision (plus\nsome irrelevant blob names) in .git/info/fast-import/svn-revs.\n\nSigned-off-by: Jonathan Nieder \nSigned-off-by: David Barr \nSigned-off-by: Jonathan Nieder \n---\nFrom David's tree.\n\n vcs-svn/fast_export.c | 1 +\n 1 files changed, 1 insertions(+), 0 deletions(-)\n\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex 260cf50..932824a 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -45,6 +45,7 @@ void fast_export_commit(uint32_t revision, uint32_t author, char *log,\n \t\t*gitsvnline = '\\0';\n \t}\n \tprintf(\"commit refs/heads/master\\n\");\n+\tprintf(\"mark :%\"PRIu32\"\\n\", revision);\n \tprintf(\"committer %s <%s@%s> %ld +0000\\n\",\n \t\t ~author ? pool_fetch(author) : \"nobody\",\n \t\t ~author ? pool_fetch(author) : \"nobody\",\n-- \n1.7.4.1\n"},{"id":"162877","messageId":"20110306230703.GE24327@elie","threadId":"26014","inReplyTo":"20110306225419.GA24327@elie","subject":"[PATCH 03/12] vcs-svn: introduce repo_read_path to check the content at a path","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-03-06T23:07:03Z","receivedAt":"2011-03-06T23:07:03Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Date: Sat, 20 Nov 2010 13:25:28 -0600\n\nThe repo_tree structure remembers, for each path in each revision, a\nmode (regular file, executable, symlink, or directory) and content\n(blob mark or directory structure). Maintaining a second copy of all\nthis information when it's already in the target repository is\nwasteful, it does not persist between svn-fe invocations, and most\nimportantly, there is no convenient way to transfer it from one\nmachine to another. So it would be nice to get rid of it.\n\nAs a first step, let's change the repo_tree API to match fast-import's\nread commands more closely. Currently to read the mode for a path,\none uses\n\n\trepo_modify_path(path, new_mode, new_content);\n\nwhich changes the mode and content as a side effect. There is no\nfunction to read the content at a path; add one.\n\nSigned-off-by: Jonathan Nieder \nSigned-off-by: David Barr \nSigned-off-by: Jonathan Nieder \n---\n vcs-svn/repo_tree.c | 12 +++++++++++-\n vcs-svn/repo_tree.h | 1 +\n 2 files changed, 12 insertions(+), 1 deletions(-)\n\ndiff --git a/vcs-svn/repo_tree.c b/vcs-svn/repo_tree.c\nindex 093c5ff..23a9371 100644\n--- a/vcs-svn/repo_tree.c\n+++ b/vcs-svn/repo_tree.c\n@@ -87,7 +87,8 @@ static struct repo_dir *repo_clone_dir(struct repo_dir *orig_dir)\n \treturn dir_pointer(new_o);\n }\n \n-static struct repo_dirent *repo_read_dirent(uint32_t revision, uint32_t *path)\n+static struct repo_dirent *repo_read_dirent(uint32_t revision,\n+\t\t\t\t\t const uint32_t *path)\n {\n \tuint32_t name = 0;\n \tstruct repo_dirent *key = dent_pointer(dent_alloc(1));\n@@ -157,6 +158,15 @@ static void repo_write_dirent(uint32_t *path, uint32_t mode,\n \t\tdent_remove(&dir_pointer(parent_dir_o)->entries, dent);\n }\n \n+uint32_t repo_read_path(const uint32_t *path)\n+{\n+\tuint32_t content_offset = 0;\n+\tstruct repo_dirent *dent = repo_read_dirent(active_commit, path);\n+\tif (dent != NULL)\n+\t\tcontent_offset = dent->content_offset;\n+\treturn content_offset;\n+}\n+\n uint32_t repo_copy(uint32_t revision, uint32_t *src, uint32_t *dst)\n {\n \tuint32_t mode = 0, content_offset = 0;\ndiff --git a/vcs-svn/repo_tree.h b/vcs-svn/repo_tree.h\nindex 68baeb5..3202bbe 100644\n--- a/vcs-svn/repo_tree.h\n+++ b/vcs-svn/repo_tree.h\n@@ -15,6 +15,7 @@ uint32_t next_blob_mark(void);\n uint32_t repo_copy(uint32_t revision, uint32_t *src, uint32_t *dst);\n void repo_add(uint32_t *path, uint32_t mode, uint32_t blob_mark);\n uint32_t repo_modify_path(uint32_t *path, uint32_t mode, uint32_t blob_mark);\n+uint32_t repo_read_path(const uint32_t *path);\n void repo_delete(uint32_t *path);\n void repo_commit(uint32_t revision, uint32_t author, char *log, uint32_t uuid,\n \t\t uint32_t url, long unsigned timestamp);\n-- \n1.7.4.1\n"},{"id":"162878","messageId":"20110306230851.GF24327@elie","threadId":"26014","inReplyTo":"20110306225419.GA24327@elie","subject":"[PATCH 04/12] vcs-svn: handle_node: use repo_read_path","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-03-06T23:08:51Z","receivedAt":"2011-03-06T23:08:51Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Date: Fri, 10 Dec 2010 04:28:06 -0600\n\nsvn-fe processes each commit in two stages: first decide on the\ncorrect content for all paths and export the relevant blobs, then\nexport a commit with the result.\n\nWe can keep less state and simplify svn-fe a great deal by doing\nexporting the commit in one stage: use 'inline' blobs for each path\nand remember nothing. This way, the repo_tree structure could be\neliminated, and we would get support for incremental imports 'for\nfree'.\n\nReorganize handle_node() along these lines. This is just a code\ncleanup; the functional changes to repo_tree and handle_revision\nwill come later.\n\n[db: backported to apply without text delta support]\n\nSigned-off-by: Jonathan Nieder \nSigned-off-by: David Barr \nSigned-off-by: Jonathan Nieder \n---\n vcs-svn/svndump.c | 33 +++++++++++++++++++++++----------\n 1 files changed, 23 insertions(+), 10 deletions(-)\n\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex ee7c0bb..f07376f 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -201,13 +201,14 @@ static void handle_node(void)\n \tuint32_t mark = 0;\n \tconst uint32_t type = node_ctx.type;\n \tconst int have_props = node_ctx.propLength != LENGTH_UNKNOWN;\n+\tconst int have_text = node_ctx.textLength != LENGTH_UNKNOWN;\n \n \tif (node_ctx.text_delta)\n \t\tdie(\"text deltas not supported\");\n-\tif (node_ctx.textLength != LENGTH_UNKNOWN)\n+\tif (have_text)\n \t\tmark = next_blob_mark();\n \tif (node_ctx.action == NODEACT_DELETE) {\n-\t\tif (mark || have_props || node_ctx.srcRev)\n+\t\tif (have_text || have_props || node_ctx.srcRev)\n \t\t\tdie(\"invalid dump: deletion node has \"\n \t\t\t\t\"copyfrom info, text, or properties\");\n \t\treturn repo_delete(node_ctx.dst);\n@@ -221,13 +222,20 @@ static void handle_node(void)\n \t\tif (node_ctx.action == NODEACT_ADD)\n \t\t\tnode_ctx.action = NODEACT_CHANGE;\n \t}\n-\tif (mark && type == REPO_MODE_DIR)\n+\tif (have_text && type == REPO_MODE_DIR)\n \t\tdie(\"invalid dump: directories cannot have text attached\");\n+\n+\t/*\n+\t * Decide on the new content (mark) and mode (node_ctx.type).\n+\t */\n \tif (node_ctx.action == NODEACT_CHANGE && !~*node_ctx.dst) {\n \t\tif (type != REPO_MODE_DIR)\n \t\t\tdie(\"invalid dump: root of tree is not a regular file\");\n \t} else if (node_ctx.action == NODEACT_CHANGE) {\n-\t\tuint32_t mode = repo_modify_path(node_ctx.dst, 0, mark);\n+\t\tuint32_t mode;\n+\t\tif (!have_text)\n+\t\t\tmark = repo_read_path(node_ctx.dst);\n+\t\tmode = repo_modify_path(node_ctx.dst, 0, 0);\n \t\tif (!mode)\n \t\t\tdie(\"invalid dump: path to be modified is missing\");\n \t\tif (mode == REPO_MODE_DIR && type != REPO_MODE_DIR)\n@@ -236,22 +244,27 @@ static void handle_node(void)\n \t\t\tdie(\"invalid dump: cannot modify a file into a directory\");\n \t\tnode_ctx.type = mode;\n \t} else if (node_ctx.action == NODEACT_ADD) {\n-\t\tif (!mark && type != REPO_MODE_DIR)\n+\t\tif (!have_text && type != REPO_MODE_DIR)\n \t\t\tdie(\"invalid dump: adds node without text\");\n-\t\trepo_add(node_ctx.dst, type, mark);\n \t} else {\n \t\tdie(\"invalid dump: Node-path block lacks Node-action\");\n \t}\n+\n+\t/*\n+\t * Adjust mode to reflect properties.\n+\t */\n \tif (have_props) {\n-\t\tconst uint32_t old_mode = node_ctx.type;\n \t\tif (!node_ctx.prop_delta)\n \t\t\tnode_ctx.type = type;\n \t\tif (node_ctx.propLength)\n \t\t\tread_props();\n-\t\tif (node_ctx.type != old_mode)\n-\t\t\trepo_modify_path(node_ctx.dst, node_ctx.type, mark);\n \t}\n-\tif (mark)\n+\n+\t/*\n+\t * Save the result.\n+\t */\n+\trepo_add(node_ctx.dst, node_ctx.type, mark);\n+\tif (have_text)\n \t\tfast_export_blob(node_ctx.type, mark,\n \t\t\t\t node_ctx.textLength, &input);\n }\n-- \n1.7.4.1\n"},{"id":"162879","messageId":"20110306230923.GG24327@elie","threadId":"26014","inReplyTo":"20110306225419.GA24327@elie","subject":"[PATCH 05/12] vcs-svn: simplify repo_modify_path and repo_copy","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-03-06T23:09:23Z","receivedAt":"2011-03-06T23:09:23Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Date: Fri, 10 Dec 2010 00:53:54 -0600\n\nRestrict the repo_tree API to functions that are actually needed.\n\n - decouple reading the mode and content of dirents from other\n operations.\n - remove repo_modify_path. It is only used to read the mode from\n dirents.\n - remove the ability to use repo_read_mode on a missing path. The\n existing code only errors out in that case, anyway.\n\nSigned-off-by: Jonathan Nieder \nSigned-off-by: David Barr \nSigned-off-by: Jonathan Nieder \n---\n vcs-svn/repo_tree.c | 27 ++++++++++-----------------\n vcs-svn/repo_tree.h | 4 ++--\n vcs-svn/svndump.c | 4 +---\n 3 files changed, 13 insertions(+), 22 deletions(-)\n\ndiff --git a/vcs-svn/repo_tree.c b/vcs-svn/repo_tree.c\nindex 23a9371..036a686 100644\n--- a/vcs-svn/repo_tree.c\n+++ b/vcs-svn/repo_tree.c\n@@ -106,7 +106,7 @@ static struct repo_dirent *repo_read_dirent(uint32_t revision,\n \treturn dent;\n }\n \n-static void repo_write_dirent(uint32_t *path, uint32_t mode,\n+static void repo_write_dirent(const uint32_t *path, uint32_t mode,\n \t\t\t uint32_t content_offset, uint32_t del)\n {\n \tuint32_t name, revision, dir_o = ~0, parent_dir_o = ~0;\n@@ -167,7 +167,15 @@ uint32_t repo_read_path(const uint32_t *path)\n \treturn content_offset;\n }\n \n-uint32_t repo_copy(uint32_t revision, uint32_t *src, uint32_t *dst)\n+uint32_t repo_read_mode(const uint32_t *path)\n+{\n+\tstruct repo_dirent *dent = repo_read_dirent(active_commit, path);\n+\tif (dent == NULL)\n+\t\tdie(\"invalid dump: path to be modified is missing\");\n+\treturn dent->mode;\n+}\n+\n+void repo_copy(uint32_t revision, const uint32_t *src, const uint32_t *dst)\n {\n \tuint32_t mode = 0, content_offset = 0;\n \tstruct repo_dirent *src_dent;\n@@ -177,7 +185,6 @@ uint32_t repo_copy(uint32_t revision, uint32_t *src, uint32_t *dst)\n \t\tcontent_offset = src_dent->content_offset;\n \t\trepo_write_dirent(dst, mode, content_offset, 0);\n \t}\n-\treturn mode;\n }\n \n void repo_add(uint32_t *path, uint32_t mode, uint32_t blob_mark)\n@@ -185,20 +192,6 @@ void repo_add(uint32_t *path, uint32_t mode, uint32_t blob_mark)\n \trepo_write_dirent(path, mode, blob_mark, 0);\n }\n \n-uint32_t repo_modify_path(uint32_t *path, uint32_t mode, uint32_t blob_mark)\n-{\n-\tstruct repo_dirent *src_dent;\n-\tsrc_dent = repo_read_dirent(active_commit, path);\n-\tif (!src_dent)\n-\t\treturn 0;\n-\tif (!blob_mark)\n-\t\tblob_mark = src_dent->content_offset;\n-\tif (!mode)\n-\t\tmode = src_dent->mode;\n-\trepo_write_dirent(path, mode, blob_mark, 0);\n-\treturn mode;\n-}\n-\n void repo_delete(uint32_t *path)\n {\n \trepo_write_dirent(path, 0, 0, 1);\ndiff --git a/vcs-svn/repo_tree.h b/vcs-svn/repo_tree.h\nindex 3202bbe..11d48c2 100644\n--- a/vcs-svn/repo_tree.h\n+++ b/vcs-svn/repo_tree.h\n@@ -12,10 +12,10 @@\n #define REPO_MAX_PATH_DEPTH 1000\n \n uint32_t next_blob_mark(void);\n-uint32_t repo_copy(uint32_t revision, uint32_t *src, uint32_t *dst);\n+void repo_copy(uint32_t revision, const uint32_t *src, const uint32_t *dst);\n void repo_add(uint32_t *path, uint32_t mode, uint32_t blob_mark);\n-uint32_t repo_modify_path(uint32_t *path, uint32_t mode, uint32_t blob_mark);\n uint32_t repo_read_path(const uint32_t *path);\n+uint32_t repo_read_mode(const uint32_t *path);\n void repo_delete(uint32_t *path);\n void repo_commit(uint32_t revision, uint32_t author, char *log, uint32_t uuid,\n \t\t uint32_t url, long unsigned timestamp);\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex f07376f..e6d84ba 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -235,9 +235,7 @@ static void handle_node(void)\n \t\tuint32_t mode;\n \t\tif (!have_text)\n \t\t\tmark = repo_read_path(node_ctx.dst);\n-\t\tmode = repo_modify_path(node_ctx.dst, 0, 0);\n-\t\tif (!mode)\n-\t\t\tdie(\"invalid dump: path to be modified is missing\");\n+\t\tmode = repo_read_mode(node_ctx.dst);\n \t\tif (mode == REPO_MODE_DIR && type != REPO_MODE_DIR)\n \t\t\tdie(\"invalid dump: cannot modify a directory into a file\");\n \t\tif (mode != REPO_MODE_DIR && type == REPO_MODE_DIR)\n-- \n1.7.4.1\n"},{"id":"162880","messageId":"20110306230951.GH24327@elie","threadId":"26014","inReplyTo":"20110306225419.GA24327@elie","subject":"[PATCH 06/12] vcs-svn: add a comment before each commit","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-03-06T23:09:52Z","receivedAt":"2011-03-06T23:09:52Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Date: Tue, 4 Jan 2011 21:53:33 -0600\n\nCurrent svn-fe produces output like this:\n\n\tblob\n\tmark :7382321\n\tdata 5\n\thello\n\n\tblob\n\tmark :7382322\n\tdata 5\n\tHello\n\n\tcommit\n\tmark :3\n[...]\n\tM 100644 :7382321 hello.c\n\tM 100644 :7382322 hello2.c\n\nThis means svn-fe has to keep track of the paths modified in each\ncommit and the corresponding marks, instead of dealing with each file\nas it arrives in input and then forgetting about it. A better\nstrategy would be to use inline blobs:\n\n\tcommit\n\tmark :3\n[...]\n\tM 100644 inline hello.c\n\tdata 5\n\thello\n[...]\n\nAs a first step towards that, teach svn-fe to notice when the\ncollection of blobs for each commit starts and write a comment\n(\"# commit 3.\") there.\n\nSigned-off-by: Jonathan Nieder \n---\n vcs-svn/fast_export.c | 5 +++++\n vcs-svn/fast_export.h | 1 +\n vcs-svn/svndump.c | 29 ++++++++++++++++++++++-------\n 3 files changed, 28 insertions(+), 7 deletions(-)\n\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex 932824a..5a105ad 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -30,6 +30,11 @@ void fast_export_modify(uint32_t depth, uint32_t *path, uint32_t mode,\n \tputchar('\\n');\n }\n \n+void fast_export_begin_commit(uint32_t revision)\n+{\n+\tprintf(\"# commit %\"PRIu32\".\\n\", revision);\n+}\n+\n static char gitsvnline[MAX_GITSVN_LINE_LEN];\n void fast_export_commit(uint32_t revision, uint32_t author, char *log,\n \t\t\tuint32_t uuid, uint32_t url,\ndiff --git a/vcs-svn/fast_export.h b/vcs-svn/fast_export.h\nindex 054e7d5..aff8005 100644\n--- a/vcs-svn/fast_export.h\n+++ b/vcs-svn/fast_export.h\n@@ -6,6 +6,7 @@\n void fast_export_delete(uint32_t depth, uint32_t *path);\n void fast_export_modify(uint32_t depth, uint32_t *path, uint32_t mode,\n \t\t\tuint32_t mark);\n+void fast_export_begin_commit(uint32_t revision);\n void fast_export_commit(uint32_t revision, uint32_t author, char *log,\n \t\t\tuint32_t uuid, uint32_t url, unsigned long timestamp);\n void fast_export_blob(uint32_t mode, uint32_t mark, uint32_t len,\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex e6d84ba..a384996 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -20,9 +20,11 @@\n #define NODEACT_CHANGE 1\n #define NODEACT_UNKNOWN 0\n \n-#define DUMP_CTX 0\n-#define REV_CTX 1\n-#define NODE_CTX 2\n+/* States: */\n+#define DUMP_CTX 0\t/* dump metadata */\n+#define REV_CTX 1\t/* revision metadata */\n+#define NODE_CTX 2\t/* node metadata */\n+#define INTERNODE_CTX 3\t/* between nodes */\n \n #define LENGTH_UNKNOWN (~0)\n #define DATE_RFC2822_LEN 31\n@@ -267,7 +269,14 @@ static void handle_node(void)\n \t\t\t\t node_ctx.textLength, &input);\n }\n \n-static void handle_revision(void)\n+static void begin_revision(void)\n+{\n+\tif (!rev_ctx.revision)\t/* revision 0 gets no git commit. */\n+\t\treturn;\n+\tfast_export_begin_commit(rev_ctx.revision);\n+}\n+\n+static void end_revision(void)\n {\n \tif (rev_ctx.revision)\n \t\trepo_commit(rev_ctx.revision, rev_ctx.author, rev_ctx.log,\n@@ -301,13 +310,17 @@ void svndump_read(const char *url)\n \t\t} else if (key == keys.revision_number) {\n \t\t\tif (active_ctx == NODE_CTX)\n \t\t\t\thandle_node();\n+\t\t\tif (active_ctx == REV_CTX)\n+\t\t\t\tbegin_revision();\n \t\t\tif (active_ctx != DUMP_CTX)\n-\t\t\t\thandle_revision();\n+\t\t\t\tend_revision();\n \t\t\tactive_ctx = REV_CTX;\n \t\t\treset_rev_ctx(atoi(val));\n \t\t} else if (key == keys.node_path) {\n \t\t\tif (active_ctx == NODE_CTX)\n \t\t\t\thandle_node();\n+\t\t\tif (active_ctx == REV_CTX)\n+\t\t\t\tbegin_revision();\n \t\t\tactive_ctx = NODE_CTX;\n \t\t\treset_node_ctx(val);\n \t\t} else if (key == keys.node_kind) {\n@@ -349,7 +362,7 @@ void svndump_read(const char *url)\n \t\t\t\tread_props();\n \t\t\t} else if (active_ctx == NODE_CTX) {\n \t\t\t\thandle_node();\n-\t\t\t\tactive_ctx = REV_CTX;\n+\t\t\t\tactive_ctx = INTERNODE_CTX;\n \t\t\t} else {\n \t\t\t\tfprintf(stderr, \"Unexpected content length header: %\"PRIu32\"\\n\", len);\n \t\t\t\tbuffer_skip_bytes(&input, len);\n@@ -358,8 +371,10 @@ void svndump_read(const char *url)\n \t}\n \tif (active_ctx == NODE_CTX)\n \t\thandle_node();\n+\tif (active_ctx == REV_CTX)\n+\t\tbegin_revision();\n \tif (active_ctx != DUMP_CTX)\n-\t\thandle_revision();\n+\t\tend_revision();\n }\n \n int svndump_init(const char *filename)\n-- \n1.7.4.1\n"},{"id":"162881","messageId":"20110306231054.GI24327@elie","threadId":"26014","inReplyTo":"20110306225419.GA24327@elie","subject":"[PATCH 07/12] vcs-svn: allow input errors to be detected promptly","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-03-06T23:10:54Z","receivedAt":"2011-03-06T23:10:54Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Date: Sun, 10 Oct 2010 21:51:21 -0500\n\nThe line_buffer library silently flags input errors until\nbuffer_deinit time; unfortunately, by that point usually errno is\ninvalid. Expose the error flag so callers can check for and\nreport errors early for easy debugging.\n\n\tsome_error_prone_operation(...);\n\tif (buffer_ferror(buf))\n\t\treturn error(\"input error: %s\", strerror(errno));\n\nSigned-off-by: Jonathan Nieder \nSigned-off-by: David Barr \nSigned-off-by: Jonathan Nieder \n---\n vcs-svn/line_buffer.c | 5 +++++\n vcs-svn/line_buffer.h | 1 +\n 2 files changed, 6 insertions(+), 0 deletions(-)\n\ndiff --git a/vcs-svn/line_buffer.c b/vcs-svn/line_buffer.c\nindex aedf105..eb8a6a7 100644\n--- a/vcs-svn/line_buffer.c\n+++ b/vcs-svn/line_buffer.c\n@@ -59,6 +59,11 @@ long buffer_tmpfile_prepare_to_read(struct line_buffer *buf)\n \treturn pos;\n }\n \n+int buffer_ferror(struct line_buffer *buf)\n+{\n+\treturn ferror(buf->infile);\n+}\n+\n int buffer_read_char(struct line_buffer *buf)\n {\n \treturn fgetc(buf->infile);\ndiff --git a/vcs-svn/line_buffer.h b/vcs-svn/line_buffer.h\nindex 96ce966..3c9629e 100644\n--- a/vcs-svn/line_buffer.h\n+++ b/vcs-svn/line_buffer.h\n@@ -21,6 +21,7 @@ int buffer_tmpfile_init(struct line_buffer *buf);\n FILE *buffer_tmpfile_rewind(struct line_buffer *buf);\t/* prepare to write. */\n long buffer_tmpfile_prepare_to_read(struct line_buffer *buf);\n \n+int buffer_ferror(struct line_buffer *buf);\n char *buffer_read_line(struct line_buffer *buf);\n char *buffer_read_string(struct line_buffer *buf, uint32_t len);\n int buffer_read_char(struct line_buffer *buf);\n-- \n1.7.4.1\n"},{"id":"162882","messageId":"20110306231130.GJ24327@elie","threadId":"26014","inReplyTo":"20110306225419.GA24327@elie","subject":"[PATCH 08/12] vcs-svn: set up channel to read fast-import cat-blob response","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-03-06T23:11:30Z","receivedAt":"2011-03-06T23:11:30Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"From: David Barr \nDate: Sat, 5 Mar 2011 13:30:23 +1100\n\nSet up some plumbing: teach the svndump lib to pass a file descriptor\nnumber to the fast_export lib, representing where cat-blob/ls\nresponses can be read from, and add a get_response_line helper\nfunction to the fast_export lib to read a line from that file.\n\nUnfortunately this means that svn-fe needs file descriptor 3 to be\nredirected from somewhere (preferrably the cat-blob stream of a\nfast-import backend); otherwise it will fail:\n\n\t$ svndump | svn-fe\n\tfatal: cannot read from file descriptor 3: Bad file descriptor\n\nFor the moment, \"svn-fe 3\nSigned-off-by: David Barr \nSigned-off-by: Jonathan Nieder \n---\n contrib/svn-fe/svn-fe.txt | 6 ++-\n t/t9010-svn-fe.sh | 118 +++++++++++++++++++++++++-------------------\n vcs-svn/fast_export.c | 28 +++++++++++\n vcs-svn/fast_export.h | 4 ++\n vcs-svn/svndump.c | 5 ++\n 5 files changed, 109 insertions(+), 52 deletions(-)\n\ndiff --git a/contrib/svn-fe/svn-fe.txt b/contrib/svn-fe/svn-fe.txt\nindex cd075b9..85f7b83 100644\n--- a/contrib/svn-fe/svn-fe.txt\n+++ b/contrib/svn-fe/svn-fe.txt\n@@ -7,7 +7,11 @@ svn-fe - convert an SVN \"dumpfile\" to a fast-import stream\n \n SYNOPSIS\n --------\n-svnadmin dump --incremental REPO | svn-fe [url] | git fast-import\n+[verse]\n+mkfifo backchannel &&\n+svnadmin dump --incremental REPO |\n+\tsvn-fe [url] 3backchannel\n \n DESCRIPTION\n -----------\ndiff --git a/t/t9010-svn-fe.sh b/t/t9010-svn-fe.sh\nindex 5a6a4b9..2ae5374 100755\n--- a/t/t9010-svn-fe.sh\n+++ b/t/t9010-svn-fe.sh\n@@ -5,8 +5,26 @@ test_description='check svn dumpfile importer'\n . ./test-lib.sh\n \n reinit_git () {\n+\tif ! test_declared_prereq PIPE\n+\tthen\n+\t\techo >&4 \"reinit_git: need to declare PIPE prerequisite\"\n+\t\treturn 127\n+\tfi\n \trm -fr .git &&\n-\tgit init\n+\trm -f stream backflow &&\n+\tgit init &&\n+\tmkfifo stream backflow\n+}\n+\n+try_dump () {\n+\tinput=$1 &&\n+\tmaybe_fail=${2:+test_$2} &&\n+\n+\t{\n+\t\t$maybe_fail test-svn-fe \"$input\" >stream 3backflow &&\n+\twait $!\n }\n \n properties () {\n@@ -35,21 +53,27 @@ text_no_props () {\n \n >empty\n \n-test_expect_success 'empty dump' '\n+test_expect_success 'setup: have pipes?' '\n+\trm -f frob &&\n+\tif mkfifo frob\n+\tthen\n+\t\ttest_set_prereq PIPE\n+\tfi\n+'\n+\n+test_expect_success PIPE 'empty dump' '\n \treinit_git &&\n \techo \"SVN-fs-dump-format-version: 2\" >input &&\n-\ttest-svn-fe input >stream &&\n-\tgit fast-import v4.dump &&\n-\ttest_must_fail test-svn-fe v4.dump >stream &&\n-\ttest_cmp empty stream\n+\ttry_dump v4.dump must_fail\n '\n \n-test_expect_failure 'empty revision' '\n+test_expect_failure PIPE 'empty revision' '\n \treinit_git &&\n \tprintf \"rev : %s\\n\" \"\" \"\" >expect &&\n \tcat >emptyrev.dump <<-\\EOF &&\n@@ -64,13 +88,12 @@ test_expect_failure 'empty revision' '\n \tContent-length: 0\n \n \tEOF\n-\ttest-svn-fe emptyrev.dump >stream &&\n-\tgit fast-import : %s\" HEAD >actual &&\n \ttest_cmp expect actual\n '\n \n-test_expect_success 'empty properties' '\n+test_expect_success PIPE 'empty properties' '\n \treinit_git &&\n \tprintf \"rev : %s\\n\" \"\" \"\" >expect &&\n \tcat >emptyprop.dump <<-\\EOF &&\n@@ -88,13 +111,12 @@ test_expect_success 'empty properties' '\n \n \tPROPS-END\n \tEOF\n-\ttest-svn-fe emptyprop.dump >stream &&\n-\tgit fast-import : %s\" HEAD >actual &&\n \ttest_cmp expect actual\n '\n \n-test_expect_success 'author name and commit message' '\n+test_expect_success PIPE 'author name and commit message' '\n \treinit_git &&\n \techo \"\" >expect.author &&\n \tcat >message <<-\\EOF &&\n@@ -121,15 +143,14 @@ test_expect_success 'author name and commit message' '\n \t\techo &&\n \t\tcat props\n \t} >log.dump &&\n-\ttest-svn-fe log.dump >stream &&\n-\tgit fast-import actual.log &&\n \tgit log --format=\"<%an, %ae>\" >actual.author &&\n \ttest_cmp message actual.log &&\n \ttest_cmp expect.author actual.author\n '\n \n-test_expect_success 'unsupported properties are ignored' '\n+test_expect_success PIPE 'unsupported properties are ignored' '\n \treinit_git &&\n \techo author >expect &&\n \tcat >extraprop.dump <<-\\EOF &&\n@@ -149,13 +170,12 @@ test_expect_success 'unsupported properties are ignored' '\n \tauthor\n \tPROPS-END\n \tEOF\n-\ttest-svn-fe extraprop.dump >stream &&\n-\tgit fast-import actual &&\n \ttest_cmp expect actual\n '\n \n-test_expect_failure 'timestamp and empty file' '\n+test_expect_failure PIPE 'timestamp and empty file' '\n \techo author@example.com >expect.author &&\n \techo 1999-01-01 >expect.date &&\n \techo file >expect.files &&\n@@ -186,8 +206,7 @@ test_expect_failure 'timestamp and empty file' '\n \n \t\tEOF\n \t} >emptyfile.dump &&\n-\ttest-svn-fe emptyfile.dump >stream &&\n-\tgit fast-import actual.author &&\n \tgit log --date=short --format=%ad HEAD >actual.date &&\n \tgit ls-tree -r --name-only HEAD >actual.files &&\n@@ -198,7 +217,7 @@ test_expect_failure 'timestamp and empty file' '\n \ttest_cmp empty file\n '\n \n-test_expect_success 'directory with files' '\n+test_expect_success PIPE 'directory with files' '\n \treinit_git &&\n \tprintf \"%s\\n\" directory/file1 directory/file2 >expect.files &&\n \techo hi >hi &&\n@@ -242,8 +261,7 @@ test_expect_success 'directory with files' '\n \t\tEOF\n \t\ttext_no_props hi\n \t} >directory.dump &&\n-\ttest-svn-fe directory.dump >stream &&\n-\tgit fast-import actual.files &&\n \tgit checkout HEAD directory &&\n@@ -252,7 +270,8 @@ test_expect_success 'directory with files' '\n \ttest_cmp hi directory/file2\n '\n \n-test_expect_success 'node without action' '\n+test_expect_success PIPE 'node without action' '\n+\treinit_git &&\n \tcat >inaction.dump <<-\\EOF &&\n \tSVN-fs-dump-format-version: 3\n \n@@ -269,10 +288,11 @@ test_expect_success 'node without action' '\n \n \tPROPS-END\n \tEOF\n-\ttest_must_fail test-svn-fe inaction.dump\n+\ttry_dump inaction.dump must_fail\n '\n \n-test_expect_success 'action: add node without text' '\n+test_expect_success PIPE 'action: add node without text' '\n+\treinit_git &&\n \tcat >textless.dump <<-\\EOF &&\n \tSVN-fs-dump-format-version: 3\n \n@@ -290,10 +310,10 @@ test_expect_success 'action: add node without text' '\n \n \tPROPS-END\n \tEOF\n-\ttest_must_fail test-svn-fe textless.dump\n+\ttry_dump textless.dump must_fail\n '\n \n-test_expect_failure 'change file mode but keep old content' '\n+test_expect_failure PIPE 'change file mode but keep old content' '\n \treinit_git &&\n \tcat >expect <<-\\EOF &&\n \tOBJID\n@@ -356,8 +376,7 @@ test_expect_failure 'change file mode but keep old content' '\n \n \tPROPS-END\n \tEOF\n-\ttest-svn-fe filemode.dump >stream &&\n-\tgit fast-import expect <<-\\EOF &&\n \tOBJID\n@@ -382,7 +401,7 @@ test_expect_success 'change file mode and reiterate content' '\n \tEOF\n \techo \"link hello\" >expect.blob &&\n \techo hello >hello &&\n-\tcat >filemode.dump <<-\\EOF &&\n+\tcat >filemode2.dump <<-\\EOF &&\n \tSVN-fs-dump-format-version: 3\n \n \tRevision-number: 1\n@@ -437,8 +456,7 @@ test_expect_success 'change file mode and reiterate content' '\n \tPROPS-END\n \tlink hello\n \tEOF\n-\ttest-svn-fe filemode.dump >stream &&\n-\tgit fast-import delta.dump &&\n-\ttest_must_fail test-svn-fe delta.dump\n+\ttest_must_fail try_dump delta.dump\n '\n \n-test_expect_success 'property deltas supported' '\n+test_expect_success PIPE 'property deltas supported' '\n \treinit_git &&\n \tcat >expect <<-\\EOF &&\n \tOBJID\n@@ -570,8 +589,7 @@ test_expect_success 'property deltas supported' '\n \t\tPROPS-END\n \t\tEOF\n \t} >propdelta.dump &&\n-\ttest-svn-fe propdelta.dump >stream &&\n-\tgit fast-import expect &&\n \tOBJID\n@@ -625,8 +643,7 @@ test_expect_success 'properties on /' '\n \n \tPROPS-END\n \tEOF\n-\ttest-svn-fe changeroot.dump >stream &&\n-\tgit fast-import expect <<-\\EOF &&\n \tOBJID\n@@ -711,8 +728,7 @@ test_expect_success 'deltas for typechange' '\n \tPROPS-END\n \tlink testing 321\n \tEOF\n-\ttest-svn-fe deleteprop.dump >stream &&\n-\tgit fast-import simple.fe &&\n+test_expect_success SVNREPO,PIPE 't9135/svn.dump' '\n+\tmkdir -p simple-git &&\n \t(\n \t\tcd simple-git &&\n-\t\tgit fast-import <../simple.fe\n+\t\treinit_git &&\n+\t\ttry_dump \"$TEST_DIRECTORY/t9135/svn.dump\"\n \t) &&\n \t(\n \t\tcd simple-svnco &&\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex 5a105ad..8786ed2 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -12,6 +12,24 @@\n #define MAX_GITSVN_LINE_LEN 4096\n \n static uint32_t first_commit_done;\n+static struct line_buffer report_buffer = LINE_BUFFER_INIT;\n+\n+void fast_export_init(int fd)\n+{\n+\tif (buffer_fdinit(&report_buffer, fd))\n+\t\tdie_errno(\"cannot read from file descriptor %d\", fd);\n+}\n+\n+void fast_export_deinit(void)\n+{\n+\tif (buffer_deinit(&report_buffer))\n+\t\tdie_errno(\"error closing fast-import feedback stream\");\n+}\n+\n+void fast_export_reset(void)\n+{\n+\tbuffer_reset(&report_buffer);\n+}\n \n void fast_export_delete(uint32_t depth, uint32_t *path)\n {\n@@ -69,6 +87,16 @@ void fast_export_commit(uint32_t revision, uint32_t author, char *log,\n \tprintf(\"progress Imported commit %\"PRIu32\".\\n\\n\", revision);\n }\n \n+static const char *get_response_line(void)\n+{\n+\tconst char *line = buffer_read_line(&report_buffer);\n+\tif (line)\n+\t\treturn line;\n+\tif (buffer_ferror(&report_buffer))\n+\t\tdie_errno(\"error reading from fast-import\");\n+\tdie(\"unexpected end of fast-import feedback\");\n+}\n+\n void fast_export_blob(uint32_t mode, uint32_t mark, uint32_t len, struct line_buffer *input)\n {\n \tif (mode == REPO_MODE_LNK) {\ndiff --git a/vcs-svn/fast_export.h b/vcs-svn/fast_export.h\nindex aff8005..09b2033 100644\n--- a/vcs-svn/fast_export.h\n+++ b/vcs-svn/fast_export.h\n@@ -3,6 +3,10 @@\n \n #include \"line_buffer.h\"\n \n+void fast_export_init(int fd);\n+void fast_export_deinit(void);\n+void fast_export_reset(void);\n+\n void fast_export_delete(uint32_t depth, uint32_t *path);\n void fast_export_modify(uint32_t depth, uint32_t *path, uint32_t mode,\n \t\t\tuint32_t mark);\ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex a384996..3cc4135 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -14,6 +14,8 @@\n #include \"obj_pool.h\"\n #include \"string_pool.h\"\n \n+#define REPORT_FILENO 3\n+\n #define NODEACT_REPLACE 4\n #define NODEACT_DELETE 3\n #define NODEACT_ADD 2\n@@ -382,6 +384,7 @@ int svndump_init(const char *filename)\n \tif (buffer_init(&input, filename))\n \t\treturn error(\"cannot open %s: %s\", filename, strerror(errno));\n \trepo_init();\n+\tfast_export_init(REPORT_FILENO);\n \treset_dump_ctx(~0);\n \treset_rev_ctx(0);\n \treset_node_ctx(NULL);\n@@ -392,6 +395,7 @@ int svndump_init(const char *filename)\n void svndump_deinit(void)\n {\n \tlog_reset();\n+\tfast_export_deinit();\n \trepo_reset();\n \treset_dump_ctx(~0);\n \treset_rev_ctx(0);\n@@ -405,6 +409,7 @@ void svndump_deinit(void)\n void svndump_reset(void)\n {\n \tlog_reset();\n+\tfast_export_reset();\n \tbuffer_reset(&input);\n \trepo_reset();\n \treset_dump_ctx(~0);\n-- \n1.7.4.1\n"},{"id":"162883","messageId":"20110306231212.GK24327@elie","threadId":"26014","inReplyTo":"20110306225419.GA24327@elie","subject":"[PATCH 09/12] vcs-svn: eliminate repo_tree structure","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-03-06T23:12:12Z","receivedAt":"2011-03-06T23:12:12Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Date: Fri, 10 Dec 2010 04:00:55 -0600\n\nRely on fast-import for information about previous revs.\n\nThis requires always setting up backward flow of information, even for\nv2 dumps. On the plus side, it simplifies the code by quite a bit and\nopens the door to further simplifications.\n\n[db: adjusted to support final version of the cat-blob patch]\n[jn: avoiding hard-coding git's name for the empty tree for\n portability to other backends]\n\nSigned-off-by: Jonathan Nieder \nSigned-off-by: David Barr \nSigned-off-by: Jonathan Nieder \n---\n vcs-svn/fast_export.c | 108 ++++++++++++--\n vcs-svn/fast_export.h | 44 +++---\n vcs-svn/repo_tree.c | 389 ++++++++-----------------------------------------\n vcs-svn/repo_tree.h | 2 +-\n vcs-svn/string_pool.c | 2 +-\n vcs-svn/string_pool.h | 2 +-\n vcs-svn/svndump.c | 53 +++++--\n 7 files changed, 222 insertions(+), 378 deletions(-)\n rewrite vcs-svn/fast_export.h (65%)\n rewrite vcs-svn/repo_tree.c (95%)\n\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex 8786ed2..a8ce5c6 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -8,6 +8,7 @@\n #include \"line_buffer.h\"\n #include \"repo_tree.h\"\n #include \"string_pool.h\"\n+#include \"strbuf.h\"\n \n #define MAX_GITSVN_LINE_LEN 4096\n \n@@ -31,7 +32,7 @@ void fast_export_reset(void)\n \tbuffer_reset(&report_buffer);\n }\n \n-void fast_export_delete(uint32_t depth, uint32_t *path)\n+void fast_export_delete(uint32_t depth, const uint32_t *path)\n {\n \tputchar('D');\n \tputchar(' ');\n@@ -39,22 +40,27 @@ void fast_export_delete(uint32_t depth, uint32_t *path)\n \tputchar('\\n');\n }\n \n-void fast_export_modify(uint32_t depth, uint32_t *path, uint32_t mode,\n-\t\t\tuint32_t mark)\n+static void fast_export_truncate(uint32_t depth, const uint32_t *path, uint32_t mode)\n+{\n+\tfast_export_modify(depth, path, mode, \"inline\");\n+\tprintf(\"data 0\\n\\n\");\n+}\n+\n+void fast_export_modify(uint32_t depth, const uint32_t *path, uint32_t mode,\n+\t\t\tconst char *dataref)\n {\n \t/* Mode must be 100644, 100755, 120000, or 160000. */\n-\tprintf(\"M %06\"PRIo32\" :%\"PRIu32\" \", mode, mark);\n+\tif (!dataref) {\n+\t\tfast_export_truncate(depth, path, mode);\n+\t\treturn;\n+\t}\n+\tprintf(\"M %06\"PRIo32\" %s \", mode, dataref);\n \tpool_print_seq(depth, path, '/', stdout);\n \tputchar('\\n');\n }\n \n-void fast_export_begin_commit(uint32_t revision)\n-{\n-\tprintf(\"# commit %\"PRIu32\".\\n\", revision);\n-}\n-\n static char gitsvnline[MAX_GITSVN_LINE_LEN];\n-void fast_export_commit(uint32_t revision, uint32_t author, char *log,\n+void fast_export_begin_commit(uint32_t revision, uint32_t author, char *log,\n \t\t\tuint32_t uuid, uint32_t url,\n \t\t\tunsigned long timestamp)\n {\n@@ -81,12 +87,31 @@ void fast_export_commit(uint32_t revision, uint32_t author, char *log,\n \t\t\tprintf(\"from refs/heads/master^0\\n\");\n \t\tfirst_commit_done = 1;\n \t}\n-\trepo_diff(revision - 1, revision);\n-\tfputc('\\n', stdout);\n+}\n \n+void fast_export_end_commit(uint32_t revision)\n+{\n \tprintf(\"progress Imported commit %\"PRIu32\".\\n\\n\", revision);\n }\n \n+static void ls_from_rev(uint32_t rev, uint32_t depth, const uint32_t *path)\n+{\n+\t/* ls :5 path/to/old/file */\n+\tprintf(\"ls :%\"PRIu32\" \", rev);\n+\tpool_print_seq(depth, path, '/', stdout);\n+\tputchar('\\n');\n+\tfflush(stdout);\n+}\n+\n+static void ls_from_active_commit(uint32_t depth, const uint32_t *path)\n+{\n+\t/* ls \"path/to/file\" */\n+\tprintf(\"ls \\\"\");\n+\tpool_print_seq(depth, path, '/', stdout);\n+\tprintf(\"\\\"\\n\");\n+\tfflush(stdout);\n+}\n+\n static const char *get_response_line(void)\n {\n \tconst char *line = buffer_read_line(&report_buffer);\n@@ -97,14 +122,69 @@ static const char *get_response_line(void)\n \tdie(\"unexpected end of fast-import feedback\");\n }\n \n-void fast_export_blob(uint32_t mode, uint32_t mark, uint32_t len, struct line_buffer *input)\n+void fast_export_data(uint32_t mode, uint32_t len, struct line_buffer *input)\n {\n \tif (mode == REPO_MODE_LNK) {\n \t\t/* svn symlink blobs start with \"link \" */\n \t\tbuffer_skip_bytes(input, 5);\n \t\tlen -= 5;\n \t}\n-\tprintf(\"blob\\nmark :%\"PRIu32\"\\ndata %\"PRIu32\"\\n\", mark, len);\n+\tprintf(\"data %\"PRIu32\"\\n\", len);\n \tbuffer_copy_bytes(input, len);\n \tfputc('\\n', stdout);\n }\n+\n+static int parse_ls_response(const char *response, uint32_t *mode,\n+\t\t\t\t\tstruct strbuf *dataref)\n+{\n+\tconst char *tab;\n+\tconst char *response_end;\n+\n+\tassert(response);\n+\tresponse_end = response + strlen(response);\n+\n+\tif (*response == 'm') {\t/* Missing. */\n+\t\terrno = ENOENT;\n+\t\treturn -1;\n+\t}\n+\n+\t/* Mode. */\n+\tif (response_end - response < strlen(\"100644\") ||\n+\t response[strlen(\"100644\")] != ' ')\n+\t\tdie(\"invalid ls response: missing mode: %s\", response);\n+\t*mode = 0;\n+\tfor (; *response != ' '; response++) {\n+\t\tchar ch = *response;\n+\t\tif (ch < '0' || ch > '7')\n+\t\t\tdie(\"invalid ls response: mode is not octal: %s\", response);\n+\t\t*mode *= 8;\n+\t\t*mode += ch - '0';\n+\t}\n+\n+\t/* ' blob ' or ' tree ' */\n+\tif (response_end - response < strlen(\" blob \") ||\n+\t (response[1] != 'b' && response[1] != 't'))\n+\t\tdie(\"unexpected ls response: not a tree or blob: %s\", response);\n+\tresponse += strlen(\" blob \");\n+\n+\t/* Dataref. */\n+\ttab = memchr(response, '\\t', response_end - response);\n+\tif (!tab)\n+\t\tdie(\"invalid ls response: missing tab: %s\", response);\n+\tstrbuf_add(dataref, response, tab - response);\n+\treturn 0;\n+}\n+\n+int fast_export_ls_rev(uint32_t rev, uint32_t depth, const uint32_t *path,\n+\t\t\t\tuint32_t *mode, struct strbuf *dataref)\n+{\n+\tls_from_rev(rev, depth, path);\n+\treturn parse_ls_response(get_response_line(), mode, dataref);\n+}\n+\n+int fast_export_ls(uint32_t depth, const uint32_t *path,\n+\t\t\t\tuint32_t *mode, struct strbuf *dataref)\n+{\n+\tls_from_active_commit(depth, path);\n+\treturn parse_ls_response(get_response_line(), mode, dataref);\n+}\ndiff --git a/vcs-svn/fast_export.h b/vcs-svn/fast_export.h\ndissimilarity index 65%\nindex 09b2033..633d219 100644\n--- a/vcs-svn/fast_export.h\n+++ b/vcs-svn/fast_export.h\n@@ -1,19 +1,25 @@\n-#ifndef FAST_EXPORT_H_\n-#define FAST_EXPORT_H_\n-\n-#include \"line_buffer.h\"\n-\n-void fast_export_init(int fd);\n-void fast_export_deinit(void);\n-void fast_export_reset(void);\n-\n-void fast_export_delete(uint32_t depth, uint32_t *path);\n-void fast_export_modify(uint32_t depth, uint32_t *path, uint32_t mode,\n-\t\t\tuint32_t mark);\n-void fast_export_begin_commit(uint32_t revision);\n-void fast_export_commit(uint32_t revision, uint32_t author, char *log,\n-\t\t\tuint32_t uuid, uint32_t url, unsigned long timestamp);\n-void fast_export_blob(uint32_t mode, uint32_t mark, uint32_t len,\n-\t\t struct line_buffer *input);\n-\n-#endif\n+#ifndef FAST_EXPORT_H_\n+#define FAST_EXPORT_H_\n+\n+struct strbuf;\n+struct line_buffer;\n+\n+void fast_export_init(int fd);\n+void fast_export_deinit(void);\n+void fast_export_reset(void);\n+\n+void fast_export_delete(uint32_t depth, const uint32_t *path);\n+void fast_export_modify(uint32_t depth, const uint32_t *path,\n+\t\t\tuint32_t mode, const char *dataref);\n+void fast_export_begin_commit(uint32_t revision, uint32_t author, char *log,\n+\t\t\tuint32_t uuid, uint32_t url, unsigned long timestamp);\n+void fast_export_end_commit(uint32_t revision);\n+void fast_export_data(uint32_t mode, uint32_t len, struct line_buffer *input);\n+\n+/* If there is no such file at that rev, returns -1, errno == ENOENT. */\n+int fast_export_ls_rev(uint32_t rev, uint32_t depth, const uint32_t *path,\n+\t\t\tuint32_t *mode_out, struct strbuf *dataref_out);\n+int fast_export_ls(uint32_t depth, const uint32_t *path,\n+\t\t\tuint32_t *mode_out, struct strbuf *dataref_out);\n+\n+#endif\ndiff --git a/vcs-svn/repo_tree.c b/vcs-svn/repo_tree.c\ndissimilarity index 95%\nindex 036a686..e75f580 100644\n--- a/vcs-svn/repo_tree.c\n+++ b/vcs-svn/repo_tree.c\n@@ -1,325 +1,64 @@\n-/*\n- * Licensed under a two-clause BSD-style license.\n- * See LICENSE for details.\n- */\n-\n-#include \"git-compat-util.h\"\n-\n-#include \"string_pool.h\"\n-#include \"repo_tree.h\"\n-#include \"obj_pool.h\"\n-#include \"fast_export.h\"\n-\n-#include \"trp.h\"\n-\n-struct repo_dirent {\n-\tuint32_t name_offset;\n-\tstruct trp_node children;\n-\tuint32_t mode;\n-\tuint32_t content_offset;\n-};\n-\n-struct repo_dir {\n-\tstruct trp_root entries;\n-};\n-\n-struct repo_commit {\n-\tuint32_t root_dir_offset;\n-};\n-\n-/* Memory pools for commit, dir and dirent */\n-obj_pool_gen(commit, struct repo_commit, 4096)\n-obj_pool_gen(dir, struct repo_dir, 4096)\n-obj_pool_gen(dent, struct repo_dirent, 4096)\n-\n-static uint32_t active_commit;\n-static uint32_t mark;\n-\n-static int repo_dirent_name_cmp(const void *a, const void *b);\n-\n-/* Treap for directory entries */\n-trp_gen(static, dent_, struct repo_dirent, children, dent, repo_dirent_name_cmp);\n-\n-uint32_t next_blob_mark(void)\n-{\n-\treturn mark++;\n-}\n-\n-static struct repo_dir *repo_commit_root_dir(struct repo_commit *commit)\n-{\n-\treturn dir_pointer(commit->root_dir_offset);\n-}\n-\n-static struct repo_dirent *repo_first_dirent(struct repo_dir *dir)\n-{\n-\treturn dent_first(&dir->entries);\n-}\n-\n-static int repo_dirent_name_cmp(const void *a, const void *b)\n-{\n-\tconst struct repo_dirent *dent1 = a, *dent2 = b;\n-\tuint32_t a_offset = dent1->name_offset;\n-\tuint32_t b_offset = dent2->name_offset;\n-\treturn (a_offset > b_offset) - (a_offset < b_offset);\n-}\n-\n-static int repo_dirent_is_dir(struct repo_dirent *dent)\n-{\n-\treturn dent != NULL && dent->mode == REPO_MODE_DIR;\n-}\n-\n-static struct repo_dir *repo_dir_from_dirent(struct repo_dirent *dent)\n-{\n-\tif (!repo_dirent_is_dir(dent))\n-\t\treturn NULL;\n-\treturn dir_pointer(dent->content_offset);\n-}\n-\n-static struct repo_dir *repo_clone_dir(struct repo_dir *orig_dir)\n-{\n-\tuint32_t orig_o, new_o;\n-\torig_o = dir_offset(orig_dir);\n-\tif (orig_o >= dir_pool.committed)\n-\t\treturn orig_dir;\n-\tnew_o = dir_alloc(1);\n-\torig_dir = dir_pointer(orig_o);\n-\t*dir_pointer(new_o) = *orig_dir;\n-\treturn dir_pointer(new_o);\n-}\n-\n-static struct repo_dirent *repo_read_dirent(uint32_t revision,\n-\t\t\t\t\t const uint32_t *path)\n-{\n-\tuint32_t name = 0;\n-\tstruct repo_dirent *key = dent_pointer(dent_alloc(1));\n-\tstruct repo_dir *dir = NULL;\n-\tstruct repo_dirent *dent = NULL;\n-\tdir = repo_commit_root_dir(commit_pointer(revision));\n-\twhile (~(name = *path++)) {\n-\t\tkey->name_offset = name;\n-\t\tdent = dent_search(&dir->entries, key);\n-\t\tif (dent == NULL || !repo_dirent_is_dir(dent))\n-\t\t\tbreak;\n-\t\tdir = repo_dir_from_dirent(dent);\n-\t}\n-\tdent_free(1);\n-\treturn dent;\n-}\n-\n-static void repo_write_dirent(const uint32_t *path, uint32_t mode,\n-\t\t\t uint32_t content_offset, uint32_t del)\n-{\n-\tuint32_t name, revision, dir_o = ~0, parent_dir_o = ~0;\n-\tstruct repo_dir *dir;\n-\tstruct repo_dirent *key;\n-\tstruct repo_dirent *dent = NULL;\n-\trevision = active_commit;\n-\tdir = repo_commit_root_dir(commit_pointer(revision));\n-\tdir = repo_clone_dir(dir);\n-\tcommit_pointer(revision)->root_dir_offset = dir_offset(dir);\n-\twhile (~(name = *path++)) {\n-\t\tparent_dir_o = dir_offset(dir);\n-\n-\t\tkey = dent_pointer(dent_alloc(1));\n-\t\tkey->name_offset = name;\n-\n-\t\tdent = dent_search(&dir->entries, key);\n-\t\tif (dent == NULL)\n-\t\t\tdent = key;\n-\t\telse\n-\t\t\tdent_free(1);\n-\n-\t\tif (dent == key) {\n-\t\t\tdent->mode = REPO_MODE_DIR;\n-\t\t\tdent->content_offset = 0;\n-\t\t\tdent = dent_insert(&dir->entries, dent);\n-\t\t}\n-\n-\t\tif (dent_offset(dent) < dent_pool.committed) {\n-\t\t\tdir_o = repo_dirent_is_dir(dent) ?\n-\t\t\t\t\tdent->content_offset : ~0;\n-\t\t\tdent_remove(&dir->entries, dent);\n-\t\t\tdent = dent_pointer(dent_alloc(1));\n-\t\t\tdent->name_offset = name;\n-\t\t\tdent->mode = REPO_MODE_DIR;\n-\t\t\tdent->content_offset = dir_o;\n-\t\t\tdent = dent_insert(&dir->entries, dent);\n-\t\t}\n-\n-\t\tdir = repo_dir_from_dirent(dent);\n-\t\tdir = repo_clone_dir(dir);\n-\t\tdent->content_offset = dir_offset(dir);\n-\t}\n-\tif (dent == NULL)\n-\t\treturn;\n-\tdent->mode = mode;\n-\tdent->content_offset = content_offset;\n-\tif (del && ~parent_dir_o)\n-\t\tdent_remove(&dir_pointer(parent_dir_o)->entries, dent);\n-}\n-\n-uint32_t repo_read_path(const uint32_t *path)\n-{\n-\tuint32_t content_offset = 0;\n-\tstruct repo_dirent *dent = repo_read_dirent(active_commit, path);\n-\tif (dent != NULL)\n-\t\tcontent_offset = dent->content_offset;\n-\treturn content_offset;\n-}\n-\n-uint32_t repo_read_mode(const uint32_t *path)\n-{\n-\tstruct repo_dirent *dent = repo_read_dirent(active_commit, path);\n-\tif (dent == NULL)\n-\t\tdie(\"invalid dump: path to be modified is missing\");\n-\treturn dent->mode;\n-}\n-\n-void repo_copy(uint32_t revision, const uint32_t *src, const uint32_t *dst)\n-{\n-\tuint32_t mode = 0, content_offset = 0;\n-\tstruct repo_dirent *src_dent;\n-\tsrc_dent = repo_read_dirent(revision, src);\n-\tif (src_dent != NULL) {\n-\t\tmode = src_dent->mode;\n-\t\tcontent_offset = src_dent->content_offset;\n-\t\trepo_write_dirent(dst, mode, content_offset, 0);\n-\t}\n-}\n-\n-void repo_add(uint32_t *path, uint32_t mode, uint32_t blob_mark)\n-{\n-\trepo_write_dirent(path, mode, blob_mark, 0);\n-}\n-\n-void repo_delete(uint32_t *path)\n-{\n-\trepo_write_dirent(path, 0, 0, 1);\n-}\n-\n-static void repo_git_add_r(uint32_t depth, uint32_t *path, struct repo_dir *dir);\n-\n-static void repo_git_add(uint32_t depth, uint32_t *path, struct repo_dirent *dent)\n-{\n-\tif (repo_dirent_is_dir(dent))\n-\t\trepo_git_add_r(depth, path, repo_dir_from_dirent(dent));\n-\telse\n-\t\tfast_export_modify(depth, path,\n-\t\t\t\t dent->mode, dent->content_offset);\n-}\n-\n-static void repo_git_add_r(uint32_t depth, uint32_t *path, struct repo_dir *dir)\n-{\n-\tstruct repo_dirent *de = repo_first_dirent(dir);\n-\twhile (de) {\n-\t\tpath[depth] = de->name_offset;\n-\t\trepo_git_add(depth + 1, path, de);\n-\t\tde = dent_next(&dir->entries, de);\n-\t}\n-}\n-\n-static void repo_diff_r(uint32_t depth, uint32_t *path, struct repo_dir *dir1,\n-\t\t\tstruct repo_dir *dir2)\n-{\n-\tstruct repo_dirent *de1, *de2;\n-\tde1 = repo_first_dirent(dir1);\n-\tde2 = repo_first_dirent(dir2);\n-\n-\twhile (de1 && de2) {\n-\t\tif (de1->name_offset < de2->name_offset) {\n-\t\t\tpath[depth] = de1->name_offset;\n-\t\t\tfast_export_delete(depth + 1, path);\n-\t\t\tde1 = dent_next(&dir1->entries, de1);\n-\t\t\tcontinue;\n-\t\t}\n-\t\tif (de1->name_offset > de2->name_offset) {\n-\t\t\tpath[depth] = de2->name_offset;\n-\t\t\trepo_git_add(depth + 1, path, de2);\n-\t\t\tde2 = dent_next(&dir2->entries, de2);\n-\t\t\tcontinue;\n-\t\t}\n-\t\tpath[depth] = de1->name_offset;\n-\n-\t\tif (de1->mode == de2->mode &&\n-\t\t de1->content_offset == de2->content_offset) {\n-\t\t\t; /* No change. */\n-\t\t} else if (repo_dirent_is_dir(de1) && repo_dirent_is_dir(de2)) {\n-\t\t\trepo_diff_r(depth + 1, path,\n-\t\t\t\t repo_dir_from_dirent(de1),\n-\t\t\t\t repo_dir_from_dirent(de2));\n-\t\t} else if (!repo_dirent_is_dir(de1) && !repo_dirent_is_dir(de2)) {\n-\t\t\trepo_git_add(depth + 1, path, de2);\n-\t\t} else {\n-\t\t\tfast_export_delete(depth + 1, path);\n-\t\t\trepo_git_add(depth + 1, path, de2);\n-\t\t}\n-\t\tde1 = dent_next(&dir1->entries, de1);\n-\t\tde2 = dent_next(&dir2->entries, de2);\n-\t}\n-\twhile (de1) {\n-\t\tpath[depth] = de1->name_offset;\n-\t\tfast_export_delete(depth + 1, path);\n-\t\tde1 = dent_next(&dir1->entries, de1);\n-\t}\n-\twhile (de2) {\n-\t\tpath[depth] = de2->name_offset;\n-\t\trepo_git_add(depth + 1, path, de2);\n-\t\tde2 = dent_next(&dir2->entries, de2);\n-\t}\n-}\n-\n-static uint32_t path_stack[REPO_MAX_PATH_DEPTH];\n-\n-void repo_diff(uint32_t r1, uint32_t r2)\n-{\n-\trepo_diff_r(0,\n-\t\t path_stack,\n-\t\t repo_commit_root_dir(commit_pointer(r1)),\n-\t\t repo_commit_root_dir(commit_pointer(r2)));\n-}\n-\n-void repo_commit(uint32_t revision, uint32_t author, char *log, uint32_t uuid,\n-\t\t uint32_t url, unsigned long timestamp)\n-{\n-\tfast_export_commit(revision, author, log, uuid, url, timestamp);\n-\tdent_commit();\n-\tdir_commit();\n-\tactive_commit = commit_alloc(1);\n-\tcommit_pointer(active_commit)->root_dir_offset =\n-\t\tcommit_pointer(active_commit - 1)->root_dir_offset;\n-}\n-\n-static void mark_init(void)\n-{\n-\tuint32_t i;\n-\tmark = 1024 * 1024 * 1024;\n-\tfor (i = 0; i < dent_pool.size; i++)\n-\t\tif (!repo_dirent_is_dir(dent_pointer(i)) &&\n-\t\t dent_pointer(i)->content_offset > mark)\n-\t\t\tmark = dent_pointer(i)->content_offset;\n-\tmark++;\n-}\n-\n-void repo_init(void)\n-{\n-\tmark_init();\n-\tif (commit_pool.size == 0) {\n-\t\t/* Create empty tree for commit 0. */\n-\t\tcommit_alloc(1);\n-\t\tcommit_pointer(0)->root_dir_offset = dir_alloc(1);\n-\t\tdir_pointer(0)->entries.trp_root = ~0;\n-\t\tdir_commit();\n-\t}\n-\t/* Preallocate next commit, ready for changes. */\n-\tactive_commit = commit_alloc(1);\n-\tcommit_pointer(active_commit)->root_dir_offset =\n-\t\tcommit_pointer(active_commit - 1)->root_dir_offset;\n-}\n-\n-void repo_reset(void)\n-{\n-\tpool_reset();\n-\tcommit_reset();\n-\tdir_reset();\n-\tdent_reset();\n-}\n+/*\n+ * Licensed under a two-clause BSD-style license.\n+ * See LICENSE for details.\n+ */\n+\n+#include \"git-compat-util.h\"\n+#include \"strbuf.h\"\n+#include \"repo_tree.h\"\n+#include \"fast_export.h\"\n+\n+const char *repo_read_path(const uint32_t *path)\n+{\n+\tint err;\n+\tuint32_t dummy;\n+\tstatic struct strbuf buf = STRBUF_INIT;\n+\n+\tstrbuf_reset(&buf);\n+\terr = fast_export_ls(REPO_MAX_PATH_DEPTH, path, &dummy, &buf);\n+\tif (err) {\n+\t\tif (errno != ENOENT)\n+\t\t\tdie_errno(\"BUG: unexpected fast_export_ls error\");\n+\t\treturn NULL;\n+\t}\n+\treturn buf.buf;\n+}\n+\n+uint32_t repo_read_mode(const uint32_t *path)\n+{\n+\tint err;\n+\tuint32_t result;\n+\tstatic struct strbuf dummy = STRBUF_INIT;\n+\n+\tstrbuf_reset(&dummy);\n+\terr = fast_export_ls(REPO_MAX_PATH_DEPTH, path, &result, &dummy);\n+\tif (err) {\n+\t\tif (errno != ENOENT)\n+\t\t\tdie_errno(\"BUG: unexpected fast_export_ls error\");\n+\t\t/* Treat missing paths as directories. */\n+\t\treturn REPO_MODE_DIR;\n+\t}\n+\treturn result;\n+}\n+\n+void repo_copy(uint32_t revision, const uint32_t *src, const uint32_t *dst)\n+{\n+\tint err;\n+\tuint32_t mode;\n+\tstatic struct strbuf data = STRBUF_INIT;\n+\n+\tstrbuf_reset(&data);\n+\terr = fast_export_ls_rev(revision, REPO_MAX_PATH_DEPTH, src, &mode, &data);\n+\tif (err) {\n+\t\tif (errno != ENOENT)\n+\t\t\tdie_errno(\"BUG: unexpected fast_export_ls_rev error\");\n+\t\tfast_export_delete(REPO_MAX_PATH_DEPTH, dst);\n+\t\treturn;\n+\t}\n+\tfast_export_modify(REPO_MAX_PATH_DEPTH, dst, mode, data.buf);\n+}\n+\n+void repo_delete(uint32_t *path)\n+{\n+\tfast_export_delete(REPO_MAX_PATH_DEPTH, path);\n+}\ndiff --git a/vcs-svn/repo_tree.h b/vcs-svn/repo_tree.h\nindex 11d48c2..d690784 100644\n--- a/vcs-svn/repo_tree.h\n+++ b/vcs-svn/repo_tree.h\n@@ -14,7 +14,7 @@\n uint32_t next_blob_mark(void);\n void repo_copy(uint32_t revision, const uint32_t *src, const uint32_t *dst);\n void repo_add(uint32_t *path, uint32_t mode, uint32_t blob_mark);\n-uint32_t repo_read_path(const uint32_t *path);\n+const char *repo_read_path(const uint32_t *path);\n uint32_t repo_read_mode(const uint32_t *path);\n void repo_delete(uint32_t *path);\n void repo_commit(uint32_t revision, uint32_t author, char *log, uint32_t uuid,\ndiff --git a/vcs-svn/string_pool.c b/vcs-svn/string_pool.c\nindex f5b1da8..c08abac 100644\n--- a/vcs-svn/string_pool.c\n+++ b/vcs-svn/string_pool.c\n@@ -65,7 +65,7 @@ uint32_t pool_tok_r(char *str, const char *delim, char **saveptr)\n \treturn token ? pool_intern(token) : ~0;\n }\n \n-void pool_print_seq(uint32_t len, uint32_t *seq, char delim, FILE *stream)\n+void pool_print_seq(uint32_t len, const uint32_t *seq, char delim, FILE *stream)\n {\n \tuint32_t i;\n \tfor (i = 0; i < len && ~seq[i]; i++) {\ndiff --git a/vcs-svn/string_pool.h b/vcs-svn/string_pool.h\nindex 222fb66..3720cf8 100644\n--- a/vcs-svn/string_pool.h\n+++ b/vcs-svn/string_pool.h\n@@ -4,7 +4,7 @@\n uint32_t pool_intern(const char *key);\n const char *pool_fetch(uint32_t entry);\n uint32_t pool_tok_r(char *str, const char *delim, char **saveptr);\n-void pool_print_seq(uint32_t len, uint32_t *seq, char delim, FILE *stream);\n+void pool_print_seq(uint32_t len, const uint32_t *seq, char delim, FILE *stream);\n uint32_t pool_tok_seq(uint32_t sz, uint32_t *seq, const char *delim, char *str);\n void pool_reset(void);\n \ndiff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c\nindex 3cc4135..7ecb227 100644\n--- a/vcs-svn/svndump.c\n+++ b/vcs-svn/svndump.c\n@@ -36,6 +36,8 @@ obj_pool_gen(log, char, 4096)\n \n static struct line_buffer input = LINE_BUFFER_INIT;\n \n+#define REPORT_FILENO 3\n+\n static char *log_copy(uint32_t length, const char *log)\n {\n \tchar *buffer;\n@@ -202,15 +204,21 @@ static void read_props(void)\n \n static void handle_node(void)\n {\n-\tuint32_t mark = 0;\n \tconst uint32_t type = node_ctx.type;\n \tconst int have_props = node_ctx.propLength != LENGTH_UNKNOWN;\n \tconst int have_text = node_ctx.textLength != LENGTH_UNKNOWN;\n+\t/*\n+\t * Old text for this node:\n+\t * NULL\t- directory or bug\n+\t * empty_blob\t- empty\n+\t * \"\"\t- data retrievable from fast-import\n+\t */\n+\tstatic const char *const empty_blob = \"::empty::\";\n+\tconst char *old_data = NULL;\n \n \tif (node_ctx.text_delta)\n \t\tdie(\"text deltas not supported\");\n-\tif (have_text)\n-\t\tmark = next_blob_mark();\n+\n \tif (node_ctx.action == NODEACT_DELETE) {\n \t\tif (have_text || have_props || node_ctx.srcRev)\n \t\t\tdie(\"invalid dump: deletion node has \"\n@@ -230,15 +238,15 @@ static void handle_node(void)\n \t\tdie(\"invalid dump: directories cannot have text attached\");\n \n \t/*\n-\t * Decide on the new content (mark) and mode (node_ctx.type).\n+\t * Find old content (old_data) and decide on the new mode.\n \t */\n \tif (node_ctx.action == NODEACT_CHANGE && !~*node_ctx.dst) {\n \t\tif (type != REPO_MODE_DIR)\n \t\t\tdie(\"invalid dump: root of tree is not a regular file\");\n+\t\told_data = NULL;\n \t} else if (node_ctx.action == NODEACT_CHANGE) {\n \t\tuint32_t mode;\n-\t\tif (!have_text)\n-\t\t\tmark = repo_read_path(node_ctx.dst);\n+\t\told_data = repo_read_path(node_ctx.dst);\n \t\tmode = repo_read_mode(node_ctx.dst);\n \t\tif (mode == REPO_MODE_DIR && type != REPO_MODE_DIR)\n \t\t\tdie(\"invalid dump: cannot modify a directory into a file\");\n@@ -246,7 +254,11 @@ static void handle_node(void)\n \t\t\tdie(\"invalid dump: cannot modify a file into a directory\");\n \t\tnode_ctx.type = mode;\n \t} else if (node_ctx.action == NODEACT_ADD) {\n-\t\tif (!have_text && type != REPO_MODE_DIR)\n+\t\tif (type == REPO_MODE_DIR)\n+\t\t\told_data = NULL;\n+\t\telse if (have_text)\n+\t\t\told_data = empty_blob;\n+\t\telse\n \t\t\tdie(\"invalid dump: adds node without text\");\n \t} else {\n \t\tdie(\"invalid dump: Node-path block lacks Node-action\");\n@@ -265,24 +277,34 @@ static void handle_node(void)\n \t/*\n \t * Save the result.\n \t */\n-\trepo_add(node_ctx.dst, node_ctx.type, mark);\n-\tif (have_text)\n-\t\tfast_export_blob(node_ctx.type, mark,\n-\t\t\t\t node_ctx.textLength, &input);\n+\tif (type == REPO_MODE_DIR)\t/* directories are not tracked. */\n+\t\treturn;\n+\tassert(old_data);\n+\tif (old_data == empty_blob)\n+\t\t/* For the fast_export_* functions, NULL means empty. */\n+\t\told_data = NULL;\n+\tif (!have_text) {\n+\t\tfast_export_modify(REPO_MAX_PATH_DEPTH, node_ctx.dst,\n+\t\t\t\t\tnode_ctx.type, old_data);\n+\t\treturn;\n+\t}\n+\tfast_export_modify(REPO_MAX_PATH_DEPTH, node_ctx.dst,\n+\t\t\t\tnode_ctx.type, \"inline\");\n+\tfast_export_data(node_ctx.type, node_ctx.textLength, &input);\n }\n \n static void begin_revision(void)\n {\n \tif (!rev_ctx.revision)\t/* revision 0 gets no git commit. */\n \t\treturn;\n-\tfast_export_begin_commit(rev_ctx.revision);\n+\tfast_export_begin_commit(rev_ctx.revision, rev_ctx.author, rev_ctx.log,\n+\t\tdump_ctx.uuid, dump_ctx.url, rev_ctx.timestamp);\n }\n \n static void end_revision(void)\n {\n \tif (rev_ctx.revision)\n-\t\trepo_commit(rev_ctx.revision, rev_ctx.author, rev_ctx.log,\n-\t\t\tdump_ctx.uuid, dump_ctx.url, rev_ctx.timestamp);\n+\t\tfast_export_end_commit(rev_ctx.revision);\n }\n \n void svndump_read(const char *url)\n@@ -383,7 +405,6 @@ int svndump_init(const char *filename)\n {\n \tif (buffer_init(&input, filename))\n \t\treturn error(\"cannot open %s: %s\", filename, strerror(errno));\n-\trepo_init();\n \tfast_export_init(REPORT_FILENO);\n \treset_dump_ctx(~0);\n \treset_rev_ctx(0);\n@@ -396,7 +417,6 @@ void svndump_deinit(void)\n {\n \tlog_reset();\n \tfast_export_deinit();\n-\trepo_reset();\n \treset_dump_ctx(~0);\n \treset_rev_ctx(0);\n \treset_node_ctx(NULL);\n@@ -411,7 +431,6 @@ void svndump_reset(void)\n \tlog_reset();\n \tfast_export_reset();\n \tbuffer_reset(&input);\n-\trepo_reset();\n \treset_dump_ctx(~0);\n \treset_rev_ctx(0);\n \treset_node_ctx(NULL);\n-- \n1.7.4.1\n"},{"id":"162884","messageId":"20110306231246.GL24327@elie","threadId":"26014","inReplyTo":"20110306225419.GA24327@elie","subject":"[PATCH 10/12] vcs-svn: quote paths correctly for ls command","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-03-06T23:12:47Z","receivedAt":"2011-03-06T23:12:47Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"From: David Barr \nDate: Sun, 12 Dec 2010 03:59:31 +1100\n\nThis bug was found while importing rev 601865 of ASF.\n\n[jn: with test]\n\nSigned-off-by: David Barr \nSigned-off-by: Jonathan Nieder \n---\n t/t9010-svn-fe.sh | 99 +++++++++++++++++++++++++++++++++++++++++++++++++\n vcs-svn/fast_export.c | 2 +-\n vcs-svn/string_pool.c | 11 +++++\n vcs-svn/string_pool.h | 1 +\n 4 files changed, 112 insertions(+), 1 deletions(-)\n\ndiff --git a/t/t9010-svn-fe.sh b/t/t9010-svn-fe.sh\nindex 2ae5374..720fd6b 100755\n--- a/t/t9010-svn-fe.sh\n+++ b/t/t9010-svn-fe.sh\n@@ -270,6 +270,105 @@ test_expect_success PIPE 'directory with files' '\n \ttest_cmp hi directory/file2\n '\n \n+test_expect_success PIPE 'branch name with backslash' '\n+\treinit_git &&\n+\tsort <<-\\EOF >expect.branch-files &&\n+\ttrunk/file1\n+\ttrunk/file2\n+\t\"branches/UpdateFOPto094\\\\/file1\"\n+\t\"branches/UpdateFOPto094\\\\/file2\"\n+\tEOF\n+\n+\techo hi >hi &&\n+\techo hello >hello &&\n+\t{\n+\t\tproperties \\\n+\t\t\tsvn:author author@example.com \\\n+\t\t\tsvn:date \"1999-02-02T00:01:02.000000Z\" \\\n+\t\t\tsvn:log \"add directory with some files in it\" &&\n+\t\techo PROPS-END\n+\t} >props.setup &&\n+\t{\n+\t\tproperties \\\n+\t\t\tsvn:author brancher@example.com \\\n+\t\t\tsvn:date \"2007-12-06T21:38:34.000000Z\" \\\n+\t\t\tsvn:log \"Updating fop to .94 and adjust fo-stylesheets\" &&\n+\t\techo PROPS-END\n+\t} >props.branch &&\n+\t{\n+\t\tcat <<-EOF &&\n+\t\tSVN-fs-dump-format-version: 3\n+\n+\t\tRevision-number: 1\n+\t\tEOF\n+\t\techo Prop-content-length: $(wc -c branch.dump &&\n+\ttry_dump branch.dump &&\n+\n+\tgit ls-tree -r --name-only HEAD |\n+\tsort >actual.branch-files &&\n+\ttest_cmp expect.branch-files actual.branch-files\n+'\n+\n test_expect_success PIPE 'node without action' '\n \treinit_git &&\n \tcat >inaction.dump <<-\\EOF &&\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex a8ce5c6..4d57efa 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -107,7 +107,7 @@ static void ls_from_active_commit(uint32_t depth, const uint32_t *path)\n {\n \t/* ls \"path/to/file\" */\n \tprintf(\"ls \\\"\");\n-\tpool_print_seq(depth, path, '/', stdout);\n+\tpool_print_seq_q(depth, path, '/', stdout);\n \tprintf(\"\\\"\\n\");\n \tfflush(stdout);\n }\ndiff --git a/vcs-svn/string_pool.c b/vcs-svn/string_pool.c\nindex c08abac..be43598 100644\n--- a/vcs-svn/string_pool.c\n+++ b/vcs-svn/string_pool.c\n@@ -4,6 +4,7 @@\n */\n \n #include \"git-compat-util.h\"\n+#include \"quote.h\"\n #include \"trp.h\"\n #include \"obj_pool.h\"\n #include \"string_pool.h\"\n@@ -75,6 +76,16 @@ void pool_print_seq(uint32_t len, const uint32_t *seq, char delim, FILE *stream)\n \t}\n }\n \n+void pool_print_seq_q(uint32_t len, const uint32_t *seq, char delim, FILE *stream)\n+{\n+\tuint32_t i;\n+\tfor (i = 0; i < len && ~seq[i]; i++) {\n+\t\tquote_c_style(pool_fetch(seq[i]), NULL, stream, 1);\n+\t\tif (i < len - 1 && ~seq[i + 1])\n+\t\t\tfputc(delim, stream);\n+\t}\n+}\n+\n uint32_t pool_tok_seq(uint32_t sz, uint32_t *seq, const char *delim, char *str)\n {\n \tchar *context = NULL;\ndiff --git a/vcs-svn/string_pool.h b/vcs-svn/string_pool.h\nindex 3720cf8..96e501d 100644\n--- a/vcs-svn/string_pool.h\n+++ b/vcs-svn/string_pool.h\n@@ -5,6 +5,7 @@ uint32_t pool_intern(const char *key);\n const char *pool_fetch(uint32_t entry);\n uint32_t pool_tok_r(char *str, const char *delim, char **saveptr);\n void pool_print_seq(uint32_t len, const uint32_t *seq, char delim, FILE *stream);\n+void pool_print_seq_q(uint32_t len, const uint32_t *seq, char delim, FILE *stream);\n uint32_t pool_tok_seq(uint32_t sz, uint32_t *seq, const char *delim, char *str);\n void pool_reset(void);\n \n-- \n1.7.4.1\n"},{"id":"162885","messageId":"20110306231337.GM24327@elie","threadId":"26014","inReplyTo":"20110306225419.GA24327@elie","subject":"[PATCH 11/12] vcs-svn: handle filenames with dq correctly","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-03-06T23:13:37Z","receivedAt":"2011-03-06T23:13:37Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Date: Sat, 11 Dec 2010 17:08:51 -0600\n\nQuote paths passed to fast-import so filenames with double quotes are\nnot misinterpreted.\n\nOne might imagine this could help with filenames with newlines, too,\nbut svn does not allow those.\n\nHelped-by: David Barr \nSigned-off-by: Jonathan Nieder \nSigned-off-by: David Barr \nSigned-off-by: Jonathan Nieder \n---\n vcs-svn/fast_export.c | 19 +++++++++----------\n 1 files changed, 9 insertions(+), 10 deletions(-)\n\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex 4d57efa..9c03f3e 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -34,10 +34,9 @@ void fast_export_reset(void)\n \n void fast_export_delete(uint32_t depth, const uint32_t *path)\n {\n-\tputchar('D');\n-\tputchar(' ');\n-\tpool_print_seq(depth, path, '/', stdout);\n-\tputchar('\\n');\n+\tprintf(\"D \\\"\");\n+\tpool_print_seq_q(depth, path, '/', stdout);\n+\tprintf(\"\\\"\\n\");\n }\n \n static void fast_export_truncate(uint32_t depth, const uint32_t *path, uint32_t mode)\n@@ -54,9 +53,9 @@ void fast_export_modify(uint32_t depth, const uint32_t *path, uint32_t mode,\n \t\tfast_export_truncate(depth, path, mode);\n \t\treturn;\n \t}\n-\tprintf(\"M %06\"PRIo32\" %s \", mode, dataref);\n-\tpool_print_seq(depth, path, '/', stdout);\n-\tputchar('\\n');\n+\tprintf(\"M %06\"PRIo32\" %s \\\"\", mode, dataref);\n+\tpool_print_seq_q(depth, path, '/', stdout);\n+\tprintf(\"\\\"\\n\");\n }\n \n static char gitsvnline[MAX_GITSVN_LINE_LEN];\n@@ -97,9 +96,9 @@ void fast_export_end_commit(uint32_t revision)\n static void ls_from_rev(uint32_t rev, uint32_t depth, const uint32_t *path)\n {\n \t/* ls :5 path/to/old/file */\n-\tprintf(\"ls :%\"PRIu32\" \", rev);\n-\tpool_print_seq(depth, path, '/', stdout);\n-\tputchar('\\n');\n+\tprintf(\"ls :%\"PRIu32\" \\\"\", rev);\n+\tpool_print_seq_q(depth, path, '/', stdout);\n+\tprintf(\"\\\"\\n\");\n \tfflush(stdout);\n }\n \n-- \n1.7.4.1\n"},{"id":"162886","messageId":"20110306231613.GN24327@elie","threadId":"26014","inReplyTo":"20110306225419.GA24327@elie","subject":"[PATCH 12/12] vcs-svn: use mark from previous import for parent commit","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-03-06T23:16:13Z","receivedAt":"2011-03-06T23:16:13Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"From: David Barr \nDate: Sun, 12 Dec 2010 13:41:38 +1100\n\nWith this patch, overlapping incremental imports work.\n\nSigned-off-by: David Barr \nSigned-off-by: Jonathan Nieder \n---\nThat's the end of the series.\n\nBug reports, suggestions, improvements, welcome, of course. The\ndocumentation in contrib/svn-fe/svn-fe.txt is probably out of date\nnow.\n\nHave fun :)\nJonathan\n\n vcs-svn/fast_export.c | 2 +-\n 1 files changed, 1 insertions(+), 1 deletions(-)\n\ndiff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c\nindex 9c03f3e..f19db9a 100644\n--- a/vcs-svn/fast_export.c\n+++ b/vcs-svn/fast_export.c\n@@ -83,7 +83,7 @@ void fast_export_begin_commit(uint32_t revision, uint32_t author, char *log,\n \t\t log, gitsvnline);\n \tif (!first_commit_done) {\n \t\tif (revision > 1)\n-\t\t\tprintf(\"from refs/heads/master^0\\n\");\n+\t\t\tprintf(\"from :%\"PRIu32\"\\n\", revision - 1);\n \t\tfirst_commit_done = 1;\n \t}\n }\n-- \n1.7.4.1\n"},{"id":"162906","messageId":"AANLkTi=+ujWER5aeYzYW1pN-LOdM3Xa2J2bZhjkoeWvB@mail.gmail.com","threadId":"26014","inReplyTo":"20110306225419.GA24327@elie","subject":"Re: [PATCH v2 00/12] vcs-svn: incremental import","fromName":"Sverre Rabbelier","fromEmail":"srabbelier@gmail.com","sentAt":"2011-03-07T12:24:16Z","receivedAt":"2011-03-07T12:24:16Z","isPatch":true,"sender":{"key":"srabbelier@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3098?v=4"},"body":"Heya,\n\nOn Sun, Mar 6, 2011 at 23:54, Jonathan Nieder wrote:\n> Patch 1 changes the mark numbers for blobs to be ridiculously high,\n> to make room for memorable commit marks (:1 for r1, :2 for r2, etc).\n\nHow high is rediculously high? There are repositories with a lot of\ncommits, will they have enough room? Ditto for repositories with a lot\nof blobs, will _those_ have enough room? What about the git\nfast-import side, what about this statement:\n\n\"Marks are stored in a sparse array, using 1 pointer (4 bytes or 8\nbytes, depending on pointer size) per mark. Although the array is\nsparse, frontends are still strongly encouraged to use marks between 1\nand n, where n is the total number of marks required for this import.\"\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"162939","messageId":"20110307212330.GA15732@elie","threadId":"26014","inReplyTo":"AANLkTi=+ujWER5aeYzYW1pN-LOdM3Xa2J2bZhjkoeWvB@mail.gmail.com","subject":"Re: [PATCH v2 00/12] vcs-svn: incremental import","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-03-07T21:23:30Z","receivedAt":"2011-03-07T21:23:30Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Sverre Rabbelier wrote:\n> On Sun, Mar 6, 2011 at 23:54, Jonathan Nieder wrote:\n\n>> Patch 1 changes the mark numbers for blobs to be ridiculously high,\n>> to make room for memorable commit marks (:1 for r1, :2 for r2, etc).\n>\n> How high is rediculously high?\n\n2^30. Later in the series (vcs-svn: eliminate repo_tree structure)\nthe blob marks are eliminated altogether.\n\n> \"Marks are stored in a sparse array, using 1 pointer (4 bytes or 8\n> bytes, depending on pointer size) per mark. Although the array is\n> sparse, frontends are still strongly encouraged to use marks between 1\n> and n, where n is the total number of marks required for this import.\"\n\nI assume that is mostly meant to make it easier to write alternate\nbackends.\n"},{"id":"163001","messageId":"7vzkp5e8gr.fsf@alter.siamese.dyndns.org","threadId":"26014","inReplyTo":"20110306230348.GC24327@elie","subject":"Re: [PATCH 01/12] vcs-svn: use higher mark numbers for blobs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-03-08T19:08:20Z","receivedAt":"2011-03-08T19:08:20Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Nieder writes:\n\n> Date: Fri, 10 Dec 2010 04:21:35 -0600\n>\n> Prepare to use mark :5 for the commit corresponding to r5 (and so on).\n>\n> 1 billion seems sufficiently high for blob marks to avoid conflicting\n> with rev marks, while still leaving room for 3 billion blobs. Such\n> high mark numbers cause trouble with ancient fast-import versions, but\n> this topic cannot support git fast-import versions before 1.7.4 (which\n> introduces the cat-blob command) anyway.\n\nHmm, 1G+3G split? Will we have HIGHMEM option someday? ;-)\n\nHow confident are you that you will never need more than two classes later\nand you will never need to split the larger space again?\n\nIf you are not, and if the topic is to introduce incompatible output,\nwould it be wiser to be even more forward looking and introduce different\nclasses of marks with a backward incompatible syntax, perhaps like using\n\":\\d+\" for anything, and using \":[a-zA-Z0-9]+:\\d+\" for some application\nspecific \"class\" of objects that is specifed by the [a-zA-Z0-9]+ part?\n"},{"id":"163045","messageId":"20110309065528.GA28098@elie","threadId":"26014","inReplyTo":"7vzkp5e8gr.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH 01/12] vcs-svn: use higher mark numbers for blobs","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-03-09T06:55:28Z","receivedAt":"2011-03-09T06:55:28Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nJunio C Hamano wrote:\n\n> Hmm, 1G+3G split? Will we have HIGHMEM option someday? ;-)\n>\n> How confident are you that you will never need more than two classes later\n> and you will never need to split the larger space again?\n>\n> If you are not, and if the topic is to introduce incompatible output,\n> would it be wiser to be even more forward looking and introduce different\n> classes of marks with a backward incompatible syntax, perhaps like using\n> \":\\d+\" for anything, and using \":[a-zA-Z0-9]+:\\d+\" for some application\n> specific \"class\" of objects that is specifed by the [a-zA-Z0-9]+ part?\n\nThat sounds very sensible (and I'd be happy to see something like\nthat).\n\nIn this particular case a later patch (\"vcs-svn: eliminate repo_tree\nstructure\") gets rid of the blob marks so the split is temporary.\nPerhaps a paragraph added to the change description would clear it up.\n\n\tA later patch will eliminate the blob marks altogether.\n\nFor the \"vcs-svn: eliminate repo_tree\" patch:\n\n\tRely on fast-import for information about previous revs.\n\n\tThis requires always setting up backward flow of information,\n\teven for v2 dumps. On the plus side:\n\n\t - No more need to include blobs in the marks table.\n\t - Given one dump that picks up where another left off, svn-fe\n\t can continue the import. Use\n\n\t\tgit fast-import --relative-marks \\\n\t\t\t--export-marks=svn-revs \\\n\t\t\t--cat-blob-fd=3 3>backchannel\n\n\t for the first import and\n\n\t\tgit fast-import --relative-marks \\\n\t\t\t--import-marks=svn-revs \\\n\t\t\t--export-marks=svn-revs \\\n\t\t\t--cat-blob-fd=3 3>backchannel\n\n\t for later ones.\n\t - It simplifies the code by quite a bit and opens the door\n\t to further simplifications.\n\nThanks for some clarity.\nJonathan\n"}]}