{"thread":{"id":"24424","subject":"[PATCHv2 0/2] non-incremental mode for fast-export","startedAt":"2010-07-17T17:00:49Z","lastAt":"2010-07-17T17:00:51Z","messageCount":3,"participants":["Elijah Newren"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"145734","messageId":"1279386051-31112-1-git-send-email-newren@gmail.com","threadId":"24424","inReplyTo":null,"subject":"[PATCHv2 0/2] non-incremental mode for fast-export","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2010-07-17T17:00:49Z","receivedAt":"2010-07-17T17:00:49Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"This patch series fixes silently dropped files in uses of fast-export\ninvolving both --import-marks and limiting files by path.  It also\nadds a new feature: a --full-tree option to switch from incremental to\ncomprehensive handling of files in commits.  The two changes are\nrelated in that the bugfix consists of simply automatically activating\nthe new feature in a case where it is required for correct operation.\n\nThere is just one change since the last submission\n  * modify one of the new testcases to not depend on the full exact\n    output of fast-export (since it is no longer the same in next)\nThis change enables the series to apply cleanly to next.  All tests\npass with this series applied to either maint or next now.\n\nI'll again note that it may make sense to split this patch series\nsince the first is a bugfix and thus makes sense to apply to maint,\nwhile the second (whose testcase was the fragile one) is a feature\naddition, though a fairly trivial one.\n\nElijah Newren (2):\n      fast-export: Fix dropping of files with --import-marks and path limiting\n      fast-export: Add a --full-tree option\n\n Documentation/git-fast-export.txt |    6 ++++++\n builtin/fast-export.c             |   11 ++++++++++-\n t/t9350-fast-export.sh            |   14 ++++++++++++++\n 3 files changed, 30 insertions(+), 1 deletions(-)\n"},{"id":"145735","messageId":"1279386051-31112-2-git-send-email-newren@gmail.com","threadId":"24424","inReplyTo":"1279386051-31112-1-git-send-email-newren@gmail.com","subject":"[PATCHv2 1/2] fast-export: Fix dropping of files with --import-marks and path limiting","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2010-07-17T17:00:50Z","receivedAt":"2010-07-17T17:00:50Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Since fast-export operates by listing file changes since the (first) parent\ncommit, when using --import-marks and path limiting and using a wider list\nof paths than in previous runs, files from the new path(s) will silently be\nomitted from the result unless or until a commit which explicitly changes\nthose files.  The resulting repository in such cases is broken and makes no\nsense.\n\nThis commit fixes this by having fast-export work with complete trees\ninstead of incremental changes (when both --import-marks and path limiting\nare used).  It works by issuing a 'deleteall' directive with each commit and\nthen listing the full set of files that make up that commit, rather than\njust showing the list of files that have changed since the (first) parent\ncommit.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\nAcked-by: Sverre Rabbelier <srabbelier@gmail.com>\n---\n builtin/fast-export.c  |    9 ++++++++-\n t/t9350-fast-export.sh |    9 +++++++++\n 2 files changed, 17 insertions(+), 1 deletions(-)\n\ndiff --git a/builtin/fast-export.c b/builtin/fast-export.c\nindex c6dd71a..25d13a1 100644\n--- a/builtin/fast-export.c\n+++ b/builtin/fast-export.c\n@@ -27,6 +27,7 @@ static enum { ABORT, VERBATIM, WARN, STRIP } signed_tag_mode = ABORT;\n static enum { ERROR, DROP, REWRITE } tag_of_filtered_mode = ABORT;\n static int fake_missing_tagger;\n static int no_data;\n+static int full_tree = 0;\n \n static int parse_opt_signed_tag_mode(const struct option *opt,\n \t\t\t\t     const char *arg, int unset)\n@@ -241,7 +242,8 @@ static void handle_commit(struct commit *commit, struct rev_info *rev)\n \t\tmessage += 2;\n \n \tif (commit->parents &&\n-\t    get_object_mark(&commit->parents->item->object) != 0) {\n+\t    get_object_mark(&commit->parents->item->object) != 0 &&\n+\t    !full_tree) {\n \t\tparse_commit(commit->parents->item);\n \t\tdiff_tree_sha1(commit->parents->item->tree->object.sha1,\n \t\t\t       commit->tree->object.sha1, \"\", &rev->diffopt);\n@@ -281,6 +283,8 @@ static void handle_commit(struct commit *commit, struct rev_info *rev)\n \t\ti++;\n \t}\n \n+\tif (full_tree)\n+\t\tprintf(\"deleteall\\n\");\n \tlog_tree_diff_flush(rev);\n \trev->diffopt.output_format = saved_output_format;\n \n@@ -608,6 +612,9 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix)\n \tif (import_filename)\n \t\timport_marks(import_filename);\n \n+\tif (import_filename && revs.prune_data)\n+\t\tfull_tree = 1;\n+\n \tget_tags_and_duplicates(&revs.pending, &extra_refs);\n \n \tif (prepare_revision_walk(&revs))\ndiff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh\nindex d43f37c..6069e1f 100755\n--- a/t/t9350-fast-export.sh\n+++ b/t/t9350-fast-export.sh\n@@ -355,6 +355,15 @@ test_expect_failure 'no exact-ref revisions included' '\n \t)\n '\n \n+test_expect_success 'path limiting with import-marks does not lose unmodified files'        '\n+\tgit checkout -b simple marks~2 &&\n+\tgit fast-export --export-marks=marks simple -- file > /dev/null &&\n+\techo more content >> file &&\n+\ttest_tick &&\n+\tgit commit -mnext file &&\n+\tgit fast-export --import-marks=marks simple -- file file0 | grep file0\n+'\n+\n test_expect_success 'set-up a few more tags for tag export tests' '\n \tgit checkout -f master &&\n \tHEAD_TREE=`git show -s --pretty=raw HEAD | grep tree | sed \"s/tree //\"` &&\n-- \n1.6.6.1\n"},{"id":"145736","messageId":"1279386051-31112-3-git-send-email-newren@gmail.com","threadId":"24424","inReplyTo":"1279386051-31112-2-git-send-email-newren@gmail.com","subject":"[PATCHv2 2/2] fast-export: Add a --full-tree option","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2010-07-17T17:00:51Z","receivedAt":"2010-07-17T17:00:51Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"This option adds symmetry with fast-import, enabling it to also work with\ncomplete trees instead of just incremental changes.  It works by issuing a\n'deleteall' directive with each commit and then listing the full set of\nfiles that make up that commit, rather than just showing the list of files\nthat have changed since the (first) parent commit.  Note that this\nfunctionality is automatically turned on when using --import-marks together\nwith path limiting in order to avoid dropping important but unchanged\nfiles.\n\nThis functionality is desired when using hand-written filters along with\n'fast-export | some-filter | fast-import' as it can be easier to write\n<some-filter> in terms of complete trees than incremental changes.\n\nWe could avoid the need to add this option by simply always turning it on.\nWhile the end result would be identical, it would slow things down slightly\nby printing many more filenames per commit which goes somewhat against the\n'fast' in 'fast-export'.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\nAcked-by: Sverre Rabbelier <srabbelier@gmail.com>\n---\n Documentation/git-fast-export.txt |    6 ++++++\n builtin/fast-export.c             |    4 +++-\n t/t9350-fast-export.sh            |    5 +++++\n 3 files changed, 14 insertions(+), 1 deletions(-)\n\ndiff --git a/Documentation/git-fast-export.txt b/Documentation/git-fast-export.txt\nindex 98ec6b5..8a6a3cb 100644\n--- a/Documentation/git-fast-export.txt\n+++ b/Documentation/git-fast-export.txt\n@@ -90,6 +90,12 @@ marks the same across runs.\n \tresulting stream can only be used by a repository which\n \talready contains the necessary objects.\n \n+--full-tree::\n+\tThis option will cause fast-export to issue a \"deleteall\"\n+\tdirective for each commit followed by a full list of all files\n+\tin the commit (as opposed to just listing the files which are\n+\tdifferent from the commit's first parent).\n+\n [git-rev-list-args...]::\n        A list of arguments, acceptable to 'git rev-parse' and\n        'git rev-list', that specifies the specific objects and references\ndiff --git a/builtin/fast-export.c b/builtin/fast-export.c\nindex 25d13a1..8c77602 100644\n--- a/builtin/fast-export.c\n+++ b/builtin/fast-export.c\n@@ -27,7 +27,7 @@ static enum { ABORT, VERBATIM, WARN, STRIP } signed_tag_mode = ABORT;\n static enum { ERROR, DROP, REWRITE } tag_of_filtered_mode = ABORT;\n static int fake_missing_tagger;\n static int no_data;\n-static int full_tree = 0;\n+static int full_tree;\n \n static int parse_opt_signed_tag_mode(const struct option *opt,\n \t\t\t\t     const char *arg, int unset)\n@@ -588,6 +588,8 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix)\n \t\t\t     \"Import marks from this file\"),\n \t\tOPT_BOOLEAN(0, \"fake-missing-tagger\", &fake_missing_tagger,\n \t\t\t     \"Fake a tagger when tags lack one\"),\n+\t\tOPT_BOOLEAN(0, \"full-tree\", &full_tree,\n+\t\t\t     \"Output full tree for each commit\"),\n \t\t{ OPTION_NEGBIT, 0, \"data\", &no_data, NULL,\n \t\t\t\"Skip output of blob data\",\n \t\t\tPARSE_OPT_NOARG | PARSE_OPT_NEGHELP, NULL, 1 },\ndiff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh\nindex 6069e1f..d831404 100755\n--- a/t/t9350-fast-export.sh\n+++ b/t/t9350-fast-export.sh\n@@ -364,6 +364,11 @@ test_expect_success 'path limiting with import-marks does not lose unmodified fi\n \tgit fast-export --import-marks=marks simple -- file file0 | grep file0\n '\n \n+test_expect_success 'full-tree re-shows unmodified files'        '\n+\tgit checkout -f simple &&\n+\ttest $(git fast-export --full-tree simple | grep -c file0) -eq 3\n+'\n+\n test_expect_success 'set-up a few more tags for tag export tests' '\n \tgit checkout -f master &&\n \tHEAD_TREE=`git show -s --pretty=raw HEAD | grep tree | sed \"s/tree //\"` &&\n-- \n1.6.6.1\n"}]}