# [PATCHv2 0/2] non-incremental mode for fast-export 3 messages from 2010-07-17 to 2010-07-17. Participants: Elijah Newren. Thread: https://gitlist.dev/t/24424 ## Elijah Newren, 2010-07-17 17:00 Subject: [PATCHv2 0/2] non-incremental mode for fast-export Message-ID: <1279386051-31112-1-git-send-email-newren@gmail.com> URL: https://gitlist.dev/e/1279386051-31112-1-git-send-email-newren%40gmail.com ``` This patch series fixes silently dropped files in uses of fast-export involving both --import-marks and limiting files by path. It also adds a new feature: a --full-tree option to switch from incremental to comprehensive handling of files in commits. The two changes are related in that the bugfix consists of simply automatically activating the new feature in a case where it is required for correct operation. There is just one change since the last submission * modify one of the new testcases to not depend on the full exact output of fast-export (since it is no longer the same in next) This change enables the series to apply cleanly to next. All tests pass with this series applied to either maint or next now. I'll again note that it may make sense to split this patch series since the first is a bugfix and thus makes sense to apply to maint, while the second (whose testcase was the fragile one) is a feature addition, though a fairly trivial one. Elijah Newren (2): fast-export: Fix dropping of files with --import-marks and path limiting fast-export: Add a --full-tree option Documentation/git-fast-export.txt | 6 ++++++ builtin/fast-export.c | 11 ++++++++++- t/t9350-fast-export.sh | 14 ++++++++++++++ 3 files changed, 30 insertions(+), 1 deletions(-) ``` ## Elijah Newren, 2010-07-17 17:00 Subject: [PATCHv2 1/2] fast-export: Fix dropping of files with --import-marks and path limiting Message-ID: <1279386051-31112-2-git-send-email-newren@gmail.com> URL: https://gitlist.dev/e/1279386051-31112-2-git-send-email-newren%40gmail.com In-Reply-To: <1279386051-31112-1-git-send-email-newren@gmail.com> ``` Since fast-export operates by listing file changes since the (first) parent commit, when using --import-marks and path limiting and using a wider list of paths than in previous runs, files from the new path(s) will silently be omitted from the result unless or until a commit which explicitly changes those files. The resulting repository in such cases is broken and makes no sense. This commit fixes this by having fast-export work with complete trees instead of incremental changes (when both --import-marks and path limiting are used). It works by issuing a 'deleteall' directive with each commit and then listing the full set of files that make up that commit, rather than just showing the list of files that have changed since the (first) parent commit. Signed-off-by: Elijah Newren Acked-by: Sverre Rabbelier --- builtin/fast-export.c | 9 ++++++++- t/t9350-fast-export.sh | 9 +++++++++ 2 files changed, 17 insertions(+), 1 deletions(-) diff --git a/builtin/fast-export.c b/builtin/fast-export.c index c6dd71a..25d13a1 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -27,6 +27,7 @@ static enum { ABORT, VERBATIM, WARN, STRIP } signed_tag_mode = ABORT; static enum { ERROR, DROP, REWRITE } tag_of_filtered_mode = ABORT; static int fake_missing_tagger; static int no_data; +static int full_tree = 0; static int parse_opt_signed_tag_mode(const struct option *opt, const char *arg, int unset) @@ -241,7 +242,8 @@ static void handle_commit(struct commit *commit, struct rev_info *rev) message += 2; if (commit->parents && - get_object_mark(&commit->parents->item->object) != 0) { + get_object_mark(&commit->parents->item->object) != 0 && + !full_tree) { parse_commit(commit->parents->item); diff_tree_sha1(commit->parents->item->tree->object.sha1, commit->tree->object.sha1, "", &rev->diffopt); @@ -281,6 +283,8 @@ static void handle_commit(struct commit *commit, struct rev_info *rev) i++; } + if (full_tree) + printf("deleteall\n"); log_tree_diff_flush(rev); rev->diffopt.output_format = saved_output_format; @@ -608,6 +612,9 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix) if (import_filename) import_marks(import_filename); + if (import_filename && revs.prune_data) + full_tree = 1; + get_tags_and_duplicates(&revs.pending, &extra_refs); if (prepare_revision_walk(&revs)) diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index d43f37c..6069e1f 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -355,6 +355,15 @@ test_expect_failure 'no exact-ref revisions included' ' ) ' +test_expect_success 'path limiting with import-marks does not lose unmodified files' ' + git checkout -b simple marks~2 && + git fast-export --export-marks=marks simple -- file > /dev/null && + echo more content >> file && + test_tick && + git commit -mnext file && + git fast-export --import-marks=marks simple -- file file0 | grep file0 +' + test_expect_success 'set-up a few more tags for tag export tests' ' git checkout -f master && HEAD_TREE=`git show -s --pretty=raw HEAD | grep tree | sed "s/tree //"` && -- 1.6.6.1 ``` ## Elijah Newren, 2010-07-17 17:00 Subject: [PATCHv2 2/2] fast-export: Add a --full-tree option Message-ID: <1279386051-31112-3-git-send-email-newren@gmail.com> URL: https://gitlist.dev/e/1279386051-31112-3-git-send-email-newren%40gmail.com In-Reply-To: <1279386051-31112-2-git-send-email-newren@gmail.com> ``` This option adds symmetry with fast-import, enabling it to also work with complete trees instead of just incremental changes. It works by issuing a 'deleteall' directive with each commit and then listing the full set of files that make up that commit, rather than just showing the list of files that have changed since the (first) parent commit. Note that this functionality is automatically turned on when using --import-marks together with path limiting in order to avoid dropping important but unchanged files. This functionality is desired when using hand-written filters along with 'fast-export | some-filter | fast-import' as it can be easier to write in terms of complete trees than incremental changes. We could avoid the need to add this option by simply always turning it on. While the end result would be identical, it would slow things down slightly by printing many more filenames per commit which goes somewhat against the 'fast' in 'fast-export'. Signed-off-by: Elijah Newren Acked-by: Sverre Rabbelier --- Documentation/git-fast-export.txt | 6 ++++++ builtin/fast-export.c | 4 +++- t/t9350-fast-export.sh | 5 +++++ 3 files changed, 14 insertions(+), 1 deletions(-) diff --git a/Documentation/git-fast-export.txt b/Documentation/git-fast-export.txt index 98ec6b5..8a6a3cb 100644 --- a/Documentation/git-fast-export.txt +++ b/Documentation/git-fast-export.txt @@ -90,6 +90,12 @@ marks the same across runs. resulting stream can only be used by a repository which already contains the necessary objects. +--full-tree:: + This option will cause fast-export to issue a "deleteall" + directive for each commit followed by a full list of all files + in the commit (as opposed to just listing the files which are + different from the commit's first parent). + [git-rev-list-args...]:: A list of arguments, acceptable to 'git rev-parse' and 'git rev-list', that specifies the specific objects and references diff --git a/builtin/fast-export.c b/builtin/fast-export.c index 25d13a1..8c77602 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -27,7 +27,7 @@ static enum { ABORT, VERBATIM, WARN, STRIP } signed_tag_mode = ABORT; static enum { ERROR, DROP, REWRITE } tag_of_filtered_mode = ABORT; static int fake_missing_tagger; static int no_data; -static int full_tree = 0; +static int full_tree; static int parse_opt_signed_tag_mode(const struct option *opt, const char *arg, int unset) @@ -588,6 +588,8 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix) "Import marks from this file"), OPT_BOOLEAN(0, "fake-missing-tagger", &fake_missing_tagger, "Fake a tagger when tags lack one"), + OPT_BOOLEAN(0, "full-tree", &full_tree, + "Output full tree for each commit"), { OPTION_NEGBIT, 0, "data", &no_data, NULL, "Skip output of blob data", PARSE_OPT_NOARG | PARSE_OPT_NEGHELP, NULL, 1 }, diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index 6069e1f..d831404 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -364,6 +364,11 @@ test_expect_success 'path limiting with import-marks does not lose unmodified fi git fast-export --import-marks=marks simple -- file file0 | grep file0 ' +test_expect_success 'full-tree re-shows unmodified files' ' + git checkout -f simple && + test $(git fast-export --full-tree simple | grep -c file0) -eq 3 +' + test_expect_success 'set-up a few more tags for tag export tests' ' git checkout -f master && HEAD_TREE=`git show -s --pretty=raw HEAD | grep tree | sed "s/tree //"` && -- 1.6.6.1 ```