# [PATCH 0/8] transport-helper: refspec support 9 messages from 2014-04-20 to 2014-04-20. Participants: Felipe Contreras. Thread: https://gitlist.dev/t/36452 ## Felipe Contreras, 2014-04-20 18:59 Subject: [PATCH 0/8] transport-helper: refspec support Message-ID: <1398020370-11406-1-git-send-email-felipe.contreras@gmail.com> URL: https://gitlist.dev/e/1398020370-11406-1-git-send-email-felipe.contreras%40gmail.com ``` Hi, Here are the patches that allow transport helpers to be completely transparent by adding the remaining missing feature: pushing refspecs and deleting branches (old:new :delete). These were already sent before, but dropped in later patch series because Junio didn't like some name. Apparently Git v2.0 is already advertising this feature, but it's missing the actual patches. Felipe Contreras (8): fast-export: improve argument parsing fast-export: add new --refspec option transport-helper: add support for old:new refspec transport-helper: add support to push symbolic refs fast-import: add support to delete refs fast-export: add support to delete refs transport-helper: add support to delete branches transport-helper: remove unnecessary strbuf resets Documentation/git-fast-export.txt | 4 ++++ Documentation/git-fast-import.txt | 3 +++ builtin/fast-export.c | 49 ++++++++++++++++++++++++++++++++++++++- fast-import.c | 13 ++++++++--- t/t5801-remote-helpers.sh | 18 +++++++++++++- t/t9300-fast-import.sh | 18 ++++++++++++++ t/t9350-fast-export.sh | 18 ++++++++++++++ transport-helper.c | 32 ++++++++++++++++++------- 8 files changed, 141 insertions(+), 14 deletions(-) -- 1.9.1+fc3.9.gc73078e ``` ## Felipe Contreras, 2014-04-20 18:59 Subject: [PATCH 1/8] fast-export: improve argument parsing Message-ID: <1398020370-11406-2-git-send-email-felipe.contreras@gmail.com> URL: https://gitlist.dev/e/1398020370-11406-2-git-send-email-felipe.contreras%40gmail.com In-Reply-To: <1398020370-11406-1-git-send-email-felipe.contreras@gmail.com> ``` We don't want to pass arguments specific to fast-export to setup_revisions. Signed-off-by: Felipe Contreras Signed-off-by: Junio C Hamano --- builtin/fast-export.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/builtin/fast-export.c b/builtin/fast-export.c index b8d8a3a..bc3490c 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -701,8 +701,9 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix) revs.topo_order = 1; revs.show_source = 1; revs.rewrite_parents = 1; + argc = parse_options(argc, argv, prefix, options, fast_export_usage, + PARSE_OPT_KEEP_ARGV0 | PARSE_OPT_KEEP_UNKNOWN); argc = setup_revisions(argc, argv, &revs, NULL); - argc = parse_options(argc, argv, prefix, options, fast_export_usage, 0); if (argc > 1) usage_with_options (fast_export_usage, options); -- 1.9.1+fc3.9.gc73078e ``` ## Felipe Contreras, 2014-04-20 18:59 Subject: [PATCH 2/8] fast-export: add new --refspec option Message-ID: <1398020370-11406-3-git-send-email-felipe.contreras@gmail.com> URL: https://gitlist.dev/e/1398020370-11406-3-git-send-email-felipe.contreras%40gmail.com In-Reply-To: <1398020370-11406-1-git-send-email-felipe.contreras@gmail.com> ``` So that we can convert the exported ref names. Signed-off-by: Felipe Contreras Signed-off-by: Junio C Hamano --- Documentation/git-fast-export.txt | 4 ++++ builtin/fast-export.c | 32 ++++++++++++++++++++++++++++++++ t/t9350-fast-export.sh | 7 +++++++ 3 files changed, 43 insertions(+) diff --git a/Documentation/git-fast-export.txt b/Documentation/git-fast-export.txt index 85f1f30..221506b 100644 --- a/Documentation/git-fast-export.txt +++ b/Documentation/git-fast-export.txt @@ -105,6 +105,10 @@ marks the same across runs. in the commit (as opposed to just listing the files which are different from the commit's first parent). +--refspec:: + Apply the specified refspec to each ref exported. Multiple of them can + be specified. + [...]:: A list of arguments, acceptable to 'git rev-parse' and 'git rev-list', that specifies the specific objects and references diff --git a/builtin/fast-export.c b/builtin/fast-export.c index bc3490c..ad9c17e 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -17,6 +17,7 @@ #include "utf8.h" #include "parse-options.h" #include "quote.h" +#include "remote.h" static const char *fast_export_usage[] = { N_("git fast-export [rev-list-opts]"), @@ -31,6 +32,8 @@ static int use_done_feature; static int no_data; static int full_tree; static struct string_list extra_refs = STRING_LIST_INIT_NODUP; +static struct refspec *refspecs; +static int refspecs_nr; static int parse_opt_signed_tag_mode(const struct option *opt, const char *arg, int unset) @@ -525,6 +528,15 @@ static void get_tags_and_duplicates(struct rev_cmdline_info *info) if (dwim_ref(e->name, strlen(e->name), sha1, &full_name) != 1) continue; + if (refspecs) { + char *private; + private = apply_refspecs(refspecs, refspecs_nr, full_name); + if (private) { + free(full_name); + full_name = private; + } + } + commit = get_commit(e, full_name); if (!commit) { warning("%s: Unexpected object of type %s, skipping.", @@ -668,6 +680,7 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix) struct commit *commit; char *export_filename = NULL, *import_filename = NULL; uint32_t lastimportid; + struct string_list refspecs_list = STRING_LIST_INIT_NODUP; struct option options[] = { OPT_INTEGER(0, "progress", &progress, N_("show progress after objects")), @@ -688,6 +701,8 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix) OPT_BOOL(0, "use-done-feature", &use_done_feature, N_("Use the done feature to terminate the stream")), OPT_BOOL(0, "no-data", &no_data, N_("Skip output of blob data")), + OPT_STRING_LIST(0, "refspec", &refspecs_list, N_("refspec"), + N_("Apply refspec to exported refs")), OPT_END() }; @@ -707,6 +722,21 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix) if (argc > 1) usage_with_options (fast_export_usage, options); + if (refspecs_list.nr) { + const char **refspecs_str; + int i; + + refspecs_str = xmalloc(sizeof(*refspecs_str) * refspecs_list.nr); + for (i = 0; i < refspecs_list.nr; i++) + refspecs_str[i] = refspecs_list.items[i].string; + + refspecs_nr = refspecs_list.nr; + refspecs = parse_fetch_refspec(refspecs_nr, refspecs_str); + + string_list_clear(&refspecs_list, 1); + free(refspecs_str); + } + if (use_done_feature) printf("feature done\n"); @@ -741,5 +771,7 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix) if (use_done_feature) printf("done\n"); + free_refspec(refspecs_nr, refspecs); + return 0; } diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index 2312dec..3d475af 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -504,4 +504,11 @@ test_expect_success 'refs are updated even if no commits need to be exported' ' test_cmp expected actual ' +test_expect_success 'use refspec' ' + git fast-export --refspec refs/heads/master:refs/heads/foobar master | \ + grep "^commit " | sort | uniq > actual && + echo "commit refs/heads/foobar" > expected && + test_cmp expected actual +' + test_done -- 1.9.1+fc3.9.gc73078e ``` ## Felipe Contreras, 2014-04-20 18:59 Subject: [PATCH 3/8] transport-helper: add support for old:new refspec Message-ID: <1398020370-11406-4-git-send-email-felipe.contreras@gmail.com> URL: https://gitlist.dev/e/1398020370-11406-4-git-send-email-felipe.contreras%40gmail.com In-Reply-To: <1398020370-11406-1-git-send-email-felipe.contreras@gmail.com> ``` By using fast-export's new --refspec option. Signed-off-by: Felipe Contreras Signed-off-by: Junio C Hamano --- t/t5801-remote-helpers.sh | 2 +- transport-helper.c | 13 ++++++++++--- 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/t/t5801-remote-helpers.sh b/t/t5801-remote-helpers.sh index 25fd2e7..52b3c99 100755 --- a/t/t5801-remote-helpers.sh +++ b/t/t5801-remote-helpers.sh @@ -87,7 +87,7 @@ test_expect_success 'push new branch by name' ' compare_refs local HEAD server refs/heads/new-name ' -test_expect_failure 'push new branch with old:new refspec' ' +test_expect_success 'push new branch with old:new refspec' ' (cd local && git push origin new-name:new-refspec ) && diff --git a/transport-helper.c b/transport-helper.c index 86e1679..3f45c47 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -856,7 +856,7 @@ static int push_refs_with_export(struct transport *transport, struct ref *ref; struct child_process *helper, exporter; struct helper_data *data = transport->data; - struct string_list revlist_args = STRING_LIST_INIT_NODUP; + struct string_list revlist_args = STRING_LIST_INIT_DUP; struct strbuf buf = STRBUF_INIT; if (!data->refspecs) @@ -894,8 +894,13 @@ static int push_refs_with_export(struct transport *transport, free(private); if (ref->peer_ref) { - if (strcmp(ref->peer_ref->name, ref->name)) - die("remote-helpers do not support old:new syntax"); + if (strcmp(ref->name, ref->peer_ref->name)) { + struct strbuf buf = STRBUF_INIT; + strbuf_addf(&buf, "%s:%s", ref->peer_ref->name, ref->name); + string_list_append(&revlist_args, "--refspec"); + string_list_append(&revlist_args, buf.buf); + strbuf_release(&buf); + } string_list_append(&revlist_args, ref->peer_ref->name); } } @@ -903,6 +908,8 @@ static int push_refs_with_export(struct transport *transport, if (get_exporter(transport, &exporter, &revlist_args)) die("Couldn't run fast-export"); + string_list_clear(&revlist_args, 1); + if (finish_command(&exporter)) die("Error while running fast-export"); push_update_refs_status(data, remote_refs, flags); -- 1.9.1+fc3.9.gc73078e ``` ## Felipe Contreras, 2014-04-20 18:59 Subject: [PATCH 4/8] transport-helper: add support to push symbolic refs Message-ID: <1398020370-11406-5-git-send-email-felipe.contreras@gmail.com> URL: https://gitlist.dev/e/1398020370-11406-5-git-send-email-felipe.contreras%40gmail.com In-Reply-To: <1398020370-11406-1-git-send-email-felipe.contreras@gmail.com> ``` For example 'HEAD'. Signed-off-by: Felipe Contreras --- t/t5801-remote-helpers.sh | 8 ++++++++ transport-helper.c | 11 ++++++++++- 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/t/t5801-remote-helpers.sh b/t/t5801-remote-helpers.sh index 52b3c99..8288669 100755 --- a/t/t5801-remote-helpers.sh +++ b/t/t5801-remote-helpers.sh @@ -94,6 +94,14 @@ test_expect_success 'push new branch with old:new refspec' ' compare_refs local HEAD server refs/heads/new-refspec ' +test_expect_success 'push new branch with HEAD:new refspec' ' + (cd local && + git checkout new-name + git push origin HEAD:new-refspec-2 + ) && + compare_refs local HEAD server refs/heads/new-refspec-2 +' + test_expect_success 'forced push' ' (cd local && git checkout -b force-test && diff --git a/transport-helper.c b/transport-helper.c index 3f45c47..f74eb57 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -896,7 +896,16 @@ static int push_refs_with_export(struct transport *transport, if (ref->peer_ref) { if (strcmp(ref->name, ref->peer_ref->name)) { struct strbuf buf = STRBUF_INIT; - strbuf_addf(&buf, "%s:%s", ref->peer_ref->name, ref->name); + const char *name; + int flag; + + /* Follow symbolic refs (mainly for HEAD). */ + name = resolve_ref_unsafe(ref->peer_ref->name, sha1, 1, &flag); + if (!name || !(flag & REF_ISSYMREF)) + name = ref->peer_ref->name; + + strbuf_addf(&buf, "%s:%s", name, ref->name); + string_list_append(&revlist_args, "--refspec"); string_list_append(&revlist_args, buf.buf); strbuf_release(&buf); -- 1.9.1+fc3.9.gc73078e ``` ## Felipe Contreras, 2014-04-20 18:59 Subject: [PATCH 5/8] fast-import: add support to delete refs Message-ID: <1398020370-11406-6-git-send-email-felipe.contreras@gmail.com> URL: https://gitlist.dev/e/1398020370-11406-6-git-send-email-felipe.contreras%40gmail.com In-Reply-To: <1398020370-11406-1-git-send-email-felipe.contreras@gmail.com> ``` Signed-off-by: Felipe Contreras Signed-off-by: Junio C Hamano --- Documentation/git-fast-import.txt | 3 +++ fast-import.c | 13 ++++++++++--- t/t9300-fast-import.sh | 18 ++++++++++++++++++ 3 files changed, 31 insertions(+), 3 deletions(-) diff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt index fd22a9a..c32a9a5 100644 --- a/Documentation/git-fast-import.txt +++ b/Documentation/git-fast-import.txt @@ -483,6 +483,9 @@ Marks must be declared (via `mark`) before they can be used. * Any valid Git SHA-1 expression that resolves to a commit. See ``SPECIFYING REVISIONS'' in linkgit:gitrevisions[7] for details. +* The special null SHA-1 (40 zeros) specifies that the branch is to be + removed. + The special case of restarting an incremental import from the current branch value should be written as: ---- diff --git a/fast-import.c b/fast-import.c index fb4738d..6707a66 100644 --- a/fast-import.c +++ b/fast-import.c @@ -248,6 +248,7 @@ struct branch { uintmax_t last_commit; uintmax_t num_notes; unsigned active : 1; + unsigned delete : 1; unsigned pack_id : PACK_ID_BITS; unsigned char sha1[20]; }; @@ -1681,10 +1682,13 @@ static int update_branch(struct branch *b) struct ref_lock *lock; unsigned char old_sha1[20]; - if (is_null_sha1(b->sha1)) - return 0; if (read_ref(b->name, old_sha1)) hashclr(old_sha1); + if (is_null_sha1(b->sha1)) { + if (b->delete) + delete_ref(b->name, old_sha1, 0); + return 0; + } lock = lock_any_ref_for_update(b->name, old_sha1, 0, NULL); if (!lock) return error("Unable to lock %s", b->name); @@ -2611,8 +2615,11 @@ static int parse_from(struct branch *b) free(buf); } else parse_from_existing(b); - } else if (!get_sha1(from, b->sha1)) + } else if (!get_sha1(from, b->sha1)) { parse_from_existing(b); + if (is_null_sha1(b->sha1)) + b->delete = 1; + } else die("Invalid ref name or SHA1 expression: %s", from); diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh index 27263df..5fc9ef2 100755 --- a/t/t9300-fast-import.sh +++ b/t/t9300-fast-import.sh @@ -2999,4 +2999,22 @@ test_expect_success 'T: ls root tree' ' test_cmp expect actual ' +test_expect_success 'T: delete branch' ' + git branch to-delete && + git fast-import <<-EOF && + reset refs/heads/to-delete + from 0000000000000000000000000000000000000000 + EOF + test_must_fail git rev-parse --verify refs/heads/to-delete +' + +test_expect_success 'T: empty reset doesnt delete branch' ' + git branch not-to-delete && + git fast-import <<-EOF && + reset refs/heads/not-to-delete + EOF + git show-ref && + git rev-parse --verify refs/heads/not-to-delete +' + test_done -- 1.9.1+fc3.9.gc73078e ``` ## Felipe Contreras, 2014-04-20 18:59 Subject: [PATCH 6/8] fast-export: add support to delete refs Message-ID: <1398020370-11406-7-git-send-email-felipe.contreras@gmail.com> URL: https://gitlist.dev/e/1398020370-11406-7-git-send-email-felipe.contreras%40gmail.com In-Reply-To: <1398020370-11406-1-git-send-email-felipe.contreras@gmail.com> ``` Signed-off-by: Felipe Contreras Signed-off-by: Junio C Hamano --- builtin/fast-export.c | 14 ++++++++++++++ t/t9350-fast-export.sh | 11 +++++++++++ 2 files changed, 25 insertions(+) diff --git a/builtin/fast-export.c b/builtin/fast-export.c index ad9c17e..ef44816 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -673,6 +673,19 @@ static void import_marks(char *input_file) fclose(f); } +static void handle_deletes(void) +{ + int i; + for (i = 0; i < refspecs_nr; i++) { + struct refspec *refspec = &refspecs[i]; + if (*refspec->src) + continue; + + printf("reset %s\nfrom %s\n\n", + refspec->dst, sha1_to_hex(null_sha1)); + } +} + int cmd_fast_export(int argc, const char **argv, const char *prefix) { struct rev_info revs; @@ -764,6 +777,7 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix) } handle_tags_and_duplicates(); + handle_deletes(); if (export_filename && lastimportid != last_idnum) export_marks(export_filename); diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index 3d475af..66c8b0a 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -511,4 +511,15 @@ test_expect_success 'use refspec' ' test_cmp expected actual ' +test_expect_success 'delete refspec' ' + git branch to-delete && + git fast-export --refspec :refs/heads/to-delete to-delete ^to-delete > actual && + cat > expected <<-EOF && + reset refs/heads/to-delete + from 0000000000000000000000000000000000000000 + + EOF + test_cmp expected actual +' + test_done -- 1.9.1+fc3.9.gc73078e ``` ## Felipe Contreras, 2014-04-20 18:59 Subject: [PATCH 7/8] transport-helper: add support to delete branches Message-ID: <1398020370-11406-8-git-send-email-felipe.contreras@gmail.com> URL: https://gitlist.dev/e/1398020370-11406-8-git-send-email-felipe.contreras%40gmail.com In-Reply-To: <1398020370-11406-1-git-send-email-felipe.contreras@gmail.com> ``` For remote-helpers that use 'export' to push. Signed-off-by: Felipe Contreras Signed-off-by: Junio C Hamano --- t/t5801-remote-helpers.sh | 8 ++++++++ transport-helper.c | 24 +++++++++++++----------- 2 files changed, 21 insertions(+), 11 deletions(-) diff --git a/t/t5801-remote-helpers.sh b/t/t5801-remote-helpers.sh index 8288669..aec924b 100755 --- a/t/t5801-remote-helpers.sh +++ b/t/t5801-remote-helpers.sh @@ -102,6 +102,14 @@ test_expect_success 'push new branch with HEAD:new refspec' ' compare_refs local HEAD server refs/heads/new-refspec-2 ' +test_expect_success 'push delete branch' ' + (cd local && + git push origin :new-name + ) && + test_must_fail git --git-dir="server/.git" \ + rev-parse --verify refs/heads/new-name +' + test_expect_success 'forced push' ' (cd local && git checkout -b force-test && diff --git a/transport-helper.c b/transport-helper.c index f74eb57..36fbf93 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -882,9 +882,6 @@ static int push_refs_with_export(struct transport *transport, char *private; unsigned char sha1[20]; - if (ref->deletion) - die("remote-helpers do not support ref deletion"); - private = apply_refspecs(data->refspecs, data->refspec_nr, ref->name); if (private && !get_sha1(private, sha1)) { strbuf_addf(&buf, "^%s", private); @@ -896,21 +893,26 @@ static int push_refs_with_export(struct transport *transport, if (ref->peer_ref) { if (strcmp(ref->name, ref->peer_ref->name)) { struct strbuf buf = STRBUF_INIT; - const char *name; - int flag; - /* Follow symbolic refs (mainly for HEAD). */ - name = resolve_ref_unsafe(ref->peer_ref->name, sha1, 1, &flag); - if (!name || !(flag & REF_ISSYMREF)) - name = ref->peer_ref->name; + if (!ref->deletion) { + const char *name; + int flag; + + /* Follow symbolic refs (mainly for HEAD). */ + name = resolve_ref_unsafe(ref->peer_ref->name, sha1, 1, &flag); + if (!name || !(flag & REF_ISSYMREF)) + name = ref->peer_ref->name; - strbuf_addf(&buf, "%s:%s", name, ref->name); + strbuf_addf(&buf, "%s:%s", name, ref->name); + } else + strbuf_addf(&buf, ":%s", ref->name); string_list_append(&revlist_args, "--refspec"); string_list_append(&revlist_args, buf.buf); strbuf_release(&buf); } - string_list_append(&revlist_args, ref->peer_ref->name); + if (!ref->deletion) + string_list_append(&revlist_args, ref->peer_ref->name); } } -- 1.9.1+fc3.9.gc73078e ``` ## Felipe Contreras, 2014-04-20 18:59 Subject: [PATCH 8/8] transport-helper: remove unnecessary strbuf resets Message-ID: <1398020370-11406-9-git-send-email-felipe.contreras@gmail.com> URL: https://gitlist.dev/e/1398020370-11406-9-git-send-email-felipe.contreras%40gmail.com In-Reply-To: <1398020370-11406-1-git-send-email-felipe.contreras@gmail.com> ``` Signed-off-by: Felipe Contreras --- transport-helper.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/transport-helper.c b/transport-helper.c index 36fbf93..a90094d 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -876,8 +876,6 @@ static int push_refs_with_export(struct transport *transport, write_constant(helper->in, "export\n"); - strbuf_reset(&buf); - for (ref = remote_refs; ref; ref = ref->next) { char *private; unsigned char sha1[20]; @@ -892,8 +890,6 @@ static int push_refs_with_export(struct transport *transport, if (ref->peer_ref) { if (strcmp(ref->name, ref->peer_ref->name)) { - struct strbuf buf = STRBUF_INIT; - if (!ref->deletion) { const char *name; int flag; -- 1.9.1+fc3.9.gc73078e ```