{"thread":{"id":"54090","subject":"[RFC v2 0/1] implement support for negative refspecs","startedAt":"2020-08-21T21:53:34Z","lastAt":"2020-09-24T23:43:07Z","messageCount":10,"participants":["Jacob Keller","Johannes Schindelin","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"404237","messageId":"20200821215247.758978-1-jacob.e.keller@intel.com","threadId":"54090","inReplyTo":null,"subject":"[RFC v2 0/1] implement support for negative refspecs","fromName":"Jacob Keller","fromEmail":"jacob.e.keller@intel.com","sentAt":"2020-08-21T21:52:46Z","receivedAt":"2020-08-21T21:53:34Z","isPatch":false,"sender":{"key":"jacob.e.keller@intel.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"From: Jacob Keller <jacob.keller@gmail.com>\n\nThis series introduces support for negative refspecs. It builds on top of a\ncouple of minor refspec cleanups which I posted at [1].\n\nThe primary motivator for negative refspecs is to allow additional control\nover refspec patterns. Today, both fetch and push support refspecs with\npatterns. This allows expressing a fetch of simple globs. Because refspec\npatterns are simple and not regular expressions, it is not possible to\nexpress some cases that a user might wish.\n\nSuppose you want to fetch all remote branches *except* for a\nspecific one. For example, a repository which has renamed its primary branch\nto \"main\" but has left the older branch \"master\" in order to avoid breaking\nexisting workflows. You wish to have only main, but not master, in your\nlocal copy. To do this, you could modify your remote config to explicitly\nlist every branch you want to fetch except for the refs/heads/master. If\nthere are many branches, this would be tedious. Further, it requires manual\nintervention for each new branch.\n\nWith negative refspecs, you can simply fetch\nrefs/heads/*:refs/remotes/origin/* along with ^refs/heads/master and git\nwill now exclude that branch from the fetch.\n\n[1]: https://lore.kernel.org/git/20200821214820.757222-1-jacob.e.keller@intel.com/T/#t\n\nRange diff since v1:\n\n1:  6a53c459e868 ! 1:  04cdd0313529 refspec: add support for negative refspecs\n    @@ Commit message\n         This is similar to how negative pathspecs work.\n     \n         Signed-off-by: Jacob Keller <jacob.keller@gmail.com>\n    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>\n     \n      ## builtin/fetch.c ##\n     @@ builtin/fetch.c: static struct ref *get_ref_map(struct remote *remote,\n      \t\ttail = &rm->next;\n      \t}\n      \n    -+\t/* apply any negative refspecs now to prune the list of refs */\n    -+\tref_map = apply_negative_refspecs(ref_map, rs);\n    ++\t/*\n    ++\t * apply negative refspecs first, before we remove duplicates. This is\n    ++\t * necessary as negative refspecs might remove an otherwise conflicting\n    ++\t * duplicate.\n    ++\t */\n    ++\tif (rs->nr)\n    ++\t\tref_map = apply_negative_refspecs(ref_map, rs);\n    ++\telse\n    ++\t\tref_map = apply_negative_refspecs(ref_map, &remote->fetch);\n     +\n      \tref_map = ref_remove_duplicates(ref_map);\n      \n    @@ refspec.c: static int parse_refspec(struct refspec_item *item, const char *refsp\n      \t * Before going on, special case \":\" (or \"+:\") as a refspec\n      \t * for pushing matching refs.\n     @@ refspec.c: static int parse_refspec(struct refspec_item *item, const char *refspec, int fet\n    + \n    + \tllen = (rhs ? (rhs - lhs - 1) : strlen(lhs));\n    + \tif (1 <= llen && memchr(lhs, '*', llen)) {\n    +-\t\tif ((rhs && !is_glob) || (!rhs && fetch))\n    ++\t\tif ((rhs && !is_glob) || (!rhs && !item->negative && fetch))\n    + \t\t\treturn 0;\n    + \t\tis_glob = 1;\n    + \t} else if (rhs && is_glob) {\n    +@@ refspec.c: static int parse_refspec(struct refspec_item *item, const char *refspec, int fet\n      \titem->src = xstrndup(lhs, llen);\n      \tflags = REFNAME_ALLOW_ONELEVEL | (is_glob ? REFNAME_REFSPEC_PATTERN : 0);\n      \n    @@ refspec.c: static int parse_refspec(struct refspec_item *item, const char *refsp\n      \tif (fetch) {\n      \t\tstruct object_id unused;\n      \n    +@@ refspec.c: void refspec_ref_prefixes(const struct refspec *rs,\n    + \t\tconst struct refspec_item *item = &rs->items[i];\n    + \t\tconst char *prefix = NULL;\n    + \n    +-\t\tif (item->exact_sha1)\n    ++\t\tif (item->exact_sha1 || item->negative)\n    + \t\t\tcontinue;\n    + \t\tif (rs->fetch == REFSPEC_FETCH)\n    + \t\t\tprefix = item->src;\n     \n      ## refspec.h ##\n     @@\n    @@ refspec.h: struct refspec_item {\n      \tchar *dst;\n     \n      ## remote.c ##\n    -@@ remote.c: static int match_explicit(struct ref *src, struct ref *dst,\n    - \tconst char *dst_value = rs->dst;\n    - \tchar *dst_guess;\n    - \n    --\tif (rs->pattern || rs->matching)\n    -+\tif (rs->pattern || rs->matching || rs->negative)\n    - \t\treturn 0;\n    - \n    - \tmatched_src = matched_dst = NULL;\n    -@@ remote.c: static char *get_ref_match(const struct refspec *rs, const struct ref *ref,\n    - \tint matching_refs = -1;\n    - \tfor (i = 0; i < rs->nr; i++) {\n    - \t\tconst struct refspec_item *item = &rs->items[i];\n    -+\n    -+\t\tif (item->negative)\n    -+\t\t\tcontinue;\n    -+\n    - \t\tif (item->matching &&\n    - \t\t    (matching_refs == -1 || item->force)) {\n    - \t\t\tmatching_refs = i;\n    -@@ remote.c: int match_push_refs(struct ref *src, struct ref **dst,\n    - \t\tstring_list_clear(&src_ref_index, 0);\n    - \t}\n    - \n    -+\t*dst = apply_negative_refspecs(*dst, rs);\n    -+\n    - \tif (errs)\n    - \t\treturn -1;\n    - \treturn 0;\n    -@@ remote.c: int get_fetch_map(const struct ref *remote_refs,\n    - {\n    - \tstruct ref *ref_map, **rmp;\n    - \n    -+\tif (refspec->negative)\n    -+\t\treturn 0;\n    -+\n    - \tif (refspec->pattern) {\n    - \t\tref_map = get_expanded_map(remote_refs, refspec);\n    - \t} else {\n    -@@ remote.c: int get_fetch_map(const struct ref *remote_refs,\n    - \treturn 0;\n    +@@ remote.c: static int match_name_with_pattern(const char *key, const char *name,\n    + \treturn ret;\n      }\n      \n     +static int refspec_match(const struct refspec_item *refspec,\n    @@ remote.c: int get_fetch_map(const struct ref *remote_refs,\n     +\treturn ref_map;\n     +}\n     +\n    - int resolve_remote_symref(struct ref *ref, struct ref *list)\n    + static void query_refspecs_multiple(struct refspec *rs,\n    + \t\t\t\t    struct refspec_item *query,\n    + \t\t\t\t    struct string_list *results)\n      {\n    - \tif (!ref->symref)\n    +-\tint i;\n    ++\tint i, matched_negative = 0;\n    + \tint find_src = !query->src;\n    ++\tstruct string_list reversed = STRING_LIST_INIT_NODUP;\n    ++\tconst char *needle = find_src ? query->dst : query->src;\n    ++\tchar **result = find_src ? &query->src : &query->dst;\n    + \n    + \tif (find_src && !query->dst)\n    + \t\tBUG(\"query_refspecs_multiple: need either src or dst\");\n    + \n    ++\t/*\n    ++\t * If a ref matches any of the negative refspecs, then we should treat\n    ++\t * it as not matching this query. Note that negative refspecs apply to\n    ++\t * the source but we're checking only the destination. Reverse and\n    ++\t * capture any pattern refspecs in order to see if the source would\n    ++\t * have matched a negative refspec.\n    ++\t */\n    ++\tfor (i = 0; i < rs->nr; i++) {\n    ++\t\tstruct refspec_item *refspec = &rs->items[i];\n    ++\t\tchar *expn_name;\n    ++\n    ++\t\tif (refspec->negative)\n    ++\t\t\tcontinue;\n    ++\n    ++\t\t/* Note the reversal of src and dst */\n    ++\t\tif (refspec->pattern) {\n    ++\t\t\tconst char *key = refspec->dst ?: refspec->src;\n    ++\t\t\tconst char *value = refspec->src;\n    ++\n    ++\t\t\tif (match_name_with_pattern(key, needle, value, &expn_name))\n    ++\t\t\t\tstring_list_append_nodup(&reversed, expn_name);\n    ++\t\t} else {\n    ++\t\t\tif (!strcmp(needle, refspec->src))\n    ++\t\t\t\tstring_list_append(&reversed, refspec->src);\n    ++\t\t}\n    ++\t}\n    ++\n    ++\tfor (i = 0; !matched_negative && i < reversed.nr; i++) {\n    ++\t\tif (omit_name_by_refspec(reversed.items[i].string, rs))\n    ++\t\t\tmatched_negative = 1;\n    ++\t}\n    ++\n    ++\tstring_list_clear(&reversed, 0);\n    ++\n    ++\tif (matched_negative)\n    ++\t\treturn;\n    ++\n    + \tfor (i = 0; i < rs->nr; i++) {\n    + \t\tstruct refspec_item *refspec = &rs->items[i];\n    + \t\tconst char *key = find_src ? refspec->dst : refspec->src;\n    + \t\tconst char *value = find_src ? refspec->src : refspec->dst;\n    +-\t\tconst char *needle = find_src ? query->dst : query->src;\n    +-\t\tchar **result = find_src ? &query->src : &query->dst;\n    + \n    +-\t\tif (!refspec->dst)\n    ++\t\tif (!refspec->dst || refspec->negative)\n    + \t\t\tcontinue;\n    + \t\tif (refspec->pattern) {\n    + \t\t\tif (match_name_with_pattern(key, needle, value, result))\n    +@@ remote.c: static void query_refspecs_multiple(struct refspec *rs,\n    + \n    + int query_refspecs(struct refspec *rs, struct refspec_item *query)\n    + {\n    +-\tint i;\n    ++\tint i, matched_negative = 0;\n    + \tint find_src = !query->src;\n    ++\tstruct string_list reversed = STRING_LIST_INIT_NODUP;\n    + \tconst char *needle = find_src ? query->dst : query->src;\n    + \tchar **result = find_src ? &query->src : &query->dst;\n    + \n    + \tif (find_src && !query->dst)\n    + \t\tBUG(\"query_refspecs: need either src or dst\");\n    + \n    ++\t/*\n    ++\t * If a ref matches any of the negative refspecs, then we should treat\n    ++\t * it as not matching this query. Note that negative refspecs apply to\n    ++\t * the source but we're checking only the destination. Reverse and\n    ++\t * capture any pattern refspecs in order to see if the source would\n    ++\t * have matched a negative refspec.\n    ++\t */\n    ++\tfor (i = 0; i < rs->nr; i++) {\n    ++\t\tstruct refspec_item *refspec = &rs->items[i];\n    ++\t\tchar *expn_name;\n    ++\n    ++\t\tif (refspec->negative)\n    ++\t\t\tcontinue;\n    ++\n    ++\t\t/* Note the reversal of src and dst */\n    ++\t\tif (refspec->pattern) {\n    ++\t\t\tconst char *key = refspec->dst ?: refspec->src;\n    ++\t\t\tconst char *value = refspec->src;\n    ++\n    ++\t\t\tif (match_name_with_pattern(key, needle, value, &expn_name))\n    ++\t\t\t\tstring_list_append_nodup(&reversed, expn_name);\n    ++\t\t} else {\n    ++\t\t\tif (!strcmp(needle, refspec->src))\n    ++\t\t\t\tstring_list_append(&reversed, refspec->src);\n    ++\t\t}\n    ++\t}\n    ++\n    ++\tfor (i = 0; !matched_negative && i < reversed.nr; i++) {\n    ++\t\tif (omit_name_by_refspec(reversed.items[i].string, rs))\n    ++\t\t\tmatched_negative = 1;\n    ++\t}\n    ++\n    ++\tstring_list_clear(&reversed, 0);\n    ++\n    ++\tif (matched_negative)\n    ++\t\treturn -1;\n    ++\n    + \tfor (i = 0; i < rs->nr; i++) {\n    + \t\tstruct refspec_item *refspec = &rs->items[i];\n    + \t\tconst char *key = find_src ? refspec->dst : refspec->src;\n    + \t\tconst char *value = find_src ? refspec->src : refspec->dst;\n    + \n    +-\t\tif (!refspec->dst)\n    ++\t\tif (!refspec->dst || refspec->negative)\n    + \t\t\tcontinue;\n    + \t\tif (refspec->pattern) {\n    + \t\t\tif (match_name_with_pattern(key, needle, value, result)) {\n    +@@ remote.c: static int match_explicit(struct ref *src, struct ref *dst,\n    + \tconst char *dst_value = rs->dst;\n    + \tchar *dst_guess;\n    + \n    +-\tif (rs->pattern || rs->matching)\n    ++\tif (rs->pattern || rs->matching || rs->negative)\n    + \t\treturn 0;\n    + \n    + \tmatched_src = matched_dst = NULL;\n    +@@ remote.c: static char *get_ref_match(const struct refspec *rs, const struct ref *ref,\n    + \tint matching_refs = -1;\n    + \tfor (i = 0; i < rs->nr; i++) {\n    + \t\tconst struct refspec_item *item = &rs->items[i];\n    ++\n    ++\t\tif (item->negative)\n    ++\t\t\tcontinue;\n    ++\n    + \t\tif (item->matching &&\n    + \t\t    (matching_refs == -1 || item->force)) {\n    + \t\t\tmatching_refs = i;\n    +@@ remote.c: int check_push_refs(struct ref *src, struct refspec *rs)\n    + \tfor (i = 0; i < rs->nr; i++) {\n    + \t\tstruct refspec_item *item = &rs->items[i];\n    + \n    +-\t\tif (item->pattern || item->matching)\n    ++\t\tif (item->pattern || item->matching || item->negative)\n    + \t\t\tcontinue;\n    + \n    + \t\tret |= match_explicit_lhs(src, item, NULL, NULL);\n    +@@ remote.c: int match_push_refs(struct ref *src, struct ref **dst,\n    + \t\tstring_list_clear(&src_ref_index, 0);\n    + \t}\n    + \n    ++\t*dst = apply_negative_refspecs(*dst, rs);\n    ++\n    + \tif (errs)\n    + \t\treturn -1;\n    + \treturn 0;\n    +@@ remote.c: int get_fetch_map(const struct ref *remote_refs,\n    + {\n    + \tstruct ref *ref_map, **rmp;\n    + \n    ++\tif (refspec->negative)\n    ++\t\treturn 0;\n    ++\n    + \tif (refspec->pattern) {\n    + \t\tref_map = get_expanded_map(remote_refs, refspec);\n    + \t} else {\n     \n      ## remote.h ##\n     @@ remote.h: int resolve_remote_symref(struct ref *ref, struct ref *list);\n    @@ remote.h: void set_ref_status_for_push(struct ref *remote_refs, int send_mirror,\n       *\n       * *tail is the pointer to the tail pointer of the list of results\n       * beforehand, and will be set to the tail pointer of the list of\n    +\n    + ## t/t5582-fetch-negative-refspec.sh (new) ##\n    +@@\n    ++#!/bin/sh\n    ++# Copyright (c) 2020, Jacob Keller.\n    ++\n    ++test_description='\"git fetch\" with negative refspecs.\n    ++\n    ++'\n    ++\n    ++. ./test-lib.sh\n    ++\n    ++test_expect_success setup '\n    ++\techo >file original &&\n    ++\tgit add file &&\n    ++\tgit commit -a -m original\n    ++'\n    ++\n    ++test_expect_success \"clone and setup child repos\" '\n    ++\tgit clone . one &&\n    ++\t(\n    ++\t\tcd one &&\n    ++\t\techo >file updated by one &&\n    ++\t\tgit commit -a -m \"updated by one\" &&\n    ++\t\tgit switch -c alternate &&\n    ++\t\techo >file updated again by one &&\n    ++\t\tgit commit -a -m \"updated by one again\" &&\n    ++\t\tgit switch master\n    ++\t) &&\n    ++\tgit clone . two &&\n    ++\t(\n    ++\t\tcd two &&\n    ++\t\tgit config branch.master.remote one &&\n    ++\t\tgit config remote.one.url ../one/.git/ &&\n    ++\t\tgit config remote.one.fetch +refs/heads/*:refs/remotes/one/* &&\n    ++\t\tgit config --add remote.one.fetch ^refs/heads/alternate\n    ++\t) &&\n    ++\tgit clone . three\n    ++'\n    ++\n    ++test_expect_success \"fetch one\" '\n    ++\techo >file updated by origin &&\n    ++\tgit commit -a -m \"updated by origin\" &&\n    ++\t(\n    ++\t\tcd two &&\n    ++\t\ttest_must_fail git rev-parse --verify refs/remotes/one/alternate &&\n    ++\t\tgit fetch one &&\n    ++\t\ttest_must_fail git rev-parse --verify refs/remotes/one/alternate &&\n    ++\t\tgit rev-parse --verify refs/remotes/one/master &&\n    ++\t\tmine=$(git rev-parse refs/remotes/one/master) &&\n    ++\t\this=$(cd ../one && git rev-parse refs/heads/master) &&\n    ++\t\ttest \"z$mine\" = \"z$his\"\n    ++\t)\n    ++'\n    ++\n    ++test_expect_success \"fetch with negative refspec on commandline\" '\n    ++\techo >file updated by origin again &&\n    ++\tgit commit -a -m \"updated by origin again\" &&\n    ++\t(\n    ++\t\tcd three &&\n    ++\t\talternate_in_one=$(cd ../one && git rev-parse refs/heads/alternate) &&\n    ++\t\techo $alternate_in_one >expect &&\n    ++\t\tgit fetch ../one/.git refs/heads/*:refs/remotes/one/* ^refs/heads/master &&\n    ++\t\tcut -f -1 .git/FETCH_HEAD >actual &&\n    ++\t\ttest_cmp expect actual\n    ++\t)\n    ++'\n    ++\n    ++test_expect_success \"fetch with negative refspec avoids duplicate conflict\" '\n    ++\tcd \"$D\" &&\n    ++\t(\n    ++\t\tcd one &&\n    ++\t\tgit branch dups/a &&\n    ++\t\tgit branch dups/b &&\n    ++\t\tgit branch dups/c &&\n    ++\t\tgit branch other/a &&\n    ++\t\tgit rev-parse --verify refs/heads/other/a >../expect &&\n    ++\t\tgit rev-parse --verify refs/heads/dups/b >>../expect &&\n    ++\t\tgit rev-parse --verify refs/heads/dups/c >>../expect\n    ++\t) &&\n    ++\t(\n    ++\t\tcd three &&\n    ++\t\tgit fetch ../one/.git ^refs/heads/dups/a refs/heads/dups/*:refs/dups/* refs/heads/other/a:refs/dups/a &&\n    ++\t\tgit rev-parse --verify refs/dups/a >../actual &&\n    ++\t\tgit rev-parse --verify refs/dups/b >>../actual &&\n    ++\t\tgit rev-parse --verify refs/dups/c >>../actual\n    ++\t) &&\n    ++\ttest_cmp expect actual\n    ++'\n    ++\n    ++test_expect_success \"push --prune with negative refspec\" '\n    ++\t(\n    ++\t\tcd two &&\n    ++\t\tgit branch prune/a &&\n    ++\t\tgit branch prune/b &&\n    ++\t\tgit branch prune/c &&\n    ++\t\tgit push ../three refs/heads/prune/* &&\n    ++\t\tgit branch -d prune/a &&\n    ++\t\tgit branch -d prune/b &&\n    ++\t\tgit push --prune ../three refs/heads/prune/* ^refs/heads/prune/b\n    ++\t) &&\n    ++\t(\n    ++\t\tcd three &&\n    ++\t\ttest_write_lines b c >expect &&\n    ++\t\tgit for-each-ref --format=\"%(refname:lstrip=3)\" refs/heads/prune/ >actual &&\n    ++\t\ttest_cmp expect actual\n    ++\t)\n    ++'\n    ++\n    ++test_expect_success \"push --prune with negative refspec apply to the destination\" '\n    ++\t(\n    ++\t\tcd two &&\n    ++\t\tgit branch ours/a &&\n    ++\t\tgit branch ours/b &&\n    ++\t\tgit branch ours/c &&\n    ++\t\tgit push ../three refs/heads/ours/*:refs/heads/theirs/* &&\n    ++\t\tgit branch -d ours/a &&\n    ++\t\tgit branch -d ours/b &&\n    ++\t\tgit push --prune ../three refs/heads/ours/*:refs/heads/theirs/* ^refs/heads/theirs/b\n    ++\t) &&\n    ++\t(\n    ++\t\tcd three &&\n    ++\t\ttest_write_lines b c >expect &&\n    ++\t\tgit for-each-ref --format=\"%(refname:lstrip=3)\" refs/heads/theirs/ >actual &&\n    ++\t\ttest_cmp expect actual\n    ++\t)\n    ++'\n    ++\n    ++test_expect_success \"fetch --prune with negative refspec\" '\n    ++\t(\n    ++\t\tcd two &&\n    ++\t\tgit branch fetch/a &&\n    ++\t\tgit branch fetch/b &&\n    ++\t\tgit branch fetch/c\n    ++\t) &&\n    ++\t(\n    ++\t\tcd three &&\n    ++\t\tgit fetch ../two/.git refs/heads/fetch/*:refs/heads/copied/*\n    ++\t) &&\n    ++\t(\n    ++\t\tcd two &&\n    ++\t\tgit branch -d fetch/a &&\n    ++\t\tgit branch -d fetch/b\n    ++\t) &&\n    ++\t(\n    ++\t\tcd three &&\n    ++\t\ttest_write_lines b c >expect &&\n    ++\t\tgit fetch -v ../two/.git --prune refs/heads/fetch/*:refs/heads/copied/* ^refs/heads/fetch/b &&\n    ++\t\tgit for-each-ref --format=\"%(refname:lstrip=3)\" refs/heads/copied/ >actual &&\n    ++\t\ttest_cmp expect actual\n    ++\t)\n    ++'\n    ++\n    ++test_done\n\nJacob Keller (1):\n  refspec: add support for negative refspecs\n\n builtin/fetch.c                   |  10 ++\n refspec.c                         |  34 ++++++-\n refspec.h                         |  14 +--\n remote.c                          | 139 +++++++++++++++++++++++++--\n remote.h                          |   9 +-\n t/t5582-fetch-negative-refspec.sh | 151 ++++++++++++++++++++++++++++++\n 6 files changed, 340 insertions(+), 17 deletions(-)\n create mode 100755 t/t5582-fetch-negative-refspec.sh\n\n\nbase-commit: 3a7f6cbded99451cc61ac8b03d3451d13e532055\n-- \n2.28.0.218.ge27853923b9d.dirty\n\n"},{"id":"404238","messageId":"20200821215247.758978-2-jacob.e.keller@intel.com","threadId":"54090","inReplyTo":"20200821215247.758978-1-jacob.e.keller@intel.com","subject":"[RFC v2 1/1] refspec: add support for negative refspecs","fromName":"Jacob Keller","fromEmail":"jacob.e.keller@intel.com","sentAt":"2020-08-21T21:52:47Z","receivedAt":"2020-08-21T21:54:06Z","isPatch":false,"sender":{"key":"jacob.e.keller@intel.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"From: Jacob Keller <jacob.keller@gmail.com>\n\nBoth fetch and push support pattern refspecs which allow fetching or\npushing references that match a specific pattern. Because these patterns\nare globs, they have somewhat limited ability to express more complex\nsituations.\n\nFor example, suppose you wish to fetch all branches from a remote except\nfor a specific one. To allow this, you must setup a set of refspecs\nwhich match only the branches you want. Because refspecs are either\nexplicit name matches, or simple globs, many patterns cannot be\nexpressed.\n\nAdd support for a new type of refspec, referred to as \"negative\"\nrefspecs. These are prefixed with a '^' and mean \"exclude any ref\nmatching this refspec\". They can only have one \"side\" which always\nrefers to the source. During a fetch, this refers to the name of the ref\non the remote. During a push, this refers to the name of the ref on the\nlocal side.\n\nWith negative refspecs, users can express more complex patterns. For\nexample:\n\n git fetch origin refs/heads/*:refs/remotes/origin/* ^refs/heads/dontwant\n\nwill fetch all branches on origin into remotes/origin, but will exclude\nfetching the branch named dontwant.\n\nRefspecs today are commutative, meaning that order doesn't expressly\nmatter. Rather than forcing an implied order, negative refspecs will\nalways be applied last. That is, in order to match, a ref must match at\nleast one positive refspec, and match none of the negative refspecs.\nThis is similar to how negative pathspecs work.\n\nSigned-off-by: Jacob Keller <jacob.keller@gmail.com>\n---\n\nI realize this probably needs to be broken down into multiple patches, but I\nhaven't quite figured out the best way to do that. I'd like to avoid the\ncase where a commit has support for parsing negative refspecs but code paths\nwhich use refspecs aren't handling them correctly. Thoughts?\n\nSplitting would also allow additional space for explanations of some of the\ntrickier logic.\n\nI am also definitely looking for more test ideas, to help make sure we\ncover a good variety of the flows.\n\n builtin/fetch.c                   |  10 ++\n refspec.c                         |  34 ++++++-\n refspec.h                         |  14 +--\n remote.c                          | 139 +++++++++++++++++++++++++--\n remote.h                          |   9 +-\n t/t5582-fetch-negative-refspec.sh | 151 ++++++++++++++++++++++++++++++\n 6 files changed, 340 insertions(+), 17 deletions(-)\n create mode 100755 t/t5582-fetch-negative-refspec.sh\n\ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex c49f0e975203..310699eb7e45 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -530,6 +530,16 @@ static struct ref *get_ref_map(struct remote *remote,\n \t\ttail = &rm->next;\n \t}\n \n+\t/*\n+\t * apply negative refspecs first, before we remove duplicates. This is\n+\t * necessary as negative refspecs might remove an otherwise conflicting\n+\t * duplicate.\n+\t */\n+\tif (rs->nr)\n+\t\tref_map = apply_negative_refspecs(ref_map, rs);\n+\telse\n+\t\tref_map = apply_negative_refspecs(ref_map, &remote->fetch);\n+\n \tref_map = ref_remove_duplicates(ref_map);\n \n \trefname_hash_init(&existing_refs);\ndiff --git a/refspec.c b/refspec.c\nindex f10ef284cef9..25755b83c5f2 100644\n--- a/refspec.c\n+++ b/refspec.c\n@@ -8,6 +8,7 @@ static struct refspec_item s_tag_refspec = {\n \t1,\n \t0,\n \t0,\n+\t0,\n \t\"refs/tags/*\",\n \t\"refs/tags/*\"\n };\n@@ -32,10 +33,17 @@ static int parse_refspec(struct refspec_item *item, const char *refspec, int fet\n \tif (*lhs == '+') {\n \t\titem->force = 1;\n \t\tlhs++;\n+\t} else if (*lhs == '^') {\n+\t\titem->negative = 1;\n+\t\tlhs++;\n \t}\n \n \trhs = strrchr(lhs, ':');\n \n+\t/* negative refspecs only have one side */\n+\tif (item->negative && rhs)\n+\t\treturn 0;\n+\n \t/*\n \t * Before going on, special case \":\" (or \"+:\") as a refspec\n \t * for pushing matching refs.\n@@ -55,7 +63,7 @@ static int parse_refspec(struct refspec_item *item, const char *refspec, int fet\n \n \tllen = (rhs ? (rhs - lhs - 1) : strlen(lhs));\n \tif (1 <= llen && memchr(lhs, '*', llen)) {\n-\t\tif ((rhs && !is_glob) || (!rhs && fetch))\n+\t\tif ((rhs && !is_glob) || (!rhs && !item->negative && fetch))\n \t\t\treturn 0;\n \t\tis_glob = 1;\n \t} else if (rhs && is_glob) {\n@@ -66,6 +74,28 @@ static int parse_refspec(struct refspec_item *item, const char *refspec, int fet\n \titem->src = xstrndup(lhs, llen);\n \tflags = REFNAME_ALLOW_ONELEVEL | (is_glob ? REFNAME_REFSPEC_PATTERN : 0);\n \n+\tif (item->negative) {\n+\t\tstruct object_id unused;\n+\n+\t\t/*\n+\t\t * Negative refspecs only have a LHS, which indicates a ref\n+\t\t * (or pattern of refs) to exclude from other matches. This\n+\t\t * can either be a simple ref, a glob pattern, or even an\n+\t\t * exact sha1 match.\n+\t\t */\n+\t\tif (!*item->src)\n+\t\t\treturn 0; /* negative refspecs must not be empty */\n+\t\telse if (llen == the_hash_algo->hexsz && !get_oid_hex(item->src, &unused))\n+\t\t\titem->exact_sha1 = 1; /* ok */\n+\t\telse if (!check_refname_format(item->src, flags))\n+\t\t\t; /* valid looking ref is ok */\n+\t\telse\n+\t\t\treturn 0;\n+\n+\t\t/* other rules for negative refspecs don't apply */\n+\t\treturn 1;\n+\t}\n+\n \tif (fetch) {\n \t\tstruct object_id unused;\n \n@@ -209,7 +239,7 @@ void refspec_ref_prefixes(const struct refspec *rs,\n \t\tconst struct refspec_item *item = &rs->items[i];\n \t\tconst char *prefix = NULL;\n \n-\t\tif (item->exact_sha1)\n+\t\tif (item->exact_sha1 || item->negative)\n \t\t\tcontinue;\n \t\tif (rs->fetch == REFSPEC_FETCH)\n \t\t\tprefix = item->src;\ndiff --git a/refspec.h b/refspec.h\nindex 8d654e3a3ac4..e5bf6d25d0f7 100644\n--- a/refspec.h\n+++ b/refspec.h\n@@ -5,12 +5,13 @@\n extern const struct refspec_item *tag_refspec;\n \n /**\n- * A struct refspec_item holds the parsed interpretation of a refspec.  If it will\n- * force updates (starts with a '+'), force is true.  If it is a pattern\n- * (sides end with '*') pattern is true.  src and dest are the two sides\n- * (including '*' characters if present); if there is only one side, it is src,\n- * and dst is NULL; if sides exist but are empty (i.e., the refspec either\n- * starts or ends with ':'), the corresponding side is \"\".\n+ * A struct refspec_item holds the parsed interpretation of a refspec.  If it\n+ * will force updates (starts with a '+'), force is true.  If it is a pattern\n+ * (sides end with '*') pattern is true.  If it is a negative refspec, (starts\n+ * with '^'), negative is true.  src and dest are the two sides (including '*'\n+ * characters if present); if there is only one side, it is src, and dst is\n+ * NULL; if sides exist but are empty (i.e., the refspec either starts or ends\n+ * with ':'), the corresponding side is \"\".\n  *\n  * remote_find_tracking(), given a remote and a struct refspec_item with either src\n  * or dst filled out, will fill out the other such that the result is in the\n@@ -22,6 +23,7 @@ struct refspec_item {\n \tunsigned pattern : 1;\n \tunsigned matching : 1;\n \tunsigned exact_sha1 : 1;\n+\tunsigned negative : 1;\n \n \tchar *src;\n \tchar *dst;\ndiff --git a/remote.c b/remote.c\nindex c5ed74f91c63..2f583d72c3f0 100644\n--- a/remote.c\n+++ b/remote.c\n@@ -686,24 +686,100 @@ static int match_name_with_pattern(const char *key, const char *name,\n \treturn ret;\n }\n \n+static int refspec_match(const struct refspec_item *refspec,\n+\t\t\t const char *name)\n+{\n+\tif (refspec->pattern)\n+\t\treturn match_name_with_pattern(refspec->src, name, NULL, NULL);\n+\n+\treturn !strcmp(refspec->src, name);\n+}\n+\n+static int omit_name_by_refspec(const char *name, struct refspec *rs)\n+{\n+\tint i;\n+\n+\tfor (i = 0; i < rs->nr; i++) {\n+\t\tif (rs->items[i].negative && refspec_match(&rs->items[i], name))\n+\t\t\treturn 1;\n+\t}\n+\treturn 0;\n+}\n+\n+struct ref *apply_negative_refspecs(struct ref *ref_map, struct refspec *rs)\n+{\n+\tstruct ref **tail;\n+\n+\tfor (tail = &ref_map; *tail; ) {\n+\t\tstruct ref *ref = *tail;\n+\n+\t\tif (omit_name_by_refspec(ref->name, rs)) {\n+\t\t\t*tail = ref->next;\n+\t\t\tfree(ref->peer_ref);\n+\t\t\tfree(ref);\n+\t\t} else\n+\t\t\ttail = &ref->next;\n+\t}\n+\n+\treturn ref_map;\n+}\n+\n static void query_refspecs_multiple(struct refspec *rs,\n \t\t\t\t    struct refspec_item *query,\n \t\t\t\t    struct string_list *results)\n {\n-\tint i;\n+\tint i, matched_negative = 0;\n \tint find_src = !query->src;\n+\tstruct string_list reversed = STRING_LIST_INIT_NODUP;\n+\tconst char *needle = find_src ? query->dst : query->src;\n+\tchar **result = find_src ? &query->src : &query->dst;\n \n \tif (find_src && !query->dst)\n \t\tBUG(\"query_refspecs_multiple: need either src or dst\");\n \n+\t/*\n+\t * If a ref matches any of the negative refspecs, then we should treat\n+\t * it as not matching this query. Note that negative refspecs apply to\n+\t * the source but we're checking only the destination. Reverse and\n+\t * capture any pattern refspecs in order to see if the source would\n+\t * have matched a negative refspec.\n+\t */\n+\tfor (i = 0; i < rs->nr; i++) {\n+\t\tstruct refspec_item *refspec = &rs->items[i];\n+\t\tchar *expn_name;\n+\n+\t\tif (refspec->negative)\n+\t\t\tcontinue;\n+\n+\t\t/* Note the reversal of src and dst */\n+\t\tif (refspec->pattern) {\n+\t\t\tconst char *key = refspec->dst ?: refspec->src;\n+\t\t\tconst char *value = refspec->src;\n+\n+\t\t\tif (match_name_with_pattern(key, needle, value, &expn_name))\n+\t\t\t\tstring_list_append_nodup(&reversed, expn_name);\n+\t\t} else {\n+\t\t\tif (!strcmp(needle, refspec->src))\n+\t\t\t\tstring_list_append(&reversed, refspec->src);\n+\t\t}\n+\t}\n+\n+\tfor (i = 0; !matched_negative && i < reversed.nr; i++) {\n+\t\tif (omit_name_by_refspec(reversed.items[i].string, rs))\n+\t\t\tmatched_negative = 1;\n+\t}\n+\n+\tstring_list_clear(&reversed, 0);\n+\n+\tif (matched_negative)\n+\t\treturn;\n+\n \tfor (i = 0; i < rs->nr; i++) {\n \t\tstruct refspec_item *refspec = &rs->items[i];\n \t\tconst char *key = find_src ? refspec->dst : refspec->src;\n \t\tconst char *value = find_src ? refspec->src : refspec->dst;\n-\t\tconst char *needle = find_src ? query->dst : query->src;\n-\t\tchar **result = find_src ? &query->src : &query->dst;\n \n-\t\tif (!refspec->dst)\n+\t\tif (!refspec->dst || refspec->negative)\n \t\t\tcontinue;\n \t\tif (refspec->pattern) {\n \t\t\tif (match_name_with_pattern(key, needle, value, result))\n@@ -716,20 +792,58 @@ static void query_refspecs_multiple(struct refspec *rs,\n \n int query_refspecs(struct refspec *rs, struct refspec_item *query)\n {\n-\tint i;\n+\tint i, matched_negative = 0;\n \tint find_src = !query->src;\n+\tstruct string_list reversed = STRING_LIST_INIT_NODUP;\n \tconst char *needle = find_src ? query->dst : query->src;\n \tchar **result = find_src ? &query->src : &query->dst;\n \n \tif (find_src && !query->dst)\n \t\tBUG(\"query_refspecs: need either src or dst\");\n \n+\t/*\n+\t * If a ref matches any of the negative refspecs, then we should treat\n+\t * it as not matching this query. Note that negative refspecs apply to\n+\t * the source but we're checking only the destination. Reverse and\n+\t * capture any pattern refspecs in order to see if the source would\n+\t * have matched a negative refspec.\n+\t */\n+\tfor (i = 0; i < rs->nr; i++) {\n+\t\tstruct refspec_item *refspec = &rs->items[i];\n+\t\tchar *expn_name;\n+\n+\t\tif (refspec->negative)\n+\t\t\tcontinue;\n+\n+\t\t/* Note the reversal of src and dst */\n+\t\tif (refspec->pattern) {\n+\t\t\tconst char *key = refspec->dst ?: refspec->src;\n+\t\t\tconst char *value = refspec->src;\n+\n+\t\t\tif (match_name_with_pattern(key, needle, value, &expn_name))\n+\t\t\t\tstring_list_append_nodup(&reversed, expn_name);\n+\t\t} else {\n+\t\t\tif (!strcmp(needle, refspec->src))\n+\t\t\t\tstring_list_append(&reversed, refspec->src);\n+\t\t}\n+\t}\n+\n+\tfor (i = 0; !matched_negative && i < reversed.nr; i++) {\n+\t\tif (omit_name_by_refspec(reversed.items[i].string, rs))\n+\t\t\tmatched_negative = 1;\n+\t}\n+\n+\tstring_list_clear(&reversed, 0);\n+\n+\tif (matched_negative)\n+\t\treturn -1;\n+\n \tfor (i = 0; i < rs->nr; i++) {\n \t\tstruct refspec_item *refspec = &rs->items[i];\n \t\tconst char *key = find_src ? refspec->dst : refspec->src;\n \t\tconst char *value = find_src ? refspec->src : refspec->dst;\n \n-\t\tif (!refspec->dst)\n+\t\tif (!refspec->dst || refspec->negative)\n \t\t\tcontinue;\n \t\tif (refspec->pattern) {\n \t\t\tif (match_name_with_pattern(key, needle, value, result)) {\n@@ -1058,7 +1172,7 @@ static int match_explicit(struct ref *src, struct ref *dst,\n \tconst char *dst_value = rs->dst;\n \tchar *dst_guess;\n \n-\tif (rs->pattern || rs->matching)\n+\tif (rs->pattern || rs->matching || rs->negative)\n \t\treturn 0;\n \n \tmatched_src = matched_dst = NULL;\n@@ -1134,6 +1248,10 @@ static char *get_ref_match(const struct refspec *rs, const struct ref *ref,\n \tint matching_refs = -1;\n \tfor (i = 0; i < rs->nr; i++) {\n \t\tconst struct refspec_item *item = &rs->items[i];\n+\n+\t\tif (item->negative)\n+\t\t\tcontinue;\n+\n \t\tif (item->matching &&\n \t\t    (matching_refs == -1 || item->force)) {\n \t\t\tmatching_refs = i;\n@@ -1339,7 +1457,7 @@ int check_push_refs(struct ref *src, struct refspec *rs)\n \tfor (i = 0; i < rs->nr; i++) {\n \t\tstruct refspec_item *item = &rs->items[i];\n \n-\t\tif (item->pattern || item->matching)\n+\t\tif (item->pattern || item->matching || item->negative)\n \t\t\tcontinue;\n \n \t\tret |= match_explicit_lhs(src, item, NULL, NULL);\n@@ -1441,6 +1559,8 @@ int match_push_refs(struct ref *src, struct ref **dst,\n \t\tstring_list_clear(&src_ref_index, 0);\n \t}\n \n+\t*dst = apply_negative_refspecs(*dst, rs);\n+\n \tif (errs)\n \t\treturn -1;\n \treturn 0;\n@@ -1810,6 +1930,9 @@ int get_fetch_map(const struct ref *remote_refs,\n {\n \tstruct ref *ref_map, **rmp;\n \n+\tif (refspec->negative)\n+\t\treturn 0;\n+\n \tif (refspec->pattern) {\n \t\tref_map = get_expanded_map(remote_refs, refspec);\n \t} else {\ndiff --git a/remote.h b/remote.h\nindex 5e3ea5a26deb..104e75e0f74d 100644\n--- a/remote.h\n+++ b/remote.h\n@@ -193,6 +193,12 @@ int resolve_remote_symref(struct ref *ref, struct ref *list);\n  */\n struct ref *ref_remove_duplicates(struct ref *ref_map);\n \n+/*\n+ * Remove all entries in the input list which match any negative refspec in\n+ * the refspec list.\n+ */\n+struct ref *apply_negative_refspecs(struct ref *ref_map, struct refspec *rs);\n+\n int query_refspecs(struct refspec *rs, struct refspec_item *query);\n char *apply_refspecs(struct refspec *rs, const char *name);\n \n@@ -205,7 +211,8 @@ void set_ref_status_for_push(struct ref *remote_refs, int send_mirror,\n /*\n  * Given a list of the remote refs and the specification of things to\n  * fetch, makes a (separate) list of the refs to fetch and the local\n- * refs to store into.\n+ * refs to store into. Note that negative refspecs are ignored here, and\n+ * should be handled separately.\n  *\n  * *tail is the pointer to the tail pointer of the list of results\n  * beforehand, and will be set to the tail pointer of the list of\ndiff --git a/t/t5582-fetch-negative-refspec.sh b/t/t5582-fetch-negative-refspec.sh\nnew file mode 100755\nindex 000000000000..83d08e513413\n--- /dev/null\n+++ b/t/t5582-fetch-negative-refspec.sh\n@@ -0,0 +1,151 @@\n+#!/bin/sh\n+# Copyright (c) 2020, Jacob Keller.\n+\n+test_description='\"git fetch\" with negative refspecs.\n+\n+'\n+\n+. ./test-lib.sh\n+\n+test_expect_success setup '\n+\techo >file original &&\n+\tgit add file &&\n+\tgit commit -a -m original\n+'\n+\n+test_expect_success \"clone and setup child repos\" '\n+\tgit clone . one &&\n+\t(\n+\t\tcd one &&\n+\t\techo >file updated by one &&\n+\t\tgit commit -a -m \"updated by one\" &&\n+\t\tgit switch -c alternate &&\n+\t\techo >file updated again by one &&\n+\t\tgit commit -a -m \"updated by one again\" &&\n+\t\tgit switch master\n+\t) &&\n+\tgit clone . two &&\n+\t(\n+\t\tcd two &&\n+\t\tgit config branch.master.remote one &&\n+\t\tgit config remote.one.url ../one/.git/ &&\n+\t\tgit config remote.one.fetch +refs/heads/*:refs/remotes/one/* &&\n+\t\tgit config --add remote.one.fetch ^refs/heads/alternate\n+\t) &&\n+\tgit clone . three\n+'\n+\n+test_expect_success \"fetch one\" '\n+\techo >file updated by origin &&\n+\tgit commit -a -m \"updated by origin\" &&\n+\t(\n+\t\tcd two &&\n+\t\ttest_must_fail git rev-parse --verify refs/remotes/one/alternate &&\n+\t\tgit fetch one &&\n+\t\ttest_must_fail git rev-parse --verify refs/remotes/one/alternate &&\n+\t\tgit rev-parse --verify refs/remotes/one/master &&\n+\t\tmine=$(git rev-parse refs/remotes/one/master) &&\n+\t\this=$(cd ../one && git rev-parse refs/heads/master) &&\n+\t\ttest \"z$mine\" = \"z$his\"\n+\t)\n+'\n+\n+test_expect_success \"fetch with negative refspec on commandline\" '\n+\techo >file updated by origin again &&\n+\tgit commit -a -m \"updated by origin again\" &&\n+\t(\n+\t\tcd three &&\n+\t\talternate_in_one=$(cd ../one && git rev-parse refs/heads/alternate) &&\n+\t\techo $alternate_in_one >expect &&\n+\t\tgit fetch ../one/.git refs/heads/*:refs/remotes/one/* ^refs/heads/master &&\n+\t\tcut -f -1 .git/FETCH_HEAD >actual &&\n+\t\ttest_cmp expect actual\n+\t)\n+'\n+\n+test_expect_success \"fetch with negative refspec avoids duplicate conflict\" '\n+\tcd \"$D\" &&\n+\t(\n+\t\tcd one &&\n+\t\tgit branch dups/a &&\n+\t\tgit branch dups/b &&\n+\t\tgit branch dups/c &&\n+\t\tgit branch other/a &&\n+\t\tgit rev-parse --verify refs/heads/other/a >../expect &&\n+\t\tgit rev-parse --verify refs/heads/dups/b >>../expect &&\n+\t\tgit rev-parse --verify refs/heads/dups/c >>../expect\n+\t) &&\n+\t(\n+\t\tcd three &&\n+\t\tgit fetch ../one/.git ^refs/heads/dups/a refs/heads/dups/*:refs/dups/* refs/heads/other/a:refs/dups/a &&\n+\t\tgit rev-parse --verify refs/dups/a >../actual &&\n+\t\tgit rev-parse --verify refs/dups/b >>../actual &&\n+\t\tgit rev-parse --verify refs/dups/c >>../actual\n+\t) &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success \"push --prune with negative refspec\" '\n+\t(\n+\t\tcd two &&\n+\t\tgit branch prune/a &&\n+\t\tgit branch prune/b &&\n+\t\tgit branch prune/c &&\n+\t\tgit push ../three refs/heads/prune/* &&\n+\t\tgit branch -d prune/a &&\n+\t\tgit branch -d prune/b &&\n+\t\tgit push --prune ../three refs/heads/prune/* ^refs/heads/prune/b\n+\t) &&\n+\t(\n+\t\tcd three &&\n+\t\ttest_write_lines b c >expect &&\n+\t\tgit for-each-ref --format=\"%(refname:lstrip=3)\" refs/heads/prune/ >actual &&\n+\t\ttest_cmp expect actual\n+\t)\n+'\n+\n+test_expect_success \"push --prune with negative refspec apply to the destination\" '\n+\t(\n+\t\tcd two &&\n+\t\tgit branch ours/a &&\n+\t\tgit branch ours/b &&\n+\t\tgit branch ours/c &&\n+\t\tgit push ../three refs/heads/ours/*:refs/heads/theirs/* &&\n+\t\tgit branch -d ours/a &&\n+\t\tgit branch -d ours/b &&\n+\t\tgit push --prune ../three refs/heads/ours/*:refs/heads/theirs/* ^refs/heads/theirs/b\n+\t) &&\n+\t(\n+\t\tcd three &&\n+\t\ttest_write_lines b c >expect &&\n+\t\tgit for-each-ref --format=\"%(refname:lstrip=3)\" refs/heads/theirs/ >actual &&\n+\t\ttest_cmp expect actual\n+\t)\n+'\n+\n+test_expect_success \"fetch --prune with negative refspec\" '\n+\t(\n+\t\tcd two &&\n+\t\tgit branch fetch/a &&\n+\t\tgit branch fetch/b &&\n+\t\tgit branch fetch/c\n+\t) &&\n+\t(\n+\t\tcd three &&\n+\t\tgit fetch ../two/.git refs/heads/fetch/*:refs/heads/copied/*\n+\t) &&\n+\t(\n+\t\tcd two &&\n+\t\tgit branch -d fetch/a &&\n+\t\tgit branch -d fetch/b\n+\t) &&\n+\t(\n+\t\tcd three &&\n+\t\ttest_write_lines b c >expect &&\n+\t\tgit fetch -v ../two/.git --prune refs/heads/fetch/*:refs/heads/copied/* ^refs/heads/fetch/b &&\n+\t\tgit for-each-ref --format=\"%(refname:lstrip=3)\" refs/heads/copied/ >actual &&\n+\t\ttest_cmp expect actual\n+\t)\n+'\n+\n+test_done\n-- \n2.28.0.218.ge27853923b9d.dirty\n\n"},{"id":"404301","messageId":"nycvar.QRO.7.76.6.2008221528170.56@tvgsbejvaqbjf.bet","threadId":"54090","inReplyTo":"20200821215247.758978-2-jacob.e.keller@intel.com","subject":"Re: [RFC v2 1/1] refspec: add support for negative refspecs","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2020-08-22T13:29:49Z","receivedAt":"2020-08-23T20:30:34Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Jake,\n\nOn Fri, 21 Aug 2020, Jacob Keller wrote:\n\n>  static void query_refspecs_multiple(struct refspec *rs,\n>  \t\t\t\t    struct refspec_item *query,\n>  \t\t\t\t    struct string_list *results)\n>  {\n> -\tint i;\n> +\tint i, matched_negative = 0;\n>  \tint find_src = !query->src;\n> +\tstruct string_list reversed = STRING_LIST_INIT_NODUP;\n> +\tconst char *needle = find_src ? query->dst : query->src;\n> +\tchar **result = find_src ? &query->src : &query->dst;\n>\n>  \tif (find_src && !query->dst)\n>  \t\tBUG(\"query_refspecs_multiple: need either src or dst\");\n>\n> +\t/*\n> +\t * If a ref matches any of the negative refspecs, then we should treat\n> +\t * it as not matching this query. Note that negative refspecs apply to\n> +\t * the source but we're checking only the destination. Reverse and\n> +\t * capture any pattern refspecs in order to see if the source would\n> +\t * have matched a negative refspec.\n> +\t */\n> +\tfor (i = 0; i < rs->nr; i++) {\n> +\t\tstruct refspec_item *refspec = &rs->items[i];\n> +\t\tchar *expn_name;\n> +\n> +\t\tif (refspec->negative)\n> +\t\t\tcontinue;\n> +\n> +\t\t/* Note the reversal of src and dst */\n> +\t\tif (refspec->pattern) {\n> +\t\t\tconst char *key = refspec->dst ?: refspec->src;\n\nThis breaks at least with Visual C. I need this to unbreak the build:\n\n-- snipsnap --\nSubject: [PATCH] fixup! refspec: add support for negative refspecs\n\nThe `?:` operator is not supported e.g. by Visual C. Let's not use it.\n\nSigned-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>\n---\n remote.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/remote.c b/remote.c\nindex 940622e0ba88..8e081863c06d 100644\n--- a/remote.c\n+++ b/remote.c\n@@ -731,7 +731,7 @@ static void query_refspecs_multiple(struct refspec *rs,\n\n \t\t/* Note the reversal of src and dst */\n \t\tif (refspec->pattern) {\n-\t\t\tconst char *key = refspec->dst ?: refspec->src;\n+\t\t\tconst char *key = refspec->dst ? refspec->dst : refspec->src;\n \t\t\tconst char *value = refspec->src;\n\n \t\t\tif (match_name_with_pattern(key, needle, value, &expn_name))\n@@ -795,7 +795,7 @@ int query_refspecs(struct refspec *rs, struct refspec_item *query)\n\n \t\t/* Note the reversal of src and dst */\n \t\tif (refspec->pattern) {\n-\t\t\tconst char *key = refspec->dst ?: refspec->src;\n+\t\t\tconst char *key = refspec->dst ? refspec->dst : refspec->src;\n \t\t\tconst char *value = refspec->src;\n\n \t\t\tif (match_name_with_pattern(key, needle, value, &expn_name))\n--\n2.28.0.windows.1\n\n"},{"id":"404325","messageId":"CA+P7+xp9CEO_wLbqvg26cTWV45mniMwJyJkY8xEmMTqLOWjoSw@mail.gmail.com","threadId":"54090","inReplyTo":"nycvar.QRO.7.76.6.2008221528170.56@tvgsbejvaqbjf.bet","subject":"Re: [RFC v2 1/1] refspec: add support for negative refspecs","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2020-08-24T15:47:54Z","receivedAt":"2020-08-24T15:48:19Z","isPatch":false,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Sun, Aug 23, 2020 at 1:29 PM Johannes Schindelin\n<Johannes.Schindelin@gmx.de> wrote:\n>\n> Hi Jake,\n>\n> On Fri, 21 Aug 2020, Jacob Keller wrote:\n>\n> >  static void query_refspecs_multiple(struct refspec *rs,\n> >                                   struct refspec_item *query,\n> >                                   struct string_list *results)\n> >  {\n> > -     int i;\n> > +     int i, matched_negative = 0;\n> >       int find_src = !query->src;\n> > +     struct string_list reversed = STRING_LIST_INIT_NODUP;\n> > +     const char *needle = find_src ? query->dst : query->src;\n> > +     char **result = find_src ? &query->src : &query->dst;\n> >\n> >       if (find_src && !query->dst)\n> >               BUG(\"query_refspecs_multiple: need either src or dst\");\n> >\n> > +     /*\n> > +      * If a ref matches any of the negative refspecs, then we should treat\n> > +      * it as not matching this query. Note that negative refspecs apply to\n> > +      * the source but we're checking only the destination. Reverse and\n> > +      * capture any pattern refspecs in order to see if the source would\n> > +      * have matched a negative refspec.\n> > +      */\n> > +     for (i = 0; i < rs->nr; i++) {\n> > +             struct refspec_item *refspec = &rs->items[i];\n> > +             char *expn_name;\n> > +\n> > +             if (refspec->negative)\n> > +                     continue;\n> > +\n> > +             /* Note the reversal of src and dst */\n> > +             if (refspec->pattern) {\n> > +                     const char *key = refspec->dst ?: refspec->src;\n>\n> This breaks at least with Visual C. I need this to unbreak the build:\n>\n\n\nThe provided fixup! looks good to me. I'll pull it in when making the\nnext revision.\n\nThanks,\nJake\n\n> -- snipsnap --\n> Subject: [PATCH] fixup! refspec: add support for negative refspecs\n>\n> The `?:` operator is not supported e.g. by Visual C. Let's not use it.\n>\n> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>\n> ---\n>  remote.c | 4 ++--\n>  1 file changed, 2 insertions(+), 2 deletions(-)\n>\n> diff --git a/remote.c b/remote.c\n> index 940622e0ba88..8e081863c06d 100644\n> --- a/remote.c\n> +++ b/remote.c\n> @@ -731,7 +731,7 @@ static void query_refspecs_multiple(struct refspec *rs,\n>\n>                 /* Note the reversal of src and dst */\n>                 if (refspec->pattern) {\n> -                       const char *key = refspec->dst ?: refspec->src;\n> +                       const char *key = refspec->dst ? refspec->dst : refspec->src;\n>                         const char *value = refspec->src;\n>\n>                         if (match_name_with_pattern(key, needle, value, &expn_name))\n> @@ -795,7 +795,7 @@ int query_refspecs(struct refspec *rs, struct refspec_item *query)\n>\n>                 /* Note the reversal of src and dst */\n>                 if (refspec->pattern) {\n> -                       const char *key = refspec->dst ?: refspec->src;\n> +                       const char *key = refspec->dst ? refspec->dst : refspec->src;\n>                         const char *value = refspec->src;\n>\n>                         if (match_name_with_pattern(key, needle, value, &expn_name))\n> --\n> 2.28.0.windows.1\n>\n"},{"id":"404345","messageId":"xmqqk0xo53pi.fsf@gitster.c.googlers.com","threadId":"54090","inReplyTo":"nycvar.QRO.7.76.6.2008221528170.56@tvgsbejvaqbjf.bet","subject":"Re: [RFC v2 1/1] refspec: add support for negative refspecs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-08-24T17:55:05Z","receivedAt":"2020-08-24T17:55:13Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n\n> The `?:` operator is not supported e.g. by Visual C. Let's not use it.\n\nThanks.  IIRC that was GNU extension?\n\n"},{"id":"404365","messageId":"CA+P7+xp_ajhC_vN0jLY6AVS+cSgLNK5SeybxSwoT16ripUdqkw@mail.gmail.com","threadId":"54090","inReplyTo":"xmqqk0xo53pi.fsf@gitster.c.googlers.com","subject":"Re: [RFC v2 1/1] refspec: add support for negative refspecs","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2020-08-24T19:26:19Z","receivedAt":"2020-08-24T19:26:35Z","isPatch":false,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Mon, Aug 24, 2020 at 10:55 AM Junio C Hamano <gitster@pobox.com> wrote:\n>\n> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n>\n> > The `?:` operator is not supported e.g. by Visual C. Let's not use it.\n>\n> Thanks.  IIRC that was GNU extension?\n>\n\nYea, probably.\n\nFrom wikipedia, [1], \"In GNU C and C++ (that is: in C and C++ with GCC\nextensions), the second operand of the ternary operator is optional.\nThis has been the case since at least GCC 2.95.3 (March 2001), and\nseems to the original elvis operator.\"\n\nThanks,\nJake\n\n[1] https://en.wikipedia.org/wiki/Elvis_operator\n"},{"id":"405797","messageId":"xmqqh7rwqhm3.fsf@gitster.c.googlers.com","threadId":"54090","inReplyTo":"20200821215247.758978-2-jacob.e.keller@intel.com","subject":"Re: [RFC v2 1/1] refspec: add support for negative refspecs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-09-17T20:21:40Z","receivedAt":"2020-09-17T20:21:46Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jacob Keller <jacob.e.keller@intel.com> writes:\n\n> From: Jacob Keller <jacob.keller@gmail.com>\n>\n> Both fetch and push support pattern refspecs which allow fetching or\n> pushing references that match a specific pattern. Because these patterns\n> are globs, they have somewhat limited ability to express more complex\n> situations.\n>\n> For example, suppose you wish to fetch all branches from a remote except\n> for a specific one. To allow this, you must setup a set of refspecs\n> which match only the branches you want. Because refspecs are either\n> explicit name matches, or simple globs, many patterns cannot be\n> expressed.\n>\n> Add support for a new type of refspec, referred to as \"negative\"\n> refspecs. These are prefixed with a '^' and mean \"exclude any ref\n> matching this refspec\". They can only have one \"side\" which always\n> refers to the source. During a fetch, this refers to the name of the ref\n> on the remote. During a push, this refers to the name of the ref on the\n> local side.\n>\n> With negative refspecs, users can express more complex patterns. For\n> example:\n>\n>  git fetch origin refs/heads/*:refs/remotes/origin/* ^refs/heads/dontwant\n>\n> will fetch all branches on origin into remotes/origin, but will exclude\n> fetching the branch named dontwant.\n>\n> Refspecs today are commutative, meaning that order doesn't expressly\n> matter. Rather than forcing an implied order, negative refspecs will\n> always be applied last. That is, in order to match, a ref must match at\n> least one positive refspec, and match none of the negative refspecs.\n> This is similar to how negative pathspecs work.\n>\n> Signed-off-by: Jacob Keller <jacob.keller@gmail.com>\n> ---\n>\n> I realize this probably needs to be broken down into multiple patches, but I\n> haven't quite figured out the best way to do that. I'd like to avoid the\n> case where a commit has support for parsing negative refspecs but code paths\n> which use refspecs aren't handling them correctly. Thoughts?\n>\n> Splitting would also allow additional space for explanations of some of the\n> trickier logic.\n>\n> I am also definitely looking for more test ideas, to help make sure we\n> cover a good variety of the flows.\n\nAnybody wants to help this move forward?\n\nI plan to send a review with the patch in the current form, without\nwaiting for any splitting, later towards the weekend, though.\n"},{"id":"405811","messageId":"xmqqzh5onea1.fsf@gitster.c.googlers.com","threadId":"54090","inReplyTo":"20200821215247.758978-2-jacob.e.keller@intel.com","subject":"Re: [RFC v2 1/1] refspec: add support for negative refspecs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-09-18T00:01:58Z","receivedAt":"2020-09-18T00:02:03Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jacob Keller <jacob.e.keller@intel.com> writes:\n\n> @@ -66,6 +74,28 @@ static int parse_refspec(struct refspec_item *item, const char *refspec, int fet\n>  \titem->src = xstrndup(lhs, llen);\n>  \tflags = REFNAME_ALLOW_ONELEVEL | (is_glob ? REFNAME_REFSPEC_PATTERN : 0);\n>  \n> +\tif (item->negative) {\n> +\t\tstruct object_id unused;\n> +\n> +\t\t/*\n> +\t\t * Negative refspecs only have a LHS, which indicates a ref\n> +\t\t * (or pattern of refs) to exclude from other matches. This\n> +\t\t * can either be a simple ref, a glob pattern, or even an\n> +\t\t * exact sha1 match.\n> +\t\t */\n> +\t\tif (!*item->src)\n> +\t\t\treturn 0; /* negative refspecs must not be empty */\n> +\t\telse if (llen == the_hash_algo->hexsz && !get_oid_hex(item->src, &unused))\n> +\t\t\titem->exact_sha1 = 1; /* ok */\n> +\t\telse if (!check_refname_format(item->src, flags))\n> +\t\t\t; /* valid looking ref is ok */\n> +\t\telse\n> +\t\t\treturn 0;\n> +\n> +\t\t/* other rules for negative refspecs don't apply */\n\nThis comment confused me a bit; did you mean \"other rules don't\napply to negative refspecs\"?\n\n> +\t\treturn 1;\n> +\t}\n> +\n>  \tif (fetch) {\n>  \t\tstruct object_id unused;\n\n\n> diff --git a/remote.c b/remote.c\n> index c5ed74f91c63..2f583d72c3f0 100644\n> --- a/remote.c\n> +++ b/remote.c\n> @@ -1058,7 +1172,7 @@ static int match_explicit(struct ref *src, struct ref *dst,\n>  \tconst char *dst_value = rs->dst;\n>  \tchar *dst_guess;\n>  \n> -\tif (rs->pattern || rs->matching)\n> +\tif (rs->pattern || rs->matching || rs->negative)\n>  \t\treturn 0;\n\nOK.  These \"special\" ones do not participate in explicit matching.\n\n> @@ -1134,6 +1248,10 @@ static char *get_ref_match(const struct refspec *rs, const struct ref *ref,\n>  \tint matching_refs = -1;\n>  \tfor (i = 0; i < rs->nr; i++) {\n>  \t\tconst struct refspec_item *item = &rs->items[i];\n> +\n> +\t\tif (item->negative)\n> +\t\t\tcontinue;\n> +\n\nAnd a negative one does not decide if a ref being pushed will be\npushed out for real at this point.  This helper is only to enumerate\nthe candidate refs to be pushed out; the caller makes a separate\ncall to apply_negative_refspecs() to cull the candidate list later.\n\nOK.\n\n> @@ -1339,7 +1457,7 @@ int check_push_refs(struct ref *src, struct refspec *rs)\n>  \tfor (i = 0; i < rs->nr; i++) {\n>  \t\tstruct refspec_item *item = &rs->items[i];\n>  \n> -\t\tif (item->pattern || item->matching)\n> +\t\tif (item->pattern || item->matching || item->negative)\n>  \t\t\tcontinue;\n>  \n>  \t\tret |= match_explicit_lhs(src, item, NULL, NULL);\n\nmatch_explicit_lhs(), like match_explicit(), are for explicit\nmatching and should not be called for the \"special\" ones.  OK.\n\n> @@ -1441,6 +1559,8 @@ int match_push_refs(struct ref *src, struct ref **dst,\n>  \t\tstring_list_clear(&src_ref_index, 0);\n>  \t}\n>  \n> +\t*dst = apply_negative_refspecs(*dst, rs);\n> +\n>  \tif (errs)\n>  \t\treturn -1;\n>  \treturn 0;\n\nAnd after grabbing all the candidate refs to be updated via this\npush, we filter out the ones that match negative pattern.  Can it\nalso produce an error, or it can never fail (to udpate errs)?\n\n> @@ -1810,6 +1930,9 @@ int get_fetch_map(const struct ref *remote_refs,\n>  {\n>  \tstruct ref *ref_map, **rmp;\n>  \n> +\tif (refspec->negative)\n> +\t\treturn 0;\n> +\n\nAgain, the idea is to let the existing codepath to only deal with\nthe positive refspec elements to keep the same behaviour, and let\nthe caller filter the ones that match negative ones out of the\nresult.  So we return without anything here for negative one.\n\nNothing jumped out at me as being suspicious so far, other than that\nthe GNU \"?<empty>:\" thing needs to be fixed as pointed out by Dscho.\n\nThanks.\n"},{"id":"406289","messageId":"CA+P7+xr-xhC3Rr3BXT-5LLUAic1Tx27pmhZAu5T0iGwfQ6qAUA@mail.gmail.com","threadId":"54090","inReplyTo":"xmqqzh5onea1.fsf@gitster.c.googlers.com","subject":"Re: [RFC v2 1/1] refspec: add support for negative refspecs","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2020-09-24T23:33:33Z","receivedAt":"2020-09-24T23:33:47Z","isPatch":false,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Thu, Sep 17, 2020 at 5:02 PM Junio C Hamano <gitster@pobox.com> wrote:\n>\n> Jacob Keller <jacob.e.keller@intel.com> writes:\n>\n> > @@ -66,6 +74,28 @@ static int parse_refspec(struct refspec_item *item, const char *refspec, int fet\n> >       item->src = xstrndup(lhs, llen);\n> >       flags = REFNAME_ALLOW_ONELEVEL | (is_glob ? REFNAME_REFSPEC_PATTERN : 0);\n> >\n> > +     if (item->negative) {\n> > +             struct object_id unused;\n> > +\n> > +             /*\n> > +              * Negative refspecs only have a LHS, which indicates a ref\n> > +              * (or pattern of refs) to exclude from other matches. This\n> > +              * can either be a simple ref, a glob pattern, or even an\n> > +              * exact sha1 match.\n> > +              */\n> > +             if (!*item->src)\n> > +                     return 0; /* negative refspecs must not be empty */\n> > +             else if (llen == the_hash_algo->hexsz && !get_oid_hex(item->src, &unused))\n> > +                     item->exact_sha1 = 1; /* ok */\n> > +             else if (!check_refname_format(item->src, flags))\n> > +                     ; /* valid looking ref is ok */\n> > +             else\n> > +                     return 0;\n> > +\n> > +             /* other rules for negative refspecs don't apply */\n>\n> This comment confused me a bit; did you mean \"other rules don't\n> apply to negative refspecs\"?\n>\n\nYea, this should be reworded.\n\n> > +             return 1;\n> > +     }\n> > +\n> >       if (fetch) {\n> >               struct object_id unused;\n>\n>\n> > diff --git a/remote.c b/remote.c\n> > index c5ed74f91c63..2f583d72c3f0 100644\n> > --- a/remote.c\n> > +++ b/remote.c\n> > @@ -1058,7 +1172,7 @@ static int match_explicit(struct ref *src, struct ref *dst,\n> >       const char *dst_value = rs->dst;\n> >       char *dst_guess;\n> >\n> > -     if (rs->pattern || rs->matching)\n> > +     if (rs->pattern || rs->matching || rs->negative)\n> >               return 0;\n>\n> OK.  These \"special\" ones do not participate in explicit matching.\n>\n> > @@ -1134,6 +1248,10 @@ static char *get_ref_match(const struct refspec *rs, const struct ref *ref,\n> >       int matching_refs = -1;\n> >       for (i = 0; i < rs->nr; i++) {\n> >               const struct refspec_item *item = &rs->items[i];\n> > +\n> > +             if (item->negative)\n> > +                     continue;\n> > +\n>\n> And a negative one does not decide if a ref being pushed will be\n> pushed out for real at this point.  This helper is only to enumerate\n> the candidate refs to be pushed out; the caller makes a separate\n> call to apply_negative_refspecs() to cull the candidate list later.\n>\n> OK.\n>\n> > @@ -1339,7 +1457,7 @@ int check_push_refs(struct ref *src, struct refspec *rs)\n> >       for (i = 0; i < rs->nr; i++) {\n> >               struct refspec_item *item = &rs->items[i];\n> >\n> > -             if (item->pattern || item->matching)\n> > +             if (item->pattern || item->matching || item->negative)\n> >                       continue;\n> >\n> >               ret |= match_explicit_lhs(src, item, NULL, NULL);\n>\n> match_explicit_lhs(), like match_explicit(), are for explicit\n> matching and should not be called for the \"special\" ones.  OK.\n>\n> > @@ -1441,6 +1559,8 @@ int match_push_refs(struct ref *src, struct ref **dst,\n> >               string_list_clear(&src_ref_index, 0);\n> >       }\n> >\n> > +     *dst = apply_negative_refspecs(*dst, rs);\n> > +\n> >       if (errs)\n> >               return -1;\n> >       return 0;\n>\n> And after grabbing all the candidate refs to be updated via this\n> push, we filter out the ones that match negative pattern.  Can it\n> also produce an error, or it can never fail (to udpate errs)?\n>\n> > @@ -1810,6 +1930,9 @@ int get_fetch_map(const struct ref *remote_refs,\n> >  {\n> >       struct ref *ref_map, **rmp;\n> >\n> > +     if (refspec->negative)\n> > +             return 0;\n> > +\n>\n> Again, the idea is to let the existing codepath to only deal with\n> the positive refspec elements to keep the same behaviour, and let\n> the caller filter the ones that match negative ones out of the\n> result.  So we return without anything here for negative one.\n>\n\nYep, that's what I went for. The only real downside here is if we\nforget a code path that should honor negative refspecs and doesn't,\nbecause it will \"accept\" the refspec list with such a negative\nrefspec, but not do anything with it.\n\n> Nothing jumped out at me as being suspicious so far, other than that\n> the GNU \"?<empty>:\" thing needs to be fixed as pointed out by Dscho.\n>\n> Thanks.\n"},{"id":"406290","messageId":"CA+P7+xqJis+DA=ZZ0Qphf0P8RGVbXyvHUHaqo30LQDytxAvs-w@mail.gmail.com","threadId":"54090","inReplyTo":"xmqqzh5onea1.fsf@gitster.c.googlers.com","subject":"Re: [RFC v2 1/1] refspec: add support for negative refspecs","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2020-09-24T23:42:54Z","receivedAt":"2020-09-24T23:43:07Z","isPatch":false,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Thu, Sep 17, 2020 at 5:02 PM Junio C Hamano <gitster@pobox.com> wrote:\n>\n> Jacob Keller <jacob.e.keller@intel.com> writes:\n>\n> > @@ -1441,6 +1559,8 @@ int match_push_refs(struct ref *src, struct ref **dst,\n> >               string_list_clear(&src_ref_index, 0);\n> >       }\n> >\n> > +     *dst = apply_negative_refspecs(*dst, rs);\n> > +\n> >       if (errs)\n> >               return -1;\n> >       return 0;\n>\n> And after grabbing all the candidate refs to be updated via this\n> push, we filter out the ones that match negative pattern.  Can it\n> also produce an error, or it can never fail (to udpate errs)?\n>\n\nIt can't fail. Either refs match a negative refspec and will get\nexcluded, or they do not match a negative refspec and will be kept. We\nalready validate the negative refspecs earlier.\n\nThanks,\nJake\n"}]}